Build a k-gram language model.
probability(): compute word continuation and sentence probabilities.
sample_sentences(): generate random text by sampling from the
language model probability distribution at arbitary temperature. See
perplexity(): Compute the language model perplexity on a test
corpus. See perplexity.
1 2 3 4 5 6 7
an object which stores the information required to build the
k-gram model. At present, necessarily a
possible additional parameters required by the smoother.
a length one character vector. Indicates the smoothing
technique to be applied to compute k-gram continuation probabilities. A list
of available smoothers can be obtained with
a length one integer. Maximum order of k-grams to use in the language
model. This muss be less than or equal to the order of the underlying
These generics are used to construct objects of class
language_model method is only needed to create copies of
language_model objects (that is to say, new copies which are not
altered by methods which modify the original object in place,
see e.g. parameters). The discussion below focuses on
language models and the
kgrams supports several k-gram language models, including
Interpolated Kneser-Ney, Stupid Backoff and others
(see smoothers). The objects created by
language_models() have methods for computing word continuation and
sentence probabilities (see probability),
random text generation (see sample_sentences)
and other type of language modeling tasks such as
(not yet implemented) computing perplexities and word
Smoothers have often tuning parameters, which need to be specified by
(exact) name through the
... arguments; otherwise,
language_model() will use default values and, once per session, throw
info(smoother) lists all parameters needed by a
specific smoother, together with their allowed parameter space.
The run-time of
language_model() may vary substantially for different
smoothing methods, depending on whether or not a method requires the
computation of additional quantities (that is to say, beyond k-gram counts)
for its operativity (this is, for instance, the case for the Kneser-Ney
1 2 3 4 5 6 7 8 9
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.