Description Usage Arguments Details Value References Examples
Weight a term-document matrix according to a combination of weights specified in SMART notation.
1 | weightSMART(m, spec = "nnn", control = list())
|
m |
A |
spec |
a character string consisting of three characters. The first letter specifies a term frequency schema, the second a document frequency schema, and the third a normalization schema. See Details for available built-in schemata. |
control |
a list of control parameters. See Details. |
Formally this function is of class WeightingFunction
with the
additional attributes name
and acronym
.
The first letter of spec
specifies a weighting schema for term
frequencies of m
:
(natural) \mathit{tf}_{i,j} counts the number of occurrences
n_{i,j} of a term t_i in a document d_j. The
input term-document matrix m
is assumed to be in this
standard term frequency format already.
(logarithm) is defined as 1 + \log_2(\mathit{tf}_{i,j}).
(augmented) is defined as 0.5 + \frac{0.5 * \mathit{tf}_{i,j}}{\max_i(\mathit{tf}_{i,j})}.
(boolean) is defined as 1 if \mathit{tf}_{i,j} > 0 and 0 otherwise.
(log average) is defined as \frac{1 + \log_2(\mathit{tf}_{i,j})}{1+\log_2(\mathrm{ave}_{i\in j}(\mathit{tf}_{i,j}))}.
The second letter of spec
specifies a weighting schema of
document frequencies for m
:
(no) is defined as 1.
(idf) is defined as \log_2 \frac{N}{\mathit{df}_t} where \mathit{df}_t denotes how often term t occurs in all documents.
(prob idf) is defined as \max(0, \log_2(\frac{N - \mathit{df}_t}{\mathit{df}_t})).
The third letter of spec
specifies a schema for normalization
of m
:
(none) is defined as 1.
(cosine) is defined as √{\mathrm{col\_sums}(m ^ 2)}.
(pivoted unique) is defined as \mathit{slope} *
√{\mathrm{col\_sums}(m ^ 2)} + (1 - \mathit{slope}) *
\mathit{pivot} where both slope
and pivot
must be set
via named tags in the control
list.
(byte size) is defined as
\frac{1}{\mathit{CharLength}^α}. The parameter
α must be set via the named tag alpha
in the control
list.
The final result is defined by multiplication of the chosen term frequency component with the chosen document frequency component with the chosen normalization component.
The weighted matrix.
Christopher D. Manning and Prabhakar Raghavan and Hinrich Schütze (2008). Introduction to Information Retrieval. Cambridge University Press, ISBN 0521865719.
1 2 3 4 5 6 | data("crude")
TermDocumentMatrix(crude,
control = list(removePunctuation = TRUE,
stopwords = TRUE,
weighting = function(x)
weightSMART(x, spec = "ntc")))
|
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.