specificTerms: List terms specific of a document or level

Description Usage Arguments Details Value Author(s) See Also

Description

List terms most associated (positively or negatively) with each document or each of a variable's levels.

Usage

1
specificTerms(dtm, variable, p = 0.1, n.max = 25, sparsity = 0.95, min.occ = 2)

Arguments

dtm

a document-term matrix.

variable

a vector whose length is the number of rows of dtm, or NULL to report specific terms by document.

p

the maximum probability up to which terms should be reported.

n.max

the maximum number of terms to report for each level.

sparsity

Optional sparsity threshold (between 0 and 1) below which terms should be skipped. See removeSparseTerms from tm.

min.occ

the minimum number of occurrences in the whole dtm below which terms should be skipped.

Details

Specific terms reported here are those whose observed frequency in the document or level has the lowest probability under an hypergeometric distribution, based on their global frequencies in the corpus and on the number of occurrences of all terms in the document or variable level considered. The positive or negative character of the association is visible from the sign of the t value, or by comparing the value of the “% Term/Level” column with that of the “Global %” column.

All terms with a probability below p are reported, up to n.max terms for each category.

Value

A list of matrices, one for each level of the variable, with seven columns:

“% Term/Level”

the percent of the term's occurrences in all terms occurrences in the level.

“% Level/Term”

the percent of the term's occurrences that appear in the level (rather than in other levels).

“Global %”

the percent of the term's occurrences in all terms occurrences in the corpus.

“Level”

the number of occurrences of the term in the level (“internal”).

“Global”

the number of occurrences of the term in the corpus.

“t value”

the quantile of a normal distribution corresponding the probability “Prob.”.

“Prob.”

the probability of observing such an extreme (high or low) number of occurrences of the term in the level, under an hypergeometric distribution.

Author(s)

Milan Bouchet-Valat

See Also

frequentTerms, DocumentTermMatrix, removeSparseTerms


RcmdrPlugin.temis documentation built on May 2, 2019, 11:10 a.m.