specificTermsDlg: List terms specific of a document or level

In RcmdrPlugin.temis: Graphical Integrated Text Mining Solution

Description

List terms most associated (positively or negatively) with each document or each of a variable's levels.

Details

Specific terms reported here are those whose observed frequency in the document or level has the lowest probability under an hypergeometric distribution, based on their global frequencies in the corpus and on the number of occurrences in the document or variable level considered. The positive or negative character of the association is visible from the sign of the t value, or by comparing the value of the “% Term/Level” column with that of the “Global %” column.

All terms with a probability below the value chosen using the first slider are reported, ignoring terms with fewer occurrences in the whole corpus than the value of the second slider (these terms can often have a low probability but are too rare to be of interest). The last slider allows limiting the number of terms that will be shown for each level.

The result is a list of matrices, one for each level of the chosen variable, with seven columns:

“% Term/Level”:

the percent of the term's occurrences in all terms occurrences in the level.

“% Level/Term”:

the percent of the term's occurrences that appear in the level (rather than in other levels).

“Global %”:

the percent of the term's occurrences in all terms occurrences in the corpus.

“Level”:

the number of occurrences of the term in the level (“internal”).

“Global”:

the number of occurrences of the term in the corpus.

“t value”:

the quantile of a normal distribution corresponding the probability “Prob.”.

“Prob.”:

the probability of observing such an extreme (high or low) number of occurrences of the term in the level, under an hypergeometric distribution.

See Also

specificTerms, setCorpusVariables, meta, restrictTermsDlg, termsDictionary

RcmdrPlugin.temis documentation built on May 2, 2019, 11:10 a.m.