compare.richness.fnc: Compare Lexical Richness of Two Texts

In languageR: Analyzing Linguistic Data: A Practical Introduction to Statistics

Description

Comparisons of lexical richness between two texts are carried out on the basis of the vocabulary size (number of types) and on the basis of the vocabulary growth rate. Variances of the number of types and of the number of hapax legomena required for the tests are estimated with the help of LNRE models.

Usage

compare.richness.fnc(text1, text2, digits = 5)

Arguments

text1	First text in the comparison.
text2	Second text in the comparison.
digits	Number of decimal digits required for the growth rate.

Details

The comparison for the vocabulary size is carried out with the test statistic

Z = (E[V_1] - E[V_2])/sqrt(VAR[V_1] + VAR[V_2])

and the comparison of the growth rates with the test statistic

Z = (E[V_1(1)]/N_1 - E[V_2(1)]/N_2)/sqrt(VAR[V_1(1)]/N_1^2 + VAR[V_2(1)]/N_2^2)

where N denotes the sample size in tokens, V the vocabulary size, and V(1) the number of hapax legomena.

Value

A summary listing the Chi-Squared measure of goodness of fit for the LNRE models (available in the zipfR package) used to estimate variances, a table listing tokens, types, hapax legomena and the vocabulary growth rate, and two-tailed tests for differences in the vocabulary sizes and growth rates with Z-score and p-value.

Note

It is probably unwise to attempt to apply this function to texts comprising more than 500,000 words.

Author(s)

R. Harald Baayen Radboud University Nijmegen and Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands. [e-mail]

References

Baayen, R. H. (2001) Word Frequency Distributions, Kluwer Academic Publishers, Dordrecht.

Examples

	## Not run: 
     data(alice, through, oz)
     compare.richness.fnc(tolower(alice), tolower(through[1:length(alice)]))
     compare.richness.fnc(tolower(alice), tolower(oz[1:25942]))
  
## End(Not run)

Example output

comparison of lexical richness for tolower(alice) and tolower(through[1:length(alice)]) 
with approximations of variances based on the LNRE models
gigp (X2 = 61.72) and gigp (X2 = 53.29)

                                  Tokens Types HapaxLegomena GrowthRate
tolower(alice)                     27269  2615          1166    0.04276
tolower(through[1:length(alice)])  27269  2727          1208    0.04430

two-tailed tests:

                             Z      p
Vocabulary Size        -2.7041 0.0068
Vocabulary Growth Rate -1.0113 0.3119

comparison of lexical richness for tolower(alice) and tolower(oz[1:25942]) 
with approximations of variances based on the LNRE models
gigp (X2 = 61.72) and gigp (X2 = 58.8)

                     Tokens Types HapaxLegomena GrowthRate
tolower(alice)        27269  2615          1166    0.04276
tolower(oz[1:25942])  25942  2333          1005    0.03874

two-tailed tests:

                            Z      p
Vocabulary Size        7.0938 0.0000
Vocabulary Growth Rate 2.6850 0.0073

languageR documentation built on May 2, 2019, 10:02 a.m.