| dispersion-score-functions | R Documentation |
defaultDispersionFunctions()
## S4 method for signature 'numeric'
dispersionAnalysis(
frequencies,
sizes,
scoreFunctions = defaultDispersionFunctions()
)
dispersionDP(frequencies, sizes)
dispersionDPnorm(frequencies, sizes)
dispersionJuillandD(frequencies, sizes)
dispersionCarrollD2(frequencies, sizes)
dispersionRosengrenS(frequencies, sizes)
dispersionRange(frequencies, sizes)
frequencies |
Numeric vector of observed or estimated hit counts, or a
|
sizes |
Numeric vector of corresponding corpus-part sizes. |
scoreFunctions |
Named list of functions with arguments
|
Calculate dispersion from absolute frequencies and corpus-part sizes. The parts must be disjoint; include positive-sized parts without hits. Sizes are token counts (or proportional sizes), not the frequencies of the query. The analysis interface and result structure may change in future releases.
dispersionDP() computes sum(abs(frequencies / sum(frequencies) - sizes / sum(sizes))) / 2. dispersionDPnorm() divides DP by
1 - min(sizes / sum(sizes)), following the corrected normalization in
Lijffijt and Gries (2012). Smaller DP and DPnorm mean more even distribution.
Juilland's D, Carroll's D2 and Rosengren's S use the versions for unequal
part sizes described by Gries (2020); larger values mean more even
distribution. These are independent implementations of the published
formulas; Gries' companion script is not included or required.
dispersionRange()
counts parts with hits, not documents unless documents are the parts.
The built-in measures other than range return NA_real_ if there are no
hits or fewer than two positive-sized parts. Range is zero without hits.
No measure is assumed to be appropriate for every research question, and
the measures are not combined into a rank or adjusted frequency.
dispersionAnalysis() returns a one-row tibble, one column per score.
defaultDispersionFunctions() returns a named list of functions.
Individual score functions return a numeric scalar.
Range describes observed reach: the number of parts containing at least one hit. It ignores part sizes and within-part frequencies, and depends on the total number of hits. Broader reach need not mean more even distribution.
DP compares each part's share of hits with its share of corpus tokens. For example, DP = 0.20 means that 20 percent of the hit distribution would have to be reallocated between parts to match the token shares, keeping the total fixed. This is hypothetical hit mass, allowing fractional counts; it is not a percentage of sources or speakers. Smaller is closer, and zero is an exact match. DPnorm rescales DP by its maximum for the given part sizes; the redistribution interpretation applies to DP itself, not directly to DPnorm.
Juilland's D describes variation in relative hit frequencies across parts; Carroll's D2 describes their entropy. Both equal one when each part has the same relative frequency. Rosengren's S compares the hit and token distributions, reaching one when they match. Larger D, D2 and S mean more even distribution under their respective definitions. The measures emphasize different aspects, so they need not rank variants in the same order.
Compare variants within the same corpus and partition. There is no universal cutoff for good dispersion, and these descriptive scores are not significance tests. Averaging ranks would depend on the selected measures, give related measures repeated votes and discard the size of differences. Inspect the underlying part counts and token sizes, and report disagreements explicitly. The leading one or two sources' shares of hits, considered alongside their token shares, can help explain concentration. Concordances are needed to investigate possible house styles, quotation or syndicated material.
Gries, Stefan Th. (2008). Dispersions and adjusted frequencies in corpora. International Journal of Corpus Linguistics, 13(4), 403-437. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1075/ijcl.13.4.02gri")}.
Gries, Stefan Th. (2020). Analyzing dispersion. In M. Paquot & S. Th. Gries (eds.), A Practical Handbook of Corpus Linguistics, 99-118. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1007/978-3-030-46216-1_5")}. Companion resources: https://www.stgries.info/research/dispersion/links.html.
Lijffijt, Jefrey & Gries, Stefan Th. (2012). Correction to "Dispersions and adjusted frequencies in corpora". International Journal of Corpus Linguistics, 17(1), 147-149. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1075/ijcl.17.1.08lij")}.
Juilland, Alphonse G. & Chang-Rodriguez, Eugenio (1964). Frequency dictionary of Spanish words. The Hague: Mouton.
Carroll, John B. (1970). An alternative to Juilland's usage coefficient for lexical frequencies, and a proposal for a standard frequency index (SFI). Computer Studies in the Humanities and Verbal Behavior, 3(2), 61-65.
Rosengren, Inger (1971). The quantitative concept of language and its relation to the structure of frequency dictionaries. Etudes de linguistique appliquee (Nouvelle Serie), 1, 103-127.
dispersionAnalysis(c(1, 2, 3, 4, 5), c(9, 10, 10, 10, 11))
dispersionAnalysis(c(10, 0), c(100, 100), scoreFunctions = list(
DP = dispersionDP,
evenness = function(frequencies, sizes) 1 - dispersionDP(frequencies, sizes)
))
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.