dispersion-score-functions: Dispersion measures and user-defined scores

dispersion-score-functionsR Documentation

Dispersion measures and user-defined scores

Description

[Experimental]

Usage

defaultDispersionFunctions()

## S4 method for signature 'numeric'
dispersionAnalysis(
  frequencies,
  sizes,
  scoreFunctions = defaultDispersionFunctions()
)

dispersionDP(frequencies, sizes)

dispersionDPnorm(frequencies, sizes)

dispersionJuillandD(frequencies, sizes)

dispersionCarrollD2(frequencies, sizes)

dispersionRosengrenS(frequencies, sizes)

dispersionRange(frequencies, sizes)

Arguments

frequencies

Numeric vector of observed or estimated hit counts, or a corpusQuery() result; see dispersionAnalysis,KorAPQuery-method for the latter.

sizes

Numeric vector of corresponding corpus-part sizes.

scoreFunctions

Named list of functions with arguments ⁠function(frequencies, sizes)⁠, each returning one numeric value. As with defaultAssociationScoreFunctions(), replace or extend the list to use your own measures. Each function receives the absolute input values, with zero-sized, zero-hit parts removed.

Details

Calculate dispersion from absolute frequencies and corpus-part sizes. The parts must be disjoint; include positive-sized parts without hits. Sizes are token counts (or proportional sizes), not the frequencies of the query. The analysis interface and result structure may change in future releases.

dispersionDP() computes sum(abs(frequencies / sum(frequencies) - sizes / sum(sizes))) / 2. dispersionDPnorm() divides DP by 1 - min(sizes / sum(sizes)), following the corrected normalization in Lijffijt and Gries (2012). Smaller DP and DPnorm mean more even distribution. Juilland's D, Carroll's D2 and Rosengren's S use the versions for unequal part sizes described by Gries (2020); larger values mean more even distribution. These are independent implementations of the published formulas; Gries' companion script is not included or required. dispersionRange() counts parts with hits, not documents unless documents are the parts.

The built-in measures other than range return NA_real_ if there are no hits or fewer than two positive-sized parts. Range is zero without hits. No measure is assumed to be appropriate for every research question, and the measures are not combined into a rank or adjusted frequency.

Value

dispersionAnalysis() returns a one-row tibble, one column per score. defaultDispersionFunctions() returns a named list of functions. Individual score functions return a numeric scalar.

Interpretation

Range describes observed reach: the number of parts containing at least one hit. It ignores part sizes and within-part frequencies, and depends on the total number of hits. Broader reach need not mean more even distribution.

DP compares each part's share of hits with its share of corpus tokens. For example, DP = 0.20 means that 20 percent of the hit distribution would have to be reallocated between parts to match the token shares, keeping the total fixed. This is hypothetical hit mass, allowing fractional counts; it is not a percentage of sources or speakers. Smaller is closer, and zero is an exact match. DPnorm rescales DP by its maximum for the given part sizes; the redistribution interpretation applies to DP itself, not directly to DPnorm.

Juilland's D describes variation in relative hit frequencies across parts; Carroll's D2 describes their entropy. Both equal one when each part has the same relative frequency. Rosengren's S compares the hit and token distributions, reaching one when they match. Larger D, D2 and S mean more even distribution under their respective definitions. The measures emphasize different aspects, so they need not rank variants in the same order.

Compare variants within the same corpus and partition. There is no universal cutoff for good dispersion, and these descriptive scores are not significance tests. Averaging ranks would depend on the selected measures, give related measures repeated votes and discard the size of differences. Inspect the underlying part counts and token sizes, and report disagreements explicitly. The leading one or two sources' shares of hits, considered alongside their token shares, can help explain concentration. Concordances are needed to investigate possible house styles, quotation or syndicated material.

References

Gries, Stefan Th. (2008). Dispersions and adjusted frequencies in corpora. International Journal of Corpus Linguistics, 13(4), 403-437. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1075/ijcl.13.4.02gri")}.

Gries, Stefan Th. (2020). Analyzing dispersion. In M. Paquot & S. Th. Gries (eds.), A Practical Handbook of Corpus Linguistics, 99-118. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1007/978-3-030-46216-1_5")}. Companion resources: https://www.stgries.info/research/dispersion/links.html.

Lijffijt, Jefrey & Gries, Stefan Th. (2012). Correction to "Dispersions and adjusted frequencies in corpora". International Journal of Corpus Linguistics, 17(1), 147-149. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1075/ijcl.17.1.08lij")}.

Juilland, Alphonse G. & Chang-Rodriguez, Eugenio (1964). Frequency dictionary of Spanish words. The Hague: Mouton.

Carroll, John B. (1970). An alternative to Juilland's usage coefficient for lexical frequencies, and a proposal for a standard frequency index (SFI). Computer Studies in the Humanities and Verbal Behavior, 3(2), 61-65.

Rosengren, Inger (1971). The quantitative concept of language and its relation to the structure of frequency dictionaries. Etudes de linguistique appliquee (Nouvelle Serie), 1, 103-127.

Examples

dispersionAnalysis(c(1, 2, 3, 4, 5), c(9, 10, 10, 10, 11))
dispersionAnalysis(c(10, 0), c(100, 100), scoreFunctions = list(
  DP = dispersionDP,
  evenness = function(frequencies, sizes) 1 - dispersionDP(frequencies, sizes)
))

RKorAPClient documentation built on Nov. 12, 2026, 5:11 p.m.