summarize_oolong: Summarize oolong objects

In oolong: Create Validation Tests for Automated Content Analysis

Summarize oolong objects

Description

This function summarizes one or more oolong objects. All oolong objects must be locked.

Usage

summarize_oolong(..., target_value = NULL, n_iter = 1500)

summarise_oolong(..., target_value = NULL, n_iter = 1500)

Arguments

...	(tm/gs) one or more oolong objects to be summarized
target_value	(gs) a vector of numeric values, the value you want to validate against the human-coded gold standard. One example of this target value is sentiment score extracted automatically from text
n_iter	(ti) number of iterations to calculate the median test

Value

An oolong summary. Depends on purpose, an oolong summary object has the following values:

$type

(gs/tm) type of analysis, either 'gs' or 'tm'

$kripp_alpha; $kripp_alpha_wsi

(wi, wsi) Krippendorff's Alpha, if more than one oolong object is analyzed.

$rater_precision; $rater_precision_wsi

(wi, wsi) Model precision

$res$rater_precision_p_value

(wi) Model precision's p-value calculated by one-sample binomial test and Fisher's Omnibus method.

$k_precision; $k_precision_wsi

(wi, wsi) precision for each topic

$tlo

(ti) vector of topic log odds

$tlo_pvalue

(ti) Median topic log odds's p-value calculated by permutation test.

$cor

(gs) Pearson's correlation between average answer and target value

$cor_length

(gs) Pearson's correlation between content length and target value

$diag_plot

(gs) diagnostic plot.

A useful summary of an object can be obtained either by print.oolong_summary or plot.oolong_summary. For details, please see the overview vignette: vignette("overview", package = "oolong")

Author(s)

Chung-hong Chan

References

Chang, J., Gerrish, S., Wang, C., Boyd-Graber, J. L., & Blei, D. M. (2009). Reading tea leaves: How humans interpret topic models. In Advances in neural information processing systems (pp. 288-296).

Song et al. (2020) In validations we trust? The impact of imperfect human annotations as a gold standard on the quality of validation of automated content analysis. Political Communication.

Ying, L., Montgomery, J. M., & Stewart, B. M. (2021). Topics, Concepts, and Measurement: A Crowdsourced Procedure for Validating Topics as Measures. Political Analysis.

Examples

# Please try this example in interactive R sessions only.
if (interactive()) {
   data(abstracts_stm)
   oolong_test1 <- create_oolong(abstracts_stm)
   oolong_test2 <- clone_oolong(oolong_test1)
   oolong_test1$do_word_intrusion_test()
   oolong_test2$do_word_intrusion_test()
   oolong_test1$lock()
   oolong_test2$lock()
   summarize_oolong(oolong_test1, oolong_test2)
}

oolong documentation built on Aug. 25, 2023, 5:16 p.m.