kappa_tidy: Calculate Kappas for interrater agreement trial

In gederajeg/happyr: The Accompanying R Package for Rajeg's (2019) PhD Thesis Titled "Metaphorical Profiles and Near Synonyms: A Corpus-based Study of Indonesian Words for Happiness"

Description

The function performs Kappas for interrater agreement between two-raters, the author of the thesis and one other coder. The calculation is based on the kappa2 function from the irr package (Gamer et al, 2012). All scripts to use this function to perform the calculation for each interrater agreement trial in Rajeg (2019, Chapter 3) are shown in the Examples section below.

Usage

kappa_tidy(
  df = NULL,
  var_names = "pattern_(coder|author)",
  split_by = "synonyms",
  round = "pre_disc"
)

Arguments

df	The input data frame for each interrater trial tasks
var_names	character strings of regular expressions for the columns involved in the calculation of the Kappa score (i.e. the interrater agreement score).
split_by	by default it is specified by "synonyms" since Kappas are computed for classification for data per synonyms included in the trial.
round	character strings indicating whether the Kappas are for the "pre-discussion" or "post-discussion" round.

Value

A tibble/data frame

References

• Gamer, M., Lemon, J., & Singh, P. (2012). irr: Various Coefficients of Interrater Reliability and Agreement. R package version 0.84.

• Rajeg, G. P. W. (2019). Metaphorical profiles and near-synonyms: A corpus-based study of Indonesian words for HAPPINESS (PhD Thesis). Monash University. Melbourne, Australia. https://doi.org/10.26180/5cac231a97fb1.

Examples

# kappa calculation for the constructional patterns
## pre-discussion
irr_cxn <- kappa_tidy(df = df_cxn_pattern,
                      var_names = "^pattern",
                      round = "pre_disc")
irr_cxn # check the output format
mean(irr_cxn[["kappa"]]) # mean
sd(irr_cxn[["kappa"]]) # SD


# kappa calculation for the metaphoricity
## pre-discussion
irr_use_1 <- kappa_tidy(df = df_meta_use_1st,
                       var_names = "^use_(coder|author)$",
                       round = "pre_disc")
mean(irr_use_1[["kappa"]]) # mean
sd(irr_use_1[["kappa"]]) # SD

## post-discussion
irr_use_2 <- kappa_tidy(df = df_meta_use_2nd,
                        var_names = "^use_2nd_",
                        round = "post_disc")
mean(irr_use_2[["kappa"]]) # mean
sd(irr_use_2[["kappa"]]) # SD

# interrater agreement for the conceptual metaphors
## pre-discussion
irr_cm_1 <- kappa_tidy(df = df_cm,
                       var_names = "_pre$",
                       round = "pre_disc")
mean(irr_cm_1[["kappa"]]) # mean
sd(irr_cm_1[["kappa"]]) # SD

## post-discussion
irr_cm_2 <- kappa_tidy(df = df_cm,
                       var_names = "_post$",
                       round = "post_disc")
mean(irr_cm_2[["kappa"]]) # mean
sd(irr_cm_2[["kappa"]]) # SD

gederajeg/happyr documentation built on Dec. 27, 2021, 3:24 a.m.