kappa: Kappa for agreement with multiple raters

In iriseekhout/Agree: Agreement and reliability between multiple raters

Kappa for agreement with multiple raters

Description

Cohen's kappa and weighed kappa for multiple raters, adapted from the ‘psych::cohen.kappa' function (Revelle, 2020). Light’s method is used to combine the scores from multiple raters.

Usage

kappa(
  data,
  weight = NULL,
  confint = FALSE,
  alpha = 0.05,
  k = NULL,
  n = NULL,
  ...
)

Arguments

data	A 'data.frame' or table with equal number of columns and rows. Or 'data.frame' that contains the scores for each rater in each column.
weight	matrix of weights for the weighed kappa.
confint	Logical indicator for confidence interval
alpha	Confidence interval level, default = 0.05.
k	number of raters; default 'k = ncol(data)'.
n	sample size; default 'n = nrow(data)'.
...	options for sumtable if 'is.data.frame(data)'

Value

vector

References

Revelle, W. (2020) psych: Procedures for Personality and Psychological Research, Northwestern University, Evanston, Illinois, USA, https://CRAN.R-project.org/package=psych Version = 2.0.12,. Light, R. J. (1971) Measures of response agreement for qualitative data: Some generalizations and alternatives, Psychological Bulletin, 76, 365-377.

Examples

df <- data.frame(r1=factor(c(1,0,1,0,0,1,1,0,0,0,1,1,0,1,1)),
                 r2=factor(c(1,1,1,1,0,1,1,0,0,0,1,1,0,1,0)),
                 r3=factor(c(1,1,1,0,0,0,1,1,1,0,0,1,0,1,1)),
                 r4=factor(c(1,1,1,1,1,1,1,1,1,1,1,1,1,1,1)))
table <- sumtable(df=df, ratings=c("r1", "r2", "r3", "r4"), levels=c("0","1"))
kappa(df)
kappa(table)

df <- data.frame(r1=factor(c(1,2,2,0,3,3,1,0,3,0,2,2,0,3,1)),
                 r2=factor(c(1,1,1,0,3,3,1,0,1,0,2,2,0,2,1)),
                 r3=factor(c(1,1,1,3,3,2,1,0,1,0,2,2,0,3,1)),
                 r4=factor(c(1,2,1,0,3,3,1,0,3,0,2,2,0,2,1)))
table <- sumtable(df=df, ratings=c("r1", "r2", "r3", "r4"), levels=c("0","1", "2", "3"))
kappa(df)
kappa(table)
kappa(df, confint = TRUE)
kappa(table)

iriseekhout/Agree documentation built on July 28, 2023, 11:24 p.m.