delta.many1: Compare dependent (multilevel) Fleiss kappas using the delta...

In svanbelle/multiagree: Comparaison of dependent kappa coefficients

Compare dependent (multilevel) Fleiss kappas using the delta method

Description

This function performs Hotelling's T square test using a variance-covariance matrix based on the bootstrap method to compare dependent multi-observers kappa coefficients

Usage

delta.many1(data, cluster_id, ncat = c(2, 3), multilevel = T,
  a.level = 0.05)

Arguments

data	a N x sum(ncat_g) matrix representing the classification of the N items by the observers in group g in the ncat_g categories. For each group, the number of categories can vary
cluster_id	a vector of lenght N with the identification number of the clusters
ncat	a vector with G elements indicating how many categories are considered to compute each kappa coefficient. For example, c(3,5) means that the three first columns correspond to the classification of subjects on a 3 categorical scale by a group of observers and the five last columns correspond to the classification of subjects on a 5 categorical scale by a group of observers.
multilevel	a binary indicator equal to TRUE in the presence of multilevel data and FALSE otherwise
a.level	the significance level

Details

This function compare several Fleiss kappa coefficients using Hotelling's T square with the variance-covariance matrix obtained by the delta method. If only one kappa is computed, it returns the estimate and confidence interval.

Value

$kappa a G x 2 matrix with the kappa coefficients in the first column and their corresponding standard error in the second column

$T_test a vector of length 2 with the value of Hotelling's T test as first element and the corresponding p-value as second element

$confidence confidence intervals for the pairwise comparisons of the measures

$cor the G x G correlation matrix for the kappa coefficients

Author(s)

Sophie Vanbelle [e-mail]

References

Fleiss J.L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin 76, 378-382.

Vanbelle S. (2017) Comparing dependent agreement coefficients obtained on multilevel data. Biometrical Journal, 59 (5):1016-1034

Vanbelle S. (submitted) On the asymptotic variability of (multilevel) multirater kappa coefficients

Examples

 
#dataset (not multilevel) (Fleiss, 1971)
data(fleiss_psy)
attach(fleiss_psy)
delta.many1(data=fleiss_psy[,2:6],cluster_id=fleiss_psy[,1],ncat=c(5),a.level=0.05,multilevel=TRUE)

svanbelle/multiagree documentation built on Feb. 9, 2023, 2:37 p.m.