fleiss.kappa.dist: Fleiss' agreement coefficient among multiple raters (2, 3, +)...

In irrCAC: Computing Chance-Corrected Agreement Coefficients (CAC)

Description

Fleiss' agreement coefficient among multiple raters (2, 3, +) when the input dataset is the distribution of raters by subject and category.

Usage

fleiss.kappa.dist(ratings, weights = "unweighted", categ = NULL,
  conflev = 0.95, N = Inf)

Arguments

ratings	An nxq matrix / data frame containing the distribution of raters by subject and category. Each cell (i,k) contains the number of raters who classsified subject i into category k.
weights	is an optional parameter that is either a string variable or a matrix. The string describes one of the predefined weights and must take one of the values ("quadratic", "ordinal", "linear", "radical", "ratio", "circular", "bipolar"). If this parameter is a matrix then it must be a square matri qxq where q is the number of posssible categories where a subject can be classified. If some of the q possible categories are not used, then it is strobgly advised to specify the complete list of possible categories as a vector in parametr categ. Otherwise, only the categories reported will be used.
categ	An optional parameter representing all categories available to raters during the experiment. This parameter may be useful if some categories were not used by any rater inspite of being available to the raters.
conflev	An optional parameter representing the confidence level associated with the confidence interval. Its default value is 0.95.
N	An optional parameter representing the population size (if any). It may be use to perform the final population correction to the variance. Its default value is infinity.

Value

A vector containing the following information: pa(the percent agreement),pe(the percent chance agreement),coeff(Fleiss' agreement coefficient), stderr(the standard error of Fleiss' coefficient),conf.int(the confidence interval of Fleiss Kappa coefficient), p.value(the p-value of Fleiss' coefficient),coeff.name ("Fleiss").

Source

Fleiss, J. L. (1981). Statistical Methods for Rates and Proportions. John Wiley & Sons.

Examples

#The dataset "distrib.6raters" comes with this package. It represents the distribution of 6 raters 
#by subject and by category. Note that each row of this dataset sums to the number of raters, which
#is 6. You may this dataset as follows:
distrib.6raters
fleiss.kappa.dist(distrib.6raters) #Fleiss' kappa, precision measures, weights & list of categories
fleiss <- fleiss.kappa.dist(distrib.6raters)$coeff #Yields Fleiss' kappa alone.
fleiss
q <- ncol(distrib.6raters) #Number of categories
fleiss.kappa.dist(distrib.6raters,weights = quadratic.weights(1:q)) #Weighted fleiss/quadratic wts

Example output

   Depression Personality.Disorder Schizophrenia Neurosis Other
1           0                    0             0        6     0
2           0                    3             0        0     3
3           0                    1             4        0     1
4           0                    0             0        0     6
5           0                    3             0        3     0
6           2                    0             4        0     0
7           0                    0             4        0     2
8           2                    0             3        1     0
9           2                    0             0        4     0
10          0                    0             0        0     6
11          1                    0             0        5     0
12          1                    1             0        4     0
13          0                    3             3        0     0
14          1                    0             0        5     0
15          0                    2             0        3     1
     coeff.name     coeff     stderr     conf.int      p.value        pa
1 Fleiss' Kappa 0.4139265 0.08119291 (0.24,0.588) 0.0001622724 0.5511111
         pe
1 0.2340741
[1] 0.4139265
     coeff.name     coeff    stderr       conf.int   p.value        pa
1 Fleiss' Kappa 0.2511909 0.1457381 (-0.061,0.564) 0.1067865 0.8544444
         pe
1 0.8056173

irrCAC documentation built on Sept. 23, 2019, 5:05 p.m.