oconnell: O'Connell-Dobson estimators for multiobserver agreement.

In magree: Implements the O'Connell-Dobson-Schouten Estimators of Agreement for Multiple Observers

Description

Use the O'Connell-Dobson estimator of agreement for nominal or ordinal data. This includes a range of statistics on agreement for assuming either distinct or homogeneous items.

Usage

oconnell(X, weights=c("unweighted","linear","quadratic"), i=NULL, score = NULL)

Arguments

X	A matrix or data-frame with observations/subjects as rows and observers as columns.
weights	"unweighted" For nominal categories - only perfect agreement is counted. "linear" For ordinal categories where disagreement is proportional to the distance between the categories. This is analogous to the agreement weights w_{i,j}=1-|score[i]-score[j]|/(max(score)-min(score)). "quadratic" For ordinal categories where disagreement is proportional to the square of the distance between the categories. This is analogous to the agreement weights w_{i,j}=1-(score[i]-score[j])^2/(max(score)-min(score))^2.
i	For nominal categories - only perfect agreement is counted. For ordinal categories where disagreement is proportional to the distance between the categories. This is analogous to the agreement weights w_{i,j}=1-|score[i]-score[j]|/(max(score)-min(score)). For ordinal categories where disagreement is proportional to the square of the distance between the categories. This is analogous to the agreement weights w_{i,j}=1-(score[i]-score[j])^2/(max(score)-min(score))^2. This argument takes precedence over weights if it is specified.
score	The scores that are to be assigned to the categories. Currently, this defaults to 1:L, where codeL is the number of categories.

Details

The Fortran code from Professor Dianne O'Connell was adapted for R.

The output object is very similar to the Fortan code. Not all of the variance terms are currently used in the print, summary and plot methods.

Value

X	As input
i	As input
nrater	Number of observers
nscore	Number of categories
nsubj	Number of subjects
p1[j,k]	Probability of observer j giving score k when observers are distinct
p2[k]	Probability of score k when observers are homogeneous
w1[j,k]	Weighted average of d[] for observer j, score k
w2[k]	Weighted average of d[] for score k when observers are homogeneous
d[j]	Amount of disagreement for subject j
s1[j]	Chance-corrected agreement statistic for subject j when observers are distinct
s2[j]	Chance-corrected agreement statistic for subject j when observers are homogeneous; s[j]=1-d[j]/expdel.
delta[j,k]	j<k: amount of disagreement expected by change for observers j and k; j>k amount of disagreement expected by chance for observers j and k when observers are homogeneous
expd1	Amount of disagreement expected by chance in null case when observers are distinct
expd2	Amount of disagreement expected by chance when observers are homogeneous
dbar	Average value of d[] over all subjects
sav1	Chance-corrected agreement statistic over all subjects when observers are distinct
sav2	Chance-corrected agreement statistic over all subjects when observers are homogeneous
var0s1	Null variance of S when observers are distinct
var0s2	Null variance of S when observers are homogeneous
vars1	Unconstrained variance of S when observers are distinct
vars2	Unconstrained variance of S when observers are homogeneous
v0sav1	Null variance of Sav when observers are distinct
v0sav2	Null variance of Sav when observers are homogeneous
vsav1	Unconstrained variance of Sav when observers are distinct
vsav2	Unconstrained variance of Sav when observers are homogeneous
p0sav1	Probability of overall agreement due to chance when observers are distinct
p0sav2	Probability of overall agreement due to chance when observers are homogeneous
resp[i,j]	Response for observer i on subject j; transpose of X (BEWARE)
score(i)	Score associated with i'th category
call	As per sys.call(), to allow for using update

See Also

magree, schouten.

Examples

## Table 1 (O'Connell and Dobson, 1984)
summary(fit <- oconnell(landis, weights="unweighted"))
update(fit, weights="linear")
update(fit, weights="quadratic")

## Table 3 (O'Connell and Dobson, 1984)
slideTypeGroups <-
    list(c(2,3,5,26,31,34,42,58,59,67,70,81,103,120),
         c(7,10:13,17,23,30,41,51,55,56,60,65,71,73,76,86,87,105,111,116,119,124),
         c(4,6,24,25,27,29,39,48,68,77,79,94,101,102,117),
         c(9,32,36,44,52,62,84,95),
         c(35,53,69,72),
         c(8,15,18,19,47,64,82,93,98,99,107,110,112,115,121),
         c(1,16,22,49,63,66,78,90,100,113),
         c(28,37,40,61,108,114,118),
         106,
         43,
         83,
         c(54,57,88,91,126),
         c(74,104),
         38,
         46,
         c(89,122),
         c(80,92,96,123),
         85)
data.frame(SlideType=1:18,
           S1=sapply(slideTypeGroups,
                     function(ids) mean(fit$s1[as.character(ids)])),
           S2=sapply(slideTypeGroups,
                     function(ids) mean(fit$s2[as.character(ids)])))

## Table 5, O'Connell and Dobson (1984)
oconnell(landis==1)
oconnell(landis==2)
oconnell(landis==3)
oconnell(landis==4)
oconnell(landis==5)

## Plot of the marginal distributions
plot(fit)

magree documentation built on Sept. 3, 2020, 9:07 a.m.