A Latent Class Model for Agreement of Two Raters

Share:

Description

Estimates a latent class model for agreement of two raters (Schuster & Smith, 2006). See Details for the description of the model.

Usage

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
lc2_agreement(y, w = rep(1, nrow(y)), type = "homo", method = "BFGS", ...)

## S3 method for class 'lc2_agreement'
summary(object, digits=3,...)

## S3 method for class 'lc2_agreement'
logLik(object, ...)

## S3 method for class 'lc2_agreement'
anova(object, ...)

Arguments

y

A data frame containing the values of two raters in columns

w

Optional vector of weights

type

Type of model specification. Can be "unif", "equal", "homo" or "hete". See Details.

method

Optimization method used in stats::optim

...

Further arguments passed to stats::optim

object

Object of class l2_agreement

digits

Number of digits for rounding

Details

The latent class model for two raters decomposes a portion of ratings which conform to true agreement and another portion of ratings which conform to a random rating of a category. Let X_r denote the rating of rater r, then for i \neq j, it is assumed that

P(X_1 = i , X_2 = j) = φ_{1i} φ_{2j} ( 1 - γ )

For i = j it is assumed that

P(X_1 = i , X_2 = i) = τ_i γ + φ_{1i} φ_{2i} ( 1 - γ )

where γ denotes the proportion of true ratings.

All τ_i and φ_{ri} parameters are estimated using type="hete". If the φ parameters are assumed as invariant across the two raters (i.e. φ_{1i}=φ_{2i}=φ_{i}), then type="homo" must be specified. The constraint τ_i = φ_i is imposed by type="equal". All φ_i parameters are set equal to each other using type="unif".

Value

model_output

Output of the fitted model

saturated_output

Output of the saturated model

LRT_output

Output of the likelihood ratio test of model fit

partable

Parameter table

parmsummary

Parameter summary

agree_true

True agreement index shich is the γ parameter

agree_chance

Agreement by chance

rel_agree

Conditional reliability of agreement

optim_output

Output of optim from the fitted model

nobs

Number of observations

type

Model type

ic

Information criteria

loglike

Log-likelihood

npars

Number of parameters

y

Used dataset

w

Used weights

Author(s)

Alexander Robitzsch

References

Schuster, C., & Smith, D. A. (2006). Estimating with a latent class model the reliability of nominal judgments upon which two raters agree. Educational and Psychological Measurement, 66(5), 739-747.

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
#############################################################################
# EXAMPLE 1: Dataset in Schuster & Smith (2006)
#############################################################################

data(data.immer08)
dat <- data.immer08

# select ratings and frequency weights
y <- dat[,1:2]
w <- dat[,3]

#*** Model 1: Uniform distribution phi parameters
mod1 <- lc2_agreement( y = y , w = w , type="unif")
summary(mod1)

#*** Model 2: Equal phi and tau parameters
mod2 <- lc2_agreement( y = y , w = w , type="equal")
summary(mod2)

## Not run: 
#*** Model 3: Homogeneous rater model
mod3 <- lc2_agreement( y = y , w = w , type="homo")
summary(mod3)

#*** Model 4: Heterogeneous rater model
mod4 <- lc2_agreement( y = y , w = w , type="hete")
summary(mod4)

#--- some model comparisons
anova(mod3,mod4)
IRT.compareModels(mod1,mod2,mod3,mod4)

## End(Not run)