icc_corr: Intraclass correlation coefficients (ICCs) for oneway and...

In irrNA: Coefficients of Interrater Reliability – Generalized for Randomly Incomplete Datasets

Intraclass correlation coefficients (ICCs) for oneway and twoway models – corrected version of icc{irr}

Description

Computes single score or average score ICCs as an index of interrater reliability of quantitative data. Additionally, F-test and confidence interval are computed. icc_corr{irrNA} corrects 3 errors of Matthias Gamer's function icc (version 0.84.1).

Usage

icc_corr(
  ratings,
  model = c("oneway", "twoway"),
  type = c("consistency", "agreement"),
  unit = c("single", "average"),
  r0 = 0,
  conf.level = 0.95
)

Arguments

ratings	n*m matrix or dataframe, n subjects m raters.
model	a character string specifying if a "oneway" model (default) with row effects random, or a "twoway" model with column and row effects random should be applied. You can specify just the initial letter.
type	a character string specifying if "consistency" (default) or "agreement" between raters should be estimated. If a '"oneway"' model is used, only "consistency" could be computed. You can specify just the initial letter.
unit	a character string specifying the unit of analysis: Must be one of "single" (default) or "average". You can specify just the initial letter.
r0	specification of the null hypothesis r ≤ r0. Note that a one sided test (H1: r > r0) is performed.
conf.level	confidence level of the interval.

Details

By this ICC-function three bugs are corrected that were found in the function icc of the irr package (version 0.84.1):
Due to the first bug the p-values of ICC(A,1) and ICC(A,k) are computed wrongly: McGraw & Wong (1996) use the variable "v" both for the computation of the CIs and for the computation of the p-values. But "v" takes different values in these calculations. In the implementation of icc{irr} (version 0.84.1) this fact is missed.
The second correction only affects the rare case of the residual mean square (of the twoway model) being zero, i.e. the case that the variance in the data may be explained completely by the two factors (Raters and Objects). In this case the F-value for determining all four twoway p-values is not correctly computet by icc.
The third correction addresses the problems arising in the rare cases of (a) no part or (b) nearly no part of variance may be explained by both factors.

Value

A list with class '"icclist"' containing the following components:

$subjects	the number of subjects examined.
$raters	the number of raters.
$model	a character string describing the selected model for the analysis.
$type	a character string describing the selected type of interrater reliability.
$unit	a character string describing the unit of analysis.
$icc.name	a character string specifying the name of ICC according to McGraw & Wong (1996).
$value	the intraclass correlation coefficient.
$r0	the specified null hypothesis.
$Fvalue	the value of the F-statistic.
$df1	the numerator degrees of freedom.
$df2	the denominator degrees of freedom.
$p.value	the p-value for a two-sided test.
$conf.level	the confidence level for the interval.
$lbound	the lower bound of the confidence interval.
$ubound	the upper bound of the confidence interval.

Author(s)

Matthias Gamer, Markus Brueckl

References

McGraw, K.O., & Wong, S.P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30–46.

Shrout, P.E., & Fleiss, J.L. (1979), Intraclass correlation: uses in assessing rater reliability. Psychological Bulletin, 86, 420–428.

See Also

icc, iccNA

Examples

# Example 1:
data(EbelFILL)
# EbelFILL is a rather arbitrary data set:
EbelFILL
# If twoway agreement ICCs are computed (e.g. the single 
# measure) with icc{irr}, the 2nd df of F and thus the 
# p-value is erroneous (please install and load the irr 
# package):
#icc(EbelFILL, model="twoway", type="agreement")
# icc_corr calculates correctly: 
icc_corr(EbelFILL, model="twoway", type="agreement")
# 
# Example 2:
data(Consist)
# Consist exhibits a perfect consistency and 
# a moderate absolute agreement between raters:
Consist
# If twoway ICCs are computed with icc{irr}, the F-value is smaller
# than zero (!) and thus the p-value is enourmously erroneous:
#icc(Consist, model="twoway", type="consistency", unit="average")
# icc_corr calculates correctly: 
icc_corr(Consist, model="twoway", type="consistency", unit="average")
#
# Example 3:
data(Indep)
# Indep exhibits zero variance between the raters just as 
# well as between the objects:
Indep
# Errors occur, if twoway agreement ICCs are computed with icc{irr}:
# ICC(A,k) just as well as its CI-bounds are (falsely) positive 
# and greater than 1...
#icc(Indep, model="twoway", type="agreement", unit="average")
# ...but must be -Inf, just as icc_corr shows:
icc_corr(Indep, model="twoway", type="agreement", unit="average")
# ICC(A,1): 2nd df of F and thus the p-value are NaN
#icc(Indep, model="twoway", type="agreement")
# icc_corr calculates correlctly:
icc_corr(Indep, model="twoway", type="agreement")

irrNA documentation built on April 5, 2022, 1:12 a.m.