reli_stats: Reliability Statistics

In SimplyAgree: Flexible and Robust Agreement and Reliability Analyses

Reliability Statistics

Description

The reli_stats and reli_aov functions produce reliability statistics described by Weir (2005). This includes intraclass correlation coefficients, the coefficient of variation, and the standard MSE of measurement.

Usage

reli_stats(
  measure,
  item,
  id,
  data,
  wide = FALSE,
  col.names = NULL,
  cv_calc = c("MSE", "residuals", "SEM"),
  conf.level = 0.95,
  other_ci = FALSE,
  type = c("perc", "norm", "basic"),
  replicates = 1999
)

reli_aov(
  measure,
  item,
  id,
  data,
  wide = FALSE,
  col.names = NULL,
  cv_calc = c("MSE", "residuals", "SEM"),
  conf.level = 0.95,
  other_ci = FALSE,
  type = "perc",
  replicates = 1999
)

Arguments

measure	Name of column containing the measurement of interest
item	Name of column containing the items. If this is a test-retest reliability study then this would indicate the time point (e.g., time1,time2, time3, etc.)
id	Column with subject identifier
data	Data frame with all data
wide	Logical value (TRUE or FALSE) indicating if data is in a "wide" format. Default is TRUE.
col.names	If wide is equal to TRUE then col.names is a list of the column names containing the measurements for reliability analysis.
cv_calc	Coefficient of variation (CV) calculation. This function allows for 3 versions of the CV. "MSE" is the default.
conf.level	the confidence level required. Default is 95%.
other_ci	Logical value (TRUE or FALSE) indicating whether to calculate confidence intervals for the CV, SEM, SEP, and SEE. Note: this will dramatically increase the computation time.
type	A character string representing the type of bootstrap confidence intervals. Only "norm", "basic", and "perc" currently supported. Bias-corrected and accelerated, bca, is the default. See ?boot::boot.ci for more details.
replicates	The number of bootstrap replicates. Passed on to the boot function. Default is 1999.

Details

These functions return intraclass correlation coefficients and other measures of reliability (CV, SEM, SEE, and SEP). The estimates of variances for any of the measures are derived from linear mixed models. When other_ci is set to TRUE, then a parametric bootstrap approach to calculating confidence intervals is used for the CV, SEM, SEE, and SEP.

reli_stats uses a linear mixed model to estimate variance components. In some cases there are convergence issues. When this occurs it is prudent to use reli_aov which instead utilizes sums of squares approach. The results may differ slightly between the functions. If reli_aov is used then rows with missing observations (e.g., if a participant has a missing observation) will be dropped.

The CV calculation has 3 versions. The "MSE" uses the "mean squared error" from the linear mixed model used to calculate the ICCs. The "SEM" option instead uses the SEM calculation and expresses CV as a ratio of the SEM to the overall mean. The "residuals" option u uses the model residuals to calculate the root mean square error which is then divided by the grand mean.

Value

Returns single list with the results of the agreement analysis.

"icc"

Table of ICC results

"lmer"

Linear mixed model from lme4

"anova"

Analysis of Variance table

"var_comp"

Table of Variance Components

"n.id"

Number of subjects/participants

"n.items"

Number of items/time points

"cv"

Coefficient of Variation

"SEM"

List with Standard MSE of Measurement estimate (est)

"SEE"

List with Standard MSE of the Estimate estimate (est)

"SEP"

List with Standard MSE of Predicitions (est)

"call"

the matched call

References

Weir, J. P. (2005). Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. The Journal of Strength & Conditioning Research, 19(1), 231-240.

Shrout, P.E. and Fleiss, J.L. (1976). Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin, 86, 420-3428.

McGraw, K. O. and Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30-46. See errata on page 390 of same volume.

Examples

data('reps')
reli_stats(data = reps, wide = TRUE, col.names = c("x","y"))

SimplyAgree documentation built on Dec. 28, 2022, 2:06 a.m.