In jmgirard/agreement: What the Package Does (One Line, Title Case)

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.path = "man/figures/README-",
  out.width = "100%"
)
set.seed(11)

agreement

The goal of the agreement package is to calculate estimates of inter-rater agreement and reliability using generalized formulas that accommodate different designs (e.g., crossed or uncrossed), missing data, and ordered or unordered categories. The package includes generalized functions for all major chance-adjusted indexes of categorical agreement (i.e., α, γ, Ir2, κ, π, and S) as well as all major intraclass correlation coefficients (i.e., one-way and two-way models, agreement and consistency types, and single measure and average measure units). Estimates include bootstrap resampling distributions, confidence intervals, and custom tidying and plotting functions.

Installation

You can install the development version from GitHub with:

# install.packages("devtools")
devtools::install_github("jmgirard/agreement")

Example

Calculate chance-adjusted indexes of categorical agreement for unordered categories

library(agreement)
# Load dataset with 4 raters assigning 12 objects to 5 unordered categories
data(unordered)
print(unordered)

# Calculate all chance-adjusted indexes for unordered categories
results1 <- cat_adjusted(unordered)
summary(results1, ci = TRUE, type = "perc")

# Transform results into a tidy data frame
tidy(results1, type = "perc")

# Plot the bootstrap resampling distributions with confidence intervals
plot(results1)

Calculate chance-adjusted indexes of categorical agreement for ordered categories

# Load dataset with 5 raters assigning 20 objects to 4 ordered categories
data(ordered)
print(ordered)

# Calculate all chance-adjusted indexes for ordered categories (linear weights)
results2 <- cat_adjusted(ordered, weighting = "linear")
summary(results2, ci = TRUE, type = "perc")

tidy(results2, type = "perc")

plot(results2)

Calculate category-specific agreement

# Calculate category-specific agreement
results3 <- cat_specific(ordered)
summary(results3, ci = TRUE, type = "bca")

tidy(results3, type = "bca")

plot(results3)

Calculate intraclass correlation coefficient for dimensional data with 1 trial

# Load dataset with 4 raters rating 15 objects in 1 trial
data(lungfun)
print(lungfun)

# Calculate average score ICC using Model 1A
results4 <- dim_icc(lungfun, model = "1A", type = "agreement", unit = "average",
                    object = Object, rater = Rater, score = Score, warnings = FALSE)
summary(results4)

tidy(results4)

plot(results4, intra = FALSE, inter = TRUE)

Calculate intraclass correlation coefficient for dimensional data with many trials

# Load dataset with 4 raters rating 8 objects in 3 trials
data(lungfun_trials)
print(lungfun_trials)

# Calculate single score ICC using Model 2A
results5 <- dim_icc(lungfun_trials, Object, Rater, Score, trial = Trial, model = "2", 
                    type = "agreement", unit = "single", warnings = FALSE)
summary(results5)

tidy(results5)

plot(results5)

Code of Conduct

Please note that the 'agreement' project is released with a Contributor Code of Conduct. By contributing to this project, you agree to abide by its terms.

jmgirard/agreement documentation built on Sept. 12, 2022, 12:39 a.m.