calculate_agreement: Calculate inter-rater agreement statistics beween team...

In nthun/metamanager: Manage Meta-analysis Workflow in R

Description

Calculate agreement summaries and Krippendorff's alpha for all reviewer pairs. Krippendorff's alpha can be calculated for nominal, ordinal, interval, and ratio scale variables.

Usage

calculate_agreement(df, decision_col = "decision", kr_method = "nominal")

Arguments

df	a data frame that contains the screened articles, and having a name<chr> column
decision_col	variable name <chr> that defines the decision column (0/1)
kr_method	The scale of the decision variable. Can be nominal (default), ordinal, interval, or ratio

Value

A data frame containing agreement summaries, and the Krippendorff's alpha statistic for all pairs of screeners. invalid_decision<int> is the total number of decisions that are not 1 or 0.

See Also

kripp.alpha

Examples

library(dplyr)
team_df <- tibble(name = c("Tom", "Jerry", "Rose"),
                  screening_effort = c(.5, .4, .1))

set.seed(1)
df <-
    workaholism_pubmed %>%
    make_id(identifier = c("doi", "pmid")) %>%
    assign_articles(team_df = team_df,
                    reviewers = 2,
                    effort_col = "screening_effort") %>%
    # Generate fake screening data
    mutate(decision = sample(c(0:1, NA_integer_),
                             prob = c(.6,.3, .1),
                             size = nrow(.),
                             replace = TRUE))

calculate_agreement(df, decision_col = "decision")

nthun/metamanager documentation built on Aug. 9, 2019, 1:37 p.m.