In jlangaa/bouldr: A Psychology-Focused Tool for ROC Analyses

git repo: https://github.com/jlangaa/bouldr.git

Setup

First, we'll set the seed and some other useful options.

set.seed(2020)
library(bouldr)
library(dplyr)
library(ggplot2)
options(digits = 2)

Next, we'll generate some test data.

dd <- generate_data(10,Diagnosis = c("Depression","Anxiety","ADHD"),
                          Measure = c("A","B"),
                          Informant = c("Self","Parent", "Teacher"))

The chunk above creates a data frame with three hypothetical diagnoses (Depression, ANxiety, and ADHD), two forms (A and B), and three informants (Self, Parent, and Teacher). The Outcome and Score variables are important for running the ROC -- they specify dichotomous criterion variable ("Outcome") and the continuous predictor we're evaluating ("Score"). The others variables are for grouping the curves (see more below).

head(dd)

Next, we'll create three bouldr objects to demonstrate how the grouping variables are handled. Note that we can run these all on the same data; variables not included in the model formula are simply ingored.

Creating a single-type bouldr object

single <- bouldr(data = dd,
            formula = Outcome ~ Score,
            test = 'delong',
            levels = c('no','yes'),
            direction = "<")

aucs(single)

This is the simplest version. The formula Outcome ~ Score indicates that we want to run a single ROC analysis using Outcome as the dichotomous outcome variable, and Score as the continuous predictor. The aucs function generates a table showing the AUC of the curve.

The other arguments, test, levels, and direction, specify respectively: the statistical test comparing the ROC curves, the levels of the outcome variable, and the relationship between the levels indicating which is the positive case. These latter two arguments get passed directly to pROC::roc_.

The plot below shows the single ROC curve defined by the formula above.

plot(single) + labs(title = "Demo of plot function using test data")

The plot is defined using ggplot2, and can be manipulated after generation, e.g., by adding a theme.

Finally, the tests function produces a table with the statistical tests

tests(single)

Including a grouping variable

grouped <- bouldr(data = dd,
            formula = Outcome ~ Score + Informant,
            test = 'delong',
            levels = c('no','yes'),
            direction = "<")

In this example, the formula contains the grouping variable, Informant, after the predictor term in the formula. A separate ROC curve will be calculated for each subset of the data specified by unique values of the grouping variable. Each curve will then be compared against each other curve using the specified statistical test.

plot(grouped) + labs(title = "Demo of plot function using test data")

The plot shows multiple ROC curves on the same axes.

aucs(grouped)

The AUC table now includes a separate AUC for each grouping variable.

tests(grouped)

The tests table shows the pairwise comparisons between ROCs for each grouping variable.

Adding a faceting variable

faceted <- bouldr(data = dd,
            formula = Outcome ~ Score + Informant + Diagnosis,
            test = 'delong',
            levels = c('no','yes'),
            direction = "<")

Finally, we can add an additional faceting variable: Diagnosis. A variable specified after the grouping variable will create discrete sets of ROC curves that will be plotted separately. Only curves within the same faceting variable will be compared to one another using the specified statistical test; ROCs across facets are not compared. The plot below shows the effect of adding a faceting variable.

plot(faceted, ncol = 1, facet_order = c("ADHD","Anxiety","Depression")) + labs(title = "Demo of plot function using test data")

Note the facet labels at the top of each plot.

aucs(faceted)

The AUC table now includes a Facet variable, in addition to the group variable seen before.

tests(faceted)

The additional variable is also included in the tests output table.

The tests function prints tests of each ROC curve against each other as well as each ROC curve against chance (U-Test).

Diagnostic Likelihood ratio

Only works for a single outcome / scale at a time.

First, compute diagnostic statistics for a range of cutpoints based on equally-spaced quantiles of the outcome variable. Specify the number of quantiles with n.cuts argument.

d.dlrs <- dd %>%
  dplyr::filter(Diagnosis == "Depression" & Measure == "A" & Informant == "Self")

dlrs(x = d.dlrs , outcomes = "Outcome",scores = "Score",positive = "yes", n.cuts = 2)

Now compute diatgnostic statistics for a specified cutpoint

dlrs(x = d.dlrs, outcomes = "Outcome", scores= "Score", positive = "yes", cut = 3)

jlangaa/bouldr documentation built on May 3, 2024, 5:40 a.m.