In ianhussey/SCED: Analyse data from AB Single Case Experimental Designs

knitr::opts_chunk$set(message = FALSE, 
                      warning = FALSE)

# dependencies
library(SCED)
library(knitr)
library(kableExtra)
library(tidyverse)

# dependencies required by SCED
library(broom)
library(coin)
library(survival)
library(effsize)
library(bootES)
library(boot)
library(stringr)
library(timesavers)  # from github/ianhussey
library(finalfit)
library(metafor)

# disable scientific notation
options(scipen = 999) 

Participant level analyses

Using the package's built in dataset. A copy of this dataset is saved to the vignette folder, for those that want to know how data must be formatted to be analysed by this package.

Robust hypothesis testing via permuted p values, and 95% CIs on robust unstandardized (median change) and standardized effect sizes (Non-overlap all pairs/Ruscio's A and Hedge's g).

Baseline trends also identified using standardized beta regression coefficients.

# save copy of simlulated data that is bunled with the package
write_csv(simulated_data, "example_data.csv")

# analyse
results <- sced_analysis(data = simulated_data)

results %>%
  kable() %>%
  kable_styling(bootstrap_options = c("striped", "hover", "condensed"), full_width = FALSE)

Plot

SCED plots with raw data, intervention point (vertical dotted line), median score within phases (horizontal dashed lines), and trends (regression lines).

sced_plot(data = simulated_data)

Meta analysis

Random effects meta analysis of standardized effect sizes across participants, plotting a standard forest plot. Includes meta analysed effect sizes, 95% confidence intervals (estimates of true effect sizes), 95% credibility intervals (estimates of effect sizes likely to be observed in future participants), and heterogenity metrics (differences in effect between participants).

You can optionally exclude participants from the meta analysis using the baseline_trend_exclusion_criterion parameter. You can change which effect size is meta analysed using the effect_size parameter.

# meta analysis
meta_analysis <- sced_meta_analysis(results = results, effect_size = "ruscios_A")

# forest plot
metafor::forest(meta_analysis$model_fit,
                xlab = "Probability of superiority (Ruscio's A)",
                digits = 2,
                transf = SCED::logodds_to_probability,  # convert log odds back to probabilities (ie Ruscio's A)
                addcred = TRUE,
                refline = 0.5)

• r meta_analysis$meta_effect
• r meta_analysis$meta_heterogeneity

Simulate SCED data

In order to learn to use the package without using the data you wish to ultimately analyse (which can be a source of unintentional p hacking), SCED data can easily be simulated using the simulate_data function. The number of participants, timepoints at A and B, and true effect size can be set to arbitrary values.

# simulate data
data <- simulate_data(participants = 2,     # two participants
                      timepoints_a = 10,    # 10 timepoints pre 
                      timepoints_b = 10,    # 10 post intervention
                      cohens_d     = 2) %>% # Cohen's d = 2
  mutate(Participant = paste("Participant", Participant))

Power analysis for SCED meta analysis

Rough approximation of power for the between-participant meta analysis of effects, using equations derived from Valentine, Pigott, & Rothstein (2010), derived by Quintana (2017), and applied to within-subject SCED designs here.

yi: Meta-analyzed hedges' g effect size. NBs 1) Although Cohen's rules of thumb (.2 = small, .5 = medium, .8 = large), larger effect sizes are often observed in SCED due to the within subject designs (e.g., above 1.0). 2) Valentine's equations assume a between-groups design but here we apply to within-subjects effect sizes. Power is therefore likely to be underestimated. 3) Valentine's equations employ Cohen's d effect size but we use the Hedges' g effect size which has been bias corrected for small samples.

ni: The average number of data points per phase (i.e., in each of A and B, before or after the intervention). Assuming equal number of total time points for all participants, this is roughly equal to half the total number of time points. NB again, Valentine's equations assume a between-groups design but here we apply to within-subjects effect sizes. Power is therefore likely to be underestimated.

k: The number of participants.

tau^2: The tau^2 metric of between participant heterogeneity, as produced by the meta analysis function. If data from previous studies is not available, historically reccommended cutoffs are .33 for small, 1.0 for moderate, & 3.0 for large. Given that SCED studies have historically argued that the method possesses strong replicability, with each participant functioning as an experiment that is then replicated, it should be possible to split data collection into exploratory and confirmatory stages, and use data from the exploratory phase (e.g., first 5 participants) to determine the likely power that will be observed in the confirmatory phase (e.g., remaining 7 participants).

# returns power (1 - beta)
power_meta(yi = 0.40, 
           ni = 30,
           k  = 6,
           tau2 = 0.3)

ianhussey/SCED documentation built on Aug. 25, 2022, 4:42 p.m.