Getting started with the LTG-SMD framework

In ltgsmd: Latent True-Score and Target-Population Anchored Geometric SMD

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.width = 6,
  fig.height = 4
)
library(ltgsmd)
set.seed(20240501)

Overview

This vignette walks through the three workflows of the companion paper:

1. Single-study analysis with an internal holdout reference (Example 1 in Section 6.1, mathematics-learning intervention).
2. Single-study analysis with an external reference distribution (Example 2 in Section 6.2, Open Psychometrics IPIP Big Five).
3. Multi-site analysis with cross-site pooled reference (Example 3 in Section 6.3, Many Labs 2 Anderson.1 effect).

The companion paper is "The Denominator Chooses the Estimand: A Target-Population True-Score Framework for Standardized Mean Differences" (Nakamura, 2026, Psychological Methods, https://doi.org/10.1037/met0000875).

1. Single-study analysis with a holdout reference

The simplest workflow uses an internal holdout split (Strategy A of Section 8.2 in the paper). The study sample provides the mean difference; the holdout sample provides reliability and variance estimates for the target reference.

# Simulated example
n <- 60; k <- 4
study_df <- data.frame(condition = rep(c("control", "treatment"), each = n))
for (i in 1:k) {
  eff <- ifelse(study_df$condition == "treatment", 0.5, 0)
  study_df[[paste0("item", i)]] <- rnorm(2 * n, mean = eff)
}
study_df$outcome <- rowMeans(study_df[, paste0("item", 1:k)])

# Holdout reference
ref_df <- data.frame(condition = rep(c("control", "treatment"), each = n))
for (i in 1:k) {
  eff <- ifelse(ref_df$condition == "treatment", 0.5, 0)
  ref_df[[paste0("item", i)]] <- rnorm(2 * n, mean = eff)
}
ref_df$outcome <- rowMeans(ref_df[, paste0("item", 1:k)])

result <- compute_ltg_smd(
  study_data     = study_df,
  reference_data = ref_df,
  group_var      = "condition",
  score_var      = "outcome",
  items          = paste0("item", 1:k),
  group_levels   = c(reference = "control", focal = "treatment")
)

print(result)

Confidence intervals

ci <- ltg_smd_ci(
  result,
  method         = c("analytic", "bootstrap"),
  boot_type      = "bc",
  B              = 2000,
  seed           = 20240501,
  study_data     = study_df,
  reference_data = ref_df,
  group_var      = "condition",
  score_var      = "outcome",
  items          = paste0("item", 1:k),
  group_levels   = c(reference = "control", focal = "treatment")
)
print(ci)

The analytic delta-method interval conditions on the reliability point estimate (Section 4.2 of the paper, $\widehat{\mathrm{SE}}_{\rho\text{-fixed}}$). The bias-corrected nonparametric bootstrap propagates uncertainty in the reliability estimate as well as in the variances. Section 8.3 of the paper recommends reporting at least one justified interval; both can be reported, with one designated as primary.

Denominator diagnostics (Table F.1 format)

diag <- denominator_diagnostics(result)
print(diag)

The diagnostic table reports group-level sample sizes, observed standard deviations, reference reliabilities, true-score standard deviations, study-to-reference ratios $c_g$, and the two decomposition factors: the reliability factor $(\rho_1 \rho_0)^{1/4}$ and the study-to-target factor $(c_1 c_0)^{-1/2}$. Their product is the predicted ratio of the within-study observed geometric SMD to the LTG-SMD (exact under the central decomposition of Section 3).

Denominator sensitivity (six symmetric denominators)

sens <- denominator_sensitivity(result)
print(sens)

When group-specific reference true-score variances are similar, all six denominators yield nearly identical estimates. When variances differ markedly, the estimates differ, reflecting that different denominators define different estimands (Sections 2.5 and 5.3 of the paper).

2. Single-study analysis with an external reference

When a separately defined external reference is available (Strategy B), pass the external sample as reference_data. The mean difference still comes from the study sample, but reliability and variance now come from the external population.

# Suppose 'study_df' is a focused US 18-30 subsample and 'norm_df' is
# the rest of an Open Psychometrics dataset serving as the broader
# reference distribution.

result_external <- compute_ltg_smd(
  study_data     = study_df,
  reference_data = norm_df,
  group_var      = "gender",
  score_var      = "neuroticism_score",
  items          = c("N1", "N2", "N3", "N4"),
  group_levels   = c(reference = "Male", focal = "Female")
)
print(result_external)

3. Multi-site analysis with cross-site pooled reference

For coordinated multi-site projects (Strategy C), use multisite_ltg_smd(), which automates per-site computation and meta-analytic pooling via metafor::rma().

ml2_result <- multisite_ltg_smd(
  data               = ml2_data,
  site_var           = "Source.Global",
  group_var          = "condition",
  score_var          = "SWB",
  items              = paste0("and_item_", 1:25),
  reference_strategy = "pooled_across_sites",
  min_n_per_group    = 50
)
print(ml2_result)

The output includes:

• site_table: site-level Hedges's $g$ and LTG-SMD with within-site SEs.
• reference_summary: pooled reference reliabilities, variances, and true-score geometric SD.
• meta_hedges and meta_ltg: full metafor::rma() outputs.
• meta_comparison: side-by-side meta-analytic summary.

As Sections 6.3 and 7.2 of the paper discuss, when within-site denominators vary symmetrically around the target-population denominator, pooled estimates and heterogeneity statistics can be similar between Hedges's $g$ and LTG-SMD even though site-level effect-size values diverge by $\pm 20\%$ or more.

4. Sensitivity to the target reference

# Compare three plausible references for an analysis
sens_ref <- sensitivity_reference(
  object                  = result,
  alternative_references  = list(
    norm_alt1 = ref_df_alt1,
    norm_alt2 = ref_df_alt2
  ),
  study_data   = study_df,
  group_var    = "condition",
  score_var    = "outcome",
  items        = paste0("item", 1:k),
  group_levels = c(reference = "control", focal = "treatment")
)
print(sens_ref)

5. Exporting a supplementary appendix

export_supplementary(result, ci = ci, file = "supplementary_appendix.md")

This writes a Markdown file with the diagnostic table, denominator sensitivity profile, and confidence intervals, ready for inclusion in supplementary materials.

Decision rule: when does the LTG-SMD framework matter?

Section 8.1 of the paper proposes three approximate regimes:

| Regime | Conditions | LTG-SMD vs Hedges's $g$ | |---|---|---| | 1 | High reliability ($\rho > 0.85$ in both groups), $c_g$ within about 10% of 1 | Roughly 10% or less | | 2 | Moderate reliability (0.65 – 0.85) or $c_g$ departing from 1 by about 10 – 30% | 10 – 25% | | 3 | Low reliability ($\rho < 0.65$) or $c_g$ departing from 1 by more than about 30% | $> 25\%$ |

The most precise diagnostic is the product of the reliability factor and the study-to-target factor in equation (5), reported in result$factors. When this product is within 5% of unity, the two estimators agree closely; when it deviates by more than 25%, they answer materially different questions.

References

The package is companion software to:

Nakamura, D. (2026). The denominator chooses the estimand: A target-population true-score framework for standardized mean differences. Psychological Methods. Advance online publication. https://doi.org/10.1037/met0000875

Run citation("ltgsmd") to obtain this reference in text and BibTeX form.

ltgsmd documentation built on Sept. 27, 2026, 5:07 p.m.