Methods Overview

In artma: Automatic Replication Tools for Meta-Analysis

Introduction

artma ships a set of runtime methods: the analytical functions that artma() runs on your data. This vignette describes what each method does, what it depends on, and what it returns, so you can pick the right ones for your analysis. For a hands-on introduction to running methods, see the Getting Started vignette.

List the methods available in your installed version at any time with:

artma::methods_list()

That prints one row per method: what it does, the data columns it needs, the methods that run before it, the optional packages it uses (with an install <pkg> status when one is missing), and whether it is opt-in. It also returns the same information as a data frame, invisibly. Pass your data to check the required columns before a run, so you see which methods would be skipped rather than finding out afterwards:

methods <- artma::methods_list(available_for = my_data)
methods[!methods$available, c("method", "missing_columns", "missing_packages")]

The tables below carry the same metadata; a test keeps them in step with what the methods register.

How methods execute

Each method is a plain function registered with register_runtime_method(), which attaches declarative metadata: a one-line description, a depends_on list of upstream methods, a required_columns list of data columns it needs, and a suggests list of optional packages it needs. methods_list() reads that metadata back, so the printed overview cannot fall out of step with what the methods declare.

When you call artma(methods = ...), the requested methods are topologically sorted by their depends_on edges, so a method that builds on another method's output (for example, best_practice_estimate builds on bma) always runs after it and receives its result as a <dependency>_result argument. Ties preserve discovery order; dependency cycles abort the run.

Before each method runs, its required_columns are checked against your data and its suggests packages against what's installed. A method that fails either check is skipped with an explanation instead of aborting the whole run, the only exception being a non-interactive run that requested exactly that one method with a missing suggested package, which aborts with a clear error. A method that throws an error is likewise caught and skipped; the run continues, and skipped/failed methods are reported at the end (as the failed_methods attribute on the returned list).

Every method returns a tables/plots/meta triple: tables are exported as CSV, plots are available for programmatic access and printing, and meta holds anything else (fitted models, fit parameters, skip reasons).

Descriptive and exploratory methods

These methods summarize or visualize the data; none depend on another method.

| Method | What it does | Required columns | | --- | --- | --- | | effect_summary_stats | Summary statistics (mean, weighted mean, CIs, median, SD) of the main effect, grouped by variables flagged in the data config | effect, study_size, study_id | | variable_summary_stats | Descriptive statistics (mean, median, min, max, SD, missingness) for each variable flagged for summary in the data config | none | | funnel_plot | Funnel plot of effect against precision, for spotting publication bias/asymmetry, with configurable outlier filtering | effect, precision | | box_plot | Box plots of the effect grouped by a categorical variable, auto-splitting into multiple plots when there are many groups | effect | | prima_facie_graphs | Density/histogram overlays of the effect distribution split by detected categorical groups (e.g. published vs. unpublished) | effect | | t_stat_histogram | Histograms of the t-statistic distribution, with a full-range and a zoomed-in view, plus significance reference lines | t_stat |

Publication-bias diagnostics

These methods test for publication bias and selective reporting; none depend on another method.

| Method | What it does | Required columns | Packages | | --- | --- | --- | --- | | linear_tests | Linear funnel-asymmetry regressions of effect on standard error (FAT-PET): OLS, panel Fixed/Between/Random Effects, and study-size/precision-weighted variants | effect, se, study_id | none | | nonlinear_tests | Non-linear publication-bias corrections: WAAP, Top10, STEM (funnel and MSE variants), a hierarchical Bayesian model, a selection-model (p-uniform-style) estimator, and the endogenous kink test | effect, se, study_id | none | | exogeneity_tests | Diagnostics that relax the exogeneity assumption: instrumental-variable regression and the p-uniform* test | effect, se, study_id, n_obs, study_size | AER | | p_hacking_tests | Caliper tests around significance thresholds and the Elliott et al. (2022) battery | effect, se, t_stat, study_id | none | | maive | The MAIVE estimator: corrects the mean effect for publication bias, p-hacking, and spurious precision by instrumenting reported variances with the inverse sample size | effect, se, n_obs | MAIVE |

maive needs MAIVE version 0.2.4 or newer.

Moderator and heterogeneity analysis

These methods examine how the effect varies with moderator variables. fma and best_practice_estimate both build on bma.

| Method | What it does | Required columns | Depends on | Packages | Opt-in | | --- | --- | --- | --- | --- | --- | | bma | Bayesian Model Averaging over moderator variables, estimating posterior inclusion probability and posterior mean/SD for each candidate moderator | effect, se | none | BMS | no | | fma | Frequentist Model Averaging over the same moderators, using the BMA model (computed on demand if not already available) to order and select predictors | effect, se | bma | BMS, quadprog | no | | best_practice_estimate | A "best-practice" point estimate and CI for the effect, plugging literature-informed or user-supplied moderator values into the BMA coefficients; also computes economic-significance metrics | effect, study_id | bma | BMS | no | | robma | Robust Bayesian meta-analysis (model-averaged publication-bias correction) via the RoBMA package; run it by requesting it by name | effect, se | none | RoBMA | yes |

If you request both bma and fma (directly or via a dependency) and both produce coefficient tables, artma adds a unified ma_table entry to the results combining them.

When moderators are selected automatically, bma always adds the standard error (se) and sample size (study_size) to the moderator set as priority variables; the standard-error term acts as a publication-bias control inside BMA by convention. They show up in the Model Averaging results alongside your own moderators, and a note is printed during variable selection.

Automatic selection also excludes columns detected as derived encodings of the effect or its standard error, such as winsorized copies of the effect, t-statistics, or inverse standard errors, which many published datasets ship alongside the raw variables. Keeping one would make BMA regress the effect on a transform of itself, hiding every genuine moderator. Detection is based on rank and outlier-clipped correlations with the effect, the SE, 1/se, and effect/se (threshold 0.9); excluded columns are listed in a warning. If such a column is configured explicitly (bma: true), it is kept but warned about. To keep one on purpose without warnings, set bma_allow_derived: true on its entry in the data config.

robma needs the RoBMA package, which in turn requires a working JAGS installation (via rjags). On Windows, CRAN's RoBMA source can fail to compile against JAGS 4.3.1; if RoBMA is not installed, the method is skipped with an explanation rather than aborting the run.

Choosing methods

Methods whose required columns are missing from your data, or whose suggested packages aren't installed, are skipped with an explanation rather than aborting the run, so it's safe to request methods = "all" even if your data or R environment doesn't support every method:

# Run everything artma supports for your data
results <- artma(methods = "all", options = "my_analysis.yaml")

# Run a specific combination
results <- artma(methods = c("funnel_plot", "bma", "fma"), options = "my_analysis.yaml")

To see the skips in advance, hand your prepared data to methods_list():

methods <- artma::methods_list(available_for = my_data)
subset(methods, !available)[, c("method", "missing_columns", "missing_packages")]

See the Getting Started vignette for the full workflow, and the Understanding Options Files vignette for how to configure each method's parameters.

artma documentation built on Oct. 27, 2026, 9:06 a.m.