View source: R/fs_fdr_report.R
| fs_fdr_report | R Documentation |
Characterizes the operating false-discovery rate (FDR) of a fitted
forestsearch analysis and tests whether requiring harm
confirmation on the de-biased effect reduces it. This is a measurement
tool, not a calibration tool:
it does not solve for thresholds that control an FDR target. For the
(c_1, c_2) pair the analysis actually used, it reports two quantities
under a null:
fs_fdr_report(
fs_analysis,
df_analysis,
null = c("hom", "harm"),
c_confirm = c(0.9, 1, 1.25),
nsims = 1000L,
n = NULL,
confounders = NULL,
n_super = 25000L,
analysis_time = 84,
cens_adjust = log(1.5),
event_min = 5L,
n_workers = NULL,
seed_base = 8316951L,
quiet = FALSE
)
fs_analysis |
A fitted |
df_analysis |
The analyst's data frame. Used only to default the
simulated sample size ( |
null |
Character vector; which null(s) to simulate. |
c_confirm |
Numeric vector of de-biased-HR harm-confirmation
thresholds at which to report B, i.e. the values |
nsims |
Integer; null replicates per null. Default 1000. |
n |
Integer or |
confounders |
Character vector or |
n_super |
Integer; super-population size for the DGM. Default 25000.
Under |
analysis_time |
Numeric; administrative censoring time (months). Default 84. |
cens_adjust |
Numeric; censoring shift passed to
|
event_min |
Integer; minimum events for a replicate to be scored. Replicates below this (or with a single treatment arm) are counted as visible failures, never silently dropped. Default 5. |
n_workers |
Integer or |
seed_base |
Integer; base seed. Each null uses a disjoint seed block and each replicate an explicit seed, so results are plan-invariant. Default 8316951. |
quiet |
Logical; suppress progress and per-null messages. Default
|
A (raw FDR): A = P(\text{declare any harm subgroup} \mid c_1, c_2).
B(g) (confirmed FDR): B(g) = P(\text{declare AND de-biased } HR \ge g).
Because harm confirmation can only remove declarations, B(g) \le A
always; the gap A - B(g) is what the de-biasing step buys at
confirmation threshold g.
All thresholds and identifier knobs are inherited from
fs_analysis$args_call_all, so the FDR reported is that of the analysis
as run. Multiplier resampling is forced on internally
(mr_inference = TRUE) regardless of the fitted setting, because B
requires the per-replicate de-biased HR.
Under "hom" the treatment effect is real but uniform, so any declared
subgroup is a manufactured heterogeneity – the winner's-curse-of-search
rate. Under "harm" a real modifier exists and the flagged region
sits exactly at HR = 1.0, so a small upward fluctuation reads as harm; this
is the more adversarial null. The declared cut need not overlap
flag_harm: under either null any declaration is a false discovery.
B(g) counts a declaration only when its de-biased HR is finite and clears
g; a declaration whose de-biased HR could not be computed counts as
unconfirmed. Such cases are counted in n_deb_na and reported, so
that A - B is never mistaken for pure de-biasing benefit when part
of it is de-biased-HR unavailability.
Identifier knobs (thresholds, subgroup_method, consistency_method,
conf_force, conf.cont_jcuts, maxk, MR arguments, ...)
transfer verbatim from args_call_all. The data-binding arguments –
the five column names and the confounder set – are rebound to the DGM,
because simulate_from_dgm() emits fixed names
(y_sim / event_sim / treat_sim / id / flag_harm). mr_inference
is forced TRUE so B is computable.
An object of class "fs_fdr_report": a list with
fdrData frame, one row per null x c_confirm, with
A, A_lo, A_hi (raw FDR and 95% Wilson CI),
B, B_lo, B_hi (confirmed FDR and CI), reduction
(A - B), and the accounting columns n_valid,
n_declared, n_deb_na, fs_errors, unhandled.
hr_structureData frame of the realized hazard ratios each DGM encodes – the structural check that the nulls are different worlds.
verdict"CLEAN" if no replicate failed across any
null, else "WARNING".
n_errorsTotal failed replicates (forestsearch errors + unhandled).
c1, c2, n, nsims, nullsThe inherited thresholds and run settings.
call_argsReproducibility metadata.
This function builds its null data through the superseded GBSG wrapper
(setup_gbsg_dgm, with calibrate_k_inter for
the "harm" null), in which the censoring model is fitted to the
resampled super-population and its covariate term is multiplied by the
ratio of the event-model scale to the censoring-model scale; this changes
the simulated censoring times, and with them the rates in fdr, but
not the true hazard ratios in hr_structure.
forestsearch (and its vocabulary section),
fs_mr_inference for the MR step this forces on,
mr_estimates_table,
forestsearch_bootstrap_dofuture for the full bootstrap,
fpr_calibration,
setup_gbsg_dgm, calibrate_k_inter,
simulate_from_dgm
## Not run:
# The GBSG trial with time in months and the column names used below
gbsg_df <- within(survival::gbsg, {
id <- pid
time <- rfstime / 30.4375
treat <- hormon
})
# A fitted survival/Cox analysis (thresholds and knobs live on the object)
fit <- forestsearch(
df.analysis = gbsg_df,
outcome.name = "time", event.name = "status",
treat.name = "treat", id.name = "id",
confounders.name = c("er", "age", "meno", "pgr", "nodes", "size", "grade"),
is.RCT = TRUE, est.scale = "hr",
subgroup_method = "consistency", consistency_method = "resample",
hr.threshold = 1.25, hr.consistency = 1.0,
pconsistency.threshold = 0.90, maxk = 2,
mr_inference = TRUE)
# Characterize its operating FDR and whether harm confirmation helps
rep <- fs_fdr_report(fit, gbsg_df, nsims = 1000)
rep # prints the realized-HR table and the A/B table
rep$fdr # one row per null x c_confirm
# Reproduce the published reference (n = 700): A ~ 0.138 (hom), 0.278 (harm)
fs_fdr_report(fit, gbsg_df, n = 700, nsims = 1000)
## End(Not run)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.