prob_signif_agree: Compute probability of "significance agreement" between...

In Replicate: Statistical Metrics for Multisite Replication Studies

Description

Given point estimates and their variances for one or multiple original studies and variances for one or more replication studies, returns a vector of probabilities that the replication estimate is "statistically significant" and in the same direction as the original. Can be computed assuming no heterogeneity or allowing for heterogeneity.

Usage

prob_signif_agree(yio, vio, vir, t2 = 0, null = 0, alpha = 0.05)

Arguments

yio	Effect estimate in the original study. Can be a vector for multiple original studies.
vio	Estimated variance of effect estimate in the original study (i.e., its squared standard error). Can be a vector for multiple original studies.
vir	Estimated variance of effect estimate in the replication study (i.e., its squared standard error). Can be a vector for multiple replication studies.
t2	Optionally (if allowing for heterogeneity), the estimated variance of true effects across replication studies.
null	Null value for the hypothesis tests.
alpha	Alpha level for the hypothesis tests.

References

1. Mathur MB & VanderWeele TJ (under review). New statistical metrics for multisite replication projects.

Examples

# replication estimates (Fisher's z scale) and SEs
# from moral credential example in Mathur & VanderWeele
# (under review)
yir = c(0.303, 0.078, 0.113, -0.055, 0.056, 0.073,
0.263, 0.056, 0.002, -0.106, 0.09, 0.024, 0.069, 0.074,
0.107, 0.01, -0.089, -0.187, 0.265, 0.076, 0.082)

seir = c(0.111, 0.092, 0.156, 0.106, 0.105, 0.057,
0.091, 0.089, 0.081, 0.1, 0.093, 0.086, 0.076,
0.094, 0.065, 0.087, 0.108, 0.114, 0.073, 0.105, 0.04)

# how many do we expect to agree?
sum( prob_signif_agree( yio = 0.21, vio = 0.004, vir = seir^2 ) )

Replicate documentation built on Dec. 1, 2019, 1:14 a.m.