mape_stability: Mean absolute predictor error (MAPE) stability plot

In pminternal: Internal Validation of Clinical Prediction Models

Mean absolute predictor error (MAPE) stability plot

Description

A MAPE (in)stability plot shows mean absolute predictor error (average absolute difference between original estimated risk and risk from B bootstrap models) as a function of apparent estimated risk (prediction from original/development model). See Riley and Collins (2023).

Usage

mape_stability(x, xlim, ylim, xlab, ylab, pch, cex, col, subset, plot = TRUE)

Arguments

x	an object produced by validate with method = "boot_*" (or boot_optimism with method="boot")
xlim	x limits (default = range of estimated risks)
ylim	y limits (default = c(0, maximum mape))
xlab	a title for the x axis
ylab	a title for the y axis
pch	plotting character (default = 16)
cex	controls point size (default = 1)
col	color of points (default = grDevices::grey(.5, .5))
subset	vector of observations to include (row indices). This can be used to select a random subset of observations.
plot	if FALSE just returns MAPE values (see value)

Value

plots calibration (in)stability. Invisibly returns a list containing individual and average MAPE.

References

Riley, R. D., & Collins, G. S. (2023). Stability of clinical prediction models developed using statistical or machine learning methods. Biometrical Journal, 65(8), 2200302. doi:10.1002/bimj.202200302

Examples

set.seed(456)
# simulate data with two predictors that interact
dat <- pmcalibration::sim_dat(N = 2000, a1 = -2, a3 = -.3)
mean(dat$y)
dat$LP <- NULL # remove linear predictor

# fit a (misspecified) logistic regression model
m1 <- glm(y ~ ., data=dat, family="binomial")

# internal validation of m1 via bootstrap optimism with 10 resamples
# B = 10 for example but should be >= 200 in practice
m1_iv <- validate(m1, method="boot_optimism", B=10)

mape_stability(m1_iv)

pminternal documentation built on April 4, 2025, 3:30 a.m.