calibrate_plot: Calibration plot

In gbm-developers/gbm3: Generalized Boosted Regression Models

Calibration plot

Description

An experimental diagnostic tool that plots the fitted values versus the actual average values. If distribution is "Bernoulli" or "Poisson", then the predictions are converted to the response scale (probability or rate). For all other distributions, the calibration plot uses least squares and predicts an expected value.

Usage

calibrate_plot(
  y,
  p,
  distribution = "Bernoulli",
  replace = TRUE,
  line.par = list(col = "black"),
  shade_col = "lightyellow",
  shade_density = NULL,
  rug.par = list(side = 1),
  xlab = "Predicted value",
  ylab = "Observed average",
  xlim = NULL,
  ylim = NULL,
  knots = NULL,
  df = 6,
  ...
)

Arguments

y	the outcome 0-1 variable
p	the predictions estimating E(y|x)
distribution	the loss function used in creating p. Bernoulli and Poisson are currently the only special options. All others default to squared error assuming Gaussian
replace	determines whether this plot will replace or overlay the current plot. replace=FALSE is useful for comparing the calibration of several methods
line.par	graphics parameters for the line
shade_col	color for shading the 2 SE region. shade_col=NA implies no 2 SE region
shade_density	the density parameter for polygon
rug.par	graphics parameters passed to rug
xlab	x-axis label corresponding to the predicted values
ylab	y-axis label corresponding to the observed average
xlim, ylim	x and y-axis limits. If not specified the function will select limits
knots, df	these parameters are passed directly to ns for constructing a natural spline smoother for the calibration curve
...	other graphics parameters passed on to the plot function

Details

Uses natural splines to estimate E(y|p). Well-calibrated predictions imply that E(y|p) = p. The plot also includes a pointwise 95

Value

calibrate.plot returns no values.

Author(s)

James Hickey, Greg Ridgeway [e-mail]

References

J.F. Yates (1982). "External correspondence: decomposition of the mean probability score," Organisational Behaviour and Human Performance 30:132-156.

D.J. Spiegelhalter (1986). "Probabilistic Prediction in Patient Management and Clinical Trials," Statistics in Medicine 5:421-433.

Examples

dataSim <- data.frame(x=rnorm(1000))
dataSim$y <- with(dataSim, rbinom(1000, 1, 1/(1+exp(-(x-0.5*x^2)))))

# showing poor calibration of a linear model
glm1 <- glm(y~x, data=dataSim, family=binomial)
p <- predict(glm1, type="response")
calibrate_plot(dataSim$y, p, xlim=c(0,1), ylim=c(0,1))

# showing better calibration with quadratic
glm1 <- glm(y~poly(x,2), data=dataSim, family=binomial)
p <- predict(glm1,type="response")
calibrate_plot(dataSim$y, p, xlim=c(0,1), ylim=c(0,1))

gbm-developers/gbm3 documentation built on April 28, 2024, 10:04 p.m.