Brier: The Brier Score

In SurvMetrics: Predictive Evaluation Metrics in Survival Analysis

The Brier Score

Description

The Brier Score was proposed by Glenn W. Brier in 1950 which is a proper score function that measures the accuracy of probabilistic predictions, usually used to measure the accuracy of a model fit for survival data. Brier can calculate the value of Brier Score at any timepoint, regardless of whether it is the event time.

Usage

Brier(object, pre_sp, t_star = -1)

Arguments

object	object of class Surv created by Surv function or a fitted survival model, including the survival model fitted by coxph, survreg, and rfsrc.
pre_sp	If the input of the parameter object is a fitted survival model, this parameter should be a survival dataset on which you want to caculate the Brier Score of the fitted model. Or with an object of class Surv as the parameter object, it should be a vector of predicted values of survival probabilities of each observation in testing set at time t_star.
t_star	the timepoint at which the Brier Score you want to calculate. If the input of the parameter object is a fitted survival model, the timepoint is necessary to be specified at which the survival probability is predicted, and this function will calculate the Brier Score at that moment. If the input object is a survival object, this parameter can be ignored and the value of this parameter will not have any effect on the result of this fuction.

Details

The Brier Score is the mean square difference between the true classes and the predicted probabilities. So the Brier Score can be thought of as a cost function. Therefore, the lower the Brier Score is for a set of predictions, the better the predictions are calibrated. The Brier Score takes on a value between zero and one, since this is the square of the largest possible difference between a predicted probability and the actual outcome. As we all know, for the cencoring samples, we do not know the real time of death, so the residual cannot be directly calculated when making the prediction. So the Brier Score is widely used in survival analysis.

The Brier Score is a strictly proper score (Gneiting and Raftery, 2007), which means that it takes its minimal value only when the predicted probabilities match the empirical probabilities.

Judging from the sparse empirical evidence, predictions of duration of survival tend to be rather inaccurate. More precision is achieved by using patient-specific survival probabilities and the Brier score as predictions to discriminate future survivors from failures.

Value

the Brier Score at time t_star between the true classes and the predicted probabilities.

Author(s)

Hanpu Zhou [e-mail]

References

Graf, Erika, Schmoor, Claudia, Sauerbrei, & Willi, et al. (1999). Assessment and comparison of prognostic classification schemes for survival data. Statist. Med., 18(1718), 2529-2545.

Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78.

Gneiting, T. , & Raftery, A. E. . (2007). Strictly Proper Scoring Rules, Prediction, and Estimation.

Examples

library(survival)
time <- rexp(50)
status <- sample(c(0, 1), 50, replace = TRUE)
pre_sp <- runif(50)
t_star <- runif(1)
Brier(Surv(time, status), pre_sp, t_star)

SurvMetrics documentation built on Sept. 4, 2022, 1:06 a.m.