Description Usage Arguments Details Value Author(s) References See Also Examples
Computes Ustatistics to test for whether predictor X1 is more
concordant than predictor X2, extending rcorr.cens
. For
method=1
, estimates the fraction of pairs for which the
x1
difference is more impressive than the x2
difference. For method=2
, estimates the fraction of pairs for
which x1
is concordant with S
but x2
is not.
For binary responses the function improveProb
provides several
assessments of whether one set of predicted probabilities is better
than another, using the methods describe in
Pencina et al (2007). This involves NRI and IDI to test for
whether predictions from model x1
are significantly different
from those obtained from predictions from model x2
. This is a
distinct improvement over comparing ROC areas, sensitivity, or
specificity.
1 2 3 4 5 6  rcorrp.cens(x1, x2, S, outx=FALSE, method=1)
improveProb(x1, x2, y)
## S3 method for class 'improveProb'
print(x, digits=3, conf.int=.95, ...)

x1 
first predictor (a probability, for 
x2 
second predictor (a probability, for 
S 
a possibly rightcensored 
outx 
set to 
method 
see above 
y 
a binary 0/1 outcome variable 
x 
the result from 
digits 
number of significant digits for use in printing the result of

conf.int 
level for confidence limits 
... 
unused 
If x1
,x2
represent predictions from models, these
functions assume either that you are using a separate sample from the
one used to build the model, or that the amount of overfitting in
x1
equals the amount of overfitting in x2
. An example
of the latter is giving both models equal opportunity to be complex so
that both models have the same number of effective degrees of freedom,
whether a predictor was included in the model or was screened out by a
variable selection scheme.
Note that in the first part of their paper, Pencina et al. presented measures that required binning the predicted probabilities. Those measures were then replaced with better continuous measures that are implementedhere.
a vector of statistics for rcorrp.cens
, or a list with class
improveProb
of statistics for improveProb
:
n 
number of cases 
na 
number of events 
nb 
number of nonevents 
pup.ev 
mean of pairwise differences in probabilities for those with events and a pairwise difference of \mbox{probabilities}>0 
pup.ne 
mean of pairwise differences in probabilities for those without events and a pairwise difference of \mbox{probabilities}>0 
pdown.ev 
mean of pairwise differences in probabilities for those with events and a pairwise difference of \mbox{probabilities}>0 
pdown.ne 
mean of pairwise differences in probabilities for those without events and a pairwise difference of \mbox{probabilities}>0 
nri 
Net Reclassification Index = (pup.evpdown.ev)(pup.nepdown.ne) 
se.nri 
standard error of NRI 
z.nri 
Z score for NRI 
nri.ev 
Net Reclassification Index = pup.evpdown.ev 
se.nri.ev 
SE of NRI of events 
z.nri.ev 
Z score for NRI of events 
nri.ne 
Net Reclassification Index = pup.nepdown.ne 
se.nri.ne 
SE of NRI of nonevents 
z.nri.ne 
Z score for NRI of nonevents 
improveSens 
improvement in sensitivity 
improveSpec 
improvement in specificity 
idi 
Integrated Discrimination Index 
se.idi 
SE of IDI 
z.idi 
Z score of IDI 
Frank Harrell
Department of Biostatistics, Vanderbilt University
fh@fharrell.com
Scott Williams
Division of Radiation Oncology
Peter MacCallum Cancer Centre, Melbourne, Australia
scott.williams@petermac.org
Pencina MJ, D'Agostino Sr RB, D'Agostino Jr RB, Vasan RS (2008): Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond. Stat in Med 27:157172. DOI: 10.1002/sim.2929
Pencina MJ, D'Agostino Sr RB, D'Agostino Jr RB, Vasan RS: Rejoinder: Comments on Integrated discrimination and net reclassification improvementsPractical advice. Stat in Med 2007; DOI: 10.1002/sim.3106
Pencina MJ, D'Agostino RB, Steyerberg EW (2011): Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat in Med 30:1121; DOI: 10.1002/sim.4085
rcorr.cens
, somers2
,
Surv
, val.prob
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18  set.seed(1)
library(survival)
x1 < rnorm(400)
x2 < x1 + rnorm(400)
d.time < rexp(400) + (x1  min(x1))
cens < runif(400,.5,2)
death < d.time <= cens
d.time < pmin(d.time, cens)
rcorrp.cens(x1, x2, Surv(d.time, death))
#rcorrp.cens(x1, x2, y) ## no censoring
set.seed(1)
x1 < runif(1000)
x2 < runif(1000)
y < sample(0:1, 1000, TRUE)
rcorrp.cens(x1, x2, y)
improveProb(x1, x2, y)

Loading required package: lattice
Loading required package: survival
Loading required package: Formula
Loading required package: ggplot2
Attaching package: 'Hmisc'
The following objects are masked from 'package:base':
format.pval, round.POSIXt, trunc.POSIXt, units
Dxy S.D. x1 more concordant x2 more concordant
8.212107e02 1.370738e01 4.589395e01 5.410605e01
n missing uncensored Relevant Pairs
4.000000e+02 0.000000e+00 1.100000e+01 4.262000e+03
Uncertain C X1 C X2 Dxy X1
1.553380e+05 9.920225e01 9.258564e01 9.840450e01
Dxy X2
8.517128e01
Dxy S.D. x1 more concordant x2 more concordant
1.077556e03 3.658641e02 5.005388e01 4.994612e01
n missing uncensored Relevant Pairs
1.000000e+03 0.000000e+00 1.000000e+03 4.992780e+05
Uncertain C X1 C X2 Dxy X1
0.000000e+00 4.871354e01 4.861620e01 2.572915e02
Dxy X2
2.767596e02
Analysis of Proportions of Subjects with Improvement in Predicted Probability
Number of events: 481 Number of nonevents: 519
Proportions of Positive and Negative Changes in Probabilities
Proportion
Increase for events (1) 0.516
Increase for nonevents (2) 0.495
Decrease for events (3) 0.484
Decrease for nonevents (4) 0.505
Net Reclassification Improvement
Index SE Z 2P Lower 0.95 Upper 0.95
NRI (13+42) 0.04082 0.0633 0.645 0.519 0.0832 0.1648
NRI for events (13) 0.03119 0.0456 0.684 0.494 0.0581 0.1205
NRI for nonevents (42) 0.00963 0.0439 0.219 0.826 0.0764 0.0957
Analysis of Changes in Predicted Probabilities
Mean Change in Probability
Increase for events (sensitivity) 0.01056
Decrease for nonevents (specificity) 0.00832
Integrated Discrimination Improvement
(average of sensitivity and 1specificity over [0,1];
also is difference in Yates' discrimination slope)
IDI SE Z 2P Lower 0.95 Upper 0.95
0.00224 0.02658 0.08423 0.93288 0.05434 0.04986
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.