Description Usage Arguments Details Value Author(s) References See Also Examples
Calibration weights require specification of tuning parameter delta or lambda. Since a single round of crossvalidation can be noisy, crossvalidation can be repeated multiple times with independent random partitions and the results be averaged. This function implements a repeated Kfold crossvalidation where tuning parameter labmda or delta is selected by maximizing standardized net benefit (sNB) (i.e. repeated cvWtTuning
procedure).
A a "onestandard error" rule can be used for selecting tuning parameters. Under the “onestandard error" rule the calibration weight tuning parameter (lambda or delta) is selected such that corresponding crossvalidated sNB is within onestandard deviation of the maximum crossvalidated sNB. This provides protection against overfitting the data and selecting a tuning parameter that is too extreme. If the "onestandard error" rule is not implemented, then the tuning parameter with the larged average crossvalidted sNB (across folds and repetition) will be selected.
1  cvRepWtTuning(y,p,r,rl,ru,kFold=5,cvRep=25,cvParm,tuneSeq,stdErrRule=TRUE,int.seed=11111)

y 
Vector of binary outcomes, with 1 indicating event (cases) and 0 indicating no event (controls) 
p 
Vector of risk score values 
r 
Clinically relevant risk threshold 
rl 
Lower bound of clinically relevant region 
ru 
Upper bound of clinically relevant region 
kFold 
Number of folds for crossvalidation 
cvRep 
Number of crossvalidation repititions 
cvParm 
Parameter to be selected via crossvalidation. Can be either delta the weight assigned to observations outside the clinically relevant region [R_l,R_u], or the lambda tuning parameter controlling exponential decay within the clinically relevant region [R_l,R_u] 
tuneSeq 
Sequence of values of tuning parameters to perform crossvalidation over 
stdErrRule 
Use "onestandard" error rule selecting tuning parameter 
int.seed 
Intial seed set for random splitting of data into K folds 
To estimate the standard deviation of the crossvalidated sNV, the dependence between the different partitions of crossvalidation needs to be accounted for. Gelman (1992) give a variance estimator of convergence diagnostic statistic used when Markov Chain Monte Carlo with multiple chains are performed. The variance estimator accounts for both the variability of the statistic “within" a single chain, and the variance of the statistic across, or “between", chains. Analogously, we can use this framework to estimate the “within" repetition variance (i.e. variation in sNB from a single round of Kfold crossvalidation) and the “between" repetition variance. We denote the ‘within" repetition variance as W and the “between" repetition variance as B . We augment this formula slightly from that given in Gelman (1992) to account for the fact that as the number of crossvalidation repetitions increases, the betweenrepetition variability should decrease. See Mishra et al (2020) for full expressions of B and W.
cv.sNB 
Standardized net benefit (sNB) of tuning parameter selected via crossvalidatoin 
cv.RAW 
Corresponding RAW value given crossvaliated selected tuning parameter 
cv.lambda 
lambda value selected via crossvalidation if cvParm=lambda, otherwise user specified lambda value 
cv.delta 
delta value selected via crossvalidation if cvParm=delta, otherwise user specified lambda value 
avgCV.res 
Averaged (acrossreplications) crossvalidated sNB for sequence of tuning parameters 
W 
Estimate of "within" repetition variance. Will only return if stdErrRule==TRUE 
B 
Estimate of "between" repetition variance. Will only return if stdErrRule==TRUE 
fullList 
List of crossvaliation results for all fold and repititions 
Anu Mishra
Mishra, A. (2019). Methods for Risk Markers that Incorporate Clinical Utility (Doctoral dissertation). (Available Upon Request)
Friedman, J., Hastie, T., & Tibshirani, R. (2001). The elements of statistical learning (Vol. 1, No. 10). New York: Springer series in statistics.
Gelman, A., & Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statistical science, 7(4), 457472.
calWt
,
RAWgrid
,
nb
,
cvWtTuning
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21  ### Load data ##
## Not run:
data(fakeData)
### Get grid of tuning parameters ###
grid < RAWgrid(r = 0.3,rl = Inf,ru = Inf,p = fakeData$p,y = fakeData$y,
cvParm = "lambda",rl.raw = 0.25,ru.raw = 0.35)
### Implement repeated kfold cross validation
repCV < cvRepWtTuning(y = fakeData$y,p = fakeData$p,rl = Inf,ru = Inf,r = 0.3,
kFold = 5,cvRep = 25,cvParm = "lambda",tuneSeq = grid,stdErrRule = TRUE)
## crossvalidation results
repCV$avgCV.res
## crossvalidation selected lambda, RAW, and sNV
cv.lambda < repCV$cv.lambda
cv.RAW < repCV$cv.RAW
cv.RAW < repCV$cv.sNB
## End(Not run)

Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.