cv_risk_y_nloglik: Cross-validated negative log-likelihood for evaluating...

In benkeser/cvma: Cross-validation-based maximal association

Description

In general, the function passed to y_weight_control$cv_risk should expect a list of outcomes and predictions in validation folds, in addition to a list called y_weight that contains the outcome weights (computed in the training sample) corresponding to this validation fold and any other information needed by y_weight_control$cv_risk (e.g., anything needed to compute confidence intervals – in this case the marginal mean of the composite outcome in the training sample). The function should return a list with names cv_measure, ci_low, ci_high, and p_value. The output of this function is returned irrespective of the names of the list; however, the names are necessary for print methods to work properly.

Usage

cv_risk_y_nloglik(input, y_weight_control)

Arguments

input	A list where each entry corresponds to a validation fold. Each entry is a list with entries: Y (matrix of outcomes for this validation fold), pred (matrix of super learner predictions for each outcomes with columns corresponding to different outcomes).
y_weight_control	Composite outcome weight control options.

Details

Confidence intervals are computed for negative log-likelihood. The p-value is for the one-sided hypothesis test that cross-validated negative log-likelihood is less than -log(0.5), which is what the value of negative log-likelihood if every observation were predicted to have Y = 1 with probability 0.5.

Value

List with named components cv_measure (cross-validated negative log-liklihood), ci_low (lower 100(1 - y_weight_control$alpha)% CI), ci_high (upper 100(1 - y_weight_control$alpha)% CI), p_value

Examples

# simulate data with proper format
input <- list(list(Y = cbind(rbinom(50,1,0.5), rbinom(50,1,0.5)), 
                   pred = cbind(runif(50,0,1), runif(50,0,1)),
                   y_weight = list(weight = c(0.5, 0.5), ybar=0.5)),
              list(Y = cbind(rbinom(50,1,0.5), rbinom(50,1,0.5)),
                   pred = cbind(runif(50,0,1), runif(50,0,1)),
                   y_weight = list(weight = c(0.25, 0.75), ybar=0.5)))
                   
# linear combination of outcomes
y_weight_control <- list(ensemble_fn = "ensemble_linear")

# get risk   
cv_risk <- cv_risk_y_nloglik(input, y_weight_control)                                       

benkeser/cvma documentation built on May 5, 2019, 1:37 p.m.