performance: performance

In AndrewLawrence/dCVnet: doubly cross-validated elastic-net regularised generalised linear models

performance

Description

Extracts the elements needed to calculate prediction performance for an object. These elements are returned with a standardised format, and the prediction performance measures can be calculated by calling the generic summary() function on the result.

Default function behaviour assumes input is a dCVnet object (and fails if this is not reasonable).

For a dCVnet object the outer-loop cross-validated performance is returned.

Applying performance to a performance object allows conversion between list and dataframe format.

For glm objects performance wraps predict.glm if newdata is specified.

Usage

performance(x, ...)

## Default S3 method:
performance(x, ...)

## S3 method for class 'dCVnet'
performance(x, as.data.frame = TRUE, ...)

## S3 method for class 'performance'
performance(x, as.data.frame = TRUE, ...)

## S3 method for class 'glm'
performance(
  x,
  as.data.frame = TRUE,
  label = deparse(substitute(x)),
  threshold = 0.5,
  newdata = NULL,
  ...
)

## S3 method for class 'glmlist'
performance(x, as.data.frame = TRUE, ...)

Arguments

x	an object from which prediction performance can be extracted.
...	arguments to pass on
as.data.frame	return a data.frame instead of a list of performance objects.
label	specify a label for the output
threshold	for binomial family glm - use specified threshold for predicting class from probability.
newdata	evaluate performance in new data

Details

Prediction performance measures differ for each model family. See InternalPerformanceSummaryFunctions.

Performance is always calculated at the level of CV-repeats. dCVnet does not report the fold-to-fold variability in CV performance.

Value

a performance object, is a dataframe (or list of dataframes) with the following columns:

• reference - the known 'true' class of the observation

• prediction - the model prediction for a case. for dCVnet this is the result of predict(model, type = "response") With "binary" response predictions are the predicted probability of the non-reference class (used for AUROC)

• classification - for binomial and multinomial models this is the predicted class assigned by the model

• label - a grouping variable when predictions come from more than one source, e.g. multiple reps

Examples

## Not run: 

data(QuickStartExample, package = "glmnet")
m <- dCVnet(QuickStartExample$y,
            QuickStartExample$x, family = "gaussian")

# a performance 'object'
performance(m)

# Performance for each repeat of the outer-loop repeated k-fold:
summary(performance(m))

# The cross-validated performance measures:
p <- report_performance_summary(m)
subset(p, select = c("Measure", "mean"))


## End(Not run)

AndrewLawrence/dCVnet documentation built on Sept. 24, 2024, 5:24 a.m.