screening: Screening measures of predictive accuracy

In DudbridgeLab/multipred: Calculates measures of accuracy for risk predictors of multiple outcomes.

Screening measures of predictive accuracy

Description

Calculates screening sensitivity, specificity, positive and negative predictive value, concordance and relative utility for a vector of predictors.

Usage

screening(x, y, thresh = NULL, prev = NULL, condprev = NULL, nsample = NULL)

Arguments

x	Matrix of predicted risks. Each row corresponds to an individual, each column to an outcome. Each entry should be a risk between 0 and 1.
y	Matrix of outcomes. Each row corresponds to an individual, each column to an outcome. Must contain binary outcomes coded as 0 and 1.
thresh	Vector of risk thresholds. For each row of x, each outcome is predicted to occur for which the risk exceeds the corresponding element of thresh. These predictions are then compared to the elements of y. If NULL, which is the default, concordance is the only measure that can be calculated.
prev	Probability of at least one event, required for calculating relative utility. If NULL, which is the default, then prev is estimated from the y matrix, ignoring ascertainment.
condprev	Probability of at least one events, conditional on the risk predictions being equal to thresh. If NULL, which is the default, then prev is set to 1- the product of the elements of (1-thresh). This working definition is exact when predictions and outcomes both are jointly independent.
nsample	Number of random pairs of samples drawn when estimating concordance. If NULL, which is the default, concordance is not calculated. If 0, all possible pairs are drawn from the data.

Details

Screening measures consider the prediction of at least one outcome to occur, without regard to whether the correct outcomes are predicted. For example, screening sensitivity is the probability that, for an individual in which at least one outcome did occur, the predicted risk exceeds the threshold for at least one outcome (but not necessary the ones that did occur). Screening specificity is the probability that, for an individual in which no outcomes did occur, the predicted risk is lower than the threshold for all outcomes.

Screening concordance is the probability that given one individual in which at least one outcome did occur, and another in which no outcomes did occur, the maximum predicted risk over all outcomes is higher in the former individual. It is calculated by randomly drawing such pairs of individuals from y. If nsample is zero, all such pairs are drawn from y; this might be time-consuming. Therefore the default is not to calculate concordance. However, a good estimate can be obtained from a limited number of random samples nsample.

prev and condprev are only required to calculate relative utility, and can be omitted otherwise.

Value

A list with the following components

sens Sensitivity

spec Specificity

PPV Positive predictive value

NPV Negative predictive value

C Concordance

RU Relative utility

Examples

attach(PRSdata)
screening(risk,disease,thresh=prevalence,nsample=1e5)

# $sens
# [1] 0.9653894

# $spec
# [1] 0.05989024

# $PPV
# [1] 0.1654311

# $NPV
# [1] 0.8996416

# $C
# [1] 0.59156

# $RU
# [1] -0.009099365

DudbridgeLab/multipred documentation built on Sept. 30, 2022, 2:24 a.m.