panelWise: Panel-wise measures of predictive accuracy

In DudbridgeLab/multipred: Calculates measures of accuracy for risk predictors of multiple outcomes.

Panel-wise measures of predictive accuracy

Description

Calculates panel-wise sensitivity, specificity, positive and negative predictive value, concordance and relative utility for a vector of predictors.

Usage

panelWise(
  x,
  y,
  thresh = NULL,
  prev0 = NULL,
  prev1 = NULL,
  condprev0 = NULL,
  condprev1 = NULL,
  nsample = NULL
)

Arguments

x	Matrix of predicted risks. Each row corresponds to an individual, each column to an outcome. Each entry should be a risk between 0 and 1.
y	Matrix of outcomes. Each row corresponds to an individual, each column to an outcome. Must contain binary outcomes coded as 0 and 1.
thresh	Vector of risk thresholds. For each row of x, each outcome is predicted to occur for which the risk exceeds the corresponding element of thresh. These predictions are then compared to the elements of y. If NULL, which is the default, concordance is the only measure that can be calculated.
prev0	Probability of at least one non-event, required for calculating relative utility. If NULL, which is the default, then prev0 is estimated from the y matrix, ignoring ascertainment.
prev1	Probability of at least one event, required for calculating relative utility. If NULL, which is the default, then prev1 is estimated from the y matrix, ignoring ascertainment.
condprev0	Probability of at least one non-event, conditional on the risk predictions being equal to thresh. If NULL, which is the default, then prev is set to 1 - the product of the elements of thresh. This working definition is exact when predictions and outcomes both are jointly independent.
condprev1	Probability of at least one event, conditional on the risk predictions being equal to thresh. If NULL, which is the default, then prev is set to 1 - the product of the elements of (1-thresh). This working definition is exact when predictions and outcomes both are jointly independent.
nsample	Number of random pairs of samples drawn when estimating concordance. If NULL, which is the default, concordance is not calculated. If 0, all possible pairs are drawn from the data.

Details

Panel-wise measures consider the prediction of at least one outcome to occur. At least one outcome that did occur must be predicted to occur. For example, panel-wise sensitivity is the probability that, for an individual in which at least one outcome did occur, the predicted risk exceeds the threshold for at least one of the outcomes that did occur. Panel-wise specificity is the probability that, for an individual in which at least one outcome did not occur, the predicted risk is lower than the threshold for all the outcomes that did not occur.

Panel-wise concordance is the probability that given one individual in which at least one outcome did occur, and another in which at least one did not occur, the maximum predicted risk over all outcomes that occurred in the former is higher than the maximum over all outcomes that did not occur in the latter. Note that under this definition an individual can be either concordant or discordant with itself. Concordance is calculated by randomly drawing such pairs of individuals from y. If nsample is zero, all such pairs are drawn from y; this might be time-consuming. Therefore the default is not to calculate condcordance. However, a good estimate of concordance can be obtained from a limited number of random samples nsample.

prev0, prev1, condprev0 and condprev1 are only required to calculate relative utility, and can be omitted otherwise.

Value

A list with the following components

sens Sensitivity

spec Specificity

PPV Positive predictive value

NPV Negative predictive value

C Concordance

RU Relative utility

Examples

attach(PRSdata)
panelWise(risk,disease,thresh=prevalence,nsample=1e5)

# $sens
# [1] 0.6266996

# $spec
# [1] 0.0701

# $PPV
# [1] 0.1073925

# $NPV
# [1] 0.9311696

# $C
# [1] 0.47999

# $RU
# [1] -5.120519

DudbridgeLab/multipred documentation built on Sept. 30, 2022, 2:24 a.m.