performance_pcp: Percentage of Correct Predictions

Description Usage Arguments Details Value References Examples

View source: R/performance_pcp.R

Description

Percentage of correct predictions (PCP) for models with binary outcome.

Usage

1
performance_pcp(model, ci = 0.95, method = "Herron", verbose = TRUE)

Arguments

model

Model with binary outcome.

ci

The level of the confidence interval.

method

Name of the method to calculate the PCP (see 'Details'). Default is "Herron". May be abbreviated.

verbose

Toggle off warnings.

Details

method = "Gelman-Hill" (or "gelman_hill") computes the PCP based on the proposal from Gelman and Hill 2017, 99, which is defined as the proportion of cases for which the deterministic prediction is wrong, i.e. the proportion where the predicted probability is above 0.5, although y=0 (and vice versa) (see also Herron 1999, 90).

method = "Herron" (or "herron") computes a modified version of the PCP (Herron 1999, 90-92), which is the sum of predicted probabilities, where y=1, plus the sum of 1 - predicted probabilities, where y=0, divided by the number of observations. This approach is said to be more accurate.

The PCP ranges from 0 to 1, where values closer to 1 mean that the model predicts the outcome better than models with an PCP closer to 0. In general, the PCP should be above 0.5 (i.e. 50\ Furthermore, the PCP of the full model should be considerably above the null model's PCP.

The likelihood-ratio test indicates whether the model has a significantly better fit than the null-model (in such cases, p < 0.05).

Value

A list with several elements: the percentage of correct predictions of the full and the null model, their confidence intervals, as well as the chi-squared and p-value from the Likelihood-Ratio-Test between the full and null model.

References

Examples

1
2
3
4
data(mtcars)
m <- glm(formula = vs ~ hp + wt, family = binomial, data = mtcars)
performance_pcp(m)
performance_pcp(m, method = "Gelman-Hill")

Example output

# Percentage of Correct Predictions from Logistic Regression Model

  Full model: 83.75% [70.96% - 96.53%]
  Null model: 50.78% [33.46% - 68.10%]

# Likelihood-Ratio-Test

  Chi-squared: 27.751
      p-value:  0.000

# Percentage of Correct Predictions from Logistic Regression Model

  Full model: 87.50% [76.04% - 98.96%]
  Null model: 56.25% [39.06% - 73.44%]

# Likelihood-Ratio-Test

  Chi-squared: 27.751
      p-value:  0.000

performance documentation built on Oct. 1, 2021, 5:08 p.m.