errorMetrics: Performance metrics for categorical response models

Description Usage Arguments Details Value See Also Examples

View source: R/serp.performance.R

Description

Calculates the performance metrics of fitted binary and multi-categorical response models. Available measures include: brier score, logloss and misclassification error.

Usage

1
2
3
4
5
errorMetrics(
             actual,
             predicted,
             type= c("brier", "logloss", "misclass"),
             control = list())

Arguments

actual

vector of actual values observed

predicted

predicted probability matrix of a categorical model or a vector of fitted values for binary models.

type

specifies the type of error metrics.

control

A list of control parameters to replace default values returned by serp.control. 'misclass.thresh' resets the default misclassification error threshold, while 'minP' assigns a near-zero constant value to the predicted values beyond certain threshold, to forestall chances of numerical problems.

Details

A numeric value of the selected error type determining how good a categorical model compares to competing models.

Value

value

the value of error measure computed.

type

the error measure used: any of brier, logLoss or misclassification error.

threshold

the misclassification threshold.

See Also

serp, anova.serp, confint.serp, vcov.serp

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
require(serp)

m1 <- serp(rating ~ temp + contact, slope = "parallel", link = "logit", data = wine)
errorMetrics(m1, type = "brier")

## objects of class other than \code{serp} require the actual
## observations with corresponding predicted values supplied.

set.seed(2)
n <- 50
y <- as.factor(rbinom(n, 1, 0.5))
m2 <- glm(y ~ rnorm(n), family = binomial())
ft <- m2$fitted.values

errorMetrics(actual=y, predicted=ft, type = "logloss")
errorMetrics(actual=y, predicted=ft, type = "misclass")

## Reset classification threshold
errorMetrics(actual=y, predicted=ft, type = "misclass",
control = list(misclass.thresh=0.4))

serp documentation built on Nov. 8, 2021, 1:08 a.m.