metrics: Prediction Evaluation

In PlayerRatings: Dynamic Updating Methods for Player Ratings Estimation

Description

Returns measures that assess prediction performance.

Usage

metrics(act, pred, cap = c(0.01,0.99), which = 1:3, na.rm = TRUE, 
  sort = TRUE, digits = 3, scale = TRUE)

Arguments

act	A numeric vector of actual values. Typically equal to one for a player one win, zero for a player two win, and one half for a draw.
pred	A numeric vector of predictions, typically values between zero and one. A matrix can also be given, in which case the jth column contains the predictions for model j.
cap	A numeric vector of length two giving values at which to cap the binomial deviance.
which	Select metrics using any subset of 1:3. All are produced by default.
na.rm	Remove missing values in predictions. The default is to remove missing values because the default predict method will predict missing values for games with new players.
sort	By default output is ordered from best to worst using the first metric specified.
digits	Round to this number of digits.
scale	If TRUE (the default), all metrics are scaled so that a value of 100 corresponds to predicting 0.5 for every game.

Details

The preferred metric for assessing predictions in chess is the capped binomial deviance. Mean squared error and mean absolute error metrics are also produced. By default all metrics are scaled so that the value 100 represents the zero information case. If not scaled, then all metrics are multiplied by 100.

Value

A numeric vector.

See Also

predict.rating

Examples

afl <- aflodds[,c(2,3,4,7)]
train <- afl[afl$Week <= 80,]
test <- afl[afl$Week > 80,]
robj <- elo(train)
metrics(test$Score, predict(robj, test))
metrics(test$Score, predict(robj, test), scale = FALSE)

Example output

  bdev    mse    mae 
96.096 97.213 90.358 
  bdev    mse    mae 
66.609 48.366 44.733 

PlayerRatings documentation built on March 1, 2020, 5:07 p.m.