measures: Performance measures.

Description Usage Arguments Format References See Also

Description

A performance measure is evaluated after a single train/predict step and returns a single number to assess the quality of the prediction (or maybe only the model, think AIC). The measure itself knows whether it wants to be minimized or maximized and for what tasks it is applicable.

All supported measures can be found by listMeasures or as a table in the tutorial appendix: http://mlr-org.github.io/mlr-tutorial/release/html/measures/.

If you want a measure for a misclassification cost matrix, look at makeCostMeasure. If you want to implement your own measure, look at makeMeasure.

Most measures can directly be accessed via the function named after the scheme measureX (e.g. measureSSE).

For clustering measures, we compact the predicted cluster IDs such that they form a continuous series starting with 1. If this is not the case, some of the measures will generate warnings.

Usage

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
featperc

timetrain

timepredict

timeboth

sse

measureSSE(truth, response)

mse

measureMSE(truth, response)

rmse

measureRMSE(truth, response)

medse

measureMEDSE(truth, response)

sae

measureSAE(truth, response)

mae

measureMAE(truth, response)

medae

measureMEDAE(truth, response)

rsq

measureRSQ(truth, response)

expvar

measureEXPVAR(truth, response)

arsq

rrse

measureRRSE(truth, response)

rae

measureRAE(truth, response)

mape

measureMAPE(truth, response)

msle

measureMSLE(truth, response)

rmsle

kendalltau

measureKendallTau(truth, response)

spearmanrho

measureSpearmanRho(truth, response)

mmce

measureMMCE(truth, response)

acc

measureACC(truth, response)

ber

multiclass.aunu

measureAUNU(probabilities, truth)

multiclass.aunp

measureAUNP(probabilities, truth)

multiclass.au1u

measureAU1U(probabilities, truth)

multiclass.au1p

measureAU1P(probabilities, truth)

multiclass.brier

measureMulticlassBrier(probabilities, truth)

logloss

measureLogloss(probabilities, truth)

ssr

measureSSR(probabilities, truth)

qsr

measureQSR(probabilities, truth)

lsr

measureLSR(probabilities, truth)

kappa

measureKAPPA(truth, response)

wkappa

measureWKAPPA(truth, response)

auc

measureAUC(probabilities, truth, negative, positive)

brier

measureBrier(probabilities, truth, negative, positive)

brier.scaled

measureBrierScaled(probabilities, truth, negative, positive)

bac

measureBAC(truth, response, negative, positive)

tp

measureTP(truth, response, positive)

tn

measureTN(truth, response, negative)

fp

measureFP(truth, response, positive)

fn

measureFN(truth, response, negative)

tpr

measureTPR(truth, response, positive)

tnr

measureTNR(truth, response, negative)

fpr

measureFPR(truth, response, negative, positive)

fnr

measureFNR(truth, response, negative, positive)

ppv

measurePPV(truth, response, positive, probabilities = NULL)

npv

measureNPV(truth, response, negative)

fdr

measureFDR(truth, response, positive)

mcc

measureMCC(truth, response, negative, positive)

f1

gmean

measureGMEAN(truth, response, negative, positive)

gpr

measureGPR(truth, response, positive)

multilabel.hamloss

measureMultilabelHamloss(truth, response)

multilabel.subset01

measureMultilabelSubset01(truth, response)

multilabel.f1

measureMultiLabelF1(truth, response)

multilabel.acc

measureMultilabelACC(truth, response)

multilabel.ppv

measureMultilabelPPV(truth, response)

multilabel.tpr

measureMultilabelTPR(truth, response)

cindex

meancosts

mcp

db

dunn

G1

G2

silhouette

Arguments

truth

[factor]
Vector of the true class.

response

[factor]
Vector of the predicted class.

probabilities

[numeric | matrix]
a) For purely binary classification measures: The predicted probabilities for the positive class as a numeric vector. b) For multiclass classification measures: The predicted probabilities for all classes, always as a numeric matrix, where columns are named with class labels.

negative

[character(1)]
The name of the negative class.

positive

[character(1)]
The name of the positive class.

Format

none

References

He, H. & Garcia, E. A. (2009) Learning from Imbalanced Data. IEEE Transactions on Knowledge and Data Engineering, vol. 21, no. 9. pp. 1263-1284.

See Also

Other performance: ConfusionMatrix, calculateConfusionMatrix, calculateROCMeasures, estimateRelativeOverfitting, makeCostMeasure, makeCustomResampledMeasure, makeMeasure, performance



Search within the mlr package
Search all R packages, documentation and source code

Questions? Problems? Suggestions? or email at ian@mutexlabs.com.

Please suggest features or report bugs with the GitHub issue tracker.

All documentation is copyright its authors; we didn't write any of that.