CSMES.ensSel: CSMES Training Stage 1: Cost-Sensitive Multicriteria Ensemble...

In CSMES: Cost-Sensitive Multi-Criteria Ensemble Selection for Uncertain Cost Conditions

CSMES Training Stage 1: Cost-Sensitive Multicriteria Ensemble Selection resulting in a Pareto frontier of candidate ensemble classifiers

Description

This function applies the first stage in the learning process of CSMES: optimizing Cost-Sensitive Multicriteria Ensemble Selection, resulting in a Pareto frontier of equivalent candidate ensemble classifiers along two objective functions. By default, cost space is optimized by optimizing false positive and false negative rates simultaneously. This results in a set of optimal ensemble classifiers, varying in the tradeoff between FNR and FPR. Optionally, other objective metrics can be specified. Currently, only binary classification is supported.

Usage

CSMES.ensSel(
  memberPreds,
  y,
  obj1 = c("FNR", "AUCC", "MSE", "AUC"),
  obj2 = c("FPR", "ensSize", "ensSizeSq", "clAmb"),
  selType = c("selection", "selectionWeighted", "weighted"),
  plotting = TRUE,
  generations = 30,
  popsize = 100
)

Arguments

memberPreds	matrix containing ensemble member library predictions
y	Vector with true class labels. Currently, a dichotomous outcome variable is supported
obj1	Specifies the first objective metric to be minimized
obj2	Specifies the second objective metric to be minimized
selType	Specifies the type of ensemble selection to be applied: "selection" for basic selection, "selectionWeighted" for weighted selection, "weighted" for weighted sum
plotting	TRUE or FALSE: Should a plot be generated showing objective function values throughout the optimization process?
generations	the number of population generations for nsga-II. Default is 30.
popsize	the population size for nsga-II. Default is 100.

Value

An object of the class CSMES.ensSel which is a list with the following components:

weights	ensemble member weights for all pareto-optimal ensemble classifiers after multicriteria ensemble selection
obj_values	optimization objective values
pareto	overview of pareto-optimal ensemble classifiers
popsize	the population size for nsga-II
generarations	the number of population generations for nsga-II
obj1	Specifies the first objective metric that was minimized
obj2	Specifies the second objective metric that was minimized
selType	the type of ensemble selection that was applied: "selection", "selectionWeighted" or "weighted"
ParetoPredictions_p	probability predictions for pareto-optimal ensemble classifiers
ParetoPredictions_c	class predictions for pareto-optimal ensebmle classifiers

Author(s)

Koen W. De Bock, [e-mail]

References

De Bock, K.W., Lessmann, S. And Coussement, K., Cost-sensitive business failure prediction when misclassification costs are uncertain: A heterogeneous ensemble selection approach, European Journal of Operational Research (2020), doi: 10.1016/j.ejor.2020.01.052.

Examples

##load data
library(rpart)
library(zoo)
library(ROCR)
library(mco)
data(BFP)
##generate random order vector
BFP_r<-BFP[sample(nrow(BFP),nrow(BFP)),]
size<-nrow(BFP_r)
##size<-300
train<-BFP_r[1:floor(size/3),]
val<-BFP_r[ceiling(size/3):floor(2*size/3),]
test<-BFP_r[ceiling(2*size/3):size,]
##generate a list containing model specifications for 100 CART decisions trees varying in the cp
##and minsplit parameters, and trained on bootstrap samples (bagging)
rpartSpecs<-list()
for (i in 1:100){
  data<-train[sample(1:ncol(train),size=ncol(train),replace=TRUE),]
  str<-paste("rpartSpecs$rpart",i,"=rpart(as.formula(Class~.),data,method=\"class\",
  control=rpart.control(minsplit=",round(runif(1, min = 1, max = 20)),",cp=",runif(1,
  min = 0.05, max = 0.4),"))",sep="")
  eval(parse(text=str))
}
##generate predictions for these models
hillclimb<-mat.or.vec(nrow(val),100)
for (i in 1:100){
  str<-paste("hillclimb[,",i,"]=predict(rpartSpecs[[i]],newdata=val)[,2]",sep="")
  eval(parse(text=str))
}
##score the validation set used for ensemble selection, to be used for ensemble selection
ESmodel<-CSMES.ensSel(hillclimb,val$Class,obj1="FNR",obj2="FPR",selType="selection",
generations=10,popsize=12,plot=TRUE)
## Create Ensemble nomination curve
enc<-CSMES.ensNomCurve(ESmodel,hillclimb,val$Class,curveType="costCurve",method="classPreds",
plot=FALSE)

CSMES documentation built on Feb. 16, 2023, 10:09 p.m.