performanceEstimation: An Infra-Structure for Performance Estimation of Predictive Models
Version 1.1.0

An infra-structure for estimating the predictive performance of predictive models. In this context, it can also be used to compare and/or select among different alternative ways of solving one or more predictive tasks. The main goal of the package is to provide a generic infra-structure to estimate the values of different metrics of predictive performance using different estimation procedures. These estimation tasks can be applied to any solutions (workflows) to the predictive tasks. The package provides easy to use standard workflows that allow the usage of any available R modeling algorithm together with some pre-defined data pre-processing steps and also prediction post- processing methods. It also provides means for addressing issues related with the statistical significance of the observed differences.

AuthorLuis Torgo [aut, cre]
Date of publication2016-10-13 20:37:05
MaintainerLuis Torgo <ltorgo@dcc.fc.up.pt>
LicenseGPL (>= 2)
Version1.1.0
URL https://github.com/ltorgo/performanceEstimation
Package repositoryView on CRAN
InstallationInstall the latest version of this package by entering the following in R:
install.packages("performanceEstimation")

Popular man pages

bootEstimates: Performance estimation using (e0 or .632) bootstrap
CDdiagram.BD: CD diagrams for the post-hoc Boferroni-Dunn test
CDdiagram.Nemenyi: CD diagrams for the post-hoc Nemenyi test
knnImp: Fill in NA values with the values of the nearest neighbours
loocvEstimates: Performance estimation using Leave One Out Cross Validation
smote: SMOTE algorithm for unbalanced classification problems
standardPOST: A function for applying post-processing steps to the...
See all...

All man pages Function index File listing

Man pages

bootEstimates: Performance estimation using (e0 or .632) bootstrap
Bootstrap-class: Class "Bootstrap"
CDdiagram.BD: CD diagrams for the post-hoc Boferroni-Dunn test
CDdiagram.Nemenyi: CD diagrams for the post-hoc Nemenyi test
classificationMetrics: Calculate some standard classification evaluation metrics of...
ComparisonResults-class: Class "ComparisonResults"
CV-class: Class "CV"
cvEstimates: Performance estimation using cross validation
EstCommon-class: Class '"EstCommon"'
EstimationMethod-class: Class "EstimationMethod"
EstimationResults-class: Class "EstimationResults"
estimationSummary: Obtain a set of descriptive statistics of the scores of a...
EstimationTask-class: Class '"EstimationTask"'
getIterationsInfo: Obtaining the information returned by a workflow when applied...
getIterationsPreds: Obtaining the predictions returned by a workflow when applied...
getScores: Obtaining the metric scores on the different iterations for a...
getWorkflow: Obtain the workflow object corresponding to an ID
hldEstimates: Performance estimation using holdout and random resampling
Holdout-class: Class "Holdout"
is.classification: Check if a certain predictive task is a classification...
is.regression: Check if a certain predictive task is a regression problem
knnImp: Fill in NA values with the values of the nearest neighbours
LOOCV-class: Class "LOOCV"
loocvEstimates: Performance estimation using Leave One Out Cross Validation
mcEstimates: Performance estimation for time series prediction tasks using...
mergeEstimationRes: Merging several 'ComparisonResults' class objects
metricNames: The evaluation metrics estimated in an experiment
metricsSummary: Obtains a summary of the individual metric scores obtained by...
MonteCarlo-class: Class "MonteCarlo"
pairedComparisons: Statistical hypothesis testing on the observed paired...
performanceEstimation: Estimate the predictive performance of modeling alternatives...
PredTask-class: Class "PredTask"
rankWorkflows: Provide a ranking of workflows involved in an estimation...
regressionMetrics: Calculate some standard regression evaluation metrics of...
responseValues: Obtain the target variable values of a prediction task
results2table: Obtains a dplyr data frame table object containing all the...
runWorkflow: Run a workflow on a predictive task
signifDiffs: Obtains a list with the set of paired differences that are...
smote: SMOTE algorithm for unbalanced classification problems
standardPOST: A function for applying post-processing steps to the...
standardPRE: A function for applying data pre-processing steps
standardWF: A function implementing a standard workflow for prediction...
subset-methods: Methods for Function 'subset' in Package...
taskNames: The prediction tasks involved in an estimation experiment
timeseriesWF: A function implementing sliding and growing window standard...
topPerformer: Obtain the workflow that best performed in terms of a metric...
topPerformers: Obtain the best scores from a performance estimation...
Workflow-class: Class "Workflow"
workflowNames: The IDs of the workflows involved in an estimation experiment
workflowVariants: Generate (parameter) variants of a workflow

Functions

Bootstrap Man page Source code
Bootstrap-class Man page
CDdiagram.BD Man page Source code
CDdiagram.Nemenyi Man page Source code
CV Man page Source code
CV-class Man page
ComparisonResults Man page Source code
ComparisonResults-class Man page
EstCommon-class Man page
EstimationMethod Man page
EstimationMethod-class Man page
EstimationResults Man page Source code
EstimationResults-class Man page
EstimationTask Man page Source code
EstimationTask-class Man page
Holdout Man page Source code
Holdout-class Man page
LOOCV Man page Source code
LOOCV-class Man page
MonteCarlo Man page Source code
MonteCarlo-class Man page
PredTask Man page Source code
PredTask-class Man page
Workflow Man page Source code
Workflow-class Man page
bootEstimates Man page Source code
classificationMetrics Man page Source code
cvEstimates Man page Source code
estimationSummary Man page Source code
getIterationsInfo Man page Source code
getIterationsPreds Man page Source code
getScores Man page Source code
getWorkflow Man page Source code
hldEstimates Man page Source code
is.classification Man page Source code
is.regression Man page Source code
knnImp Man page Source code
loocvEstimates Man page Source code
mcEstimates Man page Source code
mergeEstimationRes Man page Source code
metricNames Man page Source code
metricsSummary Man page Source code
outFold Source code
pairedComparisons Man page Source code
performanceEstimation Man page Source code
plot,ComparisonResults-method Man page
plot,EstimationResults-method Man page
rankWorkflows Man page Source code
regressionMetrics Man page Source code
responseValues Man page Source code
results2table Man page Source code
runWorkflow Man page Source code
scores2long Source code
scores2summary Source code
scoresIts Source code
show,Bootstrap-method Man page
show,CV-method Man page
show,ComparisonResults-method Man page
show,EstimationResults-method Man page
show,EstimationTask-method Man page
show,Holdout-method Man page
show,LOOCV-method Man page
show,MonteCarlo-method Man page
show,PredTask-method Man page
show,Workflow-method Man page
signifDiffs Man page Source code
smote Man page Source code
smote.exs Source code
standardPOST Man page Source code
standardPRE Man page Source code
standardWF Man page Source code
statScores Source code
subset,ANY-method Man page
subset,ComparisonResults-method Man page Man page
subset-methods Man page
summary,ComparisonResults-method Man page
summary,EstimationResults-method Man page
summary,Workflow-method Man page
taskNames Man page Source code
timeseriesWF Man page Source code
topPerformer Man page Source code
topPerformers Man page Source code
workflowNames Man page Source code
workflowVariants Man page Source code

Files

inst
inst/CITATION
inst/doc
inst/doc/performanceEstimation.pdf
inst/doc/performanceEstimation.html
inst/doc/performanceEstimation.R
inst/doc/performanceEstimation.Rmd
NAMESPACE
R
R/workflows.R
R/resultsManipulation.R
R/resultsAnalysis.R
R/methods.R
R/experiments.R
R/classDefs.R
R/smote.R
R/evaluationMetrics.R
MD5
DESCRIPTION
man
man/workflowNames.Rd
man/CV-class.Rd
man/results2table.Rd
man/CDdiagram.Nemenyi.Rd
man/standardPOST.Rd
man/responseValues.Rd
man/bootEstimates.Rd
man/standardPRE.Rd
man/taskNames.Rd
man/ComparisonResults-class.Rd
man/runWorkflow.Rd
man/signifDiffs.Rd
man/timeseriesWF.Rd
man/hldEstimates.Rd
man/regressionMetrics.Rd
man/topPerformers.Rd
man/CDdiagram.BD.Rd
man/mergeEstimationRes.Rd
man/metricsSummary.Rd
man/subset-methods.Rd
man/classificationMetrics.Rd
man/LOOCV-class.Rd
man/metricNames.Rd
man/getWorkflow.Rd
man/getScores.Rd
man/smote.Rd
man/Holdout-class.Rd
man/is.regression.Rd
man/PredTask-class.Rd
man/pairedComparisons.Rd
man/standardWF.Rd
man/EstimationTask-class.Rd
man/MonteCarlo-class.Rd
man/Bootstrap-class.Rd
man/loocvEstimates.Rd
man/workflowVariants.Rd
man/cvEstimates.Rd
man/knnImp.Rd
man/mcEstimates.Rd
man/getIterationsInfo.Rd
man/topPerformer.Rd
man/Workflow-class.Rd
man/is.classification.Rd
man/rankWorkflows.Rd
man/EstimationMethod-class.Rd
man/getIterationsPreds.Rd
man/performanceEstimation.Rd
man/EstimationResults-class.Rd
man/EstCommon-class.Rd
man/estimationSummary.Rd
CHANGES
performanceEstimation documentation built on May 19, 2017, 5:09 p.m.

Questions? Problems? Suggestions? Tweet to @rdrrHQ or email at ian@mutexlabs.com.

Please suggest features or report bugs in the GitHub issue tracker.

All documentation is copyright its authors; we didn't write any of that.