In davidnexer/elmnet: Fast Regularized and Pruned Extreme Learning Machines

knitr::opts_chunk$set(echo = TRUE, dpi = 120, fig.align = "center", fig.width=5, fig.height=3)

Nonlinear Regression and Classification with Regularized and Pruned Extreme Learning Machines

This is a BETA release and for now it works only for classification problems.

The elmnet function implements a tuning free regularized learner based on Extreme Learning Machines (ELMs) [@huang-2011]. It uses Generalized Cross Validation (GCV) [@golub-1979], a fast and efficient leave-one-out approach, to automatically define the best regularization parameter.

So, elmnet is a fast and easy to use nonlinear learner. Moreover, it uses a softmax function on the output layer to predict calibrated probabilities.

How To Install?

library('devtools')
install_github(repo = "davpinto/elmnet")

Required Packages

The following packages are required to make elmnet work properly. All of them will be automatically instaled when you install elmnet.

magrittr to use the pipe operator %>%,
matrixStats for fast row-wise and column-wise matrix operations,
doSNOW and foreach to train multiclass models in parallel. The elmnet package uses all available cores by default,
ggplot2 to plot classification decision boundaries.

Regularization Methods

Ridge Regression
Lasso (in development)
Elastic-net (in development)

Pruning Methods

Fast Pruned ELM (P-ELM) [@rong-2008] (in development)
Optimally Pruned ELM (OP-ELM) [@miche-2010] (in development)
Double-Regularized ELM (TROP-ELM) [@miche-2011] (in development)

Toy Examples

Classification decision boudary for nonlinear problems.

# Load packages
library('elmnet')
library('caTools')

Two-class example

# Load toy data
data('spirals', package = 'elmnet')
x <- spirals$x
y <- spirals$y

# Split data
set.seed(111)
tr.idx <- caTools::sample.split(y, SplitRatio = 0.7)
x.tr <- x[tr.idx,]
x.te <- x[!tr.idx,]
y.tr <- y[tr.idx]
y.te <- y[!tr.idx]

# Fit ELM model
set.seed(111)
elm.model <- elmnet(x.tr, y.tr, nodes = 300, standardize = TRUE)

# Draw classification decision boudary
elmDecisionBound(elm.model, x.te, y.te, resamp = 150)

Multi-class example

# Load toy data
data('multi_spirals', package = 'elmnet')
x <- multi_spirals$x
y <- multi_spirals$y

# Split data
set.seed(222)
tr.idx <- caTools::sample.split(y, SplitRatio = 0.7)
x.tr <- x[tr.idx,]
x.te <- x[!tr.idx,]
y.tr <- y[tr.idx]
y.te <- y[!tr.idx]

# Fit ELM model
set.seed(222)
elm.model <- elmnet(x.tr, y.tr, nodes = 300, standardize = TRUE)

# Draw classification decision boudary
elmDecisionBound(elm.model, x.te, y.te, resamp = 150)