knitr::opts_chunk$set(echo = TRUE) library(NNS) library(data.table) data.table::setDTthreads(2L) options(mc.cores = 1) Sys.setenv("OMP_THREAD_LIMIT" = 2)
library(NNS) library(data.table) require(knitr) require(rgl)
NNS.reg
is a very robust regression technique capable of nonlinear regressions of continuous variables and classification tasks in machine learning problems.
We have extended the NNS.reg
applications per the use of an ensemble method of classification in NNS.boost
. In short, NNS.reg
is the base learner instead of trees.
One major advantage NNS.boost
has over tree based methods is the ability to seamlessly extrapolate beyond the current range of observations.
Popular boosting algorithms take a series of weak learning decision tree models, and aggregate their outputs. NNS
is also a decision tree of sorts, by partitioning each regressor with respect to the dependent variable. We can directly control the number of "splits" with the NNS.reg(..., order = , ...)
parameter.
We can see how NNS
partitions each regressor by calling the $rhs.partitions
output. You will notice that each partition is not an equal interval, nor of equal length, which differentiates NNS
from other bandwidth or tree-based techniques.
Higher dependence between a regressor and the dependent variable will allow for a larger number of partitions. This is determined internally with the NNS.dep
measure.
NNS.reg(iris[,1:4], iris[,5], residual.plot = FALSE, ncores = 1)$rhs.partitions
NNS.boost()
Through resampling of the training set and letting each iterated set of data speak for themselves (while paying extra attention to the residuals throughout), we can test various regressor combinations in these dynamic decision trees...only keeping those combinations that add predictive value. From there we simply aggregate the predictions.
NNS.boost
will automatically search for an accuracy threshold
from the training set, reporting iterations remaining and level obtained in the console. A plot of the frequency of the learning accuracy on the training set is also provided.
Once a threshold
is obtained, NNS.boost
will test various feature combinations against different splits of the training set and report back the frequency of each regressor used in the final estimate.
Let's have a look and see how it works. We use 140 random iris
observations as our training set with the 10 holdout observations as our test set. For brevity, we set epochs = 10, learner.trials = 10, folds = 1
.
NOTE: Base category of response variable should be 1, not 0 for classification problems when using NNS.boost(..., type = "CLASS")
.
test.set = 141:150 a = NNS.boost(IVs.train = iris[-test.set, 1:4], DV.train = iris[-test.set, 5], IVs.test = iris[test.set, 1:4], epochs = 10, learner.trials = 10, status = FALSE, balance = TRUE, type = "CLASS", folds = 1) a$results [1] 3 3 3 3 3 3 3 3 3 3 a$feature.weights Petal.Width Petal.Length Sepal.Length 0.4285714 0.4285714 0.1428571 a$feature.frequency Petal.Width Petal.Length Sepal.Length 3 3 1 mean( a$results == as.numeric(iris[test.set, 5]) ) [1] 1
A perfect classification, using the features weighted per the output above.
NNS.stack()
The NNS.stack()
routine cross-validates for a given objective function the n.best
parameter in the multivariate NNS.reg
function as well as the threshold
parameter in the dimension reduction NNS.reg
version. NNS.stack
can be used for classification via NNS.stack(..., type = "CLASS", ...)
.
For brevity, we set folds = 1
.
NOTE: Base category of response variable should be 1, not 0 for classification problems when using NNS.stack(..., type = "CLASS")
.
b = NNS.stack(IVs.train = iris[-test.set, 1:4], DV.train = iris[-test.set, 5], IVs.test = iris[test.set, 1:4], type = "CLASS", balance = TRUE, ncores = 1, folds = 1) b
$OBJfn.reg [1] 1 $NNS.reg.n.best [1] 1 $probability.threshold [1] 0.43875 $OBJfn.dim.red [1] 0.9798658 $NNS.dim.red.threshold [1] 0.93 $reg [1] 3 3 3 3 3 3 3 3 3 3 $reg.pred.int NULL $dim.red [1] 3 3 3 3 3 3 3 3 3 3 $dim.red.pred.int NULL $stack [1] 3 3 3 3 3 3 3 3 3 3 $pred.int NULL
mean( b$stack == as.numeric(iris[test.set, 5]) )
[1] 1
depth = "max"
will force all observations to be their own partition, forcing a perfect fit of the multivariate regression. In essence, this is the basis for a kNN
nearest neighbor type of classification.
n.best = 1
will use the single nearest neighbor. When coupled with depth = "max"
, NNS
will emulate a kNN = 1
but as the dimensions increase the results diverge demonstrating NNS
is less sensitive to the curse of dimensionality than kNN
.
extreme
will use the maximum or minimum threshold
obtained, and may result in errors if that threshold cannot be eclipsed by subsequent iterations.
If the user is so motivated, detailed arguments further examples are provided within the following:
Sys.setenv("OMP_THREAD_LIMIT" = "")
Any scripts or data that you put into this service are public.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.