monmlp.fit: Fit one or more MLP or MONMLP models

In monmlp: Multi-Layer Perceptron Neural Network with Optional Monotonicity Constraints

Description

Fit an individual model or ensemble of MLP or MONMLP regression models using optimx optimization routines to minimize a least squares cost function. Optional stopped training and bootstrap aggregation (bagging) can be used to help avoid overfitting.

If invoked, the monotone argument enforces increasing behaviour between specified columns of x and model outputs. In this case, the exp function is applied to the relevant weights following initialization and during optimization; manual adjustment of init.weights may be needed.

Note: x and y are automatically standardized prior to fitting and predictions are automatically rescaled by monmlp.predict. This behaviour can be suppressed for y by the scale.y argument.

Usage

monmlp.fit(x, y, hidden1, hidden2 = 0, iter.max = 5000,
           n.trials = 1, n.ensemble = 1, bag = FALSE,
           cases.specified = NULL, iter.stopped = NULL,
           scale.y = TRUE, Th = tansig, To = linear,
           Th.prime = tansig.prime, To.prime = linear.prime,
           monotone = NULL, init.weights = NULL,
           max.exceptions = 10, silent = FALSE, method = "BFGS",
           control = list(trace = 0))

Arguments

x	covariate matrix with number of rows equal to the number of samples and number of columns equal to the number of covariates.
y	response matrix with number of rows equal to the number of samples and number of columns equal to the number of response variables.
hidden1	number of hidden nodes in the first hidden layer.
hidden2	number of hidden nodes in the second hidden layer.
iter.max	maximum number of iterations of the optimization algorithm.
n.trials	number of repeated trials used to avoid local minima.
n.ensemble	number of ensemble members to fit.
bag	logical variable indicating whether or not bootstrap aggregation (bagging) should be used.
cases.specified	if bag = TRUE, a list that specifies the bootstrapped cases to be used in each ensemble member.
iter.stopped	if bag = TRUE, specifies the number of stopped training iterations between calculation of the cost function on the out-of-bootstrap cases.
scale.y	logical determining if columns of the response matrix should be scaled to zero mean and unit variance prior to fitting. Set this to FALSE if using an output layer transfer function that limits the range of predictions.
Th	hidden layer transfer function.
To	output layer transfer function.
Th.prime	derivative of the hidden layer transfer function.
To.prime	derivative of the output layer transfer function.
monotone	column indices of covariates for which the monotonicity constraint should hold.
init.weights	either a vector giving the minimum and maximum allowable values of the random weights, an initial weight vector, or NULL to calculate based on fan-in.
max.exceptions	maximum number of exceptions of the optimization routine before fitting is terminated with an error.
silent	logical determining if diagnostic messages should be suppressed.
method	optimx optimization method.
control	list of optimx control parameters.

Value

list containing fitted weight matrices with attributes including called values of x, y, Th, To, Th.prime, To.prime, monotone, bag, iter.max, and iter.stopped, along with values of covariate/response column means and standard deviations (x.center, x.scale, y.center, y.scale), out-of-bootstrap cases oob, predicted values y.pred, and, if stopped training is switched on, the iteration iter.best and value of the cost function cost.best that minimized the out-of-bootstrap validation error.

See Also

monmlp.predict, gam.style

Examples

set.seed(123)
x <- as.matrix(seq(-10, 10, length = 100))
y <- logistic(x) + rnorm(100, sd = 0.2)

dev.new()
plot(x, y)
lines(x, logistic(x), lwd = 10, col = "gray")

## MLP w/ 2 hidden nodes
w.mlp <- monmlp.fit(x = x, y = y, hidden1 = 2, iter.max = 500)
lines(x, attr(w.mlp, "y.pred"), col = "red", lwd = 3)

## MLP w/ 2 hidden nodes and stopped training
w.stp <- monmlp.fit(x = x, y = y, hidden1 = 2, bag = TRUE,
                    iter.max = 500, iter.stopped = 10)
lines(x, attr(w.stp, "y.pred"), col = "orange", lwd = 3)

## MONMLP w/ 2 hidden nodes
w.mon <- monmlp.fit(x = x, y = y, hidden1 = 2, monotone = 1,
                    iter.max = 500)
lines(x, attr(w.mon, "y.pred"), col = "blue", lwd = 3)

Example output

Loading required package: optimx
** Ensemble 1 
0.1317504 
** 0.1317504 

** Ensemble 1 
** Bagging on
** Stopped training on
0.1932585 
	  ---> 10 0.1755113 
0.1657502 
	  ---> 20 0.136135 
0.1623037 
	  ---> 30 0.1473842 
0.1593024 
	  ---> 40 0.1349965 
0.1569279 
	  ---> 50 0.1414741 
0.1553838 
	  ---> 60 0.1427751 
0.1551755 
	  ---> 70 0.1432837 
0.1550994 
	  ---> 80 0.1438908 
0.1550512 
	  ---> 90 0.1432868 
0.1550171 
	  ---> 100 0.1435686 
0.1549929 
	  ---> 110 0.1435188 
0.1549852 
	  ---> 120 0.1427306 
0.1549764 
	  ---> 130 0.1434661 
0.154964 
	  ---> 140 0.1423122 
0.1549536 
	  ---> 150 0.1431949 
0.1549489 
	  ---> 160 0.1432554 
0.1549448 
	  ---> 170 0.1432054 
0.1549402 
	  ---> 180 0.1430867 
0.1549356 
	  ---> 190 0.1431355 
0.1549316 
	  ---> 200 0.1432282 
0.1549276 
	  ---> 210 0.1431699 
0.1549233 
	  ---> 220 0.1431287 
0.154919 
	  ---> 230 0.1430962 
0.154915 
	  ---> 240 0.1432506 
0.1549108 
	  ---> 250 0.1431629 
0.1549069 
	  ---> 260 0.1431108 
0.1549028 
	  ---> 270 0.1430877 
0.1548991 
	  ---> 280 0.1432269 
0.1548952 
	  ---> 290 0.1431466 
0.1548915 
	  ---> 300 0.1430951 
0.1548877 
	  ---> 310 0.1430719 
0.1548842 
	  ---> 320 0.1432023 
0.1548805 
	  ---> 330 0.1431296 
0.154877 
	  ---> 340 0.1430783 
0.1548734 
	  ---> 350 0.1430566 
0.15487 
	  ---> 360 0.1431804 
0.1548665 
	  ---> 370 0.1431139 
0.1548632 
	  ---> 380 0.1430621 
0.1548598 
	  ---> 390 0.1430424 
0.1548565 
	  ---> 400 0.1431611 
0.1548531 
	  ---> 410 0.1430999 
0.1548499 
	  ---> 420 0.1430467 
0.1548466 
	  ---> 430 0.1430292 
0.1548434 
	  ---> 440 0.1431437 
0.1548401 
	  ---> 450 0.1430876 
0.1548369 
	  ---> 460 0.1430322 
0.1548337 
	  ---> 470 0.1430172 
0.1548306 
	  ---> 480 0.1431279 
0.1548274 
	  ---> 490 0.1430767 
0.1548243 
	  ---> 500 0.1430185 
** 40 0.1349965 

** Ensemble 1 
0.1308002 
** 0.1308002 

monmlp documentation built on May 2, 2019, 11:14 a.m.