monmlp-package: Monotone Multi-Layer Perceptron Neural Network

In monmlp: Multi-Layer Perceptron Neural Network with Optional Monotonicity Constraints

Description

The monmlp package implements one and two hidden-layer multi-layer perceptron neural network (MLP) models. An optional monotone constraint, which guarantees monotonically increasing behaviour of model outputs with respect to specified covariates, can be added to the MLP. The resulting monotone MLP (MONMLP) regression model is based on Zhang and Zhang (1999).

Early stopping can be combined with bootstrap aggregation to control overfitting. The model reduces to a standard MLP neural network if the monotone constraint is not invoked.

MLP and MONMLP models are fit using the monmlp.fit function. Predictions from a fitted model are made using the monmlp.predict function. The gam.style function can be used to investigate fitted covariate/response relationships.

Details

Package:	monmlp
Type:	Package
License:	GPL-2
LazyLoad:	yes

References

Lang, B., 2005. Monotonic multi-layer perceptron networks as universal approximators. In: W. Duch et al. (eds.): ICANN 2005, Lecture Notes in Computer Science, 3697:31-37.
doi:10.1007/11550907

Minin, A., Velikova, M., Lang, B., and Daniels, H., 2010. Comparison of universal approximators incorporating partial monotonicity by structure. Neural Networks, 23:471-475.
doi:10.1016/j.neunet.2009.09.002

Zhang, H. and Zhang, Z., 1999. Feedforward networks with monotone constraints. In: International Joint Conference on Neural Networks, vol. 3, p. 1820-1823. doi:10.1109/IJCNN.1999.832655

Examples

set.seed(123)
x <- as.matrix(seq(-10, 10, length = 100))
y <- logistic(x) + rnorm(100, sd = 0.2)

dev.new()
plot(x, y)
lines(x, logistic(x), lwd = 10, col = "gray")

## MLP w/ 2 hidden nodes
w.mlp <- monmlp.fit(x = x, y = y, hidden1 = 2, iter.max = 500)
lines(x, attr(w.mlp, "y.pred"), col = "red", lwd = 3)

## MLP w/ 2 hidden-layers (2 nodes each) and early stopping
w.stp <- monmlp.fit(x = x, y = y, hidden1 = 2, hidden2 = 2,
                    bag = TRUE, iter.max = 500, iter.stopped = 10)
lines(x, attr(w.stp, "y.pred"), col = "orange", lwd = 3)

## MONMLP w/ 2 hidden nodes
w.mon <- monmlp.fit(x = x, y = y, hidden1 = 2, monotone = 1,
                    iter.max = 500)
lines(x, attr(w.mon, "y.pred"), col = "blue", lwd = 3)

Example output

Loading required package: optimx
** Ensemble 1 
0.1317504 
** 0.1317504 

** Ensemble 1 
** Bagging on
** Stopped training on
0.4443405 
	  ---> 10 0.2749692 
0.1636263 
	  ---> 20 0.13602 
0.1619604 
	  ---> 30 0.1414882 
0.1602621 
	  ---> 40 0.1364849 
0.1577874 
	  ---> 50 0.1452568 
0.1540762 
	  ---> 60 0.1405613 
0.1533742 
	  ---> 70 0.1402596 
0.153234 
	  ---> 80 0.1406669 
0.153152 
	  ---> 90 0.1410331 
0.1530966 
	  ---> 100 0.1406246 
0.1530579 
	  ---> 110 0.1406386 
0.152971 
	  ---> 120 0.1425328 
0.1528669 
	  ---> 130 0.140799 
0.1527747 
	  ---> 140 0.1412318 
0.1527433 
	  ---> 150 0.1406552 
0.1527163 
	  ---> 160 0.1418682 
0.1526782 
	  ---> 170 0.1415659 
0.1526484 
	  ---> 180 0.1415872 
0.1525956 
	  ---> 190 0.14069 
0.1525513 
	  ---> 200 0.1418375 
0.1524916 
	  ---> 210 0.141354 
0.1524368 
	  ---> 220 0.142005 
0.1523252 
	  ---> 230 0.1453557 
0.1521707 
	  ---> 240 0.1426114 
0.1521267 
	  ---> 250 0.1426169 
0.1520707 
	  ---> 260 0.1422372 
0.1520495 
	  ---> 270 0.1423387 
0.1520294 
	  ---> 280 0.1432292 
0.1519788 
	  ---> 290 0.1445655 
0.151943 
	  ---> 300 0.1433657 
0.1519039 
	  ---> 310 0.1443178 
0.1518796 
	  ---> 320 0.143568 
0.1518499 
	  ---> 330 0.1436556 
0.1518197 
	  ---> 340 0.1434599 
0.1516968 
	  ---> 350 0.1433032 
0.1515694 
	  ---> 360 0.1446509 
0.1515595 
	  ---> 370 0.1451241 
0.1515521 
	  ---> 380 0.1443913 
0.1515447 
	  ---> 390 0.1445243 
0.1515378 
	  ---> 400 0.1446229 
0.1515314 
	  ---> 410 0.1451276 
0.1515255 
	  ---> 420 0.144921 
0.1515202 
	  ---> 430 0.1448728 
0.151512 
	  ---> 440 0.1448554 
0.1514922 
	  ---> 450 0.1458852 
0.1514774 
	  ---> 460 0.1450831 
0.1514672 
	  ---> 470 0.1453667 
0.1514569 
	  ---> 480 0.1453465 
0.1514477 
	  ---> 490 0.1448546 
0.1514341 
	  ---> 500 0.1456572 
** 20 0.13602 

** Ensemble 1 
0.1309106 
** 0.1309106 

monmlp documentation built on May 2, 2019, 11:14 a.m.