elmNN-package: Implementation of ELM ( Extreme Learning Machine ) algorithm...

In elmNN: Implementation of ELM (Extreme Learning Machine ) algorithm for SLFN ( Single Hidden Layer Feedforward Neural Networks )

Description

ELM algorithm is an alternative training method for SLFN ( Single Hidden Layer Feedforward Networks ) which does not need any iterative tuning nor setting parameters such as learning rate, momentum, etc., which are current issues of the traditional gradient-based learning algorithms ( like backpropagation ).
Training of a SLFN with ELM is a three-step learning model:
Given a training set P = {(xi , ti )|xi E R , ti E R , i = 1,..., N}, hidden node output function G(a, b, x), and the number of hidden nodes L
1) Assign randomly hidden node parameters (ai , bi ), i = 1,..., L. It means that the arc weights between the input layer and the hidden layer and the hidden layer bias are randomly generated.
2) Calculate the hidden layer output matrix H using one of the available activation functions.
3) Calculate the output weights B: B = ginv(H) %*% T ( matrix multiplication ), where T is the target output of the training set.
ginv(H) is the Moore-Penrose generalized inverse of hidden layer output matrix H. This is calculated by the MASS package function ginv.
Once the SLFN has been trained, the output of a generic test set is simply Y = H %*% B ( matrix multiplication ). Salient features:
- The learning speed of ELM is extremely fast.
- Unlike traditional gradient-based learning algorithms which only work for differentiable activation functions, ELM works for all bounded nonconstant piecewise continuous activation functions.
- Unlike traditional gradient-based learning algorithms facing several issues like local minima, improper learning rate and overfitting, etc, ELM tends to reach the solutions straightforward without such trivial issues.
- The ELM learning algorithm looks much simpler than other popular learning algorithms: neural networks and support vector machines.

Details

Package:	elmNN
Type:	Package
Version:	1.0
Date:	2012-07-17
License:	GPL (>= 2)

To fit a neural network, the function to use is elmtrain ( default version is elmtrain.formula ). To predict values, the function to use is predict. Other functions are used internally by the training and predict functions.

Author(s)

Alberto Gosso

Maintainer: Alberto Gosso <[e-mail]>

References

http://www.ntu.edu.sg/home/egbhuang/
G.-B. Huang, H. Zhou, X. Ding, R. Zhang (2011) Extreme Learning Machine for Regression and Multiclass Classification IEEE Transactions on Systems, Man, and Cybernetics - part B: Cybernetics, vol. 42, no. 2, 513-529
G.-B. Huang, Q.-Y. Zhu, C.-K. Siew (2006) Extreme learning machine: Theory and applications Neurocomputing 70 (2006) 489-501

See Also

elmtrain.formula to train a neural network,predict.elmNN to predict values from a trained neural network

Examples

set.seed(1234)
Var1 <- runif(50, 0, 100) 
sqrt.data <- data.frame(Var1, Sqrt=sqrt(Var1))
model <- elmtrain(Sqrt~Var1, data=sqrt.data, nhid=10, actfun="sig")
new <- data.frame(Sqrt=0,Var1 = runif(50,0,100))
p <- predict(model,newdata=new)

Var2 <- runif(50, 0, 10) 
quad.data <- data.frame(Var2, Quad=(Var2)^2)
model <- elmtrain(Quad~Var2, data=quad.data, nhid=10, actfun="sig")
new <- data.frame(Quad=0,Var2 = runif(50,0,10))
p <- predict(model,newdata=new)

Example output

Loading required package: MASS

elmNN documentation built on May 29, 2017, 3:22 p.m.