train: Continue training of a Neural Network
In ANN2: Artificial Neural Networks for Anomaly Detection
Description
Usage
Arguments
Details
Value
References
Examples
Description
Continue training of a neural network object returned by neuralnetwork()
or autoencoder()
Usage
1
2
3
4
5
6
7
8
9
10
Arguments
object
object of class ANN produced by neuralnetwork()
or autoencoder()
X
matrix with explanatory variables
y
matrix with dependent variables. Not required if object is an autoencoder
n.epochs
the number of epochs to train. This parameter largely determines
the training time (one epoch is a single iteration through the training data).
batch.size
the number of observations to use in each batch. Batch learning
is computationally faster than stochastic gradient descent. However, large
batches might not result in optimal learning, see Efficient Backprop by Le Cun
for details.
drop.last
logical. Only applicable if the size of the training set is not
perfectly devisible by the batch size. Determines if the last chosen observations
should be discarded (in the current epoch) or should constitute a smaller batch.
Note that a smaller batch leads to a noisier approximation of the gradient.
val.prop
proportion of training data to use for tracking the loss on a
validation set during training. Useful for assessing the training process and
identifying possible overfitting. Set to zero for only tracking the loss on the
training data.
random.seed
optional seed for the random number generator
Details
A new validation set is randomly chosen. This can result in irregular jumps
in the plot given by plot.ANN().
Value
An ANN object. Use function plot(<object>) to assess
loss on training and optionally validation data during training process. Use
function predict(<object>, <newdata>) for prediction.
References
LeCun, Yann A., et al. "Efficient backprop." Neural networks:
Tricks of the trade. Springer Berlin Heidelberg, 2012. 9-48.
Examples
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# Train a neural network on the iris dataset
X <- iris[,1:4]
y <- iris$Species
NN <- neuralnetwork(X, y, hidden.layers = 10, sgd.momentum = 0.9,
learn.rates = 0.01, val.prop = 0.3, n.epochs = 100)
# Plot training and validation loss during training
plot(NN)
# Continue training for 1000 epochs
train(NN, X, y, n.epochs = 200, val.prop = 0.3)
# Again plot the loss - note the jump in the validation loss at the 100th epoch
# This is due to the random selection of a new validation set
plot(NN)
Example output
Artificial Neural Network:
Layer - 4 nodes - input
Layer - 10 nodes - tanh
Layer - 3 nodes - softmax
With log loss and SGD optimizer
Training progress:
[|-------------------------------------------------] 0% - Validation loss: 1.12075
[|-------------------------------------------------] 1% - Validation loss: 1.0836
[+|------------------------------------------------] 2% - Validation loss: 0.987355
[+|------------------------------------------------] 3% - Validation loss: 0.823832
[++|-----------------------------------------------] 4% - Validation loss: 0.689834
[++|-----------------------------------------------] 4% - Validation loss: 0.654725
[++|-----------------------------------------------] 5% - Validation loss: 0.595051
[+++|----------------------------------------------] 6% - Validation loss: 0.529022
[+++|----------------------------------------------] 7% - Validation loss: 0.483308
[+++|----------------------------------------------] 7% - Validation loss: 0.461439
[++++|---------------------------------------------] 8% - Validation loss: 0.453063
[++++|---------------------------------------------] 9% - Validation loss: 0.431166
[+++++|--------------------------------------------] 10% - Validation loss: 0.414912
[+++++|--------------------------------------------] 11% - Validation loss: 0.403542
[++++++|-------------------------------------------] 12% - Validation loss: 0.395634
[++++++|-------------------------------------------] 13% - Validation loss: 0.388906
[+++++++|------------------------------------------] 14% - Validation loss: 0.380221
[+++++++|------------------------------------------] 14% - Validation loss: 0.37744
[+++++++|------------------------------------------] 15% - Validation loss: 0.372048
[++++++++|-----------------------------------------] 16% - Validation loss: 0.364191
[++++++++|-----------------------------------------] 17% - Validation loss: 0.355545
[++++++++|-----------------------------------------] 17% - Validation loss: 0.350381
[+++++++++|----------------------------------------] 18% - Validation loss: 0.348108
[+++++++++|----------------------------------------] 19% - Validation loss: 0.341068
[++++++++++|---------------------------------------] 20% - Validation loss: 0.334828
[++++++++++|---------------------------------------] 21% - Validation loss: 0.329389
[+++++++++++|--------------------------------------] 22% - Validation loss: 0.324597
[+++++++++++|--------------------------------------] 23% - Validation loss: 0.321292
[++++++++++++|-------------------------------------] 24% - Validation loss: 0.319056
[++++++++++++|-------------------------------------] 24% - Validation loss: 0.318373
[++++++++++++|-------------------------------------] 25% - Validation loss: 0.315447
[+++++++++++++|------------------------------------] 26% - Validation loss: 0.311478
[+++++++++++++|------------------------------------] 27% - Validation loss: 0.307368
[+++++++++++++|------------------------------------] 27% - Validation loss: 0.304318
[++++++++++++++|-----------------------------------] 28% - Validation loss: 0.302877
[++++++++++++++|-----------------------------------] 29% - Validation loss: 0.299363
[+++++++++++++++|----------------------------------] 30% - Validation loss: 0.294754
[+++++++++++++++|----------------------------------] 31% - Validation loss: 0.289251
[++++++++++++++++|---------------------------------] 32% - Validation loss: 0.283753
[++++++++++++++++|---------------------------------] 33% - Validation loss: 0.278075
[+++++++++++++++++|--------------------------------] 34% - Validation loss: 0.272936
[+++++++++++++++++|--------------------------------] 34% - Validation loss: 0.271619
[+++++++++++++++++|--------------------------------] 35% - Validation loss: 0.268965
[++++++++++++++++++|-------------------------------] 36% - Validation loss: 0.265129
[++++++++++++++++++|-------------------------------] 37% - Validation loss: 0.261283
[++++++++++++++++++|-------------------------------] 37% - Validation loss: 0.258922
[+++++++++++++++++++|------------------------------] 38% - Validation loss: 0.258139
[+++++++++++++++++++|------------------------------] 39% - Validation loss: 0.255634
[++++++++++++++++++++|-----------------------------] 40% - Validation loss: 0.253232
[++++++++++++++++++++|-----------------------------] 41% - Validation loss: 0.2501
[+++++++++++++++++++++|----------------------------] 42% - Validation loss: 0.247112
[+++++++++++++++++++++|----------------------------] 43% - Validation loss: 0.243521
[++++++++++++++++++++++|---------------------------] 44% - Validation loss: 0.240138
[++++++++++++++++++++++|---------------------------] 44% - Validation loss: 0.239227
[++++++++++++++++++++++|---------------------------] 45% - Validation loss: 0.237186
[+++++++++++++++++++++++|--------------------------] 46% - Validation loss: 0.234582
[+++++++++++++++++++++++|--------------------------] 47% - Validation loss: 0.232428
[+++++++++++++++++++++++|--------------------------] 47% - Validation loss: 0.230735
[++++++++++++++++++++++++|-------------------------] 48% - Validation loss: 0.230063
[++++++++++++++++++++++++|-------------------------] 49% - Validation loss: 0.228072
[+++++++++++++++++++++++++|------------------------] 50% - Validation loss: 0.22566
[+++++++++++++++++++++++++|------------------------] 51% - Validation loss: 0.222739
[++++++++++++++++++++++++++|-----------------------] 52% - Validation loss: 0.220738
[++++++++++++++++++++++++++|-----------------------] 53% - Validation loss: 0.21936
[+++++++++++++++++++++++++++|----------------------] 54% - Validation loss: 0.218775
[+++++++++++++++++++++++++++|----------------------] 54% - Validation loss: 0.21888
[+++++++++++++++++++++++++++|----------------------] 55% - Validation loss: 0.21876
[++++++++++++++++++++++++++++|---------------------] 56% - Validation loss: 0.218727
[++++++++++++++++++++++++++++|---------------------] 57% - Validation loss: 0.218665
[++++++++++++++++++++++++++++|---------------------] 57% - Validation loss: 0.218247
[+++++++++++++++++++++++++++++|--------------------] 58% - Validation loss: 0.217689
[+++++++++++++++++++++++++++++|--------------------] 59% - Validation loss: 0.215853
[++++++++++++++++++++++++++++++|-------------------] 60% - Validation loss: 0.214624
[++++++++++++++++++++++++++++++|-------------------] 61% - Validation loss: 0.213587
[+++++++++++++++++++++++++++++++|------------------] 62% - Validation loss: 0.212103
[+++++++++++++++++++++++++++++++|------------------] 63% - Validation loss: 0.21067
[++++++++++++++++++++++++++++++++|-----------------] 64% - Validation loss: 0.209617
[++++++++++++++++++++++++++++++++|-----------------] 64% - Validation loss: 0.209114
[++++++++++++++++++++++++++++++++|-----------------] 65% - Validation loss: 0.208375
[+++++++++++++++++++++++++++++++++|----------------] 66% - Validation loss: 0.205751
[+++++++++++++++++++++++++++++++++|----------------] 67% - Validation loss: 0.203663
[+++++++++++++++++++++++++++++++++|----------------] 67% - Validation loss: 0.202069
[++++++++++++++++++++++++++++++++++|---------------] 68% - Validation loss: 0.201549
[++++++++++++++++++++++++++++++++++|---------------] 69% - Validation loss: 0.198673
[+++++++++++++++++++++++++++++++++++|--------------] 70% - Validation loss: 0.195683
[+++++++++++++++++++++++++++++++++++|--------------] 71% - Validation loss: 0.194309
[++++++++++++++++++++++++++++++++++++|-------------] 72% - Validation loss: 0.193422
[++++++++++++++++++++++++++++++++++++|-------------] 73% - Validation loss: 0.193454
[+++++++++++++++++++++++++++++++++++++|------------] 74% - Validation loss: 0.193783
[+++++++++++++++++++++++++++++++++++++|------------] 74% - Validation loss: 0.194004
[+++++++++++++++++++++++++++++++++++++|------------] 75% - Validation loss: 0.1954
[++++++++++++++++++++++++++++++++++++++|-----------] 76% - Validation loss: 0.19685
[++++++++++++++++++++++++++++++++++++++|-----------] 77% - Validation loss: 0.197936
[++++++++++++++++++++++++++++++++++++++|-----------] 77% - Validation loss: 0.198656
[+++++++++++++++++++++++++++++++++++++++|----------] 78% - Validation loss: 0.199033
[+++++++++++++++++++++++++++++++++++++++|----------] 79% - Validation loss: 0.200208
[++++++++++++++++++++++++++++++++++++++++|---------] 80% - Validation loss: 0.200962
[++++++++++++++++++++++++++++++++++++++++|---------] 81% - Validation loss: 0.201188
[+++++++++++++++++++++++++++++++++++++++++|--------] 82% - Validation loss: 0.198844
[+++++++++++++++++++++++++++++++++++++++++|--------] 83% - Validation loss: 0.196927
[++++++++++++++++++++++++++++++++++++++++++|-------] 84% - Validation loss: 0.19558
[++++++++++++++++++++++++++++++++++++++++++|-------] 84% - Validation loss: 0.195027
[++++++++++++++++++++++++++++++++++++++++++|-------] 85% - Validation loss: 0.194171
[+++++++++++++++++++++++++++++++++++++++++++|------] 86% - Validation loss: 0.194065
[+++++++++++++++++++++++++++++++++++++++++++|------] 87% - Validation loss: 0.194679
[+++++++++++++++++++++++++++++++++++++++++++|------] 87% - Validation loss: 0.194582
[++++++++++++++++++++++++++++++++++++++++++++|-----] 88% - Validation loss: 0.195484
[++++++++++++++++++++++++++++++++++++++++++++|-----] 89% - Validation loss: 0.19454
[+++++++++++++++++++++++++++++++++++++++++++++|----] 90% - Validation loss: 0.193121
[+++++++++++++++++++++++++++++++++++++++++++++|----] 91% - Validation loss: 0.192296
[++++++++++++++++++++++++++++++++++++++++++++++|---] 92% - Validation loss: 0.190532
[++++++++++++++++++++++++++++++++++++++++++++++|---] 93% - Validation loss: 0.189043
[+++++++++++++++++++++++++++++++++++++++++++++++|--] 94% - Validation loss: 0.18782
[+++++++++++++++++++++++++++++++++++++++++++++++|--] 94% - Validation loss: 0.187912
[+++++++++++++++++++++++++++++++++++++++++++++++|--] 95% - Validation loss: 0.188049
[++++++++++++++++++++++++++++++++++++++++++++++++|-] 96% - Validation loss: 0.188495
[++++++++++++++++++++++++++++++++++++++++++++++++|-] 97% - Validation loss: 0.188489
[++++++++++++++++++++++++++++++++++++++++++++++++|-] 97% - Validation loss: 0.187885
[+++++++++++++++++++++++++++++++++++++++++++++++++|] 98% - Validation loss: 0.187137
[+++++++++++++++++++++++++++++++++++++++++++++++++|] 99% - Validation loss: 0.18534
[++++++++++++++++++++++++++++++++++++++++++++++++++] 100% - Validation loss: 0.184198
Training progress:
[+++++++++++++++++|--------------------------------] 34% - Validation loss: 0.145225
[+++++++++++++++++|--------------------------------] 35% - Validation loss: 0.142898
[+++++++++++++++++|--------------------------------] 35% - Validation loss: 0.14334
[++++++++++++++++++|-------------------------------] 36% - Validation loss: 0.146438
[++++++++++++++++++|-------------------------------] 36% - Validation loss: 0.146798
[++++++++++++++++++|-------------------------------] 37% - Validation loss: 0.145061
[++++++++++++++++++|-------------------------------] 37% - Validation loss: 0.14218
[+++++++++++++++++++|------------------------------] 38% - Validation loss: 0.139992
[+++++++++++++++++++|------------------------------] 38% - Validation loss: 0.137713
[+++++++++++++++++++|------------------------------] 39% - Validation loss: 0.136792
[+++++++++++++++++++|------------------------------] 39% - Validation loss: 0.135021
[++++++++++++++++++++|-----------------------------] 40% - Validation loss: 0.135085
[++++++++++++++++++++|-----------------------------] 41% - Validation loss: 0.138671
[+++++++++++++++++++++|----------------------------] 42% - Validation loss: 0.134134
[+++++++++++++++++++++|----------------------------] 42% - Validation loss: 0.13356
[+++++++++++++++++++++|----------------------------] 43% - Validation loss: 0.133604
[+++++++++++++++++++++|----------------------------] 43% - Validation loss: 0.133764
[++++++++++++++++++++++|---------------------------] 44% - Validation loss: 0.134878
[++++++++++++++++++++++|---------------------------] 44% - Validation loss: 0.132988
[++++++++++++++++++++++|---------------------------] 45% - Validation loss: 0.129467
[++++++++++++++++++++++|---------------------------] 45% - Validation loss: 0.130817
[+++++++++++++++++++++++|--------------------------] 46% - Validation loss: 0.132233
[+++++++++++++++++++++++|--------------------------] 46% - Validation loss: 0.134281
[+++++++++++++++++++++++|--------------------------] 47% - Validation loss: 0.133513
[+++++++++++++++++++++++|--------------------------] 47% - Validation loss: 0.131676
[++++++++++++++++++++++++|-------------------------] 48% - Validation loss: 0.129518
[++++++++++++++++++++++++|-------------------------] 48% - Validation loss: 0.124694
[++++++++++++++++++++++++|-------------------------] 49% - Validation loss: 0.124185
[++++++++++++++++++++++++|-------------------------] 49% - Validation loss: 0.125258
[+++++++++++++++++++++++++|------------------------] 50% - Validation loss: 0.125344
[+++++++++++++++++++++++++|------------------------] 51% - Validation loss: 0.133884
[++++++++++++++++++++++++++|-----------------------] 52% - Validation loss: 0.134329
[++++++++++++++++++++++++++|-----------------------] 52% - Validation loss: 0.133795
[++++++++++++++++++++++++++|-----------------------] 53% - Validation loss: 0.129466
[++++++++++++++++++++++++++|-----------------------] 53% - Validation loss: 0.128582
[+++++++++++++++++++++++++++|----------------------] 54% - Validation loss: 0.121224
[+++++++++++++++++++++++++++|----------------------] 54% - Validation loss: 0.118914
[+++++++++++++++++++++++++++|----------------------] 55% - Validation loss: 0.120237
[+++++++++++++++++++++++++++|----------------------] 55% - Validation loss: 0.122121
[++++++++++++++++++++++++++++|---------------------] 56% - Validation loss: 0.123958
[++++++++++++++++++++++++++++|---------------------] 56% - Validation loss: 0.122739
[++++++++++++++++++++++++++++|---------------------] 57% - Validation loss: 0.119905
[++++++++++++++++++++++++++++|---------------------] 57% - Validation loss: 0.118364
[+++++++++++++++++++++++++++++|--------------------] 58% - Validation loss: 0.119385
[+++++++++++++++++++++++++++++|--------------------] 58% - Validation loss: 0.119142
[+++++++++++++++++++++++++++++|--------------------] 59% - Validation loss: 0.119518
[+++++++++++++++++++++++++++++|--------------------] 59% - Validation loss: 0.123319
[++++++++++++++++++++++++++++++|-------------------] 60% - Validation loss: 0.123213
[++++++++++++++++++++++++++++++|-------------------] 61% - Validation loss: 0.121527
[+++++++++++++++++++++++++++++++|------------------] 62% - Validation loss: 0.124623
[+++++++++++++++++++++++++++++++|------------------] 62% - Validation loss: 0.124174
[+++++++++++++++++++++++++++++++|------------------] 63% - Validation loss: 0.120527
[+++++++++++++++++++++++++++++++|------------------] 63% - Validation loss: 0.119513
[++++++++++++++++++++++++++++++++|-----------------] 64% - Validation loss: 0.115808
[++++++++++++++++++++++++++++++++|-----------------] 64% - Validation loss: 0.115692
[++++++++++++++++++++++++++++++++|-----------------] 65% - Validation loss: 0.115739
[++++++++++++++++++++++++++++++++|-----------------] 65% - Validation loss: 0.119252
[+++++++++++++++++++++++++++++++++|----------------] 66% - Validation loss: 0.121081
[+++++++++++++++++++++++++++++++++|----------------] 66% - Validation loss: 0.11965
[+++++++++++++++++++++++++++++++++|----------------] 67% - Validation loss: 0.117423
[+++++++++++++++++++++++++++++++++|----------------] 67% - Validation loss: 0.11625
[++++++++++++++++++++++++++++++++++|---------------] 68% - Validation loss: 0.115307
[++++++++++++++++++++++++++++++++++|---------------] 68% - Validation loss: 0.116134
[++++++++++++++++++++++++++++++++++|---------------] 69% - Validation loss: 0.115435
[++++++++++++++++++++++++++++++++++|---------------] 69% - Validation loss: 0.114854
[+++++++++++++++++++++++++++++++++++|--------------] 70% - Validation loss: 0.114848
[+++++++++++++++++++++++++++++++++++|--------------] 71% - Validation loss: 0.115519
[++++++++++++++++++++++++++++++++++++|-------------] 72% - Validation loss: 0.114169
[++++++++++++++++++++++++++++++++++++|-------------] 72% - Validation loss: 0.113682
[++++++++++++++++++++++++++++++++++++|-------------] 73% - Validation loss: 0.106725
[++++++++++++++++++++++++++++++++++++|-------------] 73% - Validation loss: 0.106872
[+++++++++++++++++++++++++++++++++++++|------------] 74% - Validation loss: 0.107241
[+++++++++++++++++++++++++++++++++++++|------------] 74% - Validation loss: 0.107703
[+++++++++++++++++++++++++++++++++++++|------------] 75% - Validation loss: 0.110951
[+++++++++++++++++++++++++++++++++++++|------------] 75% - Validation loss: 0.110277
[++++++++++++++++++++++++++++++++++++++|-----------] 76% - Validation loss: 0.110647
[++++++++++++++++++++++++++++++++++++++|-----------] 76% - Validation loss: 0.110511
[++++++++++++++++++++++++++++++++++++++|-----------] 77% - Validation loss: 0.109457
[++++++++++++++++++++++++++++++++++++++|-----------] 77% - Validation loss: 0.111781
[+++++++++++++++++++++++++++++++++++++++|----------] 78% - Validation loss: 0.111576
[+++++++++++++++++++++++++++++++++++++++|----------] 78% - Validation loss: 0.112351
[+++++++++++++++++++++++++++++++++++++++|----------] 79% - Validation loss: 0.112346
[+++++++++++++++++++++++++++++++++++++++|----------] 79% - Validation loss: 0.111861
[++++++++++++++++++++++++++++++++++++++++|---------] 80% - Validation loss: 0.112163
[++++++++++++++++++++++++++++++++++++++++|---------] 81% - Validation loss: 0.109398
[+++++++++++++++++++++++++++++++++++++++++|--------] 82% - Validation loss: 0.109678
[+++++++++++++++++++++++++++++++++++++++++|--------] 82% - Validation loss: 0.110236
[+++++++++++++++++++++++++++++++++++++++++|--------] 83% - Validation loss: 0.116728
[+++++++++++++++++++++++++++++++++++++++++|--------] 83% - Validation loss: 0.119022
[++++++++++++++++++++++++++++++++++++++++++|-------] 84% - Validation loss: 0.118003
[++++++++++++++++++++++++++++++++++++++++++|-------] 84% - Validation loss: 0.115238
[++++++++++++++++++++++++++++++++++++++++++|-------] 85% - Validation loss: 0.113684
[++++++++++++++++++++++++++++++++++++++++++|-------] 85% - Validation loss: 0.114235
[+++++++++++++++++++++++++++++++++++++++++++|------] 86% - Validation loss: 0.114662
[+++++++++++++++++++++++++++++++++++++++++++|------] 86% - Validation loss: 0.112996
[+++++++++++++++++++++++++++++++++++++++++++|------] 87% - Validation loss: 0.112838
[+++++++++++++++++++++++++++++++++++++++++++|------] 87% - Validation loss: 0.109501
[++++++++++++++++++++++++++++++++++++++++++++|-----] 88% - Validation loss: 0.110406
[++++++++++++++++++++++++++++++++++++++++++++|-----] 88% - Validation loss: 0.110382
[++++++++++++++++++++++++++++++++++++++++++++|-----] 89% - Validation loss: 0.110711
[++++++++++++++++++++++++++++++++++++++++++++|-----] 89% - Validation loss: 0.11019
[+++++++++++++++++++++++++++++++++++++++++++++|----] 90% - Validation loss: 0.110295
[+++++++++++++++++++++++++++++++++++++++++++++|----] 91% - Validation loss: 0.110148
[++++++++++++++++++++++++++++++++++++++++++++++|---] 92% - Validation loss: 0.109957
[++++++++++++++++++++++++++++++++++++++++++++++|---] 92% - Validation loss: 0.110452
[++++++++++++++++++++++++++++++++++++++++++++++|---] 93% - Validation loss: 0.110462
[++++++++++++++++++++++++++++++++++++++++++++++|---] 93% - Validation loss: 0.111128
[+++++++++++++++++++++++++++++++++++++++++++++++|--] 94% - Validation loss: 0.111098
[+++++++++++++++++++++++++++++++++++++++++++++++|--] 94% - Validation loss: 0.109097
[+++++++++++++++++++++++++++++++++++++++++++++++|--] 95% - Validation loss: 0.108924
[+++++++++++++++++++++++++++++++++++++++++++++++|--] 95% - Validation loss: 0.110809
[++++++++++++++++++++++++++++++++++++++++++++++++|-] 96% - Validation loss: 0.109961
[++++++++++++++++++++++++++++++++++++++++++++++++|-] 96% - Validation loss: 0.108896
[++++++++++++++++++++++++++++++++++++++++++++++++|-] 97% - Validation loss: 0.110616
[++++++++++++++++++++++++++++++++++++++++++++++++|-] 97% - Validation loss: 0.10953
[+++++++++++++++++++++++++++++++++++++++++++++++++|] 98% - Validation loss: 0.109121
[+++++++++++++++++++++++++++++++++++++++++++++++++|] 98% - Validation loss: 0.1096
[+++++++++++++++++++++++++++++++++++++++++++++++++|] 99% - Validation loss: 0.109645
[+++++++++++++++++++++++++++++++++++++++++++++++++|] 99% - Validation loss: 0.109192
[++++++++++++++++++++++++++++++++++++++++++++++++++] 100% - Validation loss: 0.109636
ANN2 documentation built on Dec. 1, 2020, 5:08 p.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Description Usage Arguments Details Value References Examples
Description
Continue training of a neural network object returned by neuralnetwork()
or autoencoder()
Usage
1 2 3 4 5 6 7 8 9 10 |
Arguments
object |
object of class |
X |
matrix with explanatory variables |
y |
matrix with dependent variables. Not required if object is an autoencoder |
n.epochs |
the number of epochs to train. This parameter largely determines the training time (one epoch is a single iteration through the training data). |
batch.size |
the number of observations to use in each batch. Batch learning is computationally faster than stochastic gradient descent. However, large batches might not result in optimal learning, see Efficient Backprop by Le Cun for details. |
drop.last |
logical. Only applicable if the size of the training set is not perfectly devisible by the batch size. Determines if the last chosen observations should be discarded (in the current epoch) or should constitute a smaller batch. Note that a smaller batch leads to a noisier approximation of the gradient. |
val.prop |
proportion of training data to use for tracking the loss on a validation set during training. Useful for assessing the training process and identifying possible overfitting. Set to zero for only tracking the loss on the training data. |
random.seed |
optional seed for the random number generator |
Details
A new validation set is randomly chosen. This can result in irregular jumps
in the plot given by plot.ANN().
Value
An ANN object. Use function plot(<object>) to assess
loss on training and optionally validation data during training process. Use
function predict(<object>, <newdata>) for prediction.
References
LeCun, Yann A., et al. "Efficient backprop." Neural networks: Tricks of the trade. Springer Berlin Heidelberg, 2012. 9-48.
Examples
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | # Train a neural network on the iris dataset
X <- iris[,1:4]
y <- iris$Species
NN <- neuralnetwork(X, y, hidden.layers = 10, sgd.momentum = 0.9,
learn.rates = 0.01, val.prop = 0.3, n.epochs = 100)
# Plot training and validation loss during training
plot(NN)
# Continue training for 1000 epochs
train(NN, X, y, n.epochs = 200, val.prop = 0.3)
# Again plot the loss - note the jump in the validation loss at the 100th epoch
# This is due to the random selection of a new validation set
plot(NN)
|
Example output
Artificial Neural Network:
Layer - 4 nodes - input
Layer - 10 nodes - tanh
Layer - 3 nodes - softmax
With log loss and SGD optimizer
Training progress:
[|-------------------------------------------------] 0% - Validation loss: 1.12075
[|-------------------------------------------------] 1% - Validation loss: 1.0836
[+|------------------------------------------------] 2% - Validation loss: 0.987355
[+|------------------------------------------------] 3% - Validation loss: 0.823832
[++|-----------------------------------------------] 4% - Validation loss: 0.689834
[++|-----------------------------------------------] 4% - Validation loss: 0.654725
[++|-----------------------------------------------] 5% - Validation loss: 0.595051
[+++|----------------------------------------------] 6% - Validation loss: 0.529022
[+++|----------------------------------------------] 7% - Validation loss: 0.483308
[+++|----------------------------------------------] 7% - Validation loss: 0.461439
[++++|---------------------------------------------] 8% - Validation loss: 0.453063
[++++|---------------------------------------------] 9% - Validation loss: 0.431166
[+++++|--------------------------------------------] 10% - Validation loss: 0.414912
[+++++|--------------------------------------------] 11% - Validation loss: 0.403542
[++++++|-------------------------------------------] 12% - Validation loss: 0.395634
[++++++|-------------------------------------------] 13% - Validation loss: 0.388906
[+++++++|------------------------------------------] 14% - Validation loss: 0.380221
[+++++++|------------------------------------------] 14% - Validation loss: 0.37744
[+++++++|------------------------------------------] 15% - Validation loss: 0.372048
[++++++++|-----------------------------------------] 16% - Validation loss: 0.364191
[++++++++|-----------------------------------------] 17% - Validation loss: 0.355545
[++++++++|-----------------------------------------] 17% - Validation loss: 0.350381
[+++++++++|----------------------------------------] 18% - Validation loss: 0.348108
[+++++++++|----------------------------------------] 19% - Validation loss: 0.341068
[++++++++++|---------------------------------------] 20% - Validation loss: 0.334828
[++++++++++|---------------------------------------] 21% - Validation loss: 0.329389
[+++++++++++|--------------------------------------] 22% - Validation loss: 0.324597
[+++++++++++|--------------------------------------] 23% - Validation loss: 0.321292
[++++++++++++|-------------------------------------] 24% - Validation loss: 0.319056
[++++++++++++|-------------------------------------] 24% - Validation loss: 0.318373
[++++++++++++|-------------------------------------] 25% - Validation loss: 0.315447
[+++++++++++++|------------------------------------] 26% - Validation loss: 0.311478
[+++++++++++++|------------------------------------] 27% - Validation loss: 0.307368
[+++++++++++++|------------------------------------] 27% - Validation loss: 0.304318
[++++++++++++++|-----------------------------------] 28% - Validation loss: 0.302877
[++++++++++++++|-----------------------------------] 29% - Validation loss: 0.299363
[+++++++++++++++|----------------------------------] 30% - Validation loss: 0.294754
[+++++++++++++++|----------------------------------] 31% - Validation loss: 0.289251
[++++++++++++++++|---------------------------------] 32% - Validation loss: 0.283753
[++++++++++++++++|---------------------------------] 33% - Validation loss: 0.278075
[+++++++++++++++++|--------------------------------] 34% - Validation loss: 0.272936
[+++++++++++++++++|--------------------------------] 34% - Validation loss: 0.271619
[+++++++++++++++++|--------------------------------] 35% - Validation loss: 0.268965
[++++++++++++++++++|-------------------------------] 36% - Validation loss: 0.265129
[++++++++++++++++++|-------------------------------] 37% - Validation loss: 0.261283
[++++++++++++++++++|-------------------------------] 37% - Validation loss: 0.258922
[+++++++++++++++++++|------------------------------] 38% - Validation loss: 0.258139
[+++++++++++++++++++|------------------------------] 39% - Validation loss: 0.255634
[++++++++++++++++++++|-----------------------------] 40% - Validation loss: 0.253232
[++++++++++++++++++++|-----------------------------] 41% - Validation loss: 0.2501
[+++++++++++++++++++++|----------------------------] 42% - Validation loss: 0.247112
[+++++++++++++++++++++|----------------------------] 43% - Validation loss: 0.243521
[++++++++++++++++++++++|---------------------------] 44% - Validation loss: 0.240138
[++++++++++++++++++++++|---------------------------] 44% - Validation loss: 0.239227
[++++++++++++++++++++++|---------------------------] 45% - Validation loss: 0.237186
[+++++++++++++++++++++++|--------------------------] 46% - Validation loss: 0.234582
[+++++++++++++++++++++++|--------------------------] 47% - Validation loss: 0.232428
[+++++++++++++++++++++++|--------------------------] 47% - Validation loss: 0.230735
[++++++++++++++++++++++++|-------------------------] 48% - Validation loss: 0.230063
[++++++++++++++++++++++++|-------------------------] 49% - Validation loss: 0.228072
[+++++++++++++++++++++++++|------------------------] 50% - Validation loss: 0.22566
[+++++++++++++++++++++++++|------------------------] 51% - Validation loss: 0.222739
[++++++++++++++++++++++++++|-----------------------] 52% - Validation loss: 0.220738
[++++++++++++++++++++++++++|-----------------------] 53% - Validation loss: 0.21936
[+++++++++++++++++++++++++++|----------------------] 54% - Validation loss: 0.218775
[+++++++++++++++++++++++++++|----------------------] 54% - Validation loss: 0.21888
[+++++++++++++++++++++++++++|----------------------] 55% - Validation loss: 0.21876
[++++++++++++++++++++++++++++|---------------------] 56% - Validation loss: 0.218727
[++++++++++++++++++++++++++++|---------------------] 57% - Validation loss: 0.218665
[++++++++++++++++++++++++++++|---------------------] 57% - Validation loss: 0.218247
[+++++++++++++++++++++++++++++|--------------------] 58% - Validation loss: 0.217689
[+++++++++++++++++++++++++++++|--------------------] 59% - Validation loss: 0.215853
[++++++++++++++++++++++++++++++|-------------------] 60% - Validation loss: 0.214624
[++++++++++++++++++++++++++++++|-------------------] 61% - Validation loss: 0.213587
[+++++++++++++++++++++++++++++++|------------------] 62% - Validation loss: 0.212103
[+++++++++++++++++++++++++++++++|------------------] 63% - Validation loss: 0.21067
[++++++++++++++++++++++++++++++++|-----------------] 64% - Validation loss: 0.209617
[++++++++++++++++++++++++++++++++|-----------------] 64% - Validation loss: 0.209114
[++++++++++++++++++++++++++++++++|-----------------] 65% - Validation loss: 0.208375
[+++++++++++++++++++++++++++++++++|----------------] 66% - Validation loss: 0.205751
[+++++++++++++++++++++++++++++++++|----------------] 67% - Validation loss: 0.203663
[+++++++++++++++++++++++++++++++++|----------------] 67% - Validation loss: 0.202069
[++++++++++++++++++++++++++++++++++|---------------] 68% - Validation loss: 0.201549
[++++++++++++++++++++++++++++++++++|---------------] 69% - Validation loss: 0.198673
[+++++++++++++++++++++++++++++++++++|--------------] 70% - Validation loss: 0.195683
[+++++++++++++++++++++++++++++++++++|--------------] 71% - Validation loss: 0.194309
[++++++++++++++++++++++++++++++++++++|-------------] 72% - Validation loss: 0.193422
[++++++++++++++++++++++++++++++++++++|-------------] 73% - Validation loss: 0.193454
[+++++++++++++++++++++++++++++++++++++|------------] 74% - Validation loss: 0.193783
[+++++++++++++++++++++++++++++++++++++|------------] 74% - Validation loss: 0.194004
[+++++++++++++++++++++++++++++++++++++|------------] 75% - Validation loss: 0.1954
[++++++++++++++++++++++++++++++++++++++|-----------] 76% - Validation loss: 0.19685
[++++++++++++++++++++++++++++++++++++++|-----------] 77% - Validation loss: 0.197936
[++++++++++++++++++++++++++++++++++++++|-----------] 77% - Validation loss: 0.198656
[+++++++++++++++++++++++++++++++++++++++|----------] 78% - Validation loss: 0.199033
[+++++++++++++++++++++++++++++++++++++++|----------] 79% - Validation loss: 0.200208
[++++++++++++++++++++++++++++++++++++++++|---------] 80% - Validation loss: 0.200962
[++++++++++++++++++++++++++++++++++++++++|---------] 81% - Validation loss: 0.201188
[+++++++++++++++++++++++++++++++++++++++++|--------] 82% - Validation loss: 0.198844
[+++++++++++++++++++++++++++++++++++++++++|--------] 83% - Validation loss: 0.196927
[++++++++++++++++++++++++++++++++++++++++++|-------] 84% - Validation loss: 0.19558
[++++++++++++++++++++++++++++++++++++++++++|-------] 84% - Validation loss: 0.195027
[++++++++++++++++++++++++++++++++++++++++++|-------] 85% - Validation loss: 0.194171
[+++++++++++++++++++++++++++++++++++++++++++|------] 86% - Validation loss: 0.194065
[+++++++++++++++++++++++++++++++++++++++++++|------] 87% - Validation loss: 0.194679
[+++++++++++++++++++++++++++++++++++++++++++|------] 87% - Validation loss: 0.194582
[++++++++++++++++++++++++++++++++++++++++++++|-----] 88% - Validation loss: 0.195484
[++++++++++++++++++++++++++++++++++++++++++++|-----] 89% - Validation loss: 0.19454
[+++++++++++++++++++++++++++++++++++++++++++++|----] 90% - Validation loss: 0.193121
[+++++++++++++++++++++++++++++++++++++++++++++|----] 91% - Validation loss: 0.192296
[++++++++++++++++++++++++++++++++++++++++++++++|---] 92% - Validation loss: 0.190532
[++++++++++++++++++++++++++++++++++++++++++++++|---] 93% - Validation loss: 0.189043
[+++++++++++++++++++++++++++++++++++++++++++++++|--] 94% - Validation loss: 0.18782
[+++++++++++++++++++++++++++++++++++++++++++++++|--] 94% - Validation loss: 0.187912
[+++++++++++++++++++++++++++++++++++++++++++++++|--] 95% - Validation loss: 0.188049
[++++++++++++++++++++++++++++++++++++++++++++++++|-] 96% - Validation loss: 0.188495
[++++++++++++++++++++++++++++++++++++++++++++++++|-] 97% - Validation loss: 0.188489
[++++++++++++++++++++++++++++++++++++++++++++++++|-] 97% - Validation loss: 0.187885
[+++++++++++++++++++++++++++++++++++++++++++++++++|] 98% - Validation loss: 0.187137
[+++++++++++++++++++++++++++++++++++++++++++++++++|] 99% - Validation loss: 0.18534
[++++++++++++++++++++++++++++++++++++++++++++++++++] 100% - Validation loss: 0.184198
Training progress:
[+++++++++++++++++|--------------------------------] 34% - Validation loss: 0.145225
[+++++++++++++++++|--------------------------------] 35% - Validation loss: 0.142898
[+++++++++++++++++|--------------------------------] 35% - Validation loss: 0.14334
[++++++++++++++++++|-------------------------------] 36% - Validation loss: 0.146438
[++++++++++++++++++|-------------------------------] 36% - Validation loss: 0.146798
[++++++++++++++++++|-------------------------------] 37% - Validation loss: 0.145061
[++++++++++++++++++|-------------------------------] 37% - Validation loss: 0.14218
[+++++++++++++++++++|------------------------------] 38% - Validation loss: 0.139992
[+++++++++++++++++++|------------------------------] 38% - Validation loss: 0.137713
[+++++++++++++++++++|------------------------------] 39% - Validation loss: 0.136792
[+++++++++++++++++++|------------------------------] 39% - Validation loss: 0.135021
[++++++++++++++++++++|-----------------------------] 40% - Validation loss: 0.135085
[++++++++++++++++++++|-----------------------------] 41% - Validation loss: 0.138671
[+++++++++++++++++++++|----------------------------] 42% - Validation loss: 0.134134
[+++++++++++++++++++++|----------------------------] 42% - Validation loss: 0.13356
[+++++++++++++++++++++|----------------------------] 43% - Validation loss: 0.133604
[+++++++++++++++++++++|----------------------------] 43% - Validation loss: 0.133764
[++++++++++++++++++++++|---------------------------] 44% - Validation loss: 0.134878
[++++++++++++++++++++++|---------------------------] 44% - Validation loss: 0.132988
[++++++++++++++++++++++|---------------------------] 45% - Validation loss: 0.129467
[++++++++++++++++++++++|---------------------------] 45% - Validation loss: 0.130817
[+++++++++++++++++++++++|--------------------------] 46% - Validation loss: 0.132233
[+++++++++++++++++++++++|--------------------------] 46% - Validation loss: 0.134281
[+++++++++++++++++++++++|--------------------------] 47% - Validation loss: 0.133513
[+++++++++++++++++++++++|--------------------------] 47% - Validation loss: 0.131676
[++++++++++++++++++++++++|-------------------------] 48% - Validation loss: 0.129518
[++++++++++++++++++++++++|-------------------------] 48% - Validation loss: 0.124694
[++++++++++++++++++++++++|-------------------------] 49% - Validation loss: 0.124185
[++++++++++++++++++++++++|-------------------------] 49% - Validation loss: 0.125258
[+++++++++++++++++++++++++|------------------------] 50% - Validation loss: 0.125344
[+++++++++++++++++++++++++|------------------------] 51% - Validation loss: 0.133884
[++++++++++++++++++++++++++|-----------------------] 52% - Validation loss: 0.134329
[++++++++++++++++++++++++++|-----------------------] 52% - Validation loss: 0.133795
[++++++++++++++++++++++++++|-----------------------] 53% - Validation loss: 0.129466
[++++++++++++++++++++++++++|-----------------------] 53% - Validation loss: 0.128582
[+++++++++++++++++++++++++++|----------------------] 54% - Validation loss: 0.121224
[+++++++++++++++++++++++++++|----------------------] 54% - Validation loss: 0.118914
[+++++++++++++++++++++++++++|----------------------] 55% - Validation loss: 0.120237
[+++++++++++++++++++++++++++|----------------------] 55% - Validation loss: 0.122121
[++++++++++++++++++++++++++++|---------------------] 56% - Validation loss: 0.123958
[++++++++++++++++++++++++++++|---------------------] 56% - Validation loss: 0.122739
[++++++++++++++++++++++++++++|---------------------] 57% - Validation loss: 0.119905
[++++++++++++++++++++++++++++|---------------------] 57% - Validation loss: 0.118364
[+++++++++++++++++++++++++++++|--------------------] 58% - Validation loss: 0.119385
[+++++++++++++++++++++++++++++|--------------------] 58% - Validation loss: 0.119142
[+++++++++++++++++++++++++++++|--------------------] 59% - Validation loss: 0.119518
[+++++++++++++++++++++++++++++|--------------------] 59% - Validation loss: 0.123319
[++++++++++++++++++++++++++++++|-------------------] 60% - Validation loss: 0.123213
[++++++++++++++++++++++++++++++|-------------------] 61% - Validation loss: 0.121527
[+++++++++++++++++++++++++++++++|------------------] 62% - Validation loss: 0.124623
[+++++++++++++++++++++++++++++++|------------------] 62% - Validation loss: 0.124174
[+++++++++++++++++++++++++++++++|------------------] 63% - Validation loss: 0.120527
[+++++++++++++++++++++++++++++++|------------------] 63% - Validation loss: 0.119513
[++++++++++++++++++++++++++++++++|-----------------] 64% - Validation loss: 0.115808
[++++++++++++++++++++++++++++++++|-----------------] 64% - Validation loss: 0.115692
[++++++++++++++++++++++++++++++++|-----------------] 65% - Validation loss: 0.115739
[++++++++++++++++++++++++++++++++|-----------------] 65% - Validation loss: 0.119252
[+++++++++++++++++++++++++++++++++|----------------] 66% - Validation loss: 0.121081
[+++++++++++++++++++++++++++++++++|----------------] 66% - Validation loss: 0.11965
[+++++++++++++++++++++++++++++++++|----------------] 67% - Validation loss: 0.117423
[+++++++++++++++++++++++++++++++++|----------------] 67% - Validation loss: 0.11625
[++++++++++++++++++++++++++++++++++|---------------] 68% - Validation loss: 0.115307
[++++++++++++++++++++++++++++++++++|---------------] 68% - Validation loss: 0.116134
[++++++++++++++++++++++++++++++++++|---------------] 69% - Validation loss: 0.115435
[++++++++++++++++++++++++++++++++++|---------------] 69% - Validation loss: 0.114854
[+++++++++++++++++++++++++++++++++++|--------------] 70% - Validation loss: 0.114848
[+++++++++++++++++++++++++++++++++++|--------------] 71% - Validation loss: 0.115519
[++++++++++++++++++++++++++++++++++++|-------------] 72% - Validation loss: 0.114169
[++++++++++++++++++++++++++++++++++++|-------------] 72% - Validation loss: 0.113682
[++++++++++++++++++++++++++++++++++++|-------------] 73% - Validation loss: 0.106725
[++++++++++++++++++++++++++++++++++++|-------------] 73% - Validation loss: 0.106872
[+++++++++++++++++++++++++++++++++++++|------------] 74% - Validation loss: 0.107241
[+++++++++++++++++++++++++++++++++++++|------------] 74% - Validation loss: 0.107703
[+++++++++++++++++++++++++++++++++++++|------------] 75% - Validation loss: 0.110951
[+++++++++++++++++++++++++++++++++++++|------------] 75% - Validation loss: 0.110277
[++++++++++++++++++++++++++++++++++++++|-----------] 76% - Validation loss: 0.110647
[++++++++++++++++++++++++++++++++++++++|-----------] 76% - Validation loss: 0.110511
[++++++++++++++++++++++++++++++++++++++|-----------] 77% - Validation loss: 0.109457
[++++++++++++++++++++++++++++++++++++++|-----------] 77% - Validation loss: 0.111781
[+++++++++++++++++++++++++++++++++++++++|----------] 78% - Validation loss: 0.111576
[+++++++++++++++++++++++++++++++++++++++|----------] 78% - Validation loss: 0.112351
[+++++++++++++++++++++++++++++++++++++++|----------] 79% - Validation loss: 0.112346
[+++++++++++++++++++++++++++++++++++++++|----------] 79% - Validation loss: 0.111861
[++++++++++++++++++++++++++++++++++++++++|---------] 80% - Validation loss: 0.112163
[++++++++++++++++++++++++++++++++++++++++|---------] 81% - Validation loss: 0.109398
[+++++++++++++++++++++++++++++++++++++++++|--------] 82% - Validation loss: 0.109678
[+++++++++++++++++++++++++++++++++++++++++|--------] 82% - Validation loss: 0.110236
[+++++++++++++++++++++++++++++++++++++++++|--------] 83% - Validation loss: 0.116728
[+++++++++++++++++++++++++++++++++++++++++|--------] 83% - Validation loss: 0.119022
[++++++++++++++++++++++++++++++++++++++++++|-------] 84% - Validation loss: 0.118003
[++++++++++++++++++++++++++++++++++++++++++|-------] 84% - Validation loss: 0.115238
[++++++++++++++++++++++++++++++++++++++++++|-------] 85% - Validation loss: 0.113684
[++++++++++++++++++++++++++++++++++++++++++|-------] 85% - Validation loss: 0.114235
[+++++++++++++++++++++++++++++++++++++++++++|------] 86% - Validation loss: 0.114662
[+++++++++++++++++++++++++++++++++++++++++++|------] 86% - Validation loss: 0.112996
[+++++++++++++++++++++++++++++++++++++++++++|------] 87% - Validation loss: 0.112838
[+++++++++++++++++++++++++++++++++++++++++++|------] 87% - Validation loss: 0.109501
[++++++++++++++++++++++++++++++++++++++++++++|-----] 88% - Validation loss: 0.110406
[++++++++++++++++++++++++++++++++++++++++++++|-----] 88% - Validation loss: 0.110382
[++++++++++++++++++++++++++++++++++++++++++++|-----] 89% - Validation loss: 0.110711
[++++++++++++++++++++++++++++++++++++++++++++|-----] 89% - Validation loss: 0.11019
[+++++++++++++++++++++++++++++++++++++++++++++|----] 90% - Validation loss: 0.110295
[+++++++++++++++++++++++++++++++++++++++++++++|----] 91% - Validation loss: 0.110148
[++++++++++++++++++++++++++++++++++++++++++++++|---] 92% - Validation loss: 0.109957
[++++++++++++++++++++++++++++++++++++++++++++++|---] 92% - Validation loss: 0.110452
[++++++++++++++++++++++++++++++++++++++++++++++|---] 93% - Validation loss: 0.110462
[++++++++++++++++++++++++++++++++++++++++++++++|---] 93% - Validation loss: 0.111128
[+++++++++++++++++++++++++++++++++++++++++++++++|--] 94% - Validation loss: 0.111098
[+++++++++++++++++++++++++++++++++++++++++++++++|--] 94% - Validation loss: 0.109097
[+++++++++++++++++++++++++++++++++++++++++++++++|--] 95% - Validation loss: 0.108924
[+++++++++++++++++++++++++++++++++++++++++++++++|--] 95% - Validation loss: 0.110809
[++++++++++++++++++++++++++++++++++++++++++++++++|-] 96% - Validation loss: 0.109961
[++++++++++++++++++++++++++++++++++++++++++++++++|-] 96% - Validation loss: 0.108896
[++++++++++++++++++++++++++++++++++++++++++++++++|-] 97% - Validation loss: 0.110616
[++++++++++++++++++++++++++++++++++++++++++++++++|-] 97% - Validation loss: 0.10953
[+++++++++++++++++++++++++++++++++++++++++++++++++|] 98% - Validation loss: 0.109121
[+++++++++++++++++++++++++++++++++++++++++++++++++|] 98% - Validation loss: 0.1096
[+++++++++++++++++++++++++++++++++++++++++++++++++|] 99% - Validation loss: 0.109645
[+++++++++++++++++++++++++++++++++++++++++++++++++|] 99% - Validation loss: 0.109192
[++++++++++++++++++++++++++++++++++++++++++++++++++] 100% - Validation loss: 0.109636
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.