backpropagation: Backpropagation learning function
In darch: Package for Deep Architectures and Restricted Boltzmann Machines
Description
Usage
Arguments
Details
Value
References
See Also
Examples
View source: R/backpropagation.R
Description
This function provides the backpropagation algorithm for deep architectures.
Usage
1
2
3
4
5
6
7
8
9
backpropagation(darch, trainData, targetData,
bp.learnRate = getParameter(".bp.learnRate", rep(1, times =
length(darch@layers))),
bp.learnRateScale = getParameter(".bp.learnRateScale"),
nesterovMomentum = getParameter(".darch.nesterovMomentum"),
dropout = getParameter(".darch.dropout", rep(0, times = length(darch@layers)
+ 1), darch), dropConnect = getParameter(".darch.dropout.dropConnect"),
matMult = getParameter(".matMult"), debugMode = getParameter(".debug", F),
...)
Arguments
darch
An instance of the class DArch.
trainData
The training data (inputs).
targetData
The target data (outputs).
bp.learnRate
Learning rates for backpropagation, length is either one
or the same as the number of weight matrices when using different learning
rates for each layer.
bp.learnRateScale
The learn rate is multiplied by this value after
each epoch.
nesterovMomentum
See darch.nesterovMomentum parameter of
darch.
dropout
See darch.dropout parameter of darch.
dropConnect
See darch.dropout.dropConnect parameter of
darch.
matMult
Matrix multiplication function, internal parameter.
debugMode
Whether debug mode is enabled, internal parameter.
...
Further parameters.
Details
The only backpropagation-specific, user-relevant parameters are
bp.learnRate and bp.learnRateScale; they can be passed to the
darch function when enabling backpropagation as the
fine-tuning function. bp.learnRate defines the backpropagation
learning rate and can either be specified as a single scalar or as a vector
with one entry for each weight matrix, allowing for per-layer learning rates.
bp.learnRateScale is a single scalar which contains a scaling factor
for the learning rate(s) which will be applied after each epoch.
Backpropagation supports dropout and uses the weight update function as
defined via the darch.weightUpdateFunction parameter of
darch.
Value
The trained deep architecture
References
Rumelhart, D., G. E. Hinton, R. J. Williams, Learning
representations by backpropagating errors, Nature 323, S. 533-536, DOI:
10.1038/323533a0, 1986.
See Also
Other fine-tuning functions: minimizeAutoencoder,
minimizeClassifier,
rpropagation
Examples
1
2
3
4
5
Example output
INFO [2017-09-07 06:24:59] The current log level is: INFO
INFO [2017-09-07 06:24:59] Start initial caret pre-processing.
INFO [2017-09-07 06:24:59] Converting non-numeric columns in data (if any)...
INFO [2017-09-07 06:24:59] Converting non-numeric columns in targets (if any)...
INFO [2017-09-07 06:24:59] Dependent factor "Species" converted to 3 new variables (1-of-n coding)
INFO [2017-09-07 06:24:59] The current log level is: INFO
INFO [2017-09-07 06:24:59] Using CPU matrix multiplication.
WARN [2017-09-07 06:24:59] No vector given for "layers" parameter, constructing shallow network with one hidden layer of 10 neurons.
INFO [2017-09-07 06:24:59] Creating and configuring new DArch instance
INFO [2017-09-07 06:24:59] Constructing a network with 3 layers (4, 10, 3 neurons).
INFO [2017-09-07 06:24:59] Generating RBMs.
INFO [2017-09-07 06:24:59] Constructing new RBM instance with 4 visible and 10 hidden units.
INFO [2017-09-07 06:24:59] Constructing new RBM instance with 10 visible and 3 hidden units.
INFO [2017-09-07 06:24:59] DArch instance ready for training, here is a summary of its configuration:
INFO [2017-09-07 06:24:59] Global parameters:
INFO [2017-09-07 06:24:59] Layers parameter was 10, resulted in network with 3 layers and 4, 10, 3 neurons
INFO [2017-09-07 06:24:59] The weights for the layers were generated with "generateWeightsGlorotUniform"
INFO [2017-09-07 06:24:59] Additionally, the following parameters were used for weight generation:
INFO [2017-09-07 06:24:59] [weights] Parameter weights.max is 0.1
INFO [2017-09-07 06:24:59] [weights] Parameter weights.min is -0.1
INFO [2017-09-07 06:24:59] [weights] Parameter weights.mean is 0
INFO [2017-09-07 06:24:59] [weights] Parameter weights.sd is 0.01
INFO [2017-09-07 06:24:59] Weight normalization is disabled
INFO [2017-09-07 06:24:59] Bootstrapping is disabled
INFO [2017-09-07 06:24:59] Train data are shuffled before each epoch
INFO [2017-09-07 06:24:59] Autosaving is disabled
INFO [2017-09-07 06:24:59] Using CPU for matrix multiplication
INFO [2017-09-07 06:24:59] Pre-processing parameters:
INFO [2017-09-07 06:24:59] [preProc] Parameter preProc.factorToNumeric is FALSE
INFO [2017-09-07 06:24:59] [preProc] Parameter preProc.factorToNumeric.targets is FALSE
INFO [2017-09-07 06:24:59] [preProc] Parameter preProc.fullRank is TRUE
INFO [2017-09-07 06:24:59] [preProc] Parameter preProc.fullRank.targets is FALSE
INFO [2017-09-07 06:24:59] [preProc] Parameter preProc.orderedToFactor.targets is TRUE
INFO [2017-09-07 06:24:59] [preProc] Parameter preProc.targets is FALSE
INFO [2017-09-07 06:24:59] Caret pre-processing is disabled
INFO [2017-09-07 06:24:59] Pre-training parameters:
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.allData is FALSE
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.batchSize is 1
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.consecutive is TRUE
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.errorFunction is "mseError"
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.finalMomentum is 0.9
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.initialMomentum is 0.5
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.lastLayer is 0
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.learnRate is 1
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.learnRateScale is 1
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.momentumRampLength is 1
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.numCD is 1
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.numEpochs is 0
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.unitFunction is "sigmoidUnitRbm"
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.updateFunction is "rbmUpdate"
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.weightDecay is 2e-04
INFO [2017-09-07 06:24:59] The selected RBMs have been trained for 0 epochs
INFO [2017-09-07 06:24:59] Fine-tuning parameters:
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.batchSize is 1
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.dither is FALSE
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.dropout is 0
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.dropout.dropConnect is FALSE
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.dropout.momentMatching is 0
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.dropout.oneMaskPerEpoch is FALSE
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.elu.alpha is 1
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.errorFunction is "crossEntropyError"
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.finalMomentum is 0.9
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.fineTuneFunction is "backpropagation"
INFO [2017-09-07 06:24:59] [backprop] Using backpropagation for fine-tuning
INFO [2017-09-07 06:24:59] [backprop] Parameter bp.learnRate is c(1, 1)
INFO [2017-09-07 06:24:59] [backprop] Parameter bp.learnRateScale is 1
INFO [2017-09-07 06:24:59] [backprop] See ?backpropagation for documentation
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.initialMomentum is 0.5
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.isClass is TRUE
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.maxout.poolSize is 2
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.maxout.unitFunction is "linearUnit"
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.momentumRampLength is 1
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.nesterovMomentum is TRUE
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.numEpochs is 100
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.returnBestModel is TRUE
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.returnBestModel.validationErrorFactor is 0.632120558828558
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.stopClassErr is -Inf
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.stopErr is -Inf
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.stopValidClassErr is -Inf
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.stopValidErr is -Inf
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.trainLayers is c(TRUE, TRUE)
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.unitFunction is "sigmoidUnit"
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.weightDecay is 0
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.weightUpdateFunction is "weightDecayWeightUpdate"
INFO [2017-09-07 06:24:59] The network has been fine-tuned for 0 epochs
INFO [2017-09-07 06:24:59] Training set consists of 150 samples.
INFO [2017-09-07 06:24:59] Start deep architecture fine-tuning for 100 epochs
INFO [2017-09-07 06:24:59] Number of Batches: 150 (batch size 1)
INFO [2017-09-07 06:24:59] Epoch: 1 of 100
INFO [2017-09-07 06:24:59] Classification error on Train set: 30% (45/150)
INFO [2017-09-07 06:24:59] Train set Cross Entropy error: 1.011
INFO [2017-09-07 06:24:59] Finished epoch 1 of 100 after 0.138 secs (1099 patterns/sec)
INFO [2017-09-07 06:24:59] Epoch: 2 of 100
INFO [2017-09-07 06:24:59] Classification error on Train set: 32% (48/150)
INFO [2017-09-07 06:24:59] Train set Cross Entropy error: 0.952
INFO [2017-09-07 06:24:59] Finished epoch 2 of 100 after 0.0717 secs (2146 patterns/sec)
INFO [2017-09-07 06:24:59] Epoch: 3 of 100
INFO [2017-09-07 06:24:59] Classification error on Train set: 18% (27/150)
INFO [2017-09-07 06:24:59] Train set Cross Entropy error: 0.789
INFO [2017-09-07 06:24:59] Finished epoch 3 of 100 after 0.056 secs (2760 patterns/sec)
INFO [2017-09-07 06:24:59] Epoch: 4 of 100
INFO [2017-09-07 06:24:59] Classification error on Train set: 16.67% (25/150)
INFO [2017-09-07 06:24:59] Train set Cross Entropy error: 0.747
INFO [2017-09-07 06:24:59] Finished epoch 4 of 100 after 0.057 secs (2713 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 5 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 22% (33/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.920
INFO [2017-09-07 06:25:00] Finished epoch 5 of 100 after 0.0558 secs (2774 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 6 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.442
INFO [2017-09-07 06:25:00] Finished epoch 6 of 100 after 0.0536 secs (2883 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 7 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 8.67% (13/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.539
INFO [2017-09-07 06:25:00] Finished epoch 7 of 100 after 0.0567 secs (2724 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 8 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 6% (9/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.517
INFO [2017-09-07 06:25:00] Finished epoch 8 of 100 after 0.0557 secs (2777 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 9 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 4.67% (7/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.533
INFO [2017-09-07 06:25:00] Finished epoch 9 of 100 after 0.0541 secs (2863 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 10 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 16% (24/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.672
INFO [2017-09-07 06:25:00] Finished epoch 10 of 100 after 0.0557 secs (2779 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 11 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 14% (21/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.586
INFO [2017-09-07 06:25:00] Finished epoch 11 of 100 after 0.054 secs (2865 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 12 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 6% (9/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.383
INFO [2017-09-07 06:25:00] Finished epoch 12 of 100 after 0.0554 secs (2790 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 13 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 15.33% (23/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.645
INFO [2017-09-07 06:25:00] Finished epoch 13 of 100 after 0.0541 secs (2864 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 14 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 16% (24/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.726
INFO [2017-09-07 06:25:00] Finished epoch 14 of 100 after 0.0552 secs (2806 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 15 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 6.67% (10/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.355
INFO [2017-09-07 06:25:00] Finished epoch 15 of 100 after 0.0538 secs (2874 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 16 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 8% (12/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.515
INFO [2017-09-07 06:25:00] Finished epoch 16 of 100 after 0.0558 secs (2773 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 17 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.280
INFO [2017-09-07 06:25:00] Finished epoch 17 of 100 after 0.0569 secs (2717 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 18 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.279
INFO [2017-09-07 06:25:00] Finished epoch 18 of 100 after 0.0552 secs (2802 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 19 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 11.33% (17/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.525
INFO [2017-09-07 06:25:00] Finished epoch 19 of 100 after 0.0575 secs (2691 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 20 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.243
INFO [2017-09-07 06:25:00] Finished epoch 20 of 100 after 0.0555 secs (2797 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 21 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.233
INFO [2017-09-07 06:25:00] Finished epoch 21 of 100 after 0.0576 secs (2687 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 22 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.302
INFO [2017-09-07 06:25:00] Finished epoch 22 of 100 after 0.0548 secs (2828 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 23 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.533
INFO [2017-09-07 06:25:01] Finished epoch 23 of 100 after 0.0609 secs (2532 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 24 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.249
INFO [2017-09-07 06:25:01] Finished epoch 24 of 100 after 0.0547 secs (2832 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 25 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.189
INFO [2017-09-07 06:25:01] Finished epoch 25 of 100 after 0.178 secs (851 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 26 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.253
INFO [2017-09-07 06:25:01] Finished epoch 26 of 100 after 0.054 secs (2875 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 27 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.209
INFO [2017-09-07 06:25:01] Finished epoch 27 of 100 after 0.0586 secs (2643 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 28 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 6.67% (10/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.306
INFO [2017-09-07 06:25:01] Finished epoch 28 of 100 after 0.0556 secs (2776 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 29 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.225
INFO [2017-09-07 06:25:01] Finished epoch 29 of 100 after 0.0526 secs (2945 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 30 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 6.67% (10/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.305
INFO [2017-09-07 06:25:01] Finished epoch 30 of 100 after 0.0541 secs (2857 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 31 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 8% (12/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.374
INFO [2017-09-07 06:25:01] Finished epoch 31 of 100 after 0.0673 secs (2296 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 32 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.214
INFO [2017-09-07 06:25:01] Finished epoch 32 of 100 after 0.0737 secs (2097 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 33 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.204
INFO [2017-09-07 06:25:01] Finished epoch 33 of 100 after 0.0598 secs (2600 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 34 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.249
INFO [2017-09-07 06:25:01] Finished epoch 34 of 100 after 0.0637 secs (2417 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 35 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.202
INFO [2017-09-07 06:25:01] Finished epoch 35 of 100 after 0.057 secs (2716 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 36 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 14% (21/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.635
INFO [2017-09-07 06:25:01] Finished epoch 36 of 100 after 0.061 secs (2533 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 37 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.235
INFO [2017-09-07 06:25:02] Finished epoch 37 of 100 after 0.0612 secs (2520 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 38 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 6.67% (10/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.307
INFO [2017-09-07 06:25:02] Finished epoch 38 of 100 after 0.0569 secs (2723 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 39 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.174
INFO [2017-09-07 06:25:02] Finished epoch 39 of 100 after 0.0584 secs (2649 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 40 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.217
INFO [2017-09-07 06:25:02] Finished epoch 40 of 100 after 0.064 secs (2410 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 41 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.182
INFO [2017-09-07 06:25:02] Finished epoch 41 of 100 after 0.0974 secs (1636 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 42 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.217
INFO [2017-09-07 06:25:02] Finished epoch 42 of 100 after 0.109 secs (1395 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 43 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.222
INFO [2017-09-07 06:25:02] Finished epoch 43 of 100 after 0.118 secs (1310 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 44 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.168
INFO [2017-09-07 06:25:02] Finished epoch 44 of 100 after 0.109 secs (1423 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 45 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.234
INFO [2017-09-07 06:25:02] Finished epoch 45 of 100 after 0.0835 secs (1834 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 46 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 7.33% (11/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.432
INFO [2017-09-07 06:25:02] Finished epoch 46 of 100 after 0.0538 secs (2874 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 47 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.231
INFO [2017-09-07 06:25:02] Finished epoch 47 of 100 after 0.0558 secs (2776 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 48 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 14% (21/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.649
INFO [2017-09-07 06:25:02] Finished epoch 48 of 100 after 0.054 secs (2866 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 49 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.165
INFO [2017-09-07 06:25:02] Finished epoch 49 of 100 after 0.0546 secs (2835 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 50 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 20% (30/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.634
INFO [2017-09-07 06:25:02] Finished epoch 50 of 100 after 0.057 secs (2711 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 51 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 6% (9/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.321
INFO [2017-09-07 06:25:03] Finished epoch 51 of 100 after 0.0543 secs (2852 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 52 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.269
INFO [2017-09-07 06:25:03] Finished epoch 52 of 100 after 0.0548 secs (2821 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 53 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.176
INFO [2017-09-07 06:25:03] Finished epoch 53 of 100 after 0.0562 secs (2750 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 54 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.252
INFO [2017-09-07 06:25:03] Finished epoch 54 of 100 after 0.0581 secs (2663 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 55 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.292
INFO [2017-09-07 06:25:03] Finished epoch 55 of 100 after 0.0547 secs (2829 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 56 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.225
INFO [2017-09-07 06:25:03] Finished epoch 56 of 100 after 0.0555 secs (2787 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 57 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.225
INFO [2017-09-07 06:25:03] Finished epoch 57 of 100 after 0.053 secs (2924 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 58 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.199
INFO [2017-09-07 06:25:03] Finished epoch 58 of 100 after 0.0545 secs (2845 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 59 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 6% (9/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.271
INFO [2017-09-07 06:25:03] Finished epoch 59 of 100 after 0.0553 secs (2796 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 60 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 4.67% (7/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.286
INFO [2017-09-07 06:25:03] Finished epoch 60 of 100 after 0.0528 secs (2939 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 61 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.196
INFO [2017-09-07 06:25:03] Finished epoch 61 of 100 after 0.0534 secs (2898 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 62 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.239
INFO [2017-09-07 06:25:03] Finished epoch 62 of 100 after 0.0549 secs (2820 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 63 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 14% (21/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.779
INFO [2017-09-07 06:25:03] Finished epoch 63 of 100 after 0.0527 secs (2934 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 64 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.157
INFO [2017-09-07 06:25:03] Finished epoch 64 of 100 after 0.0545 secs (2844 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 65 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 4.67% (7/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.224
INFO [2017-09-07 06:25:03] Finished epoch 65 of 100 after 0.0554 secs (2792 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 66 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.181
INFO [2017-09-07 06:25:03] Finished epoch 66 of 100 after 0.053 secs (2923 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 67 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.204
INFO [2017-09-07 06:25:03] Finished epoch 67 of 100 after 0.0538 secs (2879 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 68 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 9.33% (14/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.553
INFO [2017-09-07 06:25:03] Finished epoch 68 of 100 after 0.0563 secs (2747 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 69 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.200
INFO [2017-09-07 06:25:04] Finished epoch 69 of 100 after 0.0523 secs (2967 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 70 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.176
INFO [2017-09-07 06:25:04] Finished epoch 70 of 100 after 0.0537 secs (2882 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 71 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 13.33% (20/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.720
INFO [2017-09-07 06:25:04] Finished epoch 71 of 100 after 0.0552 secs (2804 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 72 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.153
INFO [2017-09-07 06:25:04] Finished epoch 72 of 100 after 0.0524 secs (2957 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 73 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.205
INFO [2017-09-07 06:25:04] Finished epoch 73 of 100 after 0.0548 secs (2828 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 74 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 7.33% (11/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.351
INFO [2017-09-07 06:25:04] Finished epoch 74 of 100 after 0.0562 secs (2751 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 75 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.284
INFO [2017-09-07 06:25:04] Finished epoch 75 of 100 after 0.0533 secs (2906 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 76 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 6.67% (10/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.295
INFO [2017-09-07 06:25:04] Finished epoch 76 of 100 after 0.0552 secs (2805 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 77 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.170
INFO [2017-09-07 06:25:04] Finished epoch 77 of 100 after 0.0563 secs (2741 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 78 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 8.67% (13/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.516
INFO [2017-09-07 06:25:04] Finished epoch 78 of 100 after 0.0525 secs (2953 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 79 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.150
INFO [2017-09-07 06:25:04] Finished epoch 79 of 100 after 0.0546 secs (2832 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 80 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 6% (9/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.428
INFO [2017-09-07 06:25:04] Finished epoch 80 of 100 after 0.0556 secs (2780 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 81 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 13.33% (20/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.637
INFO [2017-09-07 06:25:04] Finished epoch 81 of 100 after 0.0518 secs (2986 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 82 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.151
INFO [2017-09-07 06:25:04] Finished epoch 82 of 100 after 0.0537 secs (2887 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 83 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.202
INFO [2017-09-07 06:25:04] Finished epoch 83 of 100 after 0.0557 secs (2773 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 84 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 6% (9/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.244
INFO [2017-09-07 06:25:04] Finished epoch 84 of 100 after 0.053 secs (2920 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 85 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 14% (21/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.459
INFO [2017-09-07 06:25:04] Finished epoch 85 of 100 after 0.0541 secs (2862 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 86 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 8.67% (13/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.452
INFO [2017-09-07 06:25:04] Finished epoch 86 of 100 after 0.0558 secs (2778 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 87 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 10% (15/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.646
INFO [2017-09-07 06:25:05] Finished epoch 87 of 100 after 0.056 secs (2760 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 88 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.238
INFO [2017-09-07 06:25:05] Finished epoch 88 of 100 after 0.0537 secs (2887 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 89 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 16% (24/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.738
INFO [2017-09-07 06:25:05] Finished epoch 89 of 100 after 0.0589 secs (2620 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 90 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.308
INFO [2017-09-07 06:25:05] Finished epoch 90 of 100 after 0.0523 secs (2970 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 91 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.179
INFO [2017-09-07 06:25:05] Finished epoch 91 of 100 after 0.06 secs (2591 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 92 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 4.67% (7/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.269
INFO [2017-09-07 06:25:05] Finished epoch 92 of 100 after 0.0625 secs (2461 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 93 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.172
INFO [2017-09-07 06:25:05] Finished epoch 93 of 100 after 0.0526 secs (2940 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 94 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 17.33% (26/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.765
INFO [2017-09-07 06:25:05] Finished epoch 94 of 100 after 0.0534 secs (2901 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 95 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.209
INFO [2017-09-07 06:25:05] Finished epoch 95 of 100 after 0.0558 secs (2769 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 96 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.211
INFO [2017-09-07 06:25:05] Finished epoch 96 of 100 after 0.0523 secs (2958 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 97 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.195
INFO [2017-09-07 06:25:05] Finished epoch 97 of 100 after 0.0534 secs (2895 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 98 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.174
INFO [2017-09-07 06:25:05] Finished epoch 98 of 100 after 0.0563 secs (2742 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 99 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.262
INFO [2017-09-07 06:25:05] Finished epoch 99 of 100 after 0.0514 secs (3011 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 100 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 8.67% (13/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.402
INFO [2017-09-07 06:25:05] Finished epoch 100 of 100 after 0.0523 secs (2967 patterns/sec)
INFO [2017-09-07 06:25:05] Classification error on Train set (best model): 1.33% (2/150)
INFO [2017-09-07 06:25:05] Train set (best model) Cross Entropy error: 0.157
INFO [2017-09-07 06:25:05] Best model was found after epoch 64
INFO [2017-09-07 06:25:05] Fine-tuning finished after 6.062 secs
darch documentation built on May 29, 2017, 8:14 p.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Description Usage Arguments Details Value References See Also Examples
View source: R/backpropagation.R
Description
This function provides the backpropagation algorithm for deep architectures.
Usage
1 2 3 4 5 6 7 8 9 | backpropagation(darch, trainData, targetData,
bp.learnRate = getParameter(".bp.learnRate", rep(1, times =
length(darch@layers))),
bp.learnRateScale = getParameter(".bp.learnRateScale"),
nesterovMomentum = getParameter(".darch.nesterovMomentum"),
dropout = getParameter(".darch.dropout", rep(0, times = length(darch@layers)
+ 1), darch), dropConnect = getParameter(".darch.dropout.dropConnect"),
matMult = getParameter(".matMult"), debugMode = getParameter(".debug", F),
...)
|
Arguments
darch |
An instance of the class |
trainData |
The training data (inputs). |
targetData |
The target data (outputs). |
bp.learnRate |
Learning rates for backpropagation, length is either one or the same as the number of weight matrices when using different learning rates for each layer. |
bp.learnRateScale |
The learn rate is multiplied by this value after each epoch. |
nesterovMomentum |
See |
dropout |
See |
dropConnect |
See |
matMult |
Matrix multiplication function, internal parameter. |
debugMode |
Whether debug mode is enabled, internal parameter. |
... |
Further parameters. |
Details
The only backpropagation-specific, user-relevant parameters are
bp.learnRate and bp.learnRateScale; they can be passed to the
darch function when enabling backpropagation as the
fine-tuning function. bp.learnRate defines the backpropagation
learning rate and can either be specified as a single scalar or as a vector
with one entry for each weight matrix, allowing for per-layer learning rates.
bp.learnRateScale is a single scalar which contains a scaling factor
for the learning rate(s) which will be applied after each epoch.
Backpropagation supports dropout and uses the weight update function as
defined via the darch.weightUpdateFunction parameter of
darch.
Value
The trained deep architecture
References
Rumelhart, D., G. E. Hinton, R. J. Williams, Learning representations by backpropagating errors, Nature 323, S. 533-536, DOI: 10.1038/323533a0, 1986.
See Also
Other fine-tuning functions: minimizeAutoencoder,
minimizeClassifier,
rpropagation
Examples
1 2 3 4 5 |
Example output
INFO [2017-09-07 06:24:59] The current log level is: INFO
INFO [2017-09-07 06:24:59] Start initial caret pre-processing.
INFO [2017-09-07 06:24:59] Converting non-numeric columns in data (if any)...
INFO [2017-09-07 06:24:59] Converting non-numeric columns in targets (if any)...
INFO [2017-09-07 06:24:59] Dependent factor "Species" converted to 3 new variables (1-of-n coding)
INFO [2017-09-07 06:24:59] The current log level is: INFO
INFO [2017-09-07 06:24:59] Using CPU matrix multiplication.
WARN [2017-09-07 06:24:59] No vector given for "layers" parameter, constructing shallow network with one hidden layer of 10 neurons.
INFO [2017-09-07 06:24:59] Creating and configuring new DArch instance
INFO [2017-09-07 06:24:59] Constructing a network with 3 layers (4, 10, 3 neurons).
INFO [2017-09-07 06:24:59] Generating RBMs.
INFO [2017-09-07 06:24:59] Constructing new RBM instance with 4 visible and 10 hidden units.
INFO [2017-09-07 06:24:59] Constructing new RBM instance with 10 visible and 3 hidden units.
INFO [2017-09-07 06:24:59] DArch instance ready for training, here is a summary of its configuration:
INFO [2017-09-07 06:24:59] Global parameters:
INFO [2017-09-07 06:24:59] Layers parameter was 10, resulted in network with 3 layers and 4, 10, 3 neurons
INFO [2017-09-07 06:24:59] The weights for the layers were generated with "generateWeightsGlorotUniform"
INFO [2017-09-07 06:24:59] Additionally, the following parameters were used for weight generation:
INFO [2017-09-07 06:24:59] [weights] Parameter weights.max is 0.1
INFO [2017-09-07 06:24:59] [weights] Parameter weights.min is -0.1
INFO [2017-09-07 06:24:59] [weights] Parameter weights.mean is 0
INFO [2017-09-07 06:24:59] [weights] Parameter weights.sd is 0.01
INFO [2017-09-07 06:24:59] Weight normalization is disabled
INFO [2017-09-07 06:24:59] Bootstrapping is disabled
INFO [2017-09-07 06:24:59] Train data are shuffled before each epoch
INFO [2017-09-07 06:24:59] Autosaving is disabled
INFO [2017-09-07 06:24:59] Using CPU for matrix multiplication
INFO [2017-09-07 06:24:59] Pre-processing parameters:
INFO [2017-09-07 06:24:59] [preProc] Parameter preProc.factorToNumeric is FALSE
INFO [2017-09-07 06:24:59] [preProc] Parameter preProc.factorToNumeric.targets is FALSE
INFO [2017-09-07 06:24:59] [preProc] Parameter preProc.fullRank is TRUE
INFO [2017-09-07 06:24:59] [preProc] Parameter preProc.fullRank.targets is FALSE
INFO [2017-09-07 06:24:59] [preProc] Parameter preProc.orderedToFactor.targets is TRUE
INFO [2017-09-07 06:24:59] [preProc] Parameter preProc.targets is FALSE
INFO [2017-09-07 06:24:59] Caret pre-processing is disabled
INFO [2017-09-07 06:24:59] Pre-training parameters:
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.allData is FALSE
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.batchSize is 1
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.consecutive is TRUE
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.errorFunction is "mseError"
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.finalMomentum is 0.9
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.initialMomentum is 0.5
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.lastLayer is 0
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.learnRate is 1
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.learnRateScale is 1
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.momentumRampLength is 1
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.numCD is 1
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.numEpochs is 0
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.unitFunction is "sigmoidUnitRbm"
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.updateFunction is "rbmUpdate"
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.weightDecay is 2e-04
INFO [2017-09-07 06:24:59] The selected RBMs have been trained for 0 epochs
INFO [2017-09-07 06:24:59] Fine-tuning parameters:
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.batchSize is 1
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.dither is FALSE
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.dropout is 0
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.dropout.dropConnect is FALSE
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.dropout.momentMatching is 0
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.dropout.oneMaskPerEpoch is FALSE
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.elu.alpha is 1
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.errorFunction is "crossEntropyError"
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.finalMomentum is 0.9
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.fineTuneFunction is "backpropagation"
INFO [2017-09-07 06:24:59] [backprop] Using backpropagation for fine-tuning
INFO [2017-09-07 06:24:59] [backprop] Parameter bp.learnRate is c(1, 1)
INFO [2017-09-07 06:24:59] [backprop] Parameter bp.learnRateScale is 1
INFO [2017-09-07 06:24:59] [backprop] See ?backpropagation for documentation
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.initialMomentum is 0.5
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.isClass is TRUE
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.maxout.poolSize is 2
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.maxout.unitFunction is "linearUnit"
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.momentumRampLength is 1
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.nesterovMomentum is TRUE
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.numEpochs is 100
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.returnBestModel is TRUE
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.returnBestModel.validationErrorFactor is 0.632120558828558
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.stopClassErr is -Inf
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.stopErr is -Inf
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.stopValidClassErr is -Inf
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.stopValidErr is -Inf
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.trainLayers is c(TRUE, TRUE)
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.unitFunction is "sigmoidUnit"
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.weightDecay is 0
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.weightUpdateFunction is "weightDecayWeightUpdate"
INFO [2017-09-07 06:24:59] The network has been fine-tuned for 0 epochs
INFO [2017-09-07 06:24:59] Training set consists of 150 samples.
INFO [2017-09-07 06:24:59] Start deep architecture fine-tuning for 100 epochs
INFO [2017-09-07 06:24:59] Number of Batches: 150 (batch size 1)
INFO [2017-09-07 06:24:59] Epoch: 1 of 100
INFO [2017-09-07 06:24:59] Classification error on Train set: 30% (45/150)
INFO [2017-09-07 06:24:59] Train set Cross Entropy error: 1.011
INFO [2017-09-07 06:24:59] Finished epoch 1 of 100 after 0.138 secs (1099 patterns/sec)
INFO [2017-09-07 06:24:59] Epoch: 2 of 100
INFO [2017-09-07 06:24:59] Classification error on Train set: 32% (48/150)
INFO [2017-09-07 06:24:59] Train set Cross Entropy error: 0.952
INFO [2017-09-07 06:24:59] Finished epoch 2 of 100 after 0.0717 secs (2146 patterns/sec)
INFO [2017-09-07 06:24:59] Epoch: 3 of 100
INFO [2017-09-07 06:24:59] Classification error on Train set: 18% (27/150)
INFO [2017-09-07 06:24:59] Train set Cross Entropy error: 0.789
INFO [2017-09-07 06:24:59] Finished epoch 3 of 100 after 0.056 secs (2760 patterns/sec)
INFO [2017-09-07 06:24:59] Epoch: 4 of 100
INFO [2017-09-07 06:24:59] Classification error on Train set: 16.67% (25/150)
INFO [2017-09-07 06:24:59] Train set Cross Entropy error: 0.747
INFO [2017-09-07 06:24:59] Finished epoch 4 of 100 after 0.057 secs (2713 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 5 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 22% (33/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.920
INFO [2017-09-07 06:25:00] Finished epoch 5 of 100 after 0.0558 secs (2774 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 6 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.442
INFO [2017-09-07 06:25:00] Finished epoch 6 of 100 after 0.0536 secs (2883 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 7 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 8.67% (13/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.539
INFO [2017-09-07 06:25:00] Finished epoch 7 of 100 after 0.0567 secs (2724 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 8 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 6% (9/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.517
INFO [2017-09-07 06:25:00] Finished epoch 8 of 100 after 0.0557 secs (2777 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 9 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 4.67% (7/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.533
INFO [2017-09-07 06:25:00] Finished epoch 9 of 100 after 0.0541 secs (2863 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 10 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 16% (24/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.672
INFO [2017-09-07 06:25:00] Finished epoch 10 of 100 after 0.0557 secs (2779 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 11 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 14% (21/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.586
INFO [2017-09-07 06:25:00] Finished epoch 11 of 100 after 0.054 secs (2865 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 12 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 6% (9/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.383
INFO [2017-09-07 06:25:00] Finished epoch 12 of 100 after 0.0554 secs (2790 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 13 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 15.33% (23/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.645
INFO [2017-09-07 06:25:00] Finished epoch 13 of 100 after 0.0541 secs (2864 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 14 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 16% (24/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.726
INFO [2017-09-07 06:25:00] Finished epoch 14 of 100 after 0.0552 secs (2806 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 15 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 6.67% (10/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.355
INFO [2017-09-07 06:25:00] Finished epoch 15 of 100 after 0.0538 secs (2874 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 16 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 8% (12/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.515
INFO [2017-09-07 06:25:00] Finished epoch 16 of 100 after 0.0558 secs (2773 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 17 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.280
INFO [2017-09-07 06:25:00] Finished epoch 17 of 100 after 0.0569 secs (2717 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 18 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.279
INFO [2017-09-07 06:25:00] Finished epoch 18 of 100 after 0.0552 secs (2802 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 19 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 11.33% (17/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.525
INFO [2017-09-07 06:25:00] Finished epoch 19 of 100 after 0.0575 secs (2691 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 20 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.243
INFO [2017-09-07 06:25:00] Finished epoch 20 of 100 after 0.0555 secs (2797 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 21 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.233
INFO [2017-09-07 06:25:00] Finished epoch 21 of 100 after 0.0576 secs (2687 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch: 22 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.302
INFO [2017-09-07 06:25:00] Finished epoch 22 of 100 after 0.0548 secs (2828 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 23 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.533
INFO [2017-09-07 06:25:01] Finished epoch 23 of 100 after 0.0609 secs (2532 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 24 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.249
INFO [2017-09-07 06:25:01] Finished epoch 24 of 100 after 0.0547 secs (2832 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 25 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.189
INFO [2017-09-07 06:25:01] Finished epoch 25 of 100 after 0.178 secs (851 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 26 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.253
INFO [2017-09-07 06:25:01] Finished epoch 26 of 100 after 0.054 secs (2875 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 27 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.209
INFO [2017-09-07 06:25:01] Finished epoch 27 of 100 after 0.0586 secs (2643 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 28 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 6.67% (10/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.306
INFO [2017-09-07 06:25:01] Finished epoch 28 of 100 after 0.0556 secs (2776 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 29 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.225
INFO [2017-09-07 06:25:01] Finished epoch 29 of 100 after 0.0526 secs (2945 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 30 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 6.67% (10/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.305
INFO [2017-09-07 06:25:01] Finished epoch 30 of 100 after 0.0541 secs (2857 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 31 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 8% (12/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.374
INFO [2017-09-07 06:25:01] Finished epoch 31 of 100 after 0.0673 secs (2296 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 32 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.214
INFO [2017-09-07 06:25:01] Finished epoch 32 of 100 after 0.0737 secs (2097 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 33 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.204
INFO [2017-09-07 06:25:01] Finished epoch 33 of 100 after 0.0598 secs (2600 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 34 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.249
INFO [2017-09-07 06:25:01] Finished epoch 34 of 100 after 0.0637 secs (2417 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 35 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.202
INFO [2017-09-07 06:25:01] Finished epoch 35 of 100 after 0.057 secs (2716 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 36 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 14% (21/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.635
INFO [2017-09-07 06:25:01] Finished epoch 36 of 100 after 0.061 secs (2533 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch: 37 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.235
INFO [2017-09-07 06:25:02] Finished epoch 37 of 100 after 0.0612 secs (2520 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 38 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 6.67% (10/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.307
INFO [2017-09-07 06:25:02] Finished epoch 38 of 100 after 0.0569 secs (2723 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 39 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.174
INFO [2017-09-07 06:25:02] Finished epoch 39 of 100 after 0.0584 secs (2649 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 40 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.217
INFO [2017-09-07 06:25:02] Finished epoch 40 of 100 after 0.064 secs (2410 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 41 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.182
INFO [2017-09-07 06:25:02] Finished epoch 41 of 100 after 0.0974 secs (1636 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 42 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.217
INFO [2017-09-07 06:25:02] Finished epoch 42 of 100 after 0.109 secs (1395 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 43 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.222
INFO [2017-09-07 06:25:02] Finished epoch 43 of 100 after 0.118 secs (1310 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 44 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.168
INFO [2017-09-07 06:25:02] Finished epoch 44 of 100 after 0.109 secs (1423 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 45 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.234
INFO [2017-09-07 06:25:02] Finished epoch 45 of 100 after 0.0835 secs (1834 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 46 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 7.33% (11/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.432
INFO [2017-09-07 06:25:02] Finished epoch 46 of 100 after 0.0538 secs (2874 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 47 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.231
INFO [2017-09-07 06:25:02] Finished epoch 47 of 100 after 0.0558 secs (2776 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 48 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 14% (21/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.649
INFO [2017-09-07 06:25:02] Finished epoch 48 of 100 after 0.054 secs (2866 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 49 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.165
INFO [2017-09-07 06:25:02] Finished epoch 49 of 100 after 0.0546 secs (2835 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 50 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 20% (30/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.634
INFO [2017-09-07 06:25:02] Finished epoch 50 of 100 after 0.057 secs (2711 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch: 51 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 6% (9/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.321
INFO [2017-09-07 06:25:03] Finished epoch 51 of 100 after 0.0543 secs (2852 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 52 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.269
INFO [2017-09-07 06:25:03] Finished epoch 52 of 100 after 0.0548 secs (2821 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 53 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.176
INFO [2017-09-07 06:25:03] Finished epoch 53 of 100 after 0.0562 secs (2750 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 54 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.252
INFO [2017-09-07 06:25:03] Finished epoch 54 of 100 after 0.0581 secs (2663 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 55 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.292
INFO [2017-09-07 06:25:03] Finished epoch 55 of 100 after 0.0547 secs (2829 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 56 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.225
INFO [2017-09-07 06:25:03] Finished epoch 56 of 100 after 0.0555 secs (2787 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 57 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.225
INFO [2017-09-07 06:25:03] Finished epoch 57 of 100 after 0.053 secs (2924 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 58 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.199
INFO [2017-09-07 06:25:03] Finished epoch 58 of 100 after 0.0545 secs (2845 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 59 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 6% (9/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.271
INFO [2017-09-07 06:25:03] Finished epoch 59 of 100 after 0.0553 secs (2796 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 60 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 4.67% (7/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.286
INFO [2017-09-07 06:25:03] Finished epoch 60 of 100 after 0.0528 secs (2939 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 61 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.196
INFO [2017-09-07 06:25:03] Finished epoch 61 of 100 after 0.0534 secs (2898 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 62 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.239
INFO [2017-09-07 06:25:03] Finished epoch 62 of 100 after 0.0549 secs (2820 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 63 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 14% (21/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.779
INFO [2017-09-07 06:25:03] Finished epoch 63 of 100 after 0.0527 secs (2934 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 64 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.157
INFO [2017-09-07 06:25:03] Finished epoch 64 of 100 after 0.0545 secs (2844 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 65 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 4.67% (7/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.224
INFO [2017-09-07 06:25:03] Finished epoch 65 of 100 after 0.0554 secs (2792 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 66 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.181
INFO [2017-09-07 06:25:03] Finished epoch 66 of 100 after 0.053 secs (2923 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 67 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.204
INFO [2017-09-07 06:25:03] Finished epoch 67 of 100 after 0.0538 secs (2879 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 68 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 9.33% (14/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.553
INFO [2017-09-07 06:25:03] Finished epoch 68 of 100 after 0.0563 secs (2747 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch: 69 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.200
INFO [2017-09-07 06:25:04] Finished epoch 69 of 100 after 0.0523 secs (2967 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 70 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.176
INFO [2017-09-07 06:25:04] Finished epoch 70 of 100 after 0.0537 secs (2882 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 71 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 13.33% (20/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.720
INFO [2017-09-07 06:25:04] Finished epoch 71 of 100 after 0.0552 secs (2804 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 72 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.153
INFO [2017-09-07 06:25:04] Finished epoch 72 of 100 after 0.0524 secs (2957 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 73 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.205
INFO [2017-09-07 06:25:04] Finished epoch 73 of 100 after 0.0548 secs (2828 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 74 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 7.33% (11/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.351
INFO [2017-09-07 06:25:04] Finished epoch 74 of 100 after 0.0562 secs (2751 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 75 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.284
INFO [2017-09-07 06:25:04] Finished epoch 75 of 100 after 0.0533 secs (2906 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 76 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 6.67% (10/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.295
INFO [2017-09-07 06:25:04] Finished epoch 76 of 100 after 0.0552 secs (2805 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 77 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.170
INFO [2017-09-07 06:25:04] Finished epoch 77 of 100 after 0.0563 secs (2741 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 78 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 8.67% (13/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.516
INFO [2017-09-07 06:25:04] Finished epoch 78 of 100 after 0.0525 secs (2953 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 79 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.150
INFO [2017-09-07 06:25:04] Finished epoch 79 of 100 after 0.0546 secs (2832 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 80 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 6% (9/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.428
INFO [2017-09-07 06:25:04] Finished epoch 80 of 100 after 0.0556 secs (2780 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 81 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 13.33% (20/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.637
INFO [2017-09-07 06:25:04] Finished epoch 81 of 100 after 0.0518 secs (2986 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 82 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.151
INFO [2017-09-07 06:25:04] Finished epoch 82 of 100 after 0.0537 secs (2887 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 83 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.202
INFO [2017-09-07 06:25:04] Finished epoch 83 of 100 after 0.0557 secs (2773 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 84 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 6% (9/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.244
INFO [2017-09-07 06:25:04] Finished epoch 84 of 100 after 0.053 secs (2920 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 85 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 14% (21/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.459
INFO [2017-09-07 06:25:04] Finished epoch 85 of 100 after 0.0541 secs (2862 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 86 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 8.67% (13/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.452
INFO [2017-09-07 06:25:04] Finished epoch 86 of 100 after 0.0558 secs (2778 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch: 87 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 10% (15/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.646
INFO [2017-09-07 06:25:05] Finished epoch 87 of 100 after 0.056 secs (2760 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 88 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.238
INFO [2017-09-07 06:25:05] Finished epoch 88 of 100 after 0.0537 secs (2887 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 89 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 16% (24/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.738
INFO [2017-09-07 06:25:05] Finished epoch 89 of 100 after 0.0589 secs (2620 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 90 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.308
INFO [2017-09-07 06:25:05] Finished epoch 90 of 100 after 0.0523 secs (2970 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 91 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.179
INFO [2017-09-07 06:25:05] Finished epoch 91 of 100 after 0.06 secs (2591 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 92 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 4.67% (7/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.269
INFO [2017-09-07 06:25:05] Finished epoch 92 of 100 after 0.0625 secs (2461 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 93 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.172
INFO [2017-09-07 06:25:05] Finished epoch 93 of 100 after 0.0526 secs (2940 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 94 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 17.33% (26/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.765
INFO [2017-09-07 06:25:05] Finished epoch 94 of 100 after 0.0534 secs (2901 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 95 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.209
INFO [2017-09-07 06:25:05] Finished epoch 95 of 100 after 0.0558 secs (2769 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 96 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.211
INFO [2017-09-07 06:25:05] Finished epoch 96 of 100 after 0.0523 secs (2958 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 97 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.195
INFO [2017-09-07 06:25:05] Finished epoch 97 of 100 after 0.0534 secs (2895 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 98 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.174
INFO [2017-09-07 06:25:05] Finished epoch 98 of 100 after 0.0563 secs (2742 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 99 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.262
INFO [2017-09-07 06:25:05] Finished epoch 99 of 100 after 0.0514 secs (3011 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 100 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 8.67% (13/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.402
INFO [2017-09-07 06:25:05] Finished epoch 100 of 100 after 0.0523 secs (2967 patterns/sec)
INFO [2017-09-07 06:25:05] Classification error on Train set (best model): 1.33% (2/150)
INFO [2017-09-07 06:25:05] Train set (best model) Cross Entropy error: 0.157
INFO [2017-09-07 06:25:05] Best model was found after epoch 64
INFO [2017-09-07 06:25:05] Fine-tuning finished after 6.062 secs
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.