backpropagation: Backpropagation learning function

Description Usage Arguments Details Value References See Also Examples

View source: R/backpropagation.R

Description

This function provides the backpropagation algorithm for deep architectures.

Usage

1
2
3
4
5
6
7
8
9
backpropagation(darch, trainData, targetData,
  bp.learnRate = getParameter(".bp.learnRate", rep(1, times =
  length(darch@layers))),
  bp.learnRateScale = getParameter(".bp.learnRateScale"),
  nesterovMomentum = getParameter(".darch.nesterovMomentum"),
  dropout = getParameter(".darch.dropout", rep(0, times = length(darch@layers)
  + 1), darch), dropConnect = getParameter(".darch.dropout.dropConnect"),
  matMult = getParameter(".matMult"), debugMode = getParameter(".debug", F),
  ...)

Arguments

darch

An instance of the class DArch.

trainData

The training data (inputs).

targetData

The target data (outputs).

bp.learnRate

Learning rates for backpropagation, length is either one or the same as the number of weight matrices when using different learning rates for each layer.

bp.learnRateScale

The learn rate is multiplied by this value after each epoch.

nesterovMomentum

See darch.nesterovMomentum parameter of darch.

dropout

See darch.dropout parameter of darch.

dropConnect

See darch.dropout.dropConnect parameter of darch.

matMult

Matrix multiplication function, internal parameter.

debugMode

Whether debug mode is enabled, internal parameter.

...

Further parameters.

Details

The only backpropagation-specific, user-relevant parameters are bp.learnRate and bp.learnRateScale; they can be passed to the darch function when enabling backpropagation as the fine-tuning function. bp.learnRate defines the backpropagation learning rate and can either be specified as a single scalar or as a vector with one entry for each weight matrix, allowing for per-layer learning rates. bp.learnRateScale is a single scalar which contains a scaling factor for the learning rate(s) which will be applied after each epoch.

Backpropagation supports dropout and uses the weight update function as defined via the darch.weightUpdateFunction parameter of darch.

Value

The trained deep architecture

References

Rumelhart, D., G. E. Hinton, R. J. Williams, Learning representations by backpropagating errors, Nature 323, S. 533-536, DOI: 10.1038/323533a0, 1986.

See Also

darch

Other fine-tuning functions: minimizeAutoencoder, minimizeClassifier, rpropagation

Examples

1
2
3
4
5
## Not run: 
data(iris)
model <- darch(Species ~ ., iris, darch.fineTuneFunction = "backpropagation")

## End(Not run)

Example output

INFO [2017-09-07 06:24:59] The current log level is: INFO
INFO [2017-09-07 06:24:59] Start initial caret pre-processing.
INFO [2017-09-07 06:24:59] Converting non-numeric columns in data (if any)...
INFO [2017-09-07 06:24:59] Converting non-numeric columns in targets (if any)...
INFO [2017-09-07 06:24:59] Dependent factor "Species" converted to 3 new variables (1-of-n coding)
INFO [2017-09-07 06:24:59] The current log level is: INFO
INFO [2017-09-07 06:24:59] Using CPU matrix multiplication.
WARN [2017-09-07 06:24:59] No vector given for "layers" parameter, constructing shallow network with one hidden layer of 10 neurons.
INFO [2017-09-07 06:24:59] Creating and configuring new DArch instance
INFO [2017-09-07 06:24:59] Constructing a network with 3 layers (4, 10, 3 neurons).
INFO [2017-09-07 06:24:59] Generating RBMs.
INFO [2017-09-07 06:24:59] Constructing new RBM instance with 4 visible and 10 hidden units.
INFO [2017-09-07 06:24:59] Constructing new RBM instance with 10 visible and 3 hidden units.
INFO [2017-09-07 06:24:59] DArch instance ready for training, here is a summary of its configuration:
INFO [2017-09-07 06:24:59] Global parameters:
INFO [2017-09-07 06:24:59] Layers parameter was 10, resulted in network with 3 layers and 4, 10, 3 neurons
INFO [2017-09-07 06:24:59] The weights for the layers were generated with "generateWeightsGlorotUniform"
INFO [2017-09-07 06:24:59] Additionally, the following parameters were used for weight generation:
INFO [2017-09-07 06:24:59] [weights] Parameter weights.max is 0.1
INFO [2017-09-07 06:24:59] [weights] Parameter weights.min is -0.1
INFO [2017-09-07 06:24:59] [weights] Parameter weights.mean is 0
INFO [2017-09-07 06:24:59] [weights] Parameter weights.sd is 0.01
INFO [2017-09-07 06:24:59] Weight normalization is disabled
INFO [2017-09-07 06:24:59] Bootstrapping is disabled
INFO [2017-09-07 06:24:59] Train data are shuffled before each epoch
INFO [2017-09-07 06:24:59] Autosaving is disabled
INFO [2017-09-07 06:24:59] Using CPU for matrix multiplication
INFO [2017-09-07 06:24:59] Pre-processing parameters:
INFO [2017-09-07 06:24:59] [preProc] Parameter preProc.factorToNumeric is FALSE
INFO [2017-09-07 06:24:59] [preProc] Parameter preProc.factorToNumeric.targets is FALSE
INFO [2017-09-07 06:24:59] [preProc] Parameter preProc.fullRank is TRUE
INFO [2017-09-07 06:24:59] [preProc] Parameter preProc.fullRank.targets is FALSE
INFO [2017-09-07 06:24:59] [preProc] Parameter preProc.orderedToFactor.targets is TRUE
INFO [2017-09-07 06:24:59] [preProc] Parameter preProc.targets is FALSE
INFO [2017-09-07 06:24:59] Caret pre-processing is disabled
INFO [2017-09-07 06:24:59] Pre-training parameters:
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.allData is FALSE
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.batchSize is 1
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.consecutive is TRUE
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.errorFunction is "mseError"
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.finalMomentum is 0.9
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.initialMomentum is 0.5
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.lastLayer is 0
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.learnRate is 1
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.learnRateScale is 1
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.momentumRampLength is 1
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.numCD is 1
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.numEpochs is 0
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.unitFunction is "sigmoidUnitRbm"
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.updateFunction is "rbmUpdate"
INFO [2017-09-07 06:24:59] [preTrain] Parameter rbm.weightDecay is 2e-04
INFO [2017-09-07 06:24:59] The selected RBMs have been trained for 0 epochs
INFO [2017-09-07 06:24:59] Fine-tuning parameters:
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.batchSize is 1
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.dither is FALSE
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.dropout is 0
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.dropout.dropConnect is FALSE
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.dropout.momentMatching is 0
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.dropout.oneMaskPerEpoch is FALSE
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.elu.alpha is 1
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.errorFunction is "crossEntropyError"
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.finalMomentum is 0.9
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.fineTuneFunction is "backpropagation"
INFO [2017-09-07 06:24:59] [backprop] Using backpropagation for fine-tuning
INFO [2017-09-07 06:24:59] [backprop] Parameter bp.learnRate is c(1, 1)
INFO [2017-09-07 06:24:59] [backprop] Parameter bp.learnRateScale is 1
INFO [2017-09-07 06:24:59] [backprop] See ?backpropagation for documentation
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.initialMomentum is 0.5
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.isClass is TRUE
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.maxout.poolSize is 2
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.maxout.unitFunction is "linearUnit"
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.momentumRampLength is 1
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.nesterovMomentum is TRUE
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.numEpochs is 100
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.returnBestModel is TRUE
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.returnBestModel.validationErrorFactor is 0.632120558828558
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.stopClassErr is -Inf
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.stopErr is -Inf
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.stopValidClassErr is -Inf
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.stopValidErr is -Inf
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.trainLayers is c(TRUE, TRUE)
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.unitFunction is "sigmoidUnit"
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.weightDecay is 0
INFO [2017-09-07 06:24:59] [fineTune] Parameter darch.weightUpdateFunction is "weightDecayWeightUpdate"
INFO [2017-09-07 06:24:59] The network has been fine-tuned for 0 epochs
INFO [2017-09-07 06:24:59] Training set consists of 150 samples.
INFO [2017-09-07 06:24:59] Start deep architecture fine-tuning for 100 epochs
INFO [2017-09-07 06:24:59] Number of Batches: 150 (batch size 1)
INFO [2017-09-07 06:24:59] Epoch:   1 of 100
INFO [2017-09-07 06:24:59] Classification error on Train set: 30% (45/150)
INFO [2017-09-07 06:24:59] Train set Cross Entropy error: 1.011
INFO [2017-09-07 06:24:59] Finished epoch   1 of 100 after 0.138 secs (1099 patterns/sec)
INFO [2017-09-07 06:24:59] Epoch:   2 of 100
INFO [2017-09-07 06:24:59] Classification error on Train set: 32% (48/150)
INFO [2017-09-07 06:24:59] Train set Cross Entropy error: 0.952
INFO [2017-09-07 06:24:59] Finished epoch   2 of 100 after 0.0717 secs (2146 patterns/sec)
INFO [2017-09-07 06:24:59] Epoch:   3 of 100
INFO [2017-09-07 06:24:59] Classification error on Train set: 18% (27/150)
INFO [2017-09-07 06:24:59] Train set Cross Entropy error: 0.789
INFO [2017-09-07 06:24:59] Finished epoch   3 of 100 after 0.056 secs (2760 patterns/sec)
INFO [2017-09-07 06:24:59] Epoch:   4 of 100
INFO [2017-09-07 06:24:59] Classification error on Train set: 16.67% (25/150)
INFO [2017-09-07 06:24:59] Train set Cross Entropy error: 0.747
INFO [2017-09-07 06:24:59] Finished epoch   4 of 100 after 0.057 secs (2713 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch:   5 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 22% (33/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.920
INFO [2017-09-07 06:25:00] Finished epoch   5 of 100 after 0.0558 secs (2774 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch:   6 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.442
INFO [2017-09-07 06:25:00] Finished epoch   6 of 100 after 0.0536 secs (2883 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch:   7 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 8.67% (13/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.539
INFO [2017-09-07 06:25:00] Finished epoch   7 of 100 after 0.0567 secs (2724 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch:   8 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 6% (9/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.517
INFO [2017-09-07 06:25:00] Finished epoch   8 of 100 after 0.0557 secs (2777 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch:   9 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 4.67% (7/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.533
INFO [2017-09-07 06:25:00] Finished epoch   9 of 100 after 0.0541 secs (2863 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch:  10 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 16% (24/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.672
INFO [2017-09-07 06:25:00] Finished epoch  10 of 100 after 0.0557 secs (2779 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch:  11 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 14% (21/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.586
INFO [2017-09-07 06:25:00] Finished epoch  11 of 100 after 0.054 secs (2865 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch:  12 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 6% (9/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.383
INFO [2017-09-07 06:25:00] Finished epoch  12 of 100 after 0.0554 secs (2790 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch:  13 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 15.33% (23/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.645
INFO [2017-09-07 06:25:00] Finished epoch  13 of 100 after 0.0541 secs (2864 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch:  14 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 16% (24/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.726
INFO [2017-09-07 06:25:00] Finished epoch  14 of 100 after 0.0552 secs (2806 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch:  15 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 6.67% (10/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.355
INFO [2017-09-07 06:25:00] Finished epoch  15 of 100 after 0.0538 secs (2874 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch:  16 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 8% (12/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.515
INFO [2017-09-07 06:25:00] Finished epoch  16 of 100 after 0.0558 secs (2773 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch:  17 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.280
INFO [2017-09-07 06:25:00] Finished epoch  17 of 100 after 0.0569 secs (2717 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch:  18 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.279
INFO [2017-09-07 06:25:00] Finished epoch  18 of 100 after 0.0552 secs (2802 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch:  19 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 11.33% (17/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.525
INFO [2017-09-07 06:25:00] Finished epoch  19 of 100 after 0.0575 secs (2691 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch:  20 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.243
INFO [2017-09-07 06:25:00] Finished epoch  20 of 100 after 0.0555 secs (2797 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch:  21 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.233
INFO [2017-09-07 06:25:00] Finished epoch  21 of 100 after 0.0576 secs (2687 patterns/sec)
INFO [2017-09-07 06:25:00] Epoch:  22 of 100
INFO [2017-09-07 06:25:00] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:00] Train set Cross Entropy error: 0.302
INFO [2017-09-07 06:25:00] Finished epoch  22 of 100 after 0.0548 secs (2828 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch:  23 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.533
INFO [2017-09-07 06:25:01] Finished epoch  23 of 100 after 0.0609 secs (2532 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch:  24 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.249
INFO [2017-09-07 06:25:01] Finished epoch  24 of 100 after 0.0547 secs (2832 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch:  25 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.189
INFO [2017-09-07 06:25:01] Finished epoch  25 of 100 after 0.178 secs (851 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch:  26 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.253
INFO [2017-09-07 06:25:01] Finished epoch  26 of 100 after 0.054 secs (2875 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch:  27 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.209
INFO [2017-09-07 06:25:01] Finished epoch  27 of 100 after 0.0586 secs (2643 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch:  28 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 6.67% (10/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.306
INFO [2017-09-07 06:25:01] Finished epoch  28 of 100 after 0.0556 secs (2776 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch:  29 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.225
INFO [2017-09-07 06:25:01] Finished epoch  29 of 100 after 0.0526 secs (2945 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch:  30 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 6.67% (10/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.305
INFO [2017-09-07 06:25:01] Finished epoch  30 of 100 after 0.0541 secs (2857 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch:  31 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 8% (12/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.374
INFO [2017-09-07 06:25:01] Finished epoch  31 of 100 after 0.0673 secs (2296 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch:  32 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.214
INFO [2017-09-07 06:25:01] Finished epoch  32 of 100 after 0.0737 secs (2097 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch:  33 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.204
INFO [2017-09-07 06:25:01] Finished epoch  33 of 100 after 0.0598 secs (2600 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch:  34 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.249
INFO [2017-09-07 06:25:01] Finished epoch  34 of 100 after 0.0637 secs (2417 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch:  35 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.202
INFO [2017-09-07 06:25:01] Finished epoch  35 of 100 after 0.057 secs (2716 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch:  36 of 100
INFO [2017-09-07 06:25:01] Classification error on Train set: 14% (21/150)
INFO [2017-09-07 06:25:01] Train set Cross Entropy error: 0.635
INFO [2017-09-07 06:25:01] Finished epoch  36 of 100 after 0.061 secs (2533 patterns/sec)
INFO [2017-09-07 06:25:01] Epoch:  37 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.235
INFO [2017-09-07 06:25:02] Finished epoch  37 of 100 after 0.0612 secs (2520 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch:  38 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 6.67% (10/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.307
INFO [2017-09-07 06:25:02] Finished epoch  38 of 100 after 0.0569 secs (2723 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch:  39 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.174
INFO [2017-09-07 06:25:02] Finished epoch  39 of 100 after 0.0584 secs (2649 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch:  40 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.217
INFO [2017-09-07 06:25:02] Finished epoch  40 of 100 after 0.064 secs (2410 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch:  41 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.182
INFO [2017-09-07 06:25:02] Finished epoch  41 of 100 after 0.0974 secs (1636 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch:  42 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.217
INFO [2017-09-07 06:25:02] Finished epoch  42 of 100 after 0.109 secs (1395 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch:  43 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.222
INFO [2017-09-07 06:25:02] Finished epoch  43 of 100 after 0.118 secs (1310 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch:  44 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.168
INFO [2017-09-07 06:25:02] Finished epoch  44 of 100 after 0.109 secs (1423 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch:  45 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.234
INFO [2017-09-07 06:25:02] Finished epoch  45 of 100 after 0.0835 secs (1834 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch:  46 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 7.33% (11/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.432
INFO [2017-09-07 06:25:02] Finished epoch  46 of 100 after 0.0538 secs (2874 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch:  47 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.231
INFO [2017-09-07 06:25:02] Finished epoch  47 of 100 after 0.0558 secs (2776 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch:  48 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 14% (21/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.649
INFO [2017-09-07 06:25:02] Finished epoch  48 of 100 after 0.054 secs (2866 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch:  49 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.165
INFO [2017-09-07 06:25:02] Finished epoch  49 of 100 after 0.0546 secs (2835 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch:  50 of 100
INFO [2017-09-07 06:25:02] Classification error on Train set: 20% (30/150)
INFO [2017-09-07 06:25:02] Train set Cross Entropy error: 0.634
INFO [2017-09-07 06:25:02] Finished epoch  50 of 100 after 0.057 secs (2711 patterns/sec)
INFO [2017-09-07 06:25:02] Epoch:  51 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 6% (9/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.321
INFO [2017-09-07 06:25:03] Finished epoch  51 of 100 after 0.0543 secs (2852 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch:  52 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.269
INFO [2017-09-07 06:25:03] Finished epoch  52 of 100 after 0.0548 secs (2821 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch:  53 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.176
INFO [2017-09-07 06:25:03] Finished epoch  53 of 100 after 0.0562 secs (2750 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch:  54 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.252
INFO [2017-09-07 06:25:03] Finished epoch  54 of 100 after 0.0581 secs (2663 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch:  55 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.292
INFO [2017-09-07 06:25:03] Finished epoch  55 of 100 after 0.0547 secs (2829 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch:  56 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.225
INFO [2017-09-07 06:25:03] Finished epoch  56 of 100 after 0.0555 secs (2787 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch:  57 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.225
INFO [2017-09-07 06:25:03] Finished epoch  57 of 100 after 0.053 secs (2924 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch:  58 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.199
INFO [2017-09-07 06:25:03] Finished epoch  58 of 100 after 0.0545 secs (2845 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch:  59 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 6% (9/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.271
INFO [2017-09-07 06:25:03] Finished epoch  59 of 100 after 0.0553 secs (2796 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch:  60 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 4.67% (7/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.286
INFO [2017-09-07 06:25:03] Finished epoch  60 of 100 after 0.0528 secs (2939 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch:  61 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.196
INFO [2017-09-07 06:25:03] Finished epoch  61 of 100 after 0.0534 secs (2898 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch:  62 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.239
INFO [2017-09-07 06:25:03] Finished epoch  62 of 100 after 0.0549 secs (2820 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch:  63 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 14% (21/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.779
INFO [2017-09-07 06:25:03] Finished epoch  63 of 100 after 0.0527 secs (2934 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch:  64 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.157
INFO [2017-09-07 06:25:03] Finished epoch  64 of 100 after 0.0545 secs (2844 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch:  65 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 4.67% (7/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.224
INFO [2017-09-07 06:25:03] Finished epoch  65 of 100 after 0.0554 secs (2792 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch:  66 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.181
INFO [2017-09-07 06:25:03] Finished epoch  66 of 100 after 0.053 secs (2923 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch:  67 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.204
INFO [2017-09-07 06:25:03] Finished epoch  67 of 100 after 0.0538 secs (2879 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch:  68 of 100
INFO [2017-09-07 06:25:03] Classification error on Train set: 9.33% (14/150)
INFO [2017-09-07 06:25:03] Train set Cross Entropy error: 0.553
INFO [2017-09-07 06:25:03] Finished epoch  68 of 100 after 0.0563 secs (2747 patterns/sec)
INFO [2017-09-07 06:25:03] Epoch:  69 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.200
INFO [2017-09-07 06:25:04] Finished epoch  69 of 100 after 0.0523 secs (2967 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch:  70 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.176
INFO [2017-09-07 06:25:04] Finished epoch  70 of 100 after 0.0537 secs (2882 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch:  71 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 13.33% (20/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.720
INFO [2017-09-07 06:25:04] Finished epoch  71 of 100 after 0.0552 secs (2804 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch:  72 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.153
INFO [2017-09-07 06:25:04] Finished epoch  72 of 100 after 0.0524 secs (2957 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch:  73 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.205
INFO [2017-09-07 06:25:04] Finished epoch  73 of 100 after 0.0548 secs (2828 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch:  74 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 7.33% (11/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.351
INFO [2017-09-07 06:25:04] Finished epoch  74 of 100 after 0.0562 secs (2751 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch:  75 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.284
INFO [2017-09-07 06:25:04] Finished epoch  75 of 100 after 0.0533 secs (2906 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch:  76 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 6.67% (10/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.295
INFO [2017-09-07 06:25:04] Finished epoch  76 of 100 after 0.0552 secs (2805 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch:  77 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.170
INFO [2017-09-07 06:25:04] Finished epoch  77 of 100 after 0.0563 secs (2741 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch:  78 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 8.67% (13/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.516
INFO [2017-09-07 06:25:04] Finished epoch  78 of 100 after 0.0525 secs (2953 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch:  79 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.150
INFO [2017-09-07 06:25:04] Finished epoch  79 of 100 after 0.0546 secs (2832 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch:  80 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 6% (9/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.428
INFO [2017-09-07 06:25:04] Finished epoch  80 of 100 after 0.0556 secs (2780 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch:  81 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 13.33% (20/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.637
INFO [2017-09-07 06:25:04] Finished epoch  81 of 100 after 0.0518 secs (2986 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch:  82 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.151
INFO [2017-09-07 06:25:04] Finished epoch  82 of 100 after 0.0537 secs (2887 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch:  83 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.202
INFO [2017-09-07 06:25:04] Finished epoch  83 of 100 after 0.0557 secs (2773 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch:  84 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 6% (9/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.244
INFO [2017-09-07 06:25:04] Finished epoch  84 of 100 after 0.053 secs (2920 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch:  85 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 14% (21/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.459
INFO [2017-09-07 06:25:04] Finished epoch  85 of 100 after 0.0541 secs (2862 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch:  86 of 100
INFO [2017-09-07 06:25:04] Classification error on Train set: 8.67% (13/150)
INFO [2017-09-07 06:25:04] Train set Cross Entropy error: 0.452
INFO [2017-09-07 06:25:04] Finished epoch  86 of 100 after 0.0558 secs (2778 patterns/sec)
INFO [2017-09-07 06:25:04] Epoch:  87 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 10% (15/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.646
INFO [2017-09-07 06:25:05] Finished epoch  87 of 100 after 0.056 secs (2760 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch:  88 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.238
INFO [2017-09-07 06:25:05] Finished epoch  88 of 100 after 0.0537 secs (2887 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch:  89 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 16% (24/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.738
INFO [2017-09-07 06:25:05] Finished epoch  89 of 100 after 0.0589 secs (2620 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch:  90 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.308
INFO [2017-09-07 06:25:05] Finished epoch  90 of 100 after 0.0523 secs (2970 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch:  91 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 2% (3/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.179
INFO [2017-09-07 06:25:05] Finished epoch  91 of 100 after 0.06 secs (2591 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch:  92 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 4.67% (7/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.269
INFO [2017-09-07 06:25:05] Finished epoch  92 of 100 after 0.0625 secs (2461 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch:  93 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 1.33% (2/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.172
INFO [2017-09-07 06:25:05] Finished epoch  93 of 100 after 0.0526 secs (2940 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch:  94 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 17.33% (26/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.765
INFO [2017-09-07 06:25:05] Finished epoch  94 of 100 after 0.0534 secs (2901 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch:  95 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 4% (6/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.209
INFO [2017-09-07 06:25:05] Finished epoch  95 of 100 after 0.0558 secs (2769 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch:  96 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 5.33% (8/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.211
INFO [2017-09-07 06:25:05] Finished epoch  96 of 100 after 0.0523 secs (2958 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch:  97 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.195
INFO [2017-09-07 06:25:05] Finished epoch  97 of 100 after 0.0534 secs (2895 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch:  98 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 2.67% (4/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.174
INFO [2017-09-07 06:25:05] Finished epoch  98 of 100 after 0.0563 secs (2742 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch:  99 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 3.33% (5/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.262
INFO [2017-09-07 06:25:05] Finished epoch  99 of 100 after 0.0514 secs (3011 patterns/sec)
INFO [2017-09-07 06:25:05] Epoch: 100 of 100
INFO [2017-09-07 06:25:05] Classification error on Train set: 8.67% (13/150)
INFO [2017-09-07 06:25:05] Train set Cross Entropy error: 0.402
INFO [2017-09-07 06:25:05] Finished epoch 100 of 100 after 0.0523 secs (2967 patterns/sec)
INFO [2017-09-07 06:25:05] Classification error on Train set (best model): 1.33% (2/150)
INFO [2017-09-07 06:25:05] Train set (best model) Cross Entropy error: 0.157
INFO [2017-09-07 06:25:05] Best model was found after epoch 64
INFO [2017-09-07 06:25:05] Fine-tuning finished after 6.062 secs

darch documentation built on May 29, 2017, 8:14 p.m.