Optimizers: Optimizers

In kerasR: R Interface to the Keras Deep Learning Library

Optimizers

Description

Optimization functions to use in compiling a keras model. See keras_compile().

Usage

SGD(lr = 0.01, momentum = 0, decay = 0, nesterov = FALSE,
  clipnorm = -1, clipvalue = -1)

RMSprop(lr = 0.001, rho = 0.9, epsilon = 1e-08, decay = 0,
  clipnorm = -1, clipvalue = -1)

Adagrad(lr = 0.01, epsilon = 1e-08, decay = 0, clipnorm = -1,
  clipvalue = -1)

Adadelta(lr = 1, rho = 0.95, epsilon = 1e-08, decay = 0,
  clipnorm = -1, clipvalue = -1)

Adam(lr = 0.001, beta_1 = 0.9, beta_2 = 0.999, epsilon = 1e-08,
  decay = 0, clipnorm = -1, clipvalue = -1)

Adamax(lr = 0.002, beta_1 = 0.9, beta_2 = 0.999, epsilon = 1e-08,
  decay = 0, clipnorm = -1, clipvalue = -1)

Nadam(lr = 0.002, beta_1 = 0.9, beta_2 = 0.999, epsilon = 1e-08,
  schedule_decay = 0.004, clipnorm = -1, clipvalue = -1)

Arguments

lr	float >= 0. Learning rate.
momentum	float >= 0. Parameter updates momentum.
decay	float >= 0. Learning rate decay over each update.
nesterov	boolean. Whether to apply Nesterov momentum.
clipnorm	float >= 0. Gradients will be clipped when their L2 norm exceeds this value. Set to -1 to disable.
clipvalue	float >= 0. Gradients will be clipped when their absolute value exceeds this value. Set to -1 to disable.
rho	float >= 0 to be used in RMSprop
epsilon	float >= 0. Fuzz factor.
beta_1	float, 0 < beta < 1. Generally close to 1.
beta_2	float, 0 < beta < 1. Generally close to 1.
schedule_decay	float >= 0. Learning rate decay over each schedule in Nadam.

Author(s)

Taylor B. Arnold, [e-mail]

References

Chollet, Francois. 2015. Keras: Deep Learning library for Theano and TensorFlow.

Examples

if(keras_available()) {
  X_train <- matrix(rnorm(100 * 10), nrow = 100)
  Y_train <- to_categorical(matrix(sample(0:2, 100, TRUE), ncol = 1), 3)
  
  mod <- Sequential()
  mod$add(Dense(units = 50, input_shape = dim(X_train)[2]))
  mod$add(Activation("relu"))
  mod$add(Dense(units = 3))
  mod$add(Activation("softmax"))
  
  keras_compile(mod,  loss = 'categorical_crossentropy', optimizer = SGD())
  keras_fit(mod, X_train, Y_train, batch_size = 32, epochs = 5,
            verbose = 0, validation_split = 0.2)
  
  keras_compile(mod,  loss = 'categorical_crossentropy', optimizer = RMSprop())
  keras_fit(mod, X_train, Y_train, batch_size = 32, epochs = 5,
            verbose = 0, validation_split = 0.2)
  
  keras_compile(mod,  loss = 'categorical_crossentropy', optimizer = Adagrad())
  keras_fit(mod, X_train, Y_train, batch_size = 32, epochs = 5,
            verbose = 0, validation_split = 0.2)
  
  keras_compile(mod,  loss = 'categorical_crossentropy', optimizer = Adadelta())
  keras_fit(mod, X_train, Y_train, batch_size = 32, epochs = 5,
            verbose = 0, validation_split = 0.2)
  
  keras_compile(mod,  loss = 'categorical_crossentropy', optimizer = Adam())
  keras_fit(mod, X_train, Y_train, batch_size = 32, epochs = 5,
            verbose = 0, validation_split = 0.2)
  
  keras_compile(mod,  loss = 'categorical_crossentropy', optimizer = Adamax())
  keras_fit(mod, X_train, Y_train, batch_size = 32, epochs = 5,
            verbose = 0, validation_split = 0.2)
  
  keras_compile(mod,  loss = 'categorical_crossentropy', optimizer = Nadam())
  keras_fit(mod, X_train, Y_train, batch_size = 32, epochs = 5,
            verbose = 0, validation_split = 0.2)
}

kerasR documentation built on Aug. 17, 2022, 5:06 p.m.