ggml_layer_dropout: Add Dropout Layer

In ggmlR: 'GGML' Tensor Operations for Machine Learning

Add Dropout Layer

Description

Applies dropout regularization. During training, multiplies all activations by (1 - rate) (deterministic expected-value scaling). During inference (training = FALSE), the layer is an identity (no change).

Usage

ggml_layer_dropout(
  model,
  rate,
  stochastic = FALSE,
  name = NULL,
  trainable = FALSE
)

Arguments

model	A ggml_sequential_model or ggml_tensor_node.
rate	Dropout rate in [0, 1). Fraction of units to "drop".
stochastic	Logical. If TRUE, use inverted dropout with a random Bernoulli mask regenerated each epoch (proper regularization). If FALSE (default), use deterministic scaling by (1 - rate) — cheaper but weaker regularization.
name	Optional layer name.
trainable	Ignored for dropout (no weights); kept for API consistency.

Value

The model with the dropout layer appended, or a new tensor node.

Difference from Keras / inverted dropout

Keras implements inverted dropout: during training it applies a random Bernoulli mask and scales surviving activations up by 1 / (1 - rate), so the expected value of each unit is preserved and no scaling is needed at inference.

This implementation uses deterministic scaling (multiply by (1 - rate) at training, identity at inference) — equivalent in expected value but without stochastic noise. Consequences:

• No random mask → the regularization signal is weaker (no co-adaptation breaking).

• Activations at training are scaled down, not up — the magnitude seen by subsequent layers differs from Keras behaviour.

• Results are fully deterministic and reproducible without setting a seed.

Note

With stochastic = TRUE the Bernoulli mask is regenerated once per epoch (not per batch), because ggml_opt_fit processes all batches inside a single C call. This is weaker than per-batch dropout but stronger than the deterministic variant.

Examples

model <- ggml_model_sequential() |>
  ggml_layer_dense(128, activation = "relu", input_shape = 784L) |>
  ggml_layer_dropout(0.5, stochastic = TRUE) |>
  ggml_layer_dense(10, activation = "softmax")

ggmlR documentation built on July 14, 2026, 1:08 a.m.