densemlp: Fit a fast dense multilayer perceptron

In densemlp: Dense Neural Networks for Tabular Regression, Classification and Survival

Fit a fast dense multilayer perceptron

Description

A compact feedforward network for regression and classification on tabular data. Forward propagation, backpropagation, and Adam optimization are all implemented natively in C++ via RcppArmadillo – no torch / libtorch dependency.

Usage

densemlp(
  x = NULL,
  y = NULL,
  task = c("auto", "regression", "binary", "multiclass", "survival", "count"),
  loss = c("cox", "brier"),
  n_bins = 10L,
  hidden_units = c(32, 16),
  epochs = 100L,
  batch_size = 32L,
  lr = 0.001,
  validation = 0.2,
  early_stopping = TRUE,
  patience = 10L,
  min_delta = 0,
  min_epochs = max(10L, floor(epochs * 0.2)),
  residual = FALSE,
  gated = FALSE,
  dropout = 0,
  batch_norm = TRUE,
  input_projection = NULL,
  interaction = FALSE,
  ema_decay = 0,
  ensemble = 1L,
  ensemble_bootstrap = TRUE,
  lr_schedule = c("none", "cosine", "step"),
  seed = 1L,
  verbose = FALSE,
  ncores = 1L,
  formula = NULL,
  data = NULL,
  offset = NULL,
  groups = NULL,
  group_penalty = 0,
  weights = NULL
)

## S3 method for class 'densemlp'
print(x, ...)

Arguments

x	A densemlp object.
y	Outcome (x/y interface): numeric for regression, a factor/character (2 levels for binary, 3+ for multiclass) for classification, numeric proportions in ⁠[0, 1]⁠ for task = "binary" (a fractional / quasi-binomial outcome), non-negative counts for task = "count", or, for task = "survival", a survival::Surv() object or a two-column matrix/data.frame giving ⁠(time, event)⁠ (event coded 1 = event, 0 = censored).
task	"auto" infers the task from y (a survival::Surv() response, from either interface, is always detected as "survival"); otherwise one of "regression", "binary", "multiclass", "survival", "count" (Poisson log-likelihood with a log link; never inferred, so it must be requested).
loss	For task = "survival" only: "cox" (batch-wise Breslow-tie Cox partial likelihood, a single linear risk score) or "brier" (IPCW integrated Brier score over a discrete-time grid of n_bins hazard outputs; see the "Survival" section below). Ignored otherwise.
n_bins	For ⁠task = "survival", loss = "brier"⁠ only: number of discrete-time bins (quantile cutpoints of the observed follow-up times).
hidden_units	Integer vector of hidden layer sizes. An empty vector (integer(0)) gives a linear model (a GLM fitted by stochastic gradient descent).
epochs	Maximum number of training epochs (an upper bound when early_stopping is used).
batch_size	Mini-batch size.
lr	Adam learning rate.
validation	Validation fraction in ⁠[0, 1)⁠ held out for early stopping, or an integer vector of at least two row indices to hold out (for example a split by cluster or by time). Set to 0 to disable (trains for the full epochs, no BN running-stat evaluation set). Row indices cannot be combined with ensemble > 1.
early_stopping	Logical; stop once validation loss stops improving. Ignored if validation = 0.
patience	Number of non-improving epochs to wait before stopping.
min_delta	Minimum validation loss improvement to reset patience.
min_epochs	Minimum number of epochs before early stopping can trigger. Defaults to max(10, floor(epochs * 0.2)).
residual	Logical; add a residual skip to every hidden block. A learned linear projection is used when the block dimensions differ.
gated	Logical; use a learned sigmoid gate in every hidden block.
dropout	Dropout probability, either one value or one per hidden layer. Values must be in ⁠[0, 1)⁠.
batch_norm	Logical; apply batch normalization inside every hidden block (Linear -> BatchNorm -> ReLU). When FALSE, hidden blocks are Linear -> ReLU with no normalization and no learned BN affine parameters.
input_projection	Optional positive integer. When set, a plain linear layer (no activation, no batch normalization) maps the encoded predictors to input_projection dimensions before the first hidden block. NULL (default) disables it. Cannot be combined with interaction.
interaction	Logical; prepend an efficient learned cross-feature layer that models explicit second-order interactions in O(p) parameters.
ema_decay	Exponential moving-average decay for model parameters. Set to 0 to disable; values such as 0.99 enable EMA evaluation and best-epoch restoration.
ensemble	Number of internally fitted members whose predictions are averaged. 1 fits a single model and preserves the standard behavior.
ensemble_bootstrap	Logical; bootstrap rows independently for each member when ensemble > 1.
lr_schedule	Learning-rate schedule: "none", cosine annealing over epochs, or "step" decay by 0.5 every max(5, floor(epochs / 3)) epochs.
seed	Integer seed.
verbose	Logical; print training/validation loss every 10 epochs.
ncores	Number of cores used to fit ensemble members in parallel when ensemble > 1 (via parallel::mclapply on Unix-alikes, serially on Windows). Ignored when ensemble = 1.
formula	A formula, e.g. y ~ . or, for survival, survival::Surv(time, status) ~ ., as an alternative to x/y. Use either formula/data or x/y, not both. A formula may be given as the first positional argument – densemlp(y ~ ., df), densemlp(formula = y ~ ., data = df) and densemlp(x, y) all work.
data	A data.frame used with formula.
offset	Optional fixed term added to the linear predictor (the output before its activation), on the scale of the outcome for regression and of the link otherwise (log for "count", logit for "binary"). A numeric vector with one value per row, or for "multiclass" a matrix with one column per class. Not available for task = "survival".
groups	Optional grouping factor(s): a vector, or a list or data.frame of vectors, each with one value per row. Every level of every grouping factor gets its own learned intercept on the linear predictor (one per class for multiclass), trained jointly with the network. This gives fixed-effect style cluster or period intercepts, as in panel data. Not available for task = "survival".
group_penalty	Non-negative ridge penalty on the group intercepts, added to each mini-batch loss as group_penalty times the mean squared intercept of each grouping factor. Either one value for all grouping factors or one value per factor. 0 (default) leaves them unpenalized.
weights	Optional non-negative case weights, one per row. Each row's contribution to the loss (training and validation) is multiplied by its weight, so the network minimizes a weighted empirical risk (only relative weights matter: weights and c * weights give the same fit), for example importance weights p_target(x) / p_train(x) under covariate shift, survey weights, or inverse-probability weights. Training weights are rescaled to mean one (the learning rate keeps its meaning), batch normalization uses weighted batch statistics, the validation loss used for early stopping is a weighted mean, and survival uses a weighted Breslow partial likelihood or a weighted IPCW Brier score (with a weighted Kaplan-Meier censoring estimate). Numeric inputs (and a regression outcome) are standardized with weighted means and standard deviations, so integer weights behave like repeated rows (up to the n - 1 denominator of the standard deviations). Rows with weight 0 do not contribute. NULL (default) fits unweighted.
...	Unused.

Details

Hidden layers are ⁠Linear -> BatchNorm -> ReLU -> optional gate -> optional dropout -> optional residual⁠ (batch statistics during training, running mean/var at prediction time, exponential decay 0.9), with He initialization for the linear weights. Set batch_norm = FALSE to drop the normalization step, leaving Linear -> ReLU hidden blocks. Residual blocks use a learned linear projection when dimensions differ. With input_projection = k, a bare linear layer maps the encoded inputs to k dimensions before the first hidden block (mutually exclusive with interaction). The optional interaction layer is a one-layer cross network initialized as the identity. With ema_decay > 0, validation and final predictions use moving-average parameters. With ensemble > 1, fully fitted internal members are trained with successive seeds and optionally bootstrapped rows, and their response probabilities/predictions are averaged transparently. The output layer is plain ⁠Linear -> task activation⁠ (no BN): linear (regression, MSE loss), sigmoid (binary, binary cross-entropy), softmax (multiclass, categorical cross-entropy), exponential (count, Poisson log-likelihood), or a linear risk score (survival, Cox partial likelihood); the first four share the output-gradient simplification dZ = (yhat - y) / n (times the case weight when weights is given), while survival uses its own closed-form Cox gradient (see the "Survival" section below). With validation > 0 and early_stopping = TRUE (both on by default), the parameters (weights, biases, and BN affine/running-stat parameters) from the best validation epoch are restored at the end.

offset and groups both enter the linear predictor additively, after the output layer and before its activation, so the fitted model is ⁠g(E[y]) = f(x) + offset + sum_k alpha_k[group_k]⁠, where f is the network and the alpha_k are the per-level intercepts (see group_effects()).

Value

A densemlp object.

Survival

task = "survival" supports two losses, both trained batch-wise against a ⁠(time, event)⁠ outcome:

• loss = "cox" (the default) trains a single linear risk score using a batch-wise Breslow-tie Cox partial log-likelihood – the same "risk set = current mini-batch" approximation used by DeepSurv/pycox, which turns Cox regression into an ordinary per-batch loss compatible with masked-free Adam training, BN, and early stopping unchanged. For a batch sorted by descending time, with each observation's risk set approximated by the sorted prefix of observations with a time at least as large, the loss is the negative mean, over events, of the risk score minus the log of the cumulative sum of exponentiated risk scores over its risk set – computed via a single cumulative sum with a closed-form O(n) gradient (no autodiff).

• loss = "brier" trains a discrete-time hazard head with n_bins outputs (a quantile grid of the observed follow-up times) directly against the IPCW (Graf et al.) integrated Brier score. Bin k's sigmoid output is a conditional hazard, and the survival probability at that bin is the running product of one minus every hazard up to and including it, which is monotonically non-increasing by construction. Each bin's Brier term is inverse-probability-of-censoring weighted using a Kaplan-Meier estimate of the censoring distribution fit once on the training data; subjects censored before a bin are excluded from that bin's term, per Graf's correction. The gradient is closed-form (no autodiff): a bin's survival probability depends on every hazard logit at or before it, so each bin's Brier-score gradient is distributed back across all of them.

predict(fit, newdata, type = "response") returns a linear risk score for either loss (for "brier", the negative log of the predicted survival probability at the final time bin, so higher is still riskier); with loss = "brier", type = "survival" additionally returns the full n_bins-column survival-probability matrix. densemlp_metrics() reports Harrell's concordance index for task = "survival" regardless of loss.

Examples

set.seed(1)
x <- data.frame(a = rnorm(100), b = rnorm(100))
y <- x$a - 0.5 * x$b + rnorm(100, sd = 0.1)
fit <- densemlp(x, y, epochs = 50, hidden_units = c(16))
predict(fit, x[1:5, ])

densemlp documentation built on Nov. 3, 2026, 1:09 a.m.