orthoMTL: Multi-task learning with orthogonal constraints

In orthoMTL: Multi-Task Learning with Orthogonal Constraints

Multi-task learning with orthogonal constraints

Description

This function solves a multi-task problem where relationships between tasks can be complex

Usage

orthoMTL(
  X,
  Y,
  lambda = 1,
  step_size = 0.1,
  tol = 1e-05,
  stop_no_improve = 100,
  max_iter = 1e+06,
  W_0 = NULL,
  seed = NULL,
  K = NULL,
  disjoint = FALSE,
  logistic = FALSE,
  alpha = 0,
  schedule = c("sqrt", "log", "const", "linear"),
  survival = FALSE,
  censored.mat = NULL,
  verbose = 0
)

Arguments

X	a matrix of predictor variables with dimensions n x p
Y	a matrix of response variables with dimensions n x numTasks, where numTasks is the number of response variables. NAs can be used for censored data.
lambda	the regularization parameter for the OrthoPen penalty, default is 1
step_size	the step size for updating the regression coefficients in gradient descent, default is 0.1
tol	Convergence tolerance. The algorithm stops when the improvement in the objective function is less than tol. Default: 1e-5.
stop_no_improve	the number of iterations without improvement in the objective function to trigger convergence, default is 100
max_iter	the maximum number of iterations, default is 1e+06
W_0	a matrix of initial values for the regression coefficients, default is NULL, as not provided and will be randomly attributed
seed	an optional random seed for reproducibility, default is NULL (no seed is set; the ambient RNG state is used as-is). Only relevant when W_0 is not provided and disjoint = TRUE, since that is the only case where orthoMTL draws random numbers. Pass an integer to make that random initialisation reproducible.
K	a constraint matrix of weights to adjust the OrthoPen penalty, default is an identity matrix with dimensions numTasks x numTasks
disjoint	a logical value indicating whether the response variables should have disjoint supports, default is FALSE
logistic	a logical value indicating whether logistic regression should be used instead of linear regression, default is FALSE
alpha	the elastic-net mixing parameter in [0, 1], default is 0. The penalty is \lambda [(1 - \alpha)/2\, \Omega_K(W)^2 + \alpha \|W\|_1]; alpha = 0 gives the pure orthogonality penalty and alpha = 1 gives a pure Lasso.
schedule	Character; the gradient-step decay schedule – how the per-iteration scale grows with the iteration index i. The coefficient update is W \leftarrow W - \texttt{step\_size}\, \nabla / (\texttt{scale} \cdot \|\nabla\|), so a faster-growing scale means a faster-decaying effective step. One of: "sqrt" (default) \texttt{scale} = \sqrt{i}, effective step \propto 1/\sqrt{i} – the classic subgradient schedule and the historical hardcoded behaviour. "log" \texttt{scale} = \log(i) + 1 (slow decay). "const" \texttt{scale} = 1 (no decay). "linear" \texttt{scale} = i, effective step \propto 1/i (Robbins-Monro). The default "sqrt" is retained for backward compatibility: it reproduces previously published results exactly. An A/B study shipped with the package (validation/ab_s02_gradient_schedule.R) found that "sqrt" reaches the correct optimum but converges to it roughly 12–17x slower than "log" and "const" (which reach the same objective in a small fraction of the iterations), while "linear" decays too aggressively and can stall short of the optimum. Prefer "log" or "const" when convergence speed matters; note that changing the schedule changes the optimisation path and therefore the exact coefficients, so it should not be altered when reproducing published runs.
survival	a logical value indicating whether survival analysis should be performed, default is FALSE
censored.mat	a matrix indicating whether observations are censored, used only if survival=TRUE
verbose	the level of verbosity, default is 0 (no messages)

Value

a list containing the following elements:

B	a matrix of regression coefficients with dimensions p x numTasks
obj	the final objective function when algorithms stops
imax	the number of iterations

References

Kevin Vervier, Pierre Mahé, Alexandre d’Aspremont, Jean-Baptiste Veyrieras, Jean-Philippe Vert (2014). On learning matrices with orthogonal columns or disjoint supports. https://hal.science/hal-00985654/file/learningDisjointSupports.pdf

Examples

# Regression with orthogonal columns
set.seed(42)
n <- 100; p <- 10; n_tasks <- 3
X <- matrix(rnorm(n * p), n, p)
W_true <- qr.Q(qr(matrix(rnorm(p * n_tasks), p, n_tasks)))
Y <- X %*% W_true + matrix(rnorm(n * n_tasks), n) * 0.1
K <- matrix(1, n_tasks, n_tasks)
diag(K) <- 0.5
fit <- orthoMTL(X, Y, lambda = 1e-3, K = K, disjoint = FALSE)
fit$B           # coefficient matrix
fit$converged   # did optimisation converge?

orthoMTL documentation built on Aug. 23, 2026, 5:10 p.m.