mmes: Mixed model equations solver

View source: R/FUN_model_interface.R

mmesR Documentation

Mixed model equations solver

Description

mmes() is the main model-fitting function of sommer. It fits linear, generalized and nonlinear mixed models with arbitrary Kronecker-structured random and residual covariance, known relationship matrices, multiple responses, spatial structures and low-rank (factor analytic) models. Variance components are estimated by REML or ML using sparse or dense average-information, EM, PX-EM and related solvers implemented in C++. The same call can also solve the mixed model equations for known variance components (genetic evaluation).

Usage

mmes(
  fixed,
  random = NULL,
  rcov = NULL,
  data,
  W,
  weights = NULL,
  naMethodX = "exclude",
  naMethodY = "exclude",
  naMethodRandom = "exclude",
  naMethodR = "exclude",
  contrasts = NULL,
  family = stats::gaussian(),
  vcc = NULL,
  covPar = NULL,
  parameters = list(),
  curves = list(),
  start = NULL,
  method = mmes_method(),
  control = mmes_control(),
  familyParameters = list()
)

Arguments

fixed

Two-sided formula with the response(s) and fixed effects, e.g. y ~ Env. Offsets are added on the link scale. For several traits see stackTraits. The design must be full rank (see Details).

random

One-sided formula with the random effects, e.g. ~ Name + vsm(dsm(Env), ism(Name), Gu=Ainv). Terms without vsm are identity-covariance effects. See Details for the available covariance constructors.

rcov

One-sided formula with the residual covariance. The default ~ units fits one homogeneous residual variance. Structured residuals use a single vsm() term whose last factor is normally ism(units), e.g. ~ vsm(dsm(Env), ism(units)); complexity is expressed by adding Kronecker factors, not by summing terms. Data need not be sorted. In multi-trait models, replace units by a pairing key column (e.g. record from stackTraits); without a key, records are paired by order within trait and a warning is issued.

data

Data frame with the model variables. Auxiliary objects (relationship matrices, coordinates) may be found in the formula or calling environment.

W

Optional symmetric positive-definite observation precision (weight) matrix, e.g. when plots are correlated. It combines with any residual structure, including Kronecker multi-trait residuals, and with the IRLS weights of non-Gaussian PQL fits.

weights

Optional one-sided formula identifying independent blocks of W, e.g. ~trial. Rows from different groups must have zero cross entries in W. It is structural metadata only and does not generate numeric weights.

naMethodX, naMethodY, naMethodRandom, naMethodR

Missing-value policy for fixed predictors, responses, random-effect variables and residual variables. The default "exclude" removes the observation from the common mask. naMethodX="include" imputes missing covariates; naMethodY="include" imputes missing responses and "include2" retains multivariate rows with at least one observed response. Imputation changes the fitted design; use with care. Supported policies depend on the fitting route.

contrasts

Optional named list of contrasts for factors in fixed, as in lm.

family

A family object or a familym declaration. The default Gaussian identity family fits the linear mixed model. See Details for non-Gaussian families and estimation routes.

vcc

Optional data frame of equality or scaling constraints among covariance parameters (columns parameter, group and optionally scale). Parameter names come from the vcParams table returned with mmes_control(output=list(setup=TRUE)). See vcc.

covPar

Covariance parameters used as starting values, or as known values when mmes_method(task="solve"). Usually a previous mmes fit.

parameters

Named list of parameter() declarations for unknown nonlinear parameters used in fixed.

curves

Named list of nonlinear_curve() components, referenced in fixed with curve_term("name").

start

Starting values on the natural scale for nonlinear parameters (a named vector or a list with parameters); coefficient models use explicitly named coefficients.

method

A mmes_method() object declaring the objective ("reml" or "ml"), integration ("auto", "exact", "laplace"), approximation ("pql" or "likelihood") and task ("fit" or "solve").

control

A mmes_control() object with all numerical settings: solver, equation formulation, convergence, output, execution (memory, threads, GPU, precision) and diagnostics. See the sommer.mmes.options vignette.

familyParameters

Declarations of family nuisance parameters (e.g. negative binomial dispersion, zero-inflation, ordinal thresholds) as fixed or estimated.

Details

Specification layers. A model is built from four independent pieces:

  1. What is modeled: the formulas fixed, random and rcov, plus family.

  2. Which covariance structures: vsm() terms inside random and rcov (see below).

  3. What is estimated: the statistical objective, integration and task, declared with mmes_method().

  4. How it is computed: solver, formulation, convergence, output, threading and diagnostics, declared with mmes_control().

Covariance structures with vsm(). Every structured random or residual term is written as one vsm() call:

G = \sigma^2 (K_1 \otimes K_2 \otimes \cdots \otimes K_m) \otimes A,

where \sigma^2 is the single variance scale owned by vsm(), K_1,\ldots,K_m are dimensionless covariance shapes built by the constructors listed below, and A is an optional known relationship matrix among the levels of the last term. The last argument of vsm() supplies the incidence matrix of the main effect; all previous arguments are covariance factors. There is no limit on the number of factors. A random term not wrapped in vsm() is an identity-covariance effect, so random = ~ Name equals random = ~ vsm(ism(Name)).

Known relationships are given through Gu, the inverse of A, with row and column names equal to the levels of the last term and attr(Gu, "inverse") = TRUE; see vsm for the accepted formats. Matrices can be computed with A.mat and H.mat.

Covariance constructors available:

ism identity
dsm, atm diagonal (heterogeneous variances), diagonal over selected levels
usm unstructured
csm, corgm compound symmetry and correlation structures
ar1m, ar2m, ar3m autoregressive
mam (ma1m, ma2m) moving average
toeplitzm Toeplitz correlation
antem antedependence
fam factor analytic (see the factor analytic vignette)
rrm reduced rank, specific variances fixed at zero. Use fam(..., variance="homogeneous") for a common estimated specific variance
maternm Matern spatial covariance
metricm distance-based exponential, power, gaussian, spherical and circular correlations (isotropic or anisotropic)
sar, car simultaneous and conditional autoregressive spatial models
ownm user-defined or fixed covariance
dsumm direct sum: separate structure per group

Design-matrix utilities such as spl2Dc, enhancer::overlay, enhancer::leg and enhancer::redmm can be combined with vsm(). Variance components can be constrained with vcc, fixm or covm.

Examples of random and rcov specifications:

~ Name + Env:Name compound symmetry (main plus interaction)
~ vsm(dsm(Env), ism(Name)) heterogeneous genetic variance by environment
~ vsm(usm(Env), ism(Name), Gu=Ainv) unstructured GxE with known relationships
~ vsm(fam(Env, k=2), ism(Name), Gu=Ainv) factor analytic GxE
~ vsm(ar1m(Row), ar1m(Col), ism(Plot)) separable spatial AR1
rcov = ~ vsm(dsm(Env), ism(units)) heterogeneous residual variances
rcov = ~ vsm(usm(trait), ism(record)) multi-trait residual with explicit pairing key
rcov = ~ dsumm(vsm(ar1m(Row), ar1m(Col), ism(units)), by=Trial) section-specific spatial residual

Non-Gaussian responses. Non-Gaussian mixed models use penalized quasi-likelihood (PQL) by default: IRLS working responses are fitted with the Gaussian REML machinery. Because the working likelihood is not a GLMM likelihood, AIC, BIC and llik are NA and likelihood-ratio anova() is refused for PQL fits; use deviance and Wald tests (wald_mmes). Use method=mmes_method(objective="ml", integration="laplace") for the native Laplace likelihood. Supported families include binomial (logit, probit, cloglog), Poisson, Gamma, inverse Gaussian, Gaussian with other links, quasi families, nbinom2, zipoisson, ordinal, multinomial, grouped binomials via binm, and mixtures through familym. Fixed-only generalized models require objective="ml".

Nonlinear models. Unknown nonlinear parameters declared in parameters with parameter(), or native curves in curves declared with nonlinear_curve() and used in the formula through curve_term(), require mmes_method(objective="ml"). Fits are post-processed with profile_nonlinear and post_nonlinear.

Known variance components. With method=mmes_method(task="solve") and covPar set to a previous fit, the mixed model equations are solved without estimating variances (genetic evaluation, see the genetic evaluation vignette).

Missing data. All model components are evaluated before a common observation mask is applied, so response, fixed, random and residual designs always use the same rows. The naMethod* arguments set the policy per component and the audit is returned in obsInfo.

Fixed-effect rank. The fixed design must have full column rank. Aliased or all-zero columns are reported before fitting. Request mmes_control(representation=list(fixedRank="drop")) to drop aliased coefficients (returned as zeros and flagged in summary(fit)$betas), or use droplevels() and interaction(..., drop=TRUE) to remove unobserved cells.

Rotation. vsm(..., Gu=Gu, rotation=TRUE) requests Lee–van der Werf rotation for one relationship term. It requires a balanced Gaussian layout and a rotation-invariant residual structure.

Where to learn more. Vignettes (browseVignettes("sommer")):

  • vignette("sommer.models"): model specification.

  • vignette("sommer.covariance.structures"): all covariance structures and vsm() usage.

  • vignette("sommer.gxe"): genotype-by-environment models.

  • vignette("sommer.fa.and.rr.models"): factor analytic and reduced rank models.

  • vignette("sommer.spatial"): spatial models.

  • vignette("sommer.qg"): quantitative genetics.

  • vignette("sommer.genetic.evaluation"): known variance components.

  • vignette("sommer.glmm") and vignette("sommer.glmm.theory"): generalized models.

  • vignette("sommer.mmes.options"): mmes_control() and mmes_method() options.

  • vignette("sommer.vs.lme4"): translating lme4 models.

  • vignette("sommer.theory") and vignette("sommer.machinery"): parameterization and engine.

Value

A fitted object of class mmes; its class and available components depend on the model and fitting route. For linear and PQL fits the main components are:

b, u, bu

fixed effects, random effects and both together.

uList, uPevList

BLUPs and their PEVs per random term, with levels in rows and structure coordinates in columns.

theta, covPar, covParNative

estimated covariance matrices, parameters in reported coordinates, and a table of parameters on the native model scale with standard errors and z-ratios.

theta_se, InfMat

covariance-parameter uncertainty and information matrix; NULL when covarianceInformation="none".

Ci, Ci_11, Cchol

inverse and Cholesky factor of the coefficient matrix.

llik, AIC, BIC, convergence, monitor

likelihood, information criteria, convergence flag and the iteration history.

data, obsInfo, y, Dtable

retained data, the missing-data audit, the response and the factor table used by predict().

partitions, partitionsX

start and end positions of each random and fixed term.

toBoundary, remlOmissions

components forced to the boundary and structurally unestimable components omitted from REML.

engineDiagnostics

path-dependent diagnostics (compression, PCG, profile timings).

control

validated controls used in the fit.

Use the extractors and methods summary(), coef(), fitted(), residuals(), predict(), anova(), vcov(), plot(), randef, covparams_mmes, covparams_mmes_se, postPEV, postVarU, wald_mmes, boot_mmes, r2 and vcDelta.

Author(s)

Giovanny Covarrubias-Pazaran

References

Covarrubias-Pazaran G. Genome assisted prediction of quantitative traits using the R package sommer. PLoS ONE 2016, 11(6): doi:10.1371/journal.pone.0156744

Jensen J, Mantysaari EA, Madsen P, Thompson R. 1997. Residual maximum likelihood estimation of (co)variance components in multivariate mixed linear models using average information. Journal of the Indian Society of Agricultural Statistics 49:215-236.

Gilmour AR, Thompson R, Cullis BR. 1995. Average information REML: an efficient algorithm for variance parameter estimation in linear mixed models. Biometrics 51(4):1440-1450.

Sanderson C, Curtin R. 2025. Armadillo: an efficient framework for numerical linear algebra. arXiv:2502.03000.

See Also

vsm and the constructors ism, dsm, usm, csm, ar1m, fam, rrm, maternm, ownm; mmes_method, mmes_control; summary.mmes, predict.mmes, anova.mmes, wald_mmes; familym, parameter, nonlinear_curve; A.mat, H.mat, stackTraits, vcc

Examples

dat <- data.frame(x=seq_len(12), y=1+seq_len(12)/2+sin(seq_len(12))/10)
fit <- mmes(y~x, data=dat, control=mmes_control(
  diagnostics=list(verbose=FALSE, dateWarning=FALSE)))
coef(fit)


data(DT_example, package="enhancer")
DT <- DT_example

## compound symmetry: genotype main effect + genotype-by-environment
ans1 <- mmes(Yield ~ Env, random = ~ Name + Env:Name,
             rcov = ~ units, data = DT)
summary(ans1)

## known variance components: solve the MME only
sol1 <- mmes(Yield ~ Env, random = ~ Name + Env:Name, rcov = ~ units,
             data = DT, covPar = ans1,
             method = mmes_method(task = "solve"))
head(sol1$uList[[1]])

## heterogeneous genetic and residual variances by environment
DT <- DT[order(DT$Env), ]
ans2 <- mmes(Yield ~ Env,
             random = ~ Name + vsm(dsm(Env), ism(Name)),
             rcov = ~ dsumm(vsm(ism(units)), by = Env), data = DT)
summary(ans2)

## unstructured genotype-by-environment covariance
ans3 <- mmes(Yield ~ Env, random = ~ vsm(usm(Env), ism(Name)),
             rcov = ~ dsumm(vsm(ism(units)), by = Env), data = DT)
summary(ans3)

## binomial GLMM by PQL (default) and Laplace likelihood
set.seed(1)
g <- data.frame(id = factor(rep(1:30, each = 6)), x = rnorm(180))
g$y <- rbinom(180, 1, plogis(0.5*g$x + rep(rnorm(30), each = 6)))
pql <- mmes(y ~ x, random = ~ id, data = g, family = binomial())
lap <- mmes(y ~ x, random = ~ id, data = g, family = binomial(),
            method = mmes_method(objective = "ml", integration = "laplace"))
summary(lap)


sommer documentation built on Nov. 14, 2026, 5:07 p.m.