View source: R/FUN_mme_modules.R
| mmes | R Documentation |
Fits a generalized linear mixed model using model formulas, covariance
structures and observation data. Special covariance structures can be
specified with vsm(). Numerical fitting choices are supplied
through the strictly validated control object; see
mmes_control and the sommer.mmes.options vignette.
mmes(fixed, random, rcov, data, W, weights=NULL,
naMethodX="exclude", naMethodY="exclude",
naMethodRandom="exclude", naMethodR="exclude",
contrasts=NULL, family=stats::gaussian(), vcc=NULL,
covPar=NULL, control=mmes_control(), .internal=NULL, ...)
fixed |
A formula specifying the response variable(s) and fixed effects, i.e: response ~ covariate By default, the fixed-effect design must have full column rank after observation
filtering. Before fitting, Apply |
random |
A formula specifying the random effects, e.g.,
random = ~ genotype + year. A simple random term that is not wrapped
in The
where Within
fits heterogeneous variances across
specifies an arbitrary Kronecker product of covariance structures before the
relationship matrix for Covariance constructors currently available include:
Design-matrix utilities such as See A single relationship term may request Leeāvan der Werf rotation with
|
rcov |
A formula specifying the residual covariance structure. The default is rcov = ~ units, which fits a homogeneous residual variance. Structured residual covariance is specified with the same
The final term is normally
fits heterogeneous residual variances across environments, while
fits a separable row-by-column AR1 residual covariance, with one correlation per direction shared by every other Kronecker factor. For section-specific variances and correlations, use
(see Residual covariance factors must identify exactly one covariance-product
coordinate for each observation. Residual pairing key (multi-trait models). For a long-format
multi-trait model the residual must know which records belong to the same
experimental unit. With
where As with random effects, See |
data |
Optional data frame containing variables used by the model.
Model expressions are evaluated using normal R scoping rules: variables are first
sought in Observation-level variables used by the response, fixed effects, random effects,
and residual covariance specification must nevertheless have compatible lengths.
Internally, |
W |
Weights matrix (e.g., when covariance among plots exist). Internally W is squared and inverted as Wsi = solve(chol(W)), then the residual matrix is calculated as R = Wsi*O*Wsi.t(), where * is the matrix product, and O is the original residual matrix. Weights can be combined with any residual structure, including Kronecker (multi-trait) residuals. |
weights |
Optional one-sided formula describing independent row blocks
of the supplied |
control |
A |
naMethodX |
Missing-data policy for variables entering the fixed-effect
design matrix. The default, |
naMethodY |
Missing-data policy for the response variable(s). The default,
|
naMethodRandom |
Missing-data policy for observation-level variables used to
construct random-effect terms. The default is |
naMethodR |
Missing-data policy for observation-level variables used to define
the residual covariance structure. The default is |
contrasts |
an optional list. See the contrasts.arg of model.matrix.default. |
vcc |
Optional data frame of equality/scaling constraints between
covariance parameters (columns |
family |
A Long-format multi-trait data with a different family per trait are fitted by
supplying |
covPar |
Known covariance parameters for the solve task selected by
|
.internal |
Reserved for package-internal recursive fitting state. Users should not set this argument. |
... |
Deprecated legacy arguments. The supported arguments are
|
Structurally unestimable REML components
For Gaussian identity-link REML fits, mmes() checks whether an
independently scaled random component contributes anything outside the
fixed-effect design space, using the observations retained by the common
missing-data filter. A component whose entire random design lies in that
space is invisible to the restricted likelihood. Such components are
removed before optimization, with a warning, rather than assigned an
arbitrary variance by a regularized information-matrix solve.
The conservative check handles a whole random term with a free variance
scale and fixed covariance shape, or individual levels of a single
dsm() or atm() shaping factor when its scale and all variance
ratios are free. For example, with fixed locale means, a locale-specific
independent block variance is omitted when that locale has only one
observed block. A shared block variance is retained if other locales
provide information about it. Correlated, multiply shaped, constrained or
partly fixed structures are not automatically reduced at individual levels.
The check does not diagnose general aliasing between covariance terms or
weak identification near a variance boundary.
Automatic omission is not applied to ML, PQL, known-covariance solve tasks,
vcc constraints, observation rotation or factor-score augmentation.
For supported fits, omitted components retain their original positions in
random-effect and covariance outputs, with zero covariance and zero random
effects. These zeros denote an omission convention, not variances estimated
to be zero. Their native-scale standard errors and Z-ratios are NA;
summary() and covparams_mmes() label them
"unestimable (omitted)". Only retained free parameters contribute to
optimization and information criteria. Omission preserves the restricted
likelihood information about retained parameters; it does not establish
zero between-block variation in the population. Inference on fixed means
and predictions is conditional on the omission convention.
Fixed factors with many levels
In Henderson fits the fixed-effect design X is stored as a sparse
matrix. The coefficient matrix C of the mixed model equations is
factorized with sparse methods at every REML iteration, whatever the
control$output$inverse setting. The fill-reducing ordering of the
sparse factorization eliminates the fixed-effect equations within the same
factor as the random ones, which is equivalent to absorbing them. A fixed
factor with thousands of levels, such as contemporary groups, does not
create a dense fixed-effect block during estimation. Variance components,
BLUEs and BLUPs are identical to those of an explicit absorption of the
fixed effects.
control$output$inverse only controls work done after convergence:
0 (default)No inverse of C is formed. Fixed-effect
standard errors in summary() are NA.
1Elements of the inverse are computed only within the
sparsity pattern of the factor (Takahashi selected inverse), giving
random-effect PEVs. The fixed part stays sparse. Fixed-effect standard
errors in summary() remain NA.
2The complete inverse of C, including the fixed
block, is formed as a dense matrix. This needs about
8(p+q)^2 bytes for p fixed and q random effects
(around 1.4 GB for 13,000 effects) and is the mode that reports
fixed-effect standard errors in summary().
With many fixed levels, keep inverse=0 or 1. When standard
errors of particular fixed-effect combinations are needed, use
predict.mmes, which computes exact standard errors from the
sparse C without forming Ci. postPEV can add
PEVs or the full inverse after fitting.
When even the sparse REML fit is too slow, a practical alternative is to
estimate the variance components with the large factor fitted as random,
optionally on a well-connected subset of the data. Then refit with the
factor as fixed using control=mmes_control(task=list(mode="solve"))
and covPar to obtain BLUEs and BLUPs with a single factorization.
Simply omitting the factor during REML is not equivalent: its variation is
then attributed to the residual and to confounded random terms.
Numerical controls
The opt-in control$representation$equations="conditional" path fits Gaussian
FA models using sparse latent equations with exact REML AI updates by default,
or conditional-moment EM through control$update$method="em". The earlier
control$update$method="conditional-fa" selector retains its EM behavior.
Both optimizers support incomplete and
unbalanced observations. It accepts one FA relationship term and diagonal
or FA residuals. Matching fam(shared="group") tags request common
trait loadings across genetic and residual sources. The default AI path is
unchanged. See mmes_control and the conditional FA section of
vignette("sommer.mmes.options") for eligibility and output restrictions.
Conditional fits retain factorized covariance descriptors, not dense
theta matrices, public C, coefficient uncertainty, or
covariance-parameter uncertainty. Use covmatrix_mmes(..., se=FALSE)
to explicitly reconstruct a bounded-size covariance.
mmes() keeps model specification separate from numerical policy.
The control argument defaults to mmes_control(), whose
modules have strict schemas: unknown field names are errors. The default
configuration retains the former standard bundle: Henderson equations, REML,
exact AI curvature, EM recovery, no extrapolation, 30 inner iterations,
full covariance information, coefficient-matrix retention, and inherited
solver/resource defaults. The mmes_control help gives every field,
default, module owner and validation rule.
Ownership and eligibility
control$task selects fitting or known-covariance solving,
REML or ML, and outer PQL controls.
control$formulation selects Henderson or direct
observation-space equations.
control$representation describes
equation/covariance representation; control$solve owns linear solver,
preconditioning, reuse and elimination requests.
control$logdet owns determinant methods;
control$score owns score traces and covariance checks;
control$curvature owns exact or hybrid AI and reduced-direction work.
control$update owns EM/PXEM recovery, blending
and common-scale profiling. control$proposal owns step scaling and
Aitken extrapolation. Aitken and hybrid AI are mutually exclusive.
control$convergence owns inner
iteration, tolerances and stochastic accuracy; control$output owns
coefficient/covariance uncertainty and setup output.
control$execution owns memory,
hardware, threading, precision and seeds. control$diagnostics owns
verbosity, package-age warnings and profiling.
These modules do not form a Cartesian product of interchangeable algorithms.
Model-dependent admission remains authoritative; a requested mode may be
ineligible. Consult mmes_control for prerequisites, exclusions
and current behavior. No new method combinations are implied by the modular API.
The internal .mmes_capabilities() catalog is descriptive and does
not resolve eligibility or predict execution.
If all parameters are correctly indicated the program will return a list with the following information:
data |
the dataset used in model fitting after application of the common observation mask. |
obsInfo |
an observation-level audit table describing the common missing-data filter. It includes the original row index, validity indicators for response, fixed, random, and residual model components, the final inclusion indicator, and the reason for exclusion when applicable. |
remlOmissions |
when present, a data frame identifying structurally
unestimable random covariance components omitted from REML, with columns
|
Dtable |
the table to be used for the predict function to help the program recognize the factors available. |
llik |
the vector of log-likelihoods across iterations |
b |
the vector of fixed effects on the original response scale. |
u |
the vector of random effects on the original response scale, consistent
with |
bu |
the vector of fixed and random effects together on the original response
scale, formed by concatenating |
Ci |
the inverse of the coefficient matrix. |
Ci_11 |
the inverse of the coefficient matrix pertaining to the fixed effects. |
theta |
a list of estimated variance covariance matrices. Each element of the list corresponds to the different random and residual components |
covPar |
the fitted covariance parameters in their descriptor-defined reported coordinates. |
control |
the validated modular numerical controls used for the fit.
See |
pcgRecycleState |
present only when |
covarianceInformation |
|
retainCoefficientMatrix |
FALSE when an opt-in C-free saddle-point fit
omitted the final compatibility matrix. In that mode |
engineDiagnostics |
path-dependent numerical diagnostics.
|
pcgMatrixFree |
logical indicating whether the matrix-free Kronecker PCG path was used during optimization. |
residualCovarianceCompact |
logical indicating that a complete, unweighted,
non-diagonal Kronecker residual covariance was evaluated directly from its
factors without materializing the full covariance matrix. In this case the
residual element of |
covParNative |
a data frame containing all fixed and estimated covariance
parameters in the native model-facing scale defined by each CovarianceFactor.
For example, |
theta_se |
covariance-parameter uncertainty matrix; NULL when
|
InfMat |
information matrix; NULL when
|
monitor |
The values of the variance-covariance components across iterations during the REML estimation. |
AIC |
Akaike information criterion |
BIC |
Bayesian information criterion |
convergence |
a TRUE/FALSE statement indicating if the model converged. |
partitions |
a list where each element contains a matrix indicating where each random effect starts and ends. |
partitionsX |
a list where each element contains a matrix indicating where each fixed effect starts and ends. |
percDelta |
the matrix of percentage change in deltas (see |
normMonitor |
the matrix of the three norms calculated (see |
toBoundary |
the matrix of variance components that were forced to the boundary across iterations. |
Cchol |
the Cholesky decomposition of the coefficient matrix. |
y |
the response vector. |
W |
the column binded matrix W = [X Z] |
uList |
a list containing the BLUPs in data frame format where rows are levels of the random effects and column the different factors at which the random effect is fitted. This is specially useful for diagonal and unstructured models. |
uPevList |
a list containing the BLUPs in data frame format where rows are levels of the random effects and column the different factors at which the random effect is fitted. This is specially useful for diagonal and unstructured models. |
args |
the fixed, random and residual formulas from the mmes model. |
constraints |
The vector of constraints. |
Coded by Giovanny Covarrubias-Pazaran with contributions of Christelle Fernandez Camacho, Johan Aparicio-Arce, and Claudio Flavio 5.1 jaja (laughing in Spanish).
Covarrubias-Pazaran G. Genome assisted prediction of quantitative traits using the R package sommer. PLoS ONE 2016, 11(6): doi:10.1371/journal.pone.0156744
Jensen, J., Mantysaari, E. A., Madsen, P., and Thompson, R. (1997). Residual maximum likelihood estimation of (co) variance components in multivariate mixed linear models using average information. Journal of the Indian Society of Agricultural Statistics, 49, 215-236.
Sanderson, C., & Curtin, R. (2025). Armadillo: An Efficient Framework for Numerical Linear Algebra. arXiv preprint arXiv:2502.03000.
Gilmour et al. 1995. Average Information REML: An efficient algorithm for variance parameter estimation in linear mixed models. Biometrics 51(4):1440-1450.
####=========================================####
#### For CRAN time limitations most lines in the
#### examples are silenced with one '#' mark,
#### remove them and run the examples
####=========================================####
data(DT_example, package="enhancer")
DT <- DT_example
head(DT)
####=========================================####
#### Univariate homogeneous variance models ####
####=========================================####
## Compound simmetry (CS) model
ans1 <- mmes(Yield~Env,
random= ~ Name + Env:Name,
rcov= ~ units,
data=DT)
summary(ans1)
## Known variances: solve the MME only (e.g. genetic evaluation)
sol1 <- mmes(Yield~Env,
random= ~ Name + Env:Name,
rcov= ~ units,
data=DT, control=mmes_control(task=list(mode="solve")),
covPar=ans1)
head(sol1$uList[[1]])
####===========================================####
#### Univariate heterogeneous variance models ####
####===========================================####
DT=DT[with(DT, order(Env)), ]
## Compound simmetry (CS) + Diagonal (DIAG) model
ans2 <- mmes(Yield~Env,
random= ~Name + vsm(dsm(Env),ism(Name)),
rcov= ~ dsumm(vsm(ism(units)), by=Env),
data=DT)
summary(ans2)
####===========================================####
#### Univariate unstructured variance models ####
####===========================================####
ans3 <- mmes(Yield~Env,
random=~ vsm(usm(Env),ism(Name)),
rcov= ~ dsumm(vsm(ism(units)), by=Env),
data=DT)
summary(ans3)
## simulated MET
nInds=300
nMarks=1000
nEnvs=3
#random population of nInds lines with nMarks markers
M <- matrix(rep(0,nInds*nMarks),nInds,nMarks)
for (i in 1:nInds) {
M[i,] <- ifelse(runif(nMarks)<0.5,-1,1)
}
# compute the relationship matrix
MMT <- tcrossprod(M)
m <- sum(diag(MMT))/nrow(MMT)
MMT <- MMT/m
MMT <- MMT + diag(1e-05, ncol(MMT), ncol(MMT))
# get phenotypes
rg <- enhancer::simGECorMat(nEnvs, 1, mu=0.2, v=0.2)
rg <- Matrix::nearPD(rg)$mat
rg # desired genetic correlation
u <- MASS::mvrnorm(n = nMarks, rep(0, nEnvs), Sigma=rg) # marker effects
g <- crossprod(t(M),u) # genetic values
h2 <- 0.2 #heritability
envEffect <- rnorm(nEnvs, mean = abs(rnorm(1))) # environment means
e <- MASS::mvrnorm(n = nInds, envEffect, Sigma=(1-h2)/h2*diag(diag(var(g)))) # errors
p = g + e # phenotypes
y <- as.vector(p) # phenotypes as vector
env <- sort(rep(paste0("e",1:nEnvs), nrow(g))) # environment labels
ind <- rep(paste0("i",1:nrow(g)), nEnvs ) # individuals labels
colnames(MMT) <- rownames(MMT) <- paste0("i",1:nrow(g)) # assign dimnames
MMTi <- solve(MMT + diag(1e-4,ncol(MMT),ncol(MMT)))
MMTi[lower.tri(MMTi)] <- t(MMTi)[lower.tri(MMTi)] # fill the lower triangular
MMTi <- as(as(as( MMTi, "dMatrix"), "generalMatrix"), "CsparseMatrix")
attr(MMTi, 'inverse')=TRUE
DT <- data.frame(ind=ind, env=env, y=y)
# fit the models
mixm2 <- mmes(y ~ env,
random=~ vsm(dsm(env), ism(ind), Gu=MMTi),
data=DT
)
summary(mixm2)$varcomp
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.