View source: R/multilevel.cfa.R
| multilevel.cfa | R Documentation |
This function conducts multilevel confirmatory factor analysis to investigate
different types of constructs with distinct construct meanings at the individual
and cluster level (Stapleton et al., 2016) by calling the cfa function
in the R package lavaan. By default, the function specifies and estimates
a measurement model for an individual and configural construct to provides a
table with univariate sample statistics, model fit information, and parameter
estimates. Additionally, variance-covariance coverage of the data, modification
indices, residual correlation matrix, and relative Opdyke distribution percentile
matrix can be requested by specifying the argument print.
multilevel.cfa(data, ..., cluster, model = NULL, rescov = NULL,
model.w = NULL, model.b = NULL, rescov.w = NULL, rescov.b = NULL,
const = c("within", "shared", "config", "shareconf"),
fix.resid = NULL, ident = c("marker", "var", "effect"),
ls.fit = FALSE, estimator = c("ML", "MLR"),
test = c("none", "standard", "yuan.bentler", "yuan.bentler.mplus"),
se = c("none", "standard", "robust.huber.white"),
optim.method = c("nlminb", "em"), missing = c("listwise", "fiml"),
print = c("all", "summary", "coverage", "descript", "fit", "est",
"modind", "resid", "opdyke"),
mod.minval = 6.63, resid.minval = 0.1, opdyke.prec = 1,
opdyke.minmax = c(0.40, 0.60), color = "b.red",
style = c("regular", "bold", "italic"), digits = 3, p.digits = 3,
as.na = NULL, write = NULL, append = TRUE, check = TRUE, output = TRUE)
data |
a data frame. If |
... |
an expression indicating the variable names in |
cluster |
either a character string indicating the variable name of
the cluster variable in |
model |
a character vector for specifying the same factor structure
with one factor at the Within and Between Level, or a list
of character vectors for specifying the same measurement
model with more than one factor at the Within and Between
Level, e.g., |
rescov |
a character vector or a list of character vectors for specifying
residual covariances at the Within level, e.g. |
model.w |
a character vector specifying a measurement model with one factor at the Within level, or a list of character vectors for specifying a measurement model with more than one factor at the Within level. |
model.b |
a character vector specifying a measurement model with one factor at the Between level, or a list of character vectors for specifying a measurement model with more than one factor at the Between level. |
rescov.w |
a character vector or a list of character vectors for specifying residual covariances at the Within level. |
rescov.b |
a character vector or a list of character vectors for specifying residual covariances at the Between level. |
const |
a character string indicating the type of construct(s), i.e.,
|
fix.resid |
a character vector for specifying residual variances to be
fixed at 0 at the Between level, e.g., |
ident |
a character string indicating the method used for identifying
and scaling latent variables, i.e., |
ls.fit |
logical: if |
estimator |
a character string indicating the estimator to be used:
|
test |
a character string indicating the test statistics, i.e.,
|
se |
a character string indicating the standard errors, i.e.,
|
optim.method |
a character string indicating the optimizer, i.e., |
missing |
a character string indicating how to deal with missing data,
i.e., |
print |
a character string or character vector indicating which
results to show on the console, i.e. |
mod.minval |
numeric value to filter modification indices and only
show modifications with a modification index value equal
or higher than this minimum value. By default, modification
indices equal or higher 6.63 are printed. Note that a
modification index value of 6.63 is equivalent to a
significance level of |
resid.minval |
numeric value indicating the minimum absolute residual
correlation coefficients and standardized means to
highlight. By default, absolute residual
correlation coefficients and standardized means equal
or higher 0.1 are highlighted. Note that highlighting
can be disabled by setting the minimum value to 1 or
by setting the argument |
opdyke.prec |
a numeric value indicating the precision of the probability
density function calculations of the Opdyke distribution.
The default is |
opdyke.minmax |
a numeric vector with two elements indicating the
minimum and maximum percentile of the Opdyke distribution
that is considered to be acceptably close to the
observed correlation represented by the Opdyke distribution
median. Predicted correlation outside the range will be
color highlighted in line with to the argument |
color |
a character string indicating the text color for
highlighting absolute residual correlation coefficients
and standardized means equal or higher |
style |
a character vector indicating the font style for
highlighting absolute residual correlation coefficients
and standardized means equal or higher |
digits |
an integer value indicating the number of decimal places
to be used for displaying results. Note that loglikelihood,
information criteria and chi-square test statistic is
printed with |
p.digits |
an integer value indicating the number of decimal places to be used for displaying the p-value. |
as.na |
a numeric vector indicating user-defined missing values,
i.e. these values are converted to |
write |
a character string naming a file for writing the output into
either a text file with file extension |
append |
logical: if |
check |
logical: if |
output |
logical: if |
Stapleton et al. (2016) identified different types of constructs when data are collected from individuals nested within clusters characterized by different interpretations of construct meaning. At the individual level, the hypothesized construct is expected to be relevant to individual item responses, i.e., scores on the measure reflect individual variability. At the cluster level, there are two types of constructs measured using item responses from individuals: shared construct, a characteristic of the cluster itself and configural construct, which reflects a construct at the individual level. Marsh et al. (2012) introduced the terms climate construct and contextual construct, while Lüdtke et al., (2011) used the term reflective construct and formative construct to differentiate between shared and configural constructs.
Five types of confirmatory factor analysis (CFA) models were proposed for construct validation with nested data, depending on whether the construct of interest exists solely at the individual level or is also relevant at the cluster level (Stapleton et al., 2016).
Model 1 is
a single-level CFA model whose parameter estimates provide an aggregate of
the relationship between item responses and the individual construct. A design-based
estimation approach is used to adjust the standard error estimates, chi-square test
statistic, and fit indices to account for the dependency of item responses
among individuals within clusters. Note that this model is estimated using
the item.cfa with specifying the cluster argument.
This model is appropriate when a positive ICC(1) is spurious, resulting from a selection process. For example, lactose intolerance is an entirely individual characteristic, unaffected by environment. However, item responses measuring lactose intolerance might exhibit a non-negligible ICC(1) due to the selection of individuals across different clusters, rather than reflecting a true cluster-level construct. That is, lactose intolerance is a completely individual measure, not able to be influenced by environmental effect.
The model is designed for application across a broad population that may or may not be nested within clusters in a single-level model. However, if the measure aims to compare the relative positions of individuals within a cluster, a within-cluster approach is more suitable.
Model 2 is a two-level CFA model
specified by const = "within" that includes a within-cluster construct
representing an individual's standing within the cluster. The model does not assume
the existence of a cluster-level construct but allows for cluster-level variability
in each measure through a saturated variance-covariance model.
The model is designed for application within clusters to compare individuals and their relative positions within a specific cluster, while between-cluster constructs are not meaningful or not of interest.
Model 3 is a two-level
CFA model specified by const = "shared" that includes a between-cluster
construct representing a cluster-level characteristic that is measured using
individuals within a cluster as the information source. The model does not assume
the existence of an individual-level construct but allows for individual-level
variability in each measure through a saturated variance-covariance model.
For a shared construct, individual item responses should be interchangeable, reflecting perfect interrater reliability and complete within-cluster agreement. However, within-cluster variation can exist for shared constructs, as these constructs stem from the experiences, attitudes, perceptions, values, or cognitions of individual cluster members. Given evidence of adequate agreement within clusters, the aggregate value of the measure can be assigned to the cluster, i.e., the shared perceptions of individuals in a cluster are interpreted as a proxy for the cluster-level characteristic (Jak et al., 2023). From this perspective, the greater the agreement within the cluster, the more reliably the shared construct is measured, although perfect agreement is not required. A reasonable step in the shared construct modeling process is to examine the ICC(2) values for each item measuring the shared construct. Klein et al. (2000) suggested that ICC(2) values of at least 0.7 indicate acceptable reliability of the measured shared construct, values between 0.5 and 0.7 indicate marginal reliability, and values below 0.5 are considered poor.
Note that the wording of questionnaire items plays a crucial role in the context of multilevel measurement. For example, the statement "The instructor presents materials in ways that keep it interesting." would likely to elicit responses reflecting the instructor's qualities, whereas the statement "I find the class meeting interest." would likely to reflect both a characteristic of the cluster and the individual's intrinsic interest in the class topic. Accordingly, an argument should be made regarding how the item wording targets cluster characteristics.
Model 4 is a two-level
CFA model specified by const = "config" that includes a within-cluster
construct representing an individual's standing within the cluster and a
between-cluster construct representing an aggregate of the measurements of
individuals within the cluster. The configural cluster construct model requires
cross-level measurement invariance of factor loadings to interpret the individual-
and cluster-level common factors as reflecting the individual- and cluster-level
components of the same construct (Jak et al., 2023).
For a configural construct, individual item responses are not interchangeable, and it is not expected that individuals within a cluster respond similarly to the items. Marsh et al. (2012) noted that if item responses have an ICC(1) of zero, there is little justification for continuing to examine the configural cluster-level construct. However, Stapleton et al. (2016) argued that it may still be of interest to investigate the variability in individual hypothetical latent scores across clusters, i.e., the dispersion of the construct may differ across clusters and represent an important cluster characteristic.
Model 5 is
a two-level CFA model specified by const = "shareconf" that includes a
within-cluster construct representing an individual's standing within the cluster
and two between-cluster constructs: a configural construct representing the
individual-level construct, and a shared construct modeling additional covariation
among cluster-average item responses, representing rater effects. The factor loadings
of the configural construct are constrained to be equal across levels, i.e., metric
cross-level measurement invariance.
For example, suppose that teachers have provided ratings of their students' motivation using multiple items for each child. It is possible that some teachers rate more positively compared to others, resulting in two sources of covariation at the cluster level: variation due to the fact that students in some classes are truly more motivated on average than those in other classes, and variation due to rater effects, because some teachers tend to rate their students more positively or more negatively on average.
Note that what may appear to be an additional shared construct could instead reflect a spurious contextual effect. Specifically, when constraining factor loadings across levels results in a relatively poor fit compared to a model with unconstrained loadings in a configural model, this suggests that the apparent additional shared construct is actually due to measurement noninvariance at the within-cluster level.
Returns an object of class misty.object, which is a list with following
entries:
call |
function call |
type |
type of analysis |
data |
data frame used for the current analysis |
args |
specification of function arguments |
model |
specified model |
model.fit |
fitted lavaan object ( |
check |
results of the convergence and model identification check |
result |
list with result tables, i.e., |
The function uses the functions cfa, lavInspect, lavTech,
modindices, parameterEstimates, and standardizedsolution
provided in the R package lavaan by Yves Rosseel (2012). The relative Opdyke
distribution percentile matrix is based on the functions opdyke and
opdyke.percentiles provided in the R package opddyke by Dan McNeish (2026).
Takuya Yanagida takuya.yanagida@univie.ac.at
Jak, S., Jorgensen, T. D., ten Hove, D., & Nevicka, B. (2023). Modeling cluster-level constructs measured by individual responses: Configuring a shared approach. Advances in Methods and Practices in Psychological Science, 6(3), Article 25152459231182319. https://doi.org/10.1177/25152459231182319
Klein, K. J., & Kozlowski, S. W. (2000). Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions. Jossey-Bass.
Lüdtke, O., Marsh, H. W., Robitzsch, A., & Trautwein, U. (2011). A 2x2 taxonomy of multilevel latent contextual models: Accuracy-bias trade-offs in full and partial error correction models. Psychological Methods, 16, 444-467. https://doi.org/10.1037/a0024376
Marsh, H. W., Ludtke, O., Nagengast, B., Trautwein, U., Morin, A. J. S., Abduljabbar, A. S., & Koller, O. (2012). Classroom climate and contextual effects: Conceptual and methodological issues in the evaluation of group-level effects. Educational Psychologist, 47, 106-124. https://doi.org/10.1080/00461520.2012.670488
McNeish, D. (2025). Less Heuristic Approximate Local Fit Evaluation in Structural Equation Models. Structural Equation Modeling: A Multidisciplinary Journal, 32(4), 590-605. https://doi.org/10.1080/10705511.2025.2473342
McNeish D (2026). opdyke: Opdyke Percentiles for Approximate Local Fit in Structural Equation Models. R package version 1.0.0. Retrieved from https://github.com/dmcneish18/opdyke
Rosseel, Y. (2012). lavaan: An R Package for Structural Equation Modeling. Journal of Statistical Software, 48, 1-36. https://doi.org/10.18637/jss.v048.i02
Stapleton, L. M., Yang, J. S., & Hancock, G. R. (2016). Construct meaning in multilevel settings. Journal of Educational and Behavioral Statistics, 41, 481-520. http://dx.doi.org/10.3102/1076998616646200
item.cfa, multilevel.fit, multilevel.invar,
multilevel.omega, multilevel.cor, multilevel.descript
## Not run:
# Load data set "Demo.twolevel" in the lavaan package
data("Demo.twolevel", package = "lavaan")
#————————————————————————————————————————————————————————————————————————————
# Model Specification Using 'data' for a One-Factor Model
#
# Same factor structure with one factor at the Within and Between Level
#——————————————————————————————————————
## Cluster Variable Specification
# Example 1a: Specification using the argument '...'
multilevel.cfa(Demo.twolevel, y1:y4, cluster = "cluster")
# Example 1b: Alternative specification with cluster variable 'cluster' in 'data'
multilevel.cfa(Demo.twolevel[, c("y1", "y2", "y3", "y4", "cluster")], cluster = "cluster")
# Example 1c: Alternative specification with cluster variable 'cluster' not in 'data'
multilevel.cfa(Demo.twolevel[, c("y1", "y2", "y3", "y4")], cluster = Demo.twolevel$cluster)
#——————————————————————————————————————
## Type of Construct
# Example 2a: Within-cluster constructs
multilevel.cfa(Demo.twolevel, y1:y4, cluster = "cluster", const = "within")
# Example 2b: Shared cluster-level construct
multilevel.cfa(Demo.twolevel, y1:y4, cluster = "cluster", const = "shared")
# Example 2c: Individual and configural cluster construct (default)
multilevel.cfa(Demo.twolevel, y1:y4, cluster = "cluster", const = "config")
# Example 2d: Simultaneous shared and configural cluster construct
multilevel.cfa(Demo.twolevel, y1:y4, cluster = "cluster", const = "shareconf")
#——————————————————————————————————————
## Residual Covariances at the Within Level
# Example 3a: Residual covariance between 'y1' and 'y3'
multilevel.cfa(Demo.twolevel, y1:y4, cluster = "cluster", rescov = c("y1", "y3"))
# Example 3b: Residual covariance between 'y1' and 'y3', and 'y2' and 'y4'
multilevel.cfa(Demo.twolevel, y1:y4, cluster = "cluster",
rescov = list(c("y1", "y3"), c("y2", "y4")))
#——————————————————————————————————————
## Residual Variances at the Between Level fixed at 0
# Example 4a: All residual variances fixed at 0
# i.e., strong factorial invariance across clusters
multilevel.cfa(Demo.twolevel, y1:y4, cluster = "cluster", fix.resid = "all")
# Example 4b: Residual variances of 'y1', 'y2', and 'y4' fixed at 0
# i.e., partial strong factorial invariance across clusters
multilevel.cfa(Demo.twolevel, y1:y4, cluster = "cluster", fix.resid = c("y1", "y2"))
#——————————————————————————————————————
## Arguments 'print', 'mod.minval', 'resid.minval', and 'opdyke.minmax'
# Example 5a: Request all results
item.cfa(Demo.twolevel, y1:y6, cluster = "cluster", print = "all")
# Example 5b: Request modification indices with value equal or higher than 2
item.cfa(Demo.twolevel, y1:y6, cluster = "cluster", print = "modind", mod.minval = 2)
# Example 5c: Highlight absolute residual correlation equal or higher than 0.05
item.cfa(Demo.twolevel, y1:y6, cluster = "cluster", print = "resid", resid.minval = 0.05,
color = "b.blue")
# Example 5d: Highlight Opdyke distribution percentiles outside 0.45 and 0.55
item.cfa(Demo.twolevel, y1:y6, cluster = "cluster", print = "opdyke", opdyke.minmax = c(0.45, 0.55),
color = "black", style = "bold")
#————————————————————————————————————————————————————————————————————————————
# Model Specification using 'model' for One or Multiple-Factor Model
#
# Same factor structure at the Within and Between Level
# Example 6a: One-factor model
multilevel.cfa(Demo.twolevel, cluster = "cluster", model = c("y1", "y2", "y3", "y4"))
# Example 6b: Two-factor model
multilevel.cfa(Demo.twolevel, cluster = "cluster",
model = list(c("y1", "y2", "y3"), c("y4", "y5", "y6")))
# Example 6c: Two-factor model with user-specified labels for the factors
multilevel.cfa(Demo.twolevel, cluster = "cluster",
model = list(factor1 = c("y1", "y2", "y3"), factor2 = c("y4", "y5", "y6")))
#——————————————————————————————————————
# Type of Construct
# Example 7a: Within-cluster constructs
multilevel.cfa(Demo.twolevel, cluster = "cluster", const = "within",
model = list(c("y1", "y2", "y3"), c("y4", "y5", "y6")))
# Example 7b: Shared cluster-level construct
multilevel.cfa(Demo.twolevel, cluster = "cluster", const = "shared",
model = list(c("y1", "y2", "y3"), c("y4", "y5", "y6")))
# Example 7c: Individual and configural cluster construct (default)
multilevel.cfa(Demo.twolevel, cluster = "cluster", const = "config",
model = list(c("y1", "y2", "y3"), c("y4", "y5", "y6")))
# Example 7d: Simultaneous shared and configural cluster construct
multilevel.cfa(Demo.twolevel, cluster = "cluster", const = "shareconf",
model = list(c("y1", "y2", "y3"), c("y4", "y5", "y6")))
#——————————————————————————————————————
## Residual Covariances at the Within Level
# Example 8a: Residual covariance between 'y1' and 'y4' at the Within level
multilevel.cfa(Demo.twolevel, cluster = "cluster",
model = list(c("y1", "y2", "y3"), c("y4", "y5", "y6")),
rescov = c("y1", "y4"))
# Example 8b: Fix all residual variances at 0
# i.e., strong factorial invariance across clusters
multilevel.cfa(Demo.twolevel, cluster = "cluster",
model = list(c("y1", "y2", "y3"), c("y4", "y5", "y6")),
fix.resid = "all")
#————————————————————————————————————————————————————————————————————————————
# Model Specification Using 'model.w' and 'model.b' for One or Multiple_Factor Model
#
# Different factor structure at the Within and Between Level
# Example 9a: Simultaneous and level-specific fit indices
multilevel.cfa(Demo.twolevel, cluster = "cluster",
model.w = c("y1", "y2", "y3", "y4", "y5", "y6"),
model.b = c("y1", "y2", "y3", "y4", "y5", "y6"), ls.fit = TRUE)
# Example 9b: Two-factor model at the Within level and one-factor model at the Between level
multilevel.cfa(Demo.twolevel, cluster = "cluster",
model.w = list(c("y1", "y2", "y3"), c("y4", "y5", "y6")),
model.b = c("y1", "y2", "y3", "y4", "y5", "y6"))
# Example 9c: Residual covariance between 'y1' and 'y4' at the Within level
# Residual covariance between 'y5' and 'y6' at the Between level
multilevel.cfa(Demo.twolevel, cluster = "cluster",
model.w = list(c("y1", "y2", "y3"), c("y4", "y5", "y6")),
model.b = c("y1", "y2", "y3", "y4", "y5", "y6"),
rescov.w = c("y1", "y4"),
rescov.b = c("y5", "y6"))
#————————————————————————————————————————————————————————————————————————————
# lavaan Model and Summary of the Estimated Model
# Example 10
mod <- multilevel.cfa(Demo.twolevel, y1:y4, cluster = "cluster", output = FALSE)
# lavaan model syntax
cat(mod$model)
# Fitted lavaan object
lavaan::summary(mod$model.fit, standardized = TRUE, fit.measures = TRUE)
#————————————————————————————————————————————————————————————————————————————
# Write Results
# Example 11a: Write results into a text file
multilevel.cfa(Demo.twolevel, y1:y4, cluster = "cluster", print = "all",
write = "Multilevel_CFA.txt", output = FALSE)
# Example 11b: Write results into an Excel file
multilevel.cfa(Demo.twolevel, y1:y4, cluster = "cluster", print = "all",
write = "Multilevel_CFA.xlsx", output = FALSE)
## End(Not run)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.