compRelSEM: Composite Reliability using SEM
In semTools: Useful Tools for Structural Equation Modeling
compRelSEM R Documentation
Composite Reliability using SEM
Description
Calculate composite reliability from estimated factor-model parameters
Usage
compRelSEM(object, obs.var = TRUE, tau.eq = FALSE, ord.scale = TRUE,
config = character(0), shared = character(0), higher = character(0),
return.total = FALSE, dropSingle = TRUE, omit.factors = character(0),
omit.indicators = character(0), omit.imps = c("no.conv", "no.se"),
return.df = TRUE)
Arguments
object
A lavaan::lavaan or lavaan.mi::lavaan.mi object,
expected to contain only exogenous common factors (i.e., a CFA model).
obs.var
logical indicating whether to compute reliability
using observed variances in the denominator. Setting FALSE triggers
using model-implied variances in the denominator.
tau.eq
logical indicating whether to assume (essential)
tau-equivalence, yielding a coefficient analogous to \alpha.
Setting FALSE yields an \omega-type coefficient.
ord.scale
logical indicating whether to apply Green and Yang's
(2009, formula 21) correction, so that reliability is calculated for the
actual ordinal response scale (ignored for factors with continuous
indicators). Setting FALSE yields coefficients that are
only applicable to the continuous latent-response scale.
config
character vector naming any configural constructs in
a multilevel CFA. For these constructs (and optional total composite),
Lai's (2021) coefficients \omega^\textrm{W} and \omega^\textrm{2L}
are returned (or corresponding \alpha coefficients when
tau.eq=TRUE), rather than Geldhof et al.'s (2014) coefficients for
hypothetical composites of latent components (although the same formula
is used for \omega^\textrm{W} in either case). Note that the same name
must be used for the factor component represented at each level of the
model.
shared
character vector naming any shared constructs in
a multilevel CFA. For these constructs (and optional total composite),
Lai's (2021) coefficient \omega^\textrm{B} or \alpha^\textrm{B} is
returned, rather than Geldhof et al.'s (2014) between-level coefficient
for hypothetical composites of latent cluster means. Lai's (2021)
coefficient quantifies reliability relative to error associated with both
indicators (measurement error) and subjects (sampling error), like a
generalizability coefficient. Given that subjects can be considered as
raters of their cluster's shared construct, an interrater reliability
(IRR) coefficient is also returned, quantifying reliability relative to
rater/sampling error alone. To quantify reliability relative to
indicator/measurement error alone (i.e., \omega^\textrm{2L}), the
shared= construct name(s) can additionally be included in
config= argument.
higher
character vector naming any higher-order constructs in
object for which composite reliability should be calculated.
Ignored when tau.eq=TRUE because alpha is not based on a CFA model;
instead, users must fit a CFA with tau-equivalence constraints.
To obtain Lai's (2021) multilevel composite-reliability indices for a
higher-order factor, do not use this argument; instead, specify the
higher-order factor(s) using the shared= or config= argument
(compRelSEM will automatically check whether it includes latent
indicators and apply the appropriate formula).
return.total
logical indicating whether to return a final
column containing the reliability of a composite of all indicators (not
listed in omit.indicators) of first-order factors not listed in
omit.factors. Ignored in 1-factor models, and should only be set
TRUE if all factors represent scale dimensions that could be
meaningfully collapsed to a single composite (scale sum or scale mean).
Setting a negative value (e.g., -1 returns only the
total-composite reliability (excluding coefficients per factor), except
when requesting Lai's (2021) coefficients for multilevel configural
or shared= constructs.
dropSingle
logical indicating whether to exclude factors
defined by a single indicator from the returned results. If TRUE
(default), single indicators will still be included in the total
column when return.total = TRUE.
omit.factors
character vector naming any common factors
modeled in object whose composite reliability is not of
interest. For example, higher-order or method factors. Note that
reliabilityL2() should be used to calculate composite
reliability of a higher-order factor.
omit.indicators
character vector naming any observed variables
that should be omitted from the composite whose reliability is calculated.
omit.imps
character vector specifying criteria for omitting
imputations from pooled results. Can include any of
c("no.conv", "no.se", "no.npd"), the first 2 of which are the
default setting, which excludes any imputations that did not
converge or for which standard errors could not be computed. The
last option ("no.npd") would exclude any imputations which
yielded a nonpositive definite covariance matrix for observed or
latent variables, which would include any "improper solutions" such
as Heywood cases. NPD solutions are not excluded by default because
they are likely to occur due to sampling error, especially in small
samples. However, gross model misspecification could also cause
NPD solutions, users can compare pooled results with and without
this setting as a sensitivity analysis to see whether some
imputations warrant further investigation.
return.df
logical indicating whether to return reliability
coefficients in a data.frame (one row per group/level), which is
possible when every model block includes the same factors (after excluding
those in omit.factors and applying dropSingle).
Details
Several coefficients for factor-analysis reliability have been termed
"omega", which Cho (2021) argues is a misleading misnomer and argues for
using \rho to represent them all, differentiated by descriptive
subscripts. In our package, we strive to provide unlabeled coefficients,
leaving it to the user to decide on a label in their report. But we do
use the symbols \alpha and \omega in the formulas below in order
to distinguish coefficients that do (not) assume essential tau-equivalence.
For higher-order constructs with latent indicators, only \omega is
available. Lai's (2021) multilevel coefficients are labeled in accordance
with the symbols used in that article (more details below).
Bentler (1968) first introduced factor-analysis reliability for a
unidimensional factor model with congeneric indicators, labeling the
coeficients \alpha. McDonald (1999) later referred to this
and other reliability coefficients, first as \theta (in 1970),
then as \omega, which is a source of confusion when reporting
coefficients (Cho, 2021). Coefficients based on factor models were later
generalized to account for multidimenisionality (possibly with
cross-loadings) and correlated errors. The general \omega formula
implemented in this function is:
\omega = \frac{\left( \sum^{k}_{i = 1} \lambda_i \right)^{2}
Var\left( \psi \right)}{\bold{1}^\prime \hat{\Sigma} \bold{1}},
where \hat{\Sigma} can be the model-implied covariance matrix from
either the saturated model (i.e., the "observed" covariance matrix, used by
default) or from the hypothesized CFA model, controlled by the
obs.var argument. A k-dimensional vector \bold{1} is used
to sum elements in the matrix. Note that if the model includes any directed
effects (latent regression slopes), all coefficients are calculated
from total factor variances: lavInspect(object, "cov.lv").
Assuming (essential) tau-equivalence (tau.eq=TRUE) makes \omega
equivalent to coefficient \alpha from classical test theory
(Cronbach, 1951):
\alpha = \frac{k}{k - 1}\left[ 1 - \frac{\sum^{k}_{i = 1}
\sigma_{ii}}{\sum^{k}_{i = 1} \sigma_{ii} + 2\sum_{i < j} \sigma_{ij}}
\right],
where k is the number of items in a factor's composite,
\sigma_{ii} signifies item i's variance, and \sigma_{ij}
signifies the covariance between items i and j. Again, the
obs.var argument controls whether \alpha is calculated using
the observed or model-implied covariance matrix.
By setting return.total=TRUE, one can estimate reliability for a
single composite calculated using all indicators in a multidimensional
CFA (Bentler, 1972, 2009). Setting return.total = -1 will return
only the total-composite reliability (not per factor).
Higher-Order Factors:
The reliability of a composite that represents a higher-order construct
requires partitioning the model-implied factor covariance matrix \Phi
in order to isolate the common-factor variance associated only with the
higher-order factor. Using a second-order factor model, the model-implied
covariance matrix of observed indicators \hat{\Sigma} can be
partitioned into 3 sources:
the second-order common-factor (co)variance:
\Lambda \bold{B} \Phi_2 \bold{B}^{\prime} \Lambda^{\prime}
the residual variance of the first-order common factors (i.e., not
accounted for by the second-order factor):
\Lambda \Psi_{u} \Lambda^{\prime}
the measurement error of observed indicators: \Theta
where \Lambda contains first-order factor loadings, \bold{B}
contains second-order factor loadings, \Phi_2 is the model-implied
covariance matrix of the second-order factor(s), and \Psi_{u} is the
covariance matrix of first-order factor disturbances. In practice, we can
use the full \bold{B} matrix and full model-implied \Phi matrix
(i.e., including all latent factors) because the zeros in \bold{B}
will cancel out unwanted components of \Phi. Thus, we can calculate
the proportion of variance of a composite score calculated from the observed
indicators (e.g., a total score or scale mean) that is attributable to the
second-order factor (i.e., coefficient \omega):
\omega=\frac{\bold{1}^{\prime} \Lambda \bold{B} \Phi \bold{B}^{\prime}
\Lambda^{\prime} \bold{1} }{ \bold{1}^{\prime} \hat{\Sigma} \bold{1}},
where \bold{1} is the k-dimensional vector of 1s and k
is the number of observed indicators in the composite. Note that if a
higher-order factor also has observed indicators, it is necessary to model
the observed indicators as single-indicator constructs, so that all of the
higher-order factor indicators are latent (with loadings in the Beta matrix,
not Lambda).
Categorical Indicators:
When all indicators (per composite) are ordinal, the ord.scale
argument controls whether the coefficient is calculated on the
latent-response scale (FALSE) or on the observed ordinal scale
(TRUE, the default). For \omega-type coefficients
(tau.eq=FALSE), Green and Yang's (2009, formula 21) approach is used
to transform factor-model results back to the ordinal response scale. When
ord.scale=TRUE and tau.eq=TRUE, coefficient \alpha is
calculated using the covariance matrix calculated from the integer-valued
numeric weights for ordinal categories, consistent with its definition
(Chalmers, 2018) and the alpha function in the psych package;
this implies obs.var=TRUE, so obs.var=FALSE will be ignored
When ord.scale=FALSE, the standard \alpha formula is applied to
the polychoric correlation matrix ("ordinal \alpha"; Zumbo et al., 2007),
estimated from the saturated or hypothesized model (see obs.var),
and \omega is calculated from CFA results without applying Green and
Yang's (2009) correction (see Zumbo & Kroc, 2019, for a rationalization).
No method analogous to Green and Yang (2009) has been proposed for
calculating reliability with a mixture of categorical and continuous
indicators, so an error is returned if object includes factors with a
mixture of indicator types (unless omitted using omit.factors). If
categorical indicators load on a different factor(s) than continuous
indicators, then reliability will still be calculated separately for those
factors, but return.total must be FALSE (unless
omit.factors is used to isolate factors with indicators of the same
type).
Multilevel Measurement Models:
Under the default settings, compRelSEM() will apply the same formula
in each "block" (group and/or level of analysis). In the case of multilevel
(ML-)SEMs, this yields "reliability" for latent within- and between-level
components, as proposed by Geldhof et al. (2014). Although this works fine
to calculate reliability per group, this is not recommended for ML-SEMs
because the coefficients do not correspond to actual composites that would
be calculated from the observed data. Lai (2021) proposed coefficients for
reliability of actual composites, depending on the type of construct, which
requires specifying the names of constructs for which reliability is desired
(or multiple constructs whose indicators would compose a multidimensional
composite). Configural (config=) and/or shared= constructs
can be specified; the same construct can be specified in both arguments, so
that overall scale-reliability can be estimated for a shared construct by
including it in config. Instead of organizing the output by block
(the default), specifying config= and/or shared= will prompt
organizing the list of output by $config and/or $shared.
The overall (_2L) scale reliability for configural
constructs is returned, along with the reliability of a purely
individual-level composite (_W, calculated by cluster-mean
centering).
The reliability for a shared construct quantifies
generalizability across both indicators and raters (i.e., subjects rating
their cluster's construct). Lüdtke et al. (2011) refer to these as
measurement error and sampling error, respectively. An interrater
reliability (IRR) coefficient is also returned, quantifying
generalizability across rater/sampling-error only. To obtain a
scale-reliability coefficient (quantifying a shared construct's
generalizability across indicator/measurement-error only), include the
same factor name in config=. Jak et al. (2021) recommended
modeling components of the same construct at both levels, but users may
also saturate the within-level model (Lai, 2021).
Be careful about including Level-2 variables in the model, especially
whether it makes sense to include them in a total composite for a Level-2
construct. dropSingle=TRUE only prevents estimating reliability for
a single-indicator construct, not from including such an indicator in a
total composite. It is permissible for shared= constructs to have
additional indicators at Level-2 only. If it is necessary to model other
Level-2 variables (e.g., to justify the missing-at-random assumption when
using missing="FIML" estimation), they should be placed in the
omit.indicators= argument to exclude them from total composites.
Value
A numeric vector of composite reliability coefficients per
factor, or a list of vectors per "block" (group and/or level of
analysis), optionally returned as a data.frame when possible (see
return.df= argument description for caveat). If there are multiple
factors, whose multidimensional indicators combine into a single
composite, users can request return.total=TRUE to add a column
including a reliability coefficient for the total composite, or
return.total = -1 to return only the total-composite
reliability (ignored when config= or shared= is specified
because each factor's specification must be checked across levels).
Author(s)
Terrence D. Jorgensen (University of Amsterdam; TJorgensen314@gmail.com)
Uses hidden functions written by Sunthud Pornprasertmanit
(psunthud@gmail.com) for the old reliability() function.
References
Bentler, P. M. (1968). Alpha-maximized factor analysis (alphamax): Its
relation to alpha and canonical factor analysis. Psychometrika, 33(3),
335–345. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1007/BF02289328")}
Bentler, P. M. (1972). A lower-bound method for the dimension-free
measurement of internal consistency. Social Science Research, 1(4),
343–357. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1016/0049-089X(72)90082-8")}
Bentler, P. M. (2009). Alpha, dimension-free, and model-based internal
consistency reliability. Psychometrika, 74(1), 137–143.
\Sexpr[results=rd]{tools:::Rd_expr_doi("10.1007/s11336-008-9100-1")}
Chalmers, R. P. (2018). On misconceptions and the limited usefulness of
ordinal alpha. Educational and Psychological Measurement, 78(6),
1056–1071. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1177/0013164417727036")}
Cho, E. (2021) Neither Cronbach’s alpha nor McDonald’s omega: A commentary
on Sijtsma and Pfadt. Psychometrika, 86(4), 877–886.
\Sexpr[results=rd]{tools:::Rd_expr_doi("10.1007/s11336-021-09801-1")}
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of
tests. Psychometrika, 16(3), 297–334. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1007/BF02310555")}
Geldhof, G. J., Preacher, K. J., & Zyphur, M. J. (2014). Reliability
estimation in a multilevel confirmatory factor analysis framework.
Psychological Methods, 19(1), 72–91. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1037/a0032138")}
Green, S. B., & Yang, Y. (2009). Reliability of summed item scores using
structural equation modeling: An alternative to coefficient alpha.
Psychometrika, 74(1), 155–167. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1007/s11336-008-9099-3")}
Jak, S., Jorgensen, T. D., & Rosseel, Y. (2021). Evaluating cluster-level
factor models with lavaan and Mplus. Psych, 3(2),
134–152. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.3390/psych3020012")}
Lai, M. H. C. (2021). Composite reliability of multilevel data: It’s about
observed scores and construct meanings. Psychological Methods, 26(1),
90–102. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1037/met0000287")}
Lüdtke, O., Marsh, H. W., Robitzsch, A., & Trautwein, U. (2011).
A 2 \times 2 taxonomy of multilevel latent contextual models:
Accuracy–bias trade-offs in full and partial error correction models.
Psychological Methods, 16(4), 444–467. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1037/a0024376")}
McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ:
Erlbaum.
Zumbo, B. D., Gadermann, A. M., & Zeisser, C. (2007). Ordinal versions of
coefficients alpha and theta for Likert rating scales.
Journal of Modern Applied Statistical Methods, 6(1), 21–29.
\Sexpr[results=rd]{tools:::Rd_expr_doi("10.22237/jmasm/1177992180")}
Zumbo, B. D., & Kroc, E. (2019). A measurement is a choice and Stevens’
scales of measurement do not help make it: A response to Chalmers.
Educational and Psychological Measurement, 79(6), 1184–1197.
\Sexpr[results=rd]{tools:::Rd_expr_doi("10.1177/0013164419844305")}
See Also
maximalRelia() for the maximal reliability of weighted composite
Examples
data(HolzingerSwineford1939)
HS9 <- HolzingerSwineford1939[ , c("x7","x8","x9")]
HSbinary <- as.data.frame( lapply(HS9, cut, 2, labels=FALSE) )
names(HSbinary) <- c("y7","y8","y9")
HS <- cbind(HolzingerSwineford1939, HSbinary)
HS.model <- ' visual =~ x1 + x2 + x3
textual =~ x4 + x5 + x6
speed =~ y7 + y8 + y9 '
fit <- cfa(HS.model, data = HS, ordered = c("y7","y8","y9"), std.lv = TRUE)
## works for factors with exclusively continuous OR categorical indicators
compRelSEM(fit)
## reliability for ALL indicators only available when they are
## all continuous or all categorical
compRelSEM(fit, omit.factors = "speed", return.total = TRUE)
## loop over visual indicators to calculate alpha if one indicator is removed
for (i in paste0("x", 1:3)) {
cat("Drop ", i, ":\n", sep = "")
print(compRelSEM(fit, omit.factors = c("textual","speed"),
omit.indicators = i, tau.eq = TRUE))
}
## item-total correlations obtainable by adding a composite to the data
HS$Visual <- HS$x1 + HS$x2 + HS$x3
cor(HS$Visual, y = HS[paste0("x", 1:3)])
## comparable to psych::alpha(HS[paste0("x", 1:3)])
## Reliability of a composite that represents a higher-order factor
mod.hi <- ' visual =~ x1 + x2 + x3
textual =~ x4 + x5 + x6
speed =~ x7 + x8 + x9
general =~ visual + textual + speed '
fit.hi <- cfa(mod.hi, data = HolzingerSwineford1939)
compRelSEM(fit.hi, higher = "general")
## reliabilities for lower-order composites also returned
## works for multigroup models and for multilevel models (and both)
data(Demo.twolevel)
## assign clusters to arbitrary groups
Demo.twolevel$g <- ifelse(Demo.twolevel$cluster %% 2L, "type1", "type2")
model2 <- ' group: type1
level: 1
f1 =~ y1 + L2*y2 + L3*y3
f2 =~ y4 + L5*y5 + L6*y6
level: 2
f1 =~ y1 + L2*y2 + L3*y3
f2 =~ y4 + L5*y5 + L6*y6
group: type2
level: 1
f1 =~ y1 + L2*y2 + L3*y3
f2 =~ y4 + L5*y5 + L6*y6
level: 2
f1 =~ y1 + L2*y2 + L3*y3
f2 =~ y4 + L5*y5 + L6*y6
'
fit2 <- sem(model2, data = Demo.twolevel, cluster = "cluster", group = "g")
compRelSEM(fit2) # Geldhof's indices (hypothetical, for latent components)
## Lai's (2021) indices for Level-1 and configural constructs
compRelSEM(fit2, config = c("f1","f2"))
## Lai's (2021) indices for shared (Level-2) constructs
## (also an interrater reliability coefficient)
compRelSEM(fit2, shared = c("f1","f2"))
## Shared construct using saturated within-level model
mod.sat1 <- ' level: 1
y1 ~~ y1 + y2 + y3 + y4 + y5 + y6
y2 ~~ y2 + y3 + y4 + y5 + y6
y3 ~~ y3 + y4 + y5 + y6
y4 ~~ y4 + y5 + y6
y5 ~~ y5 + y6
y6 ~~ y6
level: 2
f1 =~ y1 + L2*y2 + L3*y3
f2 =~ y4 + L5*y5 + L6*y6
'
fit.sat1 <- sem(mod.sat1, data = Demo.twolevel, cluster = "cluster")
compRelSEM(fit.sat1, shared = c("f1","f2"))
## Simultaneous shared-and-configural model (Stapleton et al, 2016, 2019),
## not recommended, but possible by omitting shared or configural factor.
mod.both <- ' level: 1
fc =~ y1 + L2*y2 + L3*y3 + L4*y4 + L5*y5 + L6*y6
level: 2
## configural construct
fc =~ y1 + L2*y2 + L3*y3 + L4*y4 + L5*y5 + L6*y6
## orthogonal shared construct
fs =~ NA*y1 + y2 + y3 + y4 + y5 + y6
fs ~~ 1*fs + 0*fc
'
fit.both <- sem(mod.both, data = Demo.twolevel, cluster = "cluster")
compRelSEM(fit.both, shared = "fs", config = "fc")
semTools documentation built on April 3, 2025, 9:23 p.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| compRelSEM | R Documentation |
Composite Reliability using SEM
Description
Calculate composite reliability from estimated factor-model parameters
Usage
compRelSEM(object, obs.var = TRUE, tau.eq = FALSE, ord.scale = TRUE,
config = character(0), shared = character(0), higher = character(0),
return.total = FALSE, dropSingle = TRUE, omit.factors = character(0),
omit.indicators = character(0), omit.imps = c("no.conv", "no.se"),
return.df = TRUE)
Arguments
object |
A lavaan::lavaan or lavaan.mi::lavaan.mi object, expected to contain only exogenous common factors (i.e., a CFA model). |
obs.var |
|
tau.eq |
|
ord.scale |
|
config |
|
shared |
|
higher |
|
return.total |
|
dropSingle |
|
omit.factors |
|
omit.indicators |
|
omit.imps |
|
return.df |
|
Details
Several coefficients for factor-analysis reliability have been termed
"omega", which Cho (2021) argues is a misleading misnomer and argues for
using \rho to represent them all, differentiated by descriptive
subscripts. In our package, we strive to provide unlabeled coefficients,
leaving it to the user to decide on a label in their report. But we do
use the symbols \alpha and \omega in the formulas below in order
to distinguish coefficients that do (not) assume essential tau-equivalence.
For higher-order constructs with latent indicators, only \omega is
available. Lai's (2021) multilevel coefficients are labeled in accordance
with the symbols used in that article (more details below).
Bentler (1968) first introduced factor-analysis reliability for a
unidimensional factor model with congeneric indicators, labeling the
coeficients \alpha. McDonald (1999) later referred to this
and other reliability coefficients, first as \theta (in 1970),
then as \omega, which is a source of confusion when reporting
coefficients (Cho, 2021). Coefficients based on factor models were later
generalized to account for multidimenisionality (possibly with
cross-loadings) and correlated errors. The general \omega formula
implemented in this function is:
\omega = \frac{\left( \sum^{k}_{i = 1} \lambda_i \right)^{2}
Var\left( \psi \right)}{\bold{1}^\prime \hat{\Sigma} \bold{1}},
where \hat{\Sigma} can be the model-implied covariance matrix from
either the saturated model (i.e., the "observed" covariance matrix, used by
default) or from the hypothesized CFA model, controlled by the
obs.var argument. A k-dimensional vector \bold{1} is used
to sum elements in the matrix. Note that if the model includes any directed
effects (latent regression slopes), all coefficients are calculated
from total factor variances: lavInspect(object, "cov.lv").
Assuming (essential) tau-equivalence (tau.eq=TRUE) makes \omega
equivalent to coefficient \alpha from classical test theory
(Cronbach, 1951):
\alpha = \frac{k}{k - 1}\left[ 1 - \frac{\sum^{k}_{i = 1}
\sigma_{ii}}{\sum^{k}_{i = 1} \sigma_{ii} + 2\sum_{i < j} \sigma_{ij}}
\right],
where k is the number of items in a factor's composite,
\sigma_{ii} signifies item i's variance, and \sigma_{ij}
signifies the covariance between items i and j. Again, the
obs.var argument controls whether \alpha is calculated using
the observed or model-implied covariance matrix.
By setting return.total=TRUE, one can estimate reliability for a
single composite calculated using all indicators in a multidimensional
CFA (Bentler, 1972, 2009). Setting return.total = -1 will return
only the total-composite reliability (not per factor).
Higher-Order Factors:
The reliability of a composite that represents a higher-order construct
requires partitioning the model-implied factor covariance matrix \Phi
in order to isolate the common-factor variance associated only with the
higher-order factor. Using a second-order factor model, the model-implied
covariance matrix of observed indicators \hat{\Sigma} can be
partitioned into 3 sources:
the second-order common-factor (co)variance:
\Lambda \bold{B} \Phi_2 \bold{B}^{\prime} \Lambda^{\prime}the residual variance of the first-order common factors (i.e., not accounted for by the second-order factor):
\Lambda \Psi_{u} \Lambda^{\prime}the measurement error of observed indicators:
\Theta
where \Lambda contains first-order factor loadings, \bold{B}
contains second-order factor loadings, \Phi_2 is the model-implied
covariance matrix of the second-order factor(s), and \Psi_{u} is the
covariance matrix of first-order factor disturbances. In practice, we can
use the full \bold{B} matrix and full model-implied \Phi matrix
(i.e., including all latent factors) because the zeros in \bold{B}
will cancel out unwanted components of \Phi. Thus, we can calculate
the proportion of variance of a composite score calculated from the observed
indicators (e.g., a total score or scale mean) that is attributable to the
second-order factor (i.e., coefficient \omega):
\omega=\frac{\bold{1}^{\prime} \Lambda \bold{B} \Phi \bold{B}^{\prime}
\Lambda^{\prime} \bold{1} }{ \bold{1}^{\prime} \hat{\Sigma} \bold{1}},
where \bold{1} is the k-dimensional vector of 1s and k
is the number of observed indicators in the composite. Note that if a
higher-order factor also has observed indicators, it is necessary to model
the observed indicators as single-indicator constructs, so that all of the
higher-order factor indicators are latent (with loadings in the Beta matrix,
not Lambda).
Categorical Indicators:
When all indicators (per composite) are ordinal, the ord.scale
argument controls whether the coefficient is calculated on the
latent-response scale (FALSE) or on the observed ordinal scale
(TRUE, the default). For \omega-type coefficients
(tau.eq=FALSE), Green and Yang's (2009, formula 21) approach is used
to transform factor-model results back to the ordinal response scale. When
ord.scale=TRUE and tau.eq=TRUE, coefficient \alpha is
calculated using the covariance matrix calculated from the integer-valued
numeric weights for ordinal categories, consistent with its definition
(Chalmers, 2018) and the alpha function in the psych package;
this implies obs.var=TRUE, so obs.var=FALSE will be ignored
When ord.scale=FALSE, the standard \alpha formula is applied to
the polychoric correlation matrix ("ordinal \alpha"; Zumbo et al., 2007),
estimated from the saturated or hypothesized model (see obs.var),
and \omega is calculated from CFA results without applying Green and
Yang's (2009) correction (see Zumbo & Kroc, 2019, for a rationalization).
No method analogous to Green and Yang (2009) has been proposed for
calculating reliability with a mixture of categorical and continuous
indicators, so an error is returned if object includes factors with a
mixture of indicator types (unless omitted using omit.factors). If
categorical indicators load on a different factor(s) than continuous
indicators, then reliability will still be calculated separately for those
factors, but return.total must be FALSE (unless
omit.factors is used to isolate factors with indicators of the same
type).
Multilevel Measurement Models:
Under the default settings, compRelSEM() will apply the same formula
in each "block" (group and/or level of analysis). In the case of multilevel
(ML-)SEMs, this yields "reliability" for latent within- and between-level
components, as proposed by Geldhof et al. (2014). Although this works fine
to calculate reliability per group, this is not recommended for ML-SEMs
because the coefficients do not correspond to actual composites that would
be calculated from the observed data. Lai (2021) proposed coefficients for
reliability of actual composites, depending on the type of construct, which
requires specifying the names of constructs for which reliability is desired
(or multiple constructs whose indicators would compose a multidimensional
composite). Configural (config=) and/or shared= constructs
can be specified; the same construct can be specified in both arguments, so
that overall scale-reliability can be estimated for a shared construct by
including it in config. Instead of organizing the output by block
(the default), specifying config= and/or shared= will prompt
organizing the list of output by $config and/or $shared.
The overall (
_2L) scale reliability forconfigural constructs is returned, along with the reliability of a purely individual-level composite (_W, calculated by cluster-mean centering).The reliability for a
sharedconstruct quantifies generalizability across both indicators and raters (i.e., subjects rating their cluster's construct). Lüdtke et al. (2011) refer to these as measurement error and sampling error, respectively. An interrater reliability (IRR) coefficient is also returned, quantifying generalizability across rater/sampling-error only. To obtain a scale-reliability coefficient (quantifying a shared construct's generalizability across indicator/measurement-error only), include the same factor name inconfig=. Jak et al. (2021) recommended modeling components of the same construct at both levels, but users may also saturate the within-level model (Lai, 2021).
Be careful about including Level-2 variables in the model, especially
whether it makes sense to include them in a total composite for a Level-2
construct. dropSingle=TRUE only prevents estimating reliability for
a single-indicator construct, not from including such an indicator in a
total composite. It is permissible for shared= constructs to have
additional indicators at Level-2 only. If it is necessary to model other
Level-2 variables (e.g., to justify the missing-at-random assumption when
using missing="FIML" estimation), they should be placed in the
omit.indicators= argument to exclude them from total composites.
Value
A numeric vector of composite reliability coefficients per
factor, or a list of vectors per "block" (group and/or level of
analysis), optionally returned as a data.frame when possible (see
return.df= argument description for caveat). If there are multiple
factors, whose multidimensional indicators combine into a single
composite, users can request return.total=TRUE to add a column
including a reliability coefficient for the total composite, or
return.total = -1 to return only the total-composite
reliability (ignored when config= or shared= is specified
because each factor's specification must be checked across levels).
Author(s)
Terrence D. Jorgensen (University of Amsterdam; TJorgensen314@gmail.com)
Uses hidden functions written by Sunthud Pornprasertmanit
(psunthud@gmail.com) for the old reliability() function.
References
Bentler, P. M. (1968). Alpha-maximized factor analysis (alphamax): Its relation to alpha and canonical factor analysis. Psychometrika, 33(3), 335–345. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1007/BF02289328")}
Bentler, P. M. (1972). A lower-bound method for the dimension-free measurement of internal consistency. Social Science Research, 1(4), 343–357. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1016/0049-089X(72)90082-8")}
Bentler, P. M. (2009). Alpha, dimension-free, and model-based internal consistency reliability. Psychometrika, 74(1), 137–143. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1007/s11336-008-9100-1")}
Chalmers, R. P. (2018). On misconceptions and the limited usefulness of ordinal alpha. Educational and Psychological Measurement, 78(6), 1056–1071. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1177/0013164417727036")}
Cho, E. (2021) Neither Cronbach’s alpha nor McDonald’s omega: A commentary on Sijtsma and Pfadt. Psychometrika, 86(4), 877–886. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1007/s11336-021-09801-1")}
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1007/BF02310555")}
Geldhof, G. J., Preacher, K. J., & Zyphur, M. J. (2014). Reliability estimation in a multilevel confirmatory factor analysis framework. Psychological Methods, 19(1), 72–91. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1037/a0032138")}
Green, S. B., & Yang, Y. (2009). Reliability of summed item scores using structural equation modeling: An alternative to coefficient alpha. Psychometrika, 74(1), 155–167. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1007/s11336-008-9099-3")}
Jak, S., Jorgensen, T. D., & Rosseel, Y. (2021). Evaluating cluster-level
factor models with lavaan and Mplus. Psych, 3(2),
134–152. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.3390/psych3020012")}
Lai, M. H. C. (2021). Composite reliability of multilevel data: It’s about observed scores and construct meanings. Psychological Methods, 26(1), 90–102. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1037/met0000287")}
Lüdtke, O., Marsh, H. W., Robitzsch, A., & Trautwein, U. (2011).
A 2 \times 2 taxonomy of multilevel latent contextual models:
Accuracy–bias trade-offs in full and partial error correction models.
Psychological Methods, 16(4), 444–467. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1037/a0024376")}
McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Erlbaum.
Zumbo, B. D., Gadermann, A. M., & Zeisser, C. (2007). Ordinal versions of coefficients alpha and theta for Likert rating scales. Journal of Modern Applied Statistical Methods, 6(1), 21–29. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.22237/jmasm/1177992180")}
Zumbo, B. D., & Kroc, E. (2019). A measurement is a choice and Stevens’ scales of measurement do not help make it: A response to Chalmers. Educational and Psychological Measurement, 79(6), 1184–1197. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1177/0013164419844305")}
See Also
maximalRelia() for the maximal reliability of weighted composite
Examples
data(HolzingerSwineford1939)
HS9 <- HolzingerSwineford1939[ , c("x7","x8","x9")]
HSbinary <- as.data.frame( lapply(HS9, cut, 2, labels=FALSE) )
names(HSbinary) <- c("y7","y8","y9")
HS <- cbind(HolzingerSwineford1939, HSbinary)
HS.model <- ' visual =~ x1 + x2 + x3
textual =~ x4 + x5 + x6
speed =~ y7 + y8 + y9 '
fit <- cfa(HS.model, data = HS, ordered = c("y7","y8","y9"), std.lv = TRUE)
## works for factors with exclusively continuous OR categorical indicators
compRelSEM(fit)
## reliability for ALL indicators only available when they are
## all continuous or all categorical
compRelSEM(fit, omit.factors = "speed", return.total = TRUE)
## loop over visual indicators to calculate alpha if one indicator is removed
for (i in paste0("x", 1:3)) {
cat("Drop ", i, ":\n", sep = "")
print(compRelSEM(fit, omit.factors = c("textual","speed"),
omit.indicators = i, tau.eq = TRUE))
}
## item-total correlations obtainable by adding a composite to the data
HS$Visual <- HS$x1 + HS$x2 + HS$x3
cor(HS$Visual, y = HS[paste0("x", 1:3)])
## comparable to psych::alpha(HS[paste0("x", 1:3)])
## Reliability of a composite that represents a higher-order factor
mod.hi <- ' visual =~ x1 + x2 + x3
textual =~ x4 + x5 + x6
speed =~ x7 + x8 + x9
general =~ visual + textual + speed '
fit.hi <- cfa(mod.hi, data = HolzingerSwineford1939)
compRelSEM(fit.hi, higher = "general")
## reliabilities for lower-order composites also returned
## works for multigroup models and for multilevel models (and both)
data(Demo.twolevel)
## assign clusters to arbitrary groups
Demo.twolevel$g <- ifelse(Demo.twolevel$cluster %% 2L, "type1", "type2")
model2 <- ' group: type1
level: 1
f1 =~ y1 + L2*y2 + L3*y3
f2 =~ y4 + L5*y5 + L6*y6
level: 2
f1 =~ y1 + L2*y2 + L3*y3
f2 =~ y4 + L5*y5 + L6*y6
group: type2
level: 1
f1 =~ y1 + L2*y2 + L3*y3
f2 =~ y4 + L5*y5 + L6*y6
level: 2
f1 =~ y1 + L2*y2 + L3*y3
f2 =~ y4 + L5*y5 + L6*y6
'
fit2 <- sem(model2, data = Demo.twolevel, cluster = "cluster", group = "g")
compRelSEM(fit2) # Geldhof's indices (hypothetical, for latent components)
## Lai's (2021) indices for Level-1 and configural constructs
compRelSEM(fit2, config = c("f1","f2"))
## Lai's (2021) indices for shared (Level-2) constructs
## (also an interrater reliability coefficient)
compRelSEM(fit2, shared = c("f1","f2"))
## Shared construct using saturated within-level model
mod.sat1 <- ' level: 1
y1 ~~ y1 + y2 + y3 + y4 + y5 + y6
y2 ~~ y2 + y3 + y4 + y5 + y6
y3 ~~ y3 + y4 + y5 + y6
y4 ~~ y4 + y5 + y6
y5 ~~ y5 + y6
y6 ~~ y6
level: 2
f1 =~ y1 + L2*y2 + L3*y3
f2 =~ y4 + L5*y5 + L6*y6
'
fit.sat1 <- sem(mod.sat1, data = Demo.twolevel, cluster = "cluster")
compRelSEM(fit.sat1, shared = c("f1","f2"))
## Simultaneous shared-and-configural model (Stapleton et al, 2016, 2019),
## not recommended, but possible by omitting shared or configural factor.
mod.both <- ' level: 1
fc =~ y1 + L2*y2 + L3*y3 + L4*y4 + L5*y5 + L6*y6
level: 2
## configural construct
fc =~ y1 + L2*y2 + L3*y3 + L4*y4 + L5*y5 + L6*y6
## orthogonal shared construct
fs =~ NA*y1 + y2 + y3 + y4 + y5 + y6
fs ~~ 1*fs + 0*fc
'
fit.both <- sem(mod.both, data = Demo.twolevel, cluster = "cluster")
compRelSEM(fit.both, shared = "fs", config = "fc")
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.