compRelSEM  R Documentation 
Calculate composite reliability from estimated factormodel parameters
compRelSEM(object, obs.var = TRUE, tau.eq = FALSE, ord.scale = TRUE, config = character(0), shared = character(0), higher = character(0), return.total = FALSE, dropSingle = TRUE, omit.factors = character(0), omit.indicators = character(0), omit.imps = c("no.conv", "no.se"), return.df = TRUE)
object 
A 
obs.var 

tau.eq 

ord.scale 

config 

shared 

higher 

return.total 

dropSingle 

omit.factors 

omit.indicators 

omit.imps 

return.df 

Several coefficients for factoranalysis reliability have been termed "omega", which Cho (2021) argues is a misleading misnomer and argues for using ρ to represent them all, differentiated by descriptive subscripts. In our package, we number ω based on commonly applied calculations.
Bentler (1968) first introduced factoranalysis reliability for a unidimensional factor model with congeneric indicators, labeling the coeficients α. McDonald (1999) later referred to this and other reliability coefficients, first as θ (in 1970), then as ω, which is a source of confusion when reporting coefficients (Cho, 2021). Coefficients based on factor models were later generalized to account for multidimenisionality (possibly with crossloadings) and correlated errors. The general ω formula implemented in this function is:
ω = \frac{≤ft( ∑^{k}_{i = 1} λ_i \right)^{2} Var≤ft( ψ \right)}{\bold{1}^\prime \hat{Σ} \bold{1}},
where \hat{Σ} can be the modelimplied covariance matrix from
either the saturated model (i.e., the "observed" covariance matrix, used by
default) or from the hypothesized CFA model, controlled by the
obs.var
argument. A kdimensional vector \bold{1} is used
to sum elements in the matrix. Note that if the model includes any directed
effects (latent regression slopes), all coefficients are calculated
from total factor variances:
lavInspect(object, "cov.lv")
.
Assuming (essential) tauequivalence (tau.eq=TRUE
) makes ω
equivalent to coefficient α from classical test theory
(Cronbach, 1951):
α = \frac{k}{k  1}≤ft[ 1  \frac{∑^{k}_{i = 1} σ_{ii}}{∑^{k}_{i = 1} σ_{ii} + 2∑_{i < j} σ_{ij}} \right],
where k is the number of items in a factor's composite,
σ_{ii} signifies item i's variance, and σ_{ij}
signifies the covariance between items i and j. Again, the
obs.var
argument controls whether α is calculated using
the observed or modelimplied covariance matrix.
By setting return.total=TRUE
, one can estimate reliability for a
single composite calculated using all indicators in a multidimensional
CFA (Bentler, 1972, 2009). Setting return.total = 1
will return
only the totalcomposite reliability (not per factor).
HigherOrder Factors: The reliability of a composite that represents a higherorder construct requires partitioning the modelimplied factor covariance matrix Φ in order to isolate the commonfactor variance associated only with the higherorder factor. Using a secondorder factor model, the modelimplied covariance matrix of observed indicators \hat{Σ} can be partitioned into 3 sources:
the secondorder commonfactor (co)variance: Λ \bold{B} Φ_2 \bold{B}^{\prime} Λ^{\prime}
the residual variance of the firstorder common factors (i.e., not accounted for by the secondorder factor): Λ Ψ_{u} Λ^{\prime}
the measurement error of observed indicators: Θ
where Λ contains firstorder factor loadings, \bold{B} contains secondorder factor loadings, Φ_2 is the modelimplied covariance matrix of the secondorder factor(s), and Ψ_{u} is the covariance matrix of firstorder factor disturbances. In practice, we can use the full \bold{B} matrix and full modelimplied Φ matrix (i.e., including all latent factors) because the zeros in \bold{B} will cancel out unwanted components of Φ. Thus, we can calculate the proportion of variance of a composite score calculated from the observed indicators (e.g., a total score or scale mean) that is attributable to the secondorder factor (i.e., coefficient ω):
ω_{L1}=\frac{\bold{1}^{\prime} Λ \bold{B} Φ \bold{B}^{\prime} Λ^{\prime} \bold{1} }{ \bold{1}^{\prime} \hat{Σ} \bold{1}},
where \bold{1} is the kdimensional vector of 1s and k is the number of observed indicators in the composite. Note that a higherorder factor can also have observed indicators.
Categorical Indicators:
When all indicators (per composite) are ordinal, the ord.scale
argument controls whether the coefficient is calculated on the
latentresponse scale (FALSE
) or on the observed ordinal scale
(TRUE
, the default). For ωtype coefficients
(tau.eq=FALSE
), Green and Yang's (2009, formula 21) approach is used
to transform factormodel results back to the ordinal response scale.
When ord.scale=TRUE
, coefficient α is calculated using the
covariance matrix calculated from the integervalued numeric weights for
ordinal categories, consistent with its definition (Chalmers, 2018) and the
alpha
function in the psych
package; this implies
obs.var=TRUE
, so obs.var=FALSE
will be ignored. When
ord.scale=FALSE
, the standard α formula is applied to the
polychoric correlation matrix ("ordinal α"; Zumbo et al., 2007),
estimated from the saturated or hypothesized model (see obs.var
),
and ω is calculated from CFA results without applying Green and
Yang's (2009) correction (see Zumbo & Kroc's, 2019, for a rationalization).
No method has been proposed for calculating reliability with a mixture of
categorical and continuous indicators, so an error is returned if
object
includes factors with a mixture of indicator types (unless
omitted using omit.factors
). If categorical indicators load on a
different factor(s) than continuous indicators, then reliability will still
be calculated separately for those factors, but return.total
must be
FALSE
(unless omit.factors
is used to isolate factors with
indicators of the same type).
Multilevel Measurement Models:
Under the default settings, compRelSEM()
will apply the same formula
in each "block" (group and/or level of analysis). In the case of multilevel
SEMs, this yields "reliability" for latent within and betweenlevel
components, as proposed by Geldhof et al. (2014). This is not recommended
because the coefficients do not correspond to actual composites that would
be calculated from the observed data. Lai (2021) proposed coefficients for
reliability of actual composites, depending on the type of construct, which
requires specifying the names of constructs for which reliability is desired
(or multiple constructs whose indicators would compose a multidimensional
composite). Configural (config=
) and/or shared=
constructs
can be specified; the same construct can be specified in both arguments, so
that overall scalereliability can be estimated for a shared construct by
including it in config
. Instead of organizing the output by block
(the default), specifying config=
and/or shared=
will prompt
organizing the output by $config
and/or $shared
.
The overall (_2L
) scale reliability for config
ural
constructs is returned, along with the reliability of a purely
individuallevel composite (_W
, calculated by clustermean
centering).
The reliability for a shared
construct quantifies
generalizability across both indicators and raters (i.e., subjects rating
their cluster's construct). Lüdtke et al. (2011) refer to these as
measurement error and sampling error, respectively. An interrater
reliability (IRR) coefficient is also returned, quantifying
generalizability across rater/samplingerror only. To obtain a
scalereliability coefficient (quantifying a shared construct's
generalizability across indicator/measurementerror only), include the
same factor name in config=
. Jak et al. (2021) recommended
modeling components of the same construct at both levels, but users may
also saturate the withinlevel model (Lai, 2021).
Be careful about including Level2 variables in the model, especially
whether it makes sense to include them in a total composite for a Level2
construct. dropSingle=TRUE
only prevents estimating reliability for
a singleindicator construct, not from including such an indicator in a
total composite. It is permissible for shared=
constructs to have
indicators at Level2 only. If it is necessary to model other Level2
variables (e.g., to justify the missingatrandom assumption when using
missing = "FIML" estimation
), they should be placed in the
omit.indicators=
argument to exclude them from total composites.
A numeric
vector of composite reliability coefficients per
factor, or a list
of vectors per "block" (group and/or level of
analysis), optionally returned as a data.frame
when possible (see
return.df=
argument description for caveat). If there are multiple
factors, whose multidimensional indicators combine into a single
composite, users can request return.total=TRUE
to add a column
including a reliability coefficient for the total composite, or
return.total = 1
to return only the totalcomposite
reliability (ignored when config=
or shared=
is specified
because each factor's specification must be checked across levels).
Terrence D. Jorgensen (University of Amsterdam; TJorgensen314@gmail.com)
Uses hidden functions written by Sunthud Pornprasertmanit
(psunthud@gmail.com) for the old reliability()
function.
Bentler, P. M. (1968). Alphamaximized factor analysis (alphamax): Its relation to alpha and canonical factor analysis. Psychometrika, 33(3), 335–345. doi: 10.1007/BF02289328
Bentler, P. M. (1972). A lowerbound method for the dimensionfree measurement of internal consistency. Social Science Research, 1(4), 343–357. doi: 10.1016/0049089X(72)900828
Bentler, P. M. (2009). Alpha, dimensionfree, and modelbased internal consistency reliability. Psychometrika, 74(1), 137–143. doi: 10.1007/s1133600891001
Chalmers, R. P. (2018). On misconceptions and the limited usefulness of ordinal alpha. Educational and Psychological Measurement, 78(6), 1056–1071. doi: 10.1177/0013164417727036
Cho, E. (2021) Neither Cronbach’s alpha nor McDonald’s omega: A commentary on Sijtsma and Pfadt. Psychometrika, 86(4), 877–886. doi: 10.1007/s11336021098011
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. doi: 10.1007/BF02310555
Geldhof, G. J., Preacher, K. J., & Zyphur, M. J. (2014). Reliability estimation in a multilevel confirmatory factor analysis framework. Psychological Methods, 19(1), 72–91. doi: 10.1037/a0032138
Green, S. B., & Yang, Y. (2009). Reliability of summed item scores using structural equation modeling: An alternative to coefficient alpha. Psychometrika, 74(1), 155–167. doi: 10.1007/s1133600890993
Jak, S., Jorgensen, T. D., & Rosseel, Y. (2021). Evaluating clusterlevel
factor models with lavaan
and Mplus. Psych, 3(2),
134–152. doi: 10.3390/psych3020012
Lai, M. H. C. (2021). Composite reliability of multilevel data: It’s about observed scores and construct meanings. Psychological Methods, 26(1), 90–102. doi: 10.1037/met0000287
Lüdtke, O., Marsh, H. W., Robitzsch, A., & Trautwein, U. (2011). A 2 \times 2 taxonomy of multilevel latent contextual models: Accuracy–bias tradeoffs in full and partial error correction models. Psychological Methods, 16(4), 444–467. doi: 10.1037/a0024376
McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Erlbaum.
Zumbo, B. D., Gadermann, A. M., & Zeisser, C. (2007). Ordinal versions of coefficients alpha and theta for Likert rating scales. Journal of Modern Applied Statistical Methods, 6(1), 21–29. doi: 10.22237/jmasm/1177992180
Zumbo, B. D., & Kroc, E. (2019). A measurement is a choice and Stevens’ scales of measurement do not help make it: A response to Chalmers. Educational and Psychological Measurement, 79(6), 1184–1197. doi: 10.1177/0013164419844305
maximalRelia
for the maximal reliability of weighted composite
data(HolzingerSwineford1939) HS9 < HolzingerSwineford1939[ , c("x7","x8","x9")] HSbinary < as.data.frame( lapply(HS9, cut, 2, labels=FALSE) ) names(HSbinary) < c("y7","y8","y9") HS < cbind(HolzingerSwineford1939, HSbinary) HS.model < ' visual =~ x1 + x2 + x3 textual =~ x4 + x5 + x6 speed =~ y7 + y8 + y9 ' fit < cfa(HS.model, data = HS, ordered = c("y7","y8","y9"), std.lv = TRUE) ## works for factors with exclusively continuous OR categorical indicators compRelSEM(fit) ## reliability for ALL indicators only available when they are ## all continuous or all categorical compRelSEM(fit, omit.factors = "speed", return.total = TRUE) ## loop over visual indicators to calculate alpha if one indicator is removed for (i in paste0("x", 1:3)) { cat("Drop ", i, ":\n", sep = "") print(compRelSEM(fit, omit.factors = c("textual","speed"), omit.indicators = i, tau.eq = TRUE)) } ## Reliability of a composite that represents a higherorder factor mod.hi < ' visual =~ x1 + x2 + x3 textual =~ x4 + x5 + x6 speed =~ x7 + x8 + x9 general =~ visual + textual + speed ' fit.hi < cfa(mod.hi, data = HolzingerSwineford1939) compRelSEM(fit.hi, higher = "general") ## reliabilities for lowerorder composites also returned ## works for multigroup models and for multilevel models (and both) data(Demo.twolevel) ## assign clusters to arbitrary groups Demo.twolevel$g < ifelse(Demo.twolevel$cluster %% 2L, "type1", "type2") model2 < ' group: type1 level: 1 f1 =~ y1 + L2*y2 + L3*y3 f2 =~ y4 + L5*y5 + L6*y6 level: 2 f1 =~ y1 + L2*y2 + L3*y3 f2 =~ y4 + L5*y5 + L6*y6 group: type2 level: 1 f1 =~ y1 + L2*y2 + L3*y3 f2 =~ y4 + L5*y5 + L6*y6 level: 2 f1 =~ y1 + L2*y2 + L3*y3 f2 =~ y4 + L5*y5 + L6*y6 ' fit2 < sem(model2, data = Demo.twolevel, cluster = "cluster", group = "g") compRelSEM(fit2) # Geldhof's indices (hypothetical, for latent components) ## Lai's (2021) indices for Level1 and configural constructs compRelSEM(fit2, config = c("f1","f2")) ## Lai's (2021) indices for shared (Level2) constructs ## (also an interrater reliability coefficient) compRelSEM(fit2, shared = c("f1","f2")) ## Shared construct using saturated withinlevel model mod.sat1 < ' level: 1 y1 ~~ y1 + y2 + y3 + y4 + y5 + y6 y2 ~~ y2 + y3 + y4 + y5 + y6 y3 ~~ y3 + y4 + y5 + y6 y4 ~~ y4 + y5 + y6 y5 ~~ y5 + y6 y6 ~~ y6 level: 2 f1 =~ y1 + L2*y2 + L3*y3 f2 =~ y4 + L5*y5 + L6*y6 ' fit.sat1 < sem(mod.sat1, data = Demo.twolevel, cluster = "cluster") compRelSEM(fit.sat1, shared = c("f1","f2")) ## Simultaneous sharedandconfigural model (Stapleton et al, 2016, 2019), ## not recommended, but possible by omitting shared or configural factor. mod.both < ' level: 1 fc =~ y1 + L2*y2 + L3*y3 + L4*y4 + L5*y5 + L6*y6 level: 2 ## configural construct fc =~ y1 + L2*y2 + L3*y3 + L4*y4 + L5*y5 + L6*y6 ## orthogonal shared construct fs =~ NA*y1 + y2 + y3 + y4 + y5 + y6 fs ~~ 1*fs + 0*fc ' fit.both < sem(mod.both, data = Demo.twolevel, cluster = "cluster") compRelSEM(fit.both, shared = "fs", config = "fc")
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.