In easystats/parameters: Processing of Model Parameters
library(knitr)
options(knitr.kable.NA = "")
options(digits = 2)
knitr::opts_chunk$set(
echo = TRUE,
collapse = TRUE,
warning = FALSE,
message = FALSE,
comment = "#>",
out.width = "100%"
)
if (!requireNamespace("poorman", quietly = TRUE) ||
!requireNamespace("see", quietly = TRUE) ||
!requireNamespace("lavaan", quietly = TRUE) ||
!requireNamespace("performance", quietly = TRUE) ||
!requireNamespace("nFactors", quietly = TRUE) ||
!requireNamespace("datawizard", quietly = TRUE) ||
!requireNamespace("GPArotation", quietly = TRUE) ||
!requireNamespace("psych", quietly = TRUE)) {
knitr::opts_chunk$set(eval = FALSE)
} else {
library(parameters)
library(performance)
library(GPArotation)
library(psych)
library(nFactors)
library(poorman)
library(lavaan)
}
set.seed(333)
How to perform a Factor Analysis (FA)
The difference between PCA and EFA can be quite hard to intuitively grasp as
their output is very familiar. The idea is that PCA aims at extracting the most
variance possible from all variables of the dataset, whereas EFA aims at
creating consistent factors from the dataset without desperately trying to
represent all the variables.
This is why PCA is popular for feature reduction, as it will try to best
represent the variance contained in the original data, minimizing the loss of
information. On the other hand, EFA is usually in the context of exploring the
latent dimensions that might be hidden in the observed variables, without
necessarily striving to represent the whole dataset.
To illustrate EFA, let us use the International Personality Item Pool data available in the
psych package. It
includes 25 personality self report items. The authors built these items
following the big 5 personality structure.
Factor Structure (Sphericity and KMO)
The first step is to test if the dataset is suitable for carrying out factor analysis. There are two
-
Bartlett's Test of Sphericity: This tests whether a matrix (of correlations)
is significantly different from an identity matrix. The test provides
probability that the correlation matrix has significant correlations among at
least some of the variables in a dataset, a prerequisite for factor analysis to
work. In other words, before starting with factor analysis, one needs to check
whether Bartlett’s test of sphericity is significant.
-
Kaiser, Meyer, Olkin (KMO) Measure of Sampling Adequacy (MSA): This test
was introduced by Kaiser (1970) as the Measure of Sampling Adequacy (MSA), later
modified by Kaiser and Rice (1974). The Kaiser-Meyer-Olkin (KMO) statistic,
which can vary from 0 to 1, indicates the degree to which each variable in a set
is predicted without error by the other variables. A value of 0 indicates that
the sum of partial correlations is large relative to the sum correlations,
indicating factor analysis is likely to be inappropriate. A KMO value close to 1
indicates that the sum of partial correlations is not large relative to the sum
of correlations and so factor analysis should yield distinct and reliable
factors.
Both tests can be performed by using the performance::check_factorstructure() function. First, we set up the data.
library(parameters)
library(psych)
# Load the data
data <- psych::bfi[, 1:25] # Select only the 25 first columns corresponding to the items
data <- na.omit(data) # remove missing values
Next, we check test if the dataset is suitable for carrying out factor analysis.
library(performance)
# Check factor structure
performance::check_factorstructure(data)
Exploratory Factor Analysis (EFA)
Now that we are confident that our dataset is appropriate, we will explore a
factor structure made of 5 latent variables, corresponding to the items' authors
theory of personality.
# Fit an EFA
efa <- psych::fa(data, nfactors = 5) %>%
model_parameters(sort = TRUE, threshold = "max")
efa
As we can see, the 25 items nicely spread on the 5 latent factors, the famous
big 5. Based on this model, we can now predict back the scores for each
individual for these new variables:
predictions <- predict(
efa,
names = c("Neuroticism", "Conscientiousness", "Extraversion", "Agreeableness", "Opennness"),
verbose = FALSE
)
# let's look only at the first five individuals
head(predictions, 5)
How many factors to retain in Factor Analysis (FA)
When running a factor analysis (FA), one often needs to specify how many components (or latent variables) to retain or to extract. This decision is
often motivated or supported by some statistical indices and procedures aiming
at finding the optimal number of factors.
There are a huge number of methods exist to statistically address this issue,
and they can sometimes give very different results.
Unfortunately, there is no consensus on which method to use, or which is the
best.
The Method Agreement procedure
The Method Agreement procedure, first implemented in the
psycho
package [@makowski2018psycho], proposes to rely on the consensus of methods,
rather than on one method in particular.
This procedure can be easily used via the n_factors() function, re-implemented
and improved in the parameters
package. One can provide a dataframe, and the function will run a large number
of routines and return the optimal number of factors based on the higher
consensus.
n <- n_factors(data)
n
Interestingly, the smallest nubmer of factors that most methods suggest is 6,
which is consistent with the newer models of personality (e.g., HEXACO).
More details, as well as a summary table can be obtained as follows:
as.data.frame(n)
summary(n)
A plot can also be obtained (the see package must be loaded):
library(see)
plot(n) + theme_modern()
Confirmatory Factor Analysis (CFA)
We've seen above that while an EFA with 5 latent variables works great on our
dataset, a structure with 6 latent factors might in fact be more appropriate.
How can we statistically test if that is actually the case? This can be done
using Confirmatory Factor Analysis (CFA) (as opposed to Exploratory FA),
which bridges factor analysis with Structural Equation Modeling (SEM).
However, in order to do that cleanly, EFA should be independent from CFA:
the factor structure should be explored in a "training" set, and then tested
(or "confirmed") in a "testing" set.
In other words, the dataset used for exploration and confirmation should not be
the same, a standard widely adopted in the field of machine learning.
Partition the data
The data can be easily split into two sets with the data_partition() function,
through which we will use 70\% of the sample as training and the rest as test.
# to have reproducible result, we will also set seed here so that similar
# portions of the data are used each time we run the following code
partitions <- datawizard::data_partition(data, training_proportion = 0.7, seed = 111)
training <- partitions$p_0.7
test <- partitions$test
Create CFA structures out of EFA models
In the next step, we will run two EFA models on the training set, specifying 5
and 6 latent factors respectively, that we will then transform into CFA
structures.
structure_big5 <- psych::fa(training, nfactors = 5) %>%
efa_to_cfa()
structure_big6 <- psych::fa(training, nfactors = 6) %>%
efa_to_cfa()
# Investigate how the models look
structure_big5
structure_big6
As we can see, a structure is just a string encoding how the manifest variables (the observed variables) are integrated into latent variables.
Fit and Compare models
We can finally apply this structure to the testing dataset using the lavaan
package, and compare these models against each other:
library(lavaan)
library(performance)
big5 <- suppressWarnings(lavaan::cfa(structure_big5, data = test))
big6 <- suppressWarnings(lavaan::cfa(structure_big6, data = test))
performance::compare_performance(big5, big6, verbose = FALSE)
performance::test_likelihoodratio(big5, big6) # TODO: This doesn't work
All in all, it seems that the Big-5 structure remains quite reliable.
Structural Equation Modeling
The previous example shows one of the enormous amount of modeling possibilities for
structural equation models, in particular an example for mediation analysis,
i.e. a model that estimates indirect effects in partial mediation structures.
set.seed(1234)
X <- rnorm(100)
M <- 0.5 * X + rnorm(100)
Y <- 0.7 * M + rnorm(100)
df <- data.frame(X = X, Y = Y, M = M)
model <- " # direct effect
Y ~ c*X
# mediator
M ~ a*X
Y ~ b*M
# indirect effect (a*b)
ab := a*b
# total effect
total := c + (a*b)
"
fit <- lavaan::sem(model, data = df, test = "Satorra-Bentler")
model_parameters(fit)
References
easystats/parameters documentation built on May 2, 2024, 11:22 p.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
library(knitr) options(knitr.kable.NA = "") options(digits = 2) knitr::opts_chunk$set( echo = TRUE, collapse = TRUE, warning = FALSE, message = FALSE, comment = "#>", out.width = "100%" ) if (!requireNamespace("poorman", quietly = TRUE) || !requireNamespace("see", quietly = TRUE) || !requireNamespace("lavaan", quietly = TRUE) || !requireNamespace("performance", quietly = TRUE) || !requireNamespace("nFactors", quietly = TRUE) || !requireNamespace("datawizard", quietly = TRUE) || !requireNamespace("GPArotation", quietly = TRUE) || !requireNamespace("psych", quietly = TRUE)) { knitr::opts_chunk$set(eval = FALSE) } else { library(parameters) library(performance) library(GPArotation) library(psych) library(nFactors) library(poorman) library(lavaan) } set.seed(333)
How to perform a Factor Analysis (FA)
The difference between PCA and EFA can be quite hard to intuitively grasp as their output is very familiar. The idea is that PCA aims at extracting the most variance possible from all variables of the dataset, whereas EFA aims at creating consistent factors from the dataset without desperately trying to represent all the variables.
This is why PCA is popular for feature reduction, as it will try to best represent the variance contained in the original data, minimizing the loss of information. On the other hand, EFA is usually in the context of exploring the latent dimensions that might be hidden in the observed variables, without necessarily striving to represent the whole dataset.
To illustrate EFA, let us use the International Personality Item Pool data available in the
psych package. It
includes 25 personality self report items. The authors built these items
following the big 5 personality structure.
Factor Structure (Sphericity and KMO)
The first step is to test if the dataset is suitable for carrying out factor analysis. There are two
-
Bartlett's Test of Sphericity: This tests whether a matrix (of correlations) is significantly different from an identity matrix. The test provides probability that the correlation matrix has significant correlations among at least some of the variables in a dataset, a prerequisite for factor analysis to work. In other words, before starting with factor analysis, one needs to check whether Bartlett’s test of sphericity is significant.
-
Kaiser, Meyer, Olkin (KMO) Measure of Sampling Adequacy (MSA): This test was introduced by Kaiser (1970) as the Measure of Sampling Adequacy (MSA), later modified by Kaiser and Rice (1974). The Kaiser-Meyer-Olkin (KMO) statistic, which can vary from 0 to 1, indicates the degree to which each variable in a set is predicted without error by the other variables. A value of 0 indicates that the sum of partial correlations is large relative to the sum correlations, indicating factor analysis is likely to be inappropriate. A KMO value close to 1 indicates that the sum of partial correlations is not large relative to the sum of correlations and so factor analysis should yield distinct and reliable factors.
Both tests can be performed by using the performance::check_factorstructure() function. First, we set up the data.
library(parameters) library(psych) # Load the data data <- psych::bfi[, 1:25] # Select only the 25 first columns corresponding to the items data <- na.omit(data) # remove missing values
Next, we check test if the dataset is suitable for carrying out factor analysis.
library(performance) # Check factor structure performance::check_factorstructure(data)
Exploratory Factor Analysis (EFA)
Now that we are confident that our dataset is appropriate, we will explore a factor structure made of 5 latent variables, corresponding to the items' authors theory of personality.
# Fit an EFA efa <- psych::fa(data, nfactors = 5) %>% model_parameters(sort = TRUE, threshold = "max") efa
As we can see, the 25 items nicely spread on the 5 latent factors, the famous big 5. Based on this model, we can now predict back the scores for each individual for these new variables:
predictions <- predict( efa, names = c("Neuroticism", "Conscientiousness", "Extraversion", "Agreeableness", "Opennness"), verbose = FALSE ) # let's look only at the first five individuals head(predictions, 5)
How many factors to retain in Factor Analysis (FA)
When running a factor analysis (FA), one often needs to specify how many components (or latent variables) to retain or to extract. This decision is often motivated or supported by some statistical indices and procedures aiming at finding the optimal number of factors.
There are a huge number of methods exist to statistically address this issue, and they can sometimes give very different results.
Unfortunately, there is no consensus on which method to use, or which is the best.
The Method Agreement procedure
The Method Agreement procedure, first implemented in the
psycho
package [@makowski2018psycho], proposes to rely on the consensus of methods,
rather than on one method in particular.
This procedure can be easily used via the n_factors() function, re-implemented
and improved in the parameters
package. One can provide a dataframe, and the function will run a large number
of routines and return the optimal number of factors based on the higher
consensus.
n <- n_factors(data) n
Interestingly, the smallest nubmer of factors that most methods suggest is 6, which is consistent with the newer models of personality (e.g., HEXACO).
More details, as well as a summary table can be obtained as follows:
as.data.frame(n) summary(n)
A plot can also be obtained (the see package must be loaded):
library(see) plot(n) + theme_modern()
Confirmatory Factor Analysis (CFA)
We've seen above that while an EFA with 5 latent variables works great on our dataset, a structure with 6 latent factors might in fact be more appropriate. How can we statistically test if that is actually the case? This can be done using Confirmatory Factor Analysis (CFA) (as opposed to Exploratory FA), which bridges factor analysis with Structural Equation Modeling (SEM).
However, in order to do that cleanly, EFA should be independent from CFA: the factor structure should be explored in a "training" set, and then tested (or "confirmed") in a "testing" set.
In other words, the dataset used for exploration and confirmation should not be the same, a standard widely adopted in the field of machine learning.
Partition the data
The data can be easily split into two sets with the data_partition() function,
through which we will use 70\% of the sample as training and the rest as test.
# to have reproducible result, we will also set seed here so that similar # portions of the data are used each time we run the following code partitions <- datawizard::data_partition(data, training_proportion = 0.7, seed = 111) training <- partitions$p_0.7 test <- partitions$test
Create CFA structures out of EFA models
In the next step, we will run two EFA models on the training set, specifying 5 and 6 latent factors respectively, that we will then transform into CFA structures.
structure_big5 <- psych::fa(training, nfactors = 5) %>% efa_to_cfa() structure_big6 <- psych::fa(training, nfactors = 6) %>% efa_to_cfa() # Investigate how the models look structure_big5 structure_big6
As we can see, a structure is just a string encoding how the manifest variables (the observed variables) are integrated into latent variables.
Fit and Compare models
We can finally apply this structure to the testing dataset using the lavaan
package, and compare these models against each other:
library(lavaan) library(performance) big5 <- suppressWarnings(lavaan::cfa(structure_big5, data = test)) big6 <- suppressWarnings(lavaan::cfa(structure_big6, data = test)) performance::compare_performance(big5, big6, verbose = FALSE)
performance::test_likelihoodratio(big5, big6) # TODO: This doesn't work
All in all, it seems that the Big-5 structure remains quite reliable.
Structural Equation Modeling
The previous example shows one of the enormous amount of modeling possibilities for structural equation models, in particular an example for mediation analysis, i.e. a model that estimates indirect effects in partial mediation structures.
set.seed(1234) X <- rnorm(100) M <- 0.5 * X + rnorm(100) Y <- 0.7 * M + rnorm(100) df <- data.frame(X = X, Y = Y, M = M) model <- " # direct effect Y ~ c*X # mediator M ~ a*X Y ~ b*M # indirect effect (a*b) ab := a*b # total effect total := c + (a*b) " fit <- lavaan::sem(model, data = df, test = "Satorra-Bentler") model_parameters(fit)
References
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.