Getting Started with semanticfa
In semanticfa: Semantic Factor Analysis of Language Model Embeddings

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.width = 7,
  fig.height = 5
)

Overview

semanticfa performs exploratory factor analysis on language model embeddings of psychological scale items. Given item text, it embeds each item, computes a similarity matrix, and extracts latent factors — entirely from the text, with no human response data required.

The package is designed to feel familiar to psych and EFAtools users.

Quick start with bundled data

The package ships with the 50-item IPIP Big Five inventory and precomputed Qwen3-Embedding-8B embeddings (50 x 4096, rounded to 4 decimal places), so you can try it with zero setup:

library(semanticfa)
data(big5)

fit <- sfa(
  big5$items,
  nfactors    = 5,
  embeddings  = big5$embeddings,
  scoring     = big5$scoring
)
print(fit)

Factor retention

When you omit nfactors, sfa() uses embedding-adapted parallel analysis (random unit vectors in the embedding dimension as the null):

fit_auto <- sfa(
  big5$items,
  embeddings = big5$embeddings,
  scoring    = big5$scoring
)
cat("Auto-detected factors:", fit_auto$factors, "\n")

For a multi-method comparison, use sfa_nfactors(). Its default runs parallel analysis alone (the conventional retention default); request the battery explicitly, choosing among "kaiser", "TEFI", "EKC", "EGA", "MAP", and "semk" (the calibrated learned rule, which reports a conformal interval alongside its count):

sim <- sfa_similarity(big5$embeddings, encoding = "atomic_reversed",
                      scoring = big5$scoring)
nf <- sfa_nfactors(sim, big5$embeddings,
                   methods = c("parallel", "kaiser", "EKC"),
                   parallel_iter = 50)
print(nf)

When you want to see whether the matrix carries a crisp factor count at all, sfa_cd() computes a comparison-data misfit profile (see ?sfa_cd): a sharp elbow marks a real boundary, while embedding similarity matrices typically decline smoothly instead.

Encoding methods

The encoding argument controls how embeddings become a similarity matrix:

sim_ar  <- sfa_similarity(big5$embeddings, "atomic_reversed", big5$scoring)
sim_sq  <- sfa_similarity(big5$embeddings, "squid", big5$scoring)
sim_mcp <- sfa_similarity(big5$embeddings, "mean_centered_pearson", big5$scoring)

cat("atomic_reversed range:", range(sim_ar[lower.tri(sim_ar)]), "\n")
cat("squid range:          ", range(sim_sq[lower.tri(sim_sq)]), "\n")
cat("mean_centered_pearson:", range(sim_mcp[lower.tri(sim_mcp)]), "\n")

The ranges show what each encoding does to the sign structure. Sign-flipping (atomic_reversed) manufactures strong negatives by turning reverse-keyed items into anti-topic vectors, and SQuID's questionnaire-mean centering recovers a modest negative range. Mean-centered Pearson tracks the raw cosines, which stay positive for these items.

Visualization

plot(fit, type = "scree")

plot(fit, type = "loadings")

Comparing with empirical factor analysis

The $loadings component works directly with psych functions:

# Run human-data EFA (not run — requires response data)
human_fit <- psych::fa(response_data, nfactors = 5, rotate = "oblimin")

# Compare
psych::factor.congruence(fit$loadings, human_fit$loadings)

For NMI, ARI, Frobenius, and disattenuated correlation:

cong <- sfa_congruence(fit, big5$factors, metrics = c("nmi", "ari"))
print(cong)

Using your own embeddings

Pass any embedding model's output via embeddings=:

# With sentence-transformers (requires reticulate + Python).
# The default model is "Qwen/Qwen3-Embedding-0.6B"; larger models such as
# "Qwen/Qwen3-Embedding-4B" (8 GB RAM) or "Qwen/Qwen3-Embedding-8B" (16 GB RAM)
# recover factor structure more accurately.
emb <- sfa_embed(my_items, embed = "sbert", model = "Qwen/Qwen3-Embedding-0.6B")
fit <- sfa(my_items, embeddings = emb, scoring = my_scoring)

# Or bring your own function
my_embedder <- function(texts) {
  # ... your embedding logic ...
  # must return a numeric matrix (n_items x dim)
}
fit <- sfa(my_items, embed = my_embedder, scoring = my_scoring)