sfa_semk: Calibrated Semantic Factor Retention (sem-k)

In semanticfa: Semantic Factor Analysis of Language Model Embeddings

Calibrated Semantic Factor Retention (sem-k)

Description

Estimates the number of semantic factors in an item set from its embeddings using sem-k: a learned retention rule trained on a planted-truth corpus of LLM-written item sets with known structure, embedded in realistic encoder geometry. Unlike null-referenced eigenvalue rules, sem-k is a calibrated estimator: its error rates are measured on held-out planted configurations (65.7% exact, 73.9% within 25% on the v1–v4 corpus under Qwen3-Embedding-8B), and every verdict carries a 90% split-conformal interval.

Usage

sfa_semk(
  sim_matrix = NULL,
  embeddings = NULL,
  floor = NULL,
  seed = 42L,
  download = interactive(),
  quiet = FALSE
)

Arguments

sim_matrix	A fitted "sfa" object, or a similarity matrix (accepted for signature symmetry with the other retention criteria; sem-k computes its own similarity internally from the embeddings).
embeddings	Numeric embedding matrix (n_items x embedding_dim). Required unless sim_matrix is a fitted "sfa" object that carries embeddings.
floor	Register-floor calibration for the encoder that produced the embeddings (mean off-diagonal similarity of construct-dead survey-register items). NULL (default) uses the training encoder's floor (Qwen3-Embedding-8B, 0.478).
seed	Random seed for the feature-extraction bootstrap (verdicts are seed-invariant on 41 of 42 benchmark scales, max spread 1).
download	Permission to download the model artifact if not yet cached. Defaults to interactive() (CRAN policy: no silent downloads).
quiet	Suppress download progress messages.

Details

The estimand is semantic dimensionality: the number of distinguishable meaning clusters the items' embedding geometry supports. Across 35 scales with large response archives, semantic verdicts track empirical human-data dimensionality far better than documented textbook counts do; where the two diverge (for example, single-construct symptom inventories carrying real symptom-cluster structure), human response data typically diverges the same way. Treat sem-k as one voice alongside the granularity evidence in sfa_dimselect() when the interval is wide.

The rule generalizes across encoders: retrained and evaluated on nine encoders from six providers (Qwen 0.6B–8B, e5-mistral, NVIDIA llama-embed-nemotron, Microsoft harrier, OpenAI text-embedding-3 small/large, Google gemini-embedding-2), exact accuracy stays within 61.6–69.0% and real-scale verdicts agree across providers at mean pairwise Spearman .89. The shipped artifact is the Qwen3-Embedding-8B model; for other encoders pass the encoder's register floor via floor (see the calibration files distributed with the sem-k release).

Requires Python with numpy, scipy, scikit-learn (pinned to the 1.8 series, matching the artifact's training version), and joblib (all declared automatically via reticulate::py_require() on first use), and a one-time ~17 MB artifact download (cached under tools::R_user_dir("semanticfa", "cache")).

Value

A list of class "sfa_semk" with components:

n_factors

Integer: the sem-k point estimate of semantic k.

lo90, hi90

Integer bounds of the 90% split-conformal interval (calibrated coverage 91–96% across encoders).

floor

The register floor used.

battery

Named integer vector: the classical battery votes (kaiser, pa_iso, ekc, map) consumed as features, for reference.

artifact

Artifact identifier and training-corpus tag.

References

Yanitski, D., & Westbury, C. (in preparation). How many factors does a questionnaire mean? Validated factor retention for language-model embedding similarity matrices.

Goretzko, D., & Buhner, M. (2020). One model to rule them all? Using machine learning algorithms to determine the number of factors in exploratory factor analysis. Psychological Methods, 25(6), 776–786. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1037/met0000262")}

Examples

## Not run: 
data(big5)
sim <- sfa_similarity(big5$embeddings, "mean_centered_pearson")
sfa_semk(sim, big5$embeddings)  # 5 [2, 13]

## End(Not run)

semanticfa documentation built on Sept. 2, 2026, 1:07 a.m.