sfa_cd: Comparison-Data Misfit Profile

In semanticfa: Semantic Factor Analysis of Language Model Embeddings

Comparison-Data Misfit Profile

Description

Computes a comparison-data misfit profile for an embedding similarity structure, adapting the comparison data method of Ruscio and Roche (2012) to the response-free setting. For each candidate factor count k, the function builds a finite population of comparison data with known k-factor structure that reproduces both the model-implied correlation matrix and the empirical marginal distributions (an iterative rank-remapping refinement after the GenData program of Ruscio & Kaczetow, 2008), draws bootstrap samples of the empirical size, and records how well each sample's eigenvalue profile reproduces the observed one (root-mean-square residual, RMSR).

Usage

sfa_cd(
  x,
  input = c("embeddings", "data"),
  n_factors_max = 10L,
  n_samples = 500L,
  n_pop = 10000L,
  alpha = NULL,
  fm = "minres",
  gen_iter = 6L,
  seed = 42L
)

Arguments

x	Item embeddings (n_items x embedding_dim; also accepts a fitted "sfa" object or an "sfa_embeddings" object from sfa_load_npz()) when input = "embeddings", or a raw data matrix (cases x variables, e.g. survey responses) when input = "data".
input	Whether x holds item embeddings (default) or raw case-by-variable data.
n_factors_max	Largest factor count to profile (default 10, capped at floor(variables / 3)).
n_samples	Bootstrap samples per factor count (default 500).
n_pop	Size of each comparison population (default 10000).
alpha	Optional alpha for Ruscio and Roche's sequential Mann-Whitney stopping rule. Default NULL skips the rule; see Details for why.
fm	Factor extraction method for the comparison models (default "minres").
gen_iter	Refinement iterations for the population generator (default 6; continuous marginals converge in a few refinements).
seed	Random seed, used via withr::with_seed() without touching the global RNG state.

Details

The deliverable is the profile, not a verdict. On conventional response data with a crisp factor boundary, the profile shows a sharp elbow at the true count. On embedding similarity matrices the misfit typically declines smoothly without an elbow, because a k-factor model with diagonal uniqueness cannot reproduce the heavy anisotropic tail of an embedding spectrum, and each added factor keeps improving reproduction. For the same reason Ruscio and Roche's sequential significance rule (each k tested against k - 1 with a one-tailed Mann-Whitney test) saturates at n_factors_max on embedding matrices at any conventional alpha, and in this package's benchmark runs it inflated with the case count on response data too. The rule is therefore only run when alpha is supplied explicitly, and its verdict should be read alongside the profile shape rather than in place of it.

With input = "embeddings", cases are embedding dimensions: the Pearson correlations of the transposed embedding matrix equal the "mean_centered_pearson" similarity of sfa_similarity(), so the profile addresses exactly the matrix that encoding factors. Other encodings are not correlation matrices of any data matrix, so the profile is computed in the correlation metric regardless.

Value

A list of class "sfa_cd" with components:

median_rmsr

Numeric vector: median RMSR at each factor count (NA where extraction failed).

profile

Numeric vector: median RMSR normalized by its one-factor value.

improvement

Numeric vector: relative improvement (proportion) from each factor count to the next.

rmsr

Numeric matrix (n_samples x n_factors_max): the full RMSR distributions.

eigenvalues

Numeric vector: observed eigenvalues (descending).

n_factors

Integer: sequential-rule verdict, only when alpha was supplied (otherwise NA).

alpha, n, n_samples, n_pop

Settings used.

References

Ruscio, J., & Roche, B. (2012). Determining the number of factors to retain in an exploratory factor analysis using comparison data of known factorial structure. Psychological Assessment, 24(2), 282–292. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1037/a0025697")}

Ruscio, J., & Kaczetow, W. (2008). Simulating multivariate nonnormal data using an iterative algorithm. Multivariate Behavioral Research, 43(3), 355–381. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1080/00273170802285693")}

Goretzko, D., & Ruscio, J. (2024). The comparison data forest: A new comparison data approach to determine the number of factors in exploratory factor analysis. Behavior Research Methods, 56, 1838–1851. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.3758/s13428-023-02122-4")}

Examples

## Not run: 
data(big5)
cd <- sfa_cd(big5$embeddings, n_samples = 100)
print(cd)
plot(cd)

## End(Not run)

semanticfa documentation built on Sept. 2, 2026, 1:07 a.m.