sfa_item_fit: Vet a Candidate Scale Item Before Data Collection

In semanticfa: Semantic Factor Analysis of Language Model Embeddings

Vet a Candidate Scale Item Before Data Collection

Description

Scores draft item text against an existing scale: how well it matches each construct, whether it discriminates (low cross-loading risk), how it compares to the construct's current items, and whether it duplicates one of them — entirely response-free. Each candidate is scored on two complementary axes per construct:

Similarity to name

Cosine between the candidate and the embedding of the construct's name (e.g. "Depression"): does it sound like the construct?

Similarity to other items

Cosine between the candidate and the centroid of the construct's existing items: does it look like the other items?

When the two disagree they are informative: high name + low items is a gap-filler (on-topic but covering new ground); low name + high items is drift (looks like the items but not the construct).

Usage

sfa_item_fit(
  x,
  item,
  construct = NULL,
  reverse_key = FALSE,
  redundancy_cutoff = 0.9,
  embed = NULL,
  model = NULL
)

Arguments

x	An object of class "sfa" carrying theoretical factor labels and stored (raw) embeddings.
item	Character vector of one or more candidate items to vet.
construct	Optional name of the construct you intend the item for (matched to the factor labels by exact, case-insensitive, or unique-prefix match). When supplied, the verdict is reported relative to that construct as well as the best-matching one.
reverse_key	Logical; set TRUE to sign-flip the candidate's embedding before comparison. Against the un-flipped reference space this negates the candidate's similarities (the anti-topic direction), so it is a diagnostic contrast, not an assignment mode. Default FALSE.
redundancy_cutoff	Similarity to the nearest existing item at or above which the candidate is flagged as a near-duplicate. Default 0.90.
embed, model	Embedding backend and model used to embed the candidate(s) and the construct names. Default to those recorded on x.

Details

All comparisons run in the raw, un-flipped embedding space, matching sfa_anchor and sfa_simplify: reference items are never sign-flipped by their scoring key, so belonging reflects topic. (A sign-flipped embedding is an anti-topic vector, not a reverse-scored meaning, and flipping reference items depresses the item-similarity profile of constructs with many reverse-keyed items.)

Value

An object of class "sfa_item_fit": a list with similarity_to_name and similarity_to_items (candidate x construct matrices), a per-candidate summary data frame (best construct, the two similarities, second-best construct and gap, strength versus the average existing item, nearest item and its similarity, and a verdict), and the per-construct average existing-item similarity avg_item_fit.

References

Wulff, D. U., & Mata, R. (2025). Semantic embeddings reveal and address taxonomic incommensurability in psychological measurement. Nature Human Behaviour, 9(5), 944–954. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1038/s41562-024-02089-y")}

See Also

sfa_anchor, sfa_redundancy

Examples

## Not run: 
# embeds the candidates live, so this needs the Python backend (or embed=)
data(big5)
fit <- sfa(data.frame(code = big5$codes, item = big5$items,
                      factor = big5$factors),
           embeddings = big5$embeddings, nfactors = 5)
sfa_item_fit(fit, "I make friends easily.",
             model = "Qwen/Qwen3-Embedding-8B")
sfa_item_fit(fit, c("I am the life of every party.",
                    "I rarely feel anxious or depressed."),
             model = "Qwen/Qwen3-Embedding-8B")   # vet several at once

## End(Not run)

semanticfa documentation built on Sept. 2, 2026, 1:07 a.m.