sfa_nli_matrix: Signed Item Similarity from Natural Language Inference

In semanticfa: Semantic Factor Analysis of Language Model Embeddings

Signed Item Similarity from Natural Language Inference

Description

Builds an item-by-item similarity matrix from natural language inference (NLI) rather than cosine similarity. For each ordered item pair the NLI model returns probabilities of entailment (E) and contradiction (C); the signed relation is E - C (near +1 = same meaning/direction, near -1 = opposite). Unlike plain embeddings — which place antonyms close because they share a topic — NLI separates "means the same" from "means the opposite", so reverse-keyed items are handled directly (Bowman et al., 2015; Hommel & Arslan, 2025).

Usage

sfa_nli_matrix(
  items,
  model = "cross-encoder/nli-deberta-v3-base",
  classifier = NULL,
  symmetric = TRUE
)

Arguments

items	Character vector of item texts.
model	NLI cross-encoder model name (default "cross-encoder/nli-deberta-v3-base"), used by the default classifier.
classifier	Optional function taking two equal-length character vectors (premises, hypotheses) and returning a matrix/data frame with numeric columns entailment and contradiction (one row per pair). These are typically probabilities, but any finite numeric scores are accepted — only the signed difference entailment - contradiction is used, so the values need not lie in [0, 1]. Supply this to use a custom NLI backend (or for testing); the default uses a sentence-transformers CrossEncoder via reticulate.
symmetric	Logical: average the two directions (i,j) and (j,i) (default TRUE).

Details

The resulting matrix can be passed straight to sfa via its similarity argument.

The negative-sign-for-contradiction convention adapts Hommel and Arslan (2025), who fine-tuned a sentence-embedding model on SNLI pairs relabeled with signed similarity magnitudes. This function applies the sign convention directly at inference — entailment minus contradiction from an off-the-shelf NLI classifier — with no fine-tuning.

Value

A symmetric numeric matrix (n_items x n_items) of signed relations with 1 on the diagonal and item text as dimnames. With a probability classifier (the default) the off-diagonal values lie in [-1, 1] (1 = same direction, -1 = opposite). A custom classifier returning raw (non-probability) scores may yield values outside [-1, 1]; these are passed through unchanged, so such a matrix may not be correlation-like and may need rescaling before sfa(similarity = ...).

References

Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015). A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (pp. 632–642). Association for Computational Linguistics. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.18653/v1/D15-1075")}

Hommel, B. E., & Arslan, R. C. (2025). Language models accurately infer correlations between psychological items and scales from text alone. Advances in Methods and Practices in Psychological Science, 8(4). \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1177/25152459251377093")}

See Also

sfa, sfa_similarity

Examples

data(big5)
# custom classifier (no Python needed) returning entailment/contradiction probs
clf <- function(premise, hypothesis) {
  same <- substr(premise, 1, 3) == substr(hypothesis, 1, 3)
  data.frame(entailment    = ifelse(same, 0.8, 0.1),
             contradiction = ifelse(same, 0.05, 0.5))
}
M <- sfa_nli_matrix(big5$items[1:6], classifier = clf)
round(M, 2)

## Not run: 
# default backend uses a Python NLI cross-encoder via reticulate:
M <- sfa_nli_matrix(big5$items)
fit <- sfa(big5$items, similarity = M)

## End(Not run)

semanticfa documentation built on Sept. 2, 2026, 1:07 a.m.