sfa_build_region: Build a Construct Region from a Text Corpus

In semanticfa: Semantic Factor Analysis of Language Model Embeddings

Build a Construct Region from a Text Corpus

Description

Assembles the corpus half of a content-validity audit: a construct region, the cloud of real sentences that mention a construct term, embedded in the same space that sfa_coverage() will embed the scale's items into. The result is a self-contained object recording its own provenance (corpus, extraction parameters, encoder, date); save it with ⁠file =⁠ and archive it with your analysis so the audit is reproducible.

Usage

sfa_build_region(
  construct,
  definition,
  corpus = "fineweb-10bt",
  target = 1500,
  max_docs = 2e+07,
  sentences_per_doc = 3,
  min_chars = 30,
  max_chars = 500,
  variants = NULL,
  embed = "sbert",
  model = NULL,
  instruction = TRUE,
  cache = TRUE,
  file = NULL,
  progress = TRUE
)

Arguments

construct	Construct term to search for, e.g. "procrastination".
definition	One- or two-sentence definition of the intended sense. Stored with the region and used as the default sense-gate seed in sfa_coverage().
corpus	"fineweb-10bt" (default; streams from the Hugging Face Hub), a character vector of documents, a data frame with a text/item column, or a character vector of file or directory paths.
target	Stop once this many matching sentences are collected. Default 1500.
max_docs	Maximum number of corpus documents to scan. Default 2e7 (covers the full FineWeb 10BT sample). Lower it to cap runtime on a laptop; the saturation diagnostics in sfa_coverage() show whether the smaller region was enough.
sentences_per_doc	Maximum sentences kept per document (guards against one document flooding the region). Default 3.
min_chars, max_chars	Sentence length bounds. Defaults 30 and 500.
variants	Character vector of term spellings to match. Default NULL generates simple morphological variants of construct.
embed, model, cache	Passed to sfa_embed(); model defaults to the package's default encoder. The audit must use the same encoder, which sfa_coverage() enforces from the region's metadata.
instruction	TRUE (default) embeds sentences under the construct-retrieval instruction (recommended: this register alignment outperformed alternatives in validation), FALSE embeds raw text, or a custom instruction string.
file	Optional path; when given, the region is saved there with saveRDS() and can be reloaded with sfa_load_region().
progress	Print progress while streaming. Default TRUE.

Details

Two kinds of corpus are supported. corpus = "fineweb-10bt" streams the ⁠sample-10BT⁠ configuration of the FineWeb corpus (a documented random sample of a modern LLM-training corpus) from the Hugging Face Hub via the Python datasets package, stopping as soon as target sentences are found or max_docs documents have been scanned; common construct terms hit their quota within minutes, rare terms scan the full sample (use an HPC batch job, or lower max_docs and check saturation). Alternatively, pass your own corpus: a character vector of documents, a data frame with a text column, or paths to plain-text files or directories. A domain-specific corpus (for example, workplace communications for a workplace construct) is a fully disclosed design choice recorded in the region's provenance.

Sentences are not sense-filtered here: the region stores every mention, and sfa_coverage() applies its sense gate at audit time against the definition supplied there. This is what makes construct narrowing cheap: one region file for "procrastination" can be re-audited as "academic procrastination" by re-gating with a narrower definition, with no new extraction.

Value

An object of class "sfa_region": a list with the sentences and their sources, the embedding matrix, and full provenance metadata.

See Also

sfa_coverage() to audit a scale against the region, sfa_load_region() to reload a saved region.

Examples

## Not run: 
region <- sfa_build_region(
  construct  = "procrastination",
  definition = paste("Procrastination is the voluntary delay of an",
                     "intended action despite expecting to be worse off."),
  file       = "procrastination_region.rds"
)

# a laptop-friendly build capped at 2M documents
region <- sfa_build_region("procrastination", definition = "...",
                           max_docs = 2e6)

# your own corpus
region <- sfa_build_region("procrastination", definition = "...",
                           corpus = "~/corpora/workplace_emails/")

## End(Not run)

semanticfa documentation built on Sept. 2, 2026, 1:07 a.m.