pie_cat: Classify fetched HTML with piedomains

In rdomains: Get the Category of Content Hosted by a Domain

Classify fetched HTML with piedomains

Description

Fetches pages with collect_content() and passes their original HTML to piedomains for extraction, tokenization, calibration and classification. The Python package must be version 0.15.0 or newer. This uses its text classifier, not its image or fusion models.

Usage

pie_cat(domains = NULL, pages = NULL, threshold = 0.1, model = NULL, ...)

Arguments

domains	Character vector of domains or URLs. Supply either domains or pages.
pages	Data frame from collect_content(), including its retained html column, or previous piedomains results. Previously extracted text alone is insufficient to reproduce model preprocessing. Failed fetches are retained. Failed inference can be retried on the same pages.
threshold	Finite probability in ⁠[0,1]⁠. Include every category at or above this value from the full model distribution, always retaining the predicted category.
model	Optional Hugging Face model ID or local checkpoint directory. Overrides PIEDOMAINS_TEXT_MODEL for this call. The pinned default model is reused within a session.
...	Passed to collect_content() when pages is absent.

Details

A current fetch does not guarantee a correct current label. The shipped checkpoint has known retail/drugs confusion; see its model card. Archive and cached content retain their collection dates. details preserves the page and its ⁠fetch_*⁠ collection outcomes separately from the terminal classification status, stage, error_code and retryable columns.

Value

A tibble following classification_results. Source-specific data are retained in details; all source labels are retained in labels.

Setup

Install reticulate and declare Python requirements before initializing Python: reticulate::py_require("piedomains==0.15.0", python_version = ">=3.12"). Set RETICULATE_PYTHON to select an existing Python environment. Model weights download from Hugging Face on first use. The Python package pins its default checkpoint; PIEDOMAINS_TEXT_MODEL can select another checkpoint or local path. Results report the resolved model, revision when available, Python package version, extractor, cleaning mode and calibration temperature.

Examples

## Not run: 
pages <- collect_content(c("cnn.com", "wikipedia.org"))
pie_cat(pages = pages)

## End(Not run)

rdomains documentation built on Oct. 29, 2026, 9:07 a.m.