screaming_frog_bundle: Compose Screaming Frog node and link exports

In pagerankr: Modular Toolkit for PageRank Calculation

Compose Screaming Frog node and link exports

Description

Builds the stable 'screaming_frog_bundle' handoff object from an **Internal: All** export and one **All Inlinks** or **All Outlinks** export. The component adapters remain the source of truth: raw link observations stay separate from graph-eligible edges, and node, redirect, canonical, and indexability tables are exposed unchanged for downstream scoring.

'pagerankr' accepts a Screaming Frog **Internal: All** export as node metadata and one **All Inlinks** or **All Outlinks** bulk export as link observations. Internal: All is never treated as an edge list. Both link exports use 'Source -> Destination'; the export kind records provenance and never changes orientation.

Usage

screaming_frog_bundle(
  internal,
  links,
  link_export_kind = c("all_inlinks", "all_outlinks"),
  origin_policy = c("all", "html", "rendered"),
  endpoint_action = c("drop", "error")
)

Arguments

internal	A path/data frame accepted by [screaming_frog_internal()], or an existing 'screaming_frog_internal' object.
links	A path/data frame accepted by [screaming_frog_links()], or an existing 'screaming_frog_links' object.
link_export_kind	Declared link export kind when 'links' is not already a 'screaming_frog_links' object: '"all_inlinks"' or '"all_outlinks"'.
origin_policy, endpoint_action	Passed to [screaming_frog_links()] when 'links' is not already imported.

Value

An S3 object of class 'screaming_frog_bundle' with stable top-level fields 'nodes', 'observations', 'edges', 'redirects', 'canonicals', 'indexability', 'diagnostics', and 'provenance'.

Input boundary

Future Screaming Frog adapters accept either a file path or a data frame. Files are CSV, may contain a UTF-8 byte-order mark, and are read by first inspecting the header and then selecting only contract columns. This keeps 500+ MB link exports bounded to the columns needed by the requested adapter. Data-frame inputs follow the same alias resolution and column ordering. Unknown extra columns are ignored and reported; missing required columns fail with their normalized contract names.

Raw URLs are preserved at this boundary. URL cleaning belongs to the scoring path and is performed once with ‘pagerankr'’s pinned 'rurl' canonicalization profile.

Link observation semantics

Raw link observations and graph-eligible edges are separate tables. Duplicate observations are preserved. Only rows whose 'type' is '"Hyperlink"' are graph-eligible by default. Resource, sitemap, hreflang, redirect, and canonical observations are not silently promoted to PageRank edges; redirects and canonicals remain dedicated signals.

'follow' is the primary nofollow field. 'rel' is retained and parsed independently for diagnostics, so disagreements can be reported. Link position is normalized only for 'Navigation', 'Content', 'Footer', 'Header', and 'Aside'; 'Head', blanks, and unknown values remain unmapped rather than being guessed. Link origin and link path are preserved as provenance. Link path is an XPath-like source locator, not a URL path.

Stable bundle shape

The public 'screaming_frog_bundle' object introduced by the adapter tickets is an S3 list with these stable top-level fields, in order:

nodes

Normalized Internal: All node facts.

observations

Lossless normalized link observations.

edges

The graph-eligible 'from' / 'to' subset.

redirects

'from' / 'to' redirect signals.

canonicals

'from' / 'to' canonical signals.

indexability

URL-level indexability facts.

diagnostics

Counts, omissions, invalid values, and disagreements.

provenance

Input kind, source, schema aliases, and schema clues.

This topic freezes the contract consumed by the SF1-SF5 implementation tickets. The complete object is constructed by [screaming_frog_bundle()].

Examples

internal <- data.frame(
  Address = c("https://example.com/", "https://example.com/a"),
  `Status Code` = c("200", "200"),
  check.names = FALSE
)
links <- data.frame(
  Type = "Hyperlink",
  Source = "https://example.com/",
  Destination = "https://example.com/a",
  Follow = "TRUE",
  check.names = FALSE
)
bundle <- screaming_frog_bundle(internal, links, "all_outlinks")
bundle$edges

pagerankr documentation built on Oct. 1, 2026, 5:09 p.m.