filter_links_by_domain: Filter Edge List by Domain or Host

In pagerankr: Modular Toolkit for PageRank Calculation

Filter Edge List by Domain or Host

Description

Filters an edge list by registrable domain and/or host rules. Rows are kept only when both endpoints satisfy the keep/ignore logic. Ignore rules always override keep rules. When keep rules are provided, 'drop_third_party = TRUE' removes URLs outside the keep lists.

This function is intended as a **pre-processing step** before calling [pagerank()]. For example, to scope a PageRank analysis to a single site or exclude CDN / tracking domains.

**Ordering relative to folding:** when 'pagerank()' calls this filter internally (via its 'keep_domains' / 'exclude_domains' / 'keep_hosts' / 'exclude_hosts' arguments), the filter runs *after* redirect and canonical folding, so it scopes the post-fold (canonical) namespace. If an out-of-scope canonical/redirect rewrites the crawled domain/host onto a different one, filtering on the crawled value inside 'pagerank()' matches nothing. To domain-scope the **crawled input** instead, call 'filter_links_by_domain()' on the edge list yourself *before* folding (i.e. before passing it to 'pagerank()').

Usage

filter_links_by_domain(
  edge_list_df,
  from_col = "from",
  to_col = "to",
  keep_domains = NULL,
  keep_hosts = NULL,
  ignore_domains = NULL,
  ignore_hosts = NULL,
  drop_third_party = TRUE,
  return_report = FALSE,
  psl_section = c("all", "icann", "private"),
  rurl_params = list()
)

Arguments

edge_list_df	A data frame representing the edge list, with at least two URL columns.
from_col	Name of the source URL column. Default '"from"'.
to_col	Name of the target URL column. Default '"to"'.
keep_domains	Character vector of registrable domains to keep (e.g., '"example.com"'). Subdomains are included when their registrable domain matches.
keep_hosts	Character vector of specific hosts to keep (e.g., '"www.example.com"'). Only exact host matches are kept.
ignore_domains	Character vector of registrable domains to drop.
ignore_hosts	Character vector of specific hosts to drop.
drop_third_party	Logical. When keep lists are provided and this is 'TRUE' (default), URLs outside the keep lists are dropped. When 'FALSE', only explicitly ignored URLs are dropped.
return_report	Logical. If 'TRUE', returns a list with the filtered data frame and a filter report. Default 'FALSE'.
psl_section	Public Suffix List section used to derive registrable domains, passed to 'rurl::get_domain()' / 'rurl::safe_parse_urls()'. One of '"all"' (default, ICANN + private suffixes), '"icann"', or '"private"'. Affects domain-based (not host-based) keep/ignore matching; e.g. under '"icann"', 'user.github.io' has registrable domain 'github.io', while under '"all"' it is 'user.github.io'.
rurl_params	A list of 'rurl' canonicalization arguments overriding pagerankr's profile per key, used when extracting hosts/domains from both the edge URLs and the keep/ignore values. Pass the **same** profile used to clean the graph so the comparison keys are derived identically. The host-relevant knobs are 'host_encoding' ('"keep"'/'"idna"'/'"unicode"' — IDN folding; e.g. '"idna"' makes 'münchen.de' and 'xn–mnchen-3ya.de' match), 'www_handling', 'subdomain_levels_to_keep', 'case_handling', and 'protocol_handling'. Registrable-domain matching is encoding-independent. When called from [pagerank()], this is forwarded automatically.

Value

If 'return_report = FALSE' (default), the filtered data frame (preserving all columns). If 'TRUE', a list with elements 'filtered_df' and 'report'.

Examples

links <- data.frame(
  from = c(
    "http://www.example.com/a", "http://example.com/b",
    "http://cdn.tracker.com/c"
  ),
  to = c(
    "http://example.com/b", "http://help.example.com/d",
    "http://www.example.com/a"
  )
)

# Keep only example.com edges
filter_links_by_domain(links, keep_domains = "example.com")

# Ignore a specific subdomain
filter_links_by_domain(links, ignore_hosts = "cdn.tracker.com")

# Get a report of what was filtered
result <- filter_links_by_domain(links,
  keep_domains = "example.com",
  return_report = TRUE
)
result$report

pagerankr documentation built on Oct. 1, 2026, 5:09 p.m.