observe_wacz: Observe a WACZ web archive

View source: R/observe_wacz.R

observe_waczR Documentation

Observe a WACZ web archive

Description

Creates an observational data frame from a WACZ web archive.

A WACZ (Web Archive Collection Zipped) file is an open package format for storing and exchanging web archive collections. A WACZ archive is a ZIP container that combines one or more WARC files containing captured web resources with indexes, page metadata, and package metadata needed to locate, describe, validate, and replay the archived content.

Unlike a standalone WARC file, which primarily stores captured HTTP requests and responses and their payloads, a WACZ archive provides additional structure around those captures. Typical components include WARC files, a CDX index, a page list, and a datapackage.json manifest.

observe_wacz() extracts this structural information, combines page-level information with WARC index metadata, and returns one observational row for each archived web page.

The resulting object represents observations only. It intentionally avoids making semantic assertions about Records, Record Parts, Instantiations, or other archival entities. Such interpretation can be added later with wacz_to_recordset_df() or downstream semantic enrichment workflows.

Usage

observe_wacz(wacz)

Arguments

wacz

Path to a .wacz archive.

Details

A WACZ archive commonly contains several complementary sources of observational information. pages/pages.jsonl describes archived pages, while indexes/index.cdx indexes captured resources stored in the WARC files. The datapackage.json manifest describes the package itself and its constituent resources.

The function currently performs the following steps:

  • extracts the WACZ archive into a temporary directory;

  • reads the archive datapackage.json;

  • parses page metadata from pages/pages.jsonl;

  • parses WARC index metadata from indexes/index.cdx;

  • collapses multiple archived entries for the same resource;

  • joins page observations with archive metadata.

The resulting observations preserve evidence supplied by the archive without interpreting its archival semantics. The underlying WARC, index, page, and package structures may therefore support richer observational analysis than is exposed by the current page-level representation.

Value

A tibble containing observations extracted from the archive.

The returned object carries two attributes:

  • datapackage, containing the parsed datapackage.json metadata supplied by the WACZ archive;

  • wacz, containing the normalized path to the source archive.

Typical variables include:

  • page identifiers;

  • resource locators (URLs);

  • page titles;

  • timestamps;

  • extracted text;

  • text signatures;

  • MIME types;

  • WARC digests;

  • archive offsets;

  • version counts.

References

Webrecorder WACZ format specification: https://specs.webrecorder.net/wacz/1.1.1/

See Also

wacz_to_recordset_df()

Examples

wacz <- system.file(
  "testdata", "fscontext_020.wacz",
  package = "fscontext"
)

observe_wacz(wacz)


fscontext documentation built on Nov. 19, 2026, 5:11 p.m.