read_warc_entry: Read a WARC entry from a WARC file

In hrbrmstr/warc: Tools to Work with the Web Archive Ecosystem

Description

Given the path to a WARC file (compressed or uncompressed) and the start position of the WARC record, this function will produce an R object from the WARC record.

Usage

read_warc_entry(path, start, compressed = grepl(".gz$", path))

Arguments

path	path to WARC file
start	starting offset of WARC record

Details

WARC warinfo objects are returned classed both warc and info.

WARC response objects are returned classed both warc and httr::response and the standard httr content functions will work with the object.

WARC request objects are returned classed both warc and httr::request.

Note

warcinfo, request and response objects are currently supported.

Examples

## Not run: 
cdx <- read_cdx(system.file("extdata", "20160901.cdx", package="warc"))
i <- 1
path <- file.path(cdx$warc_path[i], cdx$file_name[i])
start <- cdx$compressed_arc_file_offset[i]

(read_warc_entry(path, start))

## End(Not run)

hrbrmstr/warc documentation built on May 17, 2019, 5:53 p.m.