warc_write_response: Write simple 'httr::GET' requests or full 'httr' 'response'...

In hrbrmstr/jwatr: Tools to Query and Create Web Archive Files Using the Java Web Archive Toolkit

Description

If a plain, single, character URL is provided, a curl request will be made and the contents of the result will be added to the WARC file.

Usage

warc_write_response(wobj, x)

Arguments

wobj	a WARC file object created with warc_file
x	Either a single, valid URL to retrieve the contents of or an httr response object

Details

If an httr response object is provided, it will be transformed and the contents of the result will be added to the WARC file.

Value

The passed-in wobj (thus, allowing for pipe chaining)

Examples

## Not run: 
tf <- tempfile("test")
wf <- warc_file(tf)
warc_write_response(wf, "https://rud.is/b/")
warc_write_response(wf, GET("https://rud.is/b/"))
warc_write_response(wf, "https://www.rstudio.com/")
warc_write_response(wf, "https://www.r-project.org/")
warc_write_response(wf, "http://che.org.il/wp-content/uploads/2016/12/pdf-sample.pdf")

POST(
  url = "https://data.police.uk/api/crimes-street/all-crime",
  query = list( lat = "52.629729", lng = "-1.131592", date = "2017-01")
) -> uk_res

warc_write_response(wf, uk_res)
warc_write_response(wf, "https://journal.r-project.org/RLogo.png")

close_warc_file(wf)
unlink(tf)

## End(Not run)

hrbrmstr/jwatr documentation built on May 31, 2019, 1:15 p.m.