warc_POST: WARC-ify an httr::GET request

In hrbrmstr/jwatr: Tools to Query and Create Web Archive Files Using the Java Web Archive Toolkit

Description

Automagically writes out the result of an httr::POST request to an open WARC file connection enabling seamless recording/caching of the response for later re-use.

Usage

warc_POST(wobj, url = NULL, config = list(), ..., body = NULL,
  encode = c("multipart", "form", "json", "raw"), handle = NULL)

Arguments

wobj	WARC file object
url	the url of the page to retrieve
config	Additional configuration settings such as http authentication (authenticate), additional headers (add_headers), cookies (set_cookies) etc. See config for full details and list of helpers.
...	Further named parameters, such as query, path, etc, passed on to modify_url. Unnamed parameters will be combined with config.
body	One of the following: FALSE: No body. This is typically not used with POST, PUT, or PATCH, but can be useful if you need to send a bodyless request (like GET) with VERB(). NULL: An empty body "": A length 0 body upload_file("path/"): The contents of a file. The mime type will be guessed from the extension, or can be supplied explicitly as the second argument to upload_file() A character or raw vector: sent as is in body. Use content_type to tell the server what sort of data you are sending. A named list: See details for encode.
encode	If the body is a named list, how should it be encoded? Can be one of form (application/x-www-form-urlencoded), multipart, (multipart/form-data), or json (application/json). For "multipart", list elements can be strings or objects created by upload_file. For "form", elements are coerced to strings and escaped, use I() to prevent double-escaping. For "json", parameters are automatically "unboxed" (i.e. length 1 vectors are converted to scalars). To preserve a length 1 vector as a vector, wrap in I(). For "raw", either a character or raw vector. You'll need to make sure to set the content_type() yourself.
handle	The handle to use with this request. If not supplied, will be retrieved and reused from the handle_pool based on the scheme, hostname and port of the url. By default httr requests to the same scheme/host/port combo. This substantially reduces connection time, and ensures that cookies are maintained over multiple requests to the same host. See handle_pool for more details.

Examples

## Not run: 
tf <- tempfile("test")
wf <- warc_file(tf)

warc_GET(wf, "https://data.police.uk/api/crimes-street/all-crime",
         query = list(lat=52.629729, lng=-1.131592, date="2017-01"))

warc_POST(
  wf,
  url = "https://data.police.uk/api/crimes-street/all-crime",
  query = list( lat = "52.629729", lng = "-1.131592", date = "2017-01")
) -> uk_res

close_warc_file(wf)
unlink(tf)

## End(Not run)

hrbrmstr/jwatr documentation built on May 31, 2019, 1:15 p.m.