fetch_page: Cache a single remote page if not already present

In jefferis/gscraper: Basic web scraping - designed to generate local cache of remote website

Description

Cache a single remote page if not already present

Usage

  fetch_page(url, cacheloc, localpathfn = remotepath,
    cookie = NA, header = FALSE, Verbose = TRUE,
    maxtime = NA, curlopts = NA, DryRun = FALSE,
    ReturnCMD = FALSE)

Arguments

url	Remote URL
cacheloc	Root of local cache
localpathfn	function that converts remote url to (relative) path
cookie	Optional cookie file (e.g. from Mozilla Firefox)
header	Whether to download header rather than content of URL
Verbose	Whether to be verbose
maxtime	Maximum time in seconds to wait for a response
curlopts	Additional options for curl
DryRun	Say what we would do rather than doing it
ReturnCMD	Return resultant shell command rather than running it.

Value

NULL if already exists / on failure, otherwise path to local file

Author(s)

jefferis

See Also

remotepath

Other fetchpage: fetch_missing_pages_parallel

Examples

## Not run: 
fetch_page("https://github.com/jefferis/gscraper/blob/master/README.md",cacheloc=mycache<-tempfile())

## End(Not run)

jefferis/gscraper documentation built on Oct. 25, 2020, 12:08 a.m.