fetchInternet: Scrape or Download a File from the Internet

In AndreMikulec/econModel: Meaningful Social and Economic Data from ALFRED and elsewhere

Description

Returned data is either a page, collection of vector lines, a connection, or a file.

Usage

fetchInternet(
  x,
  Structure = "Page",
  Collapse = "\n",
  encode = T,
  Message = paste0("function ", "fetchInternet"),
  conOut = F,
  DestFile,
  Quiet = T,
  Mode = "wb",
  ...
)

Arguments

x	URL
Structure	String. Default is "Page". Return the data as a page. Other is "Lines" to return the result as a group of lines. Another is "BinFile" to download a file; parameter "DestFile" is required.
Collapse	String. Default is the newline escape character. If Structure == "Page" then separate the lines by the newline escape character. Alternately, separate the lines by a single character Collapse. Another choice may be " ". If Structure != "Page", then this parameter is ignored.
encode	Logical. Default is TRUE. Perform URLencoding upon URL.
Message	String. Default is paste0("function ", "fetchInternet"). Message is appended to the user agent.
conOut	Logical. Default is FALSE. If TRUE, instead return immediately the connection object. This ignores (and does not do) all Structure and Collapse processing.
DestFile	String. Default none. Required only when using "Structure = "BinFile". Ignored otherwise. Path and File destination.
Quiet	Logical. Default is TRUE. Only used when using "Structure = "BinFile".
Mode	String. Default is "wb". Only used when using "Structure = "BinFile".
...	Dots. Passed to URLencode

Value

just the connecion object(conOut = T) or the one element page(Page) or many vectors of lines(Lines)

Examples

## Not run: 

writeLines(fetchInternet("https://fred.stlouisfed.org/data/RECPROUSM156N.txt"))
Title:               Smoothed U.S. Recession Probabilities
Series ID:           RECPROUSM156N
Source:              Piger, Jeremy Max, Chauvet, Marcelle
Release:             U.S. Recession Probabilities
Seasonal Adjustment: Not Seasonally Adjusted
Frequency:           Monthly
Units:               Percent
Date Range:          1967-06-01 to 2020-09-01
Last Updated:        2020-11-02 7:01 AM CST

fetchInternet("https://fred.stlouisfed.org/data/RECPROUSM156N.txt", Structure = "Lines")

# tilde expansion (writes NOT HERE but to the users HOME directory)
DirFilePath <- fetchInternet("http://www.econ.yale.edu/~shiller/data/ie_data.xls",
  Structure = "BinFile", DestFile = "~/HERE.xls")

str(DirFilePath)
chr "C:\\APPLICATIONS\\R-4.0._\\R_USER_4.0\\HERE.xls"

## End(Not run)

AndreMikulec/econModel documentation built on June 30, 2021, 9:48 a.m.