scrape_cdp: Scrape ChemiDPlus

Description Usage Arguments

View source: R/cdp-utils.R

Description

The general scrape function is modified to scrape using the special parameters that ChemiDPlus requires to avoid errors.

Usage

1
2
3
4
5
6
scrape_cdp(
  x,
  options = c("RECOVER", "NOERROR", "NOBLANKS", "HUGE"),
  sleep_time = 3,
  verbose = TRUE
)

Arguments

x

A string, a connection, or a raw vector.

A string can be either a path, a url or literal xml. Urls will be converted into connections either using base::url or, if installed, curl::curl. Local paths ending in .gz, .bz2, .xz, .zip will be automatically uncompressed.

If a connection, the complete connection is read into a raw vector before being parsed.

options

Set parsing options for the libxml2 parser. Zero or more of

RECOVER

recover on errors

NOENT

substitute entities

DTDLOAD

load the external subset

DTDATTR

default DTD attributes

DTDVALID

validate with the DTD

NOERROR

suppress error reports

NOWARNING

suppress warning reports

PEDANTIC

pedantic error reporting

NOBLANKS

remove blank nodes

SAX1

use the SAX1 interface internally

XINCLUDE

Implement XInclude substitition

NONET

Forbid network access

NODICT

Do not reuse the context dictionary

NSCLEAN

remove redundant namespaces declarations

NOCDATA

merge CDATA as text nodes

NOXINCNODE

do not generate XINCLUDE START/END nodes

COMPACT

compact small text nodes; no modification of the tree allowed afterwards (will possibly crash if you try to modify the tree)

OLD10

parse using XML-1.0 before update 5

NOBASEFIX

do not fixup XINCLUDE xml:base uris

HUGE

relax any hardcoded limit from the parser

OLDSAX

parse using SAX2 interface before 2.7.0

IGNORE_ENC

ignore internal document encoding hint

BIG_LINES

Store big lines numbers in text PSVI field

sleep_time

Time in seconds for the system to sleep before each scrape with read_html.

verbose

When reading from a slow connection, this prints some output on every iteration so you know its working.


meerapatelmd/skyscraper documentation built on Dec. 27, 2020, 7:46 a.m.