process_drug_link_url: Process the Links found in the Drug Link Table for NCIt and...

In meerapatelmd/skyscraper: Scrape Clinical Drug Data

Description

Run the full sequence that scrapes, parses, and stores the NCI Drug Dictionary found at CancerGov.org and any correlates to the NCI Thesaurus in a Postgres Database.

Usage

process_drug_link_url(
  conn,
  sleep_time = 5,
  expiration_days = 30,
  encoding = "",
  options = c("RECOVER", "NOERROR", "NOBLANKS"),
  verbose = TRUE,
  render_sql = TRUE
)

Arguments

conn	Postgres connection object.
sleep_time	Time in seconds for the system to sleep before each scrape with read_html.
verbose	When reading from a slow connection, this prints some output on every iteration so you know its working.

Drug Detail Links

The links to Drug Pages are scraped from the Data Dictionary URL over the maximum page number and are saved to a Drug Link Table in the cancergov schema. The URLs in the Drug Link Table are then scraped for any HTML Tables of synonyms and the results are written to a Drug Link Synonym Table. The links to active clinical trials and NCIt mappings are also derived and stored in their respective tables.

See Also

query,appendTable render typewrite_progress html_nodes,html_text tibble

meerapatelmd/skyscraper documentation built on Dec. 27, 2020, 7:46 a.m.