getRecordData: getRecordData
In hferg/porkysdad: Porky's Dad - Useful functions, past and present.
Description
Takes a set of links, and downloads the data for the individual records at
each of those links, saving that data to disk link by link. This is a little
slower than just collecting it all then saving it, but it means that a) we
don't have to store millions of records worth of data in memory, and b) that
if for whatever reason the connection is lost, we have all the data up to
that point in time. It also has a built-in sleep every 1000 links (approx.
half an hour) so that it stops querying the server for around ten minutes.
Usage
1
getRecordData(links, filename)
Arguments
links
Either a vector of links, or the filename of some links file
that was written out by getLinks.
filename
The name of the file that the data is going to be written
into.
hferg/porkysdad documentation built on May 28, 2019, 8:55 p.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Description
Takes a set of links, and downloads the data for the individual records at each of those links, saving that data to disk link by link. This is a little slower than just collecting it all then saving it, but it means that a) we don't have to store millions of records worth of data in memory, and b) that if for whatever reason the connection is lost, we have all the data up to that point in time. It also has a built-in sleep every 1000 links (approx. half an hour) so that it stops querying the server for around ten minutes.
Usage
1 | getRecordData(links, filename)
|
Arguments
links |
Either a vector of links, or the filename of some links file that was written out by getLinks. |
filename |
The name of the file that the data is going to be written into. |
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.