Before we can actually scrap data from the website, we need to setup a web browser that will run in the background. I recommand using Firefox (mainly because I could not manage doing it with Chrome, but you're free to try it).
RSelenium also needs an external software to create a profile file for Firexox, called RTools. The installer can be downloaded here: https://cran.r-project.org/bin/windows/Rtools/index.html. During the instalation tick the box "Add Rtools to system path"
RSelenium is a great package to scrap data from website but it is unfortunately not maintained anymore and should be installed manually from the CRAN archive. For that, yo'll need to have the library devtools installed first
install.packages("devtools")
Then, you can install the latest archived version of RSelenium (1.7.1), wdman and binman
devtools::install_version("RSelenium", "1.7.1")
devtools::install_version("wdman", "0.2.2")
devtools::install_version("binman", "0.1.0")
A brand new, not super efficient and certainly not optimized package to scrap the opentransportmap website!
devtools::install_github("vpellissier/mapscrapper")
Downloading all the zip file for a given country is quite straightforward. First, we need to know the name of the countries in the first dropdownlist
The first step is to start a Selenium server and open the website through R (this open a firefox windows that you should not close):
library(RSelenium)
rDr <- rsDriver(port = 4567L, browser = "firefox", verbose = F)
remDr <- rDr[["client"]]
Sys.sleep(5)
remDr$navigate("http://opentransportmap.info/download/")
Then, one can use the dropdown_values function wich lists all the possible values of a given NUTS level (here, 0):
library(mapscrapper)
names_nuts0 <- dropdown_values(nuts = 0, remDr = remDr)
If this return an error message, you need to re-run the lines in the last two code snippets!
The function returns a vector containing the names of the NUTS0 as they are in the website:
## character(0)
The zip files are going to be downloaded in the default download folder of your computer:
download_map_country("Estonia")
Because the website we are trying to scrap constantly returns Error500 messages, the function might either not start at all (error message) or stop in the middle (without downloading anything). Even worse, it can start downloading, and then skip one item! I have not yet figured out a way to control for that, but I'll look into it.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.