In choisy/gso: Data from the General Statistical Office (GSO) of Vietnam
library(knitr)
library(RSelenium)
opts_chunk$set(comment = "#>", error = TRUE, tidy = TRUE)
Introduction
The goal of RSelenium is to make it easy to connect to a Selenium Server/ Remote Selenium Server from within R. RSelenium provides R bindings for the Selenium Webdriver API. Selenium is a project focused on automating web browsers. RSelenium allows you to carry out unit testing and regression testing on your webapps and webpages across a range of browser/OS combinations. This allows us to integrate from within R testing and manipulation of popular projects such as shiny, sauceLabs.
This vignette is divided into six main sections:
Each section will be an introduction to a major idea in Selenium, and point to more detailed explanation in other vignettes.
Connecting to a Selenium Server.
What is a Selenium Server?
Selenium Server is a standalone java program which allows you to run HTML test suites in a range of different browsers, plus extra options like reporting.
You may, or may not, need to run a Selenium Server, depending on how you intend to use Selenium-WebDriver (RSelenium).
Do I need to run a Selenium Server?
If you intend to drive a browser on the same machine that RSelenium is running on then you will need to have Selenium Server running on that machine.
How do I get the Selenium Server stand-alone binary?
You can download the latest Selenium Server binary manually it is currently found here. Look for selenium-server-standalone-x.xx.x.jar.
How do I run the Selenium Server?
The recommended way to run a Selenium Server is by running a Docker
container. There is a seperate vignette which covers the using RSelenium
with Docker see vignette("RSelenium-docker", package = "RSelenium").
For users who are not familiar with Docker there is now a function
rsDriver which will manage the binaries needed for running a Selenium
Server. This provides a wrapper around the wdman::selenium function.
For additional options and more control see the wdman project.
Examples using the rsDriver function are given in the appendix.
Please submit any issues with running binaries to the wdman
project
Alternatively you can run the binary manually. Open a console in your OS and navigate to where the binary is located and run:
java -jar selenium-server-standalone-x.xx.x.jar
By default the Selenium Server listens for connections on port 4444.
Note for Mac OSX: The default port 4444 is sometimes used by other
programs such as kerberos. In our examples we use port 4445 in respect of
this and for cdonsistency with the Docker vignette.
How do I connect to a running server?
RSelenium has a main reference class named remoteDriver. To connect to a server you need to instantiate a new remoteDriver with appropriate options.
# Run a server for example using Docker
# docker run -d -p 4445:4444 selenium/standalone-firefox:2.53.1
# Use a debug image with a VNC viewer if you wish to view the browser
# docker run -d -p 5901:5900 -p 127.0.0.1:4445:4444 --link http-server selenium/standalone-firefox-debug:2.53.1
# See Docker vignette for more detail or run a Selenium Server manually.
require(RSelenium)
remDr <- remoteDriver(remoteServerAddr = "localhost"
, port = 4445L
, browserName = "firefox"
)
Note for Windows: If you are using Docker toolbox your remote server
address will not be localhost. You need to use the ip address of the VM
that is running docker for example
$ docker-machine ip
192.168.99.100
See the Docker vignette for further details. The newer Docker for windows
however should be accessible on the localhost.
It would have been sufficient to call remDr <- remoteDriver(port = 4445L) but the options where explicitly listed to show how one may connect to an arbitrary ip/port/browser etc. More detail maybe found on the sauceLabs vignette. To connect to the server use the open method.
remDr$open()
RSelenium should now have a connection to the Selenium Server. You can query the status of the remote Server using the status method.
> remDr$getStatus()
$build
$build$version
[1] "2.53.1"
$build$revision
[1] "a36b8b1"
$build$time
[1] "2016-06-30 17:37:03"
$os
$os$name
[1] "Linux"
$os$arch
[1] "amd64"
$os$version
[1] "4.4.0-47-generic"
$java
$java$version
[1] "1.8.0_91"
Navigating using RSelenium.
Basic Navigation
To start with we navigate to a url.
remDr$navigate("http://www.google.com/ncr")
Then we navigate to a second page.
remDr$navigate("http://www.bbc.co.uk")
> remDr$getCurrentUrl()
[[1]]
[1] "http://www.bbc.co.uk/"
We can go backwards and forwards using the methods goBack and goForward.
remDr$goBack()
> remDr$getCurrentUrl()
[[1]]
[1] "https://www.google.com/"
remDr$goForward()
> remDr$getCurrentUrl()
[[1]]
[1] "http://www.bbc.co.uk/"
To refresh the current page you can use the `refresh method.
remDr$refresh()
Accessing elements in the DOM.
The DOM stands for the Document Object Model. It is a cross-platform and language-independent convention for representing and interacting with objects in HTML, XHTML and XML documents. Interacting with the DOM will be very important for us with Selenium and the webDriver provides a number of methods in which to do this.
A basic html page is
<!DOCTYPE html>
<html>
<body>
<h1>My First Heading</h1>
<p>My first paragraph.</p>
</body>
</html>
The query box on the front page of http://www.google.com has html code
<input id=..... name="q" ...</input> associated with it. The full html associated with the input tag is:
<input spellcheck="false" dir="ltr" style="border: medium none; padding: 0px; margin: 0px; height: auto; width: 100%; background: transparent url("data:image/gif;base64,R0lGODlhAQABAID/AMDAwAAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw%3D%3D") repeat scroll 0% 0%; position: absolute; z-index: 6; left: 0px; outline: medium none;" aria-autocomplete="both" role="combobox" aria-haspopup="false" class="gsfi" id="lst-ib" maxlength="2048" name="q" autocomplete="off" title="Search" value="" aria-label="Search" type="text">
NOTE: The above HTML is very liable to change however the input node
has had an attribute name = q for sometime so we can mostly rely on this.
Search by name.
To find this element in the DOM a number of methods can be used. We can search by the name.
remDr$navigate("http://www.google.com/ncr")
webElem <- remDr$findElement(using = 'name', value = "q")
> webElem$getElementAttribute("name")
[[1]]
[1] "q"
> webElem$getElementAttribute("class")
[[1]]
[1] "gsfi lst-d-f"
> webElem$getElementAttribute("id")
[[1]]
[1] "lst-ib"
Search by id
In HTML the id of a DOM element should be unique so this is usually a
good locator to use. As noted above the google id of the query box may
change (the one we see is "lst-ib" so we use that). You may see an
alternative "id".
remDr$navigate("http://www.google.com/ncr")
webElem <- remDr$findElement(using = 'id', value = "lst-ib")
Highlight an element
An element that is visible in the DOM can usually be highlighted. Using
our webElem which is an object of class "webElement" we can use the
associated highlight method to visually indicate to us we have the
correct element. Try it:
webElem$highlightElement()
You should see the query box flashing black and yellow to indicate
visually the DOM element you have selected.
Search by class.
We can also search by class name.
remDr$navigate("http://www.google.com/ncr")
webElem <- remDr$findElement(using = 'class', "gsfi")
> webElem$getElementAttribute("class")
[[1]]
[1] "gsfi lst-d-f"
> webElem$getElementAttribute("type")
[[1]]
[1] "text"
NOTE the clas is listed as "gsfi lst-d-f" and we searched using "gsfi"?
This is an example of a compound class. The class name selector can only be
used for elements with a single class ("gsfi lst-d-f" indicates two classes
"gsfi" and "lst-d-f"). For more complicated select queries we therefore use
CSS or xpath instead.
Search using css-selectors
To replicate our name search using css selectors we could use
webElem <- remDr$findElement(using = 'css', "input[name='q']")
# OR
webElem2 <- remDr$findElement(using = 'css', "[name='q']")
We can see we get the same element (using the compareElements method of
the "webElement" class):
> webElem$compareElements(webElem2)
[[1]]
[1] TRUE
and to search on id using css-selectors (again not the id you see maybe
different)
webElem <- remDr$findElement(using = 'css', "input#lst-ib")
> webElem$getElementAttribute("name")
[[1]]
[1] "q"
and class
webElem <- remDr$findElement('css', "[class = 'gsfi lst-d-f']")
NOTE no issue with compound class names with CSS
A good example of searching using css-selectors is given here.
Search using xpath
The final option is to search using xpath. Normally one would use xpath by default when searching or CSS. Both are the go-to options.
Xpath using id.
webElem <- remDr$findElement('xpath', "//input[@id = 'lst-ib']")
Xpath using class.
webElem <- remDr$findElement('xpath', "//input[@class = 'gsfi lst-d-f']")
NOTE with xpath we have no issues using a compound class name.
Sending events to elements.
To illustrate how to interact with elements we will again use the http://www.google.com/ncr as an example.
Sending text to elements
Suppose we would like to search for R cran on google. We would need to find the element for the query box and send the appropriate text to it. We can do this using the sendKeysToElement method for the webElement class.
remDr$navigate("http://www.google.com/ncr")
webElem <- remDr$findElement(using = "css", "[name = 'q']")
webElem$sendKeysToElement(list("R Cran"))
Sending key presses to elements
We should see that the text R Cran has now been entered into the query box.
How do we press enter. We can simply send the enter key to query box. The enter key would be denoted as "\uE007"(its UTF-8 code) . So we could use:
remDr$navigate("http://www.google.com/ncr")
webElem <- remDr$findElement(using = "css", "[name = 'q']")
webElem$sendKeysToElement(list("R Cran", "\uE007"))
It is not very easy to remember UTF-8 codes for appropriate keys so a mapping has been provided in RSelenium. `?selkeys' will bring up a help page explaining the mapping. The UTF-8 codes have been mapped to easy to remember names.
To use selkeys we would send the following
remDr$navigate("http://www.google.com/ncr")
webElem <- remDr$findElement(using = "css", "[name = 'q']")
webElem$sendKeysToElement(list("R Cran", key = "enter"))
Typing selKeys into the console will bring up the list of mappings.
Sending mouse events to elements
For this example we will go back to the google frontpage and search for
R Cran then we will click the link for the The Comprehensive R Archive Network.
remDr$navigate("http://www.google.com/ncr")
webElem <- remDr$findElement(using = "css", "[name = 'q']")
webElem$sendKeysToElement(list("R Cran", key = "enter"))
The header for each link is contained in a <h3 class = "r"> tag. We will access the h3 headers first. It will be succinct to find these elements using css selectors.
webElems <- remDr$findElements(using = 'css selector', "h3.r")
resHeaders <- unlist(lapply(webElems, function(x){x$getElementText()}))
> resHeaders
[1] "The Comprehensive R Archive Network"
[2] "Download R-3.3.2 for Windows. The R-project for statistical ... - CRAN"
[3] "About Microsoft R Open: The Enhanced R Distribution . MRAN"
[4] "R (programming language) - Wikipedia"
[5] "R-Cran - StatLib - Carnegie Mellon University"
[6] "Submitting your first package to CRAN, my experience | R-bloggers"
[7] "Debian -- Package Search Results -- r-cran"
[8] "It's crantastic!"
[9] "METACRAN"
[10] "CRAN - Package PopGenReport"
NOTE this is how the headers were presented at time of writing. Class
names etc. are liable to change.
We can see that the first link is the one we want but in case googles search results change we refer to it as
webElem <- webElems[[which(resHeaders == "The Comprehensive R Archive Network")]]
How do we click the link. We can use the clickElement method:
webElem$clickElement()
> remDr$getCurrentUrl()
[[1]]
[1] "https://cran.r-project.org/"
> remDr$getTitle()
[[1]]
[1] "The Comprehensive R Archive Network"
Injecting JavaScript
Sometimes it is necessary to interact with the current url using JavaScript. This maybe necessary to call bespoke methods or to have more control over the page for example by adding the JQuery library to the page if it is missing. Selenium has two methods we can use to execute JavaScript namely
executeScript and executeAsyncScript from the remoteDriver class. We return to the google front page to investigate these methods.
Injecting JavaScript synchronously
Returning to the google homepage we can find the element for the google image. The image has id = "hplogo" and
we can use this in an xpath or search by id etc to select the element. In this case we use css selectors:
remDr$navigate("http://www.google.com/ncr")
webElem <- remDr$findElement("css", "img#hplogo")
Is the image visible? Clearly it is but we can check using javascript.
> script <- "return document.getElementById('hplogo').hidden;"
> remDr$executeScript(script, args = list())
[[1]]
[1] FALSE
Great so the image is not hidden indicated by the FALSE. We can hide it executing some simple JavaScript.
> script <- "document.getElementById('hplogo').hidden = true;
return document.getElementById('hplogo').hidden;"
> remDr$executeScript(script, args = list())
[[1]]
[1] TRUE
So now the image is hidden. We used an element here given by id = "hplogo". We had to use the JavaScript function
getElementById to access it. It would be nicer if we could have used webElem which we had specified earlier.
If we pass a webElement object as an argument to either executeScript or executeAsyncScript RSelenium will pass it in an appropriate fashion.
> script <-"arguments[0].hidden = false; return arguments[0].hidden;"
> remDr$executeScript(script, args = list(webElem))
[[1]]
[1] FALSE
Notive how we passed the web element to the method executeScript. The script argument defines the script to execute in the form of a function body. The value returned by that function will be returned to the client. The function will be invoked with the provided args. If the function returns an element then this will be returned as an object of class webElement:
script <- "return document.getElementsByName('q');"
test <- remDr$executeScript(script, args = list())
> test[[1]]
[1] "remoteDriver fields"
$remoteServerAddr
[1] "localhost"
$port
[1] 4445
$browserName
[1] "firefox"
$version
[1] ""
$platform
[1] "ANY"
$javascript
[1] TRUE
$nativeEvents
[1] TRUE
$extraCapabilities
list()
[1] "webElement fields"
$elementId
[1] "21"
> class(test[[1]])
[1] "webElement"
attr(,"package")
[1] "RSelenium"
Try to highlight the element as before:
test[[1]]$highlightElement()
Injecting JavaScript asynchronously
I will briefly touch on asynch versus sync calls here. We need to set an
appropriate asynchronous timeout firstly (that is longer than the async
operation we are likely to carryout but it will ensure we will eventually
error out in case of an issue)
remDr$navigate("http://www.google.com/ncr")
remDr$setAsyncScriptTimeout(10000)
Observe:
webElem <- remDr$findElement("css", "img#hplogo")
script <- "
cb = arguments[arguments.length -1];
webElem = arguments[0];
setTimeout(function(){webElem.hidden = true; cb('DONE');},5000);"
remDr$executeAsyncScript(script, args = list(webElem))
versus
remDr$navigate("http://www.google.com/ncr")
webElem <- remDr$findElement("css", "img#hplogo")
script <- "
webElem = arguments[0];
setTimeout(function(){webElem.hidden = true;},5000);
return 'DONE';
"
remDr$executeScript(script, args = list(webElem))
The async version waits until the callback (defined as the last argument
arguments[arguments.length -1] as JavaScript is zero-indexed)
is called whereas the sync version returns straight away. In both cases
the google logo disappears after five seconds.
Frames and Windows.
In the context of a web browser, a frame is a part of a web page or browser window which displays content independent of its container, with the ability to load content independently.
Frames in Selenium
We will demonstrate interacting with frames by way of example. The The Comprehensive R Archive Network conveniently
contains frames so we shall use RSelenium to interact with it.
Assume we have a remoteDriver opened.
remDr$navigate("https://CRAN.r-project.org/")
> XML::htmlParse(remDr$getPageSource()[[1]])
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Frameset//EN" "http://www.w3.org/TR/html4/frameset.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>The Comprehensive R Archive Network</title>
<meta content="text/html; charset=utf-8" http-equiv="content-type">
<link type="image/x-icon" href="favicon.ico" rel="icon">
<link type="image/x-icon" href="favicon.ico" rel="shortcut icon">
<link href="R.css" type="text/css" rel="stylesheet">
</head>
<frameset style="border: none;" cols="1*, 4*">
<frameset rows="120, 1*">
<frame frameborder="0" name="logo" src="logo.html">
<frame frameborder="0" name="contents" src="navbar.html">
</frameset>
<frame frameborder="0" name="banner" src="banner.shtml">
<noframes>
<h1>The Comprehensive R Archive Network</h1>
Your browser seems not to support frames,
here is the <A href="navbar.html">contents page</A> of CRAN.
</noframes>
</frameset>
</html>
We can see the content is contained in three frames and we dont appear to have access to the content within a frame. But in the browser we see all the content:
remDr$maxWindowSize()
remDr$screenshot(display = TRUE)
RProject front page
To access the content we have to switch to a frame using the switchToFrame method of the remoteDriver class.
webElems <- remDr$findElements(using = "tag name", "frame")
# webElems <- remDr$findElements(value = "//frame") # using xpath
# webElems <- remDr$findElements("css", value = "frame") # using css
> sapply(webElems, function(x){x$getElementAttribute("src")})
[[1]]
[1] "https://cran.r-project.org/logo.html"
[[2]]
[1] "https://cran.r-project.org/navbar.html"
[[3]]
[1] "https://cran.r-project.org/banner.shtml"
remDr$switchToFrame(webElems[[2]])
> XML::htmlParse(remDr$getPageSource()[[1]])
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>R Contents</title>
<meta content="text/html; charset=utf-8" http-equiv="content-type">
<link href="R.css" type="text/css" rel="stylesheet">
</head>
<body>
<em class="navigation">CRAN</em><br><a target="banner" href="mirrors.html">Mirrors</a><br><a target="banner" href="//www.R-project.org/news.html">What's new?</a><br><a target="banner" href="web/views/">Task Views</a><br><a target="banner" href="search.html">Search</a><br><!--<a href="pkg_submit.html" target="_top">Submit</a><BR>--><p>
<em class="navigation">About R</em><br><a target="_top" href="//www.R-project.org/">R Homepage</a><br><a target="_top" href="http://journal.R-project.org/">The R Journal</a>
</p>
<p>
<em class="navigation">Software</em><br><a target="banner" href="sources.html">R Sources</a><br><a target="banner" href="bin/">R Binaries</a><br><a target="banner" href="web/packages/">Packages</a><br><a target="banner" href="other-software.html">Other</a>
</p>
<p>
<em class="navigation">Documentation</em><br><a target="banner" href="manuals.html">Manuals</a><br><a target="banner" href="faqs.html">FAQs</a><br><a target="banner" href="other-docs.html">Contributed</a><br></p>
</body>
</html>
Now we see the source code of the navigation sidePanel. Notice how we used a webElement in the method switchToFrame. To further demonstrate we are now "in" this frame lets get all the href attributes:
webElems <- remDr$findElements("css", "[href]")
sapply(webElems, function(x){x$getElementAttribute("href")})
> unlist(sapply(webElems, function(x){x$getElementAttribute("href")}))
[1] "https://cran.r-project.org/R.css"
[2] "https://cran.r-project.org/mirrors.html"
[3] "https://www.r-project.org/news.html"
[4] "https://cran.r-project.org/web/views/"
[5] "https://cran.r-project.org/search.html"
[6] "https://www.r-project.org/"
[7] "http://journal.r-project.org/"
[8] "https://cran.r-project.org/sources.html"
[9] "https://cran.r-project.org/bin/"
[10] "https://cran.r-project.org/web/packages/"
[11] "https://cran.r-project.org/other-software.html"
[12] "https://cran.r-project.org/manuals.html"
[13] "https://cran.r-project.org/faqs.html"
[14] "https://cran.r-project.org/other-docs.html"
Notice if we pass a NULL value to the method switchToFrame we move back to the default view.
remDr$switchToFrame(NULL)
> XML::htmlParse(remDr$getPageSource()[[1]])
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Frameset//EN" "http://www.w3.org/TR/html4/frameset.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>The Comprehensive R Archive Network</title>
<meta content="text/html; charset=utf-8" http-equiv="content-type">
<link type="image/x-icon" href="favicon.ico" rel="icon">
<link type="image/x-icon" href="favicon.ico" rel="shortcut icon">
<link href="R.css" type="text/css" rel="stylesheet">
</head>
<frameset style="border: none;" cols="1*, 4*">
<frameset rows="120, 1*">
<frame frameborder="0" name="logo" src="logo.html">
<frame frameborder="0" name="contents" src="navbar.html">
</frameset>
<frame frameborder="0" name="banner" src="banner.shtml">
<noframes>
<h1>The Comprehensive R Archive Network</h1>
Your browser seems not to support frames,
here is the <A href="navbar.html">contents page</A> of CRAN.
</noframes>
</frameset>
<body><canvas id="fxdriver-screenshot-canvas" style="display: none;" width="1360" height="559"></canvas></body>
</html>
Finally we can switch to the main panel using a name
remDr$switchToFrame("banner")
> XML::htmlParse(remDr$getPageSource()[[1]])
<!DOCTYPE html PUBLIC "-//IETF//DTD HTML//EN">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>The Comprehensive R Archive Network</title>
<link href="R.css" type="text/css" rel="stylesheet">
</head>
<body>
<h1>The Comprehensive R Archive Network</h1>
<div align="center">
<table border="1" width="80%"><tbody>
<tr>
<td>
<h3>Download and Install R</h3>
Precompiled binary distributions of the base system and
contributed packages, <strong>Windows and
Mac</strong> users most likely want one of these versions of R:
<ul>
<li><a href="bin/linux/">Download R for Linux</a></li>
<li>
<a href="bin/macosx/">Download R for (Mac) OS X</a>
</li>
<li><a href="bin/windows/">Download R for Windows</a></li>
</ul>
R is part of many Linux distributions, you should check with your Linux package management system in addition to the link above.
</td>
</tr>
<tr>
<td>
................
................
................
<h3>Questions About R</h3>
<ul><li>
If you have questions about R like how to download and install
the software, or what the license terms are,
please read our <a href="faqs.html">answers to frequently asked
questions</a> before you send an email.
</li></ul>
</td>
</tr>
</tbody></table>
</div>
<h2>What are R and CRAN?</h2>
<p> R is GNU, a freely available language and environment for
statistical computing and graphics which provides a wide variety of
statistical and graphical techniques: linear and nonlinear modelling,
statistical tests, time series analysis, classification, clustering,
etc. Please consult the <a target="_top" href="https://www.R-project.org/">R project homepage</a> for further information.
</p>
<p> CRAN is a network of ftp and web servers around the world that
store identical, up-to-date, versions of code and documentation for
R. Please use the CRAN <a href="mirrors.html">mirror</a> nearest to you to minimize network
load.
</p>
<h2 id="submitting">Submitting to CRAN </h2>
<p>
To submit a package to CRAN,
check that your submission meets the
<a href="https://CRAN.R-project.org/web/packages/policies.html">CRAN
Repository Policy</a> and then use the
<a href="https://xmpalantir.wu.ac.at/cransubmit/">web form</a>.
</p>
<p>
If this fails, upload to
<a target="_blank" href="ftp://CRAN.R-project.org/incoming/">ftp://CRAN.R-project.org/incoming/</a>
and send an email to
<a href="mailto:CRAN@R-project.org">CRAN@R-project.org</a> following the policy.
Please do not attach submissions to emails, because this will clutter up
the mailboxes of half a dozen people.
</p>
<p>
Note that we generally do not accept submissions of precompiled
binaries due to security reasons. All binary distribution listed above
are compiled by selected maintainers, who are in charge for all
binaries of their platform, respectively.
</p>
<p>
</p>
<hr>
<!--#if expr="$CRAN_HOST" --><!--#echo encoding="none" var="CRAN_HOST"--><br><!--#endif -->
</body>
</html>
Windows in Selenium
The easiest way to illustrate Windows in RSelenium is again by way of example. We will use the "CRAN" website.
First we select the download R element in the main frame.
remDr$navigate("https://cran.r-project.org/")
remDr$switchToFrame("banner")
webElems <- remDr$findElements("partial link text", "Download R")
sapply(webElems, function(x) x$getElementText())
[[1]]
[1] "Download R for Linux"
[[2]]
[1] "Download R for (Mac) OS X"
[[3]]
[1] "Download R for Windows"
We now send a selection of key presses to the first element to open the link it points to in a new window. If you did it manually you would move the mouse to the element right click on the link press the down arrow key twice then press enter. We will do the same
loc <- webElems[[1]]$getElementLocation()
> loc[c('x','y')]
$x
[1] 158
$y
[1] 132
remDr$mouseMoveToLocation(webElement = webElems[[1]]) # move mouse to the element we selected
remDr$click(2) # 2 indicates click the right mouse button
remDr$sendKeysToActiveElement(
list(key = 'down_arrow', key = 'down_arrow', key = 'enter')
)
Notice now that a new windows has opened on the remote browser.
> remDr$getCurrentWindowHandle()
[[1]]
[1] "{573d17e5-b95a-41b9-a65f-04092b6a804b}"
> remDr$getWindowHandles()
[[1]]
[1] "{4896393a-c215-4976-b4ca-030d6b75b67d}"
[[2]]
[1] "{69c00f18-d3a7-44d7-a236-c6b5e6c264ff}"
> remDr$getTitle()
[[1]]
[1] "The Comprehensive R Archive Network"
> currWin <- remDr$getCurrentWindowHandle()
> allWins <- unlist(remDr$getWindowHandles())
> otherWindow <- allWins[!allWins %in% currWin[[1]]]
> remDr$switchToWindow(otherWindow)
> remDr$getTitle()
[[1]]
[1] "Index of /bin/linux"
So using the code above one can observe how to switch between different windows on the remote browser.
Appendix
Server functions
In previous versions of RSelenium there were two functions
checkForServer and startServer which were used to download and start a
Selenium binary. These functions are Deprecated/Defunct as users had issue
with using them cross-platform. They are still in the RSelenium package and can be accessed in the serverUtils directory file.path(find.package("RSelenium"), "examples/serverUtils")
> file.path(find.package("RSelenium"), "examples/serverUtils")
[1] "/home/john/R/x86_64-pc-linux-gnu-library/3.3/RSelenium/examples/serverUtils"
They may work for you depending on your platform and setup but are not
supported.
rsDriver
The rsDriver function is a wrapper for the selenium function from the
wdman package. It allows the
user to manage the binaries used to run a Selenium Server. It returns
an environment containing a "client" and a "server".
By default it runs a chrome browser. Other browsers (firefox, PhantomJS,
Internet Explorer (Win)) can be selected using the browser argument.
The default port a Selenium Server is run on using the rsDriver function
is port = 4567L.
rD <- rsDriver(verbose = FALSE)
rD
#> $client
#> browserName id
#> 1 chrome 1670dcc6-3c97-4717-bf28-4d1fc8eea2c1
#>
#> $server
#> Process Handle
#> command : /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Dwebdriver.chrome.driver=/home/john/.local/share/binman_chromedriver/linux64/2.27/chromedriver -Dwebdriver.gecko.driver=/home/john/.local/share/binman_geckodriver/linux64/0.13.0/geckodriver -Dphantomjs.binary.path=/home/john/.local/share/binman_phantomjs/linux64/2.1.1/phantomjs-2.1.1-linux-x86_64/bin/phantomjs -jar /home/john/.local/share/binman_seleniumserver/generic/3.0.1/selenium-server-standalone-3.0.1.jar -port 4567
#> system id : 120670
#> state : running
# assign the client to a new variable
remDr <- rD$client
remDr$navigate("http://www.r-project.org")
remDr$getTitle()
#> [[1]]
#> [1] "R: The R Project for Statistical Computing"
# check the server logs
rD$server$log()
#> $stderr
#> [1] "10:29:11.536 INFO - Selenium build info: version: '3.0.1', revision: '1969d75'"
#> [2] "10:29:11.537 INFO - Launching a standalone Selenium Server"
#> [3] "2017-01-18 10:29:11.551:INFO::main: Logging initialized @179ms"
#> [4] "10:29:11.594 INFO - Driver provider org.openqa.selenium.ie.InternetExplorerDriver registration is skipped:"
#> [5] " registration capabilities Capabilities [{ensureCleanSession=true, browserName=internet explorer, version=, platform=WINDOWS}] does not match the current platform LINUX"
#> [6] "10:29:11.595 INFO - Driver provider org.openqa.selenium.edge.EdgeDriver registration is skipped:"
#> [7] " registration capabilities Capabilities [{browserName=MicrosoftEdge, version=, platform=WINDOWS}] does not match the current platform LINUX"
#> [8] "10:29:11.595 INFO - Driver class not found: com.opera.core.systems.OperaDriver"
#> [9] "10:29:11.595 INFO - Driver provider com.opera.core.systems.OperaDriver registration is skipped:"
#> [10] "Unable to create new instances on this machine."
#> [11] "10:29:11.595 INFO - Driver class not found: com.opera.core.systems.OperaDriver"
#> [12] "10:29:11.595 INFO - Driver provider com.opera.core.systems.OperaDriver is not registered"
#> [13] "10:29:11.596 INFO - Driver provider org.openqa.selenium.safari.SafariDriver registration is skipped:"
#> [14] " registration capabilities Capabilities [{browserName=safari, version=, platform=MAC}] does not match the current platform LINUX"
#> [15] "2017-01-18 10:29:11.628:INFO:osjs.Server:main: jetty-9.2.15.v20160210"
#> [16] "2017-01-18 10:29:11.647:INFO:osjsh.ContextHandler:main: Started o.s.j.s.ServletContextHandler@2ef5e5e3{/,null,AVAILABLE}"
#> [17] "2017-01-18 10:29:11.658:INFO:osjs.ServerConnector:main: Started ServerConnector@724af044{HTTP/1.1}{0.0.0.0:4567}"
#> [18] "2017-01-18 10:29:11.658:INFO:osjs.Server:main: Started @285ms"
#> [19] "10:29:11.658 INFO - Selenium Server is up and running"
#> [20] "10:29:12.295 INFO - SessionCleaner initialized with insideBrowserTimeout 0 and clientGoneTimeout 1800000 polling every 180000"
#> [21] "10:29:12.319 INFO - Executing: [new session: Capabilities [{nativeEvents=true, browserName=chrome, javascriptEnabled=true, version=, platform=ANY}]])"
#> [22] "10:29:12.331 INFO - Creating a new session for Capabilities [{nativeEvents=true, browserName=chrome, javascriptEnabled=true, version=, platform=ANY}]"
#> [23] "Starting ChromeDriver 2.27.440175 (9bc1d90b8bfa4dd181fbbf769a5eb5e575574320) on port 12090"
#> [24] "Only local connections are allowed."
#> [25] "10:29:12.556 INFO - Attempting bi-dialect session, assuming Postel's Law holds true on the remote end"
#> [26] "10:29:13.115 INFO - Detected dialect: OSS"
#> [27] "10:29:13.144 INFO - Done: [new session: Capabilities [{nativeEvents=true, browserName=chrome, javascriptEnabled=true, version=, platform=ANY}]]"
#> [28] "10:29:13.172 INFO - Executing: [get: http://www.r-project.org])"
#> [29] "10:29:14.340 INFO - Done: [get: http://www.r-project.org]"
#> [30] "10:29:14.356 INFO - Executing: [get title])"
#> [31] "10:29:14.363 INFO - Done: [get title]"
#>
#> $stdout
#> character(0)
# close client/server
remDr$close()
rD$server$stop()
#> [1] TRUE
# check server
rD$server$process
#> Process Handle
#> command : /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Dwebdriver.chrome.driver=/home/john/.local/share/binman_chromedriver/linux64/2.27/chromedriver -Dwebdriver.gecko.driver=/home/john/.local/share/binman_geckodriver/linux64/0.13.0/geckodriver -Dphantomjs.binary.path=/home/john/.local/share/binman_phantomjs/linux64/2.1.1/phantomjs-2.1.1-linux-x86_64/bin/phantomjs -jar /home/john/.local/share/binman_seleniumserver/generic/3.0.1/selenium-server-standalone-3.0.1.jar -port 4567
#> system id : 120670
#> state : terminated
For further detail and any issues you may have with the rsDriver function please see the wdman project.
choisy/gso documentation built on Aug. 22, 2019, 10:32 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
library(knitr) library(RSelenium) opts_chunk$set(comment = "#>", error = TRUE, tidy = TRUE)
Introduction
The goal of RSelenium is to make it easy to connect to a Selenium Server/ Remote Selenium Server from within R. RSelenium provides R bindings for the Selenium Webdriver API. Selenium is a project focused on automating web browsers. RSelenium allows you to carry out unit testing and regression testing on your webapps and webpages across a range of browser/OS combinations. This allows us to integrate from within R testing and manipulation of popular projects such as shiny, sauceLabs.
This vignette is divided into six main sections:
Each section will be an introduction to a major idea in Selenium, and point to more detailed explanation in other vignettes.
Connecting to a Selenium Server.
What is a Selenium Server?
Selenium Server is a standalone java program which allows you to run HTML test suites in a range of different browsers, plus extra options like reporting. You may, or may not, need to run a Selenium Server, depending on how you intend to use Selenium-WebDriver (RSelenium).
Do I need to run a Selenium Server?
If you intend to drive a browser on the same machine that RSelenium is running on then you will need to have Selenium Server running on that machine.
How do I get the Selenium Server stand-alone binary?
You can download the latest Selenium Server binary manually it is currently found here. Look for selenium-server-standalone-x.xx.x.jar.
How do I run the Selenium Server?
The recommended way to run a Selenium Server is by running a Docker
container. There is a seperate vignette which covers the using RSelenium
with Docker see vignette("RSelenium-docker", package = "RSelenium").
For users who are not familiar with Docker there is now a function
rsDriver which will manage the binaries needed for running a Selenium
Server. This provides a wrapper around the wdman::selenium function.
For additional options and more control see the wdman project.
Examples using the rsDriver function are given in the appendix.
Please submit any issues with running binaries to the wdman
project
Alternatively you can run the binary manually. Open a console in your OS and navigate to where the binary is located and run:
java -jar selenium-server-standalone-x.xx.x.jar
By default the Selenium Server listens for connections on port 4444.
Note for Mac OSX: The default port 4444 is sometimes used by other programs such as kerberos. In our examples we use port 4445 in respect of this and for cdonsistency with the Docker vignette.
How do I connect to a running server?
RSelenium has a main reference class named remoteDriver. To connect to a server you need to instantiate a new remoteDriver with appropriate options.
# Run a server for example using Docker # docker run -d -p 4445:4444 selenium/standalone-firefox:2.53.1 # Use a debug image with a VNC viewer if you wish to view the browser # docker run -d -p 5901:5900 -p 127.0.0.1:4445:4444 --link http-server selenium/standalone-firefox-debug:2.53.1 # See Docker vignette for more detail or run a Selenium Server manually. require(RSelenium) remDr <- remoteDriver(remoteServerAddr = "localhost" , port = 4445L , browserName = "firefox" )
Note for Windows: If you are using Docker toolbox your remote server address will not be localhost. You need to use the ip address of the VM that is running docker for example
$ docker-machine ip
192.168.99.100
See the Docker vignette for further details. The newer Docker for windows however should be accessible on the localhost.
It would have been sufficient to call remDr <- remoteDriver(port = 4445L) but the options where explicitly listed to show how one may connect to an arbitrary ip/port/browser etc. More detail maybe found on the sauceLabs vignette. To connect to the server use the open method.
remDr$open()
RSelenium should now have a connection to the Selenium Server. You can query the status of the remote Server using the status method.
> remDr$getStatus() $build $build$version [1] "2.53.1" $build$revision [1] "a36b8b1" $build$time [1] "2016-06-30 17:37:03" $os $os$name [1] "Linux" $os$arch [1] "amd64" $os$version [1] "4.4.0-47-generic" $java $java$version [1] "1.8.0_91"
Navigating using RSelenium.
Basic Navigation
To start with we navigate to a url.
remDr$navigate("http://www.google.com/ncr")
Then we navigate to a second page.
remDr$navigate("http://www.bbc.co.uk")
> remDr$getCurrentUrl()
[[1]]
[1] "http://www.bbc.co.uk/"
We can go backwards and forwards using the methods goBack and goForward.
remDr$goBack() > remDr$getCurrentUrl() [[1]] [1] "https://www.google.com/" remDr$goForward() > remDr$getCurrentUrl() [[1]] [1] "http://www.bbc.co.uk/"
To refresh the current page you can use the `refresh method.
remDr$refresh()
Accessing elements in the DOM.
The DOM stands for the Document Object Model. It is a cross-platform and language-independent convention for representing and interacting with objects in HTML, XHTML and XML documents. Interacting with the DOM will be very important for us with Selenium and the webDriver provides a number of methods in which to do this. A basic html page is
<!DOCTYPE html> <html> <body> <h1>My First Heading</h1> <p>My first paragraph.</p> </body> </html>
The query box on the front page of http://www.google.com has html code
<input id=..... name="q" ...</input> associated with it. The full html associated with the input tag is:
<input spellcheck="false" dir="ltr" style="border: medium none; padding: 0px; margin: 0px; height: auto; width: 100%; background: transparent url("data:image/gif;base64,R0lGODlhAQABAID/AMDAwAAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw%3D%3D") repeat scroll 0% 0%; position: absolute; z-index: 6; left: 0px; outline: medium none;" aria-autocomplete="both" role="combobox" aria-haspopup="false" class="gsfi" id="lst-ib" maxlength="2048" name="q" autocomplete="off" title="Search" value="" aria-label="Search" type="text">
NOTE: The above HTML is very liable to change however the input node has had an attribute name = q for sometime so we can mostly rely on this.
Search by name.
To find this element in the DOM a number of methods can be used. We can search by the name.
remDr$navigate("http://www.google.com/ncr") webElem <- remDr$findElement(using = 'name', value = "q") > webElem$getElementAttribute("name") [[1]] [1] "q" > webElem$getElementAttribute("class") [[1]] [1] "gsfi lst-d-f" > webElem$getElementAttribute("id") [[1]] [1] "lst-ib"
Search by id
In HTML the id of a DOM element should be unique so this is usually a good locator to use. As noted above the google id of the query box may change (the one we see is "lst-ib" so we use that). You may see an alternative "id".
remDr$navigate("http://www.google.com/ncr") webElem <- remDr$findElement(using = 'id', value = "lst-ib")
Highlight an element
An element that is visible in the DOM can usually be highlighted. Using
our webElem which is an object of class "webElement" we can use the
associated highlight method to visually indicate to us we have the
correct element. Try it:
webElem$highlightElement()
You should see the query box flashing black and yellow to indicate visually the DOM element you have selected.
Search by class.
We can also search by class name.
remDr$navigate("http://www.google.com/ncr") webElem <- remDr$findElement(using = 'class', "gsfi") > webElem$getElementAttribute("class") [[1]] [1] "gsfi lst-d-f" > webElem$getElementAttribute("type") [[1]] [1] "text"
NOTE the clas is listed as "gsfi lst-d-f" and we searched using "gsfi"? This is an example of a compound class. The class name selector can only be used for elements with a single class ("gsfi lst-d-f" indicates two classes "gsfi" and "lst-d-f"). For more complicated select queries we therefore use CSS or xpath instead.
Search using css-selectors
To replicate our name search using css selectors we could use
webElem <- remDr$findElement(using = 'css', "input[name='q']") # OR webElem2 <- remDr$findElement(using = 'css', "[name='q']")
We can see we get the same element (using the compareElements method of
the "webElement" class):
> webElem$compareElements(webElem2) [[1]] [1] TRUE
and to search on id using css-selectors (again not the id you see maybe different)
webElem <- remDr$findElement(using = 'css', "input#lst-ib") > webElem$getElementAttribute("name") [[1]] [1] "q"
and class
webElem <- remDr$findElement('css', "[class = 'gsfi lst-d-f']")
NOTE no issue with compound class names with CSS
A good example of searching using css-selectors is given here.
Search using xpath
The final option is to search using xpath. Normally one would use xpath by default when searching or CSS. Both are the go-to options.
Xpath using id.
webElem <- remDr$findElement('xpath', "//input[@id = 'lst-ib']")
Xpath using class.
webElem <- remDr$findElement('xpath', "//input[@class = 'gsfi lst-d-f']")
NOTE with xpath we have no issues using a compound class name.
Sending events to elements.
To illustrate how to interact with elements we will again use the http://www.google.com/ncr as an example.
Sending text to elements
Suppose we would like to search for R cran on google. We would need to find the element for the query box and send the appropriate text to it. We can do this using the sendKeysToElement method for the webElement class.
remDr$navigate("http://www.google.com/ncr") webElem <- remDr$findElement(using = "css", "[name = 'q']") webElem$sendKeysToElement(list("R Cran"))
Sending key presses to elements
We should see that the text R Cran has now been entered into the query box.
How do we press enter. We can simply send the enter key to query box. The enter key would be denoted as "\uE007"(its UTF-8 code) . So we could use:
remDr$navigate("http://www.google.com/ncr") webElem <- remDr$findElement(using = "css", "[name = 'q']") webElem$sendKeysToElement(list("R Cran", "\uE007"))
It is not very easy to remember UTF-8 codes for appropriate keys so a mapping has been provided in RSelenium. `?selkeys' will bring up a help page explaining the mapping. The UTF-8 codes have been mapped to easy to remember names.
To use selkeys we would send the following
remDr$navigate("http://www.google.com/ncr") webElem <- remDr$findElement(using = "css", "[name = 'q']") webElem$sendKeysToElement(list("R Cran", key = "enter"))
Typing selKeys into the console will bring up the list of mappings.
Sending mouse events to elements
For this example we will go back to the google frontpage and search for
R Cran then we will click the link for the The Comprehensive R Archive Network.
remDr$navigate("http://www.google.com/ncr") webElem <- remDr$findElement(using = "css", "[name = 'q']") webElem$sendKeysToElement(list("R Cran", key = "enter"))
The header for each link is contained in a <h3 class = "r"> tag. We will access the h3 headers first. It will be succinct to find these elements using css selectors.
webElems <- remDr$findElements(using = 'css selector', "h3.r") resHeaders <- unlist(lapply(webElems, function(x){x$getElementText()})) > resHeaders [1] "The Comprehensive R Archive Network" [2] "Download R-3.3.2 for Windows. The R-project for statistical ... - CRAN" [3] "About Microsoft R Open: The Enhanced R Distribution . MRAN" [4] "R (programming language) - Wikipedia" [5] "R-Cran - StatLib - Carnegie Mellon University" [6] "Submitting your first package to CRAN, my experience | R-bloggers" [7] "Debian -- Package Search Results -- r-cran" [8] "It's crantastic!" [9] "METACRAN" [10] "CRAN - Package PopGenReport"
NOTE this is how the headers were presented at time of writing. Class names etc. are liable to change. We can see that the first link is the one we want but in case googles search results change we refer to it as
webElem <- webElems[[which(resHeaders == "The Comprehensive R Archive Network")]]
How do we click the link. We can use the clickElement method:
webElem$clickElement() > remDr$getCurrentUrl() [[1]] [1] "https://cran.r-project.org/" > remDr$getTitle() [[1]] [1] "The Comprehensive R Archive Network"
Injecting JavaScript
Sometimes it is necessary to interact with the current url using JavaScript. This maybe necessary to call bespoke methods or to have more control over the page for example by adding the JQuery library to the page if it is missing. Selenium has two methods we can use to execute JavaScript namely
executeScript and executeAsyncScript from the remoteDriver class. We return to the google front page to investigate these methods.
Injecting JavaScript synchronously
Returning to the google homepage we can find the element for the google image. The image has id = "hplogo" and
we can use this in an xpath or search by id etc to select the element. In this case we use css selectors:
remDr$navigate("http://www.google.com/ncr") webElem <- remDr$findElement("css", "img#hplogo")
Is the image visible? Clearly it is but we can check using javascript.
> script <- "return document.getElementById('hplogo').hidden;" > remDr$executeScript(script, args = list()) [[1]] [1] FALSE
Great so the image is not hidden indicated by the FALSE. We can hide it executing some simple JavaScript.
> script <- "document.getElementById('hplogo').hidden = true; return document.getElementById('hplogo').hidden;" > remDr$executeScript(script, args = list()) [[1]] [1] TRUE
So now the image is hidden. We used an element here given by id = "hplogo". We had to use the JavaScript function
getElementById to access it. It would be nicer if we could have used webElem which we had specified earlier.
If we pass a webElement object as an argument to either executeScript or executeAsyncScript RSelenium will pass it in an appropriate fashion.
> script <-"arguments[0].hidden = false; return arguments[0].hidden;" > remDr$executeScript(script, args = list(webElem)) [[1]] [1] FALSE
Notive how we passed the web element to the method executeScript. The script argument defines the script to execute in the form of a function body. The value returned by that function will be returned to the client. The function will be invoked with the provided args. If the function returns an element then this will be returned as an object of class webElement:
script <- "return document.getElementsByName('q');" test <- remDr$executeScript(script, args = list()) > test[[1]] [1] "remoteDriver fields" $remoteServerAddr [1] "localhost" $port [1] 4445 $browserName [1] "firefox" $version [1] "" $platform [1] "ANY" $javascript [1] TRUE $nativeEvents [1] TRUE $extraCapabilities list() [1] "webElement fields" $elementId [1] "21" > class(test[[1]]) [1] "webElement" attr(,"package") [1] "RSelenium"
Try to highlight the element as before:
test[[1]]$highlightElement()
Injecting JavaScript asynchronously
I will briefly touch on asynch versus sync calls here. We need to set an appropriate asynchronous timeout firstly (that is longer than the async operation we are likely to carryout but it will ensure we will eventually error out in case of an issue)
remDr$navigate("http://www.google.com/ncr")
remDr$setAsyncScriptTimeout(10000)
Observe:
webElem <- remDr$findElement("css", "img#hplogo") script <- " cb = arguments[arguments.length -1]; webElem = arguments[0]; setTimeout(function(){webElem.hidden = true; cb('DONE');},5000);" remDr$executeAsyncScript(script, args = list(webElem))
versus
remDr$navigate("http://www.google.com/ncr") webElem <- remDr$findElement("css", "img#hplogo") script <- " webElem = arguments[0]; setTimeout(function(){webElem.hidden = true;},5000); return 'DONE'; " remDr$executeScript(script, args = list(webElem))
The async version waits until the callback (defined as the last argument
arguments[arguments.length -1] as JavaScript is zero-indexed)
is called whereas the sync version returns straight away. In both cases
the google logo disappears after five seconds.
Frames and Windows.
In the context of a web browser, a frame is a part of a web page or browser window which displays content independent of its container, with the ability to load content independently.
Frames in Selenium
We will demonstrate interacting with frames by way of example. The The Comprehensive R Archive Network conveniently
contains frames so we shall use RSelenium to interact with it.
Assume we have a remoteDriver opened.
remDr$navigate("https://CRAN.r-project.org/")
> XML::htmlParse(remDr$getPageSource()[[1]])
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Frameset//EN" "http://www.w3.org/TR/html4/frameset.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>The Comprehensive R Archive Network</title>
<meta content="text/html; charset=utf-8" http-equiv="content-type">
<link type="image/x-icon" href="favicon.ico" rel="icon">
<link type="image/x-icon" href="favicon.ico" rel="shortcut icon">
<link href="R.css" type="text/css" rel="stylesheet">
</head>
<frameset style="border: none;" cols="1*, 4*">
<frameset rows="120, 1*">
<frame frameborder="0" name="logo" src="logo.html">
<frame frameborder="0" name="contents" src="navbar.html">
</frameset>
<frame frameborder="0" name="banner" src="banner.shtml">
<noframes>
<h1>The Comprehensive R Archive Network</h1>
Your browser seems not to support frames,
here is the <A href="navbar.html">contents page</A> of CRAN.
</noframes>
</frameset>
</html>
We can see the content is contained in three frames and we dont appear to have access to the content within a frame. But in the browser we see all the content:
remDr$maxWindowSize() remDr$screenshot(display = TRUE)
RProject front page
To access the content we have to switch to a frame using the switchToFrame method of the remoteDriver class.
webElems <- remDr$findElements(using = "tag name", "frame") # webElems <- remDr$findElements(value = "//frame") # using xpath # webElems <- remDr$findElements("css", value = "frame") # using css > sapply(webElems, function(x){x$getElementAttribute("src")}) [[1]] [1] "https://cran.r-project.org/logo.html" [[2]] [1] "https://cran.r-project.org/navbar.html" [[3]] [1] "https://cran.r-project.org/banner.shtml" remDr$switchToFrame(webElems[[2]]) > XML::htmlParse(remDr$getPageSource()[[1]]) <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <title>R Contents</title> <meta content="text/html; charset=utf-8" http-equiv="content-type"> <link href="R.css" type="text/css" rel="stylesheet"> </head> <body> <em class="navigation">CRAN</em><br><a target="banner" href="mirrors.html">Mirrors</a><br><a target="banner" href="//www.R-project.org/news.html">What's new?</a><br><a target="banner" href="web/views/">Task Views</a><br><a target="banner" href="search.html">Search</a><br><!--<a href="pkg_submit.html" target="_top">Submit</a><BR>--><p> <em class="navigation">About R</em><br><a target="_top" href="//www.R-project.org/">R Homepage</a><br><a target="_top" href="http://journal.R-project.org/">The R Journal</a> </p> <p> <em class="navigation">Software</em><br><a target="banner" href="sources.html">R Sources</a><br><a target="banner" href="bin/">R Binaries</a><br><a target="banner" href="web/packages/">Packages</a><br><a target="banner" href="other-software.html">Other</a> </p> <p> <em class="navigation">Documentation</em><br><a target="banner" href="manuals.html">Manuals</a><br><a target="banner" href="faqs.html">FAQs</a><br><a target="banner" href="other-docs.html">Contributed</a><br></p> </body> </html>
Now we see the source code of the navigation sidePanel. Notice how we used a webElement in the method switchToFrame. To further demonstrate we are now "in" this frame lets get all the href attributes:
webElems <- remDr$findElements("css", "[href]") sapply(webElems, function(x){x$getElementAttribute("href")}) > unlist(sapply(webElems, function(x){x$getElementAttribute("href")})) [1] "https://cran.r-project.org/R.css" [2] "https://cran.r-project.org/mirrors.html" [3] "https://www.r-project.org/news.html" [4] "https://cran.r-project.org/web/views/" [5] "https://cran.r-project.org/search.html" [6] "https://www.r-project.org/" [7] "http://journal.r-project.org/" [8] "https://cran.r-project.org/sources.html" [9] "https://cran.r-project.org/bin/" [10] "https://cran.r-project.org/web/packages/" [11] "https://cran.r-project.org/other-software.html" [12] "https://cran.r-project.org/manuals.html" [13] "https://cran.r-project.org/faqs.html" [14] "https://cran.r-project.org/other-docs.html"
Notice if we pass a NULL value to the method switchToFrame we move back to the default view.
remDr$switchToFrame(NULL) > XML::htmlParse(remDr$getPageSource()[[1]]) <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Frameset//EN" "http://www.w3.org/TR/html4/frameset.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <title>The Comprehensive R Archive Network</title> <meta content="text/html; charset=utf-8" http-equiv="content-type"> <link type="image/x-icon" href="favicon.ico" rel="icon"> <link type="image/x-icon" href="favicon.ico" rel="shortcut icon"> <link href="R.css" type="text/css" rel="stylesheet"> </head> <frameset style="border: none;" cols="1*, 4*"> <frameset rows="120, 1*"> <frame frameborder="0" name="logo" src="logo.html"> <frame frameborder="0" name="contents" src="navbar.html"> </frameset> <frame frameborder="0" name="banner" src="banner.shtml"> <noframes> <h1>The Comprehensive R Archive Network</h1> Your browser seems not to support frames, here is the <A href="navbar.html">contents page</A> of CRAN. </noframes> </frameset> <body><canvas id="fxdriver-screenshot-canvas" style="display: none;" width="1360" height="559"></canvas></body> </html>
Finally we can switch to the main panel using a name
remDr$switchToFrame("banner")
> XML::htmlParse(remDr$getPageSource()[[1]])
<!DOCTYPE html PUBLIC "-//IETF//DTD HTML//EN">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>The Comprehensive R Archive Network</title>
<link href="R.css" type="text/css" rel="stylesheet">
</head>
<body>
<h1>The Comprehensive R Archive Network</h1>
<div align="center">
<table border="1" width="80%"><tbody>
<tr>
<td>
<h3>Download and Install R</h3>
Precompiled binary distributions of the base system and
contributed packages, <strong>Windows and
Mac</strong> users most likely want one of these versions of R:
<ul>
<li><a href="bin/linux/">Download R for Linux</a></li>
<li>
<a href="bin/macosx/">Download R for (Mac) OS X</a>
</li>
<li><a href="bin/windows/">Download R for Windows</a></li>
</ul>
R is part of many Linux distributions, you should check with your Linux package management system in addition to the link above.
</td>
</tr>
<tr>
<td>
................
................
................
<h3>Questions About R</h3>
<ul><li>
If you have questions about R like how to download and install
the software, or what the license terms are,
please read our <a href="faqs.html">answers to frequently asked
questions</a> before you send an email.
</li></ul>
</td>
</tr>
</tbody></table>
</div>
<h2>What are R and CRAN?</h2>
<p> R is GNU, a freely available language and environment for
statistical computing and graphics which provides a wide variety of
statistical and graphical techniques: linear and nonlinear modelling,
statistical tests, time series analysis, classification, clustering,
etc. Please consult the <a target="_top" href="https://www.R-project.org/">R project homepage</a> for further information.
</p>
<p> CRAN is a network of ftp and web servers around the world that
store identical, up-to-date, versions of code and documentation for
R. Please use the CRAN <a href="mirrors.html">mirror</a> nearest to you to minimize network
load.
</p>
<h2 id="submitting">Submitting to CRAN </h2>
<p>
To submit a package to CRAN,
check that your submission meets the
<a href="https://CRAN.R-project.org/web/packages/policies.html">CRAN
Repository Policy</a> and then use the
<a href="https://xmpalantir.wu.ac.at/cransubmit/">web form</a>.
</p>
<p>
If this fails, upload to
<a target="_blank" href="ftp://CRAN.R-project.org/incoming/">ftp://CRAN.R-project.org/incoming/</a>
and send an email to
<a href="mailto:CRAN@R-project.org">CRAN@R-project.org</a> following the policy.
Please do not attach submissions to emails, because this will clutter up
the mailboxes of half a dozen people.
</p>
<p>
Note that we generally do not accept submissions of precompiled
binaries due to security reasons. All binary distribution listed above
are compiled by selected maintainers, who are in charge for all
binaries of their platform, respectively.
</p>
<p>
</p>
<hr>
<!--#if expr="$CRAN_HOST" --><!--#echo encoding="none" var="CRAN_HOST"--><br><!--#endif -->
</body>
</html>
Windows in Selenium
The easiest way to illustrate Windows in RSelenium is again by way of example. We will use the "CRAN" website.
First we select the download R element in the main frame.
remDr$navigate("https://cran.r-project.org/") remDr$switchToFrame("banner") webElems <- remDr$findElements("partial link text", "Download R") sapply(webElems, function(x) x$getElementText()) [[1]] [1] "Download R for Linux" [[2]] [1] "Download R for (Mac) OS X" [[3]] [1] "Download R for Windows"
We now send a selection of key presses to the first element to open the link it points to in a new window. If you did it manually you would move the mouse to the element right click on the link press the down arrow key twice then press enter. We will do the same
loc <- webElems[[1]]$getElementLocation() > loc[c('x','y')] $x [1] 158 $y [1] 132 remDr$mouseMoveToLocation(webElement = webElems[[1]]) # move mouse to the element we selected remDr$click(2) # 2 indicates click the right mouse button remDr$sendKeysToActiveElement( list(key = 'down_arrow', key = 'down_arrow', key = 'enter') )
Notice now that a new windows has opened on the remote browser.
> remDr$getCurrentWindowHandle() [[1]] [1] "{573d17e5-b95a-41b9-a65f-04092b6a804b}" > remDr$getWindowHandles() [[1]] [1] "{4896393a-c215-4976-b4ca-030d6b75b67d}" [[2]] [1] "{69c00f18-d3a7-44d7-a236-c6b5e6c264ff}" > remDr$getTitle() [[1]] [1] "The Comprehensive R Archive Network" > currWin <- remDr$getCurrentWindowHandle() > allWins <- unlist(remDr$getWindowHandles()) > otherWindow <- allWins[!allWins %in% currWin[[1]]] > remDr$switchToWindow(otherWindow) > remDr$getTitle() [[1]] [1] "Index of /bin/linux"
So using the code above one can observe how to switch between different windows on the remote browser.
Appendix
Server functions
In previous versions of RSelenium there were two functions
checkForServer and startServer which were used to download and start a
Selenium binary. These functions are Deprecated/Defunct as users had issue
with using them cross-platform. They are still in the RSelenium package and can be accessed in the serverUtils directory file.path(find.package("RSelenium"), "examples/serverUtils")
> file.path(find.package("RSelenium"), "examples/serverUtils")
[1] "/home/john/R/x86_64-pc-linux-gnu-library/3.3/RSelenium/examples/serverUtils"
They may work for you depending on your platform and setup but are not supported.
rsDriver
The rsDriver function is a wrapper for the selenium function from the
wdman package. It allows the
user to manage the binaries used to run a Selenium Server. It returns
an environment containing a "client" and a "server".
By default it runs a chrome browser. Other browsers (firefox, PhantomJS,
Internet Explorer (Win)) can be selected using the browser argument.
The default port a Selenium Server is run on using the rsDriver function
is port = 4567L.
rD <- rsDriver(verbose = FALSE) rD #> $client #> browserName id #> 1 chrome 1670dcc6-3c97-4717-bf28-4d1fc8eea2c1 #> #> $server #> Process Handle #> command : /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Dwebdriver.chrome.driver=/home/john/.local/share/binman_chromedriver/linux64/2.27/chromedriver -Dwebdriver.gecko.driver=/home/john/.local/share/binman_geckodriver/linux64/0.13.0/geckodriver -Dphantomjs.binary.path=/home/john/.local/share/binman_phantomjs/linux64/2.1.1/phantomjs-2.1.1-linux-x86_64/bin/phantomjs -jar /home/john/.local/share/binman_seleniumserver/generic/3.0.1/selenium-server-standalone-3.0.1.jar -port 4567 #> system id : 120670 #> state : running
# assign the client to a new variable remDr <- rD$client remDr$navigate("http://www.r-project.org") remDr$getTitle() #> [[1]] #> [1] "R: The R Project for Statistical Computing"
# check the server logs rD$server$log() #> $stderr #> [1] "10:29:11.536 INFO - Selenium build info: version: '3.0.1', revision: '1969d75'" #> [2] "10:29:11.537 INFO - Launching a standalone Selenium Server" #> [3] "2017-01-18 10:29:11.551:INFO::main: Logging initialized @179ms" #> [4] "10:29:11.594 INFO - Driver provider org.openqa.selenium.ie.InternetExplorerDriver registration is skipped:" #> [5] " registration capabilities Capabilities [{ensureCleanSession=true, browserName=internet explorer, version=, platform=WINDOWS}] does not match the current platform LINUX" #> [6] "10:29:11.595 INFO - Driver provider org.openqa.selenium.edge.EdgeDriver registration is skipped:" #> [7] " registration capabilities Capabilities [{browserName=MicrosoftEdge, version=, platform=WINDOWS}] does not match the current platform LINUX" #> [8] "10:29:11.595 INFO - Driver class not found: com.opera.core.systems.OperaDriver" #> [9] "10:29:11.595 INFO - Driver provider com.opera.core.systems.OperaDriver registration is skipped:" #> [10] "Unable to create new instances on this machine." #> [11] "10:29:11.595 INFO - Driver class not found: com.opera.core.systems.OperaDriver" #> [12] "10:29:11.595 INFO - Driver provider com.opera.core.systems.OperaDriver is not registered" #> [13] "10:29:11.596 INFO - Driver provider org.openqa.selenium.safari.SafariDriver registration is skipped:" #> [14] " registration capabilities Capabilities [{browserName=safari, version=, platform=MAC}] does not match the current platform LINUX" #> [15] "2017-01-18 10:29:11.628:INFO:osjs.Server:main: jetty-9.2.15.v20160210" #> [16] "2017-01-18 10:29:11.647:INFO:osjsh.ContextHandler:main: Started o.s.j.s.ServletContextHandler@2ef5e5e3{/,null,AVAILABLE}" #> [17] "2017-01-18 10:29:11.658:INFO:osjs.ServerConnector:main: Started ServerConnector@724af044{HTTP/1.1}{0.0.0.0:4567}" #> [18] "2017-01-18 10:29:11.658:INFO:osjs.Server:main: Started @285ms" #> [19] "10:29:11.658 INFO - Selenium Server is up and running" #> [20] "10:29:12.295 INFO - SessionCleaner initialized with insideBrowserTimeout 0 and clientGoneTimeout 1800000 polling every 180000" #> [21] "10:29:12.319 INFO - Executing: [new session: Capabilities [{nativeEvents=true, browserName=chrome, javascriptEnabled=true, version=, platform=ANY}]])" #> [22] "10:29:12.331 INFO - Creating a new session for Capabilities [{nativeEvents=true, browserName=chrome, javascriptEnabled=true, version=, platform=ANY}]" #> [23] "Starting ChromeDriver 2.27.440175 (9bc1d90b8bfa4dd181fbbf769a5eb5e575574320) on port 12090" #> [24] "Only local connections are allowed." #> [25] "10:29:12.556 INFO - Attempting bi-dialect session, assuming Postel's Law holds true on the remote end" #> [26] "10:29:13.115 INFO - Detected dialect: OSS" #> [27] "10:29:13.144 INFO - Done: [new session: Capabilities [{nativeEvents=true, browserName=chrome, javascriptEnabled=true, version=, platform=ANY}]]" #> [28] "10:29:13.172 INFO - Executing: [get: http://www.r-project.org])" #> [29] "10:29:14.340 INFO - Done: [get: http://www.r-project.org]" #> [30] "10:29:14.356 INFO - Executing: [get title])" #> [31] "10:29:14.363 INFO - Done: [get title]" #> #> $stdout #> character(0)
# close client/server remDr$close() rD$server$stop() #> [1] TRUE # check server rD$server$process #> Process Handle #> command : /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Dwebdriver.chrome.driver=/home/john/.local/share/binman_chromedriver/linux64/2.27/chromedriver -Dwebdriver.gecko.driver=/home/john/.local/share/binman_geckodriver/linux64/0.13.0/geckodriver -Dphantomjs.binary.path=/home/john/.local/share/binman_phantomjs/linux64/2.1.1/phantomjs-2.1.1-linux-x86_64/bin/phantomjs -jar /home/john/.local/share/binman_seleniumserver/generic/3.0.1/selenium-server-standalone-3.0.1.jar -port 4567 #> system id : 120670 #> state : terminated
For further detail and any issues you may have with the rsDriver function please see the wdman project.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.