In garrettgman/shiny-js-tutorials: javascript widget examples for shiny web apps

Introduction

Shiny is a web application framework that makes it easy to build interactive web applications (apps) straight from R. A key feature of shiny is that users can create powerful apps driven by R without having to know HTML, CSS, or JavaScript. However, incorporating these techniques into your apps can greatly enhance the power of a shiny app.

Furthermore, in recent years there has been a shift from using base R graphics to using interactive JavaScript web components for data analysis and data visualization. In order to use the functionality offered in these frameworks we have to construct R bindings to JavaScript libraries.

The htmlwidgets package provides a framework for creating such bindings and allows you to bring the best of JavaScript into R using a set of easy-to-follow conventions. After a widget has been constructed, embedding it into a shiny app or R Markdown document only takes a few lines of code. For some examples on what htmlwidgets can do take a look at the showcase on the main website.

Even though using an htmlwidget is easy, constructing one may be challenging at first as you have to have knowledge of a number of concepts in order to create a widget. These include things as building an R package and having a basic knowledge of javascript, css and html. However, you don't need to be an expert in these techniques to create quite useful new functionality.

The tutorials below are intended for the intermediate shiny enthusiast with limited experience in html, css and/or javascript, who wants to learn how to extend shiny. A useful first step in this process is to be able to create your own htmlwidgets.

Creating a dashboard app

The widgets we are going to build are all based on c3.js, which in turn is built on the more extensive JavaScript visualization library d3.js. C3.js provides a variety of chart types e.g. gauges, pie charts, stacked bar charts, stacked area charts as well as charts to display time series. Furthermore, c3.js provides a variety of APIs and callbacks to access and update the state of a chart after it's rendered.

In order to master the technique of creating widgets we will construct an interactive dashboard application which includes a variety of htmlwidgets based on c3.js. A screenshot of the end result can be seen below. A live version of this dashboard can be seen here, while all the code can be found here.

the end result of the tutorials

During the course of the tutorials we gradually build up complexity in the functionality the widgets offer. The end result is a fully functional dashboard. An advantage of using JavaScript over base R graphics is that we have access to various types of [events](http://www.w3schools.com/js/js_events.asp) e.g. hover, click and drag events. For instance, the dashboard has a nice brush-able timeline component.

a brush-able timeline which acts as a data filter

The timeline acts as a time based filter for all the data in the other charts. In the tutorials we take a deeper look into the notion of sending data from the client to the server and back using shiny. This functionality is mainly based around three pivotal functions i.e. the JavaScript functions `Shiny.onInputChange`, `Shiny.addCustomMessageHandler` and the R Shiny function `sendCustomMessage`. Each of these will be discussed in detail. A great introduction on these functions is offered in [this](https://ryouready.wordpress.com/2013/11/20/sending-data-from-client-to-server-and-back-using-shiny/) blog post.

Dashboard data

The data in the dashboard represents data from an insurance company that screens persons which apply for a new insurance during underwriting. Underwriting is the process in which an insurance company assesses whether or not it should accept a person into their portfolio. If the risk for specific type of claims is deemed too high, the insurer may decide to reject an application. For the purposes of the tutorials we use a toy dataset of 20,000 rows which looks like this:

load("data.RData")
library(dplyr)
dd           <- DD.Signaleringen[,c("signaleringid","detectiedatum","eindresultaat","eindargumentatie","branche","product","proces","label")]
names(dd)    <- c("id","date","score","result","branch","product","process","label")
dd$id        <- 1:nrow(dd)
rownames(dd) <- NULL
dd$score[1:6] <- c(0,60,150,25,10,5)
head(dd)

Each row represents a new screening. The column id provides a unique case, while the column date indicates the date at which a person was screened. The column score indicates the risk score estimated by the insurance company. Higher scores indicate the person has a higher estimated probability to file a claim or to commit fraud somewhere in the future. The column result refers to a discretized version of the score column. For our dashboard this is the most important variable. Scores between 0 and 50 are mapped to GREEN, while scores from 51 to 75 are mapped to AMBER. Scores above 100 are mapped to RED, which indicates the highest risk group. The final four columns indicate the branch, product, process, and label associated with the policy application. The exact meaning of these categories depends on the insurer. There main purpose is to help the insurer to process new policy applications more quickly.

In the screenshot above the 4 gauges indicated the percentage of RED cases for a specific time period. The percentages for different risk groups can be selected by the dropdown menu on the left. The 4 pie charts indicate the distibution of the screenings over the various processess, labels, product and branches. The bar + line chart is an example of a c3.js combination chart with dual y-axes and a single x-axis indicating time. The gray bars indicate the total number of screenings for each week (left y-axis), while the green, amber and red lines correspond to the percentage of cases which are estimated as GREEN, AMBER or RED, respectively (right y-axis). Finally, the bottom chart shows the same information, but this time as a stacked area chart. The toggle button on the left allows you to toggle between displaying percentages or absolute counts.

For each chart type we'll create a separate htmlwidget, while for the toggle button we'll make a shiny input binding (see below).

Creating a dynamic help system

In addition to creating widgets, you'll learn how to create an interactive, dynamic help system, complete with animated transitions. The help system is based on another JavaScript library namely intro.js. This library allows you to create a step-by-step guide for a website. It will draw a nice box around elements of your choice, combined with an annotation layer and a navigation system. Here's an example of how the help will look for one of the gauges in the dashboard.

the help system

Creating your own input binding

We end the tutorials with the creation of a custom input binding. Input bindings are components that capture events from the client i.e. the webpage, and send it to Shiny. A simple example of an input binding is an action button. It captures a click event from the user and sends it to shiny. Here we'll create a nice looking toggle switch, based on the bootstrap-switch JavaScript library. We'll use the switch to toggle a c3.js stacked area chart to display either absolute counts or percentages. Here's an example of the switch we will be creating. Press the button and see what happens!

an example of a custom input binding in the form of a toggle switch

Stand alone examples

We will start with the most simple visualization c3.js offers i.e. a gauge. Before involving R and shiny we will make a simple stand alone JavaScript HTML version to show how the component and c3.js work in general.

Each stand alone example is presented in a jsfiddle, which is a webpage where you can interactively run HTML, CSS and JavaScript code. By clicking "Edit in JSFiddle" in the panel below you can alter the code and see what the effect of your change is. Working with fiddles is a great way to learn how a specific library or piece of code works. For example, by changing the value 75 on the first line of the JavaScript panel and clicking on the "Run" button in the left corner you will see the value of the gauge change.

Recipe to construct charts

What steps are needed to create a chart in HTML / javascript compared to R? It turns out that typical steps are to:

1. include library specific css and javascript files
2. create a container element in which we can house the chart
3. make sure we have data in the right format to feed to the chart
4. render the chart in the container using library specific instructions

optionally,

1. send updates to the chart e.g. update its data or update parameters that alter the appearance of the chart

A first example

Let start with a first example, let's make a c3.js gauge chart! Here is the complete code for our first example:

<html>
<head>
    <link href="c3.min.css" rel="stylesheet" type="text/css">
</head>
<body>

<div id="chart1"></div>

<script src="d3.v3.min.js" charset="utf-8"></script>
<script src="c3.min.js"></script>

<script>
var gaugeData = {data: 75}; 

var chart1 = c3.generate({
  bindto: '#chart1',
  data: {
    json: gaugeData,
    type: 'gauge',
  },
  gauge: {
    min: 0,
    max: 100
  }
});
</script>

</body>
</html>

This code corresponds to example_01_gauge.html in the examples folder of this repository.

Let's break down this code into the steps stated in the previous section.

step 1

In our example we include one css file and two JavaScript files. This happens at lines 3, 9 and 10, respectively. The css file describes how to style elements on the screen, while the JavaScript files implement the logic to render and update the charts.

step 2

Creating a container element is easy, almost always it can be a simple div. The div tag defines a division or a section in an HTML document. This tag is one of the most common tags used in HTML pages. In our case it looks like this

<div id="chart1"></div>

Here we have given the div an id. The value of this id is unique and helps us to locate the element later on.

In case you're new to HTML and these terms are not yet familiar, don't worry. Picking up a basic understanding of how HTML pages are structured, which elements exists and what they do is not very hard. You may have to familiarize yourself with some new terminology though. In general, it greatly helps to have a basic understanding of HTML elements and css selectors such as ids and classes. Luckily, a basic understanding is often all you need. W3Schools offers a great set of quick, easy to understand tutorials on HTML, CSS and JavaScript.

The most important part here is this:

We want to be able to create elements on a page and to target them in order to be able to call a function on them, which will alter the state and content in that element.

step 3

Step 3 is often the most difficult step. Besides data collection, cleaning and organizing e.g. using dplyr and tidyr, we must send data from shiny to the web browser. R, however, works with dataframes, vectors and lists, while JavaScript works with things like objects and arrays and often uses JSON to describe data.

In the end, whatever we use in R must be translated into something the browser understands. Unfortunately, each chart library has its own conventions on how the data should be represented. As a widget builder it is your task to make the translation from R to JavaScript.

In the example above the JavaScript data looks like this

var gaugeData = {data: 75}; 

Here we have created a variable called gaugeData. In JavaScript this would be an example of an object, which has a key / value pair with a key called data and a value of 75.

When we are in R and want to set the value of the gauge to 75, we must make sure the final data the chart looks like this. In this case the data representation we're after is simple. However, in general this often is harder.

Internally, shiny and htmlwidgets use the R package jsonlite to convert R data structures to JSON. For more information, take a look at the package vignette. Especially useful are the fromJSON and toJSON functions. The first tells you how an R object will be translated into JSON, while the latter does the reverse. Just play around to see how it works!

For a quick example in R run this:

if (!require("jsonlite"))
  install.packages("jsonlite")

library(jsonlite)
toJSON(list(data = 75)) 

step 4

At this point we have included the required css and js scripts, set up a container element with an id equal to chart1 and have constructed a JSON dataset called gaugeData. We are now ready to construct the chart.

Unfortunately, different chart libraries use different conventions to set up charts, so the code you see below will be very specific to c3.js. However, learning this example helps to understand other libraries as well. The important part is that you understand which steps are needed, not specifically what syntax is used. Remember, a good chart library already makes it easy to construct a chart and given some examples the syntax is often easy to master. Luckily c3.js comes with many examples, for some inspiration take a look at these!

All c3.js components are initialized by calling the function c3.generate. This function takes a single object as its argument which supplies c3 with all information it requires to generate the component. In the example above the relevant code chunk looks like this:

var chart1 = c3.generate({
  bindto: '#chart1',
  data: {
    json: gaugeData,
    type: 'gauge',
  },
  gauge: {
    min: 0,
    max: 100
  }
});

The first item in this object is bindto which tells c3 which element on the HTML page we want to target to put the component into. Here value is #chart1. This string is an example of what's known as a CSS selector. The # symbol is CSS syntax which tells the browser to select an element from the HTML page with the specific id, which value comes after # symbol.

Next we pass in the previously constructed javascript variable gaugeData, which holds the data for the chart.

var gaugeData = {data: 75}; 

Here the data is provided as a key value pair with the name 'data' and value 75. Naming the data is not required for the gauge but when we extend the visual later on the data needs to be named. In actual widgets the data is usually passed from R to the browser by shiny, which internally relies on the R library jsonlite. In the example above, however, we manually created the data object.

In the code chunk below we pass the variable holding the data and in addition tell c3 that we want to construct a chart of type 'gauge'

data: {
  json: gaugeData,
  type: 'gauge'
}

C3 has different ways of providing components with data (see here). In our examples we typically pick the JSON format as shiny uses jsonlite to convert R data structures to JSON (see above).

Finally, we supply the component with some gauge specific options which in this example are the minimum and maximum values of the gauge.

gauge: {
  min: 0,
  max: 100
}

Extending the gauge

A desirable aspect of a well designed widget is the possibility to update the data in the widget after we have constructed the chart. Before we do this from shiny, we'll extend our first example by updating the data via JavaScript. Compared to the previous example we'll set some additional style properties to our gauge and, in addition, repeatedly change its value via a timer.

In the example below we see c3.js nicely animates the transition from one value to another. Note for this to happen we need to know both the current state of the gauge and the new state we want the gauge to be in.

Here's the code for the complete example:

<html>
<head>
    <!-- required css style file for c3.js -->
    <link href="c3.min.css" rel="stylesheet" type="text/css">
</head>
<body>

<!-- container element in which we will create the chart -->
<div id="chart1"></div>

<!-- required javascript libraries-->
<script src="d3.v3.min.js" charset="utf-8"></script>
<script src="c3.min.js"></script>

<!-- javascript block to render and update the chart-->
<script>

    var gaugeData = {'data': 80.0}

    // create a chart and set options
    // note that via the c3.js API we bind the chart to the element with id equal to chart1
    var chart = c3.generate({
        bindto: '#chart1',
        data: {
            json: gaugeData,
            type: 'gauge',
        },
        gauge: {
            label:{
                //returning here the value and not the ratio
                format: function(value, ratio){ return value;}
            },
            min: 0,
            max: 100,
            width: 15,
            units: 'value' //this is only the text for the label
        }
    });

    // this function will update every 2000 milliseconds
    // and create a new value between 0 and 100
    setInterval(function () {

        // create a random value between 0 and 100, rounded to 2 digits
        var newValue = Math.floor(100 * Math.random());

        // create a data array holding the random value
        var newData = {'data', newValue };

        // tell the chart to load the new data
        chart.load({
          json: newData
        });
    }, 2000);


</script>
</body>
</html>

This code corresponds to "example_02_gauge.html" in the examples folder of this repository. As we can see the call to c3.generate is a bit more involved. With respect to the styling options we refer to the c3.js examples and reference for details.

In the example above we wrapped the code to update the value of the gauge into another function, called setInterval. SetInterval is a standard javascript function that repeatedly executes a given function every ms milliseconds.

setInterval( functionDefinition , ms );

In our case the complete function call looks like this

setInterval(function () {

    // create a random value between 0 and 100, rounded to 2 digits
    var newValue = Math.floor(100 * Math.random());

    // create a data array holding the random value
    var newData = { 'data': newValue };

    // tell the chart to load the new data
    chart.load({
      json: newData
    });
  }, 2000);

Here we supplied the function we want to be executed as a so called anonymous function, which means we directly pass in the function definition without assigning it a name. Another way would be to first create a named function and then to pass the name of that function as a first argument to setInterval.

Inside the anonymous function we draw a random number and use it to create a new data object, using the same name as before i.e. 'data'.

// create a random value between 0 and 100, rounded to 2 digits
var newValue = Math.floor(100 * Math.random());

// create a data array holding the random value
var newData = { 'data': newValue };

Finally, we tell c3.js to load the new data to the already existing gauge.

chart.load({
  json: newData
});

Note that c3 automagically transitions the chart from the old state to the new state! The transition duration can be changed, which defaults to 350 milliseconds.

In javascript terms the function load in the previous code block is called a method , which is called on the object chart. You can think of methods as special functions that are part of a javascript object. Typically, methods perform actions on objects e.g. to set or retrieve values. C3 has many methods depending on the specific chart, see here for a complete overview.

At this stage we have covered all ingredients to make a fully functional gauge. Time to create a real htmlwidget!

Creating the widget

We are now back in the R world!

In order to create htmlwidgets we need the htmlwidgets and devtools packages. In case you don't have these installed already, please run:

install.packages("htmlwidgets")
install.packages("devtools")

Next, load the packages into R:

library("htmlwidgets")
library("devtools")

In order to create an htmlwidget we have to create an R package. This is where devtools comes in. In case your new to R package building, please see here for a proper introduction.

Luckily, the devtools and htmlwidget packages make it very easy to create a new package and to set up a basic skeleton for our widget. The next code chunk creates a new R package containing template code which we can use to build our widget:

devtools::create("C3")                                    # create package using devtools
setwd("C3")                                               # navigate to package dir
htmlwidgets::scaffoldWidget("C3Gauge","c3",edit=FALSE)    # create widget scaffolding
devtools::install()                                       # install package so we can test it

At first these steps may appear a bit technical, as well as the explanation stated below. Don't worry, soon these steps become second nature once you have created a few widgets.

The scaffoldWidget function is the function which creates the actual template.

With the first argument we specify the name we want our widget to have.

The second argument is optional and states the name of the javascript library on bower we want to use. Bower is a web component manager which you can use to install a component and all its dependencies. In our case we want to include c3.js and its dependencies. When you run the above command you can see c3.js and its dependencies i.e. d3.js, are downloaded and put in the package folder we just created. You can find them in the folder C3Gauge/inst/htmlwidgets/lib.

The "lib" folder is the default location for all the external libraries you will use in your widget. Of note, here we set edit to FALSE because for now we do not want to open an editor to edit our files.

To check if everything works we install our newly created package and with the next snippet you load the package we just created and run the template code which should display a simple "hello world!" message in the viewer pane in RStudio.

library(C3)
C3Gauge("hello, world")

Ok, what just happend?

The scaffoldWidget function created a number of files and directories for us.

First, an "htmlwidgets" directory is created within the "inst" directory of our C3Gauge package.

The "inst" folder is a generic R package folder where all external package dependencies are stored. We use this whenever we want to include files in our package.

Within the "inst" folder an "htmlwidget" subfolder is created. Here you'll find the following files:

• C3Gauge.js: This file contains the client side code of our binding. In our case this will be the javascript code described in the above examples extended with some extra logic to handle data coming from R.

• C3Gauge.yaml: This file lists all dependencies and their location in the folder structure for the widget. Because we used bower to get all the external dependencies this file is automatically filled in by R.

Within this folder we find the "lib" folder which contains the external javascript files for c3.js and d3.js downloaded by bower.

In the main folder of our project we find an "R" folder with the file "C3Gauge.R", which contains a template function 'C3Gauge' which has data and sizing information as arguments, propagates this information to the right javascript code and then displays the widget.

Besides this function, an output and a render function is created which we can use to integrate our widget in a Shiny app.

Step I: writing javascript code for our widget

In order to create our widget we have to code some javascript inside a javascript template file R automatically created. More specifically, we need to fill in the details in HTMLWidgets.widget. Let's take a closer look at the C3Gauge.js template code that R just generated:

HTMLWidgets.widget({

  name: 'C3Gauge',

  type: 'output',

  factory: function(el, width, height) {

    // TODO: define shared variables for this instance

    return {

      renderValue: function(x) {

        // TODO: code to render the widget, e.g.
        el.innerText = x.message;

      },

      resize: function(width, height) {

        // TODO: code to re-render the widget with a new size

      }

    };
  }
});

In HTMLWidgets.widget we have to give our component a name and we have to define the function factory. Currently, the value of 'type' can only have a single value being "output".

We see factory itself is a function that returns two other functions i.e. renderValue which will contain the code to setup our widget and resize which will take care of any re-sizing functionality e.g. what should the chart do when its container is resized? This code, however, is optional i.e. the chart doesn't have to resize if you don't want it to.

The factory method has three arguments: el, width and height. Here el is the container element created by htmlwidgets where we can house our chart in. By default htmlwidgets creates an empty div just like we did in the standalone examples above.

The next steps are to fill in the details of the previous functions. As a first step, let's copy the code from our previous javascript example in the renderValue function. For the moment let's not worry how javascript gets its data from R.

      renderValue: function(x) {

        var gaugeData = {'data': 80.0};

        // create a chart and set options
        // note that via the c3.js API we bind the chart to the element with id equal to chart1
        var chart = c3.generate({
            bindto: el,
            data: {
                json: gaugeData,
                type: 'gauge',
            },
            gauge: {
                label:{
                    //returning here the value and not the ratio
                    format: function(value, ratio){ return value;}
                },
                min: 0,
                max: 100,
                width: 15,
                units: 'value' //this is only the text for the label
            }
        });

      },

If we re-install our package, reload the library and execute the C3Gauge function we can see the example is rendered in R.

devtools::install()                                      
library(C3)
C3Gauge("")

Hooray, you just created your first c3 gauge widget!

Step II: writing R code for our widget

We now have a gauge we can render from R. However, we can't yet set the value of the gauge from R. In order to do so we have to adjust the code inside C3Gauge.R, another file R automatically created for us:

C3Gauge <- function(message, width = NULL, height = NULL) {

  # forward options using x
  x = list(
    message = message
  )

  # create widget
  htmlwidgets::createWidget(
    name = 'C3Gauge',
    x,
    width = width,
    height = height,
    package = 'C3'
  )
}

By default this function is created with three arguments: message, width and height.

In the default code, the message argument is any data that is passed on to the renderValue function in the C3Gauge.js code. The width and height arguments are passed to the resize function in the same file. Luckily, c3.js handles resizing automatically, so we don't have to explicitly tell our widget what to do in case its container element changes size.

The body of C3Gauge consists of two parts. First, a list called x is created which will contain all data passed on to the widget. This can be a single value, a data.frame or a complex list holding all sorts of things. Next, the createWidget function is called with a name, our data x, the widget width, height and package, the name of the package our widget is in.

If we look back at how we initialized our data in the stand alone examples, we can see the gauge wants the data to be in the following javascript format:

{'data': 80.0}

By default htmlwidgets uses the toJSON function from the jsonlite package to convert data from R to the JSON format. As stated in the previous sections, often it takes a bit of experimentation to see which R structure results in the correct JSON when passed via jsonlite::toJSON(). It turns out in this case we need:

  x <- list(data=80)

Indeed, via a call to jsonlite::toJSON(x) we see in javascript this results in:

{"data":[80]} 

The added brackets simply mean that the data has become a one dimensional array, which is equivalent to a single value.

Thus, in order to get the data in the form above, we only have to rename the message to data in the C3Gauge function inside C3Gauge.R which gives us the JSON object we want:

  # forward options using x
  x = list(
    data = message
  )

In C3Gauge.js we have to make one modification: we remove the line where we create gaugeData. Instead of passing the static gaugeData to the C3 generate function, we pass on our list created in R. In other words, we change the line where our data is specified to the following:

    json: x

After rebuilding and installing the package we now have a fully function gauge we can call from R with the following line:

  devtools::install()                                      
  library(C3Gauge)
  C3Gauge(50)

The code in C3Gauge.R now looks like this:

#' <Add Title>
#'
#' <Add Description>
#'
#' @import htmlwidgets
#'
#' @export
C3Gauge <- function(message, width = NULL, height = NULL) {

  # forward options using x
  x = list(
    data = message
  )

  # create widget
  htmlwidgets::createWidget(
    name = 'C3Gauge',
    x,
    width = width,
    height = height,
    package = 'C3Gauge'
  )
}

while the code in C3Gauge.js looks like this:

HTMLWidgets.widget({

  name: 'C3Gauge',

  type: 'output',

  factory: function(el, width, height) {

    // TODO: define shared variables for this instance

    return {

       renderValue: function(x) {

        // create a chart and set options
        // note that via the c3.js API we bind the chart to the element with id equal to chart1
        var chart = c3.generate({
            bindto: el,
            data: {
                json: x,
                type: 'gauge',
            },
            gauge: {
                label:{
                    //returning here the value and not the ratio
                    format: function(value, ratio){ return value;}
                },
                min: 0,
                max: 100,
                width: 15,
                units: 'value' //this is only the text for the label
            }
        });

      },

      resize: function(width, height) {

        // TODO: code to re-render the widget with a new size

      }

    };
  }
});

Finally, we have a complete working gauge widget, which value we can specify with R!

Next tutorial

In the next tutorial we'll put the widget we created in a shiny app and add code such that it updates its value via a smooth transition, similar to the one demonstrated in the stand alone examples. In addition, we'll create widgets for the c3 pie chart, line bar chart and stacked area charts and send events from these charts to shiny and back to the browser.

garrettgman/shiny-js-tutorials documentation built on May 16, 2019, 5:40 p.m.