HARtools: Read HTTP Archive ('HAR') Data

title: "HARtools: Basic Use" author: "John D Harrison" date: "2016-11-09" output: rmarkdown::html_vignette: toc: true toc_depth: 3 vignette: > %\VignetteIndexEntry{Vignette Title} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8}

The goal of this vignette is to describe the basic functions in the HARtools package.

HAR stands for HTTP Archive. This is a common format for recording HTTP tracing information. This file contains a variety of information. It has a record of each object being loaded by a browser. Each of these objects’ timings is recorded. The common extension for these files is .har. The specification for this format is produced by the Web Performance Working Group of the World Wide Web Consortium (W3C).

An outline of the specification can be viewed at softwareishard.com

HARtools is a lightweight package containing a number of tools to read/parse, visualise and write files in the HAR format.

readHAR is the function whoose aim is to read read/parse HAR objects. It currently can read/parse a number of formats. It takes an argument har which currently can be a character string, an R list, a URL or a file containing HAR data. It also carries out basic validation on the HAR content (checking mandatory/optional fields).

Reading/Parsing from file

The HARtools package contains a number of examples of HAR files. We will use one of these (HAR data from browsing google.com):

library(HARtools)
har <- readHAR(system.file(package = "HARtools", 
                           "exdata", "google.com.har"))
har

## --------HAR VERSION-------- 
## HAR specification version: 1.1 
## --------HAR CREATOR-------- 
## Created by: Firebug 
## version: 1.5X.0b8 
## --------HAR BROWSER-------- 
## Browser: Firefox 
## version: 3.6b6pre 
## --------HAR PAGES-------- 
## Page id: page_62143 , Page title: Google 
## --------HAR ENTRIES-------- 
## Number of entries: 5 
## REQUESTS: 
## Page: page_62143 
## Number of entries: 5 
##   -  http://www.google.cz/ 
##   -  http://www.google.cz/intl/en_com/images/logo_plain.png 
##   -  http://www.google.cz/extern_js/f/CgJjcxICY3orMAo4QUAdLCswDjgKLCswFjgULCsw... 
##   -  http://clients1.google.cz/generate_204 
##   -  http://www.google.cz/images/nav_logo7.png

names(har)

## [1] "log"

names(har[["log"]])

## [1] "version" "creator" "browser" "pages"   "entries"

So we see that a HAR object is a list with root log and children as defined per the specification.

Reading/Parsing a URL

The same HAR data we sourced from file is available online.

harData <- "http://www.janodvarko.cz/har/viewer/examples/google.com.har"
harURL <- readHAR(harData)
identical(harURL, har)

## [1] TRUE

Reading/Parsing an R character string

readHAR can read/parse an R character string

fileName <- system.file(package = "HARtools", "exdata", "google.com.har")
harString <- readChar(fileName, file.info(fileName)$size)
print(substr(harString, 1, 5*80))

## [1] "{\n  \"log\":{\n    \"version\":\"1.1\",\n    \"creator\":{\n      \"name\":\"Firebug\",\n      \"version\":\"1.5X.0b8\"\n    },\n    \"browser\":{\n      \"name\":\"Firefox\",\n      \"version\":\"3.6b6pre\"\n    },\n    \"pages\":[{\n        \"startedDateTime\":\"2010-01-02T14:51:01.186+01:00\",\n        \"id\":\"page_62143\",\n        \"title\":\"Google\",\n        \"pageTimings\":{\n          \"onContentLoad\":90,\n          \"onLoad\":245\n        }\n     "

harSTR <- readHAR(harString)
identical(harSTR, har)

## [1] TRUE

Reading/Parsing an R list

readHAR can read/parse an R list.

harL <- jsonlite::fromJSON(system.file(package = "HARtools",
                                          "exdata", "google.com.har"),
                              simplifyVector = FALSE)
harLIST <- readHAR(harL)
identical(harLIST, har)

## [1] TRUE

Parsed HAR objects of class har can be written to file using the writeHAR function:

harFile <- tempfile(fileext = ".har")
writeHAR(har, harFile)
har2 <- readHAR(harFile)
identical(har, har2)

## [1] TRUE

writeHAR exports the HAR in JSON format.

HARtools includes a HARviewer function. HARviewer is a htmlwidget which utilises the PerfCascade to create a waterfall view of an HAR object.

har <- readHAR(system.file(package = "HARtools", "exdata",
                           "r-project.org.161028_W2_11MA.har"))
hv <- HARviewer(har, width="100%", height="100%")
# view in R
hv

## Warning in file(con, "r"): file("") only supports open = "w+" and open = "w+b":
## using the former

The above HAR data was obtained from testing the r-project webpage using WebPageTest.

HARtools has a renderHARviewer function and a corresponding HARviewerOutput function which can be used in a shiny server and ui respectively to embed a HARviewer waterfall chart in a shiny app:

library(shiny)
if(interactive()){
  library(HARtools)
  har <- readHAR(system.file(package = "HARtools", "exdata",
                             "r-project.org.161028_W2_11MA.har"))
  hv <- HARviewer(har)

  app <- shinyApp(
    ui = fluidPage(
      HARviewerOutput("myHAR")
    ),
    server = function(input, output) {
      output$myHAR <- renderHARviewer(hv)
    }
  )  
  runApp(app)
}

Most modern browser have developer tools that often have the option to export a browsing session as HAR data see for example this article.

Browser Mob Proxy captures performance data for web apps (via the HAR format), as well as manipulate browser behavior and traffic, such as whitelisting and blacklisting content, simulating network traffic and latency, and rewriting HTTP requests and responses.

An R client for BMP is available at https://github.com/johndharrison/bmproxy.

WebPagetest is an open source project that is primarily being developed and supported by Google as part of their efforts to make the web faster.

WebPagetest is a tool that was originally developed by AOL for use internally and was open-sourced in 2008 under a BSD license. The platform is under active development on GitHub and is also packaged up periodically and available for download if you would like to run your own instance.

The online version at www.webpagetest.org is run for the benefit of the performance community with several companies and individuals providing the testing infrastructure around the globe.

An R client for WPT is available at https://github.com/johndharrison/webpagetestr. It allows the user to send test requests to www.webpagetest.org or any private instance of WPT a user may have access to.

johndharrison/HARtools documentation built on May 19, 2019, 4:22 p.m.

rdrr.io home R language documentation Run R code online

CRAN packages Bioconductor packages R-Forge packages GitHub packages

Note that we can't provide technical support on individual packages. You should contact the package authors for that.

johndharrison/HARtools
Read HTTP Archive ('HAR') Data

docs/articles/Basic_use.knit.utf8.md
In johndharrison/HARtools: Read HTTP Archive ('HAR') Data

HAR format

HARtools basics

Reading HAR files

Reading/Parsing from file

Reading/Parsing a URL

Reading/Parsing an R character string

Reading/Parsing an R list

Writing HAR files

Visualising HAR data

HARviewer

Embedding HAR charts in a shiny app

Obtaining HAR data

Using a browser

Using Browser Mob Proxy

Using WebPageTest

R Package Documentation

Browse R Packages

We want your feedback!

johndharrison/HARtools Read HTTP Archive ('HAR') Data

docs/articles/Basic_use.knit.utf8.md In johndharrison/HARtools: Read HTTP Archive ('HAR') Data

HAR format

HARtools basics

Reading HAR files

Reading/Parsing from file

Reading/Parsing a URL

Reading/Parsing an R character string

Reading/Parsing an R list

Writing HAR files

Visualising HAR data

HARviewer

Embedding HAR charts in a shiny app

Obtaining HAR data

Using a browser

Using Browser Mob Proxy

Using WebPageTest

R Package Documentation

Browse R Packages

We want your feedback!

johndharrison/HARtools
Read HTTP Archive ('HAR') Data

docs/articles/Basic_use.knit.utf8.md
In johndharrison/HARtools: Read HTTP Archive ('HAR') Data