Request helper functions

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)

This vignette explains the purpose and usage of:

The target audience is someone writing an R package to wrap a Google API.

library(gargle)

Why use gargle's request helpers?

Why would the developer of a Google-API-wrapping package care about the request helpers in gargle?

You can write less code and safer code, in return for a modest investment in studying your target API. That is done by ingesting the API's so-called Discovery Document.

Hundreds of Google APIs -- the ones addressed by the API Discovery Service -- share a great deal of behaviour. By ingesting the metadata provided by this service, you can use gargle's request helpers to exploit this shared data and logic, while also decreasing the chance that you and your users will submit ill-formed requests.

The request helpers in gargle check the combined inputs from user and developer against suitably prepared API metadata:

Google provides API libraries for several languages, including Java, Go, Python, JavaScript, Ruby and more (but not R). All of these libraries are machine-generated from the metadata provided by the API Discovery Service. It is the official recommendation to use the Discovery Document when building client libraries. The gargle package aims to implement key parts of this strategy, in a way that is also idiomatic for R and its developers.

High-level design pattern

gargle facilitates this design for API-wrapping packages:

Later, specific examples are given, using the googledrive package.

gargle's HTTP request helpers

gargle provides support for creating and sending HTTP requests via these functions:

request_develop(endpoint, params, base_url): a.k.a. The Smart One.

request_build(method, path, params, body, token, key, base_url): a.k.a. The Dumb One.

request_make(x, ..., user_agent): actually makes the HTTP request.

They are usually called in the above order, though they don't have to be used that way. It is also fine to ignore this part of gargle and use it only for help with auth. They are separate parts of the package.

Discovery Documents

Google's API Discovery Service "provides a lightweight, JSON-based API that exposes machine-readable metadata about Google APIs". We recommend ingesting this metadata into an R list, stored as internal data in an API-wrapping client package. Then, HTTP requests inside high-level functions can be made concisely and safely, by referring to this metadata. The combined use of this data structure and gargle's request helpers can eliminate a lot of boilerplate data and logic that are shared across Google APIs and across endpoints within an API.

The gargle package ships with some functions and scripts to facilitate the ingest of a Discovery Document. You can find these files in the gargle installation like so:

ddi_dir <- system.file("discovery-doc-ingest", package = "gargle")
list.files(ddi_dir)

Main files of interest to the developer of a client package:

The remaining files present an analysis of the Discovery Document for the Discovery API itself (very meta!) and write files that are useful for reference. Several are included at the end of this vignette.

Why aren't the ingest functions exported by gargle? First, we regard this as functionality that is needed at development time, not install or run time. This is something you'll do every few months, probably associated with preparing a release of a wrapper package. Second, the packages that are useful for wrangling JSON and lists are not existing dependencies of gargle, so putting these function in gargle would require some unappealing compromises.

Method (or endpoint) data

Our Discovery Document ingest process leaves you with an R list. Let's assume it's available in your package's namespace as an internal object named .endpoints. Each item represents one method of the API (Google's vocabulary) or an endpoint (gargle's vocabulary).

Each endpoint has an id. These ids are also used as names for the list. Examples of some ids from the Drive and Sheets APIs:

drive.about.get
drive.files.create
drive.teamdrives.list
sheets.spreadsheets.create
sheets.spreadsheets.values.clear
sheets.spreadsheets.sheets.copyTo

Retrieve the metadata for one endpoint by name, e.g.:

.endpoints[["drive.files.create"]]

That info can be passed along to request_develop(endpoint, params, base_url), which conducts sanity checks and combines this external knowledge with the data coming from the user and developer via params.

Design suggestion: forming requests

Here's the model used in googledrive. There is a low-level request helper, googledrive::request_generate(), that is used to form every request in the package. It is exported as part of a low-level API for expert use, but most users will never know it exists.

# googledrive::
request_generate <- function(endpoint = character(),
                             params = list(),
                             key = NULL,
                             token = drive_token()) {
  ept <- .endpoints[[endpoint]]
  if (is.null(ept)) {
    stop_glue("\nEndpoint not recognized:\n  * {endpoint}")
  }

  ## modifications specific to googledrive package
  params$key <- key %||% params$key %||% drive_api_key()
  if (!is.null(ept$parameters$supportsTeamDrives)) {
    params$supportsTeamDrives <- TRUE
  }

  req <- gargle::request_develop(endpoint = ept, params = params)
  gargle::request_build(
    path = req$path,
    method = req$method,
    params = req$params,
    body = req$body,
    token = token
  )
}

The endpoint argument specifies an endpoint by its name, a.k.a. its id.

params is where the processed user input goes.

key and token refer to an API key and OAuth2 token, respectively. Both can be populated by default, but it is possible to pass them explicitly. If your package ships with a default API key, you should append it above as the final fallback value for params$key.

Do not "borrow" an API key from gargle or another package; always send a key associated with your package or provided by your user. Per the Google User Data Policy https://developers.google.com/terms/api-services-user-data-policy, your application must accurately represent itself when authenticating to Google API services.

After googledrive::request_generate() takes care of everything specific to the Drive API and the user's input and task, we call gargle::request_develop(). We finish preparing the request with gargle::request_build(), which enforces the rule that we always send exactly one of key and token.

Design suggestion: making requests

The output of gargle::request_build() specifies an HTTP request.

gargle::request_make() can be used to actually execute it.

# gargle::
request_make <- function(x, ..., user_agent = gargle_user_agent()) {
  stopifnot(is.character(x$method))
  method <- switch(
    x$method,
    GET    = httr::GET,
    POST   = httr::POST,
    PATCH  = httr::PATCH,
    PUT    = httr::PUT,
    DELETE = httr::DELETE,
    abort(glue("Not a recognized HTTP method: {bt(x$method)}"))
  )
  method(
    url = x$url,
    body = x$body,
    x$token,
    user_agent,
    ...
  )
}

request_make() consults x$method to identify the httr::VERB() and then calls it with the remainder of x, ... and the user_agent.

In googledrive we have a thin wrapper around this that injects the googledrive user agent:

# googledrive::
request_make <- function(x, ...) {
  gargle::request_make(x, ..., user_agent = drive_ua())
}

Reference

derived from the Discovery Document for the Discovery Service

Properties of an endpoint

cat(readLines(fs::path(ddi_dir, "method-properties-humane.txt")), sep = "\n")

API-wide endpoint parameters (taken from Discovery API but, empirically, are shared with other APIs):

cat(readLines(fs::path(ddi_dir, "api-wide-parameters-humane.txt")), sep = "\n")

Properties of an endpoint parameters:

cat(readLines(fs::path(ddi_dir, "parameter-properties-humane.txt")), sep = "\n")


Try the gargle package in your browser

Any scripts or data that you put into this service are public.

gargle documentation built on Sept. 8, 2022, 9:07 a.m.