gar_batch_walk: Walk data through batches

In googleAuthR: Authenticate and Create Google APIs

Walk data through batches

Description

Convenience function for walking through data in batches

Usage

gar_batch_walk(
  f,
  walk_vector,
  gar_pars = NULL,
  gar_paths = NULL,
  the_body = NULL,
  pars_walk = NULL,
  path_walk = NULL,
  body_walk = NULL,
  batch_size = 10,
  batch_function = NULL,
  data_frame_output = TRUE,
  ...,
  batch_endpoint = getOption("googleAuthR.batch_endpoint", default =
    "https://www.googleapis.com/batch")
)

Arguments

f	a function from gar_api_generator
walk_vector	a vector of the parameter or path to change
gar_pars	a list of parameter arguments for f
gar_paths	a list of path arguments for f
the_body	a list of body arguments for f
pars_walk	a character vector of the parameter(s) to modify for each walk of f
path_walk	a character vector of the path(s) to modify for each walk of f
body_walk	a character vector of the body(s) to modify for each walk of f
batch_size	size of each request to Google /batch API
batch_function	a function that will act on the result list of each batch API call
data_frame_output	if the list of lists are dataframes, you can bind them all by setting to TRUE
...	further arguments passed to the data parse function of f
batch_endpoint	the batch API endpoint to send

Details

You can modify more than one parameter or path arg, but it must be the same walked vector e.g. start = end = x

Many Google APIs have batch_size limits greater than 10, 1000 is common.

The 'f' function needs to be a 'gar_api_generator()' function that uses one of 'path_args', 'pars_args' or 'body_args' to construct the URL (rather than say using 'sprintf()' to create the API URL).

You don't need to set the headers in the Google docs for batching API functions - those are done for you.

The argument 'walk_vector' needs to be a vector of the values of the arguments to walk over, which you indicate will walk over the pars/path or body arguments on the function via on of the '*_walk' arguments e.g. if walking over id=1, id=2, for a path argument then it would be 'path_walk="id"' and 'walk_vector=c(1,2,3,4)'

The 'gar_*' parameter is required to pass intended for other arguments to the function 'f' you may need to pass through.

'gar_batch_walk()' only supports changing one value at a time, for one or multiple arguments (I think only changing the 'start-date', 'end-date' example would be the case when you walk through more than one per call)

'batch_size' should be over 1 for batching to be of any benefit at all

The 'batch_function' argument gives you a way to operate on the parsed output of each call

Value

if data_frame_output is FALSE: A list of lists. Outer list the length of number of batches required, inner lists the results from the calls

if data_frame_output is TRUE: The list of lists will attempt to rbind all the results

See Also

Other batch functions: gar_batch()

Examples

## Not run: 
                               

# get a webproperty per account 
getAccountInfo <- gar_api_generator(
  "https://www.googleapis.com/analytics/v3/management/accounts",
  "GET", data_parse_function = function(x) unique(x$items$id))

getWebpropertyInfo <- gar_api_generator(
  "https://www.googleapis.com/analytics/v3/management/", # don't use sprintf to construct this
  "GET",
  path_args = list(accounts = "default", webproperties = ""),
  data_parse_function = function(x) x$items)

walkData <- function(){

  # here due to R lazy evaluation  
  accs <- getAccountInfo()
  gar_batch_walk(getWebpropertyInfo, 
                 walk_vector = accs,
                 gar_paths = list("webproperties" = ""),
                 path_walk = "accounts",
                 batch_size = 100, data_frame_output = FALSE)
                 }
                 
# do the walk
walkData()

# to walk body data, be careful to modify a top level body name:
changed_emails <- lapply(email, function(x){userRef = list(email = x)})

batched <- gar_batch_walk(users, 
                       walk_vector = changed_emails, 
                       the_body = list(
                           permissions = list(
                             local = list(permissions)
                           ),
                           userRef = list(
                             email = email[[1]]
                         )
                        ),
                        body_walk = "userRef",
                        batch_size = 300,
                        data_frame_output = FALSE)

## End(Not run)

googleAuthR documentation built on May 29, 2024, 7:22 a.m.