request_retry: Make a Google API request, repeatedly

Description Usage Arguments Details Value Special cases See Also Examples

View source: R/request_retry.R

Description

Intended primarily for internal use in client packages that provide high-level wrappers for users. It is a drop-in substitute for request_make() that also has the ability to retry the request.

Usage

1
request_retry(..., max_tries_total = 5, max_total_wait_time_in_seconds = 100)

Arguments

...

Passed along to request_make().

max_tries_total

Maximum number of tries.

max_total_wait_time_in_seconds

Total seconds we are willing to dedicate to waiting, summed across all tries. This is a technical upper bound and actual cumulative waiting will be less.

Details

Consider an example where we are willing to make a request up to 5 times.

1
2
3
try  1  2    3        4                5
     |--|----|--------|----------------|
wait  1   2      3           4

There will be up to 5 - 1 = 4 waits and we generally want the waiting period to get longer, in an exponential way. Such schemes are called exponential backoff. request_retry() implements exponential backoff with "full jitter", where each waiting time is generated from a uniform distribution, where the interval of support grows exponentially. A common alternative is "equal jitter", which adds some noise to fixed, exponentially increasing waiting times.

Either way our waiting times are based on a geometric series, which, by convention, is usually written in terms of powers of 2:

1
2
b , 2b, 4b, 8b, ...
  = b * 2^0, b * 2^1, b * 2^2, b * 2^3, ...

The terms in this series require knowledge of b, the so-called exponential base, and many retry functions and libraries require the user to specify this. But most users find it easier to declare the total amount of waiting time they can tolerate for one request. Therefore request_retry() asks for that instead and solves for b internally. This is inspired by the Opnieuw Python library for retries. Opnieuw's interface is designed to eliminate uncertainty around:

Let n be the total number of tries we're willing to make (the argument max_tries_total) and let W be the total amount of seconds we're willing to dedicate to making and retrying this request (the argument max_total_wait_time_in_seconds). Here's how we determine b:

1
2
3
4
sum_{i=0}^(n - 1) b * 2^i = W
b * sum_{i=0}^(n - 1) 2^i = W
       b * ( (2 ^ n) - 1) = W
                        b = W / ( (2 ^ n) - 1)

Value

Object of class response from httr.

Special cases

request_retry() departs from exponential backoff in three special cases:

See Also

Examples

1
2
3
4
5
6
7
8
9
## Not run: 
req <- gargle::request_build(
  method = "GET",
  path = "path/to/the/resource",
  token = "PRETEND_I_AM_TOKEN"
)
gargle::request_retry(req)

## End(Not run)

gargle documentation built on July 2, 2021, 5:07 p.m.