req_perform_sequential: Perform multiple requests in sequence

In httr2: Perform HTTP Requests and Process the Responses

Perform multiple requests in sequence

Description

Given a list of requests, this function performs each in turn, returning a list of responses. It's slower than req_perform_parallel() but has fewer limitations.

Usage

req_perform_sequential(
  reqs,
  paths = NULL,
  on_error = c("stop", "return", "continue"),
  progress = TRUE
)

Arguments

reqs	A list of requests.
paths	An optional list of paths, if you want to download the request bodies to disks. If supplied, must be the same length as reqs.
on_error	What should happen if one of the requests fails? stop, the default: stop iterating with an error. return: stop iterating, returning all the successful responses received so far, as well as an error object for the failed request. continue: continue iterating, recording errors in the result.
progress	Display a progress bar? Use TRUE to turn on a basic progress bar, use a string to give it a name, or see progress_bars to customise it in other ways.

Value

A list, the same length as reqs, containing responses and possibly error objects, if on_error is "return" or "continue" and one of the responses errors. If on_error is "return" and it errors on the ith request, the ith element of the result will be an error object, and the remaining elements will be NULL. If on_error is "continue", it will be a mix of requests and error objects.

Only httr2 errors are captured; see req_error() for more details.

Examples

# One use of req_perform_sequential() is if the API allows you to request
# data for multiple objects, you want data for more objects than can fit
# in one request.
req <- request("https://api.restful-api.dev/objects")

# Imagine we have 50 ids:
ids <- sort(sample(100, 50))

# But the API only allows us to request 10 at time. So we first use split
# and some modulo arithmetic magic to generate chunks of length 10
chunks <- unname(split(ids, (seq_along(ids) - 1) %/% 10))

# Then we use lapply to generate one request for each chunk:
reqs <- chunks |> lapply(\(idx) req |> req_url_query(id = idx, .multi = "comma"))

# Then we can perform them all and get the results
## Not run: 
resps <- reqs |> req_perform_sequential()
resps_data(resps, \(resp) resp_body_json(resp))

## End(Not run)

httr2 documentation built on Nov. 14, 2023, 5:08 p.m.