Description Exponential back-off Throttling
Geek sites rate limits are not very well documented, and may even vary in relation to server load. This makes it hard to predict when the limit will be breached. Exponential back-off is used to mitigate this.
This means that after an unsuccessful request, for example one that returned
a response with status 429 ("Too many requests"), R will pause for a time
and then try again. It will try 3 times in total, including the initial time,
increasing the pause duration before each-retry. This is implemented using
httr::RETRY()
, according to the formula 5 * 2 ^ attempt seconds
,
with the minimum of 5s pause.
This will make it more likely that a large amount of data is retrieved successfully, but will not help in other situations (e.g. badly formed request).
According to some observations, throttling can vary depending on the current server load.
This is relevant for some functions like get_games()
which allow you
to specify a vector of id numbers, and the API does not seem to place a hard
limit on how long it can be. This allows you to request hundreds of records
at a time, but if the server decides it is too busy, you will only get a 429
error back. Exponential back-off might not be enough to resolve this
if the server remains busy for a longer period of time.
To mitigate this, if you are planning to make intensive requests, please
pick a time of day when the traffic is lowest (basically night time in the
US). To help distribute the load during paging (i.e. when fetching comments
or plays), you might also use the delay_s
argument to space out
the requests.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.