In ropenscilabs/rredlist: 'IUCN' Red List Client

library("knitr")
library("rredlist")
hook_output <- knitr::knit_hooks$get("output")
knitr::knit_hooks$set(output = function(x, options) {
   lines <- options$output.lines
   if (is.null(lines)) {
     return(hook_output(x, options))  # pass to default hook
   }
   x <- unlist(strsplit(x, "\n"))
   more <- "..."
   if (length(lines)==1) {        # first n lines
     if (length(x) > lines) {
       # truncate the output, but add ....
       x <- c(head(x, lines), more)
     }
   } else {
     x <- c(if (abs(lines[1])>1) more else NULL,
            x[lines],
            if (length(x)>lines[abs(length(lines))]) more else NULL
           )
   }
   # paste these lines together
   x <- paste(c(x, ""), collapse = "\n")
   hook_output(x, options)
 })

knitr::opts_chunk$set(
  warning = FALSE,
  message = FALSE,
  collapse = TRUE,
  comment = "#>"
)

## Introduction `rredlist` provides two APIs, a higher-level one that takes slightly more time but returns the data in a more user-friendly format (a list), and a lower-level one (i.e., functions that end with "_") that takes less time but does no processing of the data (returning the raw JSON string). Both APIs return the exact same information, but it is up to the user whether the format processing is worth the extra time, especially when performing bulk operations. To help inform this decision by the user, here is some benchmarking related to the two APIs. First, we'll break down the total difference in computation time between the two APIs, then we'll dig into what components are causing this difference. We'll use `microbenchmark::microbenchmark()` which has very little computational overhead. Note that the time units vary from comparison to comparison, and the speed of these functions may be highly hardware- and network-dependent. wzxhzdk:1 ## Head-to-head benchmarks We'll start by benchmarking the two APIs head-to-head. We'll test a couple of use cases, in rough order of increasing complexity. #### 1\. Get species count wzxhzdk:2 #### 2\. Lookup individual assessment wzxhzdk:3 #### 3\. Taxonomic lookup with defaults wzxhzdk:4 #### 4\. Taxonomic lookup with query (one page of results) wzxhzdk:5 #### 5\. Taxonomic lookup with query (~10 pages of results) wzxhzdk:6 #### 6\. Taxonomic lookup with query (~40 pages of results) wzxhzdk:7 #### 7\. Taxonomic lookup with query (~900 pages of results) wzxhzdk:8 ### And the winner is... As you can see above, the two APIs take roughly the same amount of time for most use cases. I previously said that the low-level API is designed to be faster. While most of these comparisons agree with that statement, the time reduction is usually a few milliseconds per function call. When we get into more complex queries, like returning multiple pages of API results, we start to see larger time reductions, especially as the number of pages of results increases (10+ seconds for hundreds of pages). ## Query breakdown Based on the above, it doesn't seem to matter much, time-wise, whether we parse the data or not. So then what takes up all of the query time? Let's break down the process of querying the API and downloading a single page of assessments using some of the internal functions of `rredlist`: wzxhzdk:9 wzxhzdk:10 The above benchmarking shows us that the vast majority of time is spent downloading data from the IUCN API. For a single page of results, even the highest level of parsing takes only 0.15% of the time it takes to download the data. Further, while parsing to a list of dataframes (`parse = TRUE`) takes about 10 times as long as just parsing to a list of lists (`parse = FALSE`), both methods remain very quick compared to the process of downloading the data. Now let's break down a multi-page query: wzxhzdk:11 wzxhzdk:12 Again, even with about 40 pages of data to parse, the download takes the vast majority of the time. The highest-level parsing has increased to about 1% of the time it takes to download the data, but this remains less than a second compared to the ~35 second download. ## Conclusion Ultimately, both APIs take about the same amount of time because the majority of time is spent downloading the data from the IUCN database and reading it into R. For larger downloads, the parsing done by the high-level API may take an appreciable amount of time (tenths of seconds to seconds), It's possible that users who are calling these functions many (e.g., thousands) of times would appreciate this time reduction. However, for most users it probably won't matter. Furthermore, keep in mind that if you use the low-level API you will likely need to do your own processing after the fact in order to do any sort of downstream analyses. Ultimately, the choice is up to you.

ropenscilabs/rredlist documentation built on Feb. 7, 2025, 2:28 a.m.