In srvanderplas/LegoR: Package to scrape data from shop.lego.com and brickset

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  include = T,
  dpi = 300
)
if (file.exists("../.Rprofile")) source("../.Rprofile")

library(LegoR)
library(dplyr)
library(tidyr)
library(ggplot2)

Rebrickable.com contains data on lego sets, pieces, links to instructions, and more. API info is available at https://rebrickable.com/api/. If you are planning to make many calls to the API, it may be faster to download their entire database at https://rebrickable.com/downloads/ instead.

To get started using the API, you'll need to register for a key. Run the following command to get set up.

rebrickable_setup()

If you want to make your key part of your .Rprofile so that you don't have to remember it, use the rebrickable_save_credentials function:

rebrickable_save_credentials("your api key")

Basic API Calls

Every API call can return a list containing the parsed (formatted) content as well as the original call information.

colors <- rebrickable_colors(parse = F, follow_next = F)
colors$parsed_content
colors$full_res

By default, each command will follow "next" links to get all available pages of information. To turn this off, set follow_next = F.

Rebrickable contains detailed information on the parts in each set (including colors), as well as theme information, which is also available at sites like Brickset.

lego_set <- rebrickable_set_info("71040-1")

knitr::include_graphics(lego_set$set_img_url)

We can get the pieces in this particular set:

parts <- rebrickable_set_parts(set_id = "71040-1", page_size = 400, follow_next = T)

And even view the images of the 24 most common pieces in the set

knitr::include_graphics(arrange(parts, desc(quantity))$part_img_url[1:24])

Putting it all together

Lets' start by getting a data set containing each lego set by decade and year:

library(dplyr)
library(tibble)
library(tidyr)
library(purrr)
library(ggplot2)

sets_by_decade <- tibble(min_year = seq(1950, 2010, by = 10), max_year = seq(1959, 2019, by = 10)) %>%
  mutate(sets = map2(min_year, max_year, ~rebrickable_sets(min_year = .x, max_year = .y, page_length = 100, follow_next = T)))

sets_by_decade <- sets_by_decade %>%
  select(decade = min_year, sets = sets) %>%
  unnest("sets")

We can see how the number of pieces changes over time:

sets_by_decade %>%
  filter(num_parts > 1) %>%
ggplot(aes(x = year, y = num_parts)) + 
  geom_jitter(alpha = .2) + 
  geom_smooth() + 
  scale_y_log10("# Pieces") + 
  xlab("Year") + 
  ggtitle("Number of Pieces per year")

With a little bit more work, we can also see how the colors in lego have changed. Let's work with sets containing between 100 and 150 pieces.

part_list <- sets_by_decade %>%
  left_join(rebrickable_themes(), by = c("theme_id" = "id")) %>%
  filter(num_parts >= 100, num_parts <= 150) %>%
  head() %>%
  mutate(part_list = map(set_num, function(x){
    Sys.sleep(5) # be kind to servers!
    rebrickable_set_parts(set_id = x)
  }))