In srvanderplas/LegoR: Package to scrape data from shop.lego.com and brickset

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.path = "man/figures/README-",
  out.width = "100%",
  dpi = 300
)

LegoR

The goal of LegoR is to make it easy to get Lego-centric data into R.

Installation

You can install the development version from GitHub with:

# install.packages("devtools")
devtools::install_github("srvanderplas/LegoR")

# Set up and load packages
library(tidyverse)
library(LegoR)

Lego.com

The first set of functions provide a convenient way to scrape data from https://shop.lego.com/. These functions are based on the rvest package and depend on the structure of the site; site updates may break the functionality. All functions for lego.com start with the lego_ prefix.

The natural approach to gather data on all currently available lego sets is to get all sets by theme.

(themes <- lego_get_themes())

Each theme link leads to a page with one or more sets.

(architecture_sets <- lego_get_sets(themes$theme_link[1]))

If the goal is to get the price and titles, we could stop here, but more set data is available on the set-specific page.

lego_get_set_data(architecture_sets$set_link[1])

These sets are structured in order to provide easy pipe functionality:

set_data <- lego_get_themes() %>%
  filter(row_number() == 1) %>% # Don't get everything in the demo
  mutate(set_summary = purrr::map(theme_link, lego_get_sets)) %>%
  unnest(set_summary) %>%
  mutate(set_data = purrr::map(set_link, lego_get_set_data)) %>%
  unnest(set_data) %>%
  select(-set_Item) # Some variables are repeated

set_data

Brickset

https://brickset.com/ contains data on historical lego sets as well as current sets. Unlike Lego.com, we can access Brickset data using an API (application programming interface). This does require registering for a brickset account and requesting an API key. All functions for the brickset.com data start with the brickset_ prefix.

brickset_setup() # guides you through the account setup process

Once you have your credentials, you can save them to your Rprofile using brickset_save_credentials(). This will also save the credentials as global variables.

brickset_save_credentials("your_username", "your_password", "your_api_key")

Then, you can access brickset's data by authenticating. You may have to periodically reauthenticate depending on your internet configuration, but most functions should refresh the authentication automatically.

brickset_auth()

As with the Lego store, sets on brickset are organized by theme.

themes <- brickset_get_themes()

We can see what themes existed at the beginning...

# Oldest themes
arrange(themes, yearfrom)

Or the themes that have been around the longest...

# Longest running themes
arrange(themes, desc(yearto - yearfrom)) %>%
  head(10)

Most of the functions described in the API documentation have been wrapped; the exception is functions which concern a user's personal collection.

srvanderplas/LegoR documentation built on Nov. 5, 2019, 9:17 a.m.