In rstudio-conf-2020/big-data: Content and setup files for the Big Data with R class

eval_vroom <- FALSE
if(Sys.getenv("GLOBAL_EVAL") != "") eval_vroom <- Sys.getenv("GLOBAL_EVAL")

library(vroom)
library(fs)
library(purrr)
library(dplyr)

Introduction to vroom

vroom basics

Load data into R using vroom

1. Load the vroom() library r library(vroom)

2. Use the vroom() function to read the transactions_1.csv file from the /usr/share/class/files folder r vroom("/usr/share/class/files/transactions_1.csv")

3. Use the id argument to add the file name to the data frame. Use file_name as the argument's value ```r

   ```

4. Load the prior command into a variable called vr_transactions ```r

   ```

5. Load the file spec into a variable called vr_spec, using the spec() command ```r vr_spec <- spec(vr_transactions)

   vr_spec ```

Load multiple files

1. Load the fs and dplyr libraries r library(fs) library(dplyr)

2. List files in the /usr/share/class/files folder using the dir_ls() function r dir_ls("/usr/share/class/files")

3. In the dir_ls() function, use the glob argument to pass a wildcard to list CSV files only. Load to a variable named files ```r

   ```

4. Pass the files variable to vroom. Set the n_max argument to 1,000 to limit the data load for now ```r

   ```

5. Add a col_types argument with vr_specs as its value ```r

   ```

6. Use the col_select argument to pass a list object containing the following variables: order_id, date, customer_name, and price ```r

   ```

Load and modify multiple files

For files that are too large to have in memory, keep a summarization

1. Use a for() loop to print the content of each vector inside files r for(i in seq_along(files)) { print(files[i]) }

2. Switch the print() command with the vroom command, using the same arguments, except the file name. Use the files variable. Load the results into a variable called transactions. ```r for(i in seq_along(files)) {

   } ```

3. Group transactions by order_id and get the total of price and the number of records. Name them total_sales and no_items respectively. Name the new variable orders ```r for(i in seq_along(files)) {

   } ```

4. Define the orders variable as NULL prior to the for loop and add a bind_rows() step to orders to preserve each summarized view. ```r orders <- NULL for(i in seq_along(files)) {

   } ```

5. Remove the transactions variable at the end of each cycle
   ```r orders <- NULL for(i in seq_along(files)) {

   } ```

6. Preview the orders variable r orders

rstudio-conf-2020/big-data documentation built on Feb. 4, 2020, 5:24 p.m.