read_all: Read multiple files

In gmega/megautils: Collection of Utility Functions

Description

Composable utility functions for reading multiple files from a given location.

Usage

read_all()

using(.p, readfun = read_csv, ...)

into_env(.p, namefun = default_naming)

into_tibble(.p, namefun = default_naming, namecol = "origin")

Arguments

readfun	the function which will be used to read each individual file. Defaults to read_csv.
...	parameters passed on to readfun (e.g. ‘delim = ’;'')
namefun	a function which takes a string and returns a string. Each file processed by 'into_env' will call this function once to determine which key to use into the destination environment. The default function uses the file name without its extension and replaces '-' with '_'.
path	filesystem path from where to read files (e.g. '/home/user/files').
pattern	a pattern to filter files by. Accepts wildcards (e.g. ‘pattern = ’*.csv''). Defaults to '*' (read all files).
bucket	name of the GCS bucket where to read files from.
prefix	GCS prefix of the files to be read.

Details

'from_fs()'

reads a set of files from the local filesystem.

'from_gcs()'

reads a set of files from a Google Cloud Storage bucket.

'using()'

specifies which read function to apply to the downloaded data. Defaults to read_csv

'into_tibble()'

combines all files into a single Tibble. Equivalent to looping through the files, reading them, and then binding them together.

'into_env()'

reads each file and assigns its contents to a key within an environment.

Examples

# So we can use %>% syntax.
library(magrittr)

# Folder with two files
path <- system.file('extdata', 'read_all', package = 'megautils')

# Reads contents as single tibble.
schema <- list(a = readr::col_integer(), b = readr::col_integer())

read_all() %>%
  using(readr::read_csv, col_types = schema) %>%
  from_fs(path) %>%
  into_tibble()

# Reads contents into environment.
e <- read_all() %>%
   using(readr::read_csv, col_types = schema) %>%
   from_fs(path) %>%
   into_env()

ls(e)

gmega/megautils documentation built on Sept. 14, 2020, 8:06 p.m.