read_from_datalake: Read any type file stored in an AWS S3 bucket.

In StatisticsNZ/er.helpers: Helper functions commonly used in Environmental Reporting at Stats NZ

Read any type file stored in an AWS S3 bucket.

Description

This function get the specified object from an AWS S3 bucket and reads it using functions designed for each relevant file type. If the file is either a .RDS, .csv, or .xlsx; it first downloads the file to a temp directory and, therefore, it avoids the unintended consequences of saving the file in the disk. If it is not a recognisable file type, it saves it to the working directory. This function also uses key search terms and will throw an error if there is more than one file with the search terms used.

Usage

read_from_datalake(..., date = NULL, all_sheets = T, version = NULL)

Arguments

...	Key terms to search for in the AWS S3 bucket
all_sheets	If more than one sheet is present, T = read all sheets into a list, F = default to sheet specified
version	VersionId of the object key desired. Can be retrieved using get_bucket_version_df

Examples

## Not run: 
setup_datalake_access()
read_from_datalake("landcover", "concordance", "lcdb4")

## End(Not run)

StatisticsNZ/er.helpers documentation built on Oct. 2, 2023, 7:24 a.m.