cloud_s3_prep_bulk: Prepare S3 content dataframe to be used by bulk download/read...

In cloudfs: Streamlined Interface to Interact with Cloud Storage Platforms

Prepare S3 content dataframe to be used by bulk download/read functions

Description

cloud_s3_ls returns a dataframe of contents of an S3 folder. This dataframe has name and type columns. name may be either full names or short names (depending on full_names parameter of cloud_s3_ls), but names(name) will always contain full names. This function:

1. filters out folders

2. extracts names(name) into path column.

3. informs about the size of files that are to be downloaded/read and asks for confirmation

Usage

cloud_s3_prep_bulk(
  content,
  what = c("read", "upload", "download"),
  safe_size = 5e+07,
  quiet = FALSE
)

Arguments

content	(data.frame) Output of cloud_s3_ls()
what	What will be done with content, either "read" or "download". This affects only how messages will look.
safe_size	What is considered to be safe size in bytes to download in bulk. To show additional caution message in case if you accidentally run bulk reading on a folder with gigabytes of data.
quiet	All caution messages may be turned off by setting this parameter to TRUE.

Value

Transformed content dataframe.

cloudfs documentation built on May 29, 2024, 11:08 a.m.