cloud_s3_write_bulk: Write multiple objects to S3 in bulk

In cloudfs: Streamlined Interface to Interact with Cloud Storage Platforms

Write multiple objects to S3 in bulk

Description

This function allows for the bulk writing of multiple R objects to the project's designated S3 folder. To prepare a list of objects for writing, use cloud_object_ls, which generates a dataframe listing the objects and their intended destinations in a format akin to the output of cloud_s3_ls. By default, the function determines the appropriate writing method based on each file's extension. However, if a specific writing function is provided via the fun parameter, it will be applied to all files, which may not be ideal if dealing with a variety of file types.

Usage

cloud_s3_write_bulk(
  content,
  fun = NULL,
  ...,
  local = FALSE,
  quiet = FALSE,
  root = NULL
)

Arguments

content	(data.frame) output of cloud_object_ls()
fun	A custom writing function. If NULL (default), the appropriate writing function will be inferred based on the file's extension.
...	Additional arguments to pass to the writing function fun.
local	Logical, defaulting to FALSE. If TRUE, the function will also create a local copy of the file at the specified path. Note that some writing functions might not overwrite existing files unless explicitly allowed. Typically, such functions have a parameter (often named overwrite) to control this behavior. Check the documentation of the writing function used to determine the exact parameter name and pass it through the ... argument if necessary. Alternatively, you can define an anonymous function for fun that calls a writing function with the overwriting option enabled.
quiet	all caution messages may be turned off by setting this parameter to TRUE.
root	S3 path of the project root. This serves as the reference point for all relative paths. When left as NULL, the root is automatically derived from the cloudfs.s3 field of the project's DESCRIPTION file.

Value

Invisibly returns the input content dataframe.

Examples

# write two csv files: data/df_mtcars.csv and data/df_iris.csv
cloud_object_ls(
  dplyr::lst(mtcars = mtcars, iris = iris),
  path = "data",
  extension = "csv",
  prefix = "df_"
) |>  
cloud_s3_write_bulk()
  

cloudfs documentation built on May 29, 2024, 11:08 a.m.