s3PutFile: Saves data to a file in AWS S3

In madedotcom/retl: Support for common ETL operations and connections

Saves data to a file in AWS S3

Description

Convention based wrapper functions that allow to save data in S3

Usage

s3PutFile(dt, path, bucket = s3DefaultBucket(), root = s3DefaultRoot(), ...)

s3PutFile.csv(
  dt,
  path,
  bucket = s3DefaultBucket(),
  root = s3DefaultRoot(),
  na.value = ""
)

s3PutFile.gz(
  dt,
  path,
  bucket = s3DefaultBucket(),
  root = s3DefaultRoot(),
  na.value = ""
)

s3PutFile.rds(dt, path, bucket = s3DefaultBucket(), root = s3DefaultRoot())

s3PutFile.json.gz(dt, path, bucket = s3DefaultBucket(), root = s3DefaultRoot())

Arguments

dt	data.table to save
path	S3 object path starting after root folder.
bucket	name of the S3 bucket. Defaults to value in AWS_S3_BUCKET environment variable.#'
root	project root path that is appended before the path, e.g. "/prod/". Defaults to value in AWS_S3_ROOT environment variable.
...	additional parameters that will be passed into extension specific call.
na.value	the string to use for missing values in the data. Defaults to empty string.

Details

Environment variables required for aws.s3 access:

• AWS_ACCESS_KEY_ID = "mykey"

• AWS_SECRET_ACCESS_KEY = "mysecretkey"

• AWS_DEFAULT_REGION = "us-east-1"

• AWS_SESSION_TOKEN = "mytoken"

Value

TRUE if data was saved to S3.

's3PutFile.csv' saves data into '.csv' file in S3.

's3PutFile.gz' saves data into '.gz' (csv archived) file in S3.

's3PutFile.rds' saves data as '.rds' file in S3

's3PutFile.json.gz' saves data as '.json.gz' (json archived) file in S3

See Also

aws.s3 package documentation for access details: https://github.com/cloudyr/aws.s3

madedotcom/retl documentation built on Sept. 19, 2022, 3:28 a.m.