sync: S3 file sync

In cloudyr/aws.s3: 'AWS S3' Client Package

Description

Sync files/directories to/from S3

Usage

s3sync(
  path = ".",
  bucket,
  prefix = "",
  direction = c("upload", "download"),
  verbose = TRUE,
  create = FALSE,
  ...
)

Arguments

path	string, path to the directory to synchronize, it will be expanded as needed (NOTE: older versions had a files argument which expected a full list of files which was ambiguous).
bucket	Character string with the name of the bucket, or an object of class “s3_bucket”.
prefix	string, if set to non-empty string, leading part of the objects in the bucket much have that prefix, other objects are not considered. In practice, this alows the immitation of sub-directories in the bucket and in that case it is typically required that the training slash is included in the prefix.
direction	A character vector specifying whether to “upload” and/or “download” files. By default, s3sync is two-way, uploading any files missing from the bucket and downloading any objects missing from the local directory.
verbose	A logical indicating whether to be verbose (the default is TRUE).
create	logical, if TRUE the bucket is created if it doesn't exist, otherwise synchronizing a non-existing bucket is an error.
...	Additional arguments passed to s3HTTP.

Details

s3sync synchronizes specified files to an S3 bucket. If the bucket does not exist, it is created (unless create=FALSE). Similarly, if local directories do not exist (corresponding to leading portions of object keys), they are created, recursively. Object keys are generated based on files and local files are named (and organized into directories) based on object keys. A slash is interpreted as a directory level. Local objects are copied to S3 and S3 objects are copied locally. This copying is performed conditionally. Objects existing locally but not in S3 are uploaded using put_object. Objects existing in S3 but not locally, are saved using save_object. If objects exist in both places, the MD5 checksum for each is compared; when identical, no copying is performed. If the checksums differ, local files are replaced with the bucket version if the local file is older and the S3 object is replaced if the local file is newer. If checksums differ but modified times match (which seems unlikely), a warning is issued. Note that multi-part files don't have a full MD5 sum recorded in S3 so they cannot be compared and thus are always assumed to be different.

Value

A logical.

References

aws s3 sync command line

See Also

get_bucket, put_object, , save_object

Examples

## Not run: 
  put_bucket("examplebucket")

  # sync all files in current directory to bucket (upload-only)
  s3sync(bucket = "examplebucket", direction = "upload")

  # two-way sync
  s3sync(bucket = "examplebucket")

  # full sync between a subset of the bucket and a test directory in user's home
  # corresponding roughly to:
  #   aws s3 sync ~/test s3://examplebucket/test/
  #   aws s3 sync s3://examplebucket/test/ ~/test
  s3sync("~/test", "examplebucket", prefix="test/", region="us-east-2")

## End(Not run)

cloudyr/aws.s3 documentation built on May 29, 2020, 7:18 p.m.