flyio - Make data fly to R

Input and output data from R — download, upload, read and write objects from AWS S3, GoogleCloudStorage or local file system from a single interface.


flyio provides a common interface to interact with data from cloud storage providers or local storage directly from R. It currently supports AWS S3 and Google Cloud Storage, thanks to the API wrappers provided by cloudyr. flyio also supports reading or writing tables, rasters, shapefiles and R objects to the data source from memory.

For global usage, the datsource, authentication keys and bucket can be set in the environment variables of the machine so that one does not have to input it every time. - For datasource:CLOUD_STORAGE_NAME - For bucket name: flyioBucketS3 or flyioBucketGcs - For authentication: GCS_AUTH_FILE or AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_DEFAULT_REGION (For AWS S3, if the awscli is athenticated, then this step is not needed)


# Install the stable version from CRAN:

# Install the latest dev version from GitHub:

# Load the library

If you encounter a bug, please file an issue with steps to reproduce it on Github. Please use the same for any feature requests, enhancements or suggestions.


# Setting the data source

# Verify if the data source is set

# Authenticate the default data source and set bucket

# Authenticate S3 also
flyio_set_bucket("atlanhq-flyio", data_source = "s3")

# Listing the files in GCS
list_files(path = "test", pattern = "*csv")

# Saving mtcars to all the data sources using default function write.csv
export_table(mtcars, "~/Downloads/mtcars.csv", data_source = "local")
export_table(mtcars, "test/mtcars.csv") # saving to GCS, need not mention as set globally
export_table(mtcars, "test/mtcars.csv", data_source = "s3")

# Check if the file written exists in GCS

# Read the file from GCS using readr library
mtcars <- import_table("test/mtcars.csv", FUN = readr::read_csv)


Try the flyio package in your browser

Any scripts or data that you put into this service are public.

flyio documentation built on Oct. 31, 2019, 4:59 p.m.