db_compute: S3 implementation of 'db_compute' for Athena

Description Usage Arguments Value See Also Examples

Description

This is a backend function for dplyr's compute function. Users won't be required to access and run this function.

Usage

1

Arguments

con

A dbConnect object, as returned by dbConnect()

table

Table name, if left default noctua will use the default from dplyr's compute function.

sql

SQL code to be sent to the data

...

passes noctua table creation parameters: [file_type,s3_location,partition]

  • file_type: What file type to store data.frame on s3, noctua currently supports ["NULL","csv", "parquet", "json"]. "NULL" will let Athena set the file_type for you.

  • s3_location: s3 bucket to store Athena table, must be set as a s3 uri for example ("s3://mybucket/data/")

  • partition: Partition Athena table, requires to be a partitioned variable from previous table.

Value

db_compute returns table name

See Also

backend_dbplyr

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
## Not run: 
# Note: 
# - Require AWS Account to run below example.
# - Different connection methods can be used please see `noctua::dbConnect` documnentation

library(DBI)
library(dplyr)

# Demo connection to Athena using profile name 
con <- dbConnect(noctua::athena())

# Write data.frame to Athena table
copy_to(con, mtcars,
        s3_location = "s3://mybucket/data/")
             
# Write Athena table from tbl_sql
athena_mtcars <- tbl(con, "mtcars")
mtcars_filter <- athena_mtcars %>% filter(gear >=4)

# create athena with unique table name
mtcars_filer %>% 
  compute()

# create athena with specified name and s3 location
mtcars_filer %>% 
    compute("mtcars_filer",
            s3_location = "s3://mybucket/mtcars_filer/")

# Disconnect from Athena
dbDisconnect(con)

## End(Not run)

noctua documentation built on Sept. 23, 2021, 5:12 p.m.