copy_to: Copy a (very small) local data frame to Impala

Description Usage Arguments Value Note Examples

Description

copy_to inserts the contents of a local data frame into a new Impala table. copy_to currently only supports very small data frames (1000 or fewer row/column positions). It uses the SQL INSERT ... VALUES() technique, which is not suitable for loading large amounts of data.

This package does not provide tools for loading larger amounts of local data into Impala tables. This is because Impala can query data stored in several different filesystems and storage systems (HDFS, Apache Kudu, Apache HBase, Amazon S3, Microsoft ADLS, and Dell EMC Isilon) and Impala does not include built-in capability for loading local data into these systems.

Usage

1
2
3
4
5
6
## S3 method for class 'src_impala'
copy_to(dest, df, name = deparse(substitute(df)),
  overwrite = FALSE, types = NULL, temporary = TRUE,
  unique_indexes = NULL, indexes = NULL, analyze = TRUE,
  external = FALSE, force = FALSE, field_terminator = NULL,
  line_terminator = NULL, file_format = NULL, ...)

Arguments

dest

an object with class with class src_impala

df

a (very small) local data frame

name

name for the new Impala table

overwrite

whether to overwrite existing table data (currently ignored)

types

a character vector giving variable types to use for the columns

temporary

must be set to FALSE

unique_indexes

not used

indexes

not used

analyze

whether to run COMPUTE STATS after adding data to the new table

external

whether the new table will be externally managed

force

whether to silently continue if the table already exists

field_terminator

the deliminter to use between fields in text file data. Defaults to the ASCII control-A (hex 01) character

line_terminator

the line terminator. Defaults to "\n"

file_format

the storage format to use. Options are "TEXTFILE" (default) and "PARQUET"

...

other arguments passed on to methods

Value

An object with class tbl_impala, tbl_sql, tbl_lazy, tbl

Note

Impala does not support temporary tables. When using copy_to() to insert local data into an Impala table, you must set temporary = FALSE.

Examples

1
2
3
4
5
6
7
library(nycflights13)
dim(airlines) # airlines data frame is very small
# [1] 16  2

## Not run: 
copy_to(impala, airlines, temporary = FALSE)
## End(Not run)

implyr documentation built on July 21, 2019, 5:04 p.m.