bq_table_download | R Documentation |
This retrieves rows in chunks of page_size
. It is most suitable for results
of smaller queries (<100 MB, say). For larger queries, it is better to
export the results to a CSV file stored on google cloud and use the
bq command line tool to download locally.
bq_table_download(
x,
n_max = Inf,
page_size = NULL,
start_index = 0L,
max_connections = 6L,
quiet = NA,
bigint = c("integer", "integer64", "numeric", "character"),
max_results = deprecated()
)
x |
A bq_table |
n_max |
Maximum number of results to retrieve. Use |
page_size |
The number of rows requested per chunk. It is recommended to
leave this unspecified until you have evidence that the When |
start_index |
Starting row index (zero-based). |
max_connections |
Number of maximum simultaneous connections to BigQuery servers. |
quiet |
If |
bigint |
The R type that BigQuery's 64-bit integer types should be
mapped to. The default is |
max_results |
Because data retrieval may generate list-columns and the data.frame
print method can have problems with list-columns, this method returns
a tibble. If you need a data.frame
, coerce the results with
as.data.frame()
.
bigrquery will retrieve nested and repeated columns in to list-columns as follows:
Repeated values (arrays) will become a list-column of vectors.
Records will become list-columns of named lists.
Repeated records will become list-columns of data frames.
In my timings, this code takes around 1 minute per 100 MB of data. If you need to download considerably more than this, I recommend:
Export a .csv
file to Cloud Storage using bq_table_save()
.
Use the gsutil
command line utility to download it.
Read the csv file into R with readr::read_csv()
or data.table::fread()
.
Unfortunately you can not export nested or repeated formats into CSV, and the formats that BigQuery supports (arvn and ndjson) that allow for nested/repeated values, are not well supported in R.
df <- bq_table_download("publicdata.samples.natality", n_max = 35000)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.