Description Usage Arguments Examples
Reads a Avro file into Apache Spark using sparklyr.
1 2 3 4 5 6 7 8 9 | spark_read_avro(
sc,
name,
path,
readOptions = list(),
repartition = 0L,
memory = TRUE,
overwrite = TRUE
)
|
sc |
An active |
name |
The name to assign to the newly generated table. |
path |
The path to the file. Needs to be accessible from the cluster. Supports the "hdfs://", "s3n://" and "file://" protocols. |
readOptions |
A list of strings with additional options. |
repartition |
The number of partitions used to distribute the generated table. Use 0 (the default) to avoid partitioning. |
memory |
Boolean; should the data be loaded eagerly into memory? (That is, should the table be cached?) |
overwrite |
Boolean; overwrite the table with the given name if it already exists? |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | ## Not run:
## If you haven't got a Spark cluster, you can install Spark locally like this
library(sparklyr)
spark_install(version = "2.0.1")
sc <- spark_connect(master = "local")
df <- spark_read_avro(
sc,
"twitter",
system.file("extdata/twitter.avro", package = "sparkavro"),
repartition = FALSE,
memory = FALSE,
overwrite = FALSE
)
spark_disconnect(sc)
## End(Not run)
|
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.