Copy issue with sparklyr
2.0.0.
knitr::opts_chunk$set( collapse = TRUE, comment = " # " ) options(width =100)
Below is why we re-try joins against local data without using the
copy=TRUE
feature.
OSX 10.11.6. Spark installed as described at http://spark.rstudio.com
library('sparklyr') spark_install(version = "2.0.0")
library('dplyr') library('sparklyr') R.Version()$version.string packageVersion('dplyr') packageVersion('sparklyr') my_db <- sparklyr::spark_connect(version='2.0.0', master = "local") class(my_db) my_db$spark_home print(my_db) d1 <- copy_to(my_db,data.frame(x=c(1,2),y=c('a','b')),'d1') d2 <- data.frame(y=c('a','b'),z=c(3,4)) d1 %>% dplyr::inner_join(d2,by='y',copy=TRUE)
Submitted as sparklyr issue 339.
version
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.