Copy issue with sparklyr 1.6.2.

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = " # "
)
options(width =100)

Below is why we use a new column name in joins.

OSX 10.11.6. Spark installed as described at http://spark.rstudio.com

library('sparklyr')
spark_install(version = "1.6.2")
library('dplyr')
library('sparklyr')
R.Version()$version.string
packageVersion('dplyr')
packageVersion('sparklyr')
my_db <- sparklyr::spark_connect(version='1.6.2', master = "local")
class(my_db)
my_db$spark_home
print(my_db)
d1 <- copy_to(my_db,data.frame(x=c(1,2),y=c('a','b')),'d1')
d2 <- copy_to(my_db,data.frame(y=c('a','b'),z=c(3,4)),'d2')
d1 %>% dplyr::inner_join(d2,by='y')

Submitted as sparklyr issue 338.

version


WinVector/replyr documentation built on Oct. 22, 2020, 8:07 p.m.