In WinVector/replyr: Patches to Use 'dplyr' on Remote Data Sources

Duplicate columns not allowed in join

Instead of adding suffixes join fails with duplicate column message. I think this depends on the version of dplyr used. Seeing the failure with the dev-version of dplyr '0.5.0.9004' as of 5-14-2017. Confirmed it works properly for dplyr 0.5.0.

Submitted as Sparklyr issue 677 and dplyr issue 2774.

# devtools::install_github("tidyverse/dplyr")
# devtools::install_github('tidyverse/dbplyr')
suppressPackageStartupMessages(library('dplyr'))
packageVersion("dplyr")
library('sparklyr')
packageVersion("sparklyr")
if(requireNamespace("dbplyr", quietly = TRUE)) {
  packageVersion("dbplyr")
}
R.Version()$version.string

sc <- sparklyr::spark_connect(version='2.0.2', 
   master = "local")
d1 <- copy_to(sc, data.frame(x=1:3, y=4:6), 'd1')
d2 <- copy_to(sc, data.frame(x=1:3, y=7:9), 'd2')

left_join(d1, d2, by='x')

spark_disconnect(sc)
rm(list=ls())
gc(verbose = FALSE)

WinVector/replyr documentation built on Oct. 22, 2020, 8:07 p.m.