spark_dataframe.ts_rdd: Retrieve a Spark DataFrame
In sparklyr.flint: Sparklyr Extension for 'Flint'

Description Usage Arguments See Also Examples

Retrieve a Spark DataFrame from a TimeSeriesRDD object

1 2	## S3 method for class 'ts_rdd' spark_dataframe(x, ...)

`x`	An R object wrapping, or containing, a Spark DataFrame.
`...`	Optional arguments; currently unused.

Other Spark dataframe utility functions: collect.ts_rdd(), from_rdd(), from_sdf(), spark_connection.ts_rdd(), spark_jobj.ts_rdd(), to_sdf(), ts_rdd_builder()

library(sparklyr)
library(sparklyr.flint)

sc <- try_spark_connect(master = "local")

if (!is.null(sc)) {
  sdf <- copy_to(sc, tibble::tibble(t = seq(10), v = seq(10)))
  ts <- from_sdf(sdf, is_sorted = TRUE, time_unit = "SECONDS", time_column = "t")
  print(ts %>% spark_dataframe())
  print(sdf %>% spark_dataframe()) # the former should contain the same set of
                                   # rows as the latter does, modulo possible
                                   # difference in types of timestamp columns
} else {
  message("Unable to establish a Spark connection!")
}