as.spark.dataframe: Import data from a spatial RDD into a Spark Dataframe.

Description Usage Arguments Examples

View source: R/sdf_interface.R

Description

Import data from a spatial RDD (possibly with non-spatial attributes) into a Spark Dataframe.

Usage

1
as.spark.dataframe(x, non_spatial_cols = NULL, name = NULL)

Arguments

x

A spatial RDD.

non_spatial_cols

Column names for non-spatial attributes in the resulting Spark Dataframe.

name

Name to assign to the resulting Spark temporary view. If unspecified, then a random name will be assigned.

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
library(sparklyr)
library(sparklyr.sedona)

sc <- spark_connect(master = "spark://HOST:PORT")

if (!inherits(sc, "test_connection")) {
  input_location <- system.file(
    file.path("extdata", "arealm-tiny.csv"), package = "sparklyr.sedona"
  )
  rdd <- sedona_read_dsv_to_typed_rdd(
    sc,
    location = input_location,
    delimiter = ",",
    type = "point",
    first_spatial_col_index = 1L,
    repartition = 5
  )
  sdf <- as.spark.dataframe(rdd, non_spatial_cols = c("attr1", "attr2"))
}

r-spark/sparklyr.sedona documentation built on Dec. 22, 2021, 11:56 a.m.