In r-spark/sparklyr.sedona: Sparklyr Extension for Apache Sedona

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)

An important part of sparklyr.sedona is its collection of R interfaces to Sedona visualization routines. For example, the following is essentially the R equivalent of this example in Scala.

library(sparklyr)
library(sparklyr.sedona)

sc <- spark_connect(master = "local")

resolution_x <- 1000
resolution_y <- 600
boundary <- c(-126.790180, -64.630926, 24.863836, 50.000)

pt_rdd <- sedona_read_dsv_to_typed_rdd(
  sc,
  location = "arealm.csv",
  type = "point"
)
polygon_rdd <- sedona_read_dsv_to_typed_rdd(
  sc,
  location = "primaryroads-polygon.csv",
  type = "polygon"
)
pair_rdd <- sedona_spatial_join_count_by_key(
  pt_rdd,
  polygon_rdd,
  join_type = "intersect"
)

overlay <- sedona_render_scatter_plot(
  polygon_rdd,
  resolution_x,
  resolution_y,
  output_location = tempfile("scatter-plot-"),
  boundary = boundary,
  base_color = c(255, 0, 0),
  browse = FALSE
)

sedona_render_choropleth_map(
  pair_rdd,
  resolution_x,
  resolution_y,
  output_location = "/tmp/choropleth-map",
  boundary = boundary,
  overlay = overlay,
  # vary the green color channel according to relative magnitudes of data points so
  # that the resulting map will show light blue, light purple, and light gray pixels
  color_of_variation = "green",
  base_color = c(225, 225, 255)
)

It will create a scatter plot, and then overlay it on top of a choropleth map, as shown below:

See ?sparklyr.sedona::sedona_render_scatter_plot, ?sparklyr.sedona::sedona_render_heatmap, and ?sparklyr.sedona::sedona_render_choropleth_map for more details on visualization-related R interfaces currently implemented by sparklyr.sedona.

r-spark/sparklyr.sedona documentation built on Dec. 22, 2021, 11:56 a.m.