sdf_last_index: Returns the last index of a Spark DataFrame

Description Usage Arguments

View source: R/sdf_interface.R

Description

Returns the last index of a Spark DataFrame. The Spark mapPartitionsWithIndex function is used to iterate through the last nonempty partition of the RDD to find the last record.

Usage

1
sdf_last_index(x, id = "id")

Arguments

x

A spark_connection, ml_pipeline, or a tbl_spark.

id

The name of the index column.


sparklyr documentation built on June 17, 2021, 5:06 p.m.