spark_advisory_shuffle_partition_size: Retrieves or sets advisory size of the shuffle partition

In rstudio/sparklyr: R Interface to Apache Spark

Retrieves or sets advisory size of the shuffle partition

Description

Retrieves or sets advisory size in bytes of the shuffle partition during adaptive optimization

Usage

spark_advisory_shuffle_partition_size(sc, size = NULL)

Arguments

sc	A spark_connection.
size	Advisory size in bytes of the shuffle partition. Defaults to NULL to retrieve configuration entries.

See Also

Other Spark runtime configuration: spark_adaptive_query_execution(), spark_auto_broadcast_join_threshold(), spark_coalesce_initial_num_partitions(), spark_coalesce_min_num_partitions(), spark_coalesce_shuffle_partitions(), spark_session_config()

rstudio/sparklyr documentation built on Sept. 18, 2024, 6:10 a.m.