spark_jar_task: Spark Jar Task

In brickster: R Toolkit for 'Databricks'

Spark Jar Task

Description

Spark Jar Task

Usage

spark_jar_task(main_class_name, parameters = list())

Arguments

main_class_name	The full name of the class containing the main method to be executed. This class must be contained in a JAR provided as a library. The code must use SparkContext.getOrCreate to obtain a Spark context; otherwise, runs of the job fail.
parameters	Named list. Parameters passed to the main method. Use Task parameter variables to set parameters containing information about job runs.

See Also

Other Task Objects: email_notifications(), libraries(), new_cluster(), notebook_task(), pipeline_task(), python_wheel_task(), spark_python_task(), spark_submit_task()

brickster documentation built on April 12, 2025, 1:21 a.m.