processing_config: Export Airflow processing config from a SageMaker processor

In DyfanJones/sagemaker-r-workflow: sagemaker pipeline and workflows

Export Airflow processing config from a SageMaker processor

Description

Export Airflow processing config from a SageMaker processor

Usage

processing_config(
  processor,
  inputs = NULL,
  outputs = NULL,
  job_name = NULL,
  experiment_config = NULL,
  container_arguments = NULL,
  container_entrypoint = NULL,
  kms_key_id = NULL
)

Arguments

processor	(sagemaker.processor.Processor): The SageMaker processor to export Airflow config from.
inputs	(list[:class:'~sagemaker.processing.ProcessingInput']): Input files for the processing job. These must be provided as :class:'~sagemaker.processing.ProcessingInput' objects (default: None).
outputs	(list[:class:'~sagemaker.processing.ProcessingOutput']): Outputs for the processing job. These can be specified as either path strings or :class:'~sagemaker.processing.ProcessingOutput' objects (default: None).
job_name	(str): Processing job name. If not specified, the processor generates a default job name, based on the base job name and current timestamp.
experiment_config	(dict[str, str]): Experiment management configuration. Dictionary contains three optional keys: 'ExperimentName', 'TrialName', and 'TrialComponentDisplayName'.
container_arguments	([str]): The arguments for a container used to run a processing job.
container_entrypoint	([str]): The entrypoint for a container used to run a processing job.
kms_key_id	(str): The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt the processing job output. KmsKeyId can be an ID of a KMS key, ARN of a KMS key, alias of a KMS key, or alias of a KMS key. The KmsKeyId is applied to all outputs.

Value

dict: Processing config that can be directly used by SageMakerProcessingOperator in Airflow.

DyfanJones/sagemaker-r-workflow documentation built on April 3, 2022, 11:28 p.m.