dot-SageMakerContainer: Handle the lifecycle and configuration of a local container...

In DyfanJones/sagemaker-r-local: Sagemaker core classes, methods and functions

Handle the lifecycle and configuration of a local container execution.

Description

This class is responsible for creating the directories and configuration files that the docker containers will use for either training or serving.

Methods

Public methods

• .SageMakerContainer$new()

• .SageMakerContainer$process()

• .SageMakerContainer$train()

• .SageMakerContainer$serve()

• .SageMakerContainer$stop_serving()

• .SageMakerContainer$retrieve_artifacts()

• .SageMakerContainer$write_processing_config_files()

• .SageMakerContainer$write_config_files()

• .SageMakerContainer$clone()

---

Method new()

Initialize a SageMakerContainer instance It uses a :class:'sagemaker.session.Session' for general interaction with user configuration such as getting the default sagemaker S3 bucket. However this class does not call any of the SageMaker APIs.

Usage

.SageMakerContainer$new(
  instance_type,
  instance_count,
  image,
  sagemaker_session = NULL,
  container_entrypoint = NULL,
  container_arguments = NULL
)

Arguments

instance_type

(str): The instance type to use. Either 'local' or 'local_gpu'

instance_count

(int): The number of instances to create.

image

(str): docker image to use.

sagemaker_session

(sagemaker.session.Session): a sagemaker session to use when interacting with SageMaker.

container_entrypoint

(str): the container entrypoint to execute

container_arguments

(str): the container entrypoint arguments

---

Method process()

Run a processing job locally using docker-compose.

Usage

.SageMakerContainer$process(
  processing_inputs,
  processing_output_config,
  environment,
  processing_job_name
)

Arguments

processing_inputs

(dict): The processing input specification.

processing_output_config

(dict): The processing output configuration specification.

environment

(dict): The environment collection for the processing job.

processing_job_name

(str): Name of the local processing job being run.

---

Method train()

Run a training job locally using docker-compose.

Usage

.SageMakerContainer$train(
  input_data_config,
  output_data_config,
  hyperparameters,
  job_name
)

Arguments

input_data_config

(dict): The Input Data Configuration, this contains data such as the channels to be used for training.

output_data_config

: The configuration of the output data.

hyperparameters

(dict): The HyperParameters for the training job.

job_name

(str): Name of the local training job being run.

Returns

(str): Location of the trained model.

---

Method serve()

Host a local endpoint using docker-compose.

Usage

.SageMakerContainer$serve(model_dir, environment)

Arguments

model_dir

(str): pointing to a file or s3:// location.

environment

a dictionary of environment variables to be passed to the hosting container.

---

Method stop_serving()

Stop the serving container. The serving container runs in async mode to allow the SDK to do other tasks.

Usage

.SageMakerContainer$stop_serving()

---

Method retrieve_artifacts()

Get the model artifacts from all the container nodes. Used after training completes to gather the data from all the individual containers. As the official SageMaker Training Service, it will override duplicate files if multiple containers have the same file names.

Usage

.SageMakerContainer$retrieve_artifacts(
  compose_data,
  output_data_config,
  job_name
)

Arguments

compose_data

(list): Docker-Compose configuration in dictionary format.

output_data_config

: The configuration of the output data.

job_name

: The name of the job.

Returns

Local path to the collected model artifacts.

---

Method write_processing_config_files()

Write the config files for the processing containers. This method writes the hyperparameters, resources and input data configuration files.

Usage

.SageMakerContainer$write_processing_config_files(
  host,
  environment,
  processing_inputs,
  processing_output_config,
  processing_job_name
)

Arguments

host

(str): Host to write the configuration for

environment

(dict): Environment variable collection.

processing_inputs

(dict): Processing inputs.

processing_output_config

(dict): Processing output configuration.

processing_job_name

(str): Processing job name.

---

Method write_config_files()

Write the config files for the training containers. This method writes the hyperparameters, resources and input data configuration files.

Usage

.SageMakerContainer$write_config_files(
  host,
  hyperparameters,
  input_data_config
)

Arguments

host

(str): Host to write the configuration for

hyperparameters

(dict): Hyperparameters for training.

input_data_config

(dict): Training input channels to be used for training.

Returns

NULL

---

Method clone()

The objects of this class are cloneable with this method.

Usage

.SageMakerContainer$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

DyfanJones/sagemaker-r-local documentation built on June 14, 2022, 10:32 p.m.