Estimator: Sagemaker Estimator Class

In DyfanJones/sagemaker-r-mlcore: sagemaker machine learning core classes and methods

Sagemaker Estimator Class

Description

A generic Estimator to train using any supplied algorithm. This class is designed for use with algorithms that don't have their own, custom class.

Super class

sagemaker.mlcore::EstimatorBase -> Estimator

Public fields

.module

mimic python module

Methods

Public methods

• Estimator$new()

• Estimator$training_image_uri()

• Estimator$set_hyperparameters()

• Estimator$hyperparameters()

• Estimator$create_model()

• Estimator$clone()

Inherited methods

• sagemaker.mlcore::EstimatorBase$.prepare_for_training()

• sagemaker.mlcore::EstimatorBase$attach()

• sagemaker.mlcore::EstimatorBase$compile_model()

• sagemaker.mlcore::EstimatorBase$delete_endpoint()

• sagemaker.mlcore::EstimatorBase$deploy()

• sagemaker.mlcore::EstimatorBase$describe()

• sagemaker.mlcore::EstimatorBase$disable_profiling()

• sagemaker.mlcore::EstimatorBase$enable_default_profiling()

• sagemaker.mlcore::EstimatorBase$enable_network_isolation()

• sagemaker.mlcore::EstimatorBase$fit()

• sagemaker.mlcore::EstimatorBase$format()

• sagemaker.mlcore::EstimatorBase$get_vpc_config()

• sagemaker.mlcore::EstimatorBase$help()

• sagemaker.mlcore::EstimatorBase$latest_job_debugger_artifacts_path()

• sagemaker.mlcore::EstimatorBase$latest_job_profiler_artifacts_path()

• sagemaker.mlcore::EstimatorBase$latest_job_tensorboard_artifacts_path()

• sagemaker.mlcore::EstimatorBase$logs()

• sagemaker.mlcore::EstimatorBase$prepare_workflow_for_training()

• sagemaker.mlcore::EstimatorBase$register()

• sagemaker.mlcore::EstimatorBase$rule_job_summary()

• sagemaker.mlcore::EstimatorBase$transformer()

• sagemaker.mlcore::EstimatorBase$update_profiler()

• sagemaker.mlcore::EstimatorBase$wait()

---

Method new()

Initialize an “Estimator“ instance.

Usage

Estimator$new(
  image_uri,
  role,
  instance_count = NULL,
  instance_type = NULL,
  volume_size = 30,
  volume_kms_key = NULL,
  max_run = 24 * 60 * 60,
  input_mode = "File",
  output_path = NULL,
  output_kms_key = NULL,
  base_job_name = NULL,
  sagemaker_session = NULL,
  hyperparameters = NULL,
  tags = NULL,
  subnets = NULL,
  security_group_ids = NULL,
  model_uri = NULL,
  model_channel_name = "model",
  metric_definitions = NULL,
  encrypt_inter_container_traffic = FALSE,
  use_spot_instances = FALSE,
  max_wait = NULL,
  checkpoint_s3_uri = NULL,
  checkpoint_local_path = NULL,
  enable_network_isolation = FALSE,
  rules = NULL,
  debugger_hook_config = NULL,
  tensorboard_output_config = NULL,
  enable_sagemaker_metrics = NULL,
  profiler_config = NULL,
  disable_profiler = FALSE,
  environment = NULL,
  max_retry_attempts = NULL,
  ...
)

Arguments

image_uri

(str): The container image to use for training.

role

(str): An AWS IAM role (either name or full ARN). The Amazon SageMaker training jobs and APIs that create Amazon SageMaker endpoints use this role to access training data and model artifacts. After the endpoint is created, the inference code might use the IAM role, if it needs to access an AWS resource.

instance_count

(int): Number of Amazon EC2 instances to use for training.

instance_type

(str): Type of EC2 instance to use for training, for example, 'ml.c4.xlarge'.

volume_size

(int): Size in GB of the EBS volume to use for storing input data during training (default: 30). Must be large enough to store training data if File Mode is used (which is the default).

volume_kms_key

(str): Optional. KMS key ID for encrypting EBS volume attached to the training instance (default: NULL).

max_run

(int): Timeout in seconds for training (default: 24 * 60 * 60). After this amount of time Amazon SageMaker terminates the job regardless of its current status.

input_mode

(str): The input mode that the algorithm supports (default: 'File'). Valid modes: * 'File' - Amazon SageMaker copies the training dataset from the S3 location to a local directory. * 'Pipe' - Amazon SageMaker streams data directly from S3 to the container via a Unix-named pipe. This argument can be overriden on a per-channel basis using “TrainingInput.input_mode“.

output_path

(str): S3 location for saving the training result (model artifacts and output files). If not specified, results are stored to a default bucket. If the bucket with the specific name does not exist, the estimator creates the bucket during the :meth:'~sagemaker.estimator.EstimatorBase.fit' method execution.

output_kms_key

(str): Optional. KMS key ID for encrypting the training output (default: NULL).

base_job_name

(str): Prefix for training job name when the :meth:'~sagemaker.estimator.EstimatorBase.fit' method launches. If not specified, the estimator generates a default job name, based on the training image name and current timestamp.

sagemaker_session

(sagemaker.session.Session): Session object which manages interactions with Amazon SageMaker APIs and any other AWS services needed. If not specified, the estimator creates one using the default AWS configuration chain.

hyperparameters

(dict): Dictionary containing the hyperparameters to initialize this estimator with.

tags

(list[dict]): List of tags for labeling a training job. For more, see https://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.

subnets

(list[str]): List of subnet ids. If not specified training job will be created without VPC config.

security_group_ids

(list[str]): List of security group ids. If not specified training job will be created without VPC config.

model_uri

(str): URI where a pre-trained model is stored, either locally or in S3 (default: NULL). If specified, the estimator can download it. This model can be a 'model.tar.gz' from a previous training job, or other artifacts coming from a different source. In local mode, this should point to the path in which the model is located and not the file itself, as local Docker containers will try to mount the URI as a volume. More information: https://docs.aws.amazon.com/sagemaker/latest/dg/cdf-training.html#td-deserialization

model_channel_name

(str): Name of the channel where 'model_uri' will be downloaded (default: 'model').

metric_definitions

(list[dict]): A list of dictionaries that defines the metric(s) used to evaluate the training jobs. Each dictionary contains two keys: 'Name' for the name of the metric, and 'Regex' for the regular expression used to extract the metric from the logs. This should be defined only for jobs that don't use an Amazon algorithm.

encrypt_inter_container_traffic

(bool): Specifies whether traffic between training containers is encrypted for the training job (default: “False“).

use_spot_instances

(bool): Specifies whether to use SageMaker Managed Spot instances for training. If enabled then the 'max_wait' arg should also be set. More information: https://docs.aws.amazon.com/sagemaker/latest/dg/model-managed-spot-training.html (default: “False“).

max_wait

(int): Timeout in seconds waiting for spot training instances (default: NULL). After this amount of time Amazon SageMaker will stop waiting for Spot instances to become available (default: “NULL“).

checkpoint_s3_uri

(str): The S3 URI in which to persist checkpoints that the algorithm persists (if any) during training. (default: “NULL“).

checkpoint_local_path

(str): The local path that the algorithm writes its checkpoints to. SageMaker will persist all files under this path to 'checkpoint_s3_uri' continually during training. On job startup the reverse happens - data from the s3 location is downloaded to this path before the algorithm is started. If the path is unset then SageMaker assumes the checkpoints will be provided under '/opt/ml/checkpoints/'. (default: “NULL“).

enable_network_isolation

(bool): Specifies whether container will run in network isolation mode (default: “False“). Network isolation mode restricts the container access to outside networks (such as the Internet). The container does not make any inbound or outbound network calls. Also known as Internet-free mode.

rules

(list[:class:'~sagemaker.debugger.Rule']): A list of :class:'~sagemaker.debugger.Rule' objects used to define rules for continuous analysis with SageMaker Debugger (default: “NULL“). For more, see https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_debugger.html#continuous-analyses-through-rules

debugger_hook_config

(:class:'~sagemaker.debugger.DebuggerHookConfig' or bool): Configuration for how debugging information is emitted with SageMaker Debugger. If not specified, a default one is created using the estimator's “output_path“, unless the region does not support SageMaker Debugger. To disable SageMaker Debugger, set this parameter to “False“. For more, see https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_debugger.html

tensorboard_output_config

(:class:'~sagemaker.debugger.TensorBoardOutputConfig'): Configuration for customizing debugging visualization using TensorBoard (default: “NULL“). For more, see https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_debugger.html#capture-real-time-tensorboard-data-from-the-debugging-hook

enable_sagemaker_metrics

(bool): enable SageMaker Metrics Time Series. For more information see: https://docs.aws.amazon.com/sagemaker/latest/dg/API_AlgorithmSpecification.html#SageMaker-Type-AlgorithmSpecification-EnableSageMakerMetricsTimeSeries (default: “NULL“).

profiler_config

(:class:'~sagemaker.debugger.ProfilerConfig'): Configuration for how SageMaker Debugger collects monitoring and profiling information from your training job. If not specified, Debugger will be configured with a default configuration and will save system and framework metrics the estimator's default “output_path“ in Amazon S3. Use :class:'~sagemaker.debugger.ProfilerConfig' to configure this parameter. To disable SageMaker Debugger monitoring and profiling, set the “disable_profiler“ parameter to “True“.

disable_profiler

(bool): Specifies whether Debugger monitoring and profiling will be disabled (default: “False“).

environment

(dict[str, str]) : Environment variables to be set for use during training job (default: “None“)

max_retry_attempts

(int): The number of times to move a job to the STARTING status. You can specify between 1 and 30 attempts. If the value of attempts is greater than zero, the job is retried on InternalServerFailure the same number of attempts as the value. You can cap the total duration for your job by setting “max_wait“ and “max_run“ (default: “None“)

...

: additional arguements for parent class 'EstimatorBase'.

---

Method training_image_uri()

Returns the docker image to use for training. The fit() method, that does the model training, calls this method to find the image to use for model training.

Usage

Estimator$training_image_uri()

---

Method set_hyperparameters()

formats hyperparameters for model tunning

Usage

Estimator$set_hyperparameters(...)

Arguments

...

model hyperparameters

---

Method hyperparameters()

Returns the hyperparameters as a dictionary to use for training. The fit() method, that does the model training, calls this method to find the hyperparameters you specified.

Usage

Estimator$hyperparameters()

---

Method create_model()

Create a model to deploy. The serializer, deserializer, content_type, and accept arguments are only used to define a default Predictor. They are ignored if an explicit predictor class is passed in. Other arguments are passed through to the Model class.

Usage

Estimator$create_model(
  role = NULL,
  image_uri = NULL,
  predictor_cls = NULL,
  vpc_config_override = "VPC_CONFIG_DEFAULT",
  ...
)

Arguments

role

(str): The “ExecutionRoleArn“ IAM Role ARN for the “Model“, which is also used during transform jobs. If not specified, the role from the Estimator will be used.

image_uri

(str): An container image to use for deploying the model. Defaults to the image used for training.

predictor_cls

(Predictor): The predictor class to use when deploying the model.

vpc_config_override

(dict[str, list[str]]): Optional override for VpcConfig set on the model. Default: use subnets and security groups from this Estimator. * 'Subnets' (list[str]): List of subnet ids. * 'SecurityGroupIds' (list[str]): List of security group ids.

...

: Additional parameters passed to :class:'~sagemaker.model.Model' .. tip:: You can find additional parameters for using this method at :class:'~sagemaker.model.Model'.

Returns

(sagemaker.model.Model) a Model ready for deployment.

---

Method clone()

The objects of this class are cloneable with this method.

Usage

Estimator$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

DyfanJones/sagemaker-r-mlcore documentation built on May 3, 2022, 10:08 a.m.