AlgorithmEstimator: AlgorithmEstimator Class
In DyfanJones/sagemaker-r-mlcore: sagemaker machine learning core classes and methods
AlgorithmEstimator R Documentation
AlgorithmEstimator Class
Description
A generic Estimator to train using any algorithm object (with an
“algorithm_arn“). The Algorithm can be your own, or any Algorithm from AWS
Marketplace that you have a valid subscription for. This class will perform
client-side validation on all the inputs.
Super class
sagemaker.mlcore::EstimatorBase -> AlgorithmEstimator
Public fields
.hyperpameters_with_rangeThese Hyperparameter Types have a range definition.
Methods
Public methods
-
-
-
-
-
-
-
-
-
-
-
Inherited methods
Method new()
Initialize an “AlgorithmEstimator“ instance.
Usage
AlgorithmEstimator$new(
algorithm_arn,
role,
instance_count,
instance_type,
volume_size = 30,
volume_kms_key = NULL,
max_run = 24 * 60 * 60,
input_mode = "File",
output_path = NULL,
output_kms_key = NULL,
base_job_name = NULL,
sagemaker_session = NULL,
hyperparameters = NULL,
tags = NULL,
subnets = NULL,
security_group_ids = NULL,
model_uri = NULL,
model_channel_name = "model",
metric_definitions = NULL,
encrypt_inter_container_traffic = FALSE,
...
)
Arguments
algorithm_arn(str): algorithm arn used for training. Can be just the name if your
account owns the algorithm.
role(str): An AWS IAM role (either name or full ARN). The Amazon SageMaker
training jobs and APIs that create Amazon SageMaker endpoints use this role to
access training data and model artifacts. After the endpoint
is created, the inference code might use the IAM role, if it
needs to access an AWS resource.
instance_count(int): Number of Amazon EC2 instances to
use for training.
instance_type(str): Type of EC2
instance to use for training, for example, 'ml.c4.xlarge'.
volume_size(int): Size in GB of the EBS volume to use for
storing input data during training (default: 30). Must be large enough to store
training data if File Mode is used (which is the default).
volume_kms_key(str): Optional. KMS key ID for encrypting EBS volume attached
to the training instance (default: NULL).
max_run(int): Timeout in seconds for training (default: 24 * 60 * 60).
After this amount of time Amazon SageMaker terminates the
job regardless of its current status.
input_mode(str): The input mode that the algorithm supports
(default: 'File'). Valid modes:
* 'File' - Amazon SageMaker copies the training dataset from
the S3 location to a local directory.
* 'Pipe' - Amazon SageMaker streams data directly from S3 to
the container via a Unix-named pipe.
This argument can be overriden on a per-channel basis using
“TrainingInput.input_mode“.
output_path(str): S3 location for saving the training result (model artifacts and
output files). If not specified, results are stored to a default bucket. If
the bucket with the specific name does not exist, the
estimator creates the bucket during the
:meth:'~sagemaker.estimator.EstimatorBase.fit' method
execution.
output_kms_key(str): Optional. KMS key ID for encrypting the
training output (default: NULL).
base_job_name(str): Prefix for
training job name when the
:meth:'~sagemaker.estimator.EstimatorBase.fit'
method launches. If not specified, the estimator generates a
default job name, based on the training image name and
current timestamp.
sagemaker_session(sagemaker.session.Session): Session object which manages
interactions with Amazon SageMaker APIs and any other AWS services needed. If
not specified, the estimator creates one using the default
AWS configuration chain.
hyperparameters(dict): Dictionary containing the hyperparameters to
initialize this estimator with.
tags(list[dict]): List of tags for labeling a training job. For more, see
https://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.
subnets(list[str]): List of subnet ids. If not specified
training job will be created without VPC config.
security_group_ids(list[str]): List of security group ids. If
not specified training job will be created without VPC config.
model_uri(str): URI where a pre-trained model is stored, either locally or in S3
(default: NULL). If specified, the estimator will create a channel pointing to
the model so the training job can download it. This model
can be a 'model.tar.gz' from a previous training job, or
other artifacts coming from a different source.
More information:
https://docs.aws.amazon.com/sagemaker/latest/dg/cdf-training.html#td-deserialization
model_channel_name(str): Name of the channel where 'model_uri'
will be downloaded (default: 'model').
metric_definitions(list[dict]): A list of dictionaries that defines the metric(s)
used to evaluate the training jobs. Each dictionary contains two keys: 'Name' for
the name of the metric, and 'Regex' for the regular
expression used to extract the metric from the logs.
encrypt_inter_container_traffic(bool): Specifies whether traffic between training
containers is encrypted for the training job (default: “False“).
...: Additional kwargs. This is unused. It's only added for AlgorithmEstimator
to ignore the irrelevant arguments.
Method validate_train_spec()
Placeholder docstring
Usage
AlgorithmEstimator$validate_train_spec()
Method set_hyperparameter()
formats hyperparameters for model tunning
Usage
AlgorithmEstimator$set_hyperparameter(...)
Arguments
...model hyperparameters
Method hyperparameters()
Returns the hyperparameters as a dictionary to use for training.
The fit() method, that does the model training, calls this method to
find the hyperparameters you specified.
Usage
AlgorithmEstimator$hyperparameters()
Method training_image_uri()
Returns the docker image to use for training.
The fit() method, that does the model training, calls this method to
find the image to use for model training.
Usage
AlgorithmEstimator$training_image_uri()
Method enable_network_isolation()
Return True if this Estimator will need network isolation to run.
On Algorithm Estimators this depends on the algorithm being used. If
this is algorithm owned by your account it will be False. If this is an
an algorithm consumed from Marketplace it will be True.
Usage
AlgorithmEstimator$enable_network_isolation()
Returns
bool: Whether this Estimator needs network isolation or not.
Method create_model()
Create a model to deploy.
The serializer, deserializer, content_type, and accept arguments are
only used to define a default Predictor They are ignored if an
explicit predictor class is passed in. Other arguments are passed
through to the Model class.
Usage
AlgorithmEstimator$create_model(
role = NULL,
predictor_cls = NULL,
serializer = IdentitySerializer$new(),
deserializer = BytesDeserializer$new(),
vpc_config_override = "VPC_CONFIG_DEFAULT",
...
)
Arguments
role(str): The “ExecutionRoleArn“ IAM Role ARN for the “Model“,
which is also used during transform jobs. If not specified, the
role from the Estimator will be used.
predictor_cls(RealTimePredictor): The predictor class to use when
deploying the model.
serializer(callable): Should accept a single argument, the input
data, and return a sequence of bytes. May provide a content_type
attribute that defines the endpoint request content type
deserializer(callable): Should accept two arguments, the result
data and the response content type, and return a sequence of
bytes. May provide a content_type attribute that defines the
endpoint response Accept content type.
vpc_config_override(dict[str, list[str]]): Optional override for VpcConfig set on
the model. Default: use subnets and security groups from this Estimator.
* 'Subnets' (list[str]): List of subnet ids.
* 'SecurityGroupIds' (list[str]): List of security group ids.
...: Additional arguments for creating a :class:'~sagemaker.model.ModelPackage'.
.. tip::
You can find additional parameters for using this method at
:class:'~sagemaker.model.ModelPackage' and
:class:'~sagemaker.model.Model'.
Returns
a Model ready for deployment.
Method transformer()
Return a “Transformer“ that uses a SageMaker Model based on the
training job. It reuses the SageMaker Session and base job name used by
the Estimator.
Usage
AlgorithmEstimator$transformer(
instance_count,
instance_type,
strategy = NULL,
assemble_with = NULL,
output_path = NULL,
output_kms_key = NULL,
accept = NULL,
env = NULL,
max_concurrent_transforms = NULL,
max_payload = NULL,
tags = NULL,
role = NULL,
volume_kms_key = NULL
)
Arguments
instance_count(int): Number of EC2 instances to use.
instance_type(str): Type of EC2 instance to use, for example,
'ml.c4.xlarge'.
strategy(str): The strategy used to decide how to batch records in
a single request (default: None). Valid values: 'MultiRecord'
and 'SingleRecord'.
assemble_with(str): How the output is assembled (default: None).
Valid values: 'Line' or 'None'.
output_path(str): S3 location for saving the transform result. If
not specified, results are stored to a default bucket.
output_kms_key(str): Optional. KMS key ID for encrypting the
transform output (default: None).
accept(str): The accept header passed by the client to
the inference endpoint. If it is supported by the endpoint,
it will be the format of the batch transform output.
env(dict): Environment variables to be set for use during the
transform job (default: None).
max_concurrent_transforms(int): The maximum number of HTTP requests
to be made to each individual transform container at one time.
max_payload(int): Maximum size of the payload in a single HTTP
request to the container in MB.
tags(list[dict]): List of tags for labeling a transform job. If
none specified, then the tags used for the training job are used
for the transform job.
role(str): The “ExecutionRoleArn“ IAM Role ARN for the “Model“,
which is also used during transform jobs. If not specified, the
role from the Estimator will be used.
volume_kms_key(str): Optional. KMS key ID for encrypting the volume
attached to the ML compute instance (default: None).
Method fit()
Train a model using the input training dataset.
The API calls the Amazon SageMaker CreateTrainingJob API to start
model training. The API uses configuration you provided to create the
estimator and the specified input training data to send the
CreatingTrainingJob request to Amazon SageMaker.
This is a synchronous operation. After the model training
successfully completes, you can call the “deploy()“ method to host the
model using the Amazon SageMaker hosting services.
Usage
AlgorithmEstimator$fit(
inputs = NULL,
wait = TRUE,
logs = TRUE,
job_name = NULL
)
Arguments
inputs(str or dict or TrainingInput): Information
about the training data. This can be one of three types:
(str) the S3 location where training data is saved, or a file:// path in
local mode.
(dict[str, str] or dict[str, TrainingInput]) If using multiple
channels for training data, you can specify a dict mapping channel names to
strings or :func:'~TrainingInput' objects.
(TrainingInput) - channel configuration for S3 data sources that can
provide additional information as well as the path to the training dataset.
See :func:'TrainingInput' for full details.
(sagemaker.session.FileSystemInput) - channel configuration for
a file system data source that can provide additional information as well as
the path to the training dataset.
wait(bool): Whether the call should wait until the job completes (default: True).
logs([str]): A list of strings specifying which logs to print. Acceptable
strings are "All", "NULL", "Training", or "Rules". To maintain backwards
compatibility, boolean values are also accepted and converted to strings.
Only meaningful when wait is True.
job_name(str): Training job name. If not specified, the estimator generates
a default job name, based on the training image name and current timestamp.
experiment_config(dict[str, str]): Experiment management configuration.
Dictionary contains three optional keys,
'ExperimentName', 'TrialName', and 'TrialComponentDisplayName'.
Method print()
Printer.
Usage
AlgorithmEstimator$print(...)
Arguments
...(ignored).
Method clone()
The objects of this class are cloneable with this method.
Usage
AlgorithmEstimator$clone(deep = FALSE)
Arguments
deepWhether to make a deep clone.
DyfanJones/sagemaker-r-mlcore documentation built on May 3, 2022, 10:08 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| AlgorithmEstimator | R Documentation |
AlgorithmEstimator Class
Description
A generic Estimator to train using any algorithm object (with an “algorithm_arn“). The Algorithm can be your own, or any Algorithm from AWS Marketplace that you have a valid subscription for. This class will perform client-side validation on all the inputs.
Super class
sagemaker.mlcore::EstimatorBase -> AlgorithmEstimator
Public fields
.hyperpameters_with_rangeThese Hyperparameter Types have a range definition.
Methods
Public methods
Inherited methods
Method new()
Initialize an “AlgorithmEstimator“ instance.
Usage
AlgorithmEstimator$new( algorithm_arn, role, instance_count, instance_type, volume_size = 30, volume_kms_key = NULL, max_run = 24 * 60 * 60, input_mode = "File", output_path = NULL, output_kms_key = NULL, base_job_name = NULL, sagemaker_session = NULL, hyperparameters = NULL, tags = NULL, subnets = NULL, security_group_ids = NULL, model_uri = NULL, model_channel_name = "model", metric_definitions = NULL, encrypt_inter_container_traffic = FALSE, ... )
Arguments
algorithm_arn(str): algorithm arn used for training. Can be just the name if your account owns the algorithm.
role(str): An AWS IAM role (either name or full ARN). The Amazon SageMaker training jobs and APIs that create Amazon SageMaker endpoints use this role to access training data and model artifacts. After the endpoint is created, the inference code might use the IAM role, if it needs to access an AWS resource.
instance_count(int): Number of Amazon EC2 instances to use for training.
instance_type(str): Type of EC2 instance to use for training, for example, 'ml.c4.xlarge'.
volume_size(int): Size in GB of the EBS volume to use for storing input data during training (default: 30). Must be large enough to store training data if File Mode is used (which is the default).
volume_kms_key(str): Optional. KMS key ID for encrypting EBS volume attached to the training instance (default: NULL).
max_run(int): Timeout in seconds for training (default: 24 * 60 * 60). After this amount of time Amazon SageMaker terminates the job regardless of its current status.
input_mode(str): The input mode that the algorithm supports (default: 'File'). Valid modes: * 'File' - Amazon SageMaker copies the training dataset from the S3 location to a local directory. * 'Pipe' - Amazon SageMaker streams data directly from S3 to the container via a Unix-named pipe. This argument can be overriden on a per-channel basis using “TrainingInput.input_mode“.
output_path(str): S3 location for saving the training result (model artifacts and output files). If not specified, results are stored to a default bucket. If the bucket with the specific name does not exist, the estimator creates the bucket during the :meth:'~sagemaker.estimator.EstimatorBase.fit' method execution.
output_kms_key(str): Optional. KMS key ID for encrypting the training output (default: NULL).
base_job_name(str): Prefix for training job name when the :meth:'~sagemaker.estimator.EstimatorBase.fit' method launches. If not specified, the estimator generates a default job name, based on the training image name and current timestamp.
sagemaker_session(sagemaker.session.Session): Session object which manages interactions with Amazon SageMaker APIs and any other AWS services needed. If not specified, the estimator creates one using the default AWS configuration chain.
hyperparameters(dict): Dictionary containing the hyperparameters to initialize this estimator with.
tags(list[dict]): List of tags for labeling a training job. For more, see https://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.
subnets(list[str]): List of subnet ids. If not specified training job will be created without VPC config.
security_group_ids(list[str]): List of security group ids. If not specified training job will be created without VPC config.
model_uri(str): URI where a pre-trained model is stored, either locally or in S3 (default: NULL). If specified, the estimator will create a channel pointing to the model so the training job can download it. This model can be a 'model.tar.gz' from a previous training job, or other artifacts coming from a different source. More information: https://docs.aws.amazon.com/sagemaker/latest/dg/cdf-training.html#td-deserialization
model_channel_name(str): Name of the channel where 'model_uri' will be downloaded (default: 'model').
metric_definitions(list[dict]): A list of dictionaries that defines the metric(s) used to evaluate the training jobs. Each dictionary contains two keys: 'Name' for the name of the metric, and 'Regex' for the regular expression used to extract the metric from the logs.
encrypt_inter_container_traffic(bool): Specifies whether traffic between training containers is encrypted for the training job (default: “False“).
...: Additional kwargs. This is unused. It's only added for AlgorithmEstimator to ignore the irrelevant arguments.
Method validate_train_spec()
Placeholder docstring
Usage
AlgorithmEstimator$validate_train_spec()
Method set_hyperparameter()
formats hyperparameters for model tunning
Usage
AlgorithmEstimator$set_hyperparameter(...)
Arguments
...model hyperparameters
Method hyperparameters()
Returns the hyperparameters as a dictionary to use for training. The fit() method, that does the model training, calls this method to find the hyperparameters you specified.
Usage
AlgorithmEstimator$hyperparameters()
Method training_image_uri()
Returns the docker image to use for training. The fit() method, that does the model training, calls this method to find the image to use for model training.
Usage
AlgorithmEstimator$training_image_uri()
Method enable_network_isolation()
Return True if this Estimator will need network isolation to run. On Algorithm Estimators this depends on the algorithm being used. If this is algorithm owned by your account it will be False. If this is an an algorithm consumed from Marketplace it will be True.
Usage
AlgorithmEstimator$enable_network_isolation()
Returns
bool: Whether this Estimator needs network isolation or not.
Method create_model()
Create a model to deploy. The serializer, deserializer, content_type, and accept arguments are only used to define a default Predictor They are ignored if an explicit predictor class is passed in. Other arguments are passed through to the Model class.
Usage
AlgorithmEstimator$create_model( role = NULL, predictor_cls = NULL, serializer = IdentitySerializer$new(), deserializer = BytesDeserializer$new(), vpc_config_override = "VPC_CONFIG_DEFAULT", ... )
Arguments
role(str): The “ExecutionRoleArn“ IAM Role ARN for the “Model“, which is also used during transform jobs. If not specified, the role from the Estimator will be used.
predictor_cls(RealTimePredictor): The predictor class to use when deploying the model.
serializer(callable): Should accept a single argument, the input data, and return a sequence of bytes. May provide a content_type attribute that defines the endpoint request content type
deserializer(callable): Should accept two arguments, the result data and the response content type, and return a sequence of bytes. May provide a content_type attribute that defines the endpoint response Accept content type.
vpc_config_override(dict[str, list[str]]): Optional override for VpcConfig set on the model. Default: use subnets and security groups from this Estimator. * 'Subnets' (list[str]): List of subnet ids. * 'SecurityGroupIds' (list[str]): List of security group ids.
...: Additional arguments for creating a :class:'~sagemaker.model.ModelPackage'. .. tip:: You can find additional parameters for using this method at :class:'~sagemaker.model.ModelPackage' and :class:'~sagemaker.model.Model'.
Returns
a Model ready for deployment.
Method transformer()
Return a “Transformer“ that uses a SageMaker Model based on the training job. It reuses the SageMaker Session and base job name used by the Estimator.
Usage
AlgorithmEstimator$transformer( instance_count, instance_type, strategy = NULL, assemble_with = NULL, output_path = NULL, output_kms_key = NULL, accept = NULL, env = NULL, max_concurrent_transforms = NULL, max_payload = NULL, tags = NULL, role = NULL, volume_kms_key = NULL )
Arguments
instance_count(int): Number of EC2 instances to use.
instance_type(str): Type of EC2 instance to use, for example, 'ml.c4.xlarge'.
strategy(str): The strategy used to decide how to batch records in a single request (default: None). Valid values: 'MultiRecord' and 'SingleRecord'.
assemble_with(str): How the output is assembled (default: None). Valid values: 'Line' or 'None'.
output_path(str): S3 location for saving the transform result. If not specified, results are stored to a default bucket.
output_kms_key(str): Optional. KMS key ID for encrypting the transform output (default: None).
accept(str): The accept header passed by the client to the inference endpoint. If it is supported by the endpoint, it will be the format of the batch transform output.
env(dict): Environment variables to be set for use during the transform job (default: None).
max_concurrent_transforms(int): The maximum number of HTTP requests to be made to each individual transform container at one time.
max_payload(int): Maximum size of the payload in a single HTTP request to the container in MB.
tags(list[dict]): List of tags for labeling a transform job. If none specified, then the tags used for the training job are used for the transform job.
role(str): The “ExecutionRoleArn“ IAM Role ARN for the “Model“, which is also used during transform jobs. If not specified, the role from the Estimator will be used.
volume_kms_key(str): Optional. KMS key ID for encrypting the volume attached to the ML compute instance (default: None).
Method fit()
Train a model using the input training dataset. The API calls the Amazon SageMaker CreateTrainingJob API to start model training. The API uses configuration you provided to create the estimator and the specified input training data to send the CreatingTrainingJob request to Amazon SageMaker. This is a synchronous operation. After the model training successfully completes, you can call the “deploy()“ method to host the model using the Amazon SageMaker hosting services.
Usage
AlgorithmEstimator$fit( inputs = NULL, wait = TRUE, logs = TRUE, job_name = NULL )
Arguments
inputs(str or dict or TrainingInput): Information about the training data. This can be one of three types:
(str) the S3 location where training data is saved, or a file:// path in local mode.
(dict[str, str] or dict[str, TrainingInput]) If using multiple channels for training data, you can specify a dict mapping channel names to strings or :func:'~TrainingInput' objects.
(TrainingInput) - channel configuration for S3 data sources that can provide additional information as well as the path to the training dataset. See :func:'TrainingInput' for full details.
(sagemaker.session.FileSystemInput) - channel configuration for a file system data source that can provide additional information as well as the path to the training dataset.
wait(bool): Whether the call should wait until the job completes (default: True).
logs([str]): A list of strings specifying which logs to print. Acceptable strings are "All", "NULL", "Training", or "Rules". To maintain backwards compatibility, boolean values are also accepted and converted to strings. Only meaningful when wait is True.
job_name(str): Training job name. If not specified, the estimator generates a default job name, based on the training image name and current timestamp.
experiment_config(dict[str, str]): Experiment management configuration. Dictionary contains three optional keys, 'ExperimentName', 'TrialName', and 'TrialComponentDisplayName'.
Method print()
Printer.
Usage
AlgorithmEstimator$print(...)
Arguments
...(ignored).
Method clone()
The objects of this class are cloneable with this method.
Usage
AlgorithmEstimator$clone(deep = FALSE)
Arguments
deepWhether to make a deep clone.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.