EstimatorBase: Handle end-to-end Amazon SageMaker training and deployment...
In DyfanJones/sagemaker-r-mlcore: sagemaker machine learning core classes and methods
EstimatorBase R Documentation
Handle end-to-end Amazon SageMaker training and deployment tasks.
Description
For introduction to model training and deployment, see
http://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works-training.html
Subclasses must define a way to determine what image to use for training,
what hyperparameters to use, and how to create an appropriate predictor
instance.
Active bindings
model_dataThe model location in S3. Only set if Estimator has been “fit()“.
training_job_analyticsReturn a “TrainingJobAnalytics“ object for the current training job.
Methods
Public methods
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Method new()
Initialize an “EstimatorBase“ instance.
Usage
EstimatorBase$new(
role,
instance_count = NULL,
instance_type = NULL,
volume_size = 30,
volume_kms_key = NULL,
max_run = 24 * 60 * 60,
input_mode = "File",
output_path = NULL,
output_kms_key = NULL,
base_job_name = NULL,
sagemaker_session = NULL,
tags = NULL,
subnets = NULL,
security_group_ids = NULL,
model_uri = NULL,
model_channel_name = "model",
metric_definitions = NULL,
encrypt_inter_container_traffic = FALSE,
use_spot_instances = FALSE,
max_wait = NULL,
checkpoint_s3_uri = NULL,
checkpoint_local_path = NULL,
rules = NULL,
debugger_hook_config = NULL,
tensorboard_output_config = NULL,
enable_sagemaker_metrics = NULL,
enable_network_isolation = FALSE,
profiler_config = NULL,
disable_profiler = FALSE,
environment = NULL,
max_retry_attempts = NULL,
source_dir = NULL,
git_config = NULL,
hyperparameters = NULL,
container_log_level = "INFO",
code_location = NULL,
entry_point = NULL,
dependencies = NULL,
...
)
Arguments
role(str): An AWS IAM role (either name or full ARN). The Amazon
SageMaker training jobs and APIs that create Amazon SageMaker
endpoints use this role to access training data and model
artifacts. After the endpoint is created, the inference code
might use the IAM role, if it needs to access an AWS resource.
instance_count(int): Number of Amazon EC2 instances to use
for training.
instance_type(str): Type of EC2 instance to use for training,
for example, 'ml.c4.xlarge'.
volume_size(int): Size in GB of the EBS volume to use for
storing input data during training (default: 30). Must be large
enough to store training data if File Mode is used (which is the
default).
volume_kms_key(str): Optional. KMS key ID for encrypting EBS
volume attached to the training instance (default: NULL).
max_run(int): Timeout in seconds for training (default: 24 *
60 * 60). After this amount of time Amazon SageMaker terminates
the job regardless of its current status.
input_mode(str): The input mode that the algorithm supports
(default: 'File'). Valid modes: 'File' - Amazon SageMaker copies
the training dataset from the S3 location to a local directory.
'Pipe' - Amazon SageMaker streams data directly from S3 to the
container via a Unix-named pipe. This argument can be overriden
on a per-channel basis using
“TrainingInput.input_mode“.
output_path(str): S3 location for saving the training result (model
artifacts and output files). If not specified, results are
stored to a default bucket. If the bucket with the specific name
does not exist, the estimator creates the bucket during the
:meth:'~sagemaker.estimator.EstimatorBase.fit' method execution.
file:// urls are used for local mode. For example: 'file://model/'
will save to the model folder in the current directory.
output_kms_key(str): Optional. KMS key ID for encrypting the
training output (default: NULL).
base_job_name(str): Prefix for training job name when the
:meth:'~sagemaker.estimator.EstimatorBase.fit' method launches.
If not specified, the estimator generates a default job name,
based on the training image name and current timestamp.
sagemaker_session(sagemaker.session.Session): Session object which
manages interactions with Amazon SageMaker APIs and any other
AWS services needed. If not specified, the estimator creates one
using the default AWS configuration chain.
tags(list[dict]): List of tags for labeling a training job. For
more, see
https://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.
subnets(list[str]): List of subnet ids. If not specified training
job will be created without VPC config.
security_group_ids(list[str]): List of security group ids. If not
specified training job will be created without VPC config.
model_uri(str): URI where a pre-trained model is stored, either
locally or in S3 (default: NULL). If specified, the estimator
will create a channel pointing to the model so the training job
can download it. This model can be a 'model.tar.gz' from a
previous training job, or other artifacts coming from a
different source.
In local mode, this should point to the path in which the model
is located and not the file itself, as local Docker containers
will try to mount the URI as a volume.
More information:
https://docs.aws.amazon.com/sagemaker/latest/dg/cdf-training.html#td-deserialization
model_channel_name(str): Name of the channel where 'model_uri' will
be downloaded (default: 'model').
metric_definitions(list[dict]): A list of dictionaries that defines
the metric(s) used to evaluate the training jobs. Each
dictionary contains two keys: 'Name' for the name of the metric,
and 'Regex' for the regular expression used to extract the
metric from the logs. This should be defined only for jobs that
don't use an Amazon algorithm.
encrypt_inter_container_traffic(bool): Specifies whether traffic
between training containers is encrypted for the training job
(default: “False“).
use_spot_instances(bool): Specifies whether to use SageMaker
Managed Spot instances for training. If enabled then the
'max_wait' arg should also be set.
More information:
https://docs.aws.amazon.com/sagemaker/latest/dg/model-managed-spot-training.html
(default: “False“).
max_wait(int): Timeout in seconds waiting for spot training
instances (default: NULL). After this amount of time Amazon
SageMaker will stop waiting for Spot instances to become
available (default: “NULL“).
checkpoint_s3_uri(str): The S3 URI in which to persist checkpoints
that the algorithm persists (if any) during training. (default:
“NULL“).
checkpoint_local_path(str): The local path that the algorithm
writes its checkpoints to. SageMaker will persist all files
under this path to 'checkpoint_s3_uri' continually during
training. On job startup the reverse happens - data from the
s3 location is downloaded to this path before the algorithm is
started. If the path is unset then SageMaker assumes the
checkpoints will be provided under '/opt/ml/checkpoints/'.
(default: “NULL“).
rules(list[:class:'~sagemaker.debugger.Rule']): A list of
:class:'~sagemaker.debugger.Rule' objects used to define
rules for continuous analysis with SageMaker Debugger
(default: “NULL“). For more, see
https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_debugger.html#continuous-analyses-through-rules
debugger_hook_config(:class:'~sagemaker.debugger.DebuggerHookConfig' or bool):
Configuration for how debugging information is emitted with
SageMaker Debugger. If not specified, a default one is created using
the estimator's “output_path“, unless the region does not
support SageMaker Debugger. To disable SageMaker Debugger,
set this parameter to “False“. For more, see
https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_debugger.html
tensorboard_output_config(:class:'~sagemaker.debugger.TensorBoardOutputConfig'):
Configuration for customizing debugging visualization using TensorBoard
(default: “NULL“). For more, see
https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_debugger.html#capture-real-time-tensorboard-data-from-the-debugging-hook
enable_sagemaker_metrics(bool): Enables SageMaker Metrics Time
Series. For more information see:
https://docs.aws.amazon.com/sagemaker/latest/dg/API_AlgorithmSpecification.html#SageMaker-Type-AlgorithmSpecification-EnableSageMakerMetricsTimeSeries
(default: “NULL“).
enable_network_isolation(bool): Specifies whether container will
run in network isolation mode (default: “False“). Network
isolation mode restricts the container access to outside networks
(such as the Internet). The container does not make any inbound or
outbound network calls. Also known as Internet-free mode.
profiler_config(:class:'~sagemaker.debugger.ProfilerConfig'):
Configuration for how SageMaker Debugger collects
monitoring and profiling information from your training job.
If not specified, a default configuration is created using
the estimator's “output_path“, unless the region does not
support SageMaker Debugger. To disable SageMaker Debugger
monitoring and profiling, set the
“disable_profiler“ parameter to “True“.
disable_profiler(bool): Specifies whether Debugger monitoring and profiling
will be disabled (default: “False“).
environment(dict[str, str]) : Environment variables to be set for
use during training job (default: “None“)
max_retry_attempts(int): The number of times to move a job to the STARTING status.
You can specify between 1 and 30 attempts.
If the value of attempts is greater than zero,
the job is retried on InternalServerFailure
the same number of attempts as the value.
You can cap the total duration for your job by setting “max_wait“ and “max_run“
(default: “None“)
source_dir(str): The absolute, relative, or S3 URI Path to a directory
with any other training source code dependencies aside from the entry
point file (default: None). If “source_dir“ is an S3 URI, it must
point to a tar.gz file. The structure within this directory is preserved
when training on Amazon SageMaker. If 'git_config' is provided,
'source_dir' should be a relative location to a directory in the Git
repo. With the following GitHub repo directory structure,
if you need 'train.py' as the entry point and 'test.py' as
the training source code, you can assign
entry_point='train.py' and source_dir='src'.
git_config(dict[str, str]): Git configurations used for cloning
files, including “repo“, “branch“, “commit“,
“2FA_enabled“, “username“, “password“, and “token“. The
“repo“ field is required. All other fields are optional.
“repo“ specifies the Git repository where your training script
is stored. If you don't provide “branch“, the default value
'master' is used. If you don't provide “commit“, the latest
commit in the specified branch is used.
results in cloning the repo specified in 'repo', then
checking out the 'master' branch, and checking out the specified
commit.
“2FA_enabled“, “username“, “password“, and “token“ are
used for authentication. For GitHub (or other Git) accounts, set
“2FA_enabled“ to 'True' if two-factor authentication is
enabled for the account, otherwise set it to 'False'. If you do
not provide a value for “2FA_enabled“, a default value of
'False' is used. CodeCommit does not support two-factor
authentication, so do not provide "2FA_enabled" with CodeCommit
repositories.
For GitHub and other Git repos, when SSH URLs are provided, it
doesn't matter whether 2FA is enabled or disabled. You should
either have no passphrase for the SSH key pairs or have the
ssh-agent configured so that you will not be prompted for the SSH
passphrase when you run the 'git clone' command with SSH URLs. When
HTTPS URLs are provided, if 2FA is disabled, then either “token“
or “username“ and “password“ are be used for authentication if provided.
“Token“ is prioritized. If 2FA is enabled, only “token“ is used
for authentication if provided. If required authentication info
is not provided, the SageMaker Python SDK attempts to use local credentials
to authenticate. If that fails, an error message is thrown.
For CodeCommit repos, 2FA is not supported, so '2FA_enabled'
should not be provided. There is no token in CodeCommit, so
“token“ should also not be provided. When “repo“ is an SSH URL,
the requirements are the same as GitHub repos. When “repo“
is an HTTPS URL, “username“ and “password“ are used for
authentication if they are provided. If they are not provided,
the SageMaker Python SDK attempts to use either the CodeCommit
credential helper or local credential storage for authentication.
hyperparameters(dict): A dictionary containing the hyperparameters to
initialize this estimator with. (Default: None).
container_log_level(str): The log level to use within the container
(default: logging.INFO). Valid values are defined in the Python
logging module.
code_location(str): The S3 prefix URI where custom code is
uploaded (default: None). You must not include a trailing slash because
a string prepended with a "/" is appended to “code_location“. The code
file uploaded to S3 is 'code_location/job-name/source/sourcedir.tar.gz'.
If not specified, the default “code location“ is 's3://output_bucket/job-name/'.
entry_point(str): The absolute or relative path to the local Python
source file that should be executed as the entry point to
training. (Default: None). If “source_dir“ is specified, then “entry_point“
must point to a file located at the root of “source_dir“.
If 'git_config' is provided, 'entry_point' should be
a relative location to the Python source file in the Git repo.
You can assign entry_point='src/train.py'.
dependencies(list[str]): A list of absolute or relative paths to directories
with any additional libraries that should be exported
to the container (default: []). The library folders are
copied to SageMaker in the same folder where the entrypoint is
copied. If 'git_config' is provided, 'dependencies' should be a
list of relative locations to directories with any additional
libraries needed in the Git repo.
This is not supported with "local code" in Local Mode.
...: update any deprecated parameters passed into class.
Method help()
Return class documentation
Usage
EstimatorBase$help()
Method training_image_uri()
Return the Docker image to use for training.
The :meth:'~sagemaker.estimator.EstimatorBase.fit' method, which does
the model training, calls this method to find the image to use for model
training.
Usage
EstimatorBase$training_image_uri()
Returns
str: The URI of the Docker image.
Method hyperparameters()
Return the hyperparameters as a dictionary to use for training.
The :meth:'~sagemaker.estimator.EstimatorBase.fit' method, which
trains the model, calls this method to find the hyperparameters.
Usage
EstimatorBase$hyperparameters()
Returns
dict[str, str]: The hyperparameters.
Method enable_network_isolation()
Return True if this Estimator will need network isolation to run.
Usage
EstimatorBase$enable_network_isolation()
Returns
bool: Whether this Estimator needs network isolation or not.
Method prepare_workflow_for_training()
Calls _prepare_for_training. Used when setting up a workflow.
Usage
EstimatorBase$prepare_workflow_for_training(job_name = NULL)
Arguments
job_name(str): Name of the training job to be created. If not
specified, one is generated, using the base name given to the
constructor if applicable.
Method latest_job_debugger_artifacts_path()
Gets the path to the DebuggerHookConfig output artifacts.
Usage
EstimatorBase$latest_job_debugger_artifacts_path()
Returns
str: An S3 path to the output artifacts.
Method latest_job_tensorboard_artifacts_path()
Gets the path to the TensorBoardOutputConfig output artifacts.
Usage
EstimatorBase$latest_job_tensorboard_artifacts_path()
Returns
str: An S3 path to the output artifacts.
Method latest_job_profiler_artifacts_path()
Gets the path to the profiling output artifacts.
Usage
EstimatorBase$latest_job_profiler_artifacts_path()
Returns
str: An S3 path to the output artifacts.
Method fit()
Train a model using the input training dataset.
The API calls the Amazon SageMaker CreateTrainingJob API to start
model training. The API uses configuration you provided to create the
estimator and the specified input training data to send the
CreatingTrainingJob request to Amazon SageMaker.
This is a synchronous operation. After the model training
successfully completes, you can call the “deploy()“ method to host the
model using the Amazon SageMaker hosting services.
Usage
EstimatorBase$fit(
inputs = NULL,
wait = TRUE,
logs = "All",
job_name = NULL,
experiment_config = NULL
)
Arguments
inputs(str or dict or TrainingInput): Information
about the training data. This can be one of three types:
(str) the S3 location where training data is saved, or a file:// path in
local mode.
(dict[str, str] or dict[str, TrainingInput]) If using multiple
channels for training data, you can specify a dict mapping channel names to
strings or :func:'~TrainingInput' objects.
(TrainingInput) - channel configuration for S3 data sources that can
provide additional information as well as the path to the training dataset.
See :func:'TrainingInput' for full details.
(sagemaker.session.FileSystemInput) - channel configuration for
a file system data source that can provide additional information as well as
the path to the training dataset.
wait(bool): Whether the call should wait until the job completes (default: True).
logs([str]): A list of strings specifying which logs to print. Acceptable
strings are "All", "NULL", "Training", or "Rules". To maintain backwards
compatibility, boolean values are also accepted and converted to strings.
Only meaningful when wait is True.
job_name(str): Training job name. If not specified, the estimator generates
a default job name, based on the training image name and current timestamp.
experiment_config(dict[str, str]): Experiment management configuration.
Dictionary contains three optional keys,
'ExperimentName', 'TrialName', and 'TrialComponentDisplayName'.
Method wait()
Wait for an Amazon SageMaker job to complete.
Usage
EstimatorBase$wait(logs = "All")
Arguments
logs([str]): A list of strings specifying which logs to print. Acceptable
strings are "All", "NULL", "Training", or "Rules". To maintain backwards
compatibility, boolean values are also accepted and converted to strings.
Method describe()
Returns a response from the DescribeTrainingJob API call.
Usage
EstimatorBase$describe()
Method rule_job_summary()
Calls describe_training_job and returns two dictionaries.
Usage
EstimatorBase$rule_job_summary()
Returns
list[dict]: A list of DebugRuleEvaluationStatuses and ProfilerRuleEvaluationStatuses
dictionary.
Method compile_model()
Compile a Neo model using the input model.
Usage
EstimatorBase$compile_model(
target_instance_family,
input_shape,
output_path,
framework = NULL,
framework_version = NULL,
compile_max_run = 15 * 60,
tags = NULL,
target_platform_os = NULL,
target_platform_arch = NULL,
target_platform_accelerator = NULL,
compiler_options = NULL,
...
)
Arguments
target_instance_family(str): Identifies the device that you want to
run your model after compilation, for example: ml_c5. For allowed
strings see
https://docs.aws.amazon.com/sagemaker/latest/dg/API_OutputConfig.html.
input_shape(dict): Specifies the name and shape of the expected
inputs for your trained model in json dictionary form, for
example: 'data':[1,3,1024,1024], or 'var1': [1,1,28,28],
'var2':[1,1,28,28]
output_path(str): Specifies where to store the compiled model
framework(str): The framework that is used to train the original
model. Allowed values: 'mxnet', 'tensorflow', 'keras', 'pytorch',
'onnx', 'xgboost'
framework_version(str): The version of the framework
compile_max_run(int): Timeout in seconds for compilation (default:
3 * 60). After this amount of time Amazon SageMaker Neo
terminates the compilation job regardless of its current status.
tags(list[dict]): List of tags for labeling a compilation job. For
more, see
https://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.
target_platform_os(str): Target Platform OS, for example: 'LINUX'.
For allowed strings see
https://docs.aws.amazon.com/sagemaker/latest/dg/API_OutputConfig.html.
It can be used instead of target_instance_family.
target_platform_arch(str): Target Platform Architecture, for example: 'X86_64'.
For allowed strings see
https://docs.aws.amazon.com/sagemaker/latest/dg/API_OutputConfig.html.
It can be used instead of target_instance_family.
target_platform_accelerator(str, optional): Target Platform Accelerator,
for example: 'NVIDIA'. For allowed strings see
https://docs.aws.amazon.com/sagemaker/latest/dg/API_OutputConfig.html.
It can be used instead of target_instance_family.
compiler_options(dict, optional): Additional parameters for compiler.
Compiler Options are TargetPlatform / target_instance_family specific. See
https://docs.aws.amazon.com/sagemaker/latest/dg/API_OutputConfig.html for details.
...: Passed to invocation of “create_model()“.
Implementations may customize “create_model()“ to accept
“**kwargs“ to customize model creation during deploy. For
more, see the implementation docs.
Returns
sagemaker.model.Model: A SageMaker “Model“ object. See
:func:'~sagemaker.model.Model' for full details.
Method attach()
Attach to an existing training job.
Create an Estimator bound to an existing training job, each subclass
is responsible to implement
“_prepare_init_params_from_job_description()“ as this method delegates
the actual conversion of a training job description to the arguments
that the class constructor expects. After attaching, if the training job
has a Complete status, it can be “deploy()“ ed to create a SageMaker
Endpoint and return a “Predictor“.
If the training job is in progress, attach will block and display log
messages from the training job, until the training job completes.
Examples:
>>> my_estimator.fit(wait=False)
>>> training_job_name = my_estimator.latest_training_job.name
Later on:
>>> attached_estimator = Estimator.attach(training_job_name)
>>> attached_estimator.deploy()
Usage
EstimatorBase$attach(
training_job_name,
sagemaker_session = NULL,
model_channel_name = "model"
)
Arguments
training_job_name(str): The name of the training job to attach to.
sagemaker_session(sagemaker.session.Session): Session object which
manages interactions with Amazon SageMaker APIs and any other
AWS services needed. If not specified, the estimator creates one
using the default AWS configuration chain.
model_channel_name(str): Name of the channel where pre-trained
model data will be downloaded (default: 'model'). If no channel
with the same name exists in the training job, this option will
be ignored.
Returns
Instance of the calling “Estimator“ Class with the attached
training job.
Method logs()
Display the logs for Estimator's training job.
If the output is a tty or a Jupyter cell, it will be color-coded based
on which instance the log entry is from.
Usage
EstimatorBase$logs()
Method deploy()
Deploy the trained model to an Amazon SageMaker endpoint and return a
“sagemaker.Predictor“ object.
More information:
http://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works-training.html
Usage
EstimatorBase$deploy(
initial_instance_count,
instance_type,
serializer = NULL,
deserializer = NULL,
accelerator_type = NULL,
endpoint_name = NULL,
use_compiled_model = FALSE,
wait = TRUE,
model_name = NULL,
kms_key = NULL,
data_capture_config = NULL,
tags = NULL,
...
)
Arguments
initial_instance_count(int): Minimum number of EC2 instances to
deploy to an endpoint for prediction.
instance_type(str): Type of EC2 instance to deploy to an endpoint
for prediction, for example, 'ml.c4.xlarge'.
serializer(:class:'~sagemaker.serializers.BaseSerializer'): A
serializer object, used to encode data for an inference endpoint
(default: None). If “serializer“ is not None, then
“serializer“ will override the default serializer. The
default serializer is set by the “predictor_cls“.
deserializer(:class:'~sagemaker.deserializers.BaseDeserializer'): A
deserializer object, used to decode data from an inference
endpoint (default: None). If “deserializer“ is not None, then
“deserializer“ will override the default deserializer. The
default deserializer is set by the “predictor_cls“.
accelerator_type(str): Type of Elastic Inference accelerator to
attach to an endpoint for model loading and inference, for
example, 'ml.eia1.medium'. If not specified, no Elastic
Inference accelerator will be attached to the endpoint. For more
information:
https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html
endpoint_name(str): Name to use for creating an Amazon SageMaker
endpoint. If not specified, the name of the training job is
used.
use_compiled_model(bool): Flag to select whether to use compiled
(optimized) model. Default: False.
wait(bool): Whether the call should wait until the deployment of
model completes (default: True).
model_name(str): Name to use for creating an Amazon SageMaker
model. If not specified, the estimator generates a default job name
based on the training image name and current timestamp.
kms_key(str): The ARN of the KMS key that is used to encrypt the
data on the storage volume attached to the instance hosting the
endpoint.
data_capture_config(sagemaker.model_monitor.DataCaptureConfig): Specifies
configuration related to Endpoint data capture for use with
Amazon SageMaker Model Monitoring. Default: None.
tags(List[dict[str, str]]): Optional. The list of tags to attach to this specific
endpoint. Example:
>>> tags = ['Key': 'tagname', 'Value': 'tagvalue']
For more information about tags, see
https://boto3.amazonaws.com/v1/documentation\
/api/latest/reference/services/sagemaker.html#SageMaker.Client.add_tags
...: Passed to invocation of “create_model()“.
Implementations may customize “create_model()“ to accept
“**kwargs“ to customize model creation during deploy.
For more, see the implementation docs.
Returns
sagemaker.predictor.Predictor: A predictor that provides a “predict()“ method,
which can be used to send requests to the Amazon SageMaker
endpoint and obtain inferences.
Method register()
Creates a model package for creating SageMaker models or listing on Marketplace.
Usage
EstimatorBase$register(
content_types,
response_types,
inference_instances,
transform_instances,
image_uri = NULL,
model_package_name = NULL,
model_package_group_name = NULL,
model_metrics = NULL,
metadata_properties = NULL,
marketplace_cert = FALSE,
approval_status = NULL,
description = NULL,
compile_model_family = NULL,
model_name = NULL,
drift_check_baselines = NULL,
...
)
Arguments
content_types(list): The supported MIME types for the input data.
response_types(list): The supported MIME types for the output data.
inference_instances(list): A list of the instance types that are used to
generate inferences in real-time.
transform_instances(list): A list of the instance types on which a transformation
job can be run or on which an endpoint can be deployed.
image_uri(str): The container image uri for Model Package, if not specified,
Estimator's training container image will be used (default: None).
model_package_name(str): Model Package name, exclusive to 'model_package_group_name',
using 'model_package_name' makes the Model Package un-versioned (default: None).
model_package_group_name(str): Model Package Group name, exclusive to
'model_package_name', using 'model_package_group_name' makes the Model Package
versioned (default: None).
model_metrics(ModelMetrics): ModelMetrics object (default: None).
metadata_properties(MetadataProperties): MetadataProperties (default: None).
marketplace_cert(bool): A boolean value indicating if the Model Package is certified
for AWS Marketplace (default: False).
approval_status(str): Model Approval Status, values can be "Approved", "Rejected",
or "PendingManualApproval" (default: "PendingManualApproval").
description(str): Model Package description (default: None).
compile_model_family(str): Instance family for compiled model, if specified, a compiled
model will be used (default: None).
model_name(str): User defined model name (default: None).
drift_check_baselines(DriftCheckBaselines): DriftCheckBaselines object (default: None).
...: Passed to invocation of “create_model()“. Implementations may customize
“create_model()“ to accept “**kwargs“ to customize model creation during
deploy. For more, see the implementation docs.
Returns
str: A string of SageMaker Model Package ARN.
Method create_model()
Create a SageMaker “Model“ object that can be deployed to an
“Endpoint“.
Usage
EstimatorBase$create_model(...)
Arguments
...: Keyword arguments used by the implemented method for
creating the “Model“.
Returns
sagemaker.model.Model: A SageMaker “Model“ object. See
:func:'~sagemaker.model.Model' for full details.
Method delete_endpoint()
Delete an Amazon SageMaker “Endpoint“.
Usage
EstimatorBase$delete_endpoint()
Method transformer()
Return a “Transformer“ that uses a SageMaker Model based on the
training job. It reuses the SageMaker Session and base job name used by
the Estimator.
Usage
EstimatorBase$transformer(
instance_count,
instance_type,
strategy = NULL,
assemble_with = NULL,
output_path = NULL,
output_kms_key = NULL,
accept = NULL,
env = NULL,
max_concurrent_transforms = NULL,
max_payload = NULL,
tags = NULL,
role = NULL,
volume_kms_key = NULL,
vpc_config_override = "VPC_CONFIG_DEFAULT",
enable_network_isolation = NULL,
model_name = NULL
)
Arguments
instance_count(int): Number of EC2 instances to use.
instance_type(str): Type of EC2 instance to use, for example,
'ml.c4.xlarge'.
strategy(str): The strategy used to decide how to batch records in
a single request (default: NULL). Valid values: 'MultiRecord'
and 'SingleRecord'.
assemble_with(str): How the output is assembled (default: NULL).
Valid values: 'Line' or 'NULL'.
output_path(str): S3 location for saving the transform result. If
not specified, results are stored to a default bucket.
output_kms_key(str): Optional. KMS key ID for encrypting the
transform output (default: NULL).
accept(str): The accept header passed by the client to
the inference endpoint. If it is supported by the endpoint,
it will be the format of the batch transform output.
env(dict): Environment variables to be set for use during the
transform job (default: NULL).
max_concurrent_transforms(int): The maximum number of HTTP requests
to be made to each individual transform container at one time.
max_payload(int): Maximum size of the payload in a single HTTP
request to the container in MB.
tags(list[dict]): List of tags for labeling a transform job. If
NULL specified, then the tags used for the training job are used
for the transform job.
role(str): The “ExecutionRoleArn“ IAM Role ARN for the “Model“,
which is also used during transform jobs. If not specified, the
role from the Estimator will be used.
volume_kms_key(str): Optional. KMS key ID for encrypting the volume
attached to the ML compute instance (default: NULL).
vpc_config_override(dict[str, list[str]]): Optional override for the
VpcConfig set on the model.
Default: use subnets and security groups from this Estimator.
* 'Subnets' (list[str]): List of subnet ids.
* 'SecurityGroupIds' (list[str]): List of security group ids.
enable_network_isolation(bool): Specifies whether container will
run in network isolation mode. Network isolation mode restricts
the container access to outside networks (such as the internet).
The container does not make any inbound or outbound network
calls. If True, a channel named "code" will be created for any
user entry script for inference. Also known as Internet-free mode.
If not specified, this setting is taken from the estimator's
current configuration.
model_name(str): Name to use for creating an Amazon SageMaker
model. If not specified, the name of the training job is used.
Method get_vpc_config()
Returns VpcConfig dict either from this Estimator's subnets and
security groups, or else validate and return an optional override value.
Usage
EstimatorBase$get_vpc_config(vpc_config_override = "VPC_CONFIG_DEFAULT")
Arguments
vpc_config_override:
Method .prepare_for_training()
Set any values in the estimator that need to be set before training.
Usage
EstimatorBase$.prepare_for_training(job_name = NULL)
Arguments
job_name(str): Name of the training job to be created. If not
specified, one is generated, using the base name given to the
constructor if applicable.
Method enable_default_profiling()
Update training job to enable Debugger monitoring.
This method enables Debugger monitoring with
the default “profiler_config“ parameter to collect system
metrics and the default built-in “profiler_report“ rule.
Framework metrics won't be saved.
To update training job to emit framework metrics, you can use
:class:'~sagemaker.estimator.Estimator.update_profiler'
method and specify the framework metrics you want to enable.
This method is callable when the training job is in progress while
Debugger monitoring is disabled.
Usage
EstimatorBase$enable_default_profiling()
Method disable_profiling()
Update the current training job in progress to disable profiling.
Debugger stops collecting the system and framework metrics
and turns off the Debugger built-in monitoring and profiling rules.
Usage
EstimatorBase$disable_profiling()
Method update_profiler()
Update training jobs to enable profiling.
This method updates the “profiler_config“ parameter
and initiates Debugger built-in rules for profiling.
Usage
EstimatorBase$update_profiler(
rules = NULL,
system_monitor_interval_millis = NULL,
s3_output_path = NULL,
framework_profile_params = NULL,
disable_framework_metrics = FALSE
)
Arguments
rules(list[:class:'~sagemaker.debugger.ProfilerRule']): A list of
:class:'~sagemaker.debugger.ProfilerRule' objects to define
rules for continuous analysis with SageMaker Debugger. Currently, you can
only add new profiler rules during the training job. (default: “None“)
system_monitor_interval_millis(int): How often profiling system metrics are
collected; Unit: Milliseconds (default: “None“)
s3_output_path(str): The location in S3 to store the output. If profiler is enabled
once, s3_output_path cannot be changed. (default: “None“)
framework_profile_params(:class:'~sagemaker.debugger.FrameworkProfile'):
A parameter object for framework metrics profiling. Configure it using
the :class:'~sagemaker.debugger.FrameworkProfile' class.
To use the default framework profile parameters, pass “FrameworkProfile()“.
For more information about the default values,
see :class:'~sagemaker.debugger.FrameworkProfile'. (default: “None“)
disable_framework_metrics(bool): Specify whether to disable all the framework metrics.
This won't update system metrics and the Debugger built-in rules for monitoring.
To stop both monitoring and profiling,
use the :class:'~sagemaker.estimator.Estimator.desable_profiling'
method. (default: “False“)
Method format()
format class
Usage
EstimatorBase$format()
Method clone()
The objects of this class are cloneable with this method.
Usage
EstimatorBase$clone(deep = FALSE)
Arguments
deepWhether to make a deep clone.
Note
Updating the profiling configuration for TensorFlow dataloader profiling
is currently not available. If you started a TensorFlow training job only with
monitoring and want to enable profiling while the training job is running,
the dataloader profiling cannot be updated.
DyfanJones/sagemaker-r-mlcore documentation built on May 3, 2022, 10:08 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| EstimatorBase | R Documentation |
Handle end-to-end Amazon SageMaker training and deployment tasks.
Description
For introduction to model training and deployment, see http://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works-training.html Subclasses must define a way to determine what image to use for training, what hyperparameters to use, and how to create an appropriate predictor instance.
Active bindings
model_dataThe model location in S3. Only set if Estimator has been “fit()“.
training_job_analyticsReturn a “TrainingJobAnalytics“ object for the current training job.
Methods
Public methods
Method new()
Initialize an “EstimatorBase“ instance.
Usage
EstimatorBase$new( role, instance_count = NULL, instance_type = NULL, volume_size = 30, volume_kms_key = NULL, max_run = 24 * 60 * 60, input_mode = "File", output_path = NULL, output_kms_key = NULL, base_job_name = NULL, sagemaker_session = NULL, tags = NULL, subnets = NULL, security_group_ids = NULL, model_uri = NULL, model_channel_name = "model", metric_definitions = NULL, encrypt_inter_container_traffic = FALSE, use_spot_instances = FALSE, max_wait = NULL, checkpoint_s3_uri = NULL, checkpoint_local_path = NULL, rules = NULL, debugger_hook_config = NULL, tensorboard_output_config = NULL, enable_sagemaker_metrics = NULL, enable_network_isolation = FALSE, profiler_config = NULL, disable_profiler = FALSE, environment = NULL, max_retry_attempts = NULL, source_dir = NULL, git_config = NULL, hyperparameters = NULL, container_log_level = "INFO", code_location = NULL, entry_point = NULL, dependencies = NULL, ... )
Arguments
role(str): An AWS IAM role (either name or full ARN). The Amazon SageMaker training jobs and APIs that create Amazon SageMaker endpoints use this role to access training data and model artifacts. After the endpoint is created, the inference code might use the IAM role, if it needs to access an AWS resource.
instance_count(int): Number of Amazon EC2 instances to use for training.
instance_type(str): Type of EC2 instance to use for training, for example, 'ml.c4.xlarge'.
volume_size(int): Size in GB of the EBS volume to use for storing input data during training (default: 30). Must be large enough to store training data if File Mode is used (which is the default).
volume_kms_key(str): Optional. KMS key ID for encrypting EBS volume attached to the training instance (default: NULL).
max_run(int): Timeout in seconds for training (default: 24 * 60 * 60). After this amount of time Amazon SageMaker terminates the job regardless of its current status.
input_mode(str): The input mode that the algorithm supports (default: 'File'). Valid modes: 'File' - Amazon SageMaker copies the training dataset from the S3 location to a local directory. 'Pipe' - Amazon SageMaker streams data directly from S3 to the container via a Unix-named pipe. This argument can be overriden on a per-channel basis using “TrainingInput.input_mode“.
output_path(str): S3 location for saving the training result (model artifacts and output files). If not specified, results are stored to a default bucket. If the bucket with the specific name does not exist, the estimator creates the bucket during the :meth:'~sagemaker.estimator.EstimatorBase.fit' method execution. file:// urls are used for local mode. For example: 'file://model/' will save to the model folder in the current directory.
output_kms_key(str): Optional. KMS key ID for encrypting the training output (default: NULL).
base_job_name(str): Prefix for training job name when the :meth:'~sagemaker.estimator.EstimatorBase.fit' method launches. If not specified, the estimator generates a default job name, based on the training image name and current timestamp.
sagemaker_session(sagemaker.session.Session): Session object which manages interactions with Amazon SageMaker APIs and any other AWS services needed. If not specified, the estimator creates one using the default AWS configuration chain.
tags(list[dict]): List of tags for labeling a training job. For more, see https://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.
subnets(list[str]): List of subnet ids. If not specified training job will be created without VPC config.
security_group_ids(list[str]): List of security group ids. If not specified training job will be created without VPC config.
model_uri(str): URI where a pre-trained model is stored, either locally or in S3 (default: NULL). If specified, the estimator will create a channel pointing to the model so the training job can download it. This model can be a 'model.tar.gz' from a previous training job, or other artifacts coming from a different source. In local mode, this should point to the path in which the model is located and not the file itself, as local Docker containers will try to mount the URI as a volume. More information: https://docs.aws.amazon.com/sagemaker/latest/dg/cdf-training.html#td-deserialization
model_channel_name(str): Name of the channel where 'model_uri' will be downloaded (default: 'model').
metric_definitions(list[dict]): A list of dictionaries that defines the metric(s) used to evaluate the training jobs. Each dictionary contains two keys: 'Name' for the name of the metric, and 'Regex' for the regular expression used to extract the metric from the logs. This should be defined only for jobs that don't use an Amazon algorithm.
encrypt_inter_container_traffic(bool): Specifies whether traffic between training containers is encrypted for the training job (default: “False“).
use_spot_instances(bool): Specifies whether to use SageMaker Managed Spot instances for training. If enabled then the 'max_wait' arg should also be set. More information: https://docs.aws.amazon.com/sagemaker/latest/dg/model-managed-spot-training.html (default: “False“).
max_wait(int): Timeout in seconds waiting for spot training instances (default: NULL). After this amount of time Amazon SageMaker will stop waiting for Spot instances to become available (default: “NULL“).
checkpoint_s3_uri(str): The S3 URI in which to persist checkpoints that the algorithm persists (if any) during training. (default: “NULL“).
checkpoint_local_path(str): The local path that the algorithm writes its checkpoints to. SageMaker will persist all files under this path to 'checkpoint_s3_uri' continually during training. On job startup the reverse happens - data from the s3 location is downloaded to this path before the algorithm is started. If the path is unset then SageMaker assumes the checkpoints will be provided under '/opt/ml/checkpoints/'. (default: “NULL“).
rules(list[:class:'~sagemaker.debugger.Rule']): A list of :class:'~sagemaker.debugger.Rule' objects used to define rules for continuous analysis with SageMaker Debugger (default: “NULL“). For more, see https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_debugger.html#continuous-analyses-through-rules
debugger_hook_config(:class:'~sagemaker.debugger.DebuggerHookConfig' or bool): Configuration for how debugging information is emitted with SageMaker Debugger. If not specified, a default one is created using the estimator's “output_path“, unless the region does not support SageMaker Debugger. To disable SageMaker Debugger, set this parameter to “False“. For more, see https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_debugger.html
tensorboard_output_config(:class:'~sagemaker.debugger.TensorBoardOutputConfig'): Configuration for customizing debugging visualization using TensorBoard (default: “NULL“). For more, see https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_debugger.html#capture-real-time-tensorboard-data-from-the-debugging-hook
enable_sagemaker_metrics(bool): Enables SageMaker Metrics Time Series. For more information see: https://docs.aws.amazon.com/sagemaker/latest/dg/API_AlgorithmSpecification.html#SageMaker-Type-AlgorithmSpecification-EnableSageMakerMetricsTimeSeries (default: “NULL“).
enable_network_isolation(bool): Specifies whether container will run in network isolation mode (default: “False“). Network isolation mode restricts the container access to outside networks (such as the Internet). The container does not make any inbound or outbound network calls. Also known as Internet-free mode.
profiler_config(:class:'~sagemaker.debugger.ProfilerConfig'): Configuration for how SageMaker Debugger collects monitoring and profiling information from your training job. If not specified, a default configuration is created using the estimator's “output_path“, unless the region does not support SageMaker Debugger. To disable SageMaker Debugger monitoring and profiling, set the “disable_profiler“ parameter to “True“.
disable_profiler(bool): Specifies whether Debugger monitoring and profiling will be disabled (default: “False“).
environment(dict[str, str]) : Environment variables to be set for use during training job (default: “None“)
max_retry_attempts(int): The number of times to move a job to the STARTING status. You can specify between 1 and 30 attempts. If the value of attempts is greater than zero, the job is retried on InternalServerFailure the same number of attempts as the value. You can cap the total duration for your job by setting “max_wait“ and “max_run“ (default: “None“)
source_dir(str): The absolute, relative, or S3 URI Path to a directory with any other training source code dependencies aside from the entry point file (default: None). If “source_dir“ is an S3 URI, it must point to a tar.gz file. The structure within this directory is preserved when training on Amazon SageMaker. If 'git_config' is provided, 'source_dir' should be a relative location to a directory in the Git repo. With the following GitHub repo directory structure, if you need 'train.py' as the entry point and 'test.py' as the training source code, you can assign entry_point='train.py' and source_dir='src'.
git_config(dict[str, str]): Git configurations used for cloning files, including “repo“, “branch“, “commit“, “2FA_enabled“, “username“, “password“, and “token“. The “repo“ field is required. All other fields are optional. “repo“ specifies the Git repository where your training script is stored. If you don't provide “branch“, the default value 'master' is used. If you don't provide “commit“, the latest commit in the specified branch is used. results in cloning the repo specified in 'repo', then checking out the 'master' branch, and checking out the specified commit. “2FA_enabled“, “username“, “password“, and “token“ are used for authentication. For GitHub (or other Git) accounts, set “2FA_enabled“ to 'True' if two-factor authentication is enabled for the account, otherwise set it to 'False'. If you do not provide a value for “2FA_enabled“, a default value of 'False' is used. CodeCommit does not support two-factor authentication, so do not provide "2FA_enabled" with CodeCommit repositories. For GitHub and other Git repos, when SSH URLs are provided, it doesn't matter whether 2FA is enabled or disabled. You should either have no passphrase for the SSH key pairs or have the ssh-agent configured so that you will not be prompted for the SSH passphrase when you run the 'git clone' command with SSH URLs. When HTTPS URLs are provided, if 2FA is disabled, then either “token“ or “username“ and “password“ are be used for authentication if provided. “Token“ is prioritized. If 2FA is enabled, only “token“ is used for authentication if provided. If required authentication info is not provided, the SageMaker Python SDK attempts to use local credentials to authenticate. If that fails, an error message is thrown. For CodeCommit repos, 2FA is not supported, so '2FA_enabled' should not be provided. There is no token in CodeCommit, so “token“ should also not be provided. When “repo“ is an SSH URL, the requirements are the same as GitHub repos. When “repo“ is an HTTPS URL, “username“ and “password“ are used for authentication if they are provided. If they are not provided, the SageMaker Python SDK attempts to use either the CodeCommit credential helper or local credential storage for authentication.
hyperparameters(dict): A dictionary containing the hyperparameters to initialize this estimator with. (Default: None).
container_log_level(str): The log level to use within the container (default: logging.INFO). Valid values are defined in the Python logging module.
code_location(str): The S3 prefix URI where custom code is uploaded (default: None). You must not include a trailing slash because a string prepended with a "/" is appended to “code_location“. The code file uploaded to S3 is 'code_location/job-name/source/sourcedir.tar.gz'. If not specified, the default “code location“ is 's3://output_bucket/job-name/'.
entry_point(str): The absolute or relative path to the local Python source file that should be executed as the entry point to training. (Default: None). If “source_dir“ is specified, then “entry_point“ must point to a file located at the root of “source_dir“. If 'git_config' is provided, 'entry_point' should be a relative location to the Python source file in the Git repo. You can assign entry_point='src/train.py'.
dependencies(list[str]): A list of absolute or relative paths to directories with any additional libraries that should be exported to the container (default: []). The library folders are copied to SageMaker in the same folder where the entrypoint is copied. If 'git_config' is provided, 'dependencies' should be a list of relative locations to directories with any additional libraries needed in the Git repo. This is not supported with "local code" in Local Mode.
...: update any deprecated parameters passed into class.
Method help()
Return class documentation
Usage
EstimatorBase$help()
Method training_image_uri()
Return the Docker image to use for training. The :meth:'~sagemaker.estimator.EstimatorBase.fit' method, which does the model training, calls this method to find the image to use for model training.
Usage
EstimatorBase$training_image_uri()
Returns
str: The URI of the Docker image.
Method hyperparameters()
Return the hyperparameters as a dictionary to use for training. The :meth:'~sagemaker.estimator.EstimatorBase.fit' method, which trains the model, calls this method to find the hyperparameters.
Usage
EstimatorBase$hyperparameters()
Returns
dict[str, str]: The hyperparameters.
Method enable_network_isolation()
Return True if this Estimator will need network isolation to run.
Usage
EstimatorBase$enable_network_isolation()
Returns
bool: Whether this Estimator needs network isolation or not.
Method prepare_workflow_for_training()
Calls _prepare_for_training. Used when setting up a workflow.
Usage
EstimatorBase$prepare_workflow_for_training(job_name = NULL)
Arguments
job_name(str): Name of the training job to be created. If not specified, one is generated, using the base name given to the constructor if applicable.
Method latest_job_debugger_artifacts_path()
Gets the path to the DebuggerHookConfig output artifacts.
Usage
EstimatorBase$latest_job_debugger_artifacts_path()
Returns
str: An S3 path to the output artifacts.
Method latest_job_tensorboard_artifacts_path()
Gets the path to the TensorBoardOutputConfig output artifacts.
Usage
EstimatorBase$latest_job_tensorboard_artifacts_path()
Returns
str: An S3 path to the output artifacts.
Method latest_job_profiler_artifacts_path()
Gets the path to the profiling output artifacts.
Usage
EstimatorBase$latest_job_profiler_artifacts_path()
Returns
str: An S3 path to the output artifacts.
Method fit()
Train a model using the input training dataset. The API calls the Amazon SageMaker CreateTrainingJob API to start model training. The API uses configuration you provided to create the estimator and the specified input training data to send the CreatingTrainingJob request to Amazon SageMaker. This is a synchronous operation. After the model training successfully completes, you can call the “deploy()“ method to host the model using the Amazon SageMaker hosting services.
Usage
EstimatorBase$fit( inputs = NULL, wait = TRUE, logs = "All", job_name = NULL, experiment_config = NULL )
Arguments
inputs(str or dict or TrainingInput): Information about the training data. This can be one of three types:
(str) the S3 location where training data is saved, or a file:// path in local mode.
(dict[str, str] or dict[str, TrainingInput]) If using multiple channels for training data, you can specify a dict mapping channel names to strings or :func:'~TrainingInput' objects.
(TrainingInput) - channel configuration for S3 data sources that can provide additional information as well as the path to the training dataset. See :func:'TrainingInput' for full details.
(sagemaker.session.FileSystemInput) - channel configuration for a file system data source that can provide additional information as well as the path to the training dataset.
wait(bool): Whether the call should wait until the job completes (default: True).
logs([str]): A list of strings specifying which logs to print. Acceptable strings are "All", "NULL", "Training", or "Rules". To maintain backwards compatibility, boolean values are also accepted and converted to strings. Only meaningful when wait is True.
job_name(str): Training job name. If not specified, the estimator generates a default job name, based on the training image name and current timestamp.
experiment_config(dict[str, str]): Experiment management configuration. Dictionary contains three optional keys, 'ExperimentName', 'TrialName', and 'TrialComponentDisplayName'.
Method wait()
Wait for an Amazon SageMaker job to complete.
Usage
EstimatorBase$wait(logs = "All")
Arguments
logs([str]): A list of strings specifying which logs to print. Acceptable strings are "All", "NULL", "Training", or "Rules". To maintain backwards compatibility, boolean values are also accepted and converted to strings.
Method describe()
Returns a response from the DescribeTrainingJob API call.
Usage
EstimatorBase$describe()
Method rule_job_summary()
Calls describe_training_job and returns two dictionaries.
Usage
EstimatorBase$rule_job_summary()
Returns
list[dict]: A list of DebugRuleEvaluationStatuses and ProfilerRuleEvaluationStatuses dictionary.
Method compile_model()
Compile a Neo model using the input model.
Usage
EstimatorBase$compile_model( target_instance_family, input_shape, output_path, framework = NULL, framework_version = NULL, compile_max_run = 15 * 60, tags = NULL, target_platform_os = NULL, target_platform_arch = NULL, target_platform_accelerator = NULL, compiler_options = NULL, ... )
Arguments
target_instance_family(str): Identifies the device that you want to run your model after compilation, for example: ml_c5. For allowed strings see https://docs.aws.amazon.com/sagemaker/latest/dg/API_OutputConfig.html.
input_shape(dict): Specifies the name and shape of the expected inputs for your trained model in json dictionary form, for example: 'data':[1,3,1024,1024], or 'var1': [1,1,28,28], 'var2':[1,1,28,28]
output_path(str): Specifies where to store the compiled model
framework(str): The framework that is used to train the original model. Allowed values: 'mxnet', 'tensorflow', 'keras', 'pytorch', 'onnx', 'xgboost'
framework_version(str): The version of the framework
compile_max_run(int): Timeout in seconds for compilation (default: 3 * 60). After this amount of time Amazon SageMaker Neo terminates the compilation job regardless of its current status.
tags(list[dict]): List of tags for labeling a compilation job. For more, see https://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.
target_platform_os(str): Target Platform OS, for example: 'LINUX'. For allowed strings see https://docs.aws.amazon.com/sagemaker/latest/dg/API_OutputConfig.html. It can be used instead of target_instance_family.
target_platform_arch(str): Target Platform Architecture, for example: 'X86_64'. For allowed strings see https://docs.aws.amazon.com/sagemaker/latest/dg/API_OutputConfig.html. It can be used instead of target_instance_family.
target_platform_accelerator(str, optional): Target Platform Accelerator, for example: 'NVIDIA'. For allowed strings see https://docs.aws.amazon.com/sagemaker/latest/dg/API_OutputConfig.html. It can be used instead of target_instance_family.
compiler_options(dict, optional): Additional parameters for compiler. Compiler Options are TargetPlatform / target_instance_family specific. See https://docs.aws.amazon.com/sagemaker/latest/dg/API_OutputConfig.html for details.
...: Passed to invocation of “create_model()“. Implementations may customize “create_model()“ to accept “**kwargs“ to customize model creation during deploy. For more, see the implementation docs.
Returns
sagemaker.model.Model: A SageMaker “Model“ object. See :func:'~sagemaker.model.Model' for full details.
Method attach()
Attach to an existing training job. Create an Estimator bound to an existing training job, each subclass is responsible to implement “_prepare_init_params_from_job_description()“ as this method delegates the actual conversion of a training job description to the arguments that the class constructor expects. After attaching, if the training job has a Complete status, it can be “deploy()“ ed to create a SageMaker Endpoint and return a “Predictor“. If the training job is in progress, attach will block and display log messages from the training job, until the training job completes. Examples: >>> my_estimator.fit(wait=False) >>> training_job_name = my_estimator.latest_training_job.name Later on: >>> attached_estimator = Estimator.attach(training_job_name) >>> attached_estimator.deploy()
Usage
EstimatorBase$attach( training_job_name, sagemaker_session = NULL, model_channel_name = "model" )
Arguments
training_job_name(str): The name of the training job to attach to.
sagemaker_session(sagemaker.session.Session): Session object which manages interactions with Amazon SageMaker APIs and any other AWS services needed. If not specified, the estimator creates one using the default AWS configuration chain.
model_channel_name(str): Name of the channel where pre-trained model data will be downloaded (default: 'model'). If no channel with the same name exists in the training job, this option will be ignored.
Returns
Instance of the calling “Estimator“ Class with the attached training job.
Method logs()
Display the logs for Estimator's training job. If the output is a tty or a Jupyter cell, it will be color-coded based on which instance the log entry is from.
Usage
EstimatorBase$logs()
Method deploy()
Deploy the trained model to an Amazon SageMaker endpoint and return a “sagemaker.Predictor“ object. More information: http://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works-training.html
Usage
EstimatorBase$deploy( initial_instance_count, instance_type, serializer = NULL, deserializer = NULL, accelerator_type = NULL, endpoint_name = NULL, use_compiled_model = FALSE, wait = TRUE, model_name = NULL, kms_key = NULL, data_capture_config = NULL, tags = NULL, ... )
Arguments
initial_instance_count(int): Minimum number of EC2 instances to deploy to an endpoint for prediction.
instance_type(str): Type of EC2 instance to deploy to an endpoint for prediction, for example, 'ml.c4.xlarge'.
serializer(:class:'~sagemaker.serializers.BaseSerializer'): A serializer object, used to encode data for an inference endpoint (default: None). If “serializer“ is not None, then “serializer“ will override the default serializer. The default serializer is set by the “predictor_cls“.
deserializer(:class:'~sagemaker.deserializers.BaseDeserializer'): A deserializer object, used to decode data from an inference endpoint (default: None). If “deserializer“ is not None, then “deserializer“ will override the default deserializer. The default deserializer is set by the “predictor_cls“.
accelerator_type(str): Type of Elastic Inference accelerator to attach to an endpoint for model loading and inference, for example, 'ml.eia1.medium'. If not specified, no Elastic Inference accelerator will be attached to the endpoint. For more information: https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html
endpoint_name(str): Name to use for creating an Amazon SageMaker endpoint. If not specified, the name of the training job is used.
use_compiled_model(bool): Flag to select whether to use compiled (optimized) model. Default: False.
wait(bool): Whether the call should wait until the deployment of model completes (default: True).
model_name(str): Name to use for creating an Amazon SageMaker model. If not specified, the estimator generates a default job name based on the training image name and current timestamp.
kms_key(str): The ARN of the KMS key that is used to encrypt the data on the storage volume attached to the instance hosting the endpoint.
data_capture_config(sagemaker.model_monitor.DataCaptureConfig): Specifies configuration related to Endpoint data capture for use with Amazon SageMaker Model Monitoring. Default: None.
tags(List[dict[str, str]]): Optional. The list of tags to attach to this specific endpoint. Example: >>> tags = ['Key': 'tagname', 'Value': 'tagvalue'] For more information about tags, see https://boto3.amazonaws.com/v1/documentation\ /api/latest/reference/services/sagemaker.html#SageMaker.Client.add_tags
...: Passed to invocation of “create_model()“. Implementations may customize “create_model()“ to accept “**kwargs“ to customize model creation during deploy. For more, see the implementation docs.
Returns
sagemaker.predictor.Predictor: A predictor that provides a “predict()“ method, which can be used to send requests to the Amazon SageMaker endpoint and obtain inferences.
Method register()
Creates a model package for creating SageMaker models or listing on Marketplace.
Usage
EstimatorBase$register( content_types, response_types, inference_instances, transform_instances, image_uri = NULL, model_package_name = NULL, model_package_group_name = NULL, model_metrics = NULL, metadata_properties = NULL, marketplace_cert = FALSE, approval_status = NULL, description = NULL, compile_model_family = NULL, model_name = NULL, drift_check_baselines = NULL, ... )
Arguments
content_types(list): The supported MIME types for the input data.
response_types(list): The supported MIME types for the output data.
inference_instances(list): A list of the instance types that are used to generate inferences in real-time.
transform_instances(list): A list of the instance types on which a transformation job can be run or on which an endpoint can be deployed.
image_uri(str): The container image uri for Model Package, if not specified, Estimator's training container image will be used (default: None).
model_package_name(str): Model Package name, exclusive to 'model_package_group_name', using 'model_package_name' makes the Model Package un-versioned (default: None).
model_package_group_name(str): Model Package Group name, exclusive to 'model_package_name', using 'model_package_group_name' makes the Model Package versioned (default: None).
model_metrics(ModelMetrics): ModelMetrics object (default: None).
metadata_properties(MetadataProperties): MetadataProperties (default: None).
marketplace_cert(bool): A boolean value indicating if the Model Package is certified for AWS Marketplace (default: False).
approval_status(str): Model Approval Status, values can be "Approved", "Rejected", or "PendingManualApproval" (default: "PendingManualApproval").
description(str): Model Package description (default: None).
compile_model_family(str): Instance family for compiled model, if specified, a compiled model will be used (default: None).
model_name(str): User defined model name (default: None).
drift_check_baselines(DriftCheckBaselines): DriftCheckBaselines object (default: None).
...: Passed to invocation of “create_model()“. Implementations may customize “create_model()“ to accept “**kwargs“ to customize model creation during deploy. For more, see the implementation docs.
Returns
str: A string of SageMaker Model Package ARN.
Method create_model()
Create a SageMaker “Model“ object that can be deployed to an “Endpoint“.
Usage
EstimatorBase$create_model(...)
Arguments
...: Keyword arguments used by the implemented method for creating the “Model“.
Returns
sagemaker.model.Model: A SageMaker “Model“ object. See :func:'~sagemaker.model.Model' for full details.
Method delete_endpoint()
Delete an Amazon SageMaker “Endpoint“.
Usage
EstimatorBase$delete_endpoint()
Method transformer()
Return a “Transformer“ that uses a SageMaker Model based on the training job. It reuses the SageMaker Session and base job name used by the Estimator.
Usage
EstimatorBase$transformer( instance_count, instance_type, strategy = NULL, assemble_with = NULL, output_path = NULL, output_kms_key = NULL, accept = NULL, env = NULL, max_concurrent_transforms = NULL, max_payload = NULL, tags = NULL, role = NULL, volume_kms_key = NULL, vpc_config_override = "VPC_CONFIG_DEFAULT", enable_network_isolation = NULL, model_name = NULL )
Arguments
instance_count(int): Number of EC2 instances to use.
instance_type(str): Type of EC2 instance to use, for example, 'ml.c4.xlarge'.
strategy(str): The strategy used to decide how to batch records in a single request (default: NULL). Valid values: 'MultiRecord' and 'SingleRecord'.
assemble_with(str): How the output is assembled (default: NULL). Valid values: 'Line' or 'NULL'.
output_path(str): S3 location for saving the transform result. If not specified, results are stored to a default bucket.
output_kms_key(str): Optional. KMS key ID for encrypting the transform output (default: NULL).
accept(str): The accept header passed by the client to the inference endpoint. If it is supported by the endpoint, it will be the format of the batch transform output.
env(dict): Environment variables to be set for use during the transform job (default: NULL).
max_concurrent_transforms(int): The maximum number of HTTP requests to be made to each individual transform container at one time.
max_payload(int): Maximum size of the payload in a single HTTP request to the container in MB.
tags(list[dict]): List of tags for labeling a transform job. If NULL specified, then the tags used for the training job are used for the transform job.
role(str): The “ExecutionRoleArn“ IAM Role ARN for the “Model“, which is also used during transform jobs. If not specified, the role from the Estimator will be used.
volume_kms_key(str): Optional. KMS key ID for encrypting the volume attached to the ML compute instance (default: NULL).
vpc_config_override(dict[str, list[str]]): Optional override for the VpcConfig set on the model. Default: use subnets and security groups from this Estimator. * 'Subnets' (list[str]): List of subnet ids. * 'SecurityGroupIds' (list[str]): List of security group ids.
enable_network_isolation(bool): Specifies whether container will run in network isolation mode. Network isolation mode restricts the container access to outside networks (such as the internet). The container does not make any inbound or outbound network calls. If True, a channel named "code" will be created for any user entry script for inference. Also known as Internet-free mode. If not specified, this setting is taken from the estimator's current configuration.
model_name(str): Name to use for creating an Amazon SageMaker model. If not specified, the name of the training job is used.
Method get_vpc_config()
Returns VpcConfig dict either from this Estimator's subnets and security groups, or else validate and return an optional override value.
Usage
EstimatorBase$get_vpc_config(vpc_config_override = "VPC_CONFIG_DEFAULT")
Arguments
vpc_config_override:
Method .prepare_for_training()
Set any values in the estimator that need to be set before training.
Usage
EstimatorBase$.prepare_for_training(job_name = NULL)
Arguments
job_name(str): Name of the training job to be created. If not specified, one is generated, using the base name given to the constructor if applicable.
Method enable_default_profiling()
Update training job to enable Debugger monitoring. This method enables Debugger monitoring with the default “profiler_config“ parameter to collect system metrics and the default built-in “profiler_report“ rule. Framework metrics won't be saved. To update training job to emit framework metrics, you can use :class:'~sagemaker.estimator.Estimator.update_profiler' method and specify the framework metrics you want to enable. This method is callable when the training job is in progress while Debugger monitoring is disabled.
Usage
EstimatorBase$enable_default_profiling()
Method disable_profiling()
Update the current training job in progress to disable profiling. Debugger stops collecting the system and framework metrics and turns off the Debugger built-in monitoring and profiling rules.
Usage
EstimatorBase$disable_profiling()
Method update_profiler()
Update training jobs to enable profiling. This method updates the “profiler_config“ parameter and initiates Debugger built-in rules for profiling.
Usage
EstimatorBase$update_profiler( rules = NULL, system_monitor_interval_millis = NULL, s3_output_path = NULL, framework_profile_params = NULL, disable_framework_metrics = FALSE )
Arguments
rules(list[:class:'~sagemaker.debugger.ProfilerRule']): A list of :class:'~sagemaker.debugger.ProfilerRule' objects to define rules for continuous analysis with SageMaker Debugger. Currently, you can only add new profiler rules during the training job. (default: “None“)
system_monitor_interval_millis(int): How often profiling system metrics are collected; Unit: Milliseconds (default: “None“)
s3_output_path(str): The location in S3 to store the output. If profiler is enabled once, s3_output_path cannot be changed. (default: “None“)
framework_profile_params(:class:'~sagemaker.debugger.FrameworkProfile'): A parameter object for framework metrics profiling. Configure it using the :class:'~sagemaker.debugger.FrameworkProfile' class. To use the default framework profile parameters, pass “FrameworkProfile()“. For more information about the default values, see :class:'~sagemaker.debugger.FrameworkProfile'. (default: “None“)
disable_framework_metrics(bool): Specify whether to disable all the framework metrics. This won't update system metrics and the Debugger built-in rules for monitoring. To stop both monitoring and profiling, use the :class:'~sagemaker.estimator.Estimator.desable_profiling' method. (default: “False“)
Method format()
format class
Usage
EstimatorBase$format()
Method clone()
The objects of this class are cloneable with this method.
Usage
EstimatorBase$clone(deep = FALSE)
Arguments
deepWhether to make a deep clone.
Note
Updating the profiling configuration for TensorFlow dataloader profiling is currently not available. If you started a TensorFlow training job only with monitoring and want to enable profiling while the training job is running, the dataloader profiling cannot be updated.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.