ModelQualityMonitor: Amazon SageMaker model monitor to monitor quality metrics for...
In DyfanJones/sagemaker-r-mlcore: sagemaker machine learning core classes and methods
ModelQualityMonitor R Documentation
Amazon SageMaker model monitor to monitor quality metrics for an endpoint.
Description
Please see the 'initialize' method of its base class for how to instantiate it.
Super class
sagemaker.mlcore::ModelMonitor -> ModelQualityMonitor
Public fields
JOB_DEFINITION_BASE_NAMEModel definition base name
Methods
Public methods
-
-
-
-
-
-
-
-
Inherited methods
Method new()
Initializes a monitor instance.
The monitor handles baselining datasets and creating Amazon SageMaker
Monitoring Schedules to monitor SageMaker endpoints.
Usage
ModelQualityMonitor$new(
role,
instance_count = 1,
instance_type = "ml.m5.xlarge",
volume_size_in_gb = 30,
volume_kms_key = NULL,
output_kms_key = NULL,
max_runtime_in_seconds = NULL,
base_job_name = NULL,
sagemaker_session = NULL,
env = NULL,
tags = NULL,
network_config = NULL
)
Arguments
role(str): An AWS IAM role. The Amazon SageMaker jobs use this role.
instance_count(int): The number of instances to run
the jobs with.
instance_type(str): Type of EC2 instance to use for
the job, for example, 'ml.m5.xlarge'.
volume_size_in_gb(int): Size in GB of the EBS volume
to use for storing data during processing (default: 30).
volume_kms_key(str): A KMS key for the job's volume.
output_kms_key(str): The KMS key id for the job's outputs.
max_runtime_in_seconds(int): Timeout in seconds. After this amount of
time, Amazon SageMaker terminates the job regardless of its current status.
Default: 3600
base_job_name(str): Prefix for the job name. If not specified,
a default name is generated based on the training image name and
current timestamp.
sagemaker_session(sagemaker.session.Session): Session object which
manages interactions with Amazon SageMaker APIs and any other
AWS services needed. If not specified, one is created using
the default AWS configuration chain.
env(dict): Environment variables to be passed to the job.
tags([dict]): List of tags to be passed to the job.
network_config(sagemaker.network.NetworkConfig): A NetworkConfig
object that configures network isolation, encryption of
inter-container traffic, security group IDs, and subnets.
Method monitoring_type()
Type of the monitoring job.
Usage
ModelQualityMonitor$monitoring_type()
Method suggest_baseline()
Suggest baselines for use with Amazon SageMaker Model Monitoring Schedules.
Usage
ModelQualityMonitor$suggest_baseline(
baseline_dataset,
dataset_format,
problem_type,
inference_attribute = NULL,
probability_attribute = NULL,
ground_truth_attribute = NULL,
probability_threshold_attribute = NULL,
post_analytics_processor_script = NULL,
output_s3_uri = NULL,
wait = FALSE,
logs = FALSE,
job_name = NULL
)
Arguments
baseline_dataset(str): The path to the baseline_dataset file. This can be a local
path or an S3 uri.
dataset_format(dict): The format of the baseline_dataset.
problem_type(str): The type of problem of this model quality monitoring. Valid
values are "Regression", "BinaryClassification", "MulticlassClassification".
inference_attribute(str): Index or JSONpath to locate predicted label(s).
probability_attribute(str or int): Index or JSONpath to locate probabilities.
ground_truth_attribute(str): Index or JSONpath to locate actual label(s).
probability_threshold_attribute(float): threshold to convert probabilities to binaries
Only used for ModelQualityMonitor, ModelBiasMonitor and ModelExplainabilityMonitor
post_analytics_processor_script(str): The path to the record post-analytics processor
script. This can be a local path or an S3 uri.
output_s3_uri(str): Desired S3 destination Destination of the constraint_violations
and statistics json files.
Default: "s3://<default_session_bucket>/<job_name>/output"
wait(bool): Whether the call should wait until the job completes (default: False).
logs(bool): Whether to show the logs produced by the job.
Only meaningful when wait is True (default: False).
job_name(str): Processing job name. If not specified, the processor generates
a default job name, based on the image name and current timestamp.
Returns
sagemaker.processing.ProcessingJob: The ProcessingJob object representing the
baselining job.
Method create_monitorying_schedule()
Creates a monitoring schedule.
Usage
ModelQualityMonitor$create_monitorying_schedule(
endpoint_input,
ground_truth_input,
problem_type,
record_preprocessor_script = NULL,
post_analytics_processor_script = NULL,
output_s3_uri = NULL,
constraints = NULL,
monitor_schedule_name = NULL,
schedule_cron_expression = NULL,
enable_cloudwatch_metrics = TRUE
)
Arguments
endpoint_input(str or sagemaker.model_monitor.EndpointInput): The endpoint to
monitor. This can either be the endpoint name or an EndpointInput.
ground_truth_input(str): S3 URI to ground truth dataset.
problem_type(str): The type of problem of this model quality monitoring. Valid
values are "Regression", "BinaryClassification", "MulticlassClassification".
record_preprocessor_script(str): The path to the record preprocessor script. This can
be a local path or an S3 uri.
post_analytics_processor_script(str): The path to the record post-analytics processor
script. This can be a local path or an S3 uri.
output_s3_uri(str): S3 destination of the constraint_violations and analysis result.
Default: "s3://<default_session_bucket>/<job_name>/output"
constraints(sagemaker.model_monitor.Constraints or str): If provided it will be used
for monitoring the endpoint. It can be a Constraints object or an S3 uri pointing
to a constraints JSON file.
monitor_schedule_name(str): Schedule name. If not specified, the processor generates
a default job name, based on the image name and current timestamp.
schedule_cron_expression(str): The cron expression that dictates the frequency that
this job run. See sagemaker.model_monitor.CronExpressionGenerator for valid
expressions. Default: Daily.
enable_cloudwatch_metrics(bool): Whether to publish cloudwatch metrics as part of
the baselining or monitoring jobs.
Method update_monitoring_schedule()
Updates the existing monitoring schedule.
If more options than schedule_cron_expression are to be updated, a new job definition will
be created to hold them. The old job definition will not be deleted.
Usage
ModelQualityMonitor$update_monitoring_schedule(
endpoint_input = NULL,
ground_truth_input = NULL,
problem_type = NULL,
record_preprocessor_script = NULL,
post_analytics_processor_script = NULL,
output_s3_uri = NULL,
constraints = NULL,
schedule_cron_expression = NULL,
enable_cloudwatch_metrics = NULL,
role = NULL,
instance_count = NULL,
instance_type = NULL,
volume_size_in_gb = NULL,
volume_kms_key = NULL,
output_kms_key = NULL,
max_runtime_in_seconds = NULL,
env = NULL,
network_config = NULL
)
Arguments
endpoint_input(str or sagemaker.model_monitor.EndpointInput): The endpoint
to monitor. This can either be the endpoint name or an EndpointInput.
ground_truth_input(str): S3 URI to ground truth dataset.
problem_type(str): The type of problem of this model quality monitoring. Valid values
are "Regression", "BinaryClassification", "MulticlassClassification".
record_preprocessor_script(str): The path to the record preprocessor script. This can
be a local path or an S3 uri.
post_analytics_processor_script(str): The path to the record post-analytics processor
script. This can be a local path or an S3 uri.
output_s3_uri(str): S3 destination of the constraint_violations and analysis result.
Default: "s3://<default_session_bucket>/<job_name>/output"
constraints(sagemaker.model_monitor.Constraints or str): If provided it will be used
for monitoring the endpoint. It can be a Constraints object or an S3 uri pointing
to a constraints JSON file.
schedule_cron_expression(str): The cron expression that dictates the frequency that
this job run. See sagemaker.model_monitor.CronExpressionGenerator for valid
expressions. Default: Daily.
enable_cloudwatch_metrics(bool): Whether to publish cloudwatch metrics as part of
the baselining or monitoring jobs.
role(str): An AWS IAM role. The Amazon SageMaker jobs use this role.
instance_count(int): The number of instances to run
the jobs with.
instance_type(str): Type of EC2 instance to use for
the job, for example, 'ml.m5.xlarge'.
volume_size_in_gb(int): Size in GB of the EBS volume
to use for storing data during processing (default: 30).
volume_kms_key(str): A KMS key for the job's volume.
output_kms_key(str): The KMS key id for the job's outputs.
max_runtime_in_seconds(int): Timeout in seconds. After this amount of
time, Amazon SageMaker terminates the job regardless of its current status.
Default: 3600
env(dict): Environment variables to be passed to the job.
network_config(sagemaker.network.NetworkConfig): A NetworkConfig
object that configures network isolation, encryption of
inter-container traffic, security group IDs, and subnets.
Method delete_monitoring_schedule()
Deletes the monitoring schedule and its job definition."
Usage
ModelQualityMonitor$delete_monitoring_schedule()
Method attach()
Sets this object's schedule name to the name provided.
This allows subsequent describe_schedule or list_executions calls to point
to the given schedule.
Usage
ModelQualityMonitor$attach(monitor_schedule_name, sagemaker_session = NULL)
Arguments
monitor_schedule_name(str): The name of the schedule to attach to.
sagemaker_session(sagemaker.session.Session): Session object which
manages interactions with Amazon SageMaker APIs and any other
AWS services needed. If not specified, one is created using
the default AWS configuration chain.
Method clone()
The objects of this class are cloneable with this method.
Usage
ModelQualityMonitor$clone(deep = FALSE)
Arguments
deepWhether to make a deep clone.
DyfanJones/sagemaker-r-mlcore documentation built on May 3, 2022, 10:08 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| ModelQualityMonitor | R Documentation |
Amazon SageMaker model monitor to monitor quality metrics for an endpoint.
Description
Please see the 'initialize' method of its base class for how to instantiate it.
Super class
sagemaker.mlcore::ModelMonitor -> ModelQualityMonitor
Public fields
JOB_DEFINITION_BASE_NAMEModel definition base name
Methods
Public methods
Inherited methods
Method new()
Initializes a monitor instance. The monitor handles baselining datasets and creating Amazon SageMaker Monitoring Schedules to monitor SageMaker endpoints.
Usage
ModelQualityMonitor$new( role, instance_count = 1, instance_type = "ml.m5.xlarge", volume_size_in_gb = 30, volume_kms_key = NULL, output_kms_key = NULL, max_runtime_in_seconds = NULL, base_job_name = NULL, sagemaker_session = NULL, env = NULL, tags = NULL, network_config = NULL )
Arguments
role(str): An AWS IAM role. The Amazon SageMaker jobs use this role.
instance_count(int): The number of instances to run the jobs with.
instance_type(str): Type of EC2 instance to use for the job, for example, 'ml.m5.xlarge'.
volume_size_in_gb(int): Size in GB of the EBS volume to use for storing data during processing (default: 30).
volume_kms_key(str): A KMS key for the job's volume.
output_kms_key(str): The KMS key id for the job's outputs.
max_runtime_in_seconds(int): Timeout in seconds. After this amount of time, Amazon SageMaker terminates the job regardless of its current status. Default: 3600
base_job_name(str): Prefix for the job name. If not specified, a default name is generated based on the training image name and current timestamp.
sagemaker_session(sagemaker.session.Session): Session object which manages interactions with Amazon SageMaker APIs and any other AWS services needed. If not specified, one is created using the default AWS configuration chain.
env(dict): Environment variables to be passed to the job.
tags([dict]): List of tags to be passed to the job.
network_config(sagemaker.network.NetworkConfig): A NetworkConfig object that configures network isolation, encryption of inter-container traffic, security group IDs, and subnets.
Method monitoring_type()
Type of the monitoring job.
Usage
ModelQualityMonitor$monitoring_type()
Method suggest_baseline()
Suggest baselines for use with Amazon SageMaker Model Monitoring Schedules.
Usage
ModelQualityMonitor$suggest_baseline( baseline_dataset, dataset_format, problem_type, inference_attribute = NULL, probability_attribute = NULL, ground_truth_attribute = NULL, probability_threshold_attribute = NULL, post_analytics_processor_script = NULL, output_s3_uri = NULL, wait = FALSE, logs = FALSE, job_name = NULL )
Arguments
baseline_dataset(str): The path to the baseline_dataset file. This can be a local path or an S3 uri.
dataset_format(dict): The format of the baseline_dataset.
problem_type(str): The type of problem of this model quality monitoring. Valid values are "Regression", "BinaryClassification", "MulticlassClassification".
inference_attribute(str): Index or JSONpath to locate predicted label(s).
probability_attribute(str or int): Index or JSONpath to locate probabilities.
ground_truth_attribute(str): Index or JSONpath to locate actual label(s).
probability_threshold_attribute(float): threshold to convert probabilities to binaries Only used for ModelQualityMonitor, ModelBiasMonitor and ModelExplainabilityMonitor
post_analytics_processor_script(str): The path to the record post-analytics processor script. This can be a local path or an S3 uri.
output_s3_uri(str): Desired S3 destination Destination of the constraint_violations and statistics json files. Default: "s3://<default_session_bucket>/<job_name>/output"
wait(bool): Whether the call should wait until the job completes (default: False).
logs(bool): Whether to show the logs produced by the job. Only meaningful when wait is True (default: False).
job_name(str): Processing job name. If not specified, the processor generates a default job name, based on the image name and current timestamp.
Returns
sagemaker.processing.ProcessingJob: The ProcessingJob object representing the baselining job.
Method create_monitorying_schedule()
Creates a monitoring schedule.
Usage
ModelQualityMonitor$create_monitorying_schedule( endpoint_input, ground_truth_input, problem_type, record_preprocessor_script = NULL, post_analytics_processor_script = NULL, output_s3_uri = NULL, constraints = NULL, monitor_schedule_name = NULL, schedule_cron_expression = NULL, enable_cloudwatch_metrics = TRUE )
Arguments
endpoint_input(str or sagemaker.model_monitor.EndpointInput): The endpoint to monitor. This can either be the endpoint name or an EndpointInput.
ground_truth_input(str): S3 URI to ground truth dataset.
problem_type(str): The type of problem of this model quality monitoring. Valid values are "Regression", "BinaryClassification", "MulticlassClassification".
record_preprocessor_script(str): The path to the record preprocessor script. This can be a local path or an S3 uri.
post_analytics_processor_script(str): The path to the record post-analytics processor script. This can be a local path or an S3 uri.
output_s3_uri(str): S3 destination of the constraint_violations and analysis result. Default: "s3://<default_session_bucket>/<job_name>/output"
constraints(sagemaker.model_monitor.Constraints or str): If provided it will be used for monitoring the endpoint. It can be a Constraints object or an S3 uri pointing to a constraints JSON file.
monitor_schedule_name(str): Schedule name. If not specified, the processor generates a default job name, based on the image name and current timestamp.
schedule_cron_expression(str): The cron expression that dictates the frequency that this job run. See sagemaker.model_monitor.CronExpressionGenerator for valid expressions. Default: Daily.
enable_cloudwatch_metrics(bool): Whether to publish cloudwatch metrics as part of the baselining or monitoring jobs.
Method update_monitoring_schedule()
Updates the existing monitoring schedule. If more options than schedule_cron_expression are to be updated, a new job definition will be created to hold them. The old job definition will not be deleted.
Usage
ModelQualityMonitor$update_monitoring_schedule( endpoint_input = NULL, ground_truth_input = NULL, problem_type = NULL, record_preprocessor_script = NULL, post_analytics_processor_script = NULL, output_s3_uri = NULL, constraints = NULL, schedule_cron_expression = NULL, enable_cloudwatch_metrics = NULL, role = NULL, instance_count = NULL, instance_type = NULL, volume_size_in_gb = NULL, volume_kms_key = NULL, output_kms_key = NULL, max_runtime_in_seconds = NULL, env = NULL, network_config = NULL )
Arguments
endpoint_input(str or sagemaker.model_monitor.EndpointInput): The endpoint to monitor. This can either be the endpoint name or an EndpointInput.
ground_truth_input(str): S3 URI to ground truth dataset.
problem_type(str): The type of problem of this model quality monitoring. Valid values are "Regression", "BinaryClassification", "MulticlassClassification".
record_preprocessor_script(str): The path to the record preprocessor script. This can be a local path or an S3 uri.
post_analytics_processor_script(str): The path to the record post-analytics processor script. This can be a local path or an S3 uri.
output_s3_uri(str): S3 destination of the constraint_violations and analysis result. Default: "s3://<default_session_bucket>/<job_name>/output"
constraints(sagemaker.model_monitor.Constraints or str): If provided it will be used for monitoring the endpoint. It can be a Constraints object or an S3 uri pointing to a constraints JSON file.
schedule_cron_expression(str): The cron expression that dictates the frequency that this job run. See sagemaker.model_monitor.CronExpressionGenerator for valid expressions. Default: Daily.
enable_cloudwatch_metrics(bool): Whether to publish cloudwatch metrics as part of the baselining or monitoring jobs.
role(str): An AWS IAM role. The Amazon SageMaker jobs use this role.
instance_count(int): The number of instances to run the jobs with.
instance_type(str): Type of EC2 instance to use for the job, for example, 'ml.m5.xlarge'.
volume_size_in_gb(int): Size in GB of the EBS volume to use for storing data during processing (default: 30).
volume_kms_key(str): A KMS key for the job's volume.
output_kms_key(str): The KMS key id for the job's outputs.
max_runtime_in_seconds(int): Timeout in seconds. After this amount of time, Amazon SageMaker terminates the job regardless of its current status. Default: 3600
env(dict): Environment variables to be passed to the job.
network_config(sagemaker.network.NetworkConfig): A NetworkConfig object that configures network isolation, encryption of inter-container traffic, security group IDs, and subnets.
Method delete_monitoring_schedule()
Deletes the monitoring schedule and its job definition."
Usage
ModelQualityMonitor$delete_monitoring_schedule()
Method attach()
Sets this object's schedule name to the name provided. This allows subsequent describe_schedule or list_executions calls to point to the given schedule.
Usage
ModelQualityMonitor$attach(monitor_schedule_name, sagemaker_session = NULL)
Arguments
monitor_schedule_name(str): The name of the schedule to attach to.
sagemaker_session(sagemaker.session.Session): Session object which manages interactions with Amazon SageMaker APIs and any other AWS services needed. If not specified, one is created using the default AWS configuration chain.
Method clone()
The objects of this class are cloneable with this method.
Usage
ModelQualityMonitor$clone(deep = FALSE)
Arguments
deepWhether to make a deep clone.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.