KMeans: An unsupervised learning algorithm that attempts to find...

In DyfanJones/sagemaker-r-mlframework: sagemaker machine learning developed by amazon

An unsupervised learning algorithm that attempts to find discrete groupings within data.

Description

As the result of KMeans, members of a group are as similar as possible to one another and as different as possible from members of other groups. You define the attributes that you want the algorithm to use to determine similarity.

Super classes

sagemaker.mlcore::EstimatorBase -> sagemaker.mlcore::AmazonAlgorithmEstimatorBase -> KMeans

Public fields

repo_name

sagemaker repo name for framework

repo_version

version of framework

.module

mimic python module

Active bindings

k

The number of clusters to produce.

init_method

How to initialize cluster locations.

max_iterations

Maximum iterations for Lloyds EM procedure in the local kmeans used in finalize stage.

tol

Tolerance for change in ssd for early stopping in local kmeans.

num_trials

Local version is run multiple times and the one with the best loss is chosen.

local_init_method

Initialization method for local version.

half_life_time_size

The points can have a decayed weight.

epochs

Number of passes done over the training data.

center_factor

The algorithm will create “num_clusters * extra_center_factor“ as it runs.

eval_metrics

JSON list of metrics types to be used for reporting the score for the model.

Methods

Public methods

• KMeans$new()

• KMeans$create_model()

• KMeans$.prepare_for_training()

• KMeans$hyperparameters()

• KMeans$clone()

Inherited methods

• sagemaker.mlcore::EstimatorBase$attach()

• sagemaker.mlcore::EstimatorBase$compile_model()

• sagemaker.mlcore::EstimatorBase$delete_endpoint()

• sagemaker.mlcore::EstimatorBase$deploy()

• sagemaker.mlcore::EstimatorBase$describe()

• sagemaker.mlcore::EstimatorBase$disable_profiling()

• sagemaker.mlcore::EstimatorBase$enable_default_profiling()

• sagemaker.mlcore::EstimatorBase$enable_network_isolation()

• sagemaker.mlcore::EstimatorBase$format()

• sagemaker.mlcore::EstimatorBase$get_vpc_config()

• sagemaker.mlcore::EstimatorBase$help()

• sagemaker.mlcore::EstimatorBase$latest_job_debugger_artifacts_path()

• sagemaker.mlcore::EstimatorBase$latest_job_profiler_artifacts_path()

• sagemaker.mlcore::EstimatorBase$latest_job_tensorboard_artifacts_path()

• sagemaker.mlcore::EstimatorBase$logs()

• sagemaker.mlcore::EstimatorBase$register()

• sagemaker.mlcore::EstimatorBase$rule_job_summary()

• sagemaker.mlcore::EstimatorBase$transformer()

• sagemaker.mlcore::EstimatorBase$update_profiler()

• sagemaker.mlcore::AmazonAlgorithmEstimatorBase$fit()

• sagemaker.mlcore::AmazonAlgorithmEstimatorBase$prepare_workflow_for_training()

• sagemaker.mlcore::AmazonAlgorithmEstimatorBase$record_set()

• sagemaker.mlcore::AmazonAlgorithmEstimatorBase$training_image_uri()

• sagemaker.mlcore::AmazonAlgorithmEstimatorBase$wait()

---

Method new()

A k-means clustering :class:'~sagemaker.amazon.AmazonAlgorithmEstimatorBase'. Finds k clusters of data in an unlabeled dataset. This Estimator may be fit via calls to :meth:'~sagemaker.amazon.amazon_estimator.AmazonAlgorithmEstimatorBase.fit_ndarray' or :meth:'~sagemaker.amazon.amazon_estimator.AmazonAlgorithmEstimatorBase.fit'. The former allows a KMeans model to be fit on a 2-dimensional numpy array. The latter requires Amazon :class:'~sagemaker.amazon.record_pb2.Record' protobuf serialized data to be stored in S3. To learn more about the Amazon protobuf Record class and how to prepare bulk data in this format, please consult AWS technical documentation: https://docs.aws.amazon.com/sagemaker/latest/dg/cdf-training.html. After this Estimator is fit, model data is stored in S3. The model may be deployed to an Amazon SageMaker Endpoint by invoking :meth:'~sagemaker.amazon.estimator.EstimatorBase.deploy'. As well as deploying an Endpoint, “deploy“ returns a :class:'~sagemaker.amazon.kmeans.KMeansPredictor' object that can be used to k-means cluster assignments, using the trained k-means model hosted in the SageMaker Endpoint. KMeans Estimators can be configured by setting hyperparameters. The available hyperparameters for KMeans are documented below. For further information on the AWS KMeans algorithm, please consult AWS technical documentation: https://docs.aws.amazon.com/sagemaker/latest/dg/k-means.html.

Usage

KMeans$new(
  role,
  instance_count,
  instance_type,
  k,
  init_method = NULL,
  max_iterations = NULL,
  tol = NULL,
  num_trials = NULL,
  local_init_method = NULL,
  half_life_time_size = NULL,
  epochs = NULL,
  center_factor = NULL,
  eval_metrics = NULL,
  ...
)

Arguments

role

(str): An AWS IAM role (either name or full ARN). The Amazon SageMaker training jobs and APIs that create Amazon SageMaker endpoints use this role to access training data and model artifacts. After the endpoint is created, the inference code might use the IAM role, if accessing AWS resource.

instance_count

(int): Number of Amazon EC2 instances to use for training.

instance_type

(str): Type of EC2 instance to use for training, for example, 'ml.c4.xlarge'.

k

(int): The number of clusters to produce.

init_method

(str): How to initialize cluster locations. One of 'random' or 'kmeans++'.

max_iterations

(int): Maximum iterations for Lloyds EM procedure in the local kmeans used in finalize stage.

tol

(float): Tolerance for change in ssd for early stopping in local kmeans.

num_trials

(int): Local version is run multiple times and the one with the best loss is chosen. This determines how many times.

local_init_method

(str): Initialization method for local version. One of 'random', 'kmeans++'

half_life_time_size

(int): The points can have a decayed weight. When a point is observed its weight, with regard to the computation of the cluster mean is 1. This weight will decay exponentially as we observe more points. The exponent coefficient is chosen such that after observing “half_life_time_size“ points after the mentioned point, its weight will become 1/2. If set to 0, there will be no decay.

epochs

(int): Number of passes done over the training data.

center_factor

(int): The algorithm will create “num_clusters * extra_center_factor“ as it runs and reduce the number of centers to “k“ when finalizing

eval_metrics

(list): JSON list of metrics types to be used for reporting the score for the model. Allowed values are "msd" Means Square Error, "ssd": Sum of square distance. If test data is provided, the score shall be reported in terms of all requested metrics.

...

: base class keyword argument values.

---

Method create_model()

Return a :class:'~sagemaker.amazon.kmeans.KMeansModel' referencing the latest s3 model data produced by this Estimator.

Usage

KMeans$create_model(vpc_config_override = "VPC_CONFIG_DEFAULT", ...)

Arguments

vpc_config_override

(dict[str, list[str]]): Optional override for VpcConfig set on the model. Default: use subnets and security groups from this Estimator. * 'Subnets' (list[str]): List of subnet ids. * 'SecurityGroupIds' (list[str]): List of security group ids.

...

: Additional kwargs passed to the KMeansModel constructor.

---

Method .prepare_for_training()

Set hyperparameters needed for training. This method will also validate “source_dir“.

Usage

KMeans$.prepare_for_training(records, mini_batch_size = 5000, job_name = NULL)

Arguments

records

(RecordSet) – The records to train this Estimator on.

mini_batch_size

(int or None) – The size of each mini-batch to use when training. If None, a default value will be used.

job_name

(str): Name of the training job to be created. If not specified, one is generated, using the base name given to the constructor if applicable.

---

Method hyperparameters()

Return the SageMaker hyperparameters for training this KMeans Estimator

Usage

KMeans$hyperparameters()

---

Method clone()

The objects of this class are cloneable with this method.

Usage

KMeans$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

DyfanJones/sagemaker-r-mlframework documentation built on March 18, 2022, 7:41 a.m.