Experiment: A Comet Experiment object

In comet-ml/cometr: 'Comet' API for R

A Comet Experiment object

Description

A comet experiment object can be used to modify or get information about an active experiment. All methods documented here are the different ways to interact with an experiment. Use create_experiment() to create or get_experiment() to retrieve a Comet experiment object.

Value

LoggedArtifact with all relevant information about logged artifact.

LoggedArtifact with all relevant information about logged artifact.

Methods

Public methods

• Experiment$new()

• Experiment$get_key()

• Experiment$get_workspace_name()

• Experiment$get_project_name()

• Experiment$get_dynamic()

• Experiment$get_url()

• Experiment$get_metadata()

• Experiment$archive()

• Experiment$restore()

• Experiment$delete()

• Experiment$stop()

• Experiment$log_metric()

• Experiment$get_metric()

• Experiment$get_metrics_summary()

• Experiment$log_graph()

• Experiment$get_graph()

• Experiment$log_parameter()

• Experiment$get_parameters()

• Experiment$log_other()

• Experiment$get_other()

• Experiment$add_tags()

• Experiment$get_tags()

• Experiment$log_html()

• Experiment$get_html()

• Experiment$upload_asset()

• Experiment$log_remote_asset()

• Experiment$get_asset_list()

• Experiment$get_asset()

• Experiment$create_symlink()

• Experiment$log_git_metadata()

• Experiment$get_git_metadata()

• Experiment$get_git_patch()

• Experiment$get_output()

• Experiment$log_code()

• Experiment$get_code()

• Experiment$log_system_details()

• Experiment$get_system_details()

• Experiment$log_artifact()

• Experiment$get_artifact()

• Experiment$set_start_end_time()

• Experiment$print()

---

Method new()

Do not call this function directly. Use create_experiment() or get_experiment() instead.

Usage

Experiment$new(
  experiment_key,
  experiment_url = NULL,
  api_key = NULL,
  keep_active = FALSE,
  log_output = FALSE,
  log_error = FALSE,
  dynamic = TRUE,
  workspace_name = NULL,
  project_name = NULL
)

Arguments

experiment_key

The key of the Experiment.

experiment_url

The URL of the Experiment.

api_key

Comet API key (can also be specified using the COMET_API_KEY parameter as an environment variable or in a comet config file).

keep_active

If TRUE, automatically send Comet a status update every few seconds until the experiment is stopped to mark the experiment as active on the Comet web dashboard.

log_output

If TRUE, all standard output will automatically be sent to the Comet servers to display as message logs for the experiment. The output will still be shown in the console as well.

log_error

If TRUE, all output from 'stderr' (which includes errors, warnings, and messages) will be redirected to the Comet servers to display as message logs for the experiment. Note that unlike auto_log_output, if this option is on then these messages will not be shown in the console and instead they will only be logged to the Comet experiment. This option is set to FALSE by default because of this behavior.

dynamic

If TRUE the Experiment was created rather than retrieved.

workspace_name

The workspace name (can also be specified using the COMET_WORKSPACE parameter as an environment variable or in a comet config file).

project_name

The project name (can also be specified using the COMET_PROJECT_NAME parameter as an environment variable or in a comet config file).

---

Method get_key()

Get the experiment key of an experiment.

Usage

Experiment$get_key()

---

Method get_workspace_name()

Get the workspace name of an experiment.

Usage

Experiment$get_workspace_name()

---

Method get_project_name()

Get the project name of an experiment.

Usage

Experiment$get_project_name()

---

Method get_dynamic()

Get the dynamic status of an experiment.

Usage

Experiment$get_dynamic()

---

Method get_url()

Get the URL to view an experiment in the browser.

Usage

Experiment$get_url()

---

Method get_metadata()

Get an experiment's metadata.

Usage

Experiment$get_metadata()

---

Method archive()

Archive an experiment.

Usage

Experiment$archive()

---

Method restore()

Restore an archived experiment.

Usage

Experiment$restore()

---

Method delete()

Delete an experiment.

Usage

Experiment$delete()

---

Method stop()

Stop an experiment. Always call this method before creating a new experiment.

Usage

Experiment$stop()

---

Method log_metric()

Log a metric name and value. Metrics are the only items that are logged as a full time series. However, even metrics can be throttled if too much data (either by rate or by count) is attempted to be logged.

Usage

Experiment$log_metric(name, value, step = NULL, epoch = NULL, context = NULL)

Arguments

name

(Required) Name of the metric.

value

(Required) Value of the metric.

step

Step number.

epoch

Epoch.

context

Context.

---

Method get_metric()

Get All Metrics For Name

Usage

Experiment$get_metric(name)

Arguments

name

(Required) Name of metric.

---

Method get_metrics_summary()

Get an experiment's metrics summary.

Usage

Experiment$get_metrics_summary()

---

Method log_graph()

Log an experiment's associated model graph.

Usage

Experiment$log_graph(graph)

Arguments

graph

(Required) JSON representation of a graph.

---

Method get_graph()

Get an experiment's model graph.

Usage

Experiment$get_graph()

---

Method log_parameter()

Log a parameter name and value. Note that you can only retrieve parameters summary data (e.g., this is not recorded as a full time series).

Usage

Experiment$log_parameter(name, value, step = NULL)

Arguments

name

(Required) Name of the parameter.

value

(Required) Value of the parameter.

step

Step number.

---

Method get_parameters()

Get an experiment's parameters summary.

Usage

Experiment$get_parameters()

---

Method log_other()

Log a key/value 'other“ data (not a metric or parameter). Note that you can only retrieve others summary data (e.g., this is not recorded as a full time series).

Usage

Experiment$log_other(key, value)

Arguments

key

(Required) The key.

value

(Required) The value.

---

Method get_other()

Get an experiment's others (logged with log_other()) summary.

Usage

Experiment$get_other()

---

Method add_tags()

Add a list of tags to an experiment.

Usage

Experiment$add_tags(tags)

Arguments

tags

(Required) List of tags.

---

Method get_tags()

Get an experiment's tags.

Usage

Experiment$get_tags()

---

Method log_html()

Set (or append onto) an experiment's HTML.

Usage

Experiment$log_html(html, override = FALSE)

Arguments

html

(Required) An HTML string to add to the experiment.

override

If TRUE, override the previous HTML. If FALSE, append to it.

---

Method get_html()

Get an experiment's HTML.

Usage

Experiment$get_html()

---

Method upload_asset()

Upload a file to the experiment.

Usage

Experiment$upload_asset(
  file,
  step = NULL,
  overwrite = NULL,
  context = NULL,
  type = NULL,
  name = NULL,
  metadata = NULL
)

Arguments

file

(Required) Path to the file to upload.

step

Step number.

overwrite

If TRUE, overwrite any uploaded file with the same name.

context

The context.

type

The type of asset.

name

Name of the file on comet. By default the name of the file will match the file that you upload, but you can use this parameter to use a different name.

metadata

Metadata to upload along with the file.

---

Method log_remote_asset()

Logs a Remote Asset identified by an URI. A Remote Asset is an asset but its content is not uploaded and stored on Comet. Rather a link for its location is stored, so you can identify and distinguish between two experiment using different version of a dataset stored somewhere else.

Usage

Experiment$log_remote_asset(
  uri,
  remote_file_name = NULL,
  step = NULL,
  overwrite = FALSE,
  type = "asset",
  metadata = NULL
)

Arguments

uri

(Required) The remote asset location, there is no imposed format, and it could be a private link.

remote_file_name

The "name" of the remote asset, could be a dataset name, a model file name.

step

Step number.

overwrite

If TRUE, overwrite any logged asset with the same name.

type

The type of asset, default: "asset".

metadata

Metadata to log along with the asset

---

Method get_asset_list()

Get an experiment's asset list.

Usage

Experiment$get_asset_list(type = NULL)

Arguments

type

The type of assets to retrieve (by default, all assets are returned).

---

Method get_asset()

Get an asset.

Usage

Experiment$get_asset(assetId)

Arguments

assetId

(Required) The asset ID to retrieve.

---

Method create_symlink()

Add a symlink to an experiment in another project.

Usage

Experiment$create_symlink(project_name)

Arguments

project_name

(Required) Project that the experiment to should linked to.

---

Method log_git_metadata()

Log an experiment's git metadata. This should only be called once and it can be done automatically by enabling log_git_info in create_experiment() or get_experiment(). This will replace any previous git metadata that was logged.

Usage

Experiment$log_git_metadata(
  branch = NULL,
  origin = NULL,
  parent = NULL,
  user = NULL,
  root = NULL
)

Arguments

branch

Git branch name.

origin

Git repository origin.

parent

Git commit SHA.

user

Git username.

root

Git root.

---

Method get_git_metadata()

Get the git metadata of an experiment.

Usage

Experiment$get_git_metadata()

---

Method get_git_patch()

Get the git patch of an experiment.

Usage

Experiment$get_git_patch()

---

Method get_output()

Get an experiment's standard output and error.

Usage

Experiment$get_output()

---

Method log_code()

Log an experiment's source code. This should only be called once and it can be done automatically by enabling log_code in create_experiment() or get_experiment(). This will replace any previous code that was logged.

Usage

Experiment$log_code(code)

Arguments

code

The code to set as the source code.

---

Method get_code()

Get an experiment's source code.

Usage

Experiment$get_code()

---

Method log_system_details()

Log system details. This can be done automatically by enabling log_system_details in create_experiment() or get_experiment().

Usage

Experiment$log_system_details(
  command = NULL,
  executable = NULL,
  hostname = NULL,
  installed_packages = NULL,
  gpu_static_info = NULL,
  ip = NULL,
  network_interface_ips = NULL,
  additional_system_info = NULL,
  os = NULL,
  os_packages = NULL,
  os_type = NULL,
  pid = NULL,
  user = NULL,
  r_version = NULL,
  r_version_verbose = NULL
)

Arguments

command

Script and optional arguments.

executable

Executable.

hostname

Hostname.

installed_packages

List of installed R packages.

gpu_static_info

List of GPU information, where each GPU is a list() with fields gpuIndex, name, powerLimit, totalMemory, uuid.

ip

IP address.

network_interface_ips

List of network interface IPs.

additional_system_info

List of additional parameters to log, where each parameter is a list() with key and value pairs.

os

Full details about operating system.

os_packages

List of operating system packages installed.

os_type

Operating system type.

pid

Process ID.

user

User.

r_version

Short form R version.

r_version_verbose

Long form R version.

---

Method get_system_details()

Get an experiment's system details.

Usage

Experiment$get_system_details()

---

Method log_artifact()

Log an Artifact object, synchronously create a new Artifact Version and upload all local and remote assets attached to the Artifact object.

Usage

Experiment$log_artifact(artifact)

Arguments

artifact

an Artifact object.

---

Method get_artifact()

Returns a logged artifact object that can be used to access the artifact version assets and download them locally.

If no version or alias is provided, the latest version for that artifact is returned.

Usage

Experiment$get_artifact(
  artifact_name,
  workspace = NULL,
  version_or_alias = NULL
)

Arguments

artifact_name

(Required) Retrieve an artifact with that name. This could either be a fully qualified artifact name like workspace/artifact-name:versionOrAlias or just the name of the artifact like artifact-name.

workspace

Retrieve an artifact belonging to that workspace.

version_or_alias

Retrieve the artifact by the given alias or version.

Examples

\dontrun{
library(cometr)
# Assuming you have COMET_API_KEY, COMET_WORKSPACE, COMET_PROJECT_NAME variables define
exp <- create_experiment()

# Get a Comet Artifact
logged_artifact <- exp$get_artifact("workspace/artifact-name:version_or_alias")

# Which is equivalent to
logged_artifact = exp$get_artifact(artifact_name="artifact-name",
                                   workspace="workspace",
                                   version_or_alias="version_or_alias")
}

---

Method set_start_end_time()

Set an experiment's start and end time.

Usage

Experiment$set_start_end_time(start = NULL, end = NULL)

Arguments

start

Start time for the experiment (milliseconds since the Epoch)

end

End time for the experiment (milliseconds since the Epoch)

---

Method print()

Print the experiment.

Usage

Experiment$print()

Examples

## Not run: 
library(cometr)
# Assuming you have COMET_API_KEY, COMET_WORKSPACE, COMET_PROJECT_NAME variables define
exp <- create_experiment()
exp$get_key()
exp$get_metadata()
exp$add_tags(c("test", "tag2"))
exp$get_tags()
exp$log_metric("metric1", 5)
exp$get_metric("metric1")
exp$get_metrics_summary()
exp$stop()

## End(Not run)


## ------------------------------------------------
## Method `Experiment$get_artifact`
## ------------------------------------------------

## Not run: 
library(cometr)
# Assuming you have COMET_API_KEY, COMET_WORKSPACE, COMET_PROJECT_NAME variables define
exp <- create_experiment()

# Get a Comet Artifact
logged_artifact <- exp$get_artifact("workspace/artifact-name:version_or_alias")

# Which is equivalent to
logged_artifact = exp$get_artifact(artifact_name="artifact-name",
                                   workspace="workspace",
                                   version_or_alias="version_or_alias")

## End(Not run)

comet-ml/cometr documentation built on Oct. 23, 2023, 7:44 a.m.