GetDeploymentAccuracyOverTime: Retrieves accuracy statistics over time on given metrics for...

In datarobot: 'DataRobot' Predictive Modeling API

Retrieves accuracy statistics over time on given metrics for a deployment.

Description

By default this will return statistics for the last seven days prior to the next; set the start and end parameters to adjust the reporting period.

Usage

GetDeploymentAccuracyOverTime(
  deploymentId,
  metrics,
  modelId = NULL,
  start = NULL,
  end = NULL,
  bucketSize = NULL,
  segmentAttribute = NULL,
  segmentValue = NULL
)

Arguments

deploymentId	character. The ID of the deployment in question.
metrics	character. Metrics to query. See DeploymentAccuracyMetric for supported values.
modelId	character. Optional. The ID of the model to query. If provided, only data for this specific model will be retrieved; otherwise, data for the deployment's default model will be retrieved.
start	POSIXct. Optional. The start time of the reporting period for monitoring data. Defaults to seven days prior to the end of the period. Sub-hour resolution is not permitted, and the timezone must be UTC.
end	POSIXct. Optional. The end time of the reporting period for monitoring data. Defaults to the next top of the hour. Sub-hour resolution is not permitted, and the timezone must be UTC.
bucketSize	character. Optional. The time duration of a bucket. This should be a multiple of one hour and cannot be longer than the total length of the period. If not set, a default value will be calculated based on the start and end times.
segmentAttribute	character. Optional. The name of an attribute used for segment analysis. See SegmentAnalysisAttribute“ for permitted values. Added in DataRobot 2.21.
segmentValue	character. Optional. The value of segmentAttribute. Added in DataRobot 2.21.

Value

An object representing how accuracy has changed over time for the deployment, containing:

• modelId character. The ID of the deployment model for which monitoring data was retrieved.

• summary data.frame. A summary bucket across the entire reporting period.

• buckets data.frame. A list of buckets representing each interval (constrained by the bucketSize parameter) in the reporting period.

• baseline data.frame. A baseline bucket.

Each bucket contains:

• sampleSize. integer. The number of predictions made against this deployment.

• start. POSIXct. The start time of the bucket. May be NA.

• end. POSIXct. The end time of the bucket. May be NA.

• metricName. numeric. Given N metrics queried, there will be N value columns, each one named for the metric. See DeploymentAccuracyMetric for supported values. May be NA if sampleSize is 0.

See Also

Other deployment accuracy functions: GetDeploymentAccuracy(), GetDeploymentAssociationId(), SubmitActuals()

Examples

## Not run: 
deploymentId <- "59a5af20c80891534e3c2bde"
aot <- GetDeploymentAccuracyOverTime(deploymentId,
         metrics = c(DeploymentAccuracyMetric$Gamma.Deviance,
                     DeploymentAccuracyMetric$LogLoss,
                     DeploymentAccuracyMetric$RMSE))

## End(Not run)

datarobot documentation built on Nov. 3, 2023, 1:07 a.m.