vetiver_plot_metrics: Plot model metrics over time for monitoring
In tidymodels/vetiver: Version, Share, Deploy, and Monitor Models

vetiver_plot_metrics

R Documentation

Plot model metrics over time for monitoring

Description

These three functions can be used for model monitoring (such as in a monitoring dashboard):

vetiver_compute_metrics() computes metrics (such as accuracy for a classification model or RMSE for a regression model) at a chosen time aggregation period
vetiver_pin_metrics() updates an existing pin storing model metrics over time
vetiver_plot_metrics() creates a plot of metrics over time

Usage

vetiver_plot_metrics(
  df_metrics,
  .index = .index,
  .estimate = .estimate,
  .metric = .metric,
  .n = .n
)

Arguments

`df_metrics`	A tidy dataframe of metrics over time, such as created by `vetiver_compute_metrics()`.
`.index`	The variable in `df_metrics` containing the aggregated dates or date-times (from `time_var` in `data`). Defaults to `.index`.
`.estimate`	The variable in `df_metrics` containing the metric estimate. Defaults to `.estimate`.
`.metric`	The variable in `df_metrics` containing the metric type. Defaults to `.metric`.
`.n`	The variable in `df_metrics` containing the number of observations used for estimating the metric.

Value

A ggplot2 object.

Examples


library(dplyr)
library(parsnip)
data(Chicago, package = "modeldata")
Chicago <- Chicago %>% select(ridership, date, all_of(stations))
training_data <- Chicago %>% filter(date < "2009-01-01")
testing_data <- Chicago %>% filter(date >= "2009-01-01", date < "2011-01-01")
monitoring <- Chicago %>% filter(date >= "2011-01-01", date < "2012-12-31")
lm_fit <- linear_reg() %>% fit(ridership ~ ., data = training_data)

library(pins)
b <- board_temp()

## before starting monitoring, initiate the metrics and pin
## (for example, with the testing data):
original_metrics <-
    augment(lm_fit, new_data = testing_data) %>%
    vetiver_compute_metrics(date, "week", ridership, .pred, every = 4L)
pin_write(b, original_metrics, "lm_fit_metrics", type = "arrow")

## to continue monitoring with new data, compute metrics and update pin:
new_metrics <-
    augment(lm_fit, new_data = monitoring) %>%
    vetiver_compute_metrics(date, "week", ridership, .pred, every = 4L)
vetiver_pin_metrics(b, new_metrics, "lm_fit_metrics")

library(ggplot2)
vetiver_plot_metrics(new_metrics) +
    scale_size(range = c(2, 4))

tidymodels/vetiver documentation built on March 25, 2024, 6 p.m.