In tidymodels/vetiver: Version, Share, Deploy, and Monitor Models

library(tidyverse)
library(vetiver)
library(pins)
library(yardstick)
knitr::opts_chunk$set(echo = FALSE)
v <- vetiver_pin_read(params$board, params$name, version = params$version)
v_meta <- pin_meta(params$board, params$name)
theme_set(theme_light())

A model card provides brief, transparent, responsible reporting for a trained machine learning model.

Model details

• Developed by PERSON AND/OR TEAM
• r cli::pluralize("{v$description} using {ncol(v$prototype)} feature{?s}")
• More details about how model was developed and what it is predicting
• More details on feature engineering and/or data preprocessing for model
• Version r v$metadata$version of this model was published at r v_meta$created
• Citation and/or license details for the model
• If you have questions about this model, please contact [e-mail]

Intended use

• The primary intended uses of this model are ...
• The primary intended users of this model are ...
• Some use cases are out of scope for this model, such as ...

Important aspects/factors

• Aspects or factors (demographic, environmental, technical) that are relevant to the context of this model are ...
• In evaluating this model, we examined aspects such as ...

Metrics

• The metrics used to evaluate this model are ...
• These metrics are computed via ...
• We chose these metrics because ...

Training data & evaluation data

• The training dataset for this model was ...

• The training dataset for this model has the "prototype" or signature:

  r glimpse(v$prototype)

• The evaluation dataset used in this model card is ...

• We chose this evaluation data because ...

## EVALUATION DATA:

data(Sacramento, package = "modeldata")

## consider using a package like skimr or DataExplorer for automated 
## presentation of evaluation data characteristics

Quantitative analyses {.tabset}

## compute predictions for your evaluation data
## load packages needed for prediction:
library(parsnip)
library(workflows)
preds <- augment(v, Sacramento)

Overall model performance

preds %>%
    metrics(price, .pred)

Disaggregated model performance

preds %>%
    group_by(type) %>%
    metrics(price, .pred)

Visualize model performance

preds %>%
    ggplot(aes(price, .pred, color = type)) +
    geom_abline(slope = 1, lty = 2, color = "gray60", size = 1.2) +
    geom_point(alpha = 0.5, show.legend = FALSE) +
    facet_wrap(vars(type))

Make a custom plot

preds %>%
    mutate(.resid = price - .pred) %>%
    ggplot(aes(longitude, latitude, color = .resid)) +
    geom_point(alpha = 0.8) +
    scale_color_gradient2() +
    coord_fixed()

Ethical considerations

• We considered ...

Caveats & recommendations

• This model does ...
• This model does not ...
• We recommend ...

tidymodels/vetiver documentation built on Oct. 15, 2024, 4:16 p.m.