plot_eval_by: Plot a metric across multiple values of a model parameter

In simulator: An Engine for Running Simulations

Plot a metric across multiple values of a model parameter

Description

This function is to be used on simulations in which generate_model was called using the vary_along parameter. When this is a single (scalar) numeric parameter, a single plot is created in which the x-axis is this parameter. Eventually, this function should handle one or two categorical variables (in which facets are used) and one categorical combined with one continuous variable.

Usage

plot_eval_by(
  sim,
  metric_name,
  varying,
  type = c("aggregated", "raw"),
  center_aggregator = NULL,
  spread_aggregator = NULL,
  use_ggplot2 = TRUE,
  main,
  xlab,
  ylab,
  xlim,
  ylim,
  include_zero = FALSE,
  legend_location = "topright",
  method_col = seq(num_methods),
  method_lty = rep(1, num_methods),
  method_lwd = rep(1, num_methods),
  method_pch = rep(1, num_methods),
  ...
)

Arguments

sim	an object of class Simulation
metric_name	the name of a metric to plot (ignored if custom aggregator is provided)
varying	character vector giving the name of a parameter that is varied across the models in evals. For now, this parameter must be numeric and there cannot be multiple models having the same value of this parameter.
type	if "aggregated" then shows line with error bars (line represents center_aggregator and error bars represent spread_aggregator; by default these are sample mean and estimated standard error); if type is "raw" then shows the raw data as points (with smoother overlayed)
center_aggregator	ignored if type is "raw". When NULL (which is default), the sample mean aggregator is used. User can write specialized aggregators (see definition of class Aggregator) as necessary, for example, when the evaluated metric is not scalar-valued.
spread_aggregator	ignored if type is "raw". When NULL (which is default), the sample mean aggregator is used. User can write specialized aggregators (see definition of class Aggregator) as necessary, for example, when the evaluated metric is not scalar-valued. Set spread_aggregator to NA to hide error bars.
use_ggplot2	whether to use ggplot2 (requires installation of ggplot2)
main	title of plot.
xlab	the x-axis label (default is varying)
ylab	the y-axis label (default is metric_label)
xlim	the x-axis limits to use
ylim	the y-axis limits to use
include_zero	whether ylim should include 0. Ignored if ylim is passed explicitly
legend_location	location of legend. Set to NULL to remove legend.
method_col	color to use for each method
method_lty	line style to use for each method
method_lwd	line thickness to use for each method
method_pch	point style to use for each method (default is that no points, only lines are drawn)
...	additional arguments to pass to plot (only when use_ggplot2 = FALSE).

Details

When type is "raw", the individual evals are shown (one point per model-draw-method triplet) along with a loess smooth. When type is "aggregated", then center_aggregator and spread_aggregator are used. center_aggregator is used to draw a single line per method in which the individual evals computed for each draw has been been aggregated in some way. By default, the mean_aggregator is used, which simply averages the evals computed across all draws. When spread_aggregator is non-NULL, "error bars" are drawn with (half)widths computed using spread_aggregator. By default, the se_aggregator is used, which gives an estimate of the standard error of the sample mean.

The arguments method_col, method_lty, method_lwd, method_pch only apply when use_ggplot2 is FALSE.

Examples

## Not run: 
 # suppose previously we had run the following:
 sim <- new_simulation(name = "normal-example",
                       label = "Normal Mean Estimation",
                       dir = tempdir()) %>%
   generate_model(make_my_example_model,
                  n = list(10, 20, 30),
                  vary_along = "n") %>%
   simulate_from_model(nsim = 50, index = 1:3) %>%
   run_method(my_example_method) %>%
   evaluate(my_example_loss)
   # then we could plot this
   plot_eval_by(sim, "myloss", varying = "n", include_zero = TRUE)
 
## End(Not run)

simulator documentation built on Feb. 16, 2023, 9:34 p.m.