predict_runtime: Predict the elapsed runtime of the next call to 'make()' for...

In ropensci/drake: A Pipeline Toolkit for Reproducible Computation at Scale

Predict the elapsed runtime of the next call to make() for non-staged parallel backends.

Description

Take the past recorded runtimes times from build_times() and use them to predict how the targets will be distributed among the available workers in the next make(). Then, predict the overall runtime to be the runtime of the slowest (busiest) workers. Predictions only include the time it takes to run the targets, not overhead/preprocessing from drake itself.

Usage

predict_runtime(
  ...,
  targets_predict = NULL,
  from_scratch = FALSE,
  targets_only = NULL,
  jobs_predict = 1L,
  known_times = numeric(0),
  default_time = 0,
  warn = TRUE,
  config = NULL
)

Arguments

...	Arguments to make(), such as plan and targets.
targets_predict	Character vector, names of targets to include in the total runtime and worker predictions.
from_scratch	Logical, whether to predict a make() build from scratch or to take into account the fact that some targets may be already up to date and therefore skipped.
targets_only	Deprecated.
jobs_predict	The jobs argument of your next planned make().
known_times	A named numeric vector with targets/imports as names and values as hypothetical runtimes in seconds. Use this argument to overwrite any of the existing build times or the default_time.
default_time	Number of seconds to assume for any target or import with no recorded runtime (from build_times()) or anything in known_times.
warn	Logical, whether to warn the user about any targets with no available runtime, either in known_times or build_times(). The times for these targets default to default_time.
config	Deprecated.

Value

Predicted total runtime of the next call to make().

See Also

predict_workers(), build_times(), make()

Examples

## Not run: 
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
load_mtcars_example() # Get the code with drake_example("mtcars").
make(my_plan) # Run the project, build the targets.
known_times <- rep(7200, nrow(my_plan))
names(known_times) <- my_plan$target
known_times
# Predict the runtime
if (requireNamespace("lubridate", quietly = TRUE)) {
predict_runtime(
  my_plan,
  jobs_predict = 7L,
  from_scratch = TRUE,
  known_times = known_times
)
predict_runtime(
  my_plan,
  jobs_predict = 8L,
  from_scratch = TRUE,
  known_times = known_times
)
balance <- predict_workers(
  my_plan,
  jobs_predict = 7L,
  from_scratch = TRUE,
  known_times = known_times
)
balance
}
}
})

## End(Not run)

ropensci/drake documentation built on Dec. 7, 2024, 7:19 p.m.