Median stopping is an early termination policy based on running averages of primary metrics reported by the runs. This policy computes running averages across all training runs and terminates runs whose performance is worse than the median of the running averages. Specifically, a run will be canceled at interval N if its best primary metric reported up to interval N is worse than the median of the running averages for intervals 1:N across all runs.
median_stopping_policy(evaluation_interval = 1L, delay_evaluation = 0L)
An integer of the frequency for applying policy.
An integer of the number of intervals for which to delay the first evaluation.
The median stopping policy takes the following optional configuration parameters:
evaluation_interval: Optional. The frequency for applying the policy.
Each time the training script logs the primary metric counts as one
delay_evaluation: Optional. The number of intervals to delay the
policy evaluation. Use this parameter to avoid premature termination
of training runs. If specified, the policy applies every multiple of
evaluation_interval that is greater than or equal to
This policy is inspired from the research publication Google Vizier: A Service for Black-Box Optimization.
If you are looking for a conservative policy that provides savings without
terminating promising jobs, you can use a
evaluation_interval = 1 and
delay_evaluation = 5. These are conservative
settings that can provide approximately 25%-35% savings with no loss on
the primary metric (based on our evaluation data).
1 2 3 4 5 6 7 8 9
# In this example, the early termination policy is applied at every # interval starting at evaluation interval 5. A run will be terminated at # interval 5 if its best primary metric is worse than the median of the # running averages over intervals 1:5 across all training runs ## Not run: early_termination_policy = median_stopping_policy(evaluation_interval = 1L, delay_evaluation = 5L) ## End(Not run)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.