evaluation_stage_validation_plot: Draw a decile validation plot for a classification problem.
In robertzk/syberiaStages: Syberia Stages

This evaluation stage will produce a plot showing performance of the model according to a certain metric described below. The plot will be stored in a PNG file, which the user can control. (See the output option in evaluation_stage).

1	evaluation_stage_validation_plot(modelenv)

modelenv

environment. The current modeling environment.

The plot will contain the following information:

Percent of validation samples with dep_var = 1 on the y-axis.
Decile by model score on the x-axis. For example, if 0.1 is displayed on the x-axis, this should be interpreted as "the best 10 by decile score" (meaning, most likely to have dep_var = 0 as predicted by the model). If 0.2 is displayed, it means "the next best 10 of samples by decile score", that is, those falling within the 10 to 20th percentile.

In other words, the plot describes how "good" the model is at capturing the actual percent of samples with dep_var = 1. The samples whose predicted model score is within the top 10 empirical dep_var = 1, whereas the worst 10 such percentage.

robertzk/syberiaStages documentation built on May 27, 2019, 11:38 a.m.