SummarizePlot: Summarize and Plot Ensemble models

In EBMAforecast: Estimate Ensemble Bayesian Model Averaging Forecasts using Gibbs Sampling or EM-Algorithms

Summarize and Plot Ensemble models

Description

These functions summarize and plot Ensemble models that have been fit previously by the user.

Usage

## S4 method for signature 'FDatFitLogit'
summary(
  object,
  period = "calibration",
  fitStatistics = c("brier", "auc", "perCorrect", "pre"),
  threshold = 0.5,
  baseModel = 0,
  showCoefs = TRUE,
  ...
)

## S4 method for signature 'FDatFitNormal'
summary(
  object,
  period = "calibration",
  fitStatistics = c("rmse", "mae"),
  threshold = 0.5,
  baseModel = 0,
  showCoefs = TRUE,
  ...
)

## S4 method for signature 'FDatFitLogit'
plot(
  x,
  period = "calibration",
  subset = 1,
  mainLabel = "",
  xLab = "",
  yLab = "",
  cols = 1,
  ...
)

## S4 method for signature 'FDatFitNormal'
plot(
  x,
  period = "calibration",
  subset = 1,
  mainLabel = paste("Observation", subset),
  xLab = "Outcome",
  yLab = "Posterior Probability",
  cols = 2:(length(x@modelNames) + 1),
  ...
)

Arguments

object	An object of the subclass "FDatFitLogit" or "FDatFitNormal".
period	The period for which the summary should be provided or plot produced, either "calibration" or "test". The default is "calibration".
fitStatistics	A vector naming statistics that should be calculated. Possible values for objects in the "FDatFitLogit" subclass include "auc", "brier", "percCorrect", "pre". Possible values for objects in the "FDatFitNormal" subclass include "rmse" and "mae." The default is for all statistics to be calculated. Additional metrics will be made available in a future release of this package.
threshold	The threshold used to calculate when a "positive" prediction is made for a model. The default is 0.5. Not used for objects of the "FDatFitNormal" subclass.
baseModel	A vector containing predictions used to calculate proportional reduction of error ("pre"). The default is 0. Not used for objects of the "FDatFitNormal" subclass.
showCoefs	A logical indicating whether model coefficients from the ensemble should be shown. The default is TRUE.
...	Not implemented
x	An object of class "FDatFitLogit" or "FDatFitNormal"
subset	The row names or numbers for the observations the user wishes to plot. The default is the first row. Only implemented for the subclass "FDatFitNormal".
mainLabel	A vector strings to appear at the top of each predictive posterior plot. Only implemented for the subclass "FDatFitNormal"
xLab	The label for the x-axis. Only implemented for the subclass "FDatFitNormal"
yLab	The label for the y-axis. Only implemented for the subclass "FDatFitNormal"
cols	A vector containing the color for plotting the predictive PDF of each component model forecast. The default is a unique color for each PDF. Only implemented for the subclass "FDatFitNormal"

Value

Either a plot or a data object of the class 'SummaryForecastData'. The data object has the following slots:

summaryData

Under the default, the function produces a matrix containing one row for each model plus one row for the EBMA forecast. The first column is always the model weights assigned to the component models. The second and third columns display the model parameters for the transformation of the component models. The remaining columns are the requested fit statistics for all models, as calculated by the copareModels function. If showCoefs=FALSE, then the model parameters will not be shown.

Author(s)

Michael D. Ward <[e-mail]> and Jacob M. Montgomery <[e-mail]> and Florian M. Hollenbach <[e-mail]>

References

Raftery, A. E., T. Gneiting, F. Balabdaoui and M. Polakowski. (2005). Using Bayesian Model Averaging to calibrate forecast ensembles. Monthly Weather Review. 133:1155–1174.

Greenhill, B., M.D. Ward, A. Sacks. (2011). The Separation Plot: A New Visual Method For Evaluating the Fit of Binary Data. American Journal of Political Science.55: 991–1002.

Montgomery, Jacob M., Florian M. Hollenbach and Michael D. Ward. (2012). Improving Predictions Using Ensemble Bayesian Model Averaging. Political Analysis. 20: 271-291.

Montgomery, Jacob M., Florian M. Hollenbach and Michael D. Ward. (2015). Calibrating ensemble forecasting models with sparse data in the social sciences. International Journal of Forecasting. 31:930–942.

Examples

## Not run:  data(calibrationSample)
data(testSample) 

this.ForecastData <- makeForecastData(.predCalibration=calibrationSample[,c("LMER", "SAE", "GLM")],
.outcomeCalibration=calibrationSample[,"Insurgency"],.predTest=testSample[,c("LMER", "SAE", "GLM")],
.outcomeTest=testSample[,"Insurgency"], .modelNames=c("LMER", "SAE", "GLM"))

this.ensemble <- calibrateEnsemble(this.ForecastData, model="logit", tol=0.001,exp=3)

summary(this.ensemble, period="calibration") 

summary(this.ensemble, period="test",showCoefs=FALSE)

## End(Not run)

EBMAforecast documentation built on Nov. 10, 2023, 5:06 p.m.