model_judgement: Model Comparison via LOO-CV, WAIC and Raw LPPD

In Seneketh/StanDDM: Drift Diffusion Modelling with Stan

Model Comparison via LOO-CV, WAIC and Raw LPPD

Description

Applies log-likelihood based model comparison to any number of stanfit objects and extracts LOO-CV, WAIC and Raw LPPD measures and plots the models on a absolute scale ranging from 0 (best models) to 1 (worst models). WAIC and LOO-CV code has been adapted directly from the "LOO" package. Because most methods to prevent log-exp calculation-underflow failed, this function imputes underflow values with the mean of sampled values. When this happens, the function will report the amount of imputed values to inform the user. More than 5 Mind how Stan saves log-likelihood values! Consult the Stan manual to check how they are saved correctly! Because all models need to be loaded into memory, be wary if loading to big stanfits. Lighten the Stanfit objects by discarding unnecesary iterations is advised.

Usage

model_judgement(..., lik_name = "log_lik", impute_inf = TRUE)

Arguments

lik_name	Name under which the log likelihoods have been saved in the models. Needs to be identical across all Stanfit objects.
impute_inf	A boolean which regulates if underflow values should be automatically imputed or not. If FALSE, the models with such values will just be ignored. If TRUE, a report will be generated on how many values were imputed and for which models.
stanfits	At least 2 stanfit objects.

Value

Prints a table and generates a plot with the model ranked according to LOO-CV, WAIC and Raw LPPD.

Examples

load(fit1.Rdata)
load(fit2.Rdata)

model_judgement(fit1, fit2, impute_inf = TRUE)

Seneketh/StanDDM documentation built on Oct. 17, 2023, 4:26 p.m.