sim.train.test: Simulate in-sample and out-of-sample model performance
In JimmyClowes/rethinking: Statistical Rethinking book package

Description Usage Arguments Details Author(s) See Also

Simulates in-sample and out-of-sample model performance for simple quap models, returning lppd in and out of sample, WAIC, LOOIC, and LOOCV.

sim_train_test(N = 20, k = 3, rho = c(0.15, -0.4), b_sigma = 100, 
    WAIC = FALSE, LOOCV=FALSE , LOOIC=FALSE , cv.cores=1 , 
    return_model=FALSE )

# old function from 1st edition
sim.train.test( N=20 , k=3 , rho=c(0.15,-0.4) , b_sigma=100 , 
    DIC=FALSE , WAIC=FALSE, devbar=FALSE , devbarout=FALSE )

`N`	Number of cases in simulated data
`k`	Number of parameters in model to fit to data
`rho`	Vector of correlations between predictors and outcome, in simulated data
`b_sigma`	Standard deviation of beta-coefficient priors
`DIC`	If `TRUE`, returns DIC
`WAIC`	If `TRUE`, returns WAIC
`LOOIC`	If `TRUE`, returns LOOIC as produced by `LOO`
`LOOCV`	If `TRUE`, returns LOOCV as produced by `cv_quap`
`devbar`	If `TRUE`, returns the average deviance in-sample
`devbarout`	If `TRUE`, returns average deviance out-of-sample
`cv.cores`	Number or cores to use for cross-validation
`return_model`	If `TRUE`, includes fit model in result

This function simulates Gaussian data and then fits linear regression models to it, returning the lppd of the fit as produced by lppd (training, in-sample) and the deviance on a new sample, computed using the posterior from the training sample.

Richard McElreath

lppd, WAIC, LOO, cv_quap

JimmyClowes/rethinking documentation built on Dec. 18, 2021, 1:36 a.m.