runModelValidation: Run validation of MCMC model
In MJAlexander/distortr: Temporal smoothing methods for demographic time series

Description Usage Arguments Value See Also Examples

Calculate some model validation measures for MCMC models on simulated data. These are calculated by leaving out some of the available sample. The function returns values for root-meean-squared-error, coverage and interval score.

runModelValidation(input.data, nyears, method, order = NULL,
  matern.cov = TRUE, nchains = 4, nburnin = 1000, niter = 1000 + 30000,
  nthin = 30, model.file.path = NULL, leave.out.method,
  leave.out.percent = 20, nreps = 10, alpha.level = 0.05, alpha.n = 0.1,
  seed = 123)

`input.data`	Input data to JAGS. If single country, this is a dataframe of x and y observations, and standard errors around ys.
`nyears`	number of years of observations
`method`	The method of smoothing to implement (choices: ar, arma, splines, gp)
`order`	The order of splines penalization (either 1 or 2)
`matern.cov`	Whether or not to use Matern covariance function if `method=="gp"`. Default is `TRUE`.
`nchains`	Number of MCMC chains
`nburnin`	Number of iterations to throw away as burn in.
`niter`	Number of total iterations.
`nthin`	Degree of thinning of MCMC chains
`model.file.path`	Text file which contains the model to be fitted. If `NULL`, the text file is drawn from the `models` folder.
`leave.out.method`	If equal to `random`, oberservations are left out at random. If equal to `recent` the most recent x% of observations are left out.
`leave.out.percent`	The percent of observations to leave out. Default is 20%
`nreps`	The number of times to repeat the model validation if the leave out method is random
`alpha.level`	Significance level of uncertainty intervals. Default is 5%.
`alpha.n`	Significant level of interval score. Default is 10%.
`seed`	Value of random seed.

A list containing values of RMSE, a vector of coverage values for each repetition, and a vector of interval scores for each repetition.

runMCMC, simulateFluctuations

nyears <- 100
prop.sample <- 0.7
obs.err <- TRUE
sigma.y <- 0.5
seed <- 123
method <- 'splines'
params <- list(sigma.alpha = 1, order = 1)
df <- simulateFluctuations(nyears, prop.sample, method, params, obs.err, sigma.y)
df$se <- 1
validation.results <- runModelValidation(input.data = df, nyears = 100, method = "splines", order = 1,
nchains = 4, nburnin = 100, niter = 100+3000, nthin = 3,
leave.out.method = "recent", leave.out.percent = 20)