do_global_validation_run: Perform out-of-sample validation for a completed run of the...

View source: R/user_interface.R

do_global_validation_runR Documentation

Perform out-of-sample validation for a completed run of the FPEM global model

Description

This generates the MCMC chains using do_global_validation_mcmc, post-processes them using post_process_mcmc and produces plots and tables using make_results.

Usage

do_global_validation_run(
  run_desc = "",
  run_name_override = NULL,
  run_name_to_validate = NULL,
  run_name_to_validate_output_folder_path = file.path("output", run_name_to_validate),
  input_data_folder_path = NULL,
  exclude_unmet_only = FALSE,
  exclude_unmet_only_test_prop = 0.2,
  at_random = FALSE,
  at_random_min_c = 1,
  at_random_test_prop = 0.2,
  at_end = FALSE,
  at_end_not_1_obs_c = FALSE,
  at_random_no_data = FALSE,
  at_random_no_data_strata = NULL,
  at_random_no_data_test_prop = 0.2,
  leave_iso_out = FALSE,
  leave_iso_out_iso_test = NULL,
  year_cutoff = 2005,
  seed_validation = 12345,
  generate_new_set = TRUE,
  estimation_iterations = 3,
  burn_in_iterations = 1,
  steps_before_progress_report = 4,
  thinning = 2,
  chain_nums = 1:3,
  run_in_parallel = isTRUE(length(chain_nums) > 1),
  output_folder_path = NULL,
  verbose = FALSE
)

Arguments

run_desc

Character. Brief note to be appended to the auto-generated run_name. Ignored if run_name_override is non-NULL.

run_name_override

Character. User defined run name to override default generation. run_desc is ignored if this is non-NULL.

run_name_to_validate

Name of completed global run to validate.

input_data_folder_path

File path to folder containing all input data (except any map shapefiles). If NULL the value of data_csv_filename, etc., will be passed to file.path as-is. Otherwise, file.path(input_data_folder_path, data_csv_filename) will be passed. The default value points to the data directory supplied with the package.

exclude_unmet_only

Logical; do validtion exercise leaving out unmet need observations? See details.

exclude_unmet_only_test_prop

Numeric; the proportion of observations to be left out for an unmet need validation run.

at_random

Logical; do validtion exercise leaving out observations at random? See details.

at_random_min_c

Minimum number of data points per country to ensure are left in training set.

at_random_test_prop

Proportion of obs used for test set when at_random is TRUE.

at_end

Logical; do validtion exercise leaving out all observations after year_cutoff. See details.

at_end_not_1_obs_c

Logical; should obs that are the only one for their particular country be retained in the training set?

at_random_no_data

Logical; do validation exercise where all observations for a randomly selected set of countries are left out at random?

at_random_no_data_strata

Column in the country classifications file (used in the run being validated) to stratify on if at_random_no_data = TRUE. If NULL (default) do not stratify.

at_random_no_data_test_prop

Numeric; the proportion of observations to be left out when at_random_no_data = TRUE.

leave_iso_out

Logical; do validation exercise where all data for only one country is left out?

leave_iso_out_iso_test

Three-digit country ISO code, numeric or character, designating the country to leave out if leave_iso_out = TRUE.

year_cutoff

The cut-off year to use to separate test from training set if at_end = TRUE. All observations with observation year equal to or greater than year_cutoff are put in the test set.

seed_validation

Random seed used, among other things, in selection of countries to leave out.

generate_new_set

Logical; generate a new training set in validation exercise?

estimation_iterations

Numeric. Number of MCMC iterations that should be saved. This is before thinning.

burn_in_iterations

Numeric. Number of MCMC iterations that should be run as burn-in before starting to save them.

steps_before_progress_report

Numeric. The number of times progress should reported during MCMC sampling.

thinning

Numeric. The actual number of iterations saved is \frac{\code{estimation_iterations}}{\code{thinning}}.

chain_nums

Numeric. The number of MCMC chains to run, as a sequence. E.g., for three chains use 1:3. You need to run at least two chains for post-processing to be successful.

run_in_parallel

Logical. Determines if MCMC chains are run in parallel. Parallel running requires package

doParallel or doMC. Defaults to serial running if run_in_parallel = TRUE but the package is not available.

output_folder_path

Filepath to directory where outputs should be saved. If NULL, defaults to file.path("output", run_name).

verbose

Logical; print lots and lots of messages about progress?

Details

Exactly one of exclude_unmet_only, at_random, at_end, at_random_no_data, leave_iso_out should be TRUE. The possible validation exercises are

at_random = TRUE Leave out some obs at random, chosen at random
at_end = TRUE Leave out all obs after a certain year
at_random_no_data = TRUE Leave out all obs for some countries, chosen at random
leave_iso_out = TRUE Leave out all obs for a certain country

Value

A name for the run returned invisibly as a character string. Results are saved to ‘output_folder_path’.

Note

There is no “do_global_validation_all_women_run” function. You must run validation on married and unmarried women runs separately. Validation of an all women run is undefined.

Author(s)

Mark Wheldon

Examples

vignette("FPEMglobal_Intro")

FPcounts/FPEMglobal documentation built on July 20, 2024, 2:35 a.m.