doc/how_to_set_up_experiments.md

In richelbilderbeek/pirouette: Create a Bayesian Posterior from a Phylogeny

How to set up the experiments?

The pir_run argument experiments may be the most difficult one to grasp.

Below, we show how to put some research questions into the experiments.

See https://github.com/richelbilderbeek/pirouette_examples for all https://github.com/richelbilderbeek/pirouette examples.

1. What is the error BEAST2 makes from a phylogeny using the same diversification model as it was generated by?

Here one needs only one experiment:

model_type|run_if |measure_evidence |beautier::inference_model ------------|----------------|-------------------|--------------------------- generative|always |(TRUE or) FALSE|generative_model

We assume we know the generative model, which happens to be the BEAUti default:

generative_model <- beautier::create_inference_model()
testthat::expect_true(generative_model$site_model$name == "JC69")
testthat::expect_true(generative_model$clock_model$name == "strict")
testthat::expect_true(generative_model$tree_prior$name == "yule")

We create one experiment:

experiment <- create_experiment(
  model_type = "generative",
  run_if = "always",
  do_measure_evidence = FALSE,
  inference_model = generative_model
)

experiments <- list(experiment)

pir_run(
  # ...
  experiments = experiment
)

See https://github.com/richelbilderbeek/pirouette_example_1 for the complete code.

2. What is the error BEAST2 makes from a phylogeny when picking the best inference model?

To answer this question, we create as much candidate experiments as we have inference models:

model_type|run_if |measure_evidence|beautier::inference_model ------------|----------------|------------------|--------------------------- candidate |best_candidate|TRUE |inference_model_1 candidate |best_candidate|TRUE |inference_model_2 ... | ... | ... | ... candidate |best_candidate|TRUE |inference_model_39 candidate |best_candidate|TRUE |inference_model_40

experiments <- create_all_experiments()

Running:

pir_run(
  # ...
  experiments = experiments
)

3. What is the error BEAST2 makes from a phylogeny, when hand-picking an inference model, compared to the background noise?

This is the same setup as the first research question, expect now we need a twin tree.

model_type|run_if |measure_evidence |beautier::inference_model ------------|----------------|-------------------|----------------------------- generative|always |(TRUE or) FALSE|generative_model

my_favorite_inference_model <- ...

OK, put that in the experiment:

experiment <- create_experiment(
  inference_model = my_favorite_inference_model
)

experiments <- list(experiments)

pir_params <- create_pir_params(
  # ...
  experiments = list(),
  twinning_params = create_twinning_params
)

pir_run(
  # ...
  pir_params = pir_params
)

See https://github.com/richelbilderbeek/pirouette_example_11 for the complete code.

4. What is the difference in error between a known generative model and the best of the other candidates?

model_type|run_if |measure_evidence|beautier::inference_model ------------|----------------|------------------|--------------------------- generative|always |TRUE or FALSE |generative_model candidate |best_candidate|TRUE |Inference model 1 candidate |best_candidate|TRUE |Inference model 2 ... | ... | ... | ... candidate |best_candidate|TRUE |Inference model 38 candidate |best_candidate|TRUE |Inference model 39

Same as 1, but there are now two models that are run: the generative model and the best candidate model.

See https://github.com/richelbilderbeek/pirouette_example_2 for the complete code.

richelbilderbeek/pirouette documentation built on Oct. 18, 2023, 3:09 p.m.