FAMoS provides an automated and unbiased model selection algorithm that aims at determining the most appropriate subset of model parameters to describe a specific data set. Due to its flexibility with respect to the cost/optimisation function, FAMoS can handle many different mathematical structures, including for example regression models and ODEs.

You can install FAMoS from github with:

```
# install.packages("devtools")
devtools::install_github("GabelHub/FAMoS")
# alternative installation command
devtools::install_git("git://github.com/GabelHub/FAMoS.git", branch = "master")
```

FAMoS uses three different methods to find appropriate models to test:

- Forward search: Add a parameter to the currently best model
- Backward elimination: Remove a parameter from the currently best model
- Swap search: Replace a parameter of the currently best model by another. The parameters that can be swapped need to be specified by the user

FAMoS keeps track of the methods used in the previous iterations and dynamically changes them according to the outcome of each iteration.

FAMoS is designed to allow for a maximum of flexibility regarding the
fitting procedures and model types in R. While it comes with the default
option to fit the cost function via *optim*, it also allows the users to
specify their own optimisation routines, hence making it possible to
perform model selection based on various other R packages.

FAMoS makes use of the future-package which allows for easy parallelisation, meaning many different models can be tested simultaneously if the required computational resources are available.

FAMoS keeps track of previously tested models and checks also that each model fulfills all user-specified restrictions, therefore testing only relevant models and saving computational resources.

As a simple example, we generate a simple data set generated by two parameters and apply the FAMoS on a global model consisting of five different parameters.

```
library(FAMoS)
#setting data
true.p2 <- 3
true.p5 <- 2
sim.data <- cbind.data.frame(range = 1:10,
y = true.p2^2 * (1:10)^2 - exp(true.p5 * (1:10)))
#define initial parameter values and corresponding test function
inits <- c(p1 = 3, p2 = 4, p3 = -2, p4 = 2, p5 = 0)
cost_function <- function(parms, binary, data){
if(max(abs(parms)) > 5){
return(NA)
}
with(as.list(c(parms)), {
res <- p1*4 + p2^2*data$range^2 + p3*sin(data$range) + p4*data$range - exp(p5*data$range)
diff <- sum((res - data$y)^2)
#calculate AICC
nr.par <- length(which(binary == 1))
nr.data <- nrow(data)
AICC <- diff + 2*nr.par + 2*nr.par*(nr.par + 1)/(nr.data - nr.par -1)
return(AICC)
})
}
#set swap set
swaps <- list(c("p1", "p5"))
#perform model selection
res <- famos(init.par = inits,
fit.fn = cost_function,
homedir = tempdir(),
method = "swap",
swap.parameters = swaps,
init.model.type = c("p1", "p3"),
optim.runs = 1,
data = sim.data,
future.off = TRUE)
```

The FAMoS returns a lot of verbose output, telling the user what’s
currently happening (Note: The output can be turned on and off by using
the option *verbose*). In the beginning, the overall settings are
defined and the corresponding directories are created (if they don’t
exist).

```
#> Initializing...
#> Create FAMoS directory...
#>
#> Algorithm run: 001
#> Refitting disabled.
#> Starting algorithm with method 'swap'
```

In each iteration, the FAMoS identifies new models to be tested based on the current search method:

```
#> FAMoS iteration #3 - method: forward
#> Add parameter p1
#> Add parameter p2
#> Add parameter p4
#> Time passed since start: 00:00:00
```

Each model will be submitted and tested. Since FAMoS uses futures for
evaluation, the search process can be easily parallelised by setting the
corresponding future plan. Every model is subsequently evaluated by
performing (multiple) optimisation routines based either on the default
fitting routine *optim* or a user-specified fitting routine (see the
vignettes for examples).

After all models have been evaluated, the algorithm reads in the results and checks, if a better model was found

```
#> Evaluate results ...
#> Best selection criterion value of this run is 10
#> Parameter p2 was added
#> Time passed since start: 1.92 secs
```

The cycle continues until no better model can be found based on the currently used methods. After halting, the results are returned

```
#> Best model found. Algorithm stopped.
#> FAMoS run 001
#> Selection criterion value of best model: 7
#> Best model (binary): 01001
#> Best model (vector):
#> p1 p2 p3 p4 p5
#> 0 1 0 0 1
#> Estimated parameter values:
#> p1 p2 p3 p4 p5
#> 0 -3 0 0 2
#> Time needed: 3.84 secs
```

The vector *init.par* is one of the three essential variables that need
to be specified. It contains the names and initial values of *all*
parameters, that the FAMoS is supposed search through. In our example
above, we specified this vector as

```
#define initial parameter values
inits <- c(p1 = 3, p2 = 4, p3 = -2, p4 = 2, p5 = 0)
```

Depending on the starting model, FAMoS automatically extracts the corresponding values and uses them for its first iteration only. All following iterations inherit the best values found during fitting.

Additional specifications for the use of the inital parameter vector can
be supplied by the options *do.not.fit* and *default.val*.

To allow independence of specific mathematical model structures, the user can specify any cost function. The cost function has to take the complete parameter vector as an input and has to return a selection criterion value. If use.optim = TRUE, the cost function needs to return a single numeric value, which corresponds to the selection criterion value. However, if use.optim = FALSE, the cost function needs to return a list containing in its first entry the selection criterion value and in its second entry the named vector of the fitted parameter values (non-fitted parameters are internally assessed).

Due to this flexible structure, FAMoS is able to tackle many different problems, e.g. modelling approaches like linear regression, ODEs or PDEs.

The FAMoS generates and saves many different files, in order to make
results available over time as well as to simultaneously running FAMoS
runs. *homedir* specifies the folder, in which all results are going to
be stored. The default is set to the current working directory.

In order to exclude some parameters from the fitting procedures, their
names can be specified in the *do.not.fit* option. This allows to test
different model restrictions without needing to change either *init.par*
or *fit.fn*. For example, if we wanted to exclude the parameter *p4*
from our analysis, we would specify initially

```
#define initial parameter values
inits <- c(p1 = 3, p2 = 4, p3 = -2, p4 = 2, p5 = 0)
no.fit <- c("p4")
```

and pass this option on to FAMoS. Note that excluded parameters are
automatically removed from the initial model, if *init.model.type =
“random”* or *init.model.type = “global”* is used. If the
user-specified initial model contains an excluded parameter, an error
will be
returned.

```
The specified initial model violates critical conditions or the do.not.fit specifications
```

FAMoS can use three different methods to search for different models to
test: Forward search, backward elimination and swap search. As the
algorithm dynamically changes these methods over the course of each
iteration, the option *method* only specifies the starting method.

If the algorithm is able to find a better model, the current method will be used in the next iteration as well (except the swap method, which always uses a forward search next - if it doesn’t terminate in that step). If no better model is found, the algorithm will change the method according to the following scheme:

| current method | previous method | next method | | :------------: | :-----------------: | :-----------------: | | forward | backward | swap (or terminate) | | forward | forward or swap | backward | | backward | backward | forward | | backward | forward | swap (or terminate) | | swap | forward or backward | terminate |

In case the swap method is not used (due to unspecified critical or swap sets), the algorithm will terminate after a succession of an unsuccessful forward and backward search.

To verify if the FAMoS results are consistent, it is important to run
the algorithm with different starting models. To set the initial model,
the user can either use the built-in options *random* (which generates a
random model), *global* (which uses the complete model as a starting
point) or *most.distant* (uses the model most dissimilar to all
previously tested models). Alternatively, the user can specify a model
by supplying a parameter vector containing the names of the initial
model.

```
#Three options for the starting model
init.model1 <- "random" # generates a random starting model
init.model2 <- "global" # uses all available parameters
init.model3 <- "most.distant" # uses the most dissimilar model
init.model4 <- c("p1", "p4") # a user-specified model
```

In case *random* or *global* are chosen, the FAMoS automatically applies
critical conditions and removes excluded parameters (see options
*critical.parameters* and *do.not.fit*).

Before testing a model, the FAMoS checks if this model has been tested
before. In case *refit = FALSE* (default) is specified, the model will
not be tested again. If refitting is set to TRUE, FAMoS will try to
optimise the model again. If the new run returns a better fit, the old
results will be overwritten, otherwise the new run will be discarded.

Refitting makes sense if the model optimisation is dependent on the
initial parameter combination (see also *optim.runs*). If a model is
reencountered, it might well be that the new parameter set to be tested
with is much more appropriate than the previous one, especially if this
reencounter happens within the same FAMoS run.

The default fitting routine that FAMoS relies on is the built-in
function *optim*. However, by setting *use.optim = FALSE*, the user can
use any other fitting routine suitable. The fitting routine then has to
be included in the cost function *fit.fn* which needs to return a list
containing the current selection value criterion as well as the
parameter values used. See the vignettes for an example.

Finding the best fit for each model is crucial to guarantee a correct
model selection procedure. Often, fitting a model once is enough and
repeating the fitting procedure with different initial conditions does
not lead to new results. Sometimes, however, one wants to run multiple
fits for each model, e.g. if the parameter space is very large. To do
so, the user can specify *optim.runs*, which gives the number of fitting
attempts. For each optimisation run a *different* starting condition is
used. The first fitting attempt takes the inherited parameter vectors
from previous runs, while all following fitting attempts randomly
samples parameter vectors to test (see also *random.borders*).

If multiple optimisation runs are performed, FAMoS will return the best of these runs.

In each optimisation run fitting in FAMoS is either performed with the
built-in function *optim*, which is repeatedly evaluated until
convergence, or a custom optimisation routine, which is evaluated only
once. As the default optimisation method is based on the Nelder-Mead
approach, which often tends to not give reliable results if only one
optimisation is performed, the optimisation for each fitting attempt is
wrapped into a while-loop, in which the fitting procedure is repeatedly
halted and restarted (based on the options *control.optim*), until the
relative convergence tolerance in *con.tol* is reached.

The skeleton of the underlying code looks like this:

```
for(i in 1:optim.runs){#number of fitting attempts specified by optim.runs
start.parameters <- either the inherited or a randomly sampled set (for i > 2)
if(use.optim == TRUE){
#If use.optim = TRUE, the fitting routine is evaluated in a while loop
while(abs((old.optim.value - new.optim.value)/old.optim.value) < con.tol){
... run optim with start.parameters ...
start.parameters <- new parameters estimated by optim
}
}else{
#If use.optim = FALSE, the custom optimisation routine is evaluated
#only once in each optimisation run
... run custom optimisation with start parameters ...
}
}
```

Normally, FAMoS sets the parameters that are not fitted equal to zero.
However, this might not be appropriate if, for example, a parameter
describes an initial condition or a baseline turnover. Here,
*default.val* allows to specify the value that a parameter assumes, if
it is not fitted. *default.val* needs to be given as a named list, which
can either store numerical values or the name of the parameter from
which the value should be inherited. For example

```
#define initial parameter values
inits <- c(p1 = 3, p2 = 4, p3 = -2, p4 = 2, p5 = 0)
#set default values
def.val <- list(p1 = 2, p2 = -5, p3 = "p1", p4 = 0, p5 = "p4")
```

Here, the values of *p1*, *p2*, and *p4* are set to their respective
values. However, *p3* and *p5* will inherit their values from *p1* and
*p4*, respectively. This feature is useful if two rates describe similar
processes and one wants to test if the difference between them is
significant enough to warrant the fitting of an additional parameter.
Here’s a short example

```
cost.function <- function(parms){
x <- par1 + par2*x
y <- par3 + par4*x
}
def.val <- list(p1 = 0, p2 = 0, p3 = "p1", p4 = "p2")
```

Note that the parameter inheritance cannot be chained, meaning that entries that point to another parameter need a numeric value to access

```
#INCORRECT use of default.val
def.val <- list(p1 = 1, p2 = "p1", p3 = "p2", p4 = "p3")
#CORRECT use of default.val
def.val <- list(p1 = 1, p2 = "p1", p3 = "p1", p4 = "p1")
```

The swap search that FAMoS can perform relies on sets which specify
parameters that can be swapped by one another. For example, if we wanted
to allow parameters *p1*, *p2* and *p3*, as well as *p4* and *p5* to be
replaceable by each other, we would specify:

```
swap.set <- list(c("p1", "p2", "p3"), c("p4", "p5"))
```

In some cases, it does not make sense to fit certain submodels of the
global model to the data, as they might lack crucial parameters. FAMoS
can incorporate these restrictions by the specification of critical
parameter sets. For example, if at least one of the first three
parameters need to be present in the model, and all models that don’t
feature *p4* are not correct, we can specify:

```
crit.set <- list(c("p1", "p2", "p3"), c("p4"))
```

All critical sets are also automatically used in the swap search.

Since the parameters of all *optim.runs* larger than one are sampled
based on a random uniform distribution, it might be important to set the
correct sampling intervals. By default, FAMoS samples parameters with a
100% deviation of the inherited parameter values (for example, if a
model contains two parameters, and the currently best values are *p1 =
0.1* and *p2 = -1000*, the sampled values will lie in the intervals
[0,0.2] and [-2000,0], respectively). Alternatively, the user can
specify relative or absolute sampling intervals. For relative intervals,
a numeric value has to be given for each parameter denoting its relative
deviation. For absolute sampling intervals, a matrix containing the
lower and upper borders has to be specified. Here’s an example:

```
#relative sampling ranges
random.bord1 <- 0.3 # deviates all parameters by 30%
random.bord2 <- c(0.1, 0.5, 0.2) # deviates the parameters by 10%, 50% and 20%, respectively
#absolute sampling ranges
random.bord3 <- matrix(c(1,2), nrow = 1) #uses the interval [1,2] for all parameter samples
random.bord4 <- cbind(c(0,-10, 0.3), c(5, -9, 0.7)) #uses the intervals [0,5], [-10,-9] and [0.3, 0.7] to sample the respective parameters
#use a function to sample the results
random.bord5 <- rnorm #note that in this case, 'mean' and 'sd' need to passed to famos as well, if other values than the default settings should be used
```

Specifies the control options used for *optim* (see *optim.runs* for
more details).

If parameters values span over several orders of magnitudes, using the
built-in option *parscale* in *optim* can reduce the numbers of
evaluations needed. Setting *parscale.pars = TRUE* automatically adjusts
the scaling procedure in *optim*. In our experience, using
*parscale.pars = TRUE* is usually beneficial if a large number of
parameters with different orders of magnitude need to be fitted.
However, the actual performance is very problem-specific and therefore
we would recommend initially testing both approaches to see which one
performs better for the problem at hand. Also, one needs to make sure
that the other options given in *control.optim* and *con.tol* are
specified appropriately.

Specifies the relative convergence tolerance and determines when the
repeated use *optim* fits will be terminated (see *optim.runs* for more
details).

If true, a plot of the current FAMoS performance is stored in the folder “FAMoS-Results/Figures/”, which will be updated during each iteration.

To allow for parallelisation, FAMoS uses the *future* package by Henrik
Bengtsson (see https://github.com/HenrikBengtsson/future). To use
futures, the option needs to be set to TRUE and a future plan needs to
be specified.

If futures are used during a FAMoS run, there will be a message printed
every *X* seconds, informing the user which models fits are still
running. *log.interval* allows to specify the interval of *X*. Default
to 10 minutes (600 seconds).

The verbose output of FAMoS can be turned on and off. If *verbose =
FALSE*, only a minimum of information is shown.

**Any scripts or data that you put into this service are public.**

Embedding an R snippet on your website

Add the following code to your website.

For more information on customizing the embed code, read Embedding Snippets.