# Compute Model Selection Relative Frequencies

### Description

This function computes the model selection relative frequencies based on
the nonparametric bootstrap (Burnham and Anderson 2002). Models are
ranked based on the AIC, AICc, QAIC, or QAICc. The function currently
supports objects of `aov`

, `betareg`

, `clm`

, `glm`

,
`hurdle`

, `lm`

, `multinom`

, `polr`

, `rlm`

,
`survreg`

, `vglm`

, and `zeroinfl`

classes.

### Usage

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 | ```
boot.wt(cand.set, modnames = NULL, second.ord = TRUE, nobs = NULL,
sort = TRUE, nsim = 100, ...)
## S3 method for class 'AICaov.lm'
boot.wt(cand.set, modnames = NULL, second.ord = TRUE, nobs = NULL,
sort = TRUE, nsim = 100, ...)
## S3 method for class 'AICsurvreg'
boot.wt(cand.set, modnames = NULL, second.ord = TRUE, nobs = NULL,
sort = TRUE, nsim = 100, ...)
## S3 method for class 'AICsclm.clm'
boot.wt(cand.set, modnames = NULL, second.ord = TRUE, nobs = NULL,
sort = TRUE, nsim = 100, ...)
## S3 method for class 'AICglm.lm'
boot.wt(cand.set, modnames = NULL, second.ord = TRUE, nobs = NULL,
sort = TRUE, nsim = 100, c.hat = 1, ...)
## S3 method for class 'AIChurdle'
boot.wt(cand.set, modnames = NULL, second.ord = TRUE, nobs = NULL,
sort = TRUE, nsim = 100, ...)
## S3 method for class 'AIClm'
boot.wt(cand.set, modnames = NULL, second.ord = TRUE, nobs = NULL,
sort = TRUE, nsim = 100, ...)
## S3 method for class 'AICmultinom.nnet'
boot.wt(cand.set, modnames = NULL, second.ord = TRUE, nobs = NULL,
sort = TRUE, nsim = 100, c.hat = 1, ...)
## S3 method for class 'AICpolr'
boot.wt(cand.set, modnames = NULL, second.ord = TRUE, nobs = NULL,
sort = TRUE, nsim = 100, ...)
## S3 method for class 'AICrlm.lm'
boot.wt(cand.set, modnames = NULL, second.ord = TRUE, nobs = NULL,
sort = TRUE, nsim = 100, ...)
## S3 method for class 'AICsurvreg'
boot.wt(cand.set, modnames = NULL, second.ord = TRUE, nobs = NULL,
sort = TRUE, nsim = 100, ...)
## S3 method for class 'AICvglm'
boot.wt(cand.set, modnames = NULL, second.ord = TRUE, nobs = NULL,
sort = TRUE, nsim = 100, c.hat = 1, ...)
## S3 method for class 'AICzeroinfl'
boot.wt(cand.set, modnames = NULL, second.ord = TRUE, nobs = NULL,
sort = TRUE, nsim = 100, ...)
``` |

### Arguments

`cand.set` |
a list storing each of the models in the candidate model set. |

`modnames` |
a character vector of model names to facilitate the identification of
each model in the model selection table. If |

`second.ord` |
logical. If |

`nobs` |
this argument allows to specify a numeric value other than total sample
size to compute the AICc (i.e., |

`sort` |
logical. If |

`c.hat` |
value of overdispersion parameter (i.e., variance inflation factor) such
as that obtained from |

`nsim` |
the number of bootstrap iterations. Burnham and Anderson (2002) recommend at least 1000 and up to 10 000 iterations for certain problems. |

`...` |
additional arguments passed to the function. |

### Details

`boot.wt`

is implemented for `aov`

, `betareg`

,
`glm`

, `hurdle`

, `lm`

, `multinom`

, `polr`

,
`rlm`

, `survreg`

, `vglm`

, and `zeroinfl`

classes.
During each bootstrap iteration, the data are resampled with
replacement, all the models specified in `cand.set`

are updated
with the new data set, and the top-ranked model is saved. When all
iterations are completed, the relative frequency of selection is
computed for each model appearing in the candidate model set.

Relative frequencies of the models are often similar to Akaike
weights, and the latter are often preferred due to their link with
a Bayesian perspective (Burnham and Anderson 2002). `boot.wt`

is
most useful for teaching purposes of sampling-theory based relative
frequencies of model selection. The current implementation is only
appropriate with completely randomized designs. For more complex data
structures (e.g., blocks or random effects), the bootstrap should be
modified accordingly.

### Value

`boot.wt`

creates an object of class `boot.wt`

with the
following components:

`Modname` |
the names of each model of the candidate model set. |

`K ` |
the number of estimated parameters for each model. |

`(Q)AIC(c) ` |
the information criteria requested for each model (AICc, AICc, QAIC, QAICc). |

`Delta_(Q)AIC(c) ` |
the appropriate delta AIC component depending on the information criteria selected. |

`ModelLik ` |
the relative likelihood of the model given the data (exp(-0.5*delta[i])). This is not to be confused with the likelihood of the parameters given the data. The relative likelihood can then be normalized across all models to get the model probabilities. |

`(Q)AIC(c)Wt` |
the Akaike weights, also termed "model probabilities" sensu Burnham and Anderson (2002) and Anderson (2008). These measures indicate the level of support (i.e., weight of evidence) in favor of any given model being the most parsimonious among the candidate model set. |

`PiWt ` |
the relative frequencies of model selection from the bootstrap. |

`c.hat` |
if c.hat was specified as an argument, it is included in the table. |

### Author(s)

Marc J. Mazerolle

### References

Anderson, D. R. (2008) *Model-based Inference in the Life Sciences:
a primer on evidence*. Springer: New York.

Burnham, K. P., Anderson, D. R. (2002) *Model Selection and
Multimodel Inference: a practical information-theoretic
approach*. Second edition. Springer: New York.

Burnham, K. P., Anderson, D. R. (2004) Multimodel inference:
understanding AIC and BIC in model selection. *Sociological Methods
and Research* **33**, 261–304.

Mazerolle, M. J. (2006) Improving data analysis in herpetology: using
Akaike's Information Criterion (AIC) to assess the strength of
biological hypotheses. *Amphibia-Reptilia* **27**, 169–180.

### See Also

`AICc`

, `confset`

, `c_hat`

,
`evidence`

, `importance`

, `modavg`

,
`modavgShrink`

, `modavgPred`

### Examples

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 | ```
##Mazerolle (2006) frog water loss example
data(dry.frog)
##setup a subset of models of Table 1
Cand.models <- list( )
Cand.models[[1]] <- lm(log_Mass_lost ~ Shade + Substrate +
cent_Initial_mass + Initial_mass2,
data = dry.frog)
Cand.models[[2]] <- lm(log_Mass_lost ~ Shade + Substrate +
cent_Initial_mass + Initial_mass2 +
Shade:Substrate, data = dry.frog)
Cand.models[[3]] <- lm(log_Mass_lost ~ cent_Initial_mass +
Initial_mass2, data = dry.frog)
Cand.models[[4]] <- lm(log_Mass_lost ~ Shade + cent_Initial_mass +
Initial_mass2, data = dry.frog)
Cand.models[[5]] <- lm(log_Mass_lost ~ Substrate + cent_Initial_mass +
Initial_mass2, data = dry.frog)
##create a vector of names to trace back models in set
Modnames <- paste("mod", 1:length(Cand.models), sep = " ")
##generate AICc table with bootstrapped relative
##frequencies of model selection
boot.wt(cand.set = Cand.models, modnames = Modnames, sort = TRUE,
nsim = 10) #number of iterations should be much higher
##Burnham and Anderson (2002) flour beetle data
## Not run:
data(beetle)
##models as suggested by Burnham and Anderson p. 198
Cand.set <- list( )
Cand.set[[1]] <- glm(Mortality_rate ~ Dose, family =
binomial(link = "logit"), weights = Number_tested,
data = beetle)
Cand.set[[2]] <- glm(Mortality_rate ~ Dose, family =
binomial(link = "probit"), weights = Number_tested,
data = beetle)
Cand.set[[3]] <- glm(Mortality_rate ~ Dose, family =
binomial(link ="cloglog"), weights = Number_tested,
data = beetle)
##create a vector of names to trace back models in set
Modnames <- paste("Mod", 1:length(Cand.set), sep = " ")
##model selection table with bootstrapped
##relative frequencies
boot.wt(cand.set = Cand.set, modnames = Modnames)
## End(Not run)
``` |