stepAIC: Choose a model by AIC in a Stepwise Algorithm
In MASS: Support Functions and Datasets for Venables and Ripley's MASS
Description
Usage
Arguments
Details
Value
Note
References
See Also
Examples
Description
Performs stepwise model selection by AIC.
Usage
1
2
3
4
Arguments
object
an object representing a model of an appropriate class.
This is used as the initial model in the stepwise search.
scope
defines the range of models examined in the stepwise search.
This should be either a single formula, or a list containing
components upper and lower, both formulae. See the
details for how to specify the formulae and how they are used.
scale
used in the definition of the AIC statistic for selecting the models,
currently only for lm and aov models
(see extractAIC for details).
direction
the mode of stepwise search, can be one of "both",
"backward", or "forward", with a default of "both".
If the scope argument is missing the default for
direction is "backward".
trace
if positive, information is printed during the running of
stepAIC.
Larger values may give more information on the fitting process.
keep
a filter function whose input is a fitted model object and the
associated AIC statistic, and whose output is arbitrary.
Typically keep will select a subset of the components of
the object and return them. The default is not to keep anything.
steps
the maximum number of steps to be considered. The default is 1000
(essentially as many as required). It is typically used to stop the
process early.
use.start
if true the updated fits are done starting at the linear predictor for
the currently selected model. This may speed up the iterative
calculations for glm (and other fits), but it can also slow them
down. Not used in R.
k
the multiple of the number of degrees of freedom used for the penalty.
Only k = 2 gives the genuine AIC: k = log(n) is
sometimes referred to as BIC or SBC.
...
any additional arguments to extractAIC. (None are currently used.)
Details
The set of models searched is determined by the scope argument.
The right-hand-side of its lower component is always included
in the model, and right-hand-side of the model is included in the
upper component. If scope is a single formula, it
specifies the upper component, and the lower model is
empty. If scope is missing, the initial model is used as the
upper model.
Models specified by scope can be templates to update
object as used by update.formula.
There is a potential problem in using glm fits with a
variable scale, as in that case the deviance is not simply
related to the maximized log-likelihood. The glm method for
extractAIC makes the
appropriate adjustment for a gaussian family, but may need to be
amended for other cases. (The binomial and poisson
families have fixed scale by default and do not correspond
to a particular maximum-likelihood problem for variable scale.)
Where a conventional deviance exists (e.g. for lm, aov
and glm fits) this is quoted in the analysis of variance table:
it is the unscaled deviance.
Value
the stepwise-selected model is returned, with up to two additional
components. There is an "anova" component corresponding to the
steps taken in the search, as well as a "keep" component if the
keep= argument was supplied in the call. The
"Resid. Dev" column of the analysis of deviance table refers
to a constant minus twice the maximized log likelihood: it will be a
deviance only in cases where a saturated model is well-defined
(thus excluding lm, aov and survreg fits,
for example).
Note
The model fitting must apply the models to the same dataset. This may
be a problem if there are missing values and an na.action other than
na.fail is used (as is the default in R).
We suggest you remove the missing values first.
References
Venables, W. N. and Ripley, B. D. (2002)
Modern Applied Statistics with S. Fourth edition. Springer.
See Also
Examples
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
quine.hi <- aov(log(Days + 2.5) ~ .^4, quine)
quine.nxt <- update(quine.hi, . ~ . - Eth:Sex:Age:Lrn)
quine.stp <- stepAIC(quine.nxt,
scope = list(upper = ~Eth*Sex*Age*Lrn, lower = ~1),
trace = FALSE)
quine.stp$anova
cpus1 <- cpus
for(v in names(cpus)[2:7])
cpus1[[v]] <- cut(cpus[[v]], unique(quantile(cpus[[v]])),
include.lowest = TRUE)
cpus0 <- cpus1[, 2:8] # excludes names, authors' predictions
cpus.samp <- sample(1:209, 100)
cpus.lm <- lm(log10(perf) ~ ., data = cpus1[cpus.samp,2:8])
cpus.lm2 <- stepAIC(cpus.lm, trace = FALSE)
cpus.lm2$anova
example(birthwt)
birthwt.glm <- glm(low ~ ., family = binomial, data = bwt)
birthwt.step <- stepAIC(birthwt.glm, trace = FALSE)
birthwt.step$anova
birthwt.step2 <- stepAIC(birthwt.glm, ~ .^2 + I(scale(age)^2)
+ I(scale(lwt)^2), trace = FALSE)
birthwt.step2$anova
quine.nb <- glm.nb(Days ~ .^4, data = quine)
quine.nb2 <- stepAIC(quine.nb)
quine.nb2$anova
Example output
Stepwise Model Path
Analysis of Deviance Table
Initial Model:
log(Days + 2.5) ~ Eth + Sex + Age + Lrn + Eth:Sex + Eth:Age +
Eth:Lrn + Sex:Age + Sex:Lrn + Age:Lrn + Eth:Sex:Age + Eth:Sex:Lrn +
Eth:Age:Lrn + Sex:Age:Lrn
Final Model:
log(Days + 2.5) ~ Eth + Sex + Age + Lrn + Eth:Sex + Eth:Age +
Eth:Lrn + Sex:Age + Sex:Lrn + Age:Lrn + Eth:Sex:Lrn + Eth:Age:Lrn
Step Df Deviance Resid. Df Resid. Dev AIC
1 120 64.09900 -68.18396
2 - Eth:Sex:Age 3 0.973869 123 65.07287 -71.98244
3 - Sex:Age:Lrn 2 1.526754 125 66.59962 -72.59652
Stepwise Model Path
Analysis of Deviance Table
Initial Model:
log10(perf) ~ syct + mmin + mmax + cach + chmin + chmax
Final Model:
log10(perf) ~ syct + mmax + cach + chmax
Step Df Deviance Resid. Df Resid. Dev AIC
1 82 2.820105 -320.8396
2 - chmin 3 0.05516656 85 2.875272 -324.9023
3 - mmin 3 0.14906175 88 3.024334 -325.8479
brthwt> bwt <- with(birthwt, {
brthwt+ race <- factor(race, labels = c("white", "black", "other"))
brthwt+ ptd <- factor(ptl > 0)
brthwt+ ftv <- factor(ftv)
brthwt+ levels(ftv)[-(1:2)] <- "2+"
brthwt+ data.frame(low = factor(low), age, lwt, race, smoke = (smoke > 0),
brthwt+ ptd, ht = (ht > 0), ui = (ui > 0), ftv)
brthwt+ })
brthwt> options(contrasts = c("contr.treatment", "contr.poly"))
brthwt> glm(low ~ ., binomial, bwt)
Call: glm(formula = low ~ ., family = binomial, data = bwt)
Coefficients:
(Intercept) age lwt raceblack raceother smokeTRUE
0.82302 -0.03723 -0.01565 1.19241 0.74068 0.75553
ptdTRUE htTRUE uiTRUE ftv1 ftv2+
1.34376 1.91317 0.68020 -0.43638 0.17901
Degrees of Freedom: 188 Total (i.e. Null); 178 Residual
Null Deviance: 234.7
Residual Deviance: 195.5 AIC: 217.5
Stepwise Model Path
Analysis of Deviance Table
Initial Model:
low ~ age + lwt + race + smoke + ptd + ht + ui + ftv
Final Model:
low ~ lwt + race + smoke + ptd + ht + ui
Step Df Deviance Resid. Df Resid. Dev AIC
1 178 195.4755 217.4755
2 - ftv 2 1.358185 180 196.8337 214.8337
3 - age 1 1.017866 181 197.8516 213.8516
Stepwise Model Path
Analysis of Deviance Table
Initial Model:
low ~ age + lwt + race + smoke + ptd + ht + ui + ftv
Final Model:
low ~ age + lwt + smoke + ptd + ht + ui + ftv + age:ftv + smoke:ui
Step Df Deviance Resid. Df Resid. Dev AIC
1 178 195.4755 217.4755
2 + age:ftv 2 12.474896 176 183.0006 209.0006
3 + smoke:ui 1 3.056805 175 179.9438 207.9438
4 - race 2 3.129586 177 183.0734 207.0734
Start: AIC=1095.32
Days ~ (Eth + Sex + Age + Lrn)^4
Df AIC
- Eth:Sex:Age:Lrn 2 1092.7
<none> 1095.3
Step: AIC=1092.73
Days ~ Eth + Sex + Age + Lrn + Eth:Sex + Eth:Age + Eth:Lrn +
Sex:Age + Sex:Lrn + Age:Lrn + Eth:Sex:Age + Eth:Sex:Lrn +
Eth:Age:Lrn + Sex:Age:Lrn
Df AIC
- Eth:Sex:Age 3 1089.4
<none> 1092.7
- Eth:Sex:Lrn 1 1093.3
- Eth:Age:Lrn 2 1094.7
- Sex:Age:Lrn 2 1095.0
Step: AIC=1089.41
Days ~ Eth + Sex + Age + Lrn + Eth:Sex + Eth:Age + Eth:Lrn +
Sex:Age + Sex:Lrn + Age:Lrn + Eth:Sex:Lrn + Eth:Age:Lrn +
Sex:Age:Lrn
Df AIC
<none> 1089.4
- Sex:Age:Lrn 2 1091.1
- Eth:Age:Lrn 2 1091.2
- Eth:Sex:Lrn 1 1092.5
Stepwise Model Path
Analysis of Deviance Table
Initial Model:
Days ~ (Eth + Sex + Age + Lrn)^4
Final Model:
Days ~ Eth + Sex + Age + Lrn + Eth:Sex + Eth:Age + Eth:Lrn +
Sex:Age + Sex:Lrn + Age:Lrn + Eth:Sex:Lrn + Eth:Age:Lrn +
Sex:Age:Lrn
Step Df Deviance Resid. Df Resid. Dev AIC
1 118 167.4535 1095.324
2 - Eth:Sex:Age:Lrn 2 0.09746244 120 167.5509 1092.728
3 - Eth:Sex:Age 3 0.11060087 123 167.4403 1089.409
MASS documentation built on Feb. 12, 2021, 5:15 p.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Description Usage Arguments Details Value Note References See Also Examples
Description
Performs stepwise model selection by AIC.
Usage
1 2 3 4 |
Arguments
object |
an object representing a model of an appropriate class. This is used as the initial model in the stepwise search. |
scope |
defines the range of models examined in the stepwise search.
This should be either a single formula, or a list containing
components |
scale |
used in the definition of the AIC statistic for selecting the models,
currently only for |
direction |
the mode of stepwise search, can be one of |
trace |
if positive, information is printed during the running of
|
keep |
a filter function whose input is a fitted model object and the
associated |
steps |
the maximum number of steps to be considered. The default is 1000 (essentially as many as required). It is typically used to stop the process early. |
use.start |
if true the updated fits are done starting at the linear predictor for
the currently selected model. This may speed up the iterative
calculations for |
k |
the multiple of the number of degrees of freedom used for the penalty.
Only |
... |
any additional arguments to |
Details
The set of models searched is determined by the scope argument.
The right-hand-side of its lower component is always included
in the model, and right-hand-side of the model is included in the
upper component. If scope is a single formula, it
specifies the upper component, and the lower model is
empty. If scope is missing, the initial model is used as the
upper model.
Models specified by scope can be templates to update
object as used by update.formula.
There is a potential problem in using glm fits with a
variable scale, as in that case the deviance is not simply
related to the maximized log-likelihood. The glm method for
extractAIC makes the
appropriate adjustment for a gaussian family, but may need to be
amended for other cases. (The binomial and poisson
families have fixed scale by default and do not correspond
to a particular maximum-likelihood problem for variable scale.)
Where a conventional deviance exists (e.g. for lm, aov
and glm fits) this is quoted in the analysis of variance table:
it is the unscaled deviance.
Value
the stepwise-selected model is returned, with up to two additional
components. There is an "anova" component corresponding to the
steps taken in the search, as well as a "keep" component if the
keep= argument was supplied in the call. The
"Resid. Dev" column of the analysis of deviance table refers
to a constant minus twice the maximized log likelihood: it will be a
deviance only in cases where a saturated model is well-defined
(thus excluding lm, aov and survreg fits,
for example).
Note
The model fitting must apply the models to the same dataset. This may
be a problem if there are missing values and an na.action other than
na.fail is used (as is the default in R).
We suggest you remove the missing values first.
References
Venables, W. N. and Ripley, B. D. (2002) Modern Applied Statistics with S. Fourth edition. Springer.
See Also
Examples
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | quine.hi <- aov(log(Days + 2.5) ~ .^4, quine)
quine.nxt <- update(quine.hi, . ~ . - Eth:Sex:Age:Lrn)
quine.stp <- stepAIC(quine.nxt,
scope = list(upper = ~Eth*Sex*Age*Lrn, lower = ~1),
trace = FALSE)
quine.stp$anova
cpus1 <- cpus
for(v in names(cpus)[2:7])
cpus1[[v]] <- cut(cpus[[v]], unique(quantile(cpus[[v]])),
include.lowest = TRUE)
cpus0 <- cpus1[, 2:8] # excludes names, authors' predictions
cpus.samp <- sample(1:209, 100)
cpus.lm <- lm(log10(perf) ~ ., data = cpus1[cpus.samp,2:8])
cpus.lm2 <- stepAIC(cpus.lm, trace = FALSE)
cpus.lm2$anova
example(birthwt)
birthwt.glm <- glm(low ~ ., family = binomial, data = bwt)
birthwt.step <- stepAIC(birthwt.glm, trace = FALSE)
birthwt.step$anova
birthwt.step2 <- stepAIC(birthwt.glm, ~ .^2 + I(scale(age)^2)
+ I(scale(lwt)^2), trace = FALSE)
birthwt.step2$anova
quine.nb <- glm.nb(Days ~ .^4, data = quine)
quine.nb2 <- stepAIC(quine.nb)
quine.nb2$anova
|
Example output
Stepwise Model Path
Analysis of Deviance Table
Initial Model:
log(Days + 2.5) ~ Eth + Sex + Age + Lrn + Eth:Sex + Eth:Age +
Eth:Lrn + Sex:Age + Sex:Lrn + Age:Lrn + Eth:Sex:Age + Eth:Sex:Lrn +
Eth:Age:Lrn + Sex:Age:Lrn
Final Model:
log(Days + 2.5) ~ Eth + Sex + Age + Lrn + Eth:Sex + Eth:Age +
Eth:Lrn + Sex:Age + Sex:Lrn + Age:Lrn + Eth:Sex:Lrn + Eth:Age:Lrn
Step Df Deviance Resid. Df Resid. Dev AIC
1 120 64.09900 -68.18396
2 - Eth:Sex:Age 3 0.973869 123 65.07287 -71.98244
3 - Sex:Age:Lrn 2 1.526754 125 66.59962 -72.59652
Stepwise Model Path
Analysis of Deviance Table
Initial Model:
log10(perf) ~ syct + mmin + mmax + cach + chmin + chmax
Final Model:
log10(perf) ~ syct + mmax + cach + chmax
Step Df Deviance Resid. Df Resid. Dev AIC
1 82 2.820105 -320.8396
2 - chmin 3 0.05516656 85 2.875272 -324.9023
3 - mmin 3 0.14906175 88 3.024334 -325.8479
brthwt> bwt <- with(birthwt, {
brthwt+ race <- factor(race, labels = c("white", "black", "other"))
brthwt+ ptd <- factor(ptl > 0)
brthwt+ ftv <- factor(ftv)
brthwt+ levels(ftv)[-(1:2)] <- "2+"
brthwt+ data.frame(low = factor(low), age, lwt, race, smoke = (smoke > 0),
brthwt+ ptd, ht = (ht > 0), ui = (ui > 0), ftv)
brthwt+ })
brthwt> options(contrasts = c("contr.treatment", "contr.poly"))
brthwt> glm(low ~ ., binomial, bwt)
Call: glm(formula = low ~ ., family = binomial, data = bwt)
Coefficients:
(Intercept) age lwt raceblack raceother smokeTRUE
0.82302 -0.03723 -0.01565 1.19241 0.74068 0.75553
ptdTRUE htTRUE uiTRUE ftv1 ftv2+
1.34376 1.91317 0.68020 -0.43638 0.17901
Degrees of Freedom: 188 Total (i.e. Null); 178 Residual
Null Deviance: 234.7
Residual Deviance: 195.5 AIC: 217.5
Stepwise Model Path
Analysis of Deviance Table
Initial Model:
low ~ age + lwt + race + smoke + ptd + ht + ui + ftv
Final Model:
low ~ lwt + race + smoke + ptd + ht + ui
Step Df Deviance Resid. Df Resid. Dev AIC
1 178 195.4755 217.4755
2 - ftv 2 1.358185 180 196.8337 214.8337
3 - age 1 1.017866 181 197.8516 213.8516
Stepwise Model Path
Analysis of Deviance Table
Initial Model:
low ~ age + lwt + race + smoke + ptd + ht + ui + ftv
Final Model:
low ~ age + lwt + smoke + ptd + ht + ui + ftv + age:ftv + smoke:ui
Step Df Deviance Resid. Df Resid. Dev AIC
1 178 195.4755 217.4755
2 + age:ftv 2 12.474896 176 183.0006 209.0006
3 + smoke:ui 1 3.056805 175 179.9438 207.9438
4 - race 2 3.129586 177 183.0734 207.0734
Start: AIC=1095.32
Days ~ (Eth + Sex + Age + Lrn)^4
Df AIC
- Eth:Sex:Age:Lrn 2 1092.7
<none> 1095.3
Step: AIC=1092.73
Days ~ Eth + Sex + Age + Lrn + Eth:Sex + Eth:Age + Eth:Lrn +
Sex:Age + Sex:Lrn + Age:Lrn + Eth:Sex:Age + Eth:Sex:Lrn +
Eth:Age:Lrn + Sex:Age:Lrn
Df AIC
- Eth:Sex:Age 3 1089.4
<none> 1092.7
- Eth:Sex:Lrn 1 1093.3
- Eth:Age:Lrn 2 1094.7
- Sex:Age:Lrn 2 1095.0
Step: AIC=1089.41
Days ~ Eth + Sex + Age + Lrn + Eth:Sex + Eth:Age + Eth:Lrn +
Sex:Age + Sex:Lrn + Age:Lrn + Eth:Sex:Lrn + Eth:Age:Lrn +
Sex:Age:Lrn
Df AIC
<none> 1089.4
- Sex:Age:Lrn 2 1091.1
- Eth:Age:Lrn 2 1091.2
- Eth:Sex:Lrn 1 1092.5
Stepwise Model Path
Analysis of Deviance Table
Initial Model:
Days ~ (Eth + Sex + Age + Lrn)^4
Final Model:
Days ~ Eth + Sex + Age + Lrn + Eth:Sex + Eth:Age + Eth:Lrn +
Sex:Age + Sex:Lrn + Age:Lrn + Eth:Sex:Lrn + Eth:Age:Lrn +
Sex:Age:Lrn
Step Df Deviance Resid. Df Resid. Dev AIC
1 118 167.4535 1095.324
2 - Eth:Sex:Age:Lrn 2 0.09746244 120 167.5509 1092.728
3 - Eth:Sex:Age 3 0.11060087 123 167.4403 1089.409
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.