fs: Forward stepwise regression
In selectiveInference: Tools for Post-Selection Inference

Description Usage Arguments Details Value Author(s) See Also Examples

This function implements forward stepwise regression, for use in the selectiveInference package

1	fs(x, y, maxsteps=2000, intercept=TRUE, normalize=TRUE, verbose=FALSE)

`x`	Matrix of predictors (n by p)
`y`	Vector of outcomes (length n)
`maxsteps`	Maximum number of steps to take
`intercept`	Should an intercept be included on the model? Default is TRUE
`normalize`	Should the predictors be normalized? Default is TRUE. (Note: this argument has no real effect on model selection since forward stepwise is scale invariant already; however, it is included for completeness, and to match the interface for the `lar` function)
`verbose`	Print out progress along the way? Default is FALSE

This function implements forward stepwise regression, adding the predictor at each step that maximizes the absolute correlation between the predictors—once orthogonalized with respect to the current model—and the residual. This entry criterion is standard, and is equivalent to choosing the variable that achieves the biggest drop in RSS at each step; it is used, e.g., by the step function in R. Note that, for example, the lars package implements a stepwise option (with type="step"), but uses a (mildly) different entry criterion, based on maximal absolute correlation between the original (non-orthogonalized) predictors and the residual.

`action`	Vector of predictors in order of entry
`sign`	Signs of coefficients of predictors, upon entry
`df`	Degrees of freedom of each active model
`beta`	Matrix of regression coefficients for each model along the path, one column per model
`completepath`	Was the complete stepwise path computed?
`bls`	If completepath is TRUE, the full least squares coefficients
`Gamma`	Matrix that captures the polyhedral selection at each step
`nk`	Number of polyhedral constraints at each step in path
`vreg`	Matrix of linear contrasts that gives coefficients of variables to enter along the path
`x`	Matrix of predictors used
`y`	Vector of outcomes used
`bx`	Vector of column means of original x
`by`	Mean of original y
`sx`	Norm of each column of original x
`intercept`	Was an intercept included?
`normalize`	Were the predictors normalized?
`call`	The call to fs

Ryan Tibshirani, Rob Tibshirani, Jonathan Taylor, Joshua Loftus, Stephen Reid

fsInf, predict.fs,coef.fs, plot.fs

set.seed(33)
n = 50
p = 10
sigma = 1
x = matrix(rnorm(n*p),n,p)
beta = c(3,2,rep(0,p-2))
y = x%*%beta + sigma*rnorm(n)

# run forward stepwise, plot results
fsfit = fs(x,y)
plot(fsfit)

# compute sequential p-values and confidence intervals
# (sigma estimated from full model)
out = fsInf(fsfit)
out

Loading required package: glmnet
Loading required package: Matrix
Loading required package: foreach
Loaded glmnet 2.0-16

Loading required package: intervals

Attaching package: 'intervals'

The following object is masked from 'package:Matrix':

    expand

Loading required package: survival

Call:
fsInf(obj = fsfit)

Standard deviation of noise (specified or estimated) sigma = 1.027

Sequential testing results with alpha = 0.100
 Step Var   Coef Z-score P-value LowConfPt UpConfPt LowTailArea UpTailArea
    1   1  2.317  13.406   0.000     2.019    2.605       0.049      0.048
    2   2  1.703  12.996   0.000     1.486    1.922       0.048      0.050
    3   9 -0.265  -1.683   0.487    -0.782    1.152       0.050      0.050
    4   8 -0.175  -1.156   0.260    -4.764    1.532       0.050      0.050
    5  10  0.173   1.075   0.755   -12.195    3.056       0.050      0.050
    6   4 -0.178  -1.140   0.407   -11.057    7.428       0.050      0.050
    7   7  0.158   0.979   0.763    -9.225    2.137       0.050      0.050
    8   5  0.128   0.896   0.838    -6.737    0.737       0.050      0.050
    9   6 -0.036  -0.225   0.303      -Inf      Inf       0.000      0.000
   10   3  0.037   0.255   0.121    -1.478      Inf       0.050      0.000

Estimated stopping point from ForwardStop rule = 2