hierNet: A Lasso for interactions
In hierNet: A Lasso for Hierarchical Interactions
Description
Usage
Arguments
Value
Author(s)
References
See Also
Examples
Description
One of the main functions in the hierNet package.
Builds a regression model with hierarchically constrained pairwise interactions.
Required inputs are an x matrix of
features (the columns are the features) and a y vector of
values.
Reasonably fast for moderate sized problems (100-200 variables). We are currently working
on an alternate algorithm for large scale problems.
Usage
1
2
3
4
Arguments
x
A matrix of predictors, where the rows are the samples and
the columns are the predictors
y
A vector of observations, where length(y) equals
nrow(x)
lam
Regularization parameter (>0). L1 penalty param is lam * (1-delta).
delta
Elastic Net parameter. Squared L2 penalty param is lam * delta. Not a tuning parameter: Think of as fixed and small. Default 1e-8.
strong
Flag specifying strong hierarchy (TRUE) or weak
hierarchy (FALSE). Default FALSE.
diagonal
Flag specifying whether to include "pure" quadratic
terms, th_jjX_j^2, in the model. Default TRUE.
aa
An *optional* argument, a list with results from a previous call
zz
An *optional* argument, a matrix whose columns are products of features, computed by the function compute.interactions.c
center
Should features be centered? Default TRUE; FALSE should rarely be used. This option is available for special uses only
stand.main
Should main effects be standardized? Default TRUE.
stand.int
Should interactions be standardized? Default FALSE.
rho
ADMM parameter: tuning parameter (>0) for ADMM. If there are convergence
problems, try decreasing rho. Default n.
niter
ADMM parameter: number of iterations
sym.eps
ADMM parameter: threshold for symmetrizing with strong=TRUE
step
Stepsize for generalized gradient descent
maxiter
Maximum number of iterations for generalized gradient descent
backtrack
Backtrack parameter for generalized gradient descent
tol
Error tolerance parameter for generalized gradient descent
trace
Output option; trace=1 gives verbose output
Value
bp
p-vector of estimated "positive part" main effect (p=# features)
bn
p-vector of estimated "negative part" main effect; overall main effect estimated coefficients are bp-bn
th
Matrix of estimated interaction coefficients, of dimension p by p. Note: when output from hierNet is printed, th is symmetrized (set to (th+t(th))/2)
for simplicity.
obj
Value of objective function at minimum.
lam
Value of lambda used
type
Type of model fit- "gaussian" or "logistic" (binomial)
mx
p-vector of column means of x
sx
p-vector of column standard deviations of x
my
mean of y
mzz
column means of feature product matrix
szz
column standard deviations of feature product matrix
call
The call to hierNet
Author(s)
Jacob Bien and Robert Tibshirani
References
Bien, J., Taylor, J., Tibshirani, R., (2013) "A Lasso for Hierarchical Interactions." Annals of Statistics. 41(3). 1111-1141.
See Also
predict.hierNet, hierNet.cv, hierNet.path
Examples
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
set.seed(12)
# fit a single hierNet model
x=matrix(rnorm(100*10),ncol=10)
x=scale(x,TRUE,TRUE)
y=x[,1]+2*x[,2]+ x[,1]*x[,2]+3*rnorm(100)
fit=hierNet(x,y,lam=50)
print(fit)
# try strong (rather than weak) hierarchy
fit=hierNet(x,y,lam=50, strong=TRUE)
print(fit)
# a typical analysis including cross-validation
set.seed(12)
x=matrix(rnorm(100*10),ncol=10)
x=scale(x,TRUE,TRUE)
y=x[,1]+2*x[,2]+ x[,1]*x[,2]+3*rnorm(100)
fit=hierNet.path(x,y)
fitcv=hierNet.cv(fit,x,y)
print(fitcv)
lamhat=fitcv$lamhat.1se
fit2=hierNet(x,y,lam=lamhat)
yhat=predict(fit2,x)
Example output
GG converged in 31 iterations.
Call:
hierNet(x = x, y = y, lam = 50)
Non-zero coefficients:
(Rows are predictors with nonzero main effects)
(1st column is main effect)
(Next columns are nonzero interactions of row predictor)
(Last column indicates whether hierarchy constraint is tight.)
Main effect 1 2 3 Tight?
1 0.2836 0 0.6513 0 *
2 1.019 0.6513 0 0
3 0.1289 0 0 -0.1289 *
7 0.2373 0 0 0
GG converged in 20 iterations.
Theta is not symmetric.
GG converged in 18 iterations.
Theta is not symmetric.
GG converged in 12 iterations.
Theta is not symmetric.
GG converged in 10 iterations.
Theta is not symmetric.
GG converged in 11 iterations.
Theta is not symmetric.
GG converged in 10 iterations.
Theta is not symmetric.
GG converged in 9 iterations.
Theta is not symmetric.
GG converged in 8 iterations.
Theta is not symmetric.
GG converged in 7 iterations.
GG converged in 5 iterations.
GG converged in 2 iterations.
GG converged in 2 iterations.
GG converged in 2 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
Call:
hierNet(x = x, y = y, lam = 50, strong = TRUE)
Non-zero coefficients:
(Rows are predictors with nonzero main effects)
(1st column is main effect)
(Next columns are nonzero interactions of row predictor)
(Last column indicates whether hierarchy constraint is tight.)
Main effect 1 2 3 Tight?
1 0.4974 0 0.4974 0
2 0.8839 0.4974 0 0
3 0.1218 0 0 -0.1218 *
7 0.2866 0 0 0
i,lam= 1 129.72
GG converged in 1 iterations.
i,lam= 2 101.8
GG converged in 20 iterations.
i,lam= 3 79.89
GG converged in 26 iterations.
i,lam= 4 62.69
GG converged in 26 iterations.
i,lam= 5 49.2
GG converged in 26 iterations.
i,lam= 6 38.61
GG converged in 48 iterations.
i,lam= 7 30.3
GG converged in 57 iterations.
i,lam= 8 23.78
GG converged in 62 iterations.
i,lam= 9 18.66
GG converged in 55 iterations.
i,lam= 10 14.64
GG converged in 63 iterations.
i,lam= 11 11.49
GG converged in 60 iterations.
i,lam= 12 9.02
GG converged in 119 iterations.
i,lam= 13 7.08
GG converged in 122 iterations.
i,lam= 14 5.55
GG converged in 139 iterations.
i,lam= 15 4.36
GG converged in 147 iterations.
i,lam= 16 3.42
GG converged in 200 iterations.
i,lam= 17 2.68
GG converged in 374 iterations.
i,lam= 18 2.11
GG converged in 291 iterations.
i,lam= 19 1.65
GG converged in 602 iterations.
i,lam= 20 1.3
GG converged in 657 iterations.
Fold 1 :i,lam= 1 129.72
GG converged in 1 iterations.
i,lam= 2 101.8
GG converged in 21 iterations.
i,lam= 3 79.89
GG converged in 25 iterations.
i,lam= 4 62.69
GG converged in 25 iterations.
i,lam= 5 49.2
GG converged in 50 iterations.
i,lam= 6 38.61
GG converged in 64 iterations.
i,lam= 7 30.3
GG converged in 95 iterations.
i,lam= 8 23.78
GG converged in 114 iterations.
i,lam= 9 18.66
GG converged in 144 iterations.
i,lam= 10 14.64
GG converged in 144 iterations.
i,lam= 11 11.49
GG converged in 79 iterations.
i,lam= 12 9.02
GG converged in 133 iterations.
i,lam= 13 7.08
GG converged in 162 iterations.
i,lam= 14 5.55
GG converged in 156 iterations.
i,lam= 15 4.36
GG converged in 151 iterations.
i,lam= 16 3.42
GG converged in 166 iterations.
i,lam= 17 2.68
GG converged in 245 iterations.
i,lam= 18 2.11
GG converged in 307 iterations.
i,lam= 19 1.65
GG converged in 485 iterations.
i,lam= 20 1.3
GG converged in 807 iterations.
Fold 2 :i,lam= 1 129.72
GG converged in 1 iterations.
i,lam= 2 101.8
GG converged in 22 iterations.
i,lam= 3 79.89
GG converged in 22 iterations.
i,lam= 4 62.69
GG converged in 27 iterations.
i,lam= 5 49.2
GG converged in 26 iterations.
i,lam= 6 38.61
GG converged in 48 iterations.
i,lam= 7 30.3
GG converged in 44 iterations.
i,lam= 8 23.78
GG converged in 55 iterations.
i,lam= 9 18.66
GG converged in 62 iterations.
i,lam= 10 14.64
GG converged in 51 iterations.
i,lam= 11 11.49
GG converged in 60 iterations.
i,lam= 12 9.02
GG converged in 86 iterations.
i,lam= 13 7.08
GG converged in 134 iterations.
i,lam= 14 5.55
GG converged in 122 iterations.
i,lam= 15 4.36
GG converged in 222 iterations.
i,lam= 16 3.42
GG converged in 402 iterations.
i,lam= 17 2.68
GG converged in 272 iterations.
i,lam= 18 2.11
GG converged in 345 iterations.
i,lam= 19 1.65
GG converged in 630 iterations.
i,lam= 20 1.3
GG converged in 700 iterations.
Fold 3 :i,lam= 1 129.72
GG converged in 7 iterations.
i,lam= 2 101.8
GG converged in 10 iterations.
i,lam= 3 79.89
GG converged in 16 iterations.
i,lam= 4 62.69
GG converged in 16 iterations.
i,lam= 5 49.2
GG converged in 26 iterations.
i,lam= 6 38.61
GG converged in 26 iterations.
i,lam= 7 30.3
GG converged in 33 iterations.
i,lam= 8 23.78
GG converged in 44 iterations.
i,lam= 9 18.66
GG converged in 46 iterations.
i,lam= 10 14.64
GG converged in 91 iterations.
i,lam= 11 11.49
GG converged in 103 iterations.
i,lam= 12 9.02
GG converged in 139 iterations.
i,lam= 13 7.08
GG converged in 181 iterations.
i,lam= 14 5.55
GG converged in 232 iterations.
i,lam= 15 4.36
GG converged in 211 iterations.
i,lam= 16 3.42
GG converged in 293 iterations.
i,lam= 17 2.68
GG converged in 335 iterations.
i,lam= 18 2.11
GG converged in 491 iterations.
i,lam= 19 1.65
GG converged in 667 iterations.
i,lam= 20 1.3
GG converged in 878 iterations.
Fold 4 :i,lam= 1 129.72
GG converged in 9 iterations.
i,lam= 2 101.8
GG converged in 10 iterations.
i,lam= 3 79.89
GG converged in 38 iterations.
i,lam= 4 62.69
GG converged in 41 iterations.
i,lam= 5 49.2
GG converged in 35 iterations.
i,lam= 6 38.61
GG converged in 37 iterations.
i,lam= 7 30.3
GG converged in 48 iterations.
i,lam= 8 23.78
GG converged in 66 iterations.
i,lam= 9 18.66
GG converged in 76 iterations.
i,lam= 10 14.64
GG converged in 67 iterations.
i,lam= 11 11.49
GG converged in 82 iterations.
i,lam= 12 9.02
GG converged in 90 iterations.
i,lam= 13 7.08
GG converged in 132 iterations.
i,lam= 14 5.55
GG converged in 169 iterations.
i,lam= 15 4.36
GG converged in 176 iterations.
i,lam= 16 3.42
GG converged in 168 iterations.
i,lam= 17 2.68
GG converged in 183 iterations.
i,lam= 18 2.11
GG converged in 229 iterations.
i,lam= 19 1.65
GG converged in 377 iterations.
i,lam= 20 1.3
GG converged in 573 iterations.
Fold 5 :i,lam= 1 129.72
GG converged in 1 iterations.
i,lam= 2 101.8
GG converged in 19 iterations.
i,lam= 3 79.89
GG converged in 23 iterations.
i,lam= 4 62.69
GG converged in 27 iterations.
i,lam= 5 49.2
GG converged in 26 iterations.
i,lam= 6 38.61
GG converged in 27 iterations.
i,lam= 7 30.3
GG converged in 60 iterations.
i,lam= 8 23.78
GG converged in 78 iterations.
i,lam= 9 18.66
GG converged in 45 iterations.
i,lam= 10 14.64
GG converged in 79 iterations.
i,lam= 11 11.49
GG converged in 64 iterations.
i,lam= 12 9.02
GG converged in 297 iterations.
i,lam= 13 7.08
GG converged in 124 iterations.
i,lam= 14 5.55
GG converged in 138 iterations.
i,lam= 15 4.36
GG converged in 277 iterations.
i,lam= 16 3.42
GG converged in 222 iterations.
i,lam= 17 2.68
GG converged in 298 iterations.
i,lam= 18 2.11
GG converged in 308 iterations.
i,lam= 19 1.65
GG converged in 392 iterations.
i,lam= 20 1.3
GG converged in 495 iterations.
Fold 6 :i,lam= 1 129.72
GG converged in 1 iterations.
i,lam= 2 101.8
GG converged in 21 iterations.
i,lam= 3 79.89
GG converged in 25 iterations.
i,lam= 4 62.69
GG converged in 25 iterations.
i,lam= 5 49.2
GG converged in 24 iterations.
i,lam= 6 38.61
GG converged in 47 iterations.
i,lam= 7 30.3
GG converged in 34 iterations.
i,lam= 8 23.78
GG converged in 49 iterations.
i,lam= 9 18.66
GG converged in 59 iterations.
i,lam= 10 14.64
GG converged in 186 iterations.
i,lam= 11 11.49
GG converged in 193 iterations.
i,lam= 12 9.02
GG converged in 131 iterations.
i,lam= 13 7.08
GG converged in 151 iterations.
i,lam= 14 5.55
GG converged in 210 iterations.
i,lam= 15 4.36
GG converged in 150 iterations.
i,lam= 16 3.42
GG converged in 200 iterations.
i,lam= 17 2.68
GG converged in 274 iterations.
i,lam= 18 2.11
GG converged in 446 iterations.
i,lam= 19 1.65
GG converged in 528 iterations.
i,lam= 20 1.3
GG converged in 681 iterations.
Fold 7 :i,lam= 1 129.72
GG converged in 1 iterations.
i,lam= 2 101.8
GG converged in 20 iterations.
i,lam= 3 79.89
GG converged in 24 iterations.
i,lam= 4 62.69
GG converged in 29 iterations.
i,lam= 5 49.2
GG converged in 28 iterations.
i,lam= 6 38.61
GG converged in 30 iterations.
i,lam= 7 30.3
GG converged in 29 iterations.
i,lam= 8 23.78
GG converged in 89 iterations.
i,lam= 9 18.66
GG converged in 78 iterations.
i,lam= 10 14.64
GG converged in 131 iterations.
i,lam= 11 11.49
GG converged in 82 iterations.
i,lam= 12 9.02
GG converged in 187 iterations.
i,lam= 13 7.08
GG converged in 258 iterations.
i,lam= 14 5.55
GG converged in 142 iterations.
i,lam= 15 4.36
GG converged in 323 iterations.
i,lam= 16 3.42
GG converged in 387 iterations.
i,lam= 17 2.68
GG converged in 462 iterations.
i,lam= 18 2.11
GG converged in 490 iterations.
i,lam= 19 1.65
GG converged in 655 iterations.
i,lam= 20 1.3
GG converged in 626 iterations.
Fold 8 :i,lam= 1 129.72
GG converged in 1 iterations.
i,lam= 2 101.8
GG converged in 17 iterations.
i,lam= 3 79.89
GG converged in 23 iterations.
i,lam= 4 62.69
GG converged in 23 iterations.
i,lam= 5 49.2
GG converged in 25 iterations.
i,lam= 6 38.61
GG converged in 28 iterations.
i,lam= 7 30.3
GG converged in 51 iterations.
i,lam= 8 23.78
GG converged in 28 iterations.
i,lam= 9 18.66
GG converged in 69 iterations.
i,lam= 10 14.64
GG converged in 78 iterations.
i,lam= 11 11.49
GG converged in 77 iterations.
i,lam= 12 9.02
GG converged in 157 iterations.
i,lam= 13 7.08
GG converged in 124 iterations.
i,lam= 14 5.55
GG converged in 127 iterations.
i,lam= 15 4.36
GG converged in 175 iterations.
i,lam= 16 3.42
GG converged in 258 iterations.
i,lam= 17 2.68
GG converged in 263 iterations.
i,lam= 18 2.11
GG converged in 292 iterations.
i,lam= 19 1.65
GG converged in 347 iterations.
i,lam= 20 1.3
GG converged in 419 iterations.
Fold 9 :i,lam= 1 129.72
GG converged in 1 iterations.
i,lam= 2 101.8
GG converged in 17 iterations.
i,lam= 3 79.89
GG converged in 22 iterations.
i,lam= 4 62.69
GG converged in 23 iterations.
i,lam= 5 49.2
GG converged in 23 iterations.
i,lam= 6 38.61
GG converged in 57 iterations.
i,lam= 7 30.3
GG converged in 60 iterations.
i,lam= 8 23.78
GG converged in 25 iterations.
i,lam= 9 18.66
GG converged in 55 iterations.
i,lam= 10 14.64
GG converged in 65 iterations.
i,lam= 11 11.49
GG converged in 114 iterations.
i,lam= 12 9.02
GG converged in 162 iterations.
i,lam= 13 7.08
GG converged in 164 iterations.
i,lam= 14 5.55
GG converged in 198 iterations.
i,lam= 15 4.36
GG converged in 212 iterations.
i,lam= 16 3.42
GG converged in 195 iterations.
i,lam= 17 2.68
GG converged in 239 iterations.
i,lam= 18 2.11
GG converged in 417 iterations.
i,lam= 19 1.65
GG converged in 610 iterations.
i,lam= 20 1.3
GG converged in 903 iterations.
Fold 10 :i,lam= 1 129.72
GG converged in 1 iterations.
i,lam= 2 101.8
GG converged in 23 iterations.
i,lam= 3 79.89
GG converged in 27 iterations.
i,lam= 4 62.69
GG converged in 29 iterations.
i,lam= 5 49.2
GG converged in 43 iterations.
i,lam= 6 38.61
GG converged in 60 iterations.
i,lam= 7 30.3
GG converged in 56 iterations.
i,lam= 8 23.78
GG converged in 45 iterations.
i,lam= 9 18.66
GG converged in 82 iterations.
i,lam= 10 14.64
GG converged in 86 iterations.
i,lam= 11 11.49
GG converged in 83 iterations.
i,lam= 12 9.02
GG converged in 105 iterations.
i,lam= 13 7.08
GG converged in 120 iterations.
i,lam= 14 5.55
GG converged in 123 iterations.
i,lam= 15 4.36
GG converged in 201 iterations.
i,lam= 16 3.42
GG converged in 211 iterations.
i,lam= 17 2.68
GG converged in 323 iterations.
i,lam= 18 2.11
GG converged in 412 iterations.
i,lam= 19 1.65
GG converged in 506 iterations.
i,lam= 20 1.3
GG converged in 572 iterations.
Call:
hierNet.cv(fit = fit, x = x, y = y)
Lambda Number of nonzero Mean CV error SE
[1,] 129.72 1 12.49 1.27
[2,] 101.80 2 12.14 1.22
[3,] 79.89 6 10.96 1.04
[4,] 62.69 6 10.11 0.89
[5,] 49.20 6 9.51 0.78
[6,] 38.61 13 9.37 0.70
[7,] 30.30 15 9.48 0.70
[8,] 23.78 19 9.62 0.74
[9,] 18.66 19 9.75 0.79
[10,] 14.64 21 9.65 0.77
[11,] 11.49 23 9.77 0.82
[12,] 9.02 30 10.08 0.89
[13,] 7.08 33 10.40 1.01
[14,] 5.55 34 10.97 1.20
[15,] 4.36 36 12.18 1.55
[16,] 3.42 40 13.86 2.07
[17,] 2.68 43 16.00 2.71
[18,] 2.11 44 18.58 3.46
[19,] 1.65 52 21.57 4.50
[20,] 1.30 55 24.70 5.56
lamhat= 38.61 lamhat.1se= 49.2
GG converged in 31 iterations.
hierNet documentation built on Feb. 6, 2020, 5:09 p.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Description Usage Arguments Value Author(s) References See Also Examples
Description
One of the main functions in the hierNet package. Builds a regression model with hierarchically constrained pairwise interactions. Required inputs are an x matrix of features (the columns are the features) and a y vector of values. Reasonably fast for moderate sized problems (100-200 variables). We are currently working on an alternate algorithm for large scale problems.
Usage
1 2 3 4 |
Arguments
x |
A matrix of predictors, where the rows are the samples and the columns are the predictors |
y |
A vector of observations, where length(y) equals nrow(x) |
lam |
Regularization parameter (>0). L1 penalty param is |
delta |
Elastic Net parameter. Squared L2 penalty param is |
strong |
Flag specifying strong hierarchy (TRUE) or weak hierarchy (FALSE). Default FALSE. |
diagonal |
Flag specifying whether to include "pure" quadratic terms, th_jjX_j^2, in the model. Default TRUE. |
aa |
An *optional* argument, a list with results from a previous call |
zz |
An *optional* argument, a matrix whose columns are products of features, computed by the function compute.interactions.c |
center |
Should features be centered? Default TRUE; FALSE should rarely be used. This option is available for special uses only |
stand.main |
Should main effects be standardized? Default TRUE. |
stand.int |
Should interactions be standardized? Default FALSE. |
rho |
ADMM parameter: tuning parameter (>0) for ADMM. If there are convergence
problems, try decreasing |
niter |
ADMM parameter: number of iterations |
sym.eps |
ADMM parameter: threshold for symmetrizing with strong=TRUE |
step |
Stepsize for generalized gradient descent |
maxiter |
Maximum number of iterations for generalized gradient descent |
backtrack |
Backtrack parameter for generalized gradient descent |
tol |
Error tolerance parameter for generalized gradient descent |
trace |
Output option; trace=1 gives verbose output |
Value
bp |
p-vector of estimated "positive part" main effect (p=# features) |
bn |
p-vector of estimated "negative part" main effect; overall main effect estimated coefficients are bp-bn |
th |
Matrix of estimated interaction coefficients, of dimension p by p. Note: when output from hierNet is printed, th is symmetrized (set to (th+t(th))/2) for simplicity. |
obj |
Value of objective function at minimum. |
lam |
Value of lambda used |
type |
Type of model fit- "gaussian" or "logistic" (binomial) |
mx |
p-vector of column means of x |
sx |
p-vector of column standard deviations of x |
my |
mean of y |
mzz |
column means of feature product matrix |
szz |
column standard deviations of feature product matrix |
call |
The call to hierNet |
Author(s)
Jacob Bien and Robert Tibshirani
References
Bien, J., Taylor, J., Tibshirani, R., (2013) "A Lasso for Hierarchical Interactions." Annals of Statistics. 41(3). 1111-1141.
See Also
predict.hierNet, hierNet.cv, hierNet.path
Examples
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | set.seed(12)
# fit a single hierNet model
x=matrix(rnorm(100*10),ncol=10)
x=scale(x,TRUE,TRUE)
y=x[,1]+2*x[,2]+ x[,1]*x[,2]+3*rnorm(100)
fit=hierNet(x,y,lam=50)
print(fit)
# try strong (rather than weak) hierarchy
fit=hierNet(x,y,lam=50, strong=TRUE)
print(fit)
# a typical analysis including cross-validation
set.seed(12)
x=matrix(rnorm(100*10),ncol=10)
x=scale(x,TRUE,TRUE)
y=x[,1]+2*x[,2]+ x[,1]*x[,2]+3*rnorm(100)
fit=hierNet.path(x,y)
fitcv=hierNet.cv(fit,x,y)
print(fitcv)
lamhat=fitcv$lamhat.1se
fit2=hierNet(x,y,lam=lamhat)
yhat=predict(fit2,x)
|
Example output
GG converged in 31 iterations.
Call:
hierNet(x = x, y = y, lam = 50)
Non-zero coefficients:
(Rows are predictors with nonzero main effects)
(1st column is main effect)
(Next columns are nonzero interactions of row predictor)
(Last column indicates whether hierarchy constraint is tight.)
Main effect 1 2 3 Tight?
1 0.2836 0 0.6513 0 *
2 1.019 0.6513 0 0
3 0.1289 0 0 -0.1289 *
7 0.2373 0 0 0
GG converged in 20 iterations.
Theta is not symmetric.
GG converged in 18 iterations.
Theta is not symmetric.
GG converged in 12 iterations.
Theta is not symmetric.
GG converged in 10 iterations.
Theta is not symmetric.
GG converged in 11 iterations.
Theta is not symmetric.
GG converged in 10 iterations.
Theta is not symmetric.
GG converged in 9 iterations.
Theta is not symmetric.
GG converged in 8 iterations.
Theta is not symmetric.
GG converged in 7 iterations.
GG converged in 5 iterations.
GG converged in 2 iterations.
GG converged in 2 iterations.
GG converged in 2 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
GG converged in 1 iterations.
Call:
hierNet(x = x, y = y, lam = 50, strong = TRUE)
Non-zero coefficients:
(Rows are predictors with nonzero main effects)
(1st column is main effect)
(Next columns are nonzero interactions of row predictor)
(Last column indicates whether hierarchy constraint is tight.)
Main effect 1 2 3 Tight?
1 0.4974 0 0.4974 0
2 0.8839 0.4974 0 0
3 0.1218 0 0 -0.1218 *
7 0.2866 0 0 0
i,lam= 1 129.72
GG converged in 1 iterations.
i,lam= 2 101.8
GG converged in 20 iterations.
i,lam= 3 79.89
GG converged in 26 iterations.
i,lam= 4 62.69
GG converged in 26 iterations.
i,lam= 5 49.2
GG converged in 26 iterations.
i,lam= 6 38.61
GG converged in 48 iterations.
i,lam= 7 30.3
GG converged in 57 iterations.
i,lam= 8 23.78
GG converged in 62 iterations.
i,lam= 9 18.66
GG converged in 55 iterations.
i,lam= 10 14.64
GG converged in 63 iterations.
i,lam= 11 11.49
GG converged in 60 iterations.
i,lam= 12 9.02
GG converged in 119 iterations.
i,lam= 13 7.08
GG converged in 122 iterations.
i,lam= 14 5.55
GG converged in 139 iterations.
i,lam= 15 4.36
GG converged in 147 iterations.
i,lam= 16 3.42
GG converged in 200 iterations.
i,lam= 17 2.68
GG converged in 374 iterations.
i,lam= 18 2.11
GG converged in 291 iterations.
i,lam= 19 1.65
GG converged in 602 iterations.
i,lam= 20 1.3
GG converged in 657 iterations.
Fold 1 :i,lam= 1 129.72
GG converged in 1 iterations.
i,lam= 2 101.8
GG converged in 21 iterations.
i,lam= 3 79.89
GG converged in 25 iterations.
i,lam= 4 62.69
GG converged in 25 iterations.
i,lam= 5 49.2
GG converged in 50 iterations.
i,lam= 6 38.61
GG converged in 64 iterations.
i,lam= 7 30.3
GG converged in 95 iterations.
i,lam= 8 23.78
GG converged in 114 iterations.
i,lam= 9 18.66
GG converged in 144 iterations.
i,lam= 10 14.64
GG converged in 144 iterations.
i,lam= 11 11.49
GG converged in 79 iterations.
i,lam= 12 9.02
GG converged in 133 iterations.
i,lam= 13 7.08
GG converged in 162 iterations.
i,lam= 14 5.55
GG converged in 156 iterations.
i,lam= 15 4.36
GG converged in 151 iterations.
i,lam= 16 3.42
GG converged in 166 iterations.
i,lam= 17 2.68
GG converged in 245 iterations.
i,lam= 18 2.11
GG converged in 307 iterations.
i,lam= 19 1.65
GG converged in 485 iterations.
i,lam= 20 1.3
GG converged in 807 iterations.
Fold 2 :i,lam= 1 129.72
GG converged in 1 iterations.
i,lam= 2 101.8
GG converged in 22 iterations.
i,lam= 3 79.89
GG converged in 22 iterations.
i,lam= 4 62.69
GG converged in 27 iterations.
i,lam= 5 49.2
GG converged in 26 iterations.
i,lam= 6 38.61
GG converged in 48 iterations.
i,lam= 7 30.3
GG converged in 44 iterations.
i,lam= 8 23.78
GG converged in 55 iterations.
i,lam= 9 18.66
GG converged in 62 iterations.
i,lam= 10 14.64
GG converged in 51 iterations.
i,lam= 11 11.49
GG converged in 60 iterations.
i,lam= 12 9.02
GG converged in 86 iterations.
i,lam= 13 7.08
GG converged in 134 iterations.
i,lam= 14 5.55
GG converged in 122 iterations.
i,lam= 15 4.36
GG converged in 222 iterations.
i,lam= 16 3.42
GG converged in 402 iterations.
i,lam= 17 2.68
GG converged in 272 iterations.
i,lam= 18 2.11
GG converged in 345 iterations.
i,lam= 19 1.65
GG converged in 630 iterations.
i,lam= 20 1.3
GG converged in 700 iterations.
Fold 3 :i,lam= 1 129.72
GG converged in 7 iterations.
i,lam= 2 101.8
GG converged in 10 iterations.
i,lam= 3 79.89
GG converged in 16 iterations.
i,lam= 4 62.69
GG converged in 16 iterations.
i,lam= 5 49.2
GG converged in 26 iterations.
i,lam= 6 38.61
GG converged in 26 iterations.
i,lam= 7 30.3
GG converged in 33 iterations.
i,lam= 8 23.78
GG converged in 44 iterations.
i,lam= 9 18.66
GG converged in 46 iterations.
i,lam= 10 14.64
GG converged in 91 iterations.
i,lam= 11 11.49
GG converged in 103 iterations.
i,lam= 12 9.02
GG converged in 139 iterations.
i,lam= 13 7.08
GG converged in 181 iterations.
i,lam= 14 5.55
GG converged in 232 iterations.
i,lam= 15 4.36
GG converged in 211 iterations.
i,lam= 16 3.42
GG converged in 293 iterations.
i,lam= 17 2.68
GG converged in 335 iterations.
i,lam= 18 2.11
GG converged in 491 iterations.
i,lam= 19 1.65
GG converged in 667 iterations.
i,lam= 20 1.3
GG converged in 878 iterations.
Fold 4 :i,lam= 1 129.72
GG converged in 9 iterations.
i,lam= 2 101.8
GG converged in 10 iterations.
i,lam= 3 79.89
GG converged in 38 iterations.
i,lam= 4 62.69
GG converged in 41 iterations.
i,lam= 5 49.2
GG converged in 35 iterations.
i,lam= 6 38.61
GG converged in 37 iterations.
i,lam= 7 30.3
GG converged in 48 iterations.
i,lam= 8 23.78
GG converged in 66 iterations.
i,lam= 9 18.66
GG converged in 76 iterations.
i,lam= 10 14.64
GG converged in 67 iterations.
i,lam= 11 11.49
GG converged in 82 iterations.
i,lam= 12 9.02
GG converged in 90 iterations.
i,lam= 13 7.08
GG converged in 132 iterations.
i,lam= 14 5.55
GG converged in 169 iterations.
i,lam= 15 4.36
GG converged in 176 iterations.
i,lam= 16 3.42
GG converged in 168 iterations.
i,lam= 17 2.68
GG converged in 183 iterations.
i,lam= 18 2.11
GG converged in 229 iterations.
i,lam= 19 1.65
GG converged in 377 iterations.
i,lam= 20 1.3
GG converged in 573 iterations.
Fold 5 :i,lam= 1 129.72
GG converged in 1 iterations.
i,lam= 2 101.8
GG converged in 19 iterations.
i,lam= 3 79.89
GG converged in 23 iterations.
i,lam= 4 62.69
GG converged in 27 iterations.
i,lam= 5 49.2
GG converged in 26 iterations.
i,lam= 6 38.61
GG converged in 27 iterations.
i,lam= 7 30.3
GG converged in 60 iterations.
i,lam= 8 23.78
GG converged in 78 iterations.
i,lam= 9 18.66
GG converged in 45 iterations.
i,lam= 10 14.64
GG converged in 79 iterations.
i,lam= 11 11.49
GG converged in 64 iterations.
i,lam= 12 9.02
GG converged in 297 iterations.
i,lam= 13 7.08
GG converged in 124 iterations.
i,lam= 14 5.55
GG converged in 138 iterations.
i,lam= 15 4.36
GG converged in 277 iterations.
i,lam= 16 3.42
GG converged in 222 iterations.
i,lam= 17 2.68
GG converged in 298 iterations.
i,lam= 18 2.11
GG converged in 308 iterations.
i,lam= 19 1.65
GG converged in 392 iterations.
i,lam= 20 1.3
GG converged in 495 iterations.
Fold 6 :i,lam= 1 129.72
GG converged in 1 iterations.
i,lam= 2 101.8
GG converged in 21 iterations.
i,lam= 3 79.89
GG converged in 25 iterations.
i,lam= 4 62.69
GG converged in 25 iterations.
i,lam= 5 49.2
GG converged in 24 iterations.
i,lam= 6 38.61
GG converged in 47 iterations.
i,lam= 7 30.3
GG converged in 34 iterations.
i,lam= 8 23.78
GG converged in 49 iterations.
i,lam= 9 18.66
GG converged in 59 iterations.
i,lam= 10 14.64
GG converged in 186 iterations.
i,lam= 11 11.49
GG converged in 193 iterations.
i,lam= 12 9.02
GG converged in 131 iterations.
i,lam= 13 7.08
GG converged in 151 iterations.
i,lam= 14 5.55
GG converged in 210 iterations.
i,lam= 15 4.36
GG converged in 150 iterations.
i,lam= 16 3.42
GG converged in 200 iterations.
i,lam= 17 2.68
GG converged in 274 iterations.
i,lam= 18 2.11
GG converged in 446 iterations.
i,lam= 19 1.65
GG converged in 528 iterations.
i,lam= 20 1.3
GG converged in 681 iterations.
Fold 7 :i,lam= 1 129.72
GG converged in 1 iterations.
i,lam= 2 101.8
GG converged in 20 iterations.
i,lam= 3 79.89
GG converged in 24 iterations.
i,lam= 4 62.69
GG converged in 29 iterations.
i,lam= 5 49.2
GG converged in 28 iterations.
i,lam= 6 38.61
GG converged in 30 iterations.
i,lam= 7 30.3
GG converged in 29 iterations.
i,lam= 8 23.78
GG converged in 89 iterations.
i,lam= 9 18.66
GG converged in 78 iterations.
i,lam= 10 14.64
GG converged in 131 iterations.
i,lam= 11 11.49
GG converged in 82 iterations.
i,lam= 12 9.02
GG converged in 187 iterations.
i,lam= 13 7.08
GG converged in 258 iterations.
i,lam= 14 5.55
GG converged in 142 iterations.
i,lam= 15 4.36
GG converged in 323 iterations.
i,lam= 16 3.42
GG converged in 387 iterations.
i,lam= 17 2.68
GG converged in 462 iterations.
i,lam= 18 2.11
GG converged in 490 iterations.
i,lam= 19 1.65
GG converged in 655 iterations.
i,lam= 20 1.3
GG converged in 626 iterations.
Fold 8 :i,lam= 1 129.72
GG converged in 1 iterations.
i,lam= 2 101.8
GG converged in 17 iterations.
i,lam= 3 79.89
GG converged in 23 iterations.
i,lam= 4 62.69
GG converged in 23 iterations.
i,lam= 5 49.2
GG converged in 25 iterations.
i,lam= 6 38.61
GG converged in 28 iterations.
i,lam= 7 30.3
GG converged in 51 iterations.
i,lam= 8 23.78
GG converged in 28 iterations.
i,lam= 9 18.66
GG converged in 69 iterations.
i,lam= 10 14.64
GG converged in 78 iterations.
i,lam= 11 11.49
GG converged in 77 iterations.
i,lam= 12 9.02
GG converged in 157 iterations.
i,lam= 13 7.08
GG converged in 124 iterations.
i,lam= 14 5.55
GG converged in 127 iterations.
i,lam= 15 4.36
GG converged in 175 iterations.
i,lam= 16 3.42
GG converged in 258 iterations.
i,lam= 17 2.68
GG converged in 263 iterations.
i,lam= 18 2.11
GG converged in 292 iterations.
i,lam= 19 1.65
GG converged in 347 iterations.
i,lam= 20 1.3
GG converged in 419 iterations.
Fold 9 :i,lam= 1 129.72
GG converged in 1 iterations.
i,lam= 2 101.8
GG converged in 17 iterations.
i,lam= 3 79.89
GG converged in 22 iterations.
i,lam= 4 62.69
GG converged in 23 iterations.
i,lam= 5 49.2
GG converged in 23 iterations.
i,lam= 6 38.61
GG converged in 57 iterations.
i,lam= 7 30.3
GG converged in 60 iterations.
i,lam= 8 23.78
GG converged in 25 iterations.
i,lam= 9 18.66
GG converged in 55 iterations.
i,lam= 10 14.64
GG converged in 65 iterations.
i,lam= 11 11.49
GG converged in 114 iterations.
i,lam= 12 9.02
GG converged in 162 iterations.
i,lam= 13 7.08
GG converged in 164 iterations.
i,lam= 14 5.55
GG converged in 198 iterations.
i,lam= 15 4.36
GG converged in 212 iterations.
i,lam= 16 3.42
GG converged in 195 iterations.
i,lam= 17 2.68
GG converged in 239 iterations.
i,lam= 18 2.11
GG converged in 417 iterations.
i,lam= 19 1.65
GG converged in 610 iterations.
i,lam= 20 1.3
GG converged in 903 iterations.
Fold 10 :i,lam= 1 129.72
GG converged in 1 iterations.
i,lam= 2 101.8
GG converged in 23 iterations.
i,lam= 3 79.89
GG converged in 27 iterations.
i,lam= 4 62.69
GG converged in 29 iterations.
i,lam= 5 49.2
GG converged in 43 iterations.
i,lam= 6 38.61
GG converged in 60 iterations.
i,lam= 7 30.3
GG converged in 56 iterations.
i,lam= 8 23.78
GG converged in 45 iterations.
i,lam= 9 18.66
GG converged in 82 iterations.
i,lam= 10 14.64
GG converged in 86 iterations.
i,lam= 11 11.49
GG converged in 83 iterations.
i,lam= 12 9.02
GG converged in 105 iterations.
i,lam= 13 7.08
GG converged in 120 iterations.
i,lam= 14 5.55
GG converged in 123 iterations.
i,lam= 15 4.36
GG converged in 201 iterations.
i,lam= 16 3.42
GG converged in 211 iterations.
i,lam= 17 2.68
GG converged in 323 iterations.
i,lam= 18 2.11
GG converged in 412 iterations.
i,lam= 19 1.65
GG converged in 506 iterations.
i,lam= 20 1.3
GG converged in 572 iterations.
Call:
hierNet.cv(fit = fit, x = x, y = y)
Lambda Number of nonzero Mean CV error SE
[1,] 129.72 1 12.49 1.27
[2,] 101.80 2 12.14 1.22
[3,] 79.89 6 10.96 1.04
[4,] 62.69 6 10.11 0.89
[5,] 49.20 6 9.51 0.78
[6,] 38.61 13 9.37 0.70
[7,] 30.30 15 9.48 0.70
[8,] 23.78 19 9.62 0.74
[9,] 18.66 19 9.75 0.79
[10,] 14.64 21 9.65 0.77
[11,] 11.49 23 9.77 0.82
[12,] 9.02 30 10.08 0.89
[13,] 7.08 33 10.40 1.01
[14,] 5.55 34 10.97 1.20
[15,] 4.36 36 12.18 1.55
[16,] 3.42 40 13.86 2.07
[17,] 2.68 43 16.00 2.71
[18,] 2.11 44 18.58 3.46
[19,] 1.65 52 21.57 4.50
[20,] 1.30 55 24.70 5.56
lamhat= 38.61 lamhat.1se= 49.2
GG converged in 31 iterations.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.