powCD: Coordinate descent for penalized regression with power...

In maryclare/powopt: Solving Penalized Regression Problems with Power Penalty by Coordinate Descent

Coordinate descent for penalized regression with power penalty

Description

Gives the value of of the length p vector β that minimizes:

||y - Xβ||^2_2/(2σ^2) + λ ||β||^q_q

for fixed y, X, σ^2 > 0, λ > 0 and q > 0.

This corresponds to finding the posterior mode for beta given X, y, σ^2, λ and q under the model:

y = Xβ + σ Z

p(β_i) = qλ^(1/q) exp(-λ|β_i|^q)/(2 Γ(1/q)),

where elements of Z are independent, standard normal random variates.

This distribution for elements of β is a generalized normal distribution for β with scale α = λ^{-1/q} and shape β = q (Box and Tiao, 1973).

For q ≤ 1, uses coordinate descent algorithm given in Marjanovic and Solo (2014), modified to accomodate X that do not have standardized columns.

Usage

powCD(X = X, y = y, sigma.sq = 1, lambda = 1,
      q = 1, max.iter = 10000,
      print.iter = FALSE,
      tol = 10^(-7), ridge.eps = 0,
      rand.restart = 0)

Arguments

X	design matrix
y	response vector
sigma.sq	scalar value σ^2
lambda	scalar value λ
q	scalar value q
max.iter	maximum number of iterations for coordinate descent
print.iter	logical value indicating whether iteration count for coordinate descent should be printed
tol	scalar tolerance value for assessing convergence of objective function
ridge.eps	ridge regression tuning parameter for obtaining starting value of β in coordinate descent, defult is zero
rand.restart	number of times coordinate descent should be repeated from random starting value for β after an initial application of coordinate descent starting from ridge solution, needed when X is not orthogonal because the coordinate descent algorithm is not guaranteed to converge to the global optimum for all non-orthogonal X when q ≤ 1
start	starting value, set to null by default
return.obj.iter	TRUE/FALSE value indicating whether or not objectives at convergence and # of coordinate descent iterations should be returned
order	Vector indicating order of variables in coordinate descent, defaults to 1:p

Value

Returns a vector of optimal values for β. If the coordinate descent algorithm does not meet the optimality conditions given in Marjanovic and Solo (2014), a vector of NA's is returned.

Note that a non-NA solution for β for any value of q guarantees the global minimum has been attained when X is full rank and orthogonal. Otherwise, when q ≤ 1 the solution will only correspond to a global minimum when the conditions on X given in Marjanovic and Solo (2014) are satisfied.

Source

For q≤ 1, uses coordinate descent algorithm given by Marjanovic and Solo (2014), modified to accomodate X that do not have standardized columns.

Box, G. E. P., and G. C. Tiao. "Bayesian inference in statistical inference." Adison-Wesley, Reading, Mass (1973).

Marjanovic, Goran, and Victor Solo. "l_q Sparsity Penalized Linear Regression With Cyclic Descent." IEEE Transactions on Signal Processing 62.6 (2014): 1464-1475.

maryclare/powopt documentation built on Feb. 5, 2023, 6:05 p.m.