In Xia-Zhang/MixedGraphs: Mixed Graphical Models

Ridge regression

The MixedGraphs package support three different types of data, "gaussion", "logistic" and "poisson".

Gaussian

\begin{align} f(\mathbf{\beta}) &= \frac{1}{2n} {\| \mathbf{y} - \mathbf{o} - \mathbf{X} \mathbf{\beta} \|}_2^2+ \frac{1}{2}\epsilon\| \mathbf{\beta} \|_2^2 \ \mathbf{\beta}^{(k)} &= (\mathbf{X}^T \mathbf{X} / n + \epsilon\mathbf{I})^{-1} \mathbf{X}^T(\mathbf{y} - \mathbf{o})/n \end{align}

Logistic

\begin{align} f(\mathbf{\beta}) &= \frac{1}{n} \sum_{i=1}^{n} [\log (1 + e ^ {o_i + \mathbf{x_i} \mathbf{\beta}}) - y_i(o_i + \mathbf{x_i} \mathbf{\beta})] + \frac{1}{2}\epsilon\| \mathbf{\beta} \|2^2 \ \nabla\beta(f) &= \big(\frac{1}{n} \sum_{i = 1}^{n} \frac{x_{ij}}{1 + e ^ {- ({o_i + \mathbf{x_i}\mathbf{\beta}})}} - y_{i} x_{ij} \big)j + \epsilon \mathbf{\beta} \ \mathcal{H}\beta(f) &= \big(\frac{1}{n} \sum_{i = 1}^{n} \frac{x_{ij}x_{jk}e ^ {- ({o_i + \mathbf{x_i}\mathbf{\beta}})} }{{(1 + e ^ {- ({o_i + \mathbf{x_i}\mathbf{\beta}})})}^2}\big){jk} + \epsilon\mathbf{I}{p\times p} \end{align}

Poisson

\begin{align} f(\mathbf{\beta}) &= \frac{1}{n} \sum_{i=1}^{n} [- y_i(o_i + \mathbf{x_i} \mathbf{\beta}) + e ^ {o_i + \mathbf{x_i} \mathbf{\beta}} ] + \frac{1}{2}\epsilon\| \mathbf{\beta} \|2^2 \ \nabla\beta(f) &= \big( \frac{1}{n} \sum_{i=1}^{n} - y_ix_{ij} + x_{ij} e ^ {o_i + \mathbf{x_i} \mathbf{\beta}}\big)j + \epsilon \mathbf{\beta}\ \mathcal{H}\beta(f) &= \frac{1}{n}(x_{ij}x_{jk}e^{o_i + \mathbf{x_i}\mathbf{\beta}}){jk} + \epsilon\mathbf{I}{p\times p} \end{align}

Compare with glmnet

```{R message = F} library("glmnet") library("MixedGraphs") n <- 200; p <- 500 s <- 6; ## True sparsity

set.seed(8) X <- matrix(rnorm(n * p), ncol=p) beta <- rep(0, p); beta[1:s] <- runif(s, 4, 6) * c(-1, 1) y <- X %*% beta + rnorm(n) y <- y / sd(y)

glmnet_fit <- glmnet(X , y, alpha = 0, standardize=FALSE) glmRidge_fit <- glmRidge(X, y, lambda = glmnet_fit$lambda, thresh = 1)

```{R echo = FALSE}
plot(log(glmnet_fit$lambda), glmnet_fit$lambda,
     xlim=log(rev(range(glmnet_fit$lambda))),
     ylim=range(coef(glmnet_fit)),
     type="n", xlab=expression(log(lambda)),
     ylab=expression(hat(beta[j]^{lambda})),
     main="glmnet Paths")
for(i in 1:p){
      lines(log(glmnet_fit$lambda), coef(glmnet_fit)[i+1, ],
            col=ifelse(beta[i] == 0, "grey80", "red4"))
}

plot(log(glmnet_fit$lambda), glmnet_fit$lambda,
     xlim=log(rev(range(glmnet_fit$lambda))),
     ylim=range(coef(glmnet_fit)),
     type="n", xlab=expression(log(lambda)),
     ylab=expression(hat(beta[j]^{lambda})),
     main="MixedGraphs Paths")
for(i in 1:p){
      lines(log(glmnet_fit$lambda), glmRidge_fit[i+1, ],
            col=ifelse(beta[i] == 0, "grey80", "red4"))
}

Xia-Zhang/MixedGraphs documentation built on May 6, 2019, 3:29 p.m.