In fchamroukhi/HDME: LASSO Regularized Mixture of Experts Models

library(knitr)
knitr::opts_chunk$set(
    fig.align = "center",
    fig.height = 5.5,
    fig.width = 6,
    warning = FALSE,
    collapse = TRUE,
    dev.args = list(pointsize = 10),
    out.width = "90%",
    par = TRUE
)
knit_hooks$set(par = function(before, options, envir)
  { if (before && options$fig.show != "none") 
       par(family = "sans", mar = c(4.1,4.1,1.1,1.1), mgp = c(3,1,0), tcl = -0.5)
})

library(RMoE)

Introduction

GRMoE (Gaussian Regularized Mixture-of-Experts) provides a penalized MLE for the regularized Normal Mixture of Experts. GRMoE consists of a mixture of K Normal expert regressors network) gated by a softmax gating network.

It was written in R Markdown, using the knitr package for production.

See help(package="RMoE") for further details.

Application to a simulated dataset

Load data

data("gaussian")
X <- as.matrix(gaussian[, -8])
y <- gaussian$V8

Set up GRMoE model parameters

K <- 2 # Number of experts
Lambda <- 5
Gamma <- 5
opt <- FALSE # opt = FALSE: proximal Newton; opt = TRUE: proximal Newton-type

Estimation

grmoe <- GaussRMoE(Xm = X, Ym = y, K = K, Lambda = Lambda, 
                   Gamma = Gamma, option = opt, verbose = TRUE)

Parameters

You can access to the parameters via the following commands:

# Regression coefficients for each expert
grmoe$betak

# Parameters of the gating network
grmoe$wk

Plot

Log-likelihood

grmoe$plot(what = "loglik")

Proportion of zero coefficients

grmoe$plot(what = "zerocoefficients")

Application to a real dataset

Load data

data("housing")
X <- as.matrix(housing[, -15])
y <- housing$V15

Set up GRMoE model parameters

K <- 2 # Number of experts
Lambda <- 42
Gamma <- 10
opt <- FALSE # opt = FALSE: proximal Newton; opt = TRUE: proximal Newton-type

Estimation

grmoe <- GaussRMoE(Xm = X, Ym = y, K = K, Lambda = Lambda, 
                   Gamma = Gamma, option = opt, verbose = TRUE)

Parameters

You can access to the parameters via the following commands:

# Regression coefficients for each expert
grmoe$betak

# Parameters of the gating network
grmoe$wk

Plot

Log-likelihood

grmoe$plot(what = "loglik")

Proportion of zero coefficients

grmoe$plot(what = "zerocoefficients")

fchamroukhi/HDME documentation built on Nov. 4, 2019, 12:37 p.m.