nn_reglu: ReGLU Module
In mlr-org/mlr3torch: Deep Learning with 'mlr3'
nn_reglu R Documentation
ReGLU Module
Description
Rectified Gated Linear Unit (ReGLU) module.
Computes the output as \text{ReGLU}(x, g) = x \cdot \text{ReLU}(g)
where \(x\) and \(g\) are created by splitting the input tensor in half along the last dimension.
Usage
nn_reglu()
References
Shazeer N (2020).
“GLU Variants Improve Transformer.”
2002.05202, https://arxiv.org/abs/2002.05202.
Examples
x = torch::torch_randn(10, 10)
reglu = nn_reglu()
reglu(x)
mlr-org/mlr3torch documentation built on April 17, 2025, 8:22 p.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| nn_reglu | R Documentation |
ReGLU Module
Description
Rectified Gated Linear Unit (ReGLU) module.
Computes the output as \text{ReGLU}(x, g) = x \cdot \text{ReLU}(g)
where \(x\) and \(g\) are created by splitting the input tensor in half along the last dimension.
Usage
nn_reglu()
References
Shazeer N (2020). “GLU Variants Improve Transformer.” 2002.05202, https://arxiv.org/abs/2002.05202.
Examples
x = torch::torch_randn(10, 10)
reglu = nn_reglu()
reglu(x)
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.