activation_gelu: Gaussian error linear unit (GELU) activation function.

In rstudio/keras: R Interface to 'Keras'

Gaussian error linear unit (GELU) activation function.

Description

The Gaussian error linear unit (GELU) is defined as:

gelu(x) = x * P(X <= x) where P(X) ~ N(0, 1), i.e. gelu(x) = 0.5 * x * (1 + erf(x / sqrt(2))).

GELU weights inputs by their value, rather than gating inputs by their sign as in ReLU.

Usage

activation_gelu(x, approximate = FALSE)

Arguments

x	Input tensor.
approximate	A bool, whether to enable approximation.

Value

A tensor, the result from applying the activation to the input tensor x.

Reference

• Hendrycks et al., 2016

See Also

• https://keras.io/api/layers/activations#gelu-function

Other activations:
activation_elu()
activation_exponential()
activation_hard_sigmoid()
activation_leaky_relu()
activation_linear()
activation_log_softmax()
activation_mish()
activation_relu()
activation_relu6()
activation_selu()
activation_sigmoid()
activation_silu()
activation_softmax()
activation_softplus()
activation_softsign()
activation_tanh()

rstudio/keras documentation built on April 27, 2024, 10:11 p.m.