nnf_gelu: Gelu

In torch: Tensors and Neural Networks with 'GPU' Acceleration

Gelu

Description

Gelu

Usage

nnf_gelu(input, approximate = "none")

Arguments

input	(N,*) tensor, where * means, any number of additional dimensions
approximate	By default it's none, and applies element-wise x*pnorm(x), if 'tanh', then GELU is estimated. See GELU for more info.

gelu(input) -> Tensor

Applies element-wise the function GELU(x) = x * \Phi(x)

where \Phi(x) is the Cumulative Distribution Function for Gaussian Distribution.

See Gaussian Error Linear Units (GELUs).

torch documentation built on June 7, 2023, 6:19 p.m.