nnf_gelu: Gelu

View source: R/nnf-activation.R

nnf_geluR Documentation

Gelu

Description

Gelu

Usage

nnf_gelu(input, approximate = "none")

Arguments

input

(N,*) tensor, where * means, any number of additional dimensions

approximate

By default it's none, and applies element-wise x*pnorm(x), if 'tanh', then GELU is estimated. See GELU for more info.

gelu(input) -> Tensor

Applies element-wise the function GELU(x) = x * \Phi(x)

where \Phi(x) is the Cumulative Distribution Function for Gaussian Distribution.

See Gaussian Error Linear Units (GELUs).


torch documentation built on May 29, 2024, 9:54 a.m.