layer_norm: Run layer normalization

In jonathanbratt/RBERT: R Implementation of BERT

Run layer normalization

Description

Run layer normalization on the last dimension of the tensor.

Usage

layer_norm(input_tensor, name = NULL)

Arguments

input_tensor	Tensor to perform layor normalization on.
name	Optional variable_scope for layer_norm.

Details

Wrapper around tensorflow layer_norm function. From tensorflow documentation: Adds a Layer Normalization layer. Based on the paper: https://arxiv.org/abs/1607.06450.

Note: begin_norm_axis: The first normalization dimension: normalization will be performed along dimensions (begin_norm_axis : rank(inputs) )

begin_params_axis: The first parameter (beta, gamma) dimension: scale and centering parameters will have dimensions (begin_params_axis : rank(inputs) ) and will be broadcast with the normalized inputs accordingly.

Value

A Tensor of the same shape and type as 'input_tensor', with normalization applied.

Examples

## Not run: 
tfx <- tensorflow::tf$get_variable("example", tensorflow::shape(10L))
layer_norm(tfx)

## End(Not run)

jonathanbratt/RBERT documentation built on Jan. 26, 2023, 4:15 p.m.