proj_add_norm | R Documentation |
Takes in two tensors, an "input" and a "residual". Applies a linear projector to the input (changing the size to match residual), performs dropout, adds the result to the residual, then applies layer normalization to the sum.
proj_add_norm(input_size, output_size, hidden_dropout = 0.1)
input_size |
Integer; the size of input tensor. |
output_size |
Integer; the size of output tensor (must match residual). |
hidden_dropout |
Numeric; dropout probability applied after projection. |
Inputs:
input: (*, input_size)
residual: (*, output_size)
Output:
(*, output_size)
in_size <- 4L
out_size <- 3L
model <- proj_add_norm(input_size = in_size, output_size = out_size)
input <- torch::torch_randn(in_size)
residual <- torch::torch_randn(out_size)
model(input, residual)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.