ReLU: Rectified linear outout activation

In frhl/neuralnetr: Neural Nets in R

Description

Each activation module has a forward method that takes in a batch of pre-activations Z and returns a batch of activations A.

Each activation module has a backward method that takes in dLdA and returns dLdZ, with the exception of SoftMax, where we assume dLdZ is passed in.

Super class

neuralnetr::ClassModule -> ReLU

Public fields

A

the activation vector

Methods

Public methods

• ReLU$forward()

• ReLU$backward()

• ReLU$clone()

Inherited methods

• neuralnetr::ClassModule$sgd_step()

---

Method forward()

Usage

ReLU$forward(Z)

Arguments

Z

a vector of pre-activations

Returns

a vector of activations.

---

Method backward()

Usage

ReLU$backward(dLdA)

Arguments

dLdA

vector of gradients.

Returns

a vector gradeints.

---

Method clone()

The objects of this class are cloneable with this method.

Usage

ReLU$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

See Also

Other activation: Sigmoid, SoftMax, Tanh

frhl/neuralnetr documentation built on Nov. 9, 2020, 2:24 p.m.