act_method: Activation Function

In UAHDataScienceSC: Learn Supervised Classification Methods Through Examples and Code

Activation Function

Description

Upon a received input, calculates the output based on the selected activation function

Usage

act_method(method, x)

Arguments

method	Activation function to be used. It must be one of "step", "sine", "tangent", "linear", "relu", "gelu" or "swish".
x	Input value to be used in the activation function.

Details

Formulae used:

step

f(x) = \begin{cases} 0 & \text{if } x < \text{threshold} \\ 1 & \text{if } x \geq \text{threshold} \end{cases}

sine

f(x) = \sinh(x)

tangent

f(x) = \tanh(x)

linear

x

relu

f(x) = \begin{cases} x & \text{if } x > 0 \\ 0 & \text{if } x \leq 0 \end{cases}

gelu

f(x) = \frac{1}{2} \cdot x \cdot \left(1 + \tanh\left(\sqrt{\frac{2}{\pi}} \cdot (x + 0.044715 \cdot x^3)\right)\right)

swish

f(x) = \frac{x}{1 + e^{-x}}

Value

List with the weights of the inputs.

Author(s)

Víctor Amador Padilla, [e-mail]

Examples

# example code
act_method("step", 0.3)
act_method("gelu", 0.7)

UAHDataScienceSC documentation built on April 3, 2025, 8:58 p.m.