R/document_layer.R

In multiRL: Reinforcement Learning Tools for Multi-Armed Bandit

#' @title Layers and Loss Functions (RNN)
#' @name layer
#' @description
#'  Current supported recurrent layer types and available loss functions in
#'  the package.
#'
#' @section Recurrent Layer:
#' \itemize{
#'    \item \code{"RNN"} (Simple Recurrent Neural Network) and \code{"BiRNN"}:
#'      A fully-connected RNN where the output from the previous time step is
#'        fed back to the next time step. It's the most basic type of
#'        recurrent layer but can struggle with long-term dependencies due to
#'        the vanishing gradient problem.
#'
#'    \item \code{"GRU"} (Gated Recurrent Unit) and \code{"BiGRU"}:
#'      A modern recurrent unit that uses gating mechanisms to control
#'        information flow, enabling it to capture long-range dependencies.
#'        GRUs are generally simpler and computationally faster than LSTMs
#'        while often achieving comparable performance.
#'
#'    \item \code{"LSTM"} (Long Short-Term Memory)  and \code{"BiLSTM"} :
#'      A powerful recurrent unit with dedicated memory cells and gating
#'        mechanisms (input, forget, output). LSTMs excel at learning
#'        long-term dependencies and are robust against the vanishing
#'        gradient problem, making them ideal for very long sequences.
#'
#' }
#'
#' @section Loss Function:
#'      The loss function defines the objective that the model minimizes
#'        during training. The choice of loss function is critical as it
#'        determines what aspect of the prediction the model prioritizes.
#'      \itemize{
#'        \item \code{"MSE"} (Mean Squared Error):
#'          Calculates the average of the squared differences between predicted
#'            and true parameter values. By squaring the error, it heavily
#'            penalizes large mistakes. It is the standard choice for regression
#'            and implicitly assumes that the errors are normally distributed.
#'            However, its sensitivity to outliers can sometimes be a drawback.
#'
#'        \item \code{"MAE"} (Mean Absolute Error):
#'          Calculates the average of the absolute differences between predicted
#'            and true values. It treats all errors equally on a linear scale,
#'            making it more robust to outliers than MSE. It is a good choice
#'            when the dataset contains anomalies that should not dominate the
#'            training process.
#'
#'        \item \code{"HBR"} (Huber Loss):
#'          A hybrid loss function that combines the best properties of MSE and
#'            MAE. It behaves like MSE for small errors, providing a smooth and
#'            stable gradient, but switches to behaving like MAE for large
#'            errors. This makes it less sensitive to outliers than MSE while
#'            remaining differentiable at zero.
#'
#'        \item \code{"NLL"} (Negative Log-Likelihood):
#'          This loss is used for probabilistic regression. Instead of
#'            predicting a single value for each parameter, the network
#'            predicts the parameters of a probability distribution (here, a
#'            Gaussian: its mean \code{mu} and variance \code{sigma^2}). The
#'            loss is the negative log-likelihood of the true parameters under
#'            the predicted distribution. This allows the model to learn and
#'            express its own uncertainty about its predictions.
#'
#'        \item \code{"QRL"} (Quantile Regression Loss):
#'          Allows the model to estimate specific quantiles of the parameter
#'            distribution, rather than just its mean. This package's
#'            implementation predicts the 5th, 50th (median), and 95th
#'            percentiles. It uses a "pinball loss" function that is asymmetric,
#'            guiding the model to the desired quantile. It is useful for
#'            understanding the full range of parameter uncertainty and is
#'            naturally robust to outliers.
#'
#'        \item \code{"MDN"} (Mixture Density Network):
#'          The most flexible but complex option. An MDN learns to predict the
#'            parameters of a mixture of distributions (e.g., a mix of multiple
#'            Gaussians). This allows it to model highly complex, multi-modal
#'            (multiple peaks), or skewed posterior distributions. The network
#'            outputs the means, variances, and mixing weights for each
#'            component in the mixture.
#'      }
#'
#' @section Example:
#' \preformatted{ # supported recurrent layer and loss function
#'  control = list(
#'    layer = c("RNN", "GRU", "LSTM", "BiRNN", "BiGRU", "BiLSTM"),
#'    loss = c("MSE", "MAE", "HBR", "NLL", "QRL", "MDN")
#'  )
#' }
#'
NULL