In torekleppe/RAutoDiff: Easy to use first and second order Automatic Differentiation

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)

library(RAutoDiff)

Introduction

This document provides an overview of the different functions currently implemented (overloaded) for used with the Automatic Differentiation provided in package RAutoDiff. Most of the functions are written in order to "work" in the same way as the corresponding functions taking regular R-variables as inputs. Therefore this document provides only limited description of these, and the documentation elsewhere is also rather sparse.

Summary of functions and classes

Classes

The package provides three S4 classes and one class union:

• classes fAD and fAD2 represent numeric vectors (slot val), the Jacobian matrix (slot jac) of that vector with respect to the independent variables, and in the case of fAD2, an array of Hessian matrices (slot hessian, with respect to independent variables).
• classUnion ADtype is simply the union of fAD and fAD2.
• class AD_matrix is a matrix class were the matrix element values (slot vals) are stored as an ADtype (in row-major order).

In what remains, fAD and fAD2 should have a similar behavior (in the sense that the methods available for manipulating these should have the same names) as the built in numeric vector class numeric, and AD_matrix should have a similar behavior as built in numeric matrix class

Examples: (in general please avoid accessing and manipulating the slots of the classes directly)

x.n <- c(1,2)
x <- independent2(x.n)
print(x@val) # value of variable
print(x@jac) # jacobian wrt to x.n
print(x@hessian) # hessians of x[i] with respect to x.n

m <- diag(x) # diagonal 2x2 AD_matrix
print(m@nrow)
print(m@ncol)
print(m@vals) 

In general, calculation with fAD-types is somewhat faster (close to fully vectorized) than for fAD2 (several functions have for-loops), and hence fAD2 should be avoided if you do not need second derivatives.

Creation and extraction

Creating AD types

• independent() creates an fAD with values equal to the arguement and Jacobian equal to the identity
• independent() creates an fAD2 with values equal to the arguement and Jacobian equal to the identity, and all Hessians equal to zero.

Extracting from AD types

• value() Numeric value of the ADtype or AD_matrix (also overloaded as identity operations for numeric and matrix)
• gradient() Extract gradient of a scalar argument (length 1 ADtype or 1 by 1 AD_matrix)
• jacobian() Extract Jacobian of a vector (i.e. ADtype)
• hessian() Extract Hessian from a scalar fAD2

Available functions for ADtypes

Non-numerical

• Element access [,[<- e.g x[1]<-x[2] or x[1:2] <- x[2]
• length()
• Replications rep.int() and rep_len() e.g. rep.int(x,4)
• show()

Binary numerical

All of the below apply also for mixed (numeric and ADtype) class argument, e.g. 3 + x or x * c(1,2.5). The methods recycle element in case of different sizes of the arguments.

• +, -, *, /, ^

Binary non-numerical

• ==, != >, >=, <, <=

Unary numerical

• +, - e.g. f <- -x
• exp(), log()
• sqrt()
• square() (square(x) same as x^2 but faster, also available for numeric)
• lgamma()
• sin(), cos()
• abs()
• To do: more trigonometric, erf or pnorm, qnorm,cube

Reductions

• sum()
• prod()
• To do: min(), max(), pmax(), pmin()

Available functions for AD_matrix

Non-numerical

• Element access [,[<- e.g x[1,1]<-x[2,2] or x[1,] <- x[2]
• length(), nrow(), ncol()
• matrix() where argument data is an ADtype
• diag() extract diagonal or create diagonal matrix
• show()
• dim()

Elementwise binary numerical

Including between mixed types:

• +, -, *, /, ^

Elementwise unary numerical

• +, -
• exp(), log()
• sqrt()
• square()
• lgamma()
• sin(), cos()
• abs()

Linear Algebra

• %*% for various combinations of arguments
• backsolve() and forwardsolve()
• t()
• isSymmetric()
• solve()
• cholL(), chol() and solve.chol() for SPD matrices

Reductions

• sum(), colSums(), rowSums()

torekleppe/RAutoDiff documentation built on Dec. 23, 2021, noon