newtons_method: Evaluate steepest descent on a quadratic function, assuming a...
In jasmyace/rNeuralNetworkDesign: Learning Neural Networks by Programming in R

Description Usage Arguments Details Value Author(s)

Evaluate steepest descent on a quadratic function, assuming a positive-definite symmetric matrix.

1	newtons_method(x_0, fn, verbose = FALSE, tol = 1e-06)

`x_0`	Initial values for the parameters to be optimized over.
`fn`	A function to be minimized (or maximized), with first argument the vector of parameters over which minimization is to take place. It should return a scalar result. Currently unused.
`verbose`	A logical indicating of the estimated location returned from steepest descent should be reported to the Console for each iteration. Default is `FALSE`.
`tol`	A tolerance to which the algorithm should calculate and surpass.
`A`	A symmetric numeric matrix.
`d`	A numeric vector of the same size as `A`.
`c`	A numeric real number.
`alpha`	A numeric step size.

The arguments A, d, and c together form the quadratic function F(\mathbf{x}) = \frac{1}{2}\mathbf{x}^T\mathbf{A}\mathbf{x} + \mathbf{d}^T\mathbf{x} + c, where \mathbf{A} is of size 2\times 2, \mathbf{x} and \mathbf{d} are of size 2 \times 1, and c \in \mathbb{R}.

The steepest descent algorithm calculates, for each iteration k, values \mathbf{x}_{k+1} = \mathbf{x}_k + α_k \mathbf{g}_k, where α_k > 0 is the learning rate, and \mathbf{g}_k is the gradient of the provided quadratic function F(\mathbf{x}). Typically, α_k is a small positive number. The steepest descent algorithm continues iterating until the distance between \mathbf{x}_{k+1} - \mathbf{x}_k, calculated here via the 2-norm, is less than the tolerance tol.

A list with two elements. The first, numeric vector xStar, with the steepest-descent-found minimum of the provided quadratic function. The second, tibble conv, containing the sequential steps evaluated via steepest descent to find xStar.

Jason Mitchell

jasmyace/rNeuralNetworkDesign documentation built on Jan. 2, 2022, 4:04 p.m.