| ao | R Documentation |
Alternating optimization (AO) is an iterative process for optimizing a real-valued function jointly over all its parameters by alternating restricted optimization over parameter partitions.
ao(
f,
initial,
target = NULL,
npar = NULL,
gradient = NULL,
hessian = NULL,
...,
partition = "sequential",
new_block_probability = 0.3,
minimum_block_number = 1,
minimize = TRUE,
lower = NULL,
upper = NULL,
iteration_limit = Inf,
seconds_limit = Inf,
tolerance_value = 1e-06,
tolerance_parameter = 1e-06,
tolerance_parameter_norm = function(x, y) sqrt(sum((x - y)^2)),
tolerance_history = 1,
base_optimizer = optimizeR::Optimizer$new("stats::optim", method = "L-BFGS-B", control
= list(maxit = 10)),
verbose = FALSE,
hide_warnings = TRUE,
add_details = TRUE
)
f |
[ The first argument of If |
initial |
[ This can also be a |
target |
[ This can only be If |
npar |
[ Must be specified if more than one target argument is specified via
the Can be |
gradient |
[ The function signature of Ignored if |
hessian |
[ The function signature of Ignored if |
... |
Additional arguments to be passed to |
partition |
[
This can also be a |
new_block_probability |
[ The probability of creating a new parameter block when creating a random partition. Values close to 0 result in larger parameter blocks, values close to 1 result in smaller parameter blocks. |
minimum_block_number |
[ The minimum number of blocks in random partitions. |
minimize |
[ If |
lower, upper |
[ The values in Ignored if |
iteration_limit |
[ Can also be |
seconds_limit |
[ Can also be Note that this stopping criterion refers to the time for solving the sub-problems and is only checked after a sub-problem is solved and not within solving a sub-problem, so the actual process time can exceed this limit. |
tolerance_value |
[ Can be |
tolerance_parameter |
[ Can be By default, the distance is measured using the Euclidean norm, but another
norm can be specified via the |
tolerance_parameter_norm |
[ It must be of the form |
tolerance_history |
[ |
base_optimizer |
[ By default, the This can also be a |
verbose |
[ Not supported when using multiple processes, see details. |
hide_warnings |
[ |
add_details |
[ |
AO can suffer from local optima. To increase the likelihood of finding a better optimum, you can specify:
multiple starting parameters
multiple parameter partitions
multiple base optimizers
Use the initial, partition, and/or base_optimizer arguments to provide
a list of possible values for each parameter. Each combination of initial
values, parameter partitions, and base optimizers will create a separate AO
process.
In the case of multiple processes, the output values refer to the best AO process with respect to function value.
If add_details = TRUE, the following elements are added:
estimates is a list of optimal parameters in each process.
values is a list of optimal function values in each process.
details combines details of the single processes and has an
additional column process with an index for the different processes.
seconds_each gives the computation time in seconds for each process.
stopping_reasons gives the termination message for each process.
processes gives details on how the processes were specified.
By default, processes run sequentially. However, since they are independent,
they can be parallelized. To enable parallel computation, use the
{future} framework, which requires the
{future.apply} package to be installed. For example, run the following
before the ao() call:
future::plan(future::multisession, workers = 4)
When using multiple processes, setting verbose = TRUE to print tracing
details during AO is not supported. However, you can still track the progress
using the {progressr} framework,
which requires the {progressr} package to be installed.
For example, run the following before the ao() call:
progressr::handlers(global = TRUE)
progressr::handlers(
progressr::handler_progress(":percent :eta :message")
)
A list with the following elements:
estimate is the parameter vector with the best function value in the AO
process.
estimate_split is estimate split by target (only when applicable).
value is the function value at estimate.
details is a data.frame with information about the AO process:
For each iteration (column iteration) it contains the function value
(column value), parameter values (columns starting with p followed by
the parameter index), the active parameter block (columns starting with b
followed by the parameter index, where 1 stands for a parameter contained
in the active parameter block and 0 if not), and computation times in
seconds (column seconds). Only available if add_details = TRUE.
seconds is the total time in seconds for solving the sub-problems.
stopping_reason is a message explaining why the AO process terminated.
In the case of multiple processes, the output changes slightly, see details.
# Example 1: Minimization of Himmelblau's function --------------------------
himmelblau <- function(x) (x[1]^2 + x[2] - 11)^2 + (x[1] + x[2]^2 - 7)^2
ao(f = himmelblau, initial = c(0, 0))
# Example 2: Maximization of 2-class Gaussian mixture log-likelihood --------
normal_mixture_loglik_uc <- function(mu, logsd, eta, data) {
sd <- exp(logsd)
e <- exp(eta[1])
den <- 1 + e
q1 <- e / den
q2 <- 1 / den
l1 <- log(q1) + dnorm(data, mu[1], sd[1], log = TRUE)
l2 <- log(q2) + dnorm(data, mu[2], sd[2], log = TRUE)
m <- pmax(l1, l2)
sum(m + log(exp(l1 - m) + exp(l2 - m)))
}
set.seed(123)
data <- datasets::faithful$eruptions
fit <- ao(
f = normal_mixture_loglik_uc,
initial = c(mean(data) + c(-1, 1), rep(log(sd(data)), 2), 0),
target = c("mu", "logsd", "eta"),
npar = c(2, 2, 1),
data = data,
partition = "random",
base_optimizer = optimizeR::Optimizer$new("stats::nlm"),
minimize = FALSE,
add_details = FALSE
)
(muhat <- fit$estimate_split$mu)
(sdhat <- exp(fit$estimate_split$logsd))
e <- exp(fit$estimate_split$eta)
den <- 1 + e
(qhat <- c(e / den, 1 / den))
# Example 3: Constrained Optimization in the Setting of Example 2 -----------
# target arguments:
# - class means mu (2, unrestricted)
# - class standard deviations sd (2, must be non-negative)
# - class proportion lambda (only 1 for identification, must be in [0, 1])
normal_mixture_loglik <- function(mu, sd, lambda, data) {
c1 <- lambda * dnorm(data, mu[1], sd[1])
c2 <- (1 - lambda) * dnorm(data, mu[2], sd[2])
sum(log(c1 + c2))
}
set.seed(123)
ao(
f = normal_mixture_loglik,
initial = runif(5),
target = c("mu", "sd", "lambda"),
npar = c(2, 2, 1),
data = datasets::faithful$eruptions,
partition = list("sequential", "random", "none"),
minimize = FALSE,
lower = c(-Inf, -Inf, 0, 0, 0),
upper = c(Inf, Inf, Inf, Inf, 1),
add_details = FALSE
)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.