stopt.SA: Optimization over Stiefel Manifold with Simulated Annealing

In RiemStiefel: Inference, Learning, and Optimization on Stiefel Manifold

Description

Simulated Annealing is a black-box, derivative-free optimization algorithm that iterates via stochastic search in the neighborhood of current position.

Usage

stopt.SA(
  func,
  size,
  n.start = 10,
  stepsize = 0.1,
  maxiter = 100,
  cooling = c("exponential", 10, 0.9),
  init.val = NULL,
  print.progress = FALSE
)

Arguments

func	a function to be minimized.
size	a length-2 vector containing dimension information (p,r).
n.start	number of runs, i.e., algorithm is executed n.start times.
stepsize	size of random walk on each component.
maxiter	maximum number of iterations for each run.
cooling	triplet for cooling schedule. See the section for the usage.
init.val	if NULL, starts from a random point. Otherwise, a Stiefel matrix of size (p,r) should be provided for fixed starting point.
print.progress	a logical; TRUE to show

Value

a named list containing:

cost

minimized function value.

solution

a (p\times r) matrix that attains the cost.

accfreq

frequency of acceptance moves.

Examples

## Optimization for eigen-decomposition
#  Let's find top-3 eigenvalues 
set.seed(121)                         # set seed
A = cov(matrix(rnorm(100*5), ncol=5)) # define covariance
myfunc <- function(p){                # cost function
  return(sum(-diag(t(p)%*%A%*%p)))
} 

#  Solve the optimization problem
Aout = stopt.SA(myfunc, size=c(5,3), n.start=40, maxiter=500)

#  Compute 3 Eigenvalues
#  1. use computed basis
abase   = Aout$solution
eig3sol = sort(diag(t(abase)%*%A%*%abase), decreasing=TRUE)

#  2. use 'eigen' function
eig3dec = sort(eigen(A)$values, decreasing=TRUE)[1:3]


#   Visualize
opar <- par(no.readonly=TRUE)
yran = c(min(min(eig3sol),min(eig3dec))*0.95,
         max(max(eig3sol),max(eig3dec))*1.05)
plot(1:3, eig3sol, type="b", col="red",  pch=19, ylim=yran,
     xlab="index", ylab="eigenvalue", main="compare top 3 eigenvalues")
lines(1:3, eig3dec, type="b", col="blue", pch=19)
legend(1, 1, legend=c("optimization","decomposition"), col=c("red","blue"),
       lty=rep(1,2), pch=19)
par(opar)

RiemStiefel documentation built on March 26, 2020, 7:48 p.m.