knn.forecast.randomsearch.tuning: Tune 'knn.forecast()' Hyperparameters with Random Search
In knnwtsim: K Nearest Neighbor Forecasting with a Tailored Similarity Metric

knn.forecast.randomsearch.tuning

R Documentation

Tune `knn.forecast()` Hyperparameters with Random Search

Description

A simplistic automated hyperparameter tuning function which randomly generates a grid of hyperparameter sets used to build corresponding S_w similarity matrices which are used in knn.forecast() test against the last test.h points of y.in after any val.holdout.len points are removed from the end of y.in. The best performing set of parameters based on MAPE over over the forecast horizon of test.h points are returned as part of a list alongside the 'optimum' weighted similarity matrix Sw.opt, the Grid of tested sets, and the MAPE results. MAPE is the average of absolute percent errors for each point calculated as: abs((test.actuals - test.forecast.i) / test.actuals) * 100. Where test.forecast.i and test.actuals are both numeric vectors.

Usage

knn.forecast.randomsearch.tuning(
  grid.len = 100,
  St.in,
  Sp.in,
  Sx.in,
  y.in,
  test.h = 1,
  max.k = NULL,
  val.holdout.len = 0,
  min.k = 1
)

Arguments

`grid.len`	integer value representing the number of hyperparameter sets to generate and test, must be `>= 1`.
`St.in`	numeric and symmetric matrix of similarities, can be generated with `StMatrixCalc()`.
`Sp.in`	numeric and symmetric matrix of similarities, can be generated with `SpMatrixCalc()`.
`Sx.in`	numeric and symmetric matrix of similarities, can be generated with `SxMatrixCalc()`.
`y.in`	numeric vector of the response series to be forecast.
`test.h`	integer value representing the number of points in the test forecast horizon, must be `>= 1`.
`max.k`	integer value representing the maximum value of k, `knn.forecast()` should use, will be set to `min(floor((length(y.in)) * .4), length(y.in) - val.holdout.len - test.h)` if `NULL` or `NA` is passed. Note this `NA` behavior differs from `knnwtsim` version 0.1.0. If a numeric value is passed it must be `>= 1`.
`val.holdout.len`	integer value representing the number of observations at the end of the series to be removed in testing forecast if desired to leave a validation set after tuning, must be `>= 0`.
`min.k`	integer value representing the minimum value of k, `knn.forecast()` should use, must be `>= 1`.

Value

list of the following components:

weight.opt: numeric vector of the 3 weights to generate Sw.opt in alpha, beta, gamma order which achieved the best performance in terms of MAPE.
k.opt: integer value of neighbors used in knn.forecast() which achieved the best performance in terms of MAPE.
Sw.opt: numeric matrix of similarities calculated using S_w, with the best performing set of hyperparameters.
Test.MAPE: numeric value of the MAPE result for the optimum hyperparamter set achieved on the test points.
MAPE.all: numeric vector of MAPE results, each observation corresponds to the row in Grid of the same index.
Grid: dataframe of all hyperparameter sets tested in the tuning.

Examples

data("simulation_master_list")
series.index <- 15
ex.series <- simulation_master_list[[series.index]]$series.lin.coef.chng.x

df <- data.frame(ex.series)
# Generate vector of time orders
df$t <- c(1:nrow(df))
# Generate vector of periods
nperiods <- simulation_master_list[[series.index]]$seasonal.periods
df$p <- rep(1:nperiods, length.out = nrow(df))
# Pull corresponding exogenous predictor(s)
X <- as.matrix(simulation_master_list[[series.index]]$x.chng)

St.ex <- StMatrixCalc(df$t)
Sp.ex <- SpMatrixCalc(df$p, nPeriods = nperiods)
Sx.ex <- SxMatrixCalc(X)

tuning.test <- knn.forecast.randomsearch.tuning(
  grid.len = 10,
  y.in = ex.series,
  St.in = St.ex,
  Sp.in = Sp.ex,
  Sx.in = Sx.ex,
  test.h = 3,
  max.k = 10,
  val.holdout.len = 3
)

knnwtsim documentation built on March 18, 2022, 6 p.m.