fineTuning,knnFineTune | R Documentation |

Adds various extra features to grid search for specified tuning parameter/hyperparameter combinations: There is a plot() function, using parallel coordinates graphs to show trends among the different combinations; and Bonferroni confidence intervals are computed to avoid p-hacking. An experimental smoothing facility is also included.

fineTuning(dataset,pars,regCall,nCombs=NULL,specCombs=NULL,nTst=500, nXval=1,up=TRUE,k=NULL,dispOrderSmoothed=FALSE, showProgress=TRUE,...) ## S3 method for class 'tuner' plot(x,...) knnFineTune(data,yName,k,expandVars,ws,classif=FALSE,seed=9999) fineTuningPar(cls,dataset,pars,regCall,nCombs=NULL,specCombs=NULL, nTst=500,nXval=1,up=TRUE,k=NULL,dispOrderSmoothed=FALSE)

`...` |
Arguments to be passed on by |

`x` |
Output object from |

`cls` |
A |

`dataset` |
Data frame etc. containing the data to be analyzed. |

`data` |
The data to be analyzed. |

`yName` |
Quoted name of "Y" in the column names of |

`expandVars` |
Indices of columns in |

`ws` |
Weights to be used for |

`classif` |
Set to TRUE for classification problems. |

`seed` |
Seed for random number generation. |

`pars` |
R list, showing the desired tuning parameter values. |

`regCall` |
Function to be called at each parameter combination, performing the model fit etc. |

`nCombs` |
Number of parameter combinations to run. If Null, all will be run |

.

`nTst` |
Number of data points to be in the test set. |

`nXval` |
Number of folds to be run for a given data partition and parameter combination. |

`k` |
Nearest-neighbor smoothing parameter. |

`up` |
If TRUE, display results in ascending order of performance value. |

`dispOrderSmoothed` |
Display in order of smoothed results. |

`showProgress` |
If TRUE, print each output line as it becomes ready. |

`specCombs` |
A data frame in which the user specifies # hyperparameter parameter combinations to evaluate. |

The user specifies the values for each tuning parameter in
`pars`

. This leads to a number of possible combinations of the
parameters. In many cases, there are more combinations than the user
wishes to try, so `nCombs`

of them will be chosen at random.

For each combination, the function will run the analysis specified by
the user in `regCall`

. The latter must have the call form

`ftnName(dtrn,dtst,cmbi`

Again, note that it is `fineTuning`

that calls this function. It
will provide the training and test sets `dtrn`

and `dtst`

, as
well as `cmbi`

("combination i"), the particular parameter
combination to be run at this moment.

Each chosen combination is run in `nXval`

folds. All specified
combinations are run fully, as opposed to a directional "hill descent"
search that hopes it might eliminate poor combinations early in the process.

The function `knnFineTune`

is a wrapper for `fineTuning`

for
k-NN problems.

The function `plot.tuner`

draws a parallel coordinates plot to
visualize the grid. The argument `x`

is the output of
`fineTuning`

. Arguments to specify in the ellipsis are:
`col`

is the column to be plotted;
`disp`

is the number to display, with `0`

, `-m`

and
`+m`

meaning cases with the `m`

smallest 'smoothed' values, all
cases and the `m`

largest values of 'smoothed', respectively;
`jit`

avoids plotting coincident lines by adding jitter in the
amount `jit * range(x) * runif(n,-0.5,0.5)`

.

Object of class **”tuner'**. Contains the grid results, including upper bounds of approximate one-sided 95 univariate and Bonferroni-Dunn (adjusted for the number of parameter combinations).

Norm Matloff

# mlb data set, predict weight using k-NN, try various values of k tc <- function(dtrn,dtst,cmbi,...) { knnout <- kNN(dtrn[,-3],dtrn[,3],dtst[,-3],as.integer(cmbi[1])) preds <- knnout$regests mean(abs(preds - dtst[,3])) } data(mlb) mlb <- mlb[,3:6] mlb.d <- factorsToDummies(mlb) fineTuning(mlb.d,list(k=c(5,25)),tc,nTst=100,nXval=2)

Embedding an R snippet on your website

Add the following code to your website.

For more information on customizing the embed code, read Embedding Snippets.