lekprofile: Sensitivity analysis using Lek's profile method
In NeuralNetTools: Visualization and Analysis Tools for Neural Networks

Description Usage Arguments Details Value References Examples

Conduct a sensitivity analysis of model responses in a neural network to input variables using Lek's profile method

lekprofile(mod_in, ...)

## Default S3 method:
lekprofile(
  mod_in,
  xvars,
  ysel = NULL,
  xsel = NULL,
  steps = 100,
  group_vals = seq(0, 1, by = 0.2),
  val_out = FALSE,
  group_show = FALSE,
  grp_nms = NULL,
  position = "dodge",
  ...
)

## S3 method for class 'nnet'
lekprofile(mod_in, xsel = NULL, ysel = NULL, ...)

## S3 method for class 'mlp'
lekprofile(mod_in, xvars, yvars, xsel = NULL, ysel = NULL, ...)

## S3 method for class 'train'
lekprofile(mod_in, xsel = NULL, ysel = NULL, ...)

## S3 method for class 'nn'
lekprofile(mod_in, xsel = NULL, ysel = NULL, ...)

`mod_in`	input object for which an organized model list is desired. The input can be an object of class `nnet` or `mlp`
`...`	arguments passed to other methods
`xvars`	`data.frame` of explanatory variables used to create the input model, only needed for `mlp` objects
`ysel`	chr string indicating which response variables to plot if more than one, defaults to all
`xsel`	chr string of names of explanatory variables to plot, defaults to all
`steps`	numeric value indicating number of observations to evaluate for each explanatory variable from minimum to maximum value, default 100
`group_vals`	numeric vector with values from 0-1 indicating quantile values at which to hold other explanatory variables constant or a single value indicating number of clusters to define grouping scheme, see details
`val_out`	logical value indicating if actual sensitivity values are returned rather than a plot, default `FALSE`
`group_show`	logical if a barplot is returned that shows the values at which explanatory variables were held constant while not being evaluated
`grp_nms`	optional chr string of alternative names for groups in legend
`position`	chr string indicating bar position (e.g., 'dodge', 'fill', 'stack'), passed to `geom_bar`, used if `group_show = TRUE`
`yvars`	`data.frame` of explanatory variables used to create the input model, only needed for `mlp` objects

The Lek profile method is described briefly in Lek et al. 1996 and in more detail in Gevrey et al. 2003. The profile method is fairly generic and can be extended to any statistical model in R with a predict method. However, it is one of few methods used to evaluate sensitivity in neural networks.

The profile method can be used to evaluate the effect of explanatory variables by returning a plot of the predicted response across the range of values for each separate variable. The original profile method evaluated the effects of each variable while holding the remaining explanatory variables at different quantiles (e.g., minimum, 20th percentile, maximum). This is implemented in in the function by creating a matrix of values for explanatory variables where the number of rows is the number of observations and the number of columns is the number of explanatory variables. All explanatory variables are held at their mean (or other constant value) while the variable of interest is sequenced from its minimum to maximum value across the range of observations. This matrix (or data frame) is then used to predict values of the response variable from a fitted model object. This is repeated for each explanatory variable to obtain all response curves. Values passed to group_vals must range from zero to one to define the quantiles for holding unevaluated explanatory variables.

An alternative implementation of the profile method is to group the unevaluated explanatory variables using groupings defined by the statistical properties of the data. Covariance among predictors may present unlikely scenarios if holding all unevaluated variables at the same level. To address this issue, the function provides an option to hold unevaluated variable at mean values defined by natural clusters in the data. kmeans clustering is used on the input data.frame of explanatory variables if the argument passed to group_vals is an integer value greater than one. The centers of the clusters are then used as constant values for the unevaluated variables. An arbitrary grouping scheme can also be passed to group_vals as a data.frame where the user can specify exact values for holding each value constant (see the examples).

For all plots, the legend with the 'Groups' label indicates the colors that correspond to each group. The groups describe the values at which unevaluated explanatory variables were held constant, either as specific quantiles, group assignments based on clustering, or in the arbitrary grouping defined by the user. The constant values of each explanatory variable for each group can be viewed as a barplot by using group_show = TRUE.

Note that there is no predict method for neuralnet objects from the nn package. The lekprofile method for nn objects uses the nnet package to recreate the input model, which is then used for the sensitivity predictions. This approach only works for networks with one hidden layer.

A ggplot object for plotting if val_out = FALSE, otherwise a two-element list is returned with a data.frame in long form showing the predicted responses at different values of the explanatory variables and the grouping scheme that was used to hold unevaluated variables constant.

Beck, M.W. 2018. NeuralNetTools: Visualization and Analysis Tools for Neural Networks. Journal of Statistical Software. 85(11):1-20.

Lek, S., Delacoste, M., Baran, P., Dimopoulos, I., Lauga, J., Aulagnier, S. 1996. Application of neural networks to modelling nonlinear relationships in Ecology. Ecological Modelling. 90:39-52.

Gevrey, M., Dimopoulos, I., Lek, S. 2003. Review and comparison of methods to study the contribution of variables in artificial neural network models. Ecological Modelling. 160:249-264.

Olden, J.D., Joy, M.K., Death, R.G. 2004. An accurate comparison of methods for quantifying variable importance in artificial neural networks using simulated data. Ecological Modelling. 178:389-397.

## using nnet

library(nnet)

set.seed(123)

mod <- nnet(Y1 ~ X1 + X2 + X3, data = neuraldat, size = 5)
 
lekprofile(mod)  

## Not run: 
## using RSNNS, no bias layers

library(RSNNS)

x <- neuraldat[, c('X1', 'X2', 'X3')]
y <- neuraldat[, 'Y1', drop = FALSE]

mod <- mlp(x, y, size = 5)

lekprofile(mod, xvars = x)

## using neuralnet

library(neuralnet)

mod <- neuralnet(Y1 ~ X1 + X2 + X3, data = neuraldat, hidden = 5)

lekprofile(mod)

## back to nnet, not using formula to create model
## y variable must have a name attribute

mod <- nnet(x, y, size = 5)

lekprofile(mod)

## using caret

library(caret)

mod <- train(Y1 ~ X1 + X2 + X3, method = 'nnet', data = neuraldat, linout = TRUE)

lekprofile(mod)

## group by clusters instead of sequencing by quantiles

mod <- nnet(Y1 ~ X1 + X2 + X3, data = neuraldat, size = 5)
 
lekprofile(mod, group_vals = 6) # six clusters

## enter an arbitrary grouping scheme for the group values
## i.e. hold all values at 0.5
group_vals <- rbind(rep(0.5, length = ncol(x)))
group_vals <- data.frame(group_vals)
names(group_vals) <- names(group_vals)

lekprofile(mod, group_vals = group_vals, xsel = 'X3')

## End(Not run)