compute_MSEv_eval_crit: Mean Squared Error of the Contribution Function 'v(S)'
In shapr: Prediction Explanation with Dependence-Aware Shapley Values

compute_MSEv_eval_crit

R Documentation

Mean Squared Error of the Contribution Function `v(S)`

Description

Function that computes the Mean Squared Error (MSEv) of the contribution function v(s) as proposed by Frye et al. (2019) and used by Olsen et al. (2022).

Usage

compute_MSEv_eval_crit(
  internal,
  dt_vS,
  MSEv_uniform_comb_weights,
  MSEv_skip_empty_full_comb = TRUE
)

Arguments

`internal`	List. Holds all parameters, data, functions and computed objects used within `explain()` The list contains one or more of the elements `parameters`, `data`, `objects`, `iter_list`, `timing_list`, `main_timing_list`, `output`, and `iter_timing_list`.
`dt_vS`	Data.table of dimension `n_coalitions` times `n_explain + 1` containing the contribution function estimates. The first column is assumed to be named `id_coalition` and containing the ids of the coalitions. The last row is assumed to be the full coalition, i.e., it contains the predicted responses for the observations which are to be explained.
`MSEv_uniform_comb_weights`	Logical. If `TRUE` (default), then the function weights the coalitions uniformly when computing the MSEv criterion. If `FALSE`, then the function use the Shapley kernel weights to weight the coalitions when computing the MSEv criterion. Note that the Shapley kernel weights are replaced by the sampling frequency when not all coalitions are considered.
`MSEv_skip_empty_full_comb`	Logical. If `TRUE` (default), we exclude the empty and grand coalitions when computing the MSEv evaluation criterion. This is reasonable as they are identical for all methods, i.e., their contribution function is independent of the used method as they are special cases not effected by the used method. If `FALSE`, we include the empty and grand coalitions. In this situation, we also recommend setting `MSEv_uniform_comb_weights = TRUE`, as otherwise the large weights for the empty and grand coalitions will outweigh all other coalitions and make the MSEv criterion uninformative.

Details

The MSEv evaluation criterion does not rely on access to the true contribution functions nor the true Shapley values to be computed. A lower value indicates better approximations, however, the scale and magnitude of the MSEv criterion is not directly interpretable in regard to the precision of the final estimated Shapley values. Olsen et al. (2024) illustrates in Figure 11 a fairly strong linear relationship between the MSEv criterion and the MAE between the estimated and true Shapley values in a simulation study. Note that explicands refer to the observations whose predictions we are to explain.

Value

List containing:

MSEv: A data.table with the overall MSEv evaluation criterion averaged over both the coalitions and observations/explicands. The data.table also contains the standard deviation of the MSEv values for each explicand (only averaged over the coalitions) divided by the square root of the number of explicands.
MSEv_explicand: A data.table with the mean squared error for each explicand, i.e., only averaged over the coalitions.
MSEv_coalition: A data.table with the mean squared error for each coalition, i.e., only averaged over the explicands/observations. The data.table also contains the standard deviation of the MSEv values for each coalition divided by the square root of the number of explicands.