compute_MSEv_eval_crit: Mean Squared Error of the Contribution Function 'v(S)'

In NorskRegnesentral/shapr: Prediction Explanation with Dependence-Aware Shapley Values

Mean Squared Error of the Contribution Function v(S)

Description

Function that computes the Mean Squared Error (MSEv) of the contribution function v(s) as proposed by Frye et al. (2019) and used by Olsen et al. (2022).

Usage

compute_MSEv_eval_crit(
  internal,
  dt_vS,
  MSEv_uniform_comb_weights,
  MSEv_skip_empty_full_comb = TRUE
)

Arguments

internal	List. Holds all parameters, data, functions and computed objects used within explain() The list contains one or more of the elements parameters, data, objects, output.
dt_vS	Data.table of dimension n_combinations times n_explain + 1 containing the contribution function estimates. The first column is assumed to be named id_combination and containing the ids of the combinations. The last row is assumed to be the full combination, i.e., it contains the predicted responses for the observations which are to be explained.
MSEv_uniform_comb_weights	Logical. If TRUE (default), then the function weights the combinations uniformly when computing the MSEv criterion. If FALSE, then the function use the Shapley kernel weights to weight the combinations when computing the MSEv criterion. Note that the Shapley kernel weights are replaced by the sampling frequency when not all combinations are considered.
MSEv_skip_empty_full_comb	Logical. If TRUE (default), we exclude the empty and grand combinations/coalitions when computing the MSEv evaluation criterion. This is reasonable as they are identical for all methods, i.e., their contribution function is independent of the used method as they are special cases not effected by the used method. If FALSE, we include the empty and grand combinations/coalitions. In this situation, we also recommend setting MSEv_uniform_comb_weights = TRUE, as otherwise the large weights for the empty and grand combinations/coalitions will outweigh all other combinations and make the MSEv criterion uninformative.

Details

The MSEv evaluation criterion does not rely on access to the true contribution functions nor the true Shapley values to be computed. A lower value indicates better approximations, however, the scale and magnitude of the MSEv criterion is not directly interpretable in regard to the precision of the final estimated Shapley values. Olsen et al. (2022) illustrates in Figure 11 a fairly strong linear relationship between the MSEv criterion and the MAE between the estimated and true Shapley values in a simulation study. Note that explicands refer to the observations whose predictions we are to explain.

Value

List containing:

MSEv

A data.table with the overall MSEv evaluation criterion averaged over both the combinations/coalitions and observations/explicands. The data.table also contains the standard deviation of the MSEv values for each explicand (only averaged over the combinations) divided by the square root of the number of explicands.

MSEv_explicand

A data.table with the mean squared error for each explicand, i.e., only averaged over the combinations/coalitions.

MSEv_combination

A data.table with the mean squared error for each combination/coalition, i.e., only averaged over the explicands/observations. The data.table also contains the standard deviation of the MSEv values for each combination divided by the square root of the number of explicands.

Author(s)

Lars Henry Berge Olsen

NorskRegnesentral/shapr documentation built on April 19, 2024, 1:19 p.m.