predictor_assessment: Model quality assessment

In VEZY/sticRs: A Package to Set, Manage and Analyze STICS Simulations https://vezy.github.io/sticRs/

Model quality assessment

Description

Provide several metrics to assess the quality of the predictions of a model (see note) against observations.

Usage

R2(sim, obs, na.action = stats::na.omit)

SS_res(sim, obs, na.rm = T)

RMSE(sim, obs, na.rm = T)

nRMSE(sim, obs, na.rm = T)

MAE(sim, obs, na.rm = T)

ABS(sim, obs, na.rm = T)

MSE(sim, obs, na.rm = T)

EF(sim, obs, na.rm = T)

NSE(sim, obs, na.rm = T)

Bias(sim, obs, na.rm = T)

MAPE(sim, obs, na.rm = T)

FVU(sim, obs, na.rm = T)

RME(sim, obs, na.rm = T)

Arguments

sim	Simulated values
obs	Observed values
na.action	A function which indicates what should happen when the data contain NAs.
na.rm	Boolean. Remove NA values if TRUE (default)

Details

The statistics for model quality can differ between sources. Here is a short description of each statistic and its equation (see html version for LATEX):

• R2(): coefficient of determination, computed using stats::lm() on obs~sim.

• SS_res(): residual sum of squares (see notes).

• RMSE(): Root Mean Squared Error, computed as

  RMSE = \sqrt{\frac{\sum_1^n(\hat{y_i}-y_i)^2}{n}}

• NSE(): Nash-Sutcliffe Efficiency, alias of EF, provided for user convenience.

• nRMSE(): Normalized Root Mean Squared Error, also denoted as CV(RMSE), and computed as:

  nRMSE = \frac{RMSE}{\hat{y}}\cdot100

• MAE(): Mean Absolute Error, computed as:

  MAE = \frac{\sum_1^n(\left|\hat{y_i}-y_i\right|)}{n}

• ABS(): Mean Absolute Bias, which is an alias of MAE()

• FVU(): Fraction of variance unexplained, computed as:

  FVU = \frac{SS_{res}}{SS_{tot}}

• MSE(): Mean squared Error, computed as:

  MSE = \frac{1}{n}\sum_{i=1}^n(Y_i-\hat{Y_i})^2

• EF(): Model efficiency, also called Nash-Sutcliffe efficiency (NSE). This statistic is related to the FVU as EF= 1-FVU. It is also related to the R^2 because they share the same equation, except SStot is applied relative to the identity function (i.e. 1:1 line) instead of the regression line. It is computed as:

  EF = 1-\frac{SS_{res}}{SS_{tot}}

• Bias(): Modelling bias, simply computed as:

  Bias = \frac{\sum_1^n(\hat{y_i}-y_i)}{n}

• MAPE(): Mean Absolute Percent Error, computed as:

  MAPE = \frac{\sum_1^n(\frac{\left|\hat{y_i}-y_i\right|}{y_i})}{n}

• RME(): Relative mean error (\

  RME = \frac{\sum_1^n(\frac{\hat{y_i}-y_i}{y_i})}{n}

Value

A statistic depending on the function used.

Note

SS_{res} is the residual sum of squares and SS_{tot} the total sum of squares. They are computed as:

SS_{res} = \sum_{i=1}^n (y_i - \hat{y_i})^2

SS_{tot} = \sum_{i=1}^{n}\left(y_{i}-\bar{y}\right)^2

Also, it should be noted that y_i refers to the observed values and \hat{y_i} to the predicted values, and \bar{y} to the mean value of observations.

See Also

This function was inspired from the evaluate() function from the SticsEvalR package. This function is used by stics_eval()

Examples

library(sticRs)
sim= rnorm(n = 5,mean = 1,sd = 1)
obs= rnorm(n = 5,mean = 1,sd = 1)
RMSE(sim,obs)

VEZY/sticRs documentation built on Oct. 26, 2023, 7:37 a.m.