evaluateModelFit: evaluateModelFit

In Hmsc: Hierarchical Model of Species Communities

evaluateModelFit

Description

Computes measures of model fit for a Hmsc model

Usage

evaluateModelFit(hM, predY)

Arguments

hM	a fitted Hmsc model object
predY	array of predictions, typically posterior sample

Details

All measures of model fit are based on comparing the posterior predictive distribution (predY)) to the observed values (hM$Y). The predicted distribution is first summarized to a single matrix of predicted values by taking the posterior mean (for normal and probit models) or posterior median (for Poisson models). All measures of model fit are given as vectors with one value for each species.

The kinds of measures of model fit depend on the type of response variable. For all types of response variables, root-mean-square error (RMSE) between predicted and observed values is computed. For normal models, R2 is computed as squared pearson correlation between observed and predicted values, times the sign of the correlation. For probit models, Tjur R2 and AUC are computed. For Poisson models, a pseudo-R2 is computed as squared spearman correlation between observed and predicted values, times the sign of the correlation (SR2). For Poisson models, the observed and predicted data are also truncated to occurrences (presence-absences), for which the same measures are given as for the probit models (O.RMSE, O.AUC and O.TjurR2). For Poisson models, the observed and predicted data are also subsetted to conditional on presence, for which the root-mean-square error and pseudo-R2 based on squared spearman correlation are computed (C.RMSE, C.SR2).

The measures O.RMSE, O.AUC, O.TjurR2, C.RMSE and C.SR2 can be computed only if the option expected=FALSE has been used when making the predictions

If the model includes a mixture of response variable types, the resulting measures of model fit contain NA's for those response variables for which they cannot be computed.

Value

a list of measures of model fit

Examples

# Evaluate model fit
preds = computePredictedValues(TD$m)
MF = evaluateModelFit(hM=TD$m, predY=preds)

# Evaluate model performance based on cross validation: this will be slow
## Not run: 
partition = createPartition(TD$m, nfolds = 2)
predsCV1 = computePredictedValues(TD$m, partition=partition)
MF = evaluateModelFit(hM=TD$m, predY=predsCV1)

## End(Not run)

Hmsc documentation built on Aug. 11, 2022, 5:11 p.m.