performance: Compare performance of catch-only methods

In cfree14/datalimited2: More Stock Assessment Methods for Data-limited Fisheries

Compare performance of catch-only methods

Description

Compares the continuous and/or categorical performance of catch-only stock assessment methods. Creates both a plot and table of performance metrics.

Usage

performance(status, methods)

Arguments

status	A matrix or dataframe of continuous (e.g., B/BMSY) or categorical (e.g., under, fully, overexploited) status estimates in which the first column is the true (or data-rich) status estimate and additional columns are status estimates from catch-only methods.
methods	A character vector of the names of the catch-only models being compared, listed in the order they are presented in the status matrix or dataframe.

Details

The continuous performance of each COM is evaluated by measuring each method’s bias, accuracy, and ability to correctly rank or correlate across populations. We measured bias as the median proportional error (MPE) and accuracy as the median absolute proportional error (MAPE). Proportional error is calculated as (θest-θ)/|θ|, where θest and θ represent predicted and “true” (or data-rich stock assessment) B/BMSY values. The ability to correctly rank populations is measured as Spearman’s rank-order correlation between predicted and “true” values.

The categorical performance of each COM is evaluated using both percentage agreement (accuracy) and Cohen’s kappa. Cohen’s kappa measures inter-rate agreement between categorical items and is more robust than simple percentage agreement because it takes into account the probability of agreement occurring by chance alone (Cohen 1968). Although there are no definitive rules for interpreting Cohen’s kappa, general guidelines suggest that values >0.70 are ‘excellent’, 0.4-0.7 are ‘good’, 0.2-0.4 are ‘fair’, and <0.2 are ‘poor’ (Landis & Koch 1977; Fleiss 1973).

Value

A plot and table of continuous or categorical performance metrics.

Examples

bbmsy <- select(preds, bbmsy,  mprm, comsir, sscom, cmsy13, cmsy17, zbrt, ocom, super1)
status <- bbmsy2catg(bbmsy)
methods <- c("mPRM", "COMSIR", "SSCOM", "cMSY-13", "cMSY-17", "zBRT", "OCOM", "Super")
performance(bbmsy, methods)
performance(status, methods)

cfree14/datalimited2 documentation built on Aug. 21, 2023, 2:26 p.m.