calc_performance_metrics: Calculates performance metrics of a business-as-usual model

In ubair: Effects of External Conditions on Air Quality

Calculates performance metrics of a business-as-usual model

Description

Model agnostic function to calculate a number of common performance metrics on the reference time window. Uses the true data value and the predictions prediction for this calculation. The coverage is calculated from the columns value, prediction_lower and prediction_upper. Removes dates in the effect and buffer range as the model is not expected to be performing correctly for these times. The incorrectness is precisely what we are using for estimating the effect.

Usage

calc_performance_metrics(predictions, date_effect_start = NULL, buffer = 0)

Arguments

predictions	data.table or data.frame with the following columns date Date of the observation. Needs to be comparable to date_effect_start element. value True observed value of the station prediction Predicted model output for the same time and station as value prediction_lower Lower end of the prediction interval prediction_upper Upper end of the prediction interval
date_effect_start	A date. Start date of the effect that is to be evaluated. The data from this point onwards is disregarded for calculating model performance
buffer	Integer. An additional buffer window before date_effect_start to account for uncertainty in the effect start point. Disregards additional buffer data points for model evaluation

Value

Named vector with performance metrics of the model

ubair documentation built on April 12, 2025, 2:12 a.m.