RatioRMS: Computes the Ratio Between The RMSE of Two Experiments

In s2dverification: Set of Common Tools for Forecast Verification

Computes the Ratio Between The RMSE of Two Experiments

Description

Calculates the ratio of the RMSE for two forecasts of the same observations.
The ratio RMSE(ens, obs) / RMSE(ens.ref, obs) is output.
The p-value is provided by a two-sided Fischer test.

.RatioRMS provides the same functionality but taking two matrices of ensemble members (ens and ens.ref) as input.

Usage

RatioRMS(var_exp1, var_exp2, var_obs, posRMS = 1, pval = TRUE)

.RatioRMS(exp, exp_ref, obs, pval = TRUE)

Arguments

var_exp1	Array of experimental data 1.
var_exp2	Array of experimental data 2.
var_obs	Array of observations.
posRMS	Dimension along which the RMSE are to be computed = the position of the start dates.
pval	Whether to compute the p-value of Ho : RMSE1/RMSE2 = 1 or not. TRUE by default.
exp	Matrix of experimental data 1.
exp_ref	Matrix of experimental data 2.
obs	Vector of observations.

Value

RatioRMS:
Matrix with the same dimensions as var_exp1/var_exp2/var_obs except along posRMS where the dimension has length 2 if 'pval = TRUE', or 1 otherwise. The dimension of length 2 corresponds to the ratio between the RMSE (RMSE1/RMSE2) and the p-value of the two-sided Fisher test with Ho: RMSE1/RMSE2 = 1.

.RatioRMS:

• ratiormsThe ratio of the RMSE of the two experimental datasets

• p_valThe p-value

Author(s)

History:
0.1 - 2011-11 (V. Guemas) - Original code
1.0 - 2013-09 (N. Manubens) - Formatting to R CRAN
1.1 - 2017-02 (A. Hunter) - Adapted to veriApply()

Examples

# See examples on Load() to understand the first lines in this example
 ## Not run: 
data_path <- system.file('sample_data', package = 's2dverification')
expA <- list(name = 'experiment', path = file.path(data_path,
            'model/$EXP_NAME$/$STORE_FREQ$_mean/$VAR_NAME$_3hourly',
            '$VAR_NAME$_$START_DATE$.nc'))
obsX <- list(name = 'observation', path = file.path(data_path,
            '$OBS_NAME$/$STORE_FREQ$_mean/$VAR_NAME$',
            '$VAR_NAME$_$YEAR$$MONTH$.nc'))

# Now we are ready to use Load().
startDates <- c('19851101', '19901101', '19951101', '20001101', '20051101')
sampleData <- Load('tos', list(expA), list(obsX), startDates,
                  output = 'lonlat', latmin = 27, latmax = 48, 
                  lonmin = -12, lonmax = 40)
 
## End(Not run)
 
# Compute DJF seasonal means and anomalies.
leadtimes_dimension <- 4
initial_month <- 11
mean_start_month <- 12
mean_stop_month <- 2                                
sampleData$mod <- Season(sampleData$mod, leadtimes_dimension, initial_month, 
                        mean_start_month, mean_stop_month)
sampleData$obs <- Season(sampleData$obs, leadtimes_dimension, initial_month, 
                        mean_start_month, mean_stop_month)                              
clim <- Clim(sampleData$mod, sampleData$obs)
ano_exp <- Ano(sampleData$mod, clim$clim_exp)
ano_obs <- Ano(sampleData$obs, clim$clim_obs)
# Generate two experiments with 2 and 1 members from the only experiment 
# available in the sample data. Take only data values for a single forecast
# time step.
ano_exp_1 <- Subset(ano_exp, 'member', c(1, 2))
ano_exp_2 <- Subset(ano_exp, 'member', c(3))
ano_exp_1 <- Subset(ano_exp_1, c('dataset', 'ftime'), list(1, 1), drop = 'selected')
ano_exp_2 <- Subset(ano_exp_2, c('dataset', 'ftime'), list(1, 1), drop = 'selected')
ano_obs <- Subset(ano_obs, c('dataset', 'ftime'), list(1, 1), drop = 'selected')
# Compute ensemble mean and provide as inputs to RatioRMS.
rrms <- RatioRMS(Mean1Dim(ano_exp_1, 1), 
                Mean1Dim(ano_exp_2, 1), 
                Mean1Dim(ano_obs, 1))
# Plot the RatioRMS for the first forecast time step.
PlotEquiMap(rrms[1, , ], sampleData$lon, sampleData$lat, 
           toptitle = 'Ratio RMSE')

# The following example uses veriApply combined with .RatioRMS instead of RatioRMS
 ## Not run: 
require(easyVerification)
# The name of the function has to end in 'ss' in order for veriApply() to 
# detect it as a skill score.
RatioRMSss <- s2dverification:::.RatioRMS
rrms2 <- veriApply("RatioRMSss", ano_exp_1,
                  # see ?veriApply for how to use the 'parallel' option
                  Mean1Dim(ano_obs, 1),
                  ano_exp_2,
                  tdim = 2, ensdim = 1)
 
## End(Not run)

s2dverification documentation built on April 20, 2022, 9:06 a.m.