read_point_forecast: Read point forecasts from FCTABLE files

In andrew-MET/harpIO: IO functions and data wrangling for harp

Read point forecasts from FCTABLE files

Description

Reads point forecasts from an sqlite file produced by read_eps_interpolate or read_det_interpolate. The function generates the file names from the start date, end date, forecast model(s), parameter, lead time etc. supplied as arguments. The members, stations, forecast dates and lead times to be retrieved from the files can also be passed as arguments.

Usage

read_point_forecast(
  dttm,
  fcst_model,
  fcst_type,
  parameter,
  lead_time = seq(0, 48, 3),
  lags = "0s",
  merge_lags = TRUE,
  file_path = ".",
  file_template = "fctable",
  drop_any_na = TRUE,
  stations = NULL,
  members = NULL,
  accumulate = TRUE,
  vertical_coordinate = c(NA_character_, "pressure", "model", "height"),
  get_lat_and_lon = FALSE,
  force_param_name = FALSE,
  start_date = NULL,
  end_date = NULL,
  by = "6h"
)

Arguments

dttm	A vector of date time strings to read. Can be in YYYYMMDD, YYYYMMDDhh, YYYYMMDDhhmm, or YYYYMMDDhhmmss format. Can be numeric or character. seq_dttm can be used to generate a vector of equally spaced date-time strings.
fcst_model	The forecast model to read - this is typically used to construct the file name. Can be a character vector of model names.
fcst_type	The type of forecast to read. Set to "det" for deterministic or "eps" for ensemble.
parameter	The forecast parameter to read. This is usually only used to construct the filename, and in accumumlating precipitation.
lead_time	The lead times to be retrieved. Can be used to construct the file names and to set which lead times are retrieved.
lags	The lags that were used when the forecast was run. If, for example, the FCTABLE files are constructed from lagged model runs the lags must be given here to ensure that the correct file names are generated. If, however, you simply want to add lagged members to a forecast, you should do that using lag_forecast.
merge_lags	A logical that, if set to TRUE (the default), lagged ensemble members will be shifted in time and joined to the parent forecast as derived from start_date and by.
file_path	The path to the data.
file_template	The template for the file names of the files to be read from. This would normally be one of the "fctable_*" templates that can be seen in show_file_templates. Can be a single string, a character vector or list of the same length as fcst_model. If not named, the order of templates is assumed to be the same as in fcst_model. If named, the names must match the entries in fcst_model.
drop_any_na	Set to TRUE (the default) to remove all cases where there is at least one missing value. This ensures that when you come to analyse a forecast, only those forecasts with a full set of ensmeble members / data are read in. For reading lagged ensembles, this is automatically set to FALSE. The cases with at least one missing member are then dropped when the lagged members are created using lag_forecast.
stations	The stations to retrieve forecasts for. This should be a vector of station ID numbers. Set to NULL to retrieve all stations.
members	The members to retrieve if reading an EPS forecast. To select the same members for all forecast models, this should be a numeric vector. For specific members from specific models a named list with each element having the name of the forecast model and containing a a numeric vector. e.g. members = list(eps_model1 = seq(0, 3), eps_model2 = c(2, 3)). For multi model ensembles, each element of this named list should contain another named list with sub model name followed by the desired members, e.g. members = list(eps_model1 = list(sub_model1 = seq(0, 3), sub_model2 = c(2, 3)))
accumulate	TRUE or FALSE. Whether to automatically accumulate parameters based on the accumulation time. Set to FALSE if the data to be read in have already been accumulated.
vertical_coordinate	If upper air for multiple levels are to be read, the vertical coordinate of the data is given here. The default is "pressure", but can also be "model" for model levels, or "height" for height above ground /sea level.
get_lat_and_lon	Logical indicating whether to also extract the latitude and longitude of the point forecasts from the sqlite files.
force_param_name	Logical. Default is FALSE. Force the parameter name in the file template. For upper air data it is assumed that vertical profiles of the data are in the files so the vertical level information is removed when constructing the file name. If you want the full parameter name to be used in the file name set to TRUE.
start_date, end_date, by	The use of start_date, end_date and by is no longer supported. dttm together with seq_dttm should be used to generate equally spaced date-times.

Details

In the case of lagged forecasts, no lagged members are created at this stage, but the lags argument is used to ensure that all of the necessary data for creating lagged members are read in.

Value

A list with an element for each forecast model, or in the case of a multi model ensemble, another list with an element for each sub model. The list elements each contain a data frame with columns for station ID (SID), the forecast initialisation time (fcdate), the lead time (leadtime), the time for which the forecast is valid (validdate), and a column for the forecast with a heading of the model name in the case of a deterministic forecast, or multiple columns with heading names usually of the form <model_name>_mbrXXX, where XXX is the member number, in the case of an ensemble forecast.

Examples

if (requireNamespace("harpData", quietly = TRUE)) {
  # Read data from one deterministic model
  read_point_forecast(
    start_date = 2019021700,
    end_date   = 2019021718,
    fcst_model = "AROME_Arctic_prod",
    fcst_type  = "det",
    parameter  = "S10m",
    by         = "6h",
    file_path  = system.file("FCTABLE/deterministic", package = "harpData")
  )

  # Read data from more than one deterministic model
  read_point_forecast(
    start_date = 2019021700,
    end_date   = 2019021718,
    fcst_model = c("AROME_Arctic_prod", "MEPS_prod"),
    fcst_type  = "det",
    parameter  = "S10m",
    by         = "6h",
    file_path  = system.file("FCTABLE/deterministic", package = "harpData")
  )

  # Reading ensemble data works exactly the same way, but with fcst_type = "eps".
  # Note that all lead times are combined in the same file so
  # file_template = "fctable_eps_all_leads".
  read_point_forecast(
    start_date    = 2019021700,
    end_date      = 2019021718,
    fcst_model    = "MEPS_prod",
    fcst_type     = "eps",
    parameter     = "T2m",
    lead_time     = seq(0, 12, 3),
    by            = "6h",
    file_path     = system.file("FCTABLE/ensemble", package = "harpData"),
    file_template = "fctable_eps_all_leads"
  )

  # You can select which members you want to read
  read_point_forecast(
    start_date    = 2019021700,
    end_date      = 2019021718,
    fcst_model    = "MEPS_prod",
    fcst_type     = "eps",
    parameter     = "T2m",
    lead_time     = seq(0, 12, 3),
    by            = "6h",
    file_path     = system.file("FCTABLE/ensemble", package = "harpData"),
    file_template = "fctable_eps_all_leads",
    members       = c(0, 3, 6)
  )

  # If reading in vertical profiles, you must specify the vertical coordinate
  read_point_forecast(
    start_date          = 2019021700,
    end_date            = 2019021718,
    fcst_model          = "MEPS_prod",
    fcst_type           = "eps",
    parameter           = "T",
    lead_time           = seq(0, 12, 3),
    by                  = "6h",
    file_path           = system.file("FCTABLE/ensemble", package = "harpData"),
    file_template       = "fctable_eps_all_leads",
    members             = c(0, 3, 6),
    vertical_coordinate = "pressure"
  )

  # When reading a lagged ensemble, the lags must be set for times at which
  # lagged members exist. If member numbers are duplicated "_lag" is
  # added to the member name.
  read_point_forecast(
    start_date    = 2019021700,
    end_date      = 2019021718,
    fcst_model    = "CMEPS_prod",
    fcst_type     = "EPS",
    parameter     = "T2m",
    lead_time     = seq(0, 12, 3),
    by            = "6h",
    lags          = paste0(seq(0, 5), "h"),
    file_path     = system.file("FCTABLE/ensemble", package = "harpData"),
    file_template = "fctable_eps_all_leads"
  )

  # If more than one forecast model, and at least one is lagged, lags must
  # be specified for all forecast models as a named list.
  read_point_forecast(
    start_date    = 2019021700,
    end_date      = 2019021718,
    fcst_model    = c("CMEPS_prod", "MEPS_prod"),
    fcst_type     = "EPS",
    parameter     = "T2m",
    lead_time     = seq(0, 12, 3),
    by            = "6h",
    lags          = list(
      CMEPS_prod = paste0(seq(0, 5), "h"),
      MEPS_prod  = "0h"
    ),
    file_path     = system.file("FCTABLE/ensemble", package = "harpData"),
    file_template = "fctable_eps_all_leads"
  )
}

andrew-MET/harpIO documentation built on March 7, 2024, 7:48 p.m.