CDownloadS: Downloading Data from ECMWF Climate Data Store
In ErikKusch/KrigR: Downloading, Aggregating, and Kriging of ECMWF CDS-Data
CDownloadS R Documentation
Downloading Data from ECMWF Climate Data Store
Description
This function is used to obtain data from the Climate Data Store (CDS) hosted by the Copernicus Climate Change Service (C3S). By default, this function breaks down download calls into intervals so as to avoid submitting queries which fail, downloads queried data from ECMWF servers according to user-specification, and fuses the downloaded files together according to user-demands. The actual time to download is dependent on ECMWF download queues. Users need an API key for download staging and accept terms and conditions for the specific queried dataset(s).
Usage
CDownloadS(
Variable = NULL,
CumulVar = FALSE,
DataSet = "reanalysis-era5-land",
Type = NA,
DateStart,
DateStop,
TZone = "UTC",
TResolution = "month",
TStep = 1,
FUN = "mean",
Extent,
Buffer = 0.5,
Dir = getwd(),
FileName,
FileExtension = ".nc",
Compression = 9,
API_User,
API_Key,
TryDown = 10,
TimeOut = 36000,
TChunkSize = 6000,
Cores = 1,
verbose = TRUE,
Keep_Raw = FALSE,
closeConnections = TRUE
)
Arguments
Variable
Character. Desired variable from queried dataset. See Meta.Variables for options per dataset.
CumulVar
Logical. Some ECMWF CDS data is recorded in cumulative steps per hour/month from the 00:00 time mark per day. Setting CumulVar to TRUE converts these into records which represent the total records per hour using the Temporal.Cumul function. Monthly cumulative records express the average daily total value. Setting this argument to TRUE multiplies monthly records by the number of days per the respective month(s) to get to total records instead of average. Default is FALSE. This argument can only be set to TRUE for cumulatively recorded variables. See Meta.Variables for an overview of which variables at recorded cumulatively per dataset.
DataSet
Character. Which dataset to query data from. See currently supported datasets by calling Meta.List.
Type
Either NA or Character. Which kind of sub-type to query per data set. See Meta.QucikFacts for options per dataset.
DateStart
Character. Date ('YYYY-MM-DD HH:SS') at which to start time series of downloaded data.
DateStop
Character. Date ('YYYY-MM-DD HH:SS') at which to stop time series of downloaded data.
TZone
Character. Time zone in which to represent and evaluate time dimension of data. See the output of OlsonNames() for a full overview of supported specifications. Default is UTC.
TResolution
Character. Temporal resolution of final product. 'hour', 'day', 'month', or 'year'.
TStep
Numeric. Which time steps to consider for temporal resolution. For example, specify bi-monthly data records by setting TResolution to 'month' and TStep to 2.
FUN
A raster calculation argument as passed to ‘terra::tapp()'. This controls what kind of data to obtain for temporal aggregates of reanalysis data. Specify ’mean' (default) for mean values, 'min' for minimum values, and 'max' for maximum values, among others.
Extent
Optional, download data according to desired spatial specification. If missing/unspecified, total area of queried data set is used. Can be specified either as a raster object, an sf object, a terra object, or a data.frame. If Extent is a raster or terra object, data will be queried according to rectangular extent thereof. If Extent is an sf (MULTI-)POLYGON object, this will be treated as a shapefile and the output will be cropped and masked to this shapefile. If Extent is a data.frame of geo-referenced point records, it needs to contain Lat and Lon columns around which a buffered shapefile will be created using the Buffer argument.
Buffer
Optional, Numeric. Identifies how big a circular buffer to draw around points if Extent is a data.frame of points. Buffer is expressed as centessimal degrees.
Dir
Character/Directory Pointer. Directory specifying where to download data to.
FileName
Character. A file name for the produced file.
FileExtension
Character. A file extension for the produced file. Supported values are ".nc" (default) and ".tif" (better support for metadata).
Compression
Integer between 1 to 9. Applied to final .nc file that the function writes to hard drive. Same as compression argument in terra::writeCDF(). Ignored if FileExtension = ".tif".
API_User
Character; ECMWF cds user number.
API_Key
Character; ECMWF cds API key.
TryDown
Optional, numeric. Legacy, ignored when querying data from new CDS (https://cds-beta.climate.copernicus.eu/; this happens when the package version of ecmwfr is >= 2.0.0). How often to attempt the download of each individual file that the function queries from the CDS. This is to circumvent having to restart the entire function when encountering connectivity issues.
TimeOut
Numeric. Legacy, ignored when querying data from new CDS (https://cds-beta.climate.copernicus.eu/; this happens when the package version of ecmwfr is >= 2.0.0). The timeout for each download in seconds. Default 36000 seconds (10 hours).
TChunkSize
Numeric. Number of layers to bundle in each individual download. Default is 6000 to adhere to most restrictive CDS limits: https://cds.climate.copernicus.eu/live/limits.
Cores
Numeric. How many cores to use when carrying out temporal aggregation. Default is 1.
verbose
Logical. Whether to print/message function progress in console or not.
Keep_Raw
Logical. Whether to retain raw downloaded data or not. Default is FALSE.
closeConnections
Logical. Whether to close all connections at the end of function execution. When executing this function often after another, this can be very useful to avoid errors.
Value
A SpatRaster object containing the downloaded, cropped/masked, and subsequently temporally aggregated data, and a file (either .nc or .tif) in the specified directory.
The SpatRaster contains metadata/attributes as a named vector that can be retrieved with terra::metags(...):
Citation - A string which to use for in-line citation of the data product obtained with CDownloadS.
KrigRCall.X - Arguments passed to the CDownloadS function that produced the file (API credentials are omitted from these metadata).
ATTENTION: If data is loaded again from disk at a later point with a different function, take note that the time zone will have to be set anew and existing time parameters in the .nc contents will need to be converted to the desired time zone. Likewise, citation and KrigR-call metadata will not be loaded properly from a .nc when loading data through a different function. CDownloads() handles these .nc specific issues when loading .nc files created previously with CDownloadS from disk.
See Also
Meta.List, Meta.Variables, Meta.QuickFacts, Plot.SpatRast.
Examples
## Not run:
## Raw data for one month of full globe
RawGlobe_rast <- CDownloadS(
Variable = "2m_temperature",
DataSet = "reanalysis-era5-land-monthly-means",
Type = "monthly_averaged_reanalysis",
# time-window, default set to range of dataset-type
DateStart = "1995-01-01 00:00",
DateStop = "1995-01-01 23:00",
TZone = "CET",
# temporal aggregation
TResolution = "month",
TStep = 1,
# file storing
FileName = "RawGlobe",
# API credentials
API_User = API_User,
API_Key = API_Key
)
Plot.SpatRast(RawGlobe_rast)
## Monthly air temperature aggregated to bi-annual maximum by SpatRaster
CDS_rast <- terra::rast(system.file("extdata", "CentralNorway.nc", package = "KrigR"))
BiAnnAirTemp_rast <- CDownloadS(
Variable = "2m_temperature",
DataSet = "reanalysis-era5-land-monthly-means",
Type = "monthly_averaged_reanalysis",
# time-window, default set to range of dataset-type
DateStart = "1995-01-01 00:00",
DateStop = "1996-12-31 23:00",
TZone = "EET",
# temporal aggregation
TResolution = "year",
TStep = 2,
# spatial
Extent = CDS_rast,
# file storing
FileName = "BiAnnAirTemp",
# API credentials
API_User = API_User,
API_Key = API_Key
)
Plot.SpatRast(BiAnnAirTemp_rast)
## Hourly back-calculated precipitation aggregated to daily averages by shapefiles
data("Jotunheimen_poly")
Jotunheimen_poly
DailyBackCPrecip_rast <- CDownloadS(
Variable = "total_precipitation",
CumulVar = TRUE,
# time-window, default set to range of dataset-type
DateStart = "1995-01-01 00:00",
DateStop = "1995-01-03 23:00",
TZone = "CET",
# temporal aggregation
TResolution = "day",
# spatial
Extent = Jotunheimen_poly,
# file storing
FileName = "DailyBackCPrecip",
# API credentials
API_User = API_User,
API_Key = API_Key
)
Plot.SpatRast(DailyBackCPrecip_rast, SF = Jotunheimen_poly, Legend = "Precipitation [m]", COL = rev(viridis::cividis(100)))
## 6-hourly ensemble member spread sum for air temperature by buffered points
data("Mountains_df")
EnsembleSpreadSum6hour_rast <- CDownloadS(
Variable = "2m_temperature",
DataSet = "reanalysis-era5-single-levels",
Type = "ensemble_spread",
# time-window, default set to range of dataset-type
DateStart = "1995-01-01 00:00:00",
DateStop = "1995-01-01 21:00:00",
TZone = "UTC",
# temporal aggregation
TResolution = "hour",
TStep = 6,
FUN = sum,
# spatial
Extent = Mountains_df,
Buffer = 0.2,
# file storing
FileName = "EnsembleSpreadSum6hour",
FileExtension = ".tif",
# API credentials
API_User = API_User,
API_Key = API_Key,
Keep_Raw = TRUE
)
Plot.SpatRast(EnsembleSpreadSum6hour_rast, Legend = "Air Temperature Uncertainty [K]")
## End(Not run)
ErikKusch/KrigR documentation built on Feb. 17, 2025, 2:09 p.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| CDownloadS | R Documentation |
Downloading Data from ECMWF Climate Data Store
Description
This function is used to obtain data from the Climate Data Store (CDS) hosted by the Copernicus Climate Change Service (C3S). By default, this function breaks down download calls into intervals so as to avoid submitting queries which fail, downloads queried data from ECMWF servers according to user-specification, and fuses the downloaded files together according to user-demands. The actual time to download is dependent on ECMWF download queues. Users need an API key for download staging and accept terms and conditions for the specific queried dataset(s).
Usage
CDownloadS(
Variable = NULL,
CumulVar = FALSE,
DataSet = "reanalysis-era5-land",
Type = NA,
DateStart,
DateStop,
TZone = "UTC",
TResolution = "month",
TStep = 1,
FUN = "mean",
Extent,
Buffer = 0.5,
Dir = getwd(),
FileName,
FileExtension = ".nc",
Compression = 9,
API_User,
API_Key,
TryDown = 10,
TimeOut = 36000,
TChunkSize = 6000,
Cores = 1,
verbose = TRUE,
Keep_Raw = FALSE,
closeConnections = TRUE
)
Arguments
Variable |
Character. Desired variable from queried dataset. See |
CumulVar |
Logical. Some ECMWF CDS data is recorded in cumulative steps per hour/month from the 00:00 time mark per day. Setting CumulVar to TRUE converts these into records which represent the total records per hour using the |
DataSet |
Character. Which dataset to query data from. See currently supported datasets by calling |
Type |
Either NA or Character. Which kind of sub-type to query per data set. See |
DateStart |
Character. Date ('YYYY-MM-DD HH:SS') at which to start time series of downloaded data. |
DateStop |
Character. Date ('YYYY-MM-DD HH:SS') at which to stop time series of downloaded data. |
TZone |
Character. Time zone in which to represent and evaluate time dimension of data. See the output of OlsonNames() for a full overview of supported specifications. Default is UTC. |
TResolution |
Character. Temporal resolution of final product. 'hour', 'day', 'month', or 'year'. |
TStep |
Numeric. Which time steps to consider for temporal resolution. For example, specify bi-monthly data records by setting TResolution to 'month' and TStep to 2. |
FUN |
A raster calculation argument as passed to ‘terra::tapp()'. This controls what kind of data to obtain for temporal aggregates of reanalysis data. Specify ’mean' (default) for mean values, 'min' for minimum values, and 'max' for maximum values, among others. |
Extent |
Optional, download data according to desired spatial specification. If missing/unspecified, total area of queried data set is used. Can be specified either as a raster object, an sf object, a terra object, or a data.frame. If Extent is a raster or terra object, data will be queried according to rectangular extent thereof. If Extent is an sf (MULTI-)POLYGON object, this will be treated as a shapefile and the output will be cropped and masked to this shapefile. If Extent is a data.frame of geo-referenced point records, it needs to contain Lat and Lon columns around which a buffered shapefile will be created using the Buffer argument. |
Buffer |
Optional, Numeric. Identifies how big a circular buffer to draw around points if Extent is a data.frame of points. Buffer is expressed as centessimal degrees. |
Dir |
Character/Directory Pointer. Directory specifying where to download data to. |
FileName |
Character. A file name for the produced file. |
FileExtension |
Character. A file extension for the produced file. Supported values are ".nc" (default) and ".tif" (better support for metadata). |
Compression |
Integer between 1 to 9. Applied to final .nc file that the function writes to hard drive. Same as compression argument in terra::writeCDF(). Ignored if FileExtension = ".tif". |
API_User |
Character; ECMWF cds user number. |
API_Key |
Character; ECMWF cds API key. |
TryDown |
Optional, numeric. Legacy, ignored when querying data from new CDS (https://cds-beta.climate.copernicus.eu/; this happens when the package version of ecmwfr is >= 2.0.0). How often to attempt the download of each individual file that the function queries from the CDS. This is to circumvent having to restart the entire function when encountering connectivity issues. |
TimeOut |
Numeric. Legacy, ignored when querying data from new CDS (https://cds-beta.climate.copernicus.eu/; this happens when the package version of ecmwfr is >= 2.0.0). The timeout for each download in seconds. Default 36000 seconds (10 hours). |
TChunkSize |
Numeric. Number of layers to bundle in each individual download. Default is 6000 to adhere to most restrictive CDS limits: https://cds.climate.copernicus.eu/live/limits. |
Cores |
Numeric. How many cores to use when carrying out temporal aggregation. Default is 1. |
verbose |
Logical. Whether to print/message function progress in console or not. |
Keep_Raw |
Logical. Whether to retain raw downloaded data or not. Default is FALSE. |
closeConnections |
Logical. Whether to close all connections at the end of function execution. When executing this function often after another, this can be very useful to avoid errors. |
Value
A SpatRaster object containing the downloaded, cropped/masked, and subsequently temporally aggregated data, and a file (either .nc or .tif) in the specified directory.
The SpatRaster contains metadata/attributes as a named vector that can be retrieved with terra::metags(...):
Citation - A string which to use for in-line citation of the data product obtained with CDownloadS.
KrigRCall.X - Arguments passed to the CDownloadS function that produced the file (API credentials are omitted from these metadata).
ATTENTION: If data is loaded again from disk at a later point with a different function, take note that the time zone will have to be set anew and existing time parameters in the .nc contents will need to be converted to the desired time zone. Likewise, citation and KrigR-call metadata will not be loaded properly from a .nc when loading data through a different function. CDownloads() handles these .nc specific issues when loading .nc files created previously with CDownloadS from disk.
See Also
Meta.List, Meta.Variables, Meta.QuickFacts, Plot.SpatRast.
Examples
## Not run:
## Raw data for one month of full globe
RawGlobe_rast <- CDownloadS(
Variable = "2m_temperature",
DataSet = "reanalysis-era5-land-monthly-means",
Type = "monthly_averaged_reanalysis",
# time-window, default set to range of dataset-type
DateStart = "1995-01-01 00:00",
DateStop = "1995-01-01 23:00",
TZone = "CET",
# temporal aggregation
TResolution = "month",
TStep = 1,
# file storing
FileName = "RawGlobe",
# API credentials
API_User = API_User,
API_Key = API_Key
)
Plot.SpatRast(RawGlobe_rast)
## Monthly air temperature aggregated to bi-annual maximum by SpatRaster
CDS_rast <- terra::rast(system.file("extdata", "CentralNorway.nc", package = "KrigR"))
BiAnnAirTemp_rast <- CDownloadS(
Variable = "2m_temperature",
DataSet = "reanalysis-era5-land-monthly-means",
Type = "monthly_averaged_reanalysis",
# time-window, default set to range of dataset-type
DateStart = "1995-01-01 00:00",
DateStop = "1996-12-31 23:00",
TZone = "EET",
# temporal aggregation
TResolution = "year",
TStep = 2,
# spatial
Extent = CDS_rast,
# file storing
FileName = "BiAnnAirTemp",
# API credentials
API_User = API_User,
API_Key = API_Key
)
Plot.SpatRast(BiAnnAirTemp_rast)
## Hourly back-calculated precipitation aggregated to daily averages by shapefiles
data("Jotunheimen_poly")
Jotunheimen_poly
DailyBackCPrecip_rast <- CDownloadS(
Variable = "total_precipitation",
CumulVar = TRUE,
# time-window, default set to range of dataset-type
DateStart = "1995-01-01 00:00",
DateStop = "1995-01-03 23:00",
TZone = "CET",
# temporal aggregation
TResolution = "day",
# spatial
Extent = Jotunheimen_poly,
# file storing
FileName = "DailyBackCPrecip",
# API credentials
API_User = API_User,
API_Key = API_Key
)
Plot.SpatRast(DailyBackCPrecip_rast, SF = Jotunheimen_poly, Legend = "Precipitation [m]", COL = rev(viridis::cividis(100)))
## 6-hourly ensemble member spread sum for air temperature by buffered points
data("Mountains_df")
EnsembleSpreadSum6hour_rast <- CDownloadS(
Variable = "2m_temperature",
DataSet = "reanalysis-era5-single-levels",
Type = "ensemble_spread",
# time-window, default set to range of dataset-type
DateStart = "1995-01-01 00:00:00",
DateStop = "1995-01-01 21:00:00",
TZone = "UTC",
# temporal aggregation
TResolution = "hour",
TStep = 6,
FUN = sum,
# spatial
Extent = Mountains_df,
Buffer = 0.2,
# file storing
FileName = "EnsembleSpreadSum6hour",
FileExtension = ".tif",
# API credentials
API_User = API_User,
API_Key = API_Key,
Keep_Raw = TRUE
)
Plot.SpatRast(EnsembleSpreadSum6hour_rast, Legend = "Air Temperature Uncertainty [K]")
## End(Not run)
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.