title: "North Sea and Wadden Sea model validation database at Deltares" output: html_document: df_print: paged
Data from different sources are retrieved, harmonized and stored locally in a PostGreSQL/PostGIS database. This is done by scripting, so that it is easy to update. Data are then made available via a Geoserver for easy retrieval by users. For use in validation scripts, the data can be queried and downloaded in a ready-to-use form via scripting in Matlab, Python or R. This document describes this process.
NWDM includes North Sea observation data intended for model validation.
typeName = "NWDM:data_owner"
url = create_url(cql_list = NULL, typeName = typeName,
maxFeatures = NULL, outputFormat = "application/json",
username = nwdm$username, password = nwdm$password)
st_read(url, quiet = T) %>% st_drop_geometry() %>%
select(id) %>% mutate(`data owner` = gsub("data_owner.", "", id)) %>%
select(-id) %>%
knitr::kable(caption = "Owners of data sources included in NWDM")
Table: (#tab:unnamed-chunk-2)Owners of data sources included in NWDM
|data owner | |:--------------------| |PML | |NIOZ | |VLIZ | |Schlesswig-Holstein | |RBINS | |Danish MSFD database | |CEFAS | |IFREMER | |IMR | |MSS | |SMHI | |NLWKN | |SOCAT | |Rijkswaterstaat | |Macovei | |HZG | |EMODnet |
All data are harmonized to BODC vocabulary terms.
Link to BODC vocabs
The database uses parameter codes that were standardized by BODC (Britisch Oceanographic Data Center). Due to historical reasons there are two different parameter codes that the user needs to be aware of: P01 and P35. P01 parameter codes are very descriptive and very extensive and they are harmonized (to a certain extent) in the P35 parameter codes. Using the SeaDataNet website, one can search for the parameter codes available in both P01 and P35.
PO1: https://vocab.seadatanet.org/v_bodc_vocab_v2/search.asp?lib=P01
P35: https://vocab.seadatanet.org/v_bodc_vocab_v2/search.asp?lib=P35
Within the database P01 parameter codes can be accessed via the column "parameter_code", while P35 parameter codes are summarized in the column "p35code".
The Geoserver provides a Web Feature Service (WFS). The service is called by a URL. In the URL, specifications and criteria can be included.
Geoserver can deliver different formats, e.g. csv text. This is shown in the next examples. Calling the URL starts the download of this csv file. The file can be downloaded to your local machine, or read directly into an object in your favourite scripting language
Currently, the following tables are available in the Geoserver:
knitr::include_graphics("c:/repos_checkouts/Wozep/NWDM/doc/nwdm layers.svg")
(\#fig:databaseOverview)Overview of all data sources, and Geoserver WFS layers (bold).
Below is a map with all locations (excluding EMODnet and Ferrybox data) currently in the database. Data is read into an r dataframe using read_csv from the readr (tidyverse) package, converted to a spatial object, transformed to a suitable coordinate reference system for interactive plotting, and plotted using the leaflet library.
typeName = "NWDM:location"
url = create_url(cql_list = NULL, typeName = typeName,
maxFeatures = NULL, outputFormat = "application/json",
username = nwdm$username, password = nwdm$password)
locations <- st_read(url, quiet = T)
locations %>%
arrange(-number_of_observations) %>%
filter(number_of_observations > 500) %>%
leaflet::leaflet() %>%
addTiles() %>%
addCircleMarkers(label = ~location_code, radius = 1)
(\#fig:example1-1)Example of nitrate winter concentrations in time over the whole North Sea.
(\#fig:example1-2)Example of nitrate winter concentrations in time over the whole North Sea.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.