In pokyah/agrometeoR: Spatial interpolation for Agromet project @ CRA-W

Introduction and context

The aim of the Agromet project is to provide a near real-time hourly gridded datasets of weather parameters at the resolution of 1 km² for the whole region of Wallonia characterized by a quality indicator.

The Agromet project is largely inspired by what the ZEPP has developed in the context of their late blight warning services [@zeuner_use_2007]

Before starting the development of our own service, we decided to submit a survey to our end users and to perform a preliminary benchmark of weather data interpolation facilities developed by other institutions.

This document compiles the useful information we have gathered during our benchmarking and synthetise the main ideas to keep in mind while building our platform.

Literature review

An extensive literature review of weather spatialization techniques has been performed.

Recommanded academic papers

Here is a short selection of the most useful papers regarding the comprehension of meteorological data spatialization for our walloon agricultural context :

• Interpolation methods for climate data literature review - [@sluiter_interpolation_2008]
• Use of geographic information systems in warning services for late blight - [@zeuner_use_2007]
• Decision Support Systems in Agriculture: Administration of Meteorological Data, Use of Geographic Information Systems(GIS) and Validation Methods in Crop Protection Warning Service - [@paolo_decision_2011]
• Spatial interpolation of air pollution measurements using CORINE land cover data - [@janssen_spatial_2008]
• Spatial interpolation of ambient ozone concentrations from sparse monitoring points in Belgium - [@hooyberghs_spatial_2006]
• The developments in spatialization of meteorological and climatological elements - [@tveito_developments_2007]
• Solar irradiance-corrected spatial interpolation of hourly temperature in complex terrain - [@chung_solar_2004]

In the coming months we plan to organize and share our full bibliography.

Reference books

These books focus on the theory relative to the general principles of spatialization techniques :

• Spatial Interpolation for Climate Data - The Use of GIS in Climatology and Meteorology - [@dobesch_spatial_2013]
• Principles of Geographic Information Systems - An introductory textbook - [@huisman_principles_2009]
• Principles Of Geographical Information Systems - [@burrough_principles_2015]
• Using ArcGIS Geostatistical Analyst - [@johnston_using_2001]
• A review of spatial interpolation methods for environmental scientists - [@li_review_2011]
• A Practical Guide to Geostatistical Mapping of Environmental Variables - [@hengl_practical_2009]

European experts in weather data spatialization

Here is a list of european experts in terms of weather spatialization worth following.

| Country | Author | Institution | Publication | |--------- |----------------------- |------------------------------------------------ |----------------------------------------------------------------------------------------------------------------------------------------------- | | Allemagne | T. Zeuner | ZEPP German Crop Protection Services | Use of geographic information systems in warning services for late blight | | Serbie | Milan Kilibarda | University of Belgrade | Spatio-temporal interpolation of daily temperatures for global land areas at 1 km resolution | | Pays-bas | Raymond Sluiter | KNMI | Interpolation methods for climate data - literature Review | | Pays-bas | Tomislav Hengl | ISRIC World Soil Information Institute | R-package Spatial Analyst | | Norwegian | Jean-Marie Lepioufle | Norwegian Meteorological Institute | Recent developments in spatial interpolation of precipitation and temperature at MET Norway | | Grèce | Kostas philippopoulos | University of Reading | Artificial Neural Network Modeling of Relative Humidity and Air Temperature Spatial and Temporal Distributions Over Complex Terrains | | Portugual | Silva Alvaro | Instituto Português do Mar e da Atmosfera | Neural Networks application to spatial interpolation of climate variables | | Slovénie | Luka honzak | Bo Mo LTD | WEATHER SCENARIO APP | | France | Mehdi Sine | Vigicultures par Arvalis - institut du Végétal | VIGICULTURES – An early warning system for crop pest management | | Belgique | Aurore Degré | Faculté Gemboux | Different methods for spatial interpolation of rainfall data for operational hydrology and hydrological modeling at watershed scale: a review | | Pologne | Maciej Kryza | university de Wroclaw

Key learnings from the review

The literature reveals that a lot of spatial interpolation methods have been developed the last decades. These techniques have been borrowed from other fields (e.g. oil prospection) and transposed in the field of meteorology where the comprehension and modelisation of the processes is much more technical due to the complexity and the spatial heterogeneity of weather events.

In such, there is not an out-of-the box recipe to apply to each weather parameter. The choice of the right interpolation method depends of many factors such as the spatial distribution of the weather station network, the topography, the number of stations, local gradients such as global circulation effects, etc.

Moreover, more attention has been ported on the spatialization of climate data rather than hourly meteorological data which is our concern. Therefore, an important phase of testing, benchmarking and tweaking of the processes described here above is required in order to efficiently produce useful and sensible gridded outputs that could be used profitably by agronomical models.

These phase will require a deep knowledge of the principles of these geostatistical spatialization method combined with the development of programming skills required to explore the data and conduct practical analysis.

The exploratory phases needed for the development of an adjusted data analysis technique able to deal with data scarcity should be performed in a way such as the the most simple solutions are evaluated first. Depending of the results of the evaluation of the investigated technique, we will decide if further investigations are required. If no significant extra-value is added by are more complex process, the later will not be retained.

The first results suggest that for the temperature, we will need to apply a correction model to the Pameseb measurments recorded around the daily maximal temperature hours.

Understanding our end-users needs

A web-survey has been submitted to our potential end-users. Its purpose was to insure that the platform integrates the real needs of the future end-users (walloon crop warning system managers and academic research). The results of this survey also serve as a development priority list. The results are available in this report.

Choosing the right software

Among all the available programming languages, we choose R : - Fully open-source and free (like beer and freedom) - large user base and more and more used - R is developed by statisticians for statisticians - R is already used by other institutions implicated in weather data spatialization and internally at CRA-W - Many libraries (packages) cover our needs

| package | purpose | |------------------------------------------------------------------------------------ |--------------------------------------------------------------------------------- | | gstat | Spatial and Spatio-Temporal Geostatistical Modelling, Prediction and Simulation | | meteo | Spatio-Temporal Analysis and Mapping of Meteorological Observations | | sf | Simple Features for R | | raster | Geographic Data Analysis and Modeling | | automap | Automatic interpolation package | | dplyr | A Grammar of Data Manipulation | | rgdal | Bindings for the Geospatial Data Abstraction Library | | ggplot2 | Create Elegant Data Visualisations Using the Grammar of Graphics | | shiny | Web Application Framework for R | | geoR | Analysis of Geostatistical Data | | validate | Data Validation Infrastructure | | mlr | Machine Learning in R | Here is an infography that compares R to python.

Extra investigations

If extra time remains, we could investigate to incorporate crowdsourced datasets. At the present time, both the KNMI and the RMI work on such a process in the context of the WOW experiment initiated by the UK metoffice.

Data dissemination policy

A particular attention will be given to make our data INSPIRE compliant and their origin will be described using the W3C recommandations.

As developers we push for the adoption of an open-data policy as the Community Data License Agreement. However, the final decision regarding the choice of the policy will be taken at higher levels.

Here is a selection of publications in agreement with an open-data approach :

• open-source software in sciences paper
• KNMI data policy
• Eu general principle for open access to scientific publications in Horizon 2020
• Brussels administrations and open data
• Belgian federal state and open data
• Wallonia and open data
• The value of open source
• Open-source et cooperativism

References

pokyah/agrometeoR documentation built on May 26, 2019, 7 p.m.