In KWB-R/fakin.doc: Does not matter.

Other Projects {#other-projects}

Here we are just suming up the data workflows and created tools in (old) KWB projects, but -- in contrast to the case studies -- the workflows and tools were not tested explicitly within the FAKIN project.

The goal of this chapter is to enlarge the internal knowledge base at KWB about what and how data challenges have already been successfully solved in the past.

Spree2011 (2007)

Used data by source (data formats in parentheses)

• KWB:
  • water level and discharge at one monitoring site (Text/CSV)
  • rain (Text/CSV)
• BWB:
  • pumping rates in the pumping stations (Excel)
  • water levels in the pumping stations (Excel)
  • rain at some gauges near the monitoring site (Excel)

Tasks and methods by topic

• Dry-weather and wet-weather calibration of a sewer network model (Infoworks)

  • InfoWorks: Creating rain input files
  • InfoWorks: Creating RTC input files

Questions that arose:

• Where to store presentations (trainee vs. employee)?
• Where to store the raw data (personal drive of the trainee)?
• How does Infoworks interpret timestamps, how do BWB provide timestamps? -> metadata

MIACSO (2009) {#miacso}

Monitoring

• sites: one site in the sewer (monitoring container), more sites in the river
• variables: water quantity and quality
• devices: online sensors

Modelling

• Sewerage: Infoworks
• River hydraulics: Hydrax
• River quality: QSim

Data storage

• High amount of data -> extra server: moby
• We put some effort in planning good folder structures for the data. Nevertheless the structure at the end of the project is not as clean as it was planned.
• Data that we received from project partners was stored in Daten/EXTERN.
• Raw data was stored in a folder Daten/RAW which was write-protected and required a special user-account for storing new data.

Daten/
  ACCESS/ # MS Access databases, containing raw data
  EXTERN/ # External data (by organisation)
  META/   # MS Access databases, containing metadata 
          # (about calibration, maintenane, sites, variables)
  RAW/    # Text files containing raw data, from KWB-own devices only, by site

Metadata

• many devices in the container -> meta data about device cleaning and maintenance important -> tool: META_Maintenance.mdb

Methods and Tools

• We imported most of the data from text files into MS Access databases in -> tool: MiaCsoRawImport.mdb
• We calibrated the sensors offline by using SQL queries to provide calibrated data from raw data -> tool: MiaCsoMetaCalibControl.mdb
• We used SQL queries to perform data processing -> tool: MiaCsoStatAnalysis.mdb
• Data validation (outlier detection) was done in a two step procedure:
  1. Automatic preselection using MS Access tool MiaCsoStatAnalysis.mdb
  2. Manual selection using self-developed graphical tool in Origin

Developed Tools:

• MS Access Applications
  • MetaMaint.mdb: Monitoring Metadata Management
  • MiaCsoRawImport.mdb: Text File Import to MS Access
  • MiaCsoStatAnalysis.mdb (project deliverable): Definition and automatic execution of sequences of SQL queries
• Origin extension to interactively select and store outliers graphically
• R packages
  • kwb.mia.evalCritO2 (project deliverable): graphical evaluation of critical oxygen conditions in the river
  • kwb.mia.iw: Calculation of file sizes of InfoWorks result csv-files exported from InfoWorks.
  • kwb.miacso: functions used in MIA-CSO, for example for plotting data availabilities.

KURAS {#kuras}

Developed Tools:

• Frontend for KURAS Database of Rainwater Management Measures: KURAS_DB_Acc2003_hs.mdb

• R package kwb.kuras: Interface to KURAS database

OGRE {#ogre}

• Decision to use CUAHSI Community Observations Data Model (ODM)
• R script to import lab data from Excel to MS Access database implementing ODM

Developed Tools:

• R packages
  • kwb.ogre
  • kwb.ogre.model
  • kwb.odm
  • kwb.odmx

Flusshygiene {#flusshygiene}

• Adaptation of free online monitoring data visualisation HydroServerLite
• Reusage of lab data import script developed in OGRE

DEMEAU

Developed R packages for groundwater modelling:

• kwb.hantush r citep(manual["Rustler_2016a"])

• kwb.vs2dh

• kwb.demeau

DEMOWARE

R package kwb.qmra r citep(manual["Rustler_2016b"]) is a generic QMRA (Quantitative Microbiological Risk Assessment) calculation engine assessing the performance of water supply systems.

It was successfully applicated for the Old Ford wasterwater treatment plant r citep(manual[c("Krausetal_2016", "Rustler_2016c")]). A detailed documentation for the R package and its usage is available online.

OPTIWELLS {#optiwells}

Created R packages:

• kwb.wtaq r citep(manual["Sonnenberg_2016"]): groundwater modelling, e.g. for assessing the impact of production well (e.g. well diameter, pumping rates) and aquifer characteristics (e.g. hydraulic conductivity) on the resulting drawdawn. A detailled tutorial is available online,

• kwb.epanet: wrapper for (pressurised)pipe network simulation model EPANET

RWE {#rwe}

(Semi)automated creation of a complex groundwater simulation model with MODFLOW-2005 for the project Maxflow. The developed model consisted of up to three model layers with up to 1000 abstraction wells with temporal-spatial varying pumping rates within the simulation period. For automating the model generation the Python package flopy v3.2.5 r citep(c("10.1111/gwat.12413.", "10.5066/F7BK19FH")) was used with some minor modifications by KWB (see here). Github was used as version control software for tracking changes in the code and each model scenario was stored in its Github branch (for details, see here).

KWB-R/fakin.doc documentation built on Sept. 27, 2019, 9:53 p.m.