Open Source Components as Building Blocks for Integrated Hydrological Models
OpenStreams provides the building blocks that make up integrated hydrological models. These blocks can be complete models but also parts of models that can be used to build other models. It does not aim to be a complete framework for building and connecting models, that task is left to layers (such as OpenMI) that can be build on top of the component or collections of components.
The main goal of the OpenStreams is re-use of models (components) in different frameworks and settings, thus reducing the effort in maintaining model code. For Deltares this implies that components can be used in Deltashell, Delft-FEWS, OpenDA, RTC-Tool and experimental models written in python from a single source. This does mean that for each framework (e.g. Deltashell) a (thin) layer is needed to link the models to the framework. To be able to make this happen each component must expose it's functionality in an API that allows the framework in which it is to be wrapped to completely control the model.
Secondly, the OpenStreams project performs a number of cases (applications) that demonstrate the use of the components in projects and existing pieces of software. Here we also look at language barriers as the components we use are written in different programming languages. Within at least one of the demonstration cases we will investigate the effect of the (usually simple) numerical solutions choosen in conceptual hydrological models vs more robust solutions.
In general we expect that the components will not be used independently by an end-user (other than researchers interested in model development) but several combinations of components will be distributed as models or model packages within a framework.
Some of the expected impacts of the project are:
- better/more re-use (parts of) hydrological components while maintaining each component at one place only
- demonstration of the functionality of combined components by executing case studies
- better facilities for the research on hydrological model concepts
- better numerical solutions of (simple) conceptual hydrological models
Specific Objectives, models and languages
OpenStreams aims to be a collection of (distributed) hydrological models and model components. In the end we come to a set of components that can work independently (e.g. through a command line interface) but can also be linked together through industry standards (such OpenMI) and more closely linked interfaces if needed. The system combines hydrological software written in different languages (Python with PCRaster extensions, C++, Fortran, C#) and links these together a separate (thin) layer that implements a standard interface. The following components are presently being developed or adjusted:
- RTC-Tools - real time control and simple water balance models
- A generic framework for making water balance models (e.g. bucket models)
- C# wrappers for cPython (wflow) to connect to OpenMI and Deltashell
- java/jepp wrapper to connect cPython (wflow) to OpenDA
The list above will be extended with other concepts; one of the first that will be investigated is the Representative Elementary Watershed model (REW).
Linking components in OpenStreams
There are a number of frameworks available for linking (hydrological) models (for example OMS: http://www.javaforge.com/project/oms, OpenMI: http://www.openmi.org/, CSDMS: http://csdms.colorado.edu, hydromad: http://hydromad.catchment.org/, Deltashell, Open source integrated modeling environment Delta Shell, EGU2012-13183). In OpenStreams we do not want to build another framework, instead we try to buildup a set of components that can be used with different frameworks to maximize the reuse of the components. For this to succeed it is crucial that the API provided by the components provides an interface that allows a high level of control over the model. The amount of detail should be more than that needed by the different frameworks. Figure 1 shows how models that provide a low level API can be linked using wrappers to different frameworks such as OpenMI for linking models and data assimilation and calibration using OpenDA (http://www.openda.org, EGU2012-4039, this session). In addition, very close integration using the native API remains possible.
Figure 1: Diagram showing two models that provide a low level API.
A key feature of OpenStreams is that the components can also be used easily by researchers using a simple command-line interface or by directly interfacing with the model API. This is also one of the reasons we have chosen to also include python as a supported langue. Python is easy to learn and also comes with a number of packages that greatly simplify scientific computing.
Cases and applications
The following cases are part of the OpenStreams project:
- Integrated model for the River rhine catchment
- Testing two distributed hydrological model concepts in the River Rhine
- Rapid flood risk assements using a distributed hydrological model and freely available data
- Nutrient load calculations for Johor strait using a distributed hydrological model in combinations with Delwaq
Discussion and Future work
- This page describes preliminary work on building integrated hydrological models based on (open) components. Work has started on extending the Rhine test-case to include the Dutch Delta area. This work will result in a model that covers whole Rhine basin from the Alps to the North-Sea.
- The current PCRaster/Python based models include a flaw that exists in many hydrological models: the differential equations describing the stores are solved explicitly. Future work in OpenStreams will include versions of the models that use proper numerical solutions. This is especially important when combining models at the timestep level and when using automated calibration frameworks such as those found in OpenDA.
- OpenStreams is starting-up at this moment. Researchers and institutes interested in contributing are very welcome!
Gennadii Donchyts, Bert Jagers (2010), DeltaShell - an open modelling environment, International Environmental Modelling and Software Society (iEMSs) 2010 International Congress on Environmental Modelling and Software
Karssenberg, D., Schmitz, O., Salamon, P., De Jong, K., & Bierkens, M. F. P. (2010). A software framework for construction of process-based stochastic spatio-temporal models and data assimilation. Environmental Modelling & Software, 25, 489-502. doi:http://dx.doi.org/10.1016/j.envsoft.2009.10.004
Sutanudjaja, E.H., van Beek, L.P.H., de Jong, S.M., van Geer, F.C., Bierkens, M.F.P., 2011. Large-scale groundwater modeling using global datasets: a test case for the Rhine-Meuse basin. Hydrol. Earth Syst. Sci. 15, 2913--2935.
wflow - PCRaster-Python based distributed hydrological models, http://publicwiki.deltares.nl/display/OpenS/wflow+-+PCRaster-Python+based+distributed+hydrological+models
Contact and further information
Related information: oss.deltares.nl, oss.deltares.nl/web/rtc-tools, www.pcraster.eu
Deltares is an independent, institute for applied research in the field of water, subsurface and infrastructure. Throughout the world, we work on smart solutions, innovations and applications for people, environment and society. Our main focus is on deltas, coastal regions and river basins. Managing these densely populated and vulnerable areas is complex, which is why we work closely with governments, businesses, other research institutes and universities at home and abroad