Over many years Delft University of Technology and Deltares, together developed OpenEarth as a free and open source alternative to the project-by-project and institution-by-institution approaches to deal with data, models and tools (e.g Van Koningsveld et al. 2004). OpenEarth at its most abstract level represents the philosophy that data, models and tools should flow as freely and openly as possible across the artificial boundaries of projects and organizations (or at least departments). Put in practice OpenEarth exists only because there is a robust user community that works according to this philosophy (a bottom up approach). In its most concrete and operational form OpenEarth facilitates collaboration within its user community by providing an open ICT infrastructure, built from the best available open source components, in combination with a well-defined workflow, described in open protocols based as much as possible on widely accepted international standards.

OpenEarth as a whole (philosophy, user community, infrastructure and workflow) is the first comprehensive approach to handling data, models and tools that actually works in practice at a truly significant scale. It is implemented not only at its originally founding organizations Delft University of Technology and Deltares but also in a number of research programs with multiple partners. As a result OpenEarth is now carried by a rapidly growing user community that currently consists of more than 1000 users, of which over 100 developers made over 10.000 contributions. This community originates from tens of organizations from multiple countries. Together they share and co-develop thousands of tools, tera-bytes of data and numerous models (source code as well as model schematisations).

The OpenEarth infrastructure and workflow

Improper management of data, models and tools can easily result in a wide range of very recognisable frustrations:

• Accidentally using older versions of data, models or tools
• Not knowing where the most recent version is and what its status is
• Making the same mistake twice due to lack of control over versions
• Losing important datasets that are extremely hard to replace
• Uncertainty as to what quantities have been measured and which units apply
• Uncertainty as to the geographical location of measurements
• Uncertainty as to the time and time zone the measurements were taken in
• Lack of insight in the approach taken and the methods used
• A wide range of formats of incoming (raw) data
• Getting the feeling that a certain issue must have been addressed before by another analyst
• Running into a multitude of tools for the same thing
• Running into a multitude of databases each in its own language and style
• Etc.
  
  

Although the above-described frustrations are very common throughout the hydraulic engineering industry, no practical and widely accepted remedy used to be available. Since 2003, OpenEarth is developed to fill this gap providing an infrastructure to support a bottom-up approach for long-term project-transcending collaboration adhering to four basic criteria:

1. Open standards & open source - The infrastructure should be based on open standards, not require nonlibre software and be transferable
2. Complete transparency - The collected data should be reproducible, unambiguous and self-descriptive.
   Tools and models should be open source, well documented and tested
3. Centralised web-access - The collection and dissemination procedure for data, models and tools should be web-based and centralised to maximize access, promote collaboration and impede divergence.
4. Clear ownership and responsibility – Although data, models and tools are collected and disseminated via the OpenEarth infrastructure, the responsibility for the quality of each product remains at its original owner.

The heart of the OpenEarth infrastructure is formed by three kinds of web-services (see Figure):

1. a web service for version control and back-up of raw data, model schematisations and computer code.
2. a web service for accessing data numbers, and
3. a web service for accessing data graphics,
   
   

Once data, models and tools are stored according to the OpenEarth principles, it becomes quite straightforward to develop specific tools that address a particular user need. Examples of such applications developed within Building with Nature are:

• Visualisation of open-source data - OpenEarth-Viewer
• Interactive Dredge Planning Tool - Singapore
• Interactive group modelling - MapTable
  
  

Usage skills

OpenEarth is useful to various actors in different roles in the science and engineering context. Here we discuss the required usage skills for:

• project engineers/hands-on researchers,
• project leaders/principal investigators and program managers.
  
  

Project engineers/hands-on researchers

OpenEarth is primarily meant for those project engineers/hands-on researchers who analyse data and model outputs on a regular basis.

Data examples are elevation recordings and bathymetry soundings, counts of species, remote sensing imagery and in situ measurements with buckets, ADCP, OBS, conductivity sensor, thermometer, waverider. Model examples include Delft3D, Delwaq, SWAN, ASMITA, Unibest and XBeach.

The usage skills required for project engineers/hands-on researchers are that they have to learn about version control and about some international standards for exchange of data over the web. These can simply be incorporated in their existing workflows, so they can keep using the skills that they already have, whether it is Matlab, python, R, ArcGIS. The reason is that OpenEarth is simply a workflow for embedding existing processing into an overall framework for collaboration. OpenEarth selected the most promising standards that have been implemented worldwide for raw data, standard data, tailored data, graphics of data and meta-data of data . For these standards dedicated tutorials have been compiled for the most common general analysis tools in collaboration with other institutes: Matlab (IHE), python (TUD), R (IMARES), Delft3D, OPeNDAP (KNMI, Rijkswaterstaat), WxS (KNMI). In addition, a few times per year free hands-on courses, so called sprint sessions, are organized. The aim is that within one day participants can work with these international standards with their own data, using their own favourite analysis tool. These standards are gradually made compulsory at EU level by the INSPIRE legislation.

Project leaders/principal investigators and program managers

Project leaders/principal investigators and program managers can also adopt OpenEarth in their project. The only skill they need is that when they compose a project team, they manage to finance enough manpower in expertise in standardisation and automation of data and model processing. And that they show enough leadership to enforce use of it. Experts tend to use artisan approaches that are tailored to their specific needs, hampering exchange between different disciplines in projects. In addition, they hardly guarantee provenance of their data. This has allowed many debacles to occur like with the IPCC (Intergovernmental Panel on Climate Change) data or the wide-spread fraud of a Nijmegen professor. The lack of proper control of automation in science has been discussed in a recent Nature article. OpenEarth offers a complete workflow to circumvent these issues. Project leaders can therefore force their project teams to adopt common standards, following the consensus approach of OpenEarth. In a sense project leaders/principal investigators and program managers are therefore the most influential users of OpenEarth, without having to know about the details of the standards.

Building with Nature interest

The Data and Knowledge Management part of BwN adopted the infrastructure and standards made available by OpenEarth. Benefit of joining the OpenEarth initiative is that previously uploaded datasets as well as the hundreds of tools that have already been developed in other projects will become available to the BwN partners (all routines in OpenEarth are open source, distributed under the GNU Lesser General Public License conditions). Furthermore an active international community of practice is already established, facilitating optimal use of available resources. Training in the use of the available tools is given on a regular basis and can easily be extended to the BwN community.

Important for the BwN program is to establish maximum overlap between on-going activities related to data, model and tool storage and dissemination conforming as much as possible to existing standards (adopting successes of previous projects) on the one hand while striving for maximum usability (avoid pitfalls of previous projects) on the other. Data is put under version control, converted to a commonly used data standard (NetCDF for grids or PostgreSQL for biodiversity data) and disseminated through a web-based data server (OPeNDAP). Models are put under version control, stored and disseminated through a repository. Tools, also put under version control, are shared by creating a common toolbox that can be used by all partners.

Holland Coast case

The Netherlands is well-known for its advanced coastal protection policies and in association a lot of projects have dealt with this particular stretch of coast before. Invariably each of these projects always dealt with (subsets of) Rijkswaterstaat’s datasets on bathymetry (transects as well as grids), hydrodynamics (waves, currents and waterlevels) and water quality (TSS, etc), TNO’s datasets on topography (the general elevation data of the Netherlands), bathymetry (Dutch continental shelf bathymetry) and soil composition (soil types and grain size distributions), and KNMI’s datasets on meteorology (wind and pressure fields).

All these datasets are now available through OpenEarth in readily accessible uniform netCDF files. The datasets are stored in raw form in the OpenEarthRawdata repository. The same repository also contains the Matlab and Python scripts that can process these data into netCDF files. The raw data have been enriched with meta information and transformed to netCDF. These files are stored on an OPeNDAP server. This server allows everyone to make calculations with these data without the need to download the full data collection that covers over 10 GB.

Finally, for non-specialists that do not need to perform calculations, all data are also presented as readily processed KML feeds for straightforward visualization on Google Earth. Because all this data is already available, the Holland Coast case needs to reserve less of its budget for gathering and processing of historic external datasets. As a result more budget remains for additional data acquisition and/or more detailed (data) analysis and reporting. Just as the Holland Coast case benefits from all this previously gathered data, any future project will benefit from the additional data gathered specifically by this project and new or improved tools for data analysis.

Availability of important basic analysis tools

Besides the use of various datasets a number of important coastal state indicators is associated with the coastal policies that apply to this coastal region.The decision recipe for this policy is based on a volume trend approach applying the Momentane Coastline (MCL) concept. This concept is used in a maintenance indicator comparing the extrapolated expected coastline position (TCL) with a reference value: the 1990 coastline position (BCL) (TAW, 1995; Van Koningsveld and Mulder, 2004).

Since well tested tools associated with important indicators are already available from the OpenEarthTools repository, the Holland Coast case needs to reserve less of its budget for tool development. As a result, more budget remains for thorough application of these tools to the also available datasets. The availability and accessibility of data and tools in this case challenges the analyst to apply his/her analysis routines not to just one transect or a couple of transects near the area of interest, but simply to all transects available in the database. This is an excellent way to (1) see whether or not the routine is robust enough to deal with variations/flaws in the data, and (2) to see whether an observed phenomenon can be detected throughout the dataset or just in a small sub selection of the data. Currently the Building with Nature consortium is executing a field campaign at Vluchtenburg, located in the southern region of the Holland Coast. Data that is collected ranges from Argus video imagery, Aeolian sediment transport, grain size distributions and 3D laser mapping bathymetries to jetski based near shore bathymetries. This data, gathered by various partners, is already routinely stored in the OpenEarthRawData repository and will in the near future become available through OpenEarth for other projects as well.

Various data examples

 

This is a demonstration of various data visualizations that we have made using a combination of Matlab, the free tools in the OpenEarth toolbox, and the Google Earth ^TM mapping service. To make your own, just sign up and check out the tutorials for the googleplot toolbox. In an NCK 2012 paper we have shown that plotting data into Google Earth is possible for a wide variety of coastal datasets.

A few of the other *.kml files with which the screen shots below have been made are available in this (temporary) open dir. Note that some of the bigger files might need a bit of time to load, just be patient.

There are more initiatives worldwide that offer data in Google Earth.

RIjkswaterstaat scanfish (meetvis) salinity data for one month		Beach and foreshore elevation of Sand engine gathered with DGPS on quad and wheel barrow during JERICO 2014 summerschool	Beach and foreshore elevation of Sand engine gathered with DGPS on quad and wheel barrow during JERICO 2014 summerschool
Beach and foreshore elevation of Sand engine gathered with DGPS on quad and wheel barrow during JERICO 2014 summerschool	Beach and foreshore elevation of Sand engine gathered with DGPS on quad and wheel barrow during JERICO 2014 summerschool	ICES 2009 salinity obtained with OET tool getICESdata.m from SOAP service http://ocean.ices.dk/webservices/hydchem.asmx	Sedimentation/erosion rate for mud (grain size < 63 mum) in the Wadden Sea based on vaklodingen 1985-2013 in combination with the sediment atlas (grain size) data.
Sedimentation/erosion rate within the Wadden Sea based on the vaklodingen dataset 1985 to 2013.	Wadden Sea most recent bathymetry based on 'Vaklodingen'-dataset.	sediment atlas waddenzee with grains zie disdistribution graphs	Vessels of Opportunity temperature data
vaklodingen tile KB122_2120 2D view created with KMLsurf.m	vaklodingen tile KB122_2120 3D view created with KMLsurf.m	Observations of common scoters near Voordelta (PMR-NCV project Deltares)	Rijkswaterstaat kusthoogte lidar data with its airplane tracks (Voordelta)
Rijkswaterstaat kusthoogte lidar data with its airplane tracks (Ems-Dollard)	Delft3D model of Patos Lagoon (Brazil) plotted with colormap "bathymetry_vaklodingen". Model is part of OpenearthModels.	Delft3D model of Patos Lagoon (Brazil) plotted with colormap "GMT_globe". Model is part of OpenearthModels.	Delft3D model of Patos Lagoon (Brazil) plotted with colormap "GMT_ocean". Model is part of OpenearthModels.
Legally binding VTV grain sizes along Dutch coast	Overview of coastal indicators 2, Bas Huisman, framework of BwN HK3.8 and HK4.1	Overview of coastal indicators, Bas Huisman, framework of BwN HK4.1 and HK3.8	NIOZ TESO veerboot metingen
NIOZ TESO veerboot metingen	NIOZ TESO veerboot metingen	Kustlijnkaartenboek2012: Ameland	HIRLAM model output absolute velocites at 10m as isolines on top of the ground pressures scaled to z levels (gray tones) (multiple time steps).
HIRLAM model output ground pressures scaled to z levels and isolines (1 time step).	HIRLAM model output ground pressures scaled to z levels and isolines (multiple time steps).	HIRLAM model output absolute velocites at 10m as isolines on top of the ground pressures scaled to z levels (gray tones) (1 time step).	Surface heights for the Dutch coast (Rijkswaterstaat)
3D cores of the NCP by Geological Survery of the Netherlands (TNO dinoloket.nl)	3D cores of the NCP by Geological Survery of the Netherlands (TNO dinoloket.nl)	3D cores of the NCP by Geological Survery of the Netherlands (TNO dinoloket.nl)	3D cores of the NCP by Geological Survery of the Netherlands (TNO dinoloket.nl)
Rijkswaterstaat vaklodingen + JarKus	Bed topography of DVR model (Delft3D) for the Waal River near Nijmegen	Bed topography of DVR model (Delft3D) for the Waal River near Nijmegen	DVR numerical grid for the Waal River near Nijmegen
Rijkswaterstaat - beach and foreshore nourishments with features (1952 - 2010)	Rijkswaterstaat - All the information about time series Jarkus	Deltares - Bed profile along transects, Sao Tome	TU Delft / SKLEC fieldmeasurement Chongming Island (Shanghai) by Steven te Slaa
	TU Delft 3D simulation of Rhine ROFI with Delft3D (doi:10.1016/j.csr.2007.06.011)	TU Delft 3D simulation of Rhine ROFI with Delft3D (doi:10.1016/j.csr.2007.06.011)	Rijkswaterstaat - time series Jarkus beach and foreshore profiles
Rijkswaterstaat - time series Jarkus beach and foreshore profiles	Rijkswaterstaat - time series Jarkus beach and foreshore profiles	TU Delft 3D simulation of Rhine ROFI with Delft3D (doi:10.1016/j.csr.2007.06.011)	TU Delft 3D simulation of Rhine ROFI with Delft3D (doi:10.1016/j.csr.2007.06.011)
faux data of 3D atmospheric velocity field	RWS JarKus and coastlines at Banjaard location, Zeeland	Rijkswaterstaat Lidar data of coast: slufter, Texel (Lidar data higlighted)	Rijkswaterstaat Lidar data of coast: slufter, Texel (aereal imagery higlighted)
Rijkswaterstaat Lidar data of WaddenZee	Rijkswaterstaat vaklodingen	Netherlands Hydrographic Office sea bathymetry & AHN land elevation	Netherlands Hydrographic Office sea bathymetry & AHN land elevation
Netherlands Hydrographic Office sea bathymetry of Dutch Continental Shelf	Netherlands Hydrographic Office sea bathymetry & AHN land elevation	Netherlands Hydrographic Office sea bathymetry & AHN land elevation	TNO Bouw en Ondergrond sediment data (silt in bed) van Dutch Continental Shelf
TNO Bouw en Ondergrond sediment data (D50) van Dutch Continental Shelf	TNO Bouw en Ondergrond sediment data (D50) van Dutch Continental Shelf	TNO Bouw en Ondergrond sediment data (D10) van Dutch Continental Shelf	Delft3D modeling results after 200 yrs of bathymetric development in Western Scheldt starting from an initially flat bed based on 1998 sediment volume. Mick van der Wegen, UNESCO-IHE.
Delft3D modeling results after 100 yrs of bathymetric development in Western Scheldt starting from an initially flat bed based on 1998 sediment volume. Mick van der Wegen, UNESCO-IHE.	Delft3D modeling results after 50 yrs of bathymetric development in Western Scheldt starting from an initially flat bed based on 1998 sediment volume. Mick van der Wegen, UNESCO-IHE.	Delft3D modeling results after 25 yrs of bathymetric development in Western Scheldt starting from an initially flat bed based on 1998 sediment volume. Mick van der Wegen, UNESCO-IHE.	3D kml of dunes
Dijkringen, residential areas and habitat areas in larger Rotterdam area	Dijkring 14 with residential areas and habitat areas and kustfundament	3D mesh of faux data "christo"	Deltares Delft3D bathymetry of Markermeer
Filled 3D contour plot of faux data	Filled 3D contour plot of faux data	TNO NCP sediment grain size d10 map (gridded)	Rijkswaterstaat vaklodingen kaartbladen overview
NASA MODIS image of 2009 Dec 22nd [http://onearth.jpl.nasa.gov] & Rijkswaterstaat vaklodingen	Rijkswaterstaat beach and foreshore nourishments since 1952	AHN/TNO Actueel Hoogtebestand Nederland (100m)	AHN/TNO Actueel Hoogtebestand Nederland (100m) & Rijkswaterstaat vaklodingen
AHN/TNO Actueel Hoogtebestand Nederland (100m) & Rijkswaterstaat vaklodingen	Deltares Delft3D/SWAN model grids Gibraltar	Rijkswaterstaat JarKus beach measurements & AHN/TNO Actueel Hoogtebestand Nederland (100m)	Rijkswaterstaat JarKus beach measurements
Rijkswaterstaat JarKus beach measurements	Rijkswaterstaat JarKus beach measurements (gridded)	AHN/TNO Actueel Hoogtebestand Nederland (100m)	USGS beach data
USGS beach data	USGS beach data	USGS beach data	Deltares Delft3D Grevelingen model
Deltares Delft3D Grevelingen model	Rijkswaterstaat Sediment Atlas Waddenzee	Rijkswaterstaat Sediment Atlas Waddenzee	Rijkswaterstaat vaklodingen
Rijkswaterstaat waterbase locations	Rijkswaterstaat vaklodingen	Rijkswaterstaat vaklodingen	Deltares Delft3D ZUNO fijn model
Deltares Delft3D ZUNO model	Rijkswaterstaat WAQUA / Deltares Delft3D - DCSM model	NOAA/KNMI SST sea surface temperature satellite image	NOAA/KNMI SST sea surface temperature satellite image
NOAA/KNMI SST sea surface temperature satellite image	Deltares Delft3D ZUNO model	Deltares Delft3D results	KNMI wind locations
Deltares Delft3D results	Deltares Delft3D bathymetry Westerschelde	Deltares Delft3D bathymetry	Deltares Delft3D bathymetry
Deltares Delft3D results	Rijkswaterstaat vaklodingen	Rijkswaterstaat vaklodingen kaartblad	Rijkswaterstaat vaklodingen
Deltares Delft3D bathymetry	Rijkswaterstaat waterbase locations	KNMI HIRLAM surface temperature	Rijkswaterstaat vaklodingen Rijksdriehoek grid
Deltares Delft3D simulation of Katrina	Rijkswaterstaat kustlijnen	Rijkswaterstaat JarKus beach measurements	KNMI HIRLAM wind vectors

• Baart, F.; Van Gelder, P. H. A. J. M.; De Ronde, J.; Van Koningsveld, M. & Wouters, B., 2012. "The effect of the 18.6 year lunar nodal cycle on regional sea level rise estimates." Journal of Coastal Research, 2012
• Ciavola, P.; Ferreir, O.; Haerens, P.; Van Koningsveld, M.; Armaroli, C. & Lequeux, Q. (2011). "Storm impacts along European coastlines. Part 1: The joint effort of the MICORE and ConHaz Projects." Environmental Science & Policy 14(7): pp. 912-923
• Ciavola, P.; Ferreir, O.; Haerens, P.; Van Koningsveld, M. & Armaroli, C. (2011). "Storm impacts along European coastlines. Part 2: lessons learned from the MICORE project." Environmental Science & Policy 14(7): pp. 924-933
• Den Heijer, C., Baart, F. & Van Koningsveld, M., 2011. "Assessment of dune failure along the Dutch coast using a fully probabilistic approach." Geomorphology
• Van Koningsveld, M., G.J. de Boer, F. Baart, T. Damsma, K. den Heijer, P. van Geer and B. de Sonneville, 2010. OpenEarth: inter-company management of data, models, tools and knowledge. Proceedings WODCON XIX conference Beijing, China, 2010
• Boer de, Gerben J.; Baart, Fedor; Bruens, Ankie; Damsma, Thijs; Geer van, Pieter; Grasmeijer, Bart; Heijer den, Kees; Koningsveld van, Mark (2012). OpenEarth: using Google Earth as outreach for NCK’s data. In: NCK-days 2012 : Crossing borders in coastal research., 13 March 2012 - 16 March 2012, Enschede, the Netherlands.

Access tool:

• http://www.openearth.nl