Data Standards Training Course

Agenda course 2008-11-14

The course is scheduled for Friday November 14th from 9:00 AM to approximately 16:00 PM. We have reserved Deltares' meetingroom Innovation, which is located at far right corner on the ground floor of Deltares' main building (Rotterdamseweg 185, Delft). The course starts 09:00 AM sharp. We have a full schedule so please be on time!

• 9:00-9:05 Data management in Micore (by Mark van Koningsveld)
  • Opening and outline of the course
    1. Extract Taking measurements and storing the measured data into files.
    2. Transform Enriching gathered data with metadata and storing in a standard file format.
    3. Load Storing the files in a database - not dealt with in this course.
    4. Provide Giving access to the database.

• 9:05-9:30 Open Earth Initiative (by Gerben de Boer)
  • Explanation of Open Earth tools (former McTools/UCIT)

• 9:30-10:30 Extract (by Fedor Baart)
  • Loading data into subversion

• 10:30-12:00 Transform(by Fedor Baart)
  • Storing metadata
  • Converting to netcdf

• 12:00-13:00 Lunch

• 13:00-14:00 Provide (by Fedor Baart)
  • Reading netcdf and opendap
  • Excercise: selection of data

• 14:00-16:00 Using database (e.g. for modelling) (by Kees de Heijer)
  • useful tools available (General)
  • useful tools available (XBeach)
  • Excercise: grid -> xbeach

Table of contents

Data management in Micore (WP2)

Download presentation

Brief reiteration of WP2 objectives

The goal of this work package is to define a data standard and archiving protocol to provide end-users with a comprehensive standardised database. The protocol will become part of the Open Earth Initiative (formerly known as McTools) as developed by Deltares and be freely available. The data standard and archiving protocol provides a four step database guideline for MICORE users to:

• Extract - Taking measurements and storing the measured data into files,
• Transform - Enriching gathered data with metadata and storing in a standard file format,
• Load - Storing the files in a database - not dealt with in this course, and
• Provide - Giving access to the database.

physical data for each pilot site (bathymetries, waves, tides etc.) in a form convenient for the assessment of storm risk along the European coast. This will include (for each pilot site) data collected from regular monitoring schemes, as well as historic physical data (collected in WP1) and measured physical data (collected in WP3). Where possible, the UCIT (pronounced Use it!) methodology will be used and developed further. Where needed other approaches will be adopted.

More detailed info on WP2 can be found here.

Open Earth Initiative

Download presentation

• • Explanation of the Open Earth Initiative (former McTools/UCIT)
  • Overview of Open Earth building blocks
  • Attention for a few developing conventions

Extract

Download presentation

• • Working with subversion
  • Excercise: Loading raw data into subversion

Transform

Download presentation

• • Excercise: Filling in required fields using the inspire metadata editor
  • Excercise: Storing metadata file on OpenEarthRawData
  • Excercise: Loading raw data in memory
  • Excercise: Converting to netcdf
  • Using CF convention

Provide

Download presentation

• • Reading NetCDF and OPeNDap
  • Selection of data

Using database

Download presentation

• • Useful tools available (General)
  • Useful tools available (XBeach)
  • Several examples
  • Excercise: run examples and vary input

Using the database - course examples

Things that came up

• • Consider to store models in the OpenEarthModels repository
    • Include metadata about the models (describe who set it up, what it is used for, grid, used data, software version etc.)
  • Store XBeach tools in OpenEarthTools repository
  • Consider to upload a list to the wiki with standard organisation names
  • Consider to upload a list to the wiki with standard data set terms (e.g. all should use bathymetry rather than bathy, bottom, etc.)

Photos