The Data Standards Training Course was developed in the framework of the EU FP7 Project MICORE and involves one day in which the participants are taken through a wide range of excercises. Although the schedule can be adapted to better match with the participants' specific needs and background knowledge the schedule of this course generally is as follows:
- 9:00-9:05 Data management in general
- Opening and outline of the course
- Extract Taking measurements and storing the measured data into files.
- Transform Enriching gathered data with metadata and storing in a standard file format.
- Load Storing the files in a database - not dealt with in this course.
- Provide Giving access to the database.
- Opening and outline of the course
- 9:05-9:30 Open Earth initiative
- General introduction of the OpenEarth initiative
- 9:30-10:30 Extract
- Excercise: loading data into subversion
- 10:30-12:00 Transform
- Excercise: storing metadata
- Excercise: converting to netcdf
- 12:00-13:00 Lunch
- 13:00-14:00 Provide
- Excercise: reading netcdf and opendap
- Excercise: selection of data
- 14:00-16:00 Using database (e.g. for modelling)
- useful tools available (General)
- useful tools available (XBeach)
- Excercise: grid -> xbeach
Table of contents
Data management
Brief reiteration of objectives
The EU FP7 Project MICORE defined a data standard and archiving protocol to provide end-users with a comprehensive standardised database. The protocol has become part of the Open Earth Initiative (formerly known as McTools) as developed by Deltares and is available freely. The data standard and archiving protocol provides a four step database guideline for OpenEarth users and developers to:
- Extract - Taking measurements and storing the measured data into files,
- Transform - Enriching gathered data with metadata and storing in a standard file format,
- Load - Storing the files in a database - not dealt with in this course, and
- Provide - Giving access to the database.
physical data (bathymetries, waves, tides etc.) in a form convenient for the assessment of direct and easy use in research and consultancy projects.
Open Earth Initiative
-
- Explanation of the Open Earth Initiative (open source version of McTools/UCIT)
- Overview of Open Earth building blocks
- Attention for a few developing conventions
Extract
-
- Working with subversion
- Excercise: Loading raw data into subversion
Transform
-
- Excercise: Filling in required fields using the inspire metadata editor
- Excercise: Storing metadata file on OpenEarthRawData
- Excercise: Loading raw data in memory
- Excercise: Converting to netcdf
- Using CF convention
Provide
-
- Reading NetCDF and OPeNDap
- Selection of data
Using database
-
- Useful tools available (General)
- Useful tools available (XBeach)
- Several examples
- Excercise: run examples and vary input
