How to put netCDF files on the OPeNDAP server of OpenEarth.

This protocol is will be a deliverable for the Building with Nature project DM 1.1

• Verify whether the data are open, and you can publish them on interne. (Keep a copy of the email or a hard-copy of the letter, avoid nasty discussions afterward).

• Put the raw data in the OpenEarthRawData repository.
  • Make a new directory per institute (lower case).
  • Add an INSPIRE *.xml file.
  • Add a *.url internet shortcut file to the intetnet source (if any).
  • Make subdirectories /raw, /cache and /scripts.
    • Store the raw data in /raw
    • /cache is for data that could in principle be downloaded, so there is no need for version control. However, some sources, notably www.waterbase.nl, are down for a long time, so caching is useful.
    • Make a script to transform the data into netCDF files. Store the script in /scripts. Make sure the script only needs tools from /script and OpenEarth tools.

• Always put new netCDF first in your local synchronized local copy of the data in the OPeNDAP server. Make this local copy available to your entire company by making it accessible through your local network. This network copy is for optional fast access within your institute. Within Deltares this is P:\mcdata\opendap\ (ask for write access Gerben de Boer). For users outside Deltares we are working on ftp access to the filer server under OPeNDAP, which enable you to make synchronized local copy (with rsync).

• Make a new directory per institute or consortium that owns the copyright of the data. In principle the following sources need exactly the same directory structure:
  • the OpenEarthRawData repository
  • the OPeNDAP servers,
  • your synchronized local copy of the data in the OPeNDAP servers (within Deltares: P:\mcdata\opendap)
  • the *.kml direcory (zie beneden)

• From a windows PC - and currently only from within the Deltares network - you can add netCDF files from local copy to the OPeNDAP servers with the free program WinSCP (http://winscp.net/). You can also use fabric fabric.

• Currently OpenEarth has employed two OPeNDAP servers:
  1) production server: http://opendap.deltares.nl
  2) test/acceptation/fall-back server: http://dtvirt5.deltares.nl:8080
  Always try the data first on the dtvirt5 (test functie), and only on the real production server afterwards. If the data assessed as OK, do put the data on both servers, also on the dtvirt5 (fall-back function). If you are going to make changes to the OPeNDAP server isself, please test this first successfully on the dtvirt5, before installing/employing this on the production OPeNDAP server (acceptation function).

• Log in with your je Deltares username. Your password is known to you for the dtvirt5, and is your regular Deltares password for the productionserver. You can change your je passwords on the linux machines with command passwd.

• Copy the netCDF files to the data directory:
  @opendap.deltares.nl: /drbd/opendap-data/
  @dtvirt5.deltares.nl: /data

• Refresh your internet browser if it was already open and low-and-behold, the dataset appears: http://opendap.deltares.nl http://dtvirt5.deltares.nl:8080

• If you made Google Earth *.kml/*.kmz files too, you can put them on the dtvirt5 too. The www.OpenEarth.nl wiki links to the kml directory: @dtvirt5.deltares.nl: /data/kml
  Please use the same directory structure again (organised per originator/copyright owner.

• If you added a nice dataset, do post a news item on www.OpenEarth.nl.