Versions Compared

Old Version 8

changes.mady.by.user Jan Verkade

Saved on

compared with

New Version 9

changes.mady.by.user Camiel van Breugel

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

File-based storage

The OpenArchive Deltares Open Archive is a file-based storage solution. The data is not stored in a database but in regular files. The Forecasting Shell server when running a Delft-FEWS FSS archive export writes the data files directly to the archive data folder from an archive export workflow. The timeseries time series are stored in NetCDFnetCDF-files. Other data types are stored in xml or regular text files. In some cases, for example for security reasons, it is not possible for If for practical / security reasons the Delft-FEWS FSS to write directly Archive Export task cannot directly export to the archive data folder. In that case , the Delft-FEWS FSS should write to an intermediate folder and an a copy script should copy the data files in into the archive data folder.

Elastic catalogue automatically populated by Harvester

To facilitate searching in the archive a catalogue is used. The catalogue is based on Elastic (https://www.elastic.co/), which is an open source tool for indexing data. The catalogue is populated by the so-called harvester. The harvester scans the data folder to discover which data sets are available in the archive and updates the catalogue accordingly.

THREDDS webserver

THREDDS https://www.unidata.ucar.edu/software/thredds/current/tds/ , is an open source software tool, is used to make the data accessible by webserver providing access to forecast data for Delft-FEWS and other third party applications. By using THREDDS the The data can be downloaded by using HTTP. It addition THREDDS also supports a variety of supported protocols such as HTTP, OpenDAP and WMS.

 

...


Image Added


Delft-FEWS PI-webservice and the Deltares Open Archive

The Delft-FEWS PI-webservice can be used in combination with the OpenArchiveDeltares Open Archive. The seamless integration connects the Delft-FEWS data store with the archive. If data is requested from Delft-FEWS the data store will automatically retrieve the data from the OpenArchive Deltares Open Archive if the data is expired from the FEWS data store.