File-based storage
The Deltares Open Archive is a file-based storage solution. The data is not stored in a database but in regular files. The Forecasting Shell server when running a Delft-FEWS archive export writes the data files directly to the archive data folder from an archive export workflow. The time series are stored in netCDF-files. Other data types are stored in xml or regular text files. If for practical / security reasons the Delft-FEWS Archive Export task cannot directly export to the archive data folder, the Delft-FEWS FSS should write to an intermediate folder and a copy script should copy the data files into the archive data folder.
Elastic catalogue automatically populated by Harvester
To facilitate searching in the archive a catalogue is used. The catalogue is based on Elastic (https://www.elastic.co/), which is an open source tool for indexing data. The catalogue is populated by the so-called harvester. The harvester scans the data folder to discover which data sets are available in the archive and updates the catalogue accordingly.
THREDDS webserver
THREDDS https://www.unidata.ucar.edu/software/thredds/current/tds/ is an open source webserver providing access to forecast data for Delft-FEWS and other third party applications. The data can be downloaded by a variety of supported protocols such as HTTP, OpenDAP and WMS.
Delft-FEWS Webservices and the Deltares Open Archive
The Delft-FEWS Webservices can be used in combination with the Deltares Open Archive. The seamless integration connects the Delft-FEWS data store with the archive. If data is requested from Delft-FEWS the data store will automatically retrieve the data from the Deltares Open Archive if the data is expired from the FEWS data store.
