Ingest datasets in Delft-FEWS stand alone
Once downloaded, the data can be ingested by a Delft-FEWS stand alone with the aim to bring the local datastore back in the 'original state', i.e. with a similar time series set definition as used when Delft-FEWS ran the workflow to produce the data. This exact timeseries definition is included in the netcdf files during the archive export process. The importArchiveModule has been implemented to uses this definition included in the netcdf files to put the data back in Delft-FEWS local datastore
Figure 4.16 and Table 4.10 describe the configuration details for the import module.
Figure 4.16 Configuration of archive Import (importArchiveModule.xsd)
Table 4.10 importArchiveModule configuration
Element | Format | Description |
importSimulated/importFolder | string (path) | Full path where the simulated datasets are made available for import by Delft-FEWS |
importObserved/importFolder | string (path) | Full path where the observed datasets are made available for import by Delft-FEWS |
importMessages/importFolder | string (path) | Full path where the messages datasets are made available for import by Delft-FEWS |
importExternalForecast/importFolder | string (path) | Full path where the external forecasts are made available for import by Delft-FEWS |
importRatingCurves/importFolder | string (path) | Full path where the rating curve datasets are made available for import by Delft-FEWS |
Ingest of Historic events in Delft-FEWS (FSS)
Historic events are a special datatype to Delft-FEWS as they can be used to overlay on an existing time series graph. Historic events are the only datasets which can be ingested in a Delft-FEWS client server system. Normally, the archive server has a process running (the HistoricEventsExporter) which extracts historic events data from the archive and pushes them to the Forecasting Shell Server import directories for ingest in the operational database.
This import process has its own data administration process, and does not use the timeseries definition as embedded in the netcdf files. Hence an idMap may be needed to translate netcdf variables into Delft-FEWS parameters and locations. In addition, backup and failure folders can be defined to prevent loss of data in the automated process.
Figure 4.17 and Table 4.11 describe the configuration for importing historic events. This module should be executed in a separate workflow compared to the other archived datasets.
Figure 4.17 Configuration of historical events import (importArchiveModule.xsd)
Table 4.11 Historical events import configuration (importArchiveModule)
Element | Format | Description |
importHistoricalEvents | importHistoricalEvents ComplexType | root element for import of Historical events into Delft-FEWS database |
importHistoricalEvents ComplexType |
|
|
importFolder | string (path) | Full path where the historical events datasets are made available for import by Delft-FEWS |
failedFolder | string (path | Full path where Delft-FEWS can put any dataset which failed on import |
backupFolder | string (path | Full path where Delft-FEWS can back up any dataset which is being imported |
idMapId | string | Idmap to translate NetCDF variables to FEWS parameters/locations |