12 Archive Module - EOL in 2022

Archive Forecast Module

Introduction

The standard version of Delft FEWS (FEWS) includes archiving functionality. DELFT-FEWS can create archives for selected forecasts, thresholds, configurations and timeseries. Delft FEWS can also restore a forecast and its data from an archive. An archive of a forecast contains all result data from the forecast run, but also includes all the data used by the forecast at the time of the run, including any initial module states. The archive can be used for hindcasting and analysis; using the data that was available at the time of the forecast. Forecast archives, with all associated data, can be created manually or automatically. The archives are placed as zip files in a user defined folder. Retrieving of archives can be done by importing the zip files.

Management of the folders with archives is the responsibility of the system manager. When not selectively managed the disk space required for archiving will quickly increase depending on the data volumes used and produced by a given FEWS configuration.

As all functional tasks are run by DELFT-FEWS through a workflow, a moduleInstance is created to allow archives to be made. The module instances must be correctly registered in the moduleInstanceDescriptors (see Regional Configuration), and point to the relevant class in the moduleDescriptors configuration (see System Configuration).

Archive modules

Delft-FEWS contains several modules that can create or restore archives:

• ConfigurationArchiver: archives the configuration (incremental changes)
• ThresholdEventsArchiver: archives thresholdcrossings
• ForecastArchiver: archives all forecasts and the data used to make them
• TimeseriesArchiver: archives timeseries that are not included in the forecast Archives
• snapShotArchiver: exports a localDataStore replicating the central database

Configuration of a ConfigurationArchiver Module Instance

The ConfigurationArchiver can be used to make archives of configuration changes in the FEWS database. In the example below, the ConfigurationArchiver is scheduled to run once every 24 hours. The ConfigurationArchiver checks if there have been any configuration changes in the last 24 hours. If so, it stores the configuration changes in a zip file and stores these in the folder configured in the global.properties file with the Tag ARCHIVE_EXPORT_PATH.

<?xml version="1.0" encoding="UTF-8"?>
<archiveRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:schemaLocation="http://www.wldelft.nl/fews https://fewsdocs.deltares.nl/schemas/version1.0/archiveRun.xsd">
	<importDirectory>$ARCHIVE_IMPORT_PATH$</importDirectory>
	<exportDirectory>$ARCHIVE_EXPORT_PATH$</exportDirectory>
	<exportArchiveRun>
		<archivePeriod start="-24" end="0" unit="hour"/>
		<archiveType>ConfigurationArchive</archiveType>
	</exportArchiveRun>
</archiveRun>

Configuration of a ThresholdEventsArchiver Module Instance

The ThresholdEventsArchiver can be used to make archives of threshold crossings that have been stored in the FEWS database. Threshold events are created by the FEWS Threshold module, and stored in the FEWS ThresholdEvents table. The threshold events can be used by the FEWS Performance module to analyse the performance of the Forecasting System.
An example of the ThresholdEventsArchiver is shown in the example below. As with the ConfigurationArchiver, the ThresholdEventsArchiver is scheduled to run once every 24 hours. The ThresholdEventsArchiver checks if there have been any threshold events in the last 24 hours. If so, it stores the threshold events in a zip file and stores these in the folder configured in the global.properties file with the Tag ARCHIVE_EXPORT_PATH.

 
<?xml version="1.0" encoding="UTF-8"?>
<archiveRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/archiveRun.xsd">
	<importDirectory>$ARCHIVE_IMPORT_PATH$</importDirectory>
	<exportDirectory>$ARCHIVE_EXPORT_PATH$</exportDirectory>
	<exportArchiveRun>
		<archivePeriod start="-24" end="0" unit="hour"/>
		<archiveType>ThresholdEventsArchive</archiveType>
	</exportArchiveRun>
</archiveRun>

Configuration of a ForecastArchiver Module Instance

The ForecastArchiver can be used to make archives of forecasts that have been stored in the FEWS database. All forecasts that have been made, together with the data that has been used to make the forecasts will be stored in the forecast archive. An example of the ForecastArchiver is shown in the example below, the ForecastArchiver is scheduled to run once every 24 hours. The ForecastArchiver stores all forecasts in a zip file and stores these in the folder configured in the global.properties file with the Tag ARCHIVE_EXPORT_PATH.

 
<?xml version="1.0" encoding="UTF-8"?>
<archiveRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/archiveRun.xsd">
	<importDirectory>$ARCHIVE_IMPORT_PATH$</importDirectory>
	<exportDirectory>$ARCHIVE_EXPORT_PATH$</exportDirectory>
	<exportArchiveRun>
		<archivePeriod start="-24" end="0" unit="hour"/>
		<exportToFile>true</exportToFile>
	</exportArchiveRun>
</archiveRun>

Configuration of a TimeSeriesArchiver Module Instance

The TimeSeriesArchiver can be used to make archives of timeseries from selected module instances that will not be stored in a normal forecast archive. These time series can be performance indicators or imported data that is not used by any forecast run. All timeseries that have been stored in the database with the selected module instances will be stored in the forecast archive. An example of the TimeSeriesArchiver is shown in the example below, the TimeSeriesArchiver is scheduled to run once every 24 hours. The TimeSeriesArchiver stores all forecasts in a zip file and stores these in the folder configured in the global.properties file with the Tag ARCHIVE_EXPORT_PATH.

 
<?xml version="1.0" encoding="UTF-8"?>
<archiveRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/archiveRun.xsd">
	<importDirectory>$ARCHIVE_IMPORT_PATH$</importDirectory>
	<exportDirectory>$ARCHIVE_EXPORT_PATH$</exportDirectory>
	<exportArchiveRun>
		<archivePeriod start="-24" end="0" unit="hour"/>
		<archiveType>TimeSeriesArchive</archiveType>
		<includeModuleInstanceId>Import_Telemetry</includeModuleInstanceId>
		<includeModuleInstanceId>Import_CWB_Database</includeModuleInstanceId>
		<includeModuleInstanceId>Import_CWB_Grid</includeModuleInstanceId>
		<includeModuleInstanceId>Import_Qpesums</includeModuleInstanceId>
		<includeModuleInstanceId>Import_GFS</includeModuleInstanceId>
		<includeModuleInstanceId>Import_TRMM</includeModuleInstanceId>
		<includeModuleInstanceId>Import_WRF</includeModuleInstanceId>
	</exportArchiveRun>
</archiveRun>

Workflow to create archives

The standard procedure is to run a scheduled archive workflow every 24 hours. The Archive workflow can also be started from the Manual Forecast Display. An example of the Archive Workflow is shown below.

  
<?xml version="1.0" encoding="UTF-8"?>
<workflow xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/workflow.xsd" version="1.1">
	<activity>
		<runIndependent>true</runIndependent>
		<moduleInstanceId>Archive_Forecast</moduleInstanceId>
	</activity>
	<activity>
		<runIndependent>true</runIndependent>
		<moduleInstanceId>Archive_Thresholds</moduleInstanceId>
	</activity>
	<activity>
		<runIndependent>true</runIndependent>
		<moduleInstanceId>Archive_TimeSeries</moduleInstanceId>
	</activity>
	<activity>
		<runIndependent>true</runIndependent>
		<moduleInstanceId>Archive_Configuration</moduleInstanceId>
	</activity>
</workflow>

Workflow to Import Archives

Archives can only be imported in a FEWS database from a Stand Alone System, it is not possible to import archives in a FEWS Operator Client. To import an archive workflow must be configured that runs the archive modules again, this time to import the archives from an import folder.

 
<?xml version="1.0" encoding="UTF-8"?>
<workflow xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/workflow.xsd" version="1.1">
	<!-- Run Import Archive Files on Stand Alone -->
	<activity>
		<moduleInstanceId>Import_Archive_Forecast</moduleInstanceId>
	</activity>
	<activity>
		<moduleInstanceId>Import_Archive_Thresholds</moduleInstanceId>
	</activity>
	<activity>
		<moduleInstanceId>Import_Archive_TimeSeries</moduleInstanceId>
	</activity>
</workflow>

The module instances for importing forecasts, timeseries and threshold events are  similar. Instead of configuring an exportArchiveRun, an importArchiveRun must be configured.  For archiveType ‘ForecastArchive’ and ‘TimeSeriesArchive’ an option includeGrids  is relevant.   When includeGrids is false, the gridded time series are not imported. 

The following example shows a configuration of an import archive module instance to import forecasts.

<?xml version="1.0" encoding="UTF-8"?>
<archiveRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/archiveRun.xsd">
	<importDirectory>$ARCHIVE_IMPORT_PATH$</importDirectory>
	<exportDirectory>$ARCHIVE_EXPORT_PATH$</exportDirectory>
	<importArchiveRun>
		<archiveType>ForecastArchive</archiveType>
	</importArchiveRun>
</archiveRun>

Archive Display

It is possible to configure an Archive display that facilitates the retrieval of archives using a Stand Alone system. This Archive display can only be used when an Archive Server has been installed. For more information on installation of an Archive Server, please contact the FEWS Product Manager.