Page History

...

The Flood Periods The thresholdExport module can be used to export threshold crossing log messages that have been created in a workflow with the threshold crossing module. The threshold crossing module will check time series against threshold values, and add records in a workflow to create flood period from imported warning forecasts and threshold crossings of observed time series for forecast locations.  This Flood Periods module has functionality that is also partly included in the Threshold Crossing module, the difference is that the Flood Periods module will generate flood periods for catchments/basins and not for forecast locations. The Flood Periods module has its own XML configuration schema and can be scheduled in a Delft-FEWS system, similar to standard workflows. The module will generate automatically new Flood Periods for basins that are based on observed and/or warning forecasts for locations in the basin. The Flood Period information on basin level is stored in a new database table ‘Floodperiods’. This new Flood Periods database table is created in the Delft-FEWS database with threshold crossing information. These threshold crossings can be exported to files in different formats. The threshold export module uses a template file that specifies the format of the exported file. The procedure how the elements in the template file are filled is similar to the report module.

Configuration

operational database and will be included in the Delft-FEWS MC-MC synchronization process, database clean-up processes and export to the Delft-FEWS Open Archive. The purpose of the Flood Periods module is that it can create a flood period for a catchment or basin, based on threshold crossings of observed time series or forecast time series for locations within a catchment. The Flood Periods module will check all time series in a timeseriesSet and group the timeseriesSet locations in larger units, these will be called areas. If a direct link with the FEWS Open archive is required, the areas must be identical to the configured Archive Areas.

Configuration

The configuration of the Flood Periods The threshold export module can export threshold crossing events for use in other systems. The configuration of the module is split into several elements, information is provided below.

exportTypeStandard

This type specifies which writer should be used to write the file. The type must be one from the enumeration. Presently (2017.01) only CAP-AU is included in this list.

folder

Folder (directory) in which to store the exported files.

exportFileName

This elements describes how to construct the filename(s) of the exported file(s). If only the name element is given a fixed name is used for each export. The prefix and suffix elements describe how to create a filename prefix and/or suffix. 

eventCodePattern

This element is used as a filter for all event codes logged. Only the event codes from this list will be exported. The * character is allowed at the start or end of the event code

templateFile

The template file will be used as template for the exported log event information. The procedure how the elements in the template file are filled is similar to the report module.

In the CAP template file there are 3 types of TAGS used. The Threshold Export module will replace these TAGS with information from the content of the log messages, location attributes and system properties.

•  $propertie$: All tags between $ will be replaced by elements from the global.properties file and are system specific
• @attribute@: All tags between @ will be replaced by location attributes of the location that has a threshold crossing log message.
• %information%: All tags between % will be replaced by information from the exported threshold. There will be a list of configurable %information% tags that the export module will support.

dateFormat

configurable data formats to be used in the export file

numberFormat

configurable number formats to be used in the export file

filter

Filter the threshold crossings based on locationSets or parameters

thresholdHistorySearchTimeSpan

Time span to search in the database for previous threshold exports. This information will be used to find out if an export is an Alert, Update or Cancel. If no threshold export is found within the specified time span, FEWS will log an Alert. If threshold exports are found, FEWS will log an Update or Cancel.

eventExpiryTime

Expiry time of the alert in the export file. This time span will also be used to find the threshold exports that will be included in the Reference element when the ALERT_HISTORY Tag is included in the template.

identifier

configurable identifier used in the exported message, must be unique.

sender

Must be a valid e-mail address.

mapThreshold

Mapping of the Thresholds configured in FEWS and the Threshold name exported in the CAP file. The order of the thresholdId’s in this mapping table will also be used to check the severity of the threshold.

Configurable TAGS

The %information% tags will be hard coded in the Threshold Export module. In this example the following %information% tags are required:

• %EXPORT_TIME(DF1)%: Will be replaced by the time the message is exported by FEWS. Time format is specified in the Export Module configuration file.
• %MESSAGE_TYPE%: Enumeration: Alert, Update, Cancel. New alerts use Alert, then subsequent should use Update, and required references. For WL alerts we are assuming that the first threshold crossing should be Alert, and then subsequent threshold changes are updates (both up and down crossing), and a cancel is used when drops below lowest thresholds.
• %ALERT_HISTORY (DF1)%: Include the history of the threshold exports within the configured eventExpiryTime period. Will be included in the CAP file as Reference element.
• %THRESHOLD_NAME%: Will be the threshold name as configured in the Thresholds.xml file. When required a mapping table of Threshold ID’s from HyFS can be used.
• %EXPIRY_TIME(DF1)%: Will be replaced by the time the message can expire, is general for all threshold exports and configurable in the Export Module configuration file.
• %LOCATION_NAME%: Name of the location that crossed a threshold.
• %THRESHOLD_VALUE%. Will be replaced by the threshold value that is crossed.
• %THRESHOLD_MESSAGE%: Will be the complete threshold crossing message that is logged by HyFS.
• %VALUE%. Will be replaced by the time series value that is logged to cross a threshold.
• %X%,%Y%: Will be replaced by the X and Y coordinates of the location.

Example configuration file

The Floodperiods module can check two types of time series when creating a new 'provisional' flood period:

• Compare observed and forecast time series with thresholds and start a new 'provisional' flood period when a threshold is crossed.
• Check in the database if there are new warnings imported and start a new 'provisional' flood period when a new forecast value is found.

 The configuration of the function to check observed time series for threshold crossings is shown below; this is the observed element.

Image Added

• newFloodPeriodLogEventCode: This is the event code that will be logged when a new flood period is created.
• areaLocationAttributeId: This is the location attribute used to group the forecast locations in areas.
• skipLocationsWithoutAreaId: When true then FEWS will not generate an Error message when a location has no location attribute.
• expiryTime: Set the expiry time of the flood period records in the database. This period must be larger than the expected length of a flood period, otherwise it cannot be closed.
• periodInitialLength: When a new flood period is created, this is the minimum initial length of the flood period. The end date of a flood period will be set using this configured period length.
• maximumPeriodExtensionLength: This is the maximum length an existing flood period can be extended, in the past or in the future, when a new threshold event is found. When a threshold event is found outside this period length then a new flood period will be created.
• thresholdGroupId: This is the thresholdgroupID, from the Thresholds.xml file, used to check the time series for threshold crossings.
• timeSeriesSet: This is a standard Delft-FEWS time series, in this example a Processed Observed Water Level series.

With the above configuration, the Floodperiods module will check all time series included in the timeSeriesSet and group the thresholds for the areas in the REPORT_FOLDER location attribute.

The configuration of the function to check if new warnings are imported is shown below; this is the importedThresholdCrossings element, normally in a forecast element.

Image Added

Additional elements for forecasts are:

• forecastSearchPeriod: This is the period before To that will be used to find forecast series. The period -2 days to 0 means that only forecasts will be checked that have a forecast time in the database of -2 days to 0 days before T0.
• timeSeriesProperty - key: This is the property of the imported warning series that contains a threshold level.
• timeSeriesProperty – map: This is a mapping table to map the threshold elements in the warning files with the ThresholdID’s of the Delft-FEWS Application (configured in the Thresholds.xml file).

 With the above configuration, the Floodperiods module will check all time series included in the timeSeriesSet and group the thresholds for the areas in the REPORT_FOLDER location attribute.

 Some additional remarks on Flood Periods:

• The Delft-FEWS module will create new flood periods and set a start date
• The Delft-FEWS module will set an end date for a flood period with a length, periodInitialLength, after the last recorded threshold crossing or imported warning.
• Flood Periods can only be automatically created when new warnings are imported in the operational database and/or observed time series are up-crossing level thresholds.
• The module will compare the two start times (new warning imported and observed time series up-crossing level thresholds) and choose the earlier one as the flood period start time.
• When a long historic period must be checked for forecast threshold crossings, a FEWS batch task must be used with the start and end date of the batch task covering the analysis period and the batch interval set to a period smaller than the forecastSearchPeriod. 

Example configuration file

An example Flood Periods module configuration file is provided belowAn example threshold Export configuration file is provided below, as well as a template file with included TAGS.

<thresholdExport<?xml version="1.0" encoding="UTF-8"?>
<floodPeriodsModule xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews httphttps://fewsfewsdocs.wldelftdeltares.nl/schemas/version1.0/thresholdExportfloodPeriodsModule.xsd">
<export>
	<exportTypeStandard>CAP-AU</exportTypeStandard>	<newFloodPeriodLogEventCode>FloodPeriod.New</newFloodPeriodLogEventCode>
	<folder>$EXPORT_FOLDER$\CAP_Alerts</folder>
	<exportFileName>
		<name>_RiverAlert_CAP_%LOCATION_ID%_@STATE@_%THRESHOLD_ID%_.xml</name>
		<prefix>
			<timeZeroFormattingString>yyyyMMddHHmm</timeZeroFormattingString>
		</prefix>
		<suffix>
			<currentTimeFormattingString>yyyyMMddHHmmss</currentTimeFormattingString>
		</suffix>
	</exportFileName>
	<eventCodePattern>River_Alert.*</eventCodePattern>
	<eventCodePattern>River_Minor.*</eventCodePattern>
	<eventCodePattern>River_Moderate.*</eventCodePattern>
	<eventCodePattern>River_Major.*</eventCodePattern>
	<templateFile>CAP_Export_Template</templateFile>
	<dateFormat id="DF1">
		<timeZone>
			<timeZoneName>GMT</timeZoneName>
		</timeZone>
		<dateTimePattern>yyyy-MM-dd'T'HH:mm:ss+00:00</dateTimePattern>
	</dateFormat>
	<dateFormat id="DF2">
		<timeZone>
			<timeZoneName>ACST</timeZoneName>
		</timeZone>
		<dateTimePattern>dd-MM-yyyy</dateTimePattern>
	</dateFormat>
	<numberFormat id="NF1">
		<pattern>{0,number,##0.0}</pattern>
	</numberFormat>
	<filter>
		<parameterId>H.obs.proc</parameterId>
	</filter>
	<thresholdHistorySearchTimeSpan unit="day" multiplier="7"/>
	<eventExpiryTime unit="day" multiplier="1"/>
	<identifier>AusBoM-PRODUCT_ID-%LOCATION_ID%-%EXPORT_TIME(DF1)%</identifier>
	<sender>CAP.Message@bom.gov.au</sender>
	<mapThreshold thresholdId="River Alert" value="Unknown"/>
	<mapThreshold thresholdId="Minor Flooding" value="Minor"/>
	<mapThreshold thresholdId="Moderate Flooding" value="Moderate"/>
	<mapThreshold thresholdId="Major Flooding" value="Severe"/>
</export>
</thresholdExport>

<alert xmlns="urn:oasis:names:tc:emergency:cap:1.2">
	<identifier>%IDENTIFIER%</identifier>
	<sender>%SENDER%</sender>
	<sent>%EXPORT_TIME(DF1)%</sent>
	<status>Actual</status>
	<msgType>%MESSAGE_TYPE%</msgType>
	<source>@STATE@, STATE_AGENCY</source>
	<scope>Public</scope>
	<code>urn:oasis:names:tc:emergency:cap:1.2:profile:CAP-AU:1.0</code>
	%ALERT_HISTORY(DF1)%
	<info>
		<language>en-AU</language>
		<category>Met</category>
		<event>EVENT_EVENT</event>
		<urgency>Unknown</urgency>
		<severity>%SEVERITY%</severity>
		<certainty>Observed</certainty>
		<eventCode>
			<valueName>https://govshare.gov.au/xmlui/handle/10772/6495</valueName>
			<value>EVENTCODE</value>
		</eventCode>
		<expires>%EXPIRY_TIME(DF1)%</expires>
		<senderName>Australian Government</senderName>
		<headline>%MESSAGE_TYPE%: River Alert at %LOCATION_NAME% %CROSSING_DIRECTION% %THRESHOLD_NAME% level at %CROSSING_TIME(DF1)% - %THRESHOLD_VALUE(NF1)%  Value: %VALUE(NF1)% (@STATE@)</headline>
		<description>%THRESHOLD_MESSAGE%</description>
		<contact>http://www.bom.gov.au/inside/contacts.shtml</contact>
		<area>
			<areaDesc>%LOCATION_NAME% (RIVER_NAME) - @STATE@</areaDesc>
			<circle>%Y%,%X% 1</circle>
			<geocode>
				<valueName>http://www.iso.org/iso/country_codes.html</valueName>
				<value>AU-@STATE@</value>
			</geocode>
		</area>
	</info>
</alert>
<areaLocationAttributeId>REPORT_FOLDER</areaLocationAttributeId>
	<skipLocationsWithoutAreaId>true</skipLocationsWithoutAreaId>
	<expiryTime unit="day" multiplier="365"/>
	<periodInitialLength unit="day" multiplier="1"/>
	<maximumPeriodExtensionLength unit="day" multiplier="5"/>
	<forecastSearchPeriod unit="day" start="-2" end="0"/>
	<observed>
		<thresholdValuesSetsCrossings>
			<thresholdGroupId>Waterlevel_VAT</thresholdGroupId>
			<timeSeriesSet>
				<moduleInstanceId>Normalize_RiverTelemetry</moduleInstanceId>
				<valueType>scalar</valueType>
				<parameterId>H.obs.proc</parameterId>
				<qualifierId>15m</qualifierId>
				<locationSetId>AUS_Stations_Forecast_Location</locationSetId>
				<timeSeriesType>external historical</timeSeriesType>
				<timeStep unit="minute" multiplier="15"/>
				<relativeViewPeriod unit="day" start="-4" end="0" startOverrulable="true"/>
				<readWriteMode>add originals</readWriteMode>
			</timeSeriesSet>
		</thresholdValuesSetsCrossings>
	</observed>
	<forecasted>
		<importedThresholdCrossings>
			<timeSeriesProperty key="Prediction_Level">
				<map value="BelowMinor" thresholdId="Minor Flooding"/>
				<map value="Minor" thresholdId="Minor Flooding"/>
				<map value="Moderate" thresholdId="Moderate Flooding"/>
				<map value="Major" thresholdId="Major Flooding"/>
			</timeSeriesProperty>
			<timeSeriesSet>
				<moduleInstanceId>ImportContentReviewer</moduleInstanceId>
				<valueType>scalar</valueType>
				<parameterId>H.issued.fcst</parameterId>
				<locationSetId>AUS_Stations_Forecast_Location</locationSetId>
				<timeSeriesType>external forecasting</timeSeriesType>
				<timeStep unit="nonequidistant"/>
				<relativeViewPeriod unit="day" start="-4" end="10"/>
				<readWriteMode>add originals</readWriteMode>
			</timeSeriesSet>
		</importedThresholdCrossings>
	</forecasted>
</floodPeriodsModule>