Export Log Messages (Threshold Events) from the Delft-Fews database to several file formats
The thresholdExport module can be used to export threshold crossing log messages that have been created in a workflow with the threshold crossing module. The threshold crossing module will check time series against threshold values, and add records in the Delft-FEWS database with threshold crossing information. These threshold crossings can be exported to files in different formats. The threshold export module uses a template file that specifies the format of the exported file. The procedure how the elements in the template file are filled is similar to the report module.
Note: By design the threshold export module will only export threshold crossings that are created in the same workflow. It is therefore important to have one workflow that first logs the threshold crossings and then export these thresholds.
The threshold export module can export threshold crossing events for use in other systems. The configuration of the module is split into several elements, information is provided below.
This type specifies which writer should be used to write the file. The type must be one from the enumeration. Presently (2017.01) only CAP-AU is included in this list.
Folder (directory) in which to store the exported files.
This elements describes how to construct the filename(s) of the exported file(s). If only the name element is given a fixed name is used for each export. The prefix and suffix elements describe how to create a filename prefix and/or suffix.
This element is used as a filter for all event codes logged. Only the event codes from this list will be exported. The * character is allowed at the start or end of the event code
The template file will be used as template for the exported log event information. The procedure how the elements in the template file are filled is similar to the report module.
In the CAP template file there are 3 types of TAGS used. The Threshold Export module will replace these TAGS with information from the content of the log messages, location attributes and system properties.
- $propertie$: All tags between $ will be replaced by elements from the global.properties file and are system specific
- @attribute@: All tags between @ will be replaced by location attributes of the location that has a threshold crossing log message.
- %information%: All tags between % will be replaced by information from the exported threshold. There will be a list of configurable %information% tags that the export module will support.
configurable data formats to be used in the export file
configurable number formats to be used in the export file
Filter the threshold crossings based on locationSets or parameters
Time span to search in the database for previous threshold exports. This information will be used to find out if an export is an Alert, Update or Cancel. If no threshold export is found within the specified time span, FEWS will log an Alert. If threshold exports are found, FEWS will log an Update or Cancel.
Expiry time of the alert in the export file. This time span will also be used to find the threshold exports that will be included in the Reference element when the ALERT_HISTORY Tag is included in the template.
configurable identifier used in the exported message, must be unique.
Must be a valid e-mail address.
Mapping of the Thresholds configured in FEWS and the Threshold name exported in the CAP file. The order of the thresholdId’s in this mapping table will also be used to check the severity of the threshold.
To translate %SEVERITY% at least 1 map threshold element is needed. When the threshold is not present in the map no file will be exported.
At least 1 map threshold element is needed for level thresholds to determine the %MESSAGE_TYPE% because it needs the correct order of the threshold levels to find the lowest available for a parameter-location-time combination.
When the lowest available threshold can not be determined (for a parameter-location-time combination) no file will be exported.
For rate thresholds %MESSAGE_TYPE% is always "Alert" because there is no specific crossing order.
At least 1 map threshold element is needed for level thresholds to determine the %ALERT_HISTORY (DF1)% to see if lower or higher thresholds were crossed before. Otherwise it will remain empty.
For rate thresholds it will always be empty.
The %information% tags will be hard coded in the Threshold Export module. In this example the following %information% tags are required:
- %EXPORT_TIME(DF1)%: Will be replaced by the time the message is exported by FEWS. Time format is specified in the Export Module configuration file.
- %MESSAGE_TYPE%: Enumeration: Alert, Update, Cancel. New alerts use Alert, then subsequent should use Update, and required references. For WL alerts we are assuming that the first threshold crossing should be Alert, and then subsequent threshold changes are updates (both up and down crossing), and a cancel is used when drops below lowest thresholds.
- %ALERT_HISTORY (DF1)%: Include the history of the threshold exports within the configured eventExpiryTime period. Will be included in the CAP file as Reference element.
- %THRESHOLD_NAME%: Will be the threshold name as configured in the Thresholds.xml file. When required a mapping table of Threshold ID’s from HyFS can be used.
- %EXPIRY_TIME(DF1)%: Will be replaced by the time the message can expire, is general for all threshold exports and configurable in the Export Module configuration file.
- %LOCATION_NAME%: Name of the location that crossed a threshold.
- %THRESHOLD_VALUE%. Will be replaced by the threshold value that is crossed.
- %THRESHOLD_MESSAGE%: Will be the complete threshold crossing message that is logged by HyFS.
- %VALUE%. Will be replaced by the time series value that is logged to cross a threshold.
- %X%,%Y%: Will be replaced by the X and Y coordinates of the location.
- %CROSSING_DIRECTION%: Will be replaced by above or below is it is an up-crossing or down-crossing.
Example configuration file
An example threshold Export configuration file is provided below, as well as a template file with included TAGS.
Multivalued Attribute Tags (since 2019.02)
Since 2019.02 it is possible to define custom tags that will be replaced in the template by multivalued attributes.
The multivalued index will be found by matching the threshold event value with the value of specific multivalued matching attribute value.
Also different attributes can be used for up and down crossings.
When a tag needs to be replaced with text that is larger than 255 characters, it should be split into multiple tags because the maximum length of a text attribute value is 255.