10 ThresholdValueSets

Complementary to the definition of the types of thresholds identified, the values of the thresholds are defined in the ThresholdValueSets configuration (ThresholdValueSets.xml on a local filesystem).

A thresholdValueSet is build up out of:

• The values and what type each value is
• (Optional) Stage/Discharge conversions
• The timeSeriesSet a thresholdValueSet is associated with

Thresholds can be used for display purposes in the graphs, spatial display, icons in the map or in reports. It is also possible to have messages logged when thresholds are crossed.

What are the different value types?

There are 3 types of thresholdValues:

• levelThresholdValues
• rateThresholdValues
• maxThresholdValues (also known as peakThresholdValues)

levelThresholdValues are compared with single values from a timeSeries. It is crossed if a timeSeries value is bigger and the previous timeSeries value is smaller then the levelThresholdValue. i.a.w. the first time a timeSeries value is bigger. This means that the values in the timeSeries will first have to become smaller then the levelThresholdValues before it can be crossed again. Even tho the name "level" is asociated with water levels, it can be used for any parameter type (i.e. temperature, rainfall, wind).

rateThresholdValues are compared with a rate of change. To calculate this, each value in a timeSeries is matched with a value in the future that is exactly "timespan" away. The threshold is crossed if the difference between these two values is greater then the rateThresholdValue.

maxThresholdValues are compared with single values from a timeSeries. But only peak values are considered. A peak value has a previous and next value that are both lower. The threshold is crossed if a peak value is above the maxThresholdValue. If two crossings are within the timeSpan, only the first crossing will actually occur.

TimeSeriesSet Association

ThresholdValues are associated with all timeSeriesSets that are defined in the same thresholdValueSetSequence (With some extra associations for Stage/Discharge conversions). When the unit of the threshold values do not correspond with the internal unit, the unit element can be used to define the unit of the threshold values.

When configuring thresholdValueSets, make sure to only configure one thresholdValueSet per unique timeseriesSet. If the same timeseriesSet is used in a second thresholdValueSets, the FEWS code will ignore this second thresholdValueSet. When the FEWS system is started, an ERROR message will be logged when there are multiple thresholdValueSets configured for the same timeseriesSet.

Stage/Discharge conversions

Rating Curves are used in fews to convert between stage (water level) and discharge (water flow) values. The optional stage and discharge elements in the thresholdValueSet should only and always be used if a rating curve is used in combination with thresholds.

A timeSeriesSet is put in the stage or discharge element based on what parameter is has. So putting a timeSeriesSet in the stage element says that this timeSeriesSet has a stage parameter. Putting the thresholds in the stage element is very similar to the timeSeriesSets in that it shows that these thresholds are a stage threshold. This indirectly shows what unit the threshold values are in.

A rating curve is normally associated to a timeSeriesSet by its locationId. So when leaving out any ratingCurve related config in the thresholdValueSet, ratingCurves are searched for in this way. When a different ratingCurve is needed, it can be specified with use of the ratingCurveLocationId and ratingCurveQualifierId elements.

After all these definitions it is clear:

• What thresholds are stage and discharge
• What timeseries are stage and discharge
• What ratingCurves need to be used and consequently how to convert between stage and discharge values

Now the stage thresholds will be converted to discharge thresholds and vice versa after which they are assigned to the timeSeriesSet. So the stage and converted discharge thresholds will be assigned to the timeSeriesSets that were placed in the stage element. And similarly, the discharge and converted stage thresholds will be assigned to the timeSeriesSets in the discharge element.

Threshold Module

Thresholds can be used to log messages when threshols are crossed. A predefined workflow activity must be activated in a workflow to have threshold crossings logged. 

	<activity>
		<runIndependent>true</runIndependent>
		<predefinedActivity>threshold event crossing</predefinedActivity>
	</activity>

It is common practice to use the log messages to initiate an action by the Master Controller when using thresholds in a live forecasting system; check the Event Action Mappings documentation in the system admin pages.

EventTypeIds (otherwise known as EventCodes) are used as a prefix in logging messages for the MC to identify which threshold is crossed. When threshold messages need to be logged, a special predefined threshod activity is required in the workflows.

<levelThresholdValue>
			<levelThresholdId>UKV</levelThresholdId>
			<valueFunction>1</valueFunction>
			<upActionLogEventTypeId>UKVRRCThreshold.crossed</upActionLogEventTypeId>
		</levelThresholdValue>

ThresholdValueSet Schema

The latest version of the ThresholdValueSet schema can be found on the Deltares schema site: http://fews.wldelft.nl/schemas/version1.0/thresholdValueSets.xsd. The elements in the schema are shortly described below.

ThresholdValueSet

Contains level- and/or rate- and/or max-thresholds set for a given time series

description

Optional description for the ThresholdValueSet. Used for reference purposes only.

considerQualifiers

Since 2016.01 #67260. Default = true.  If set to false, qualifiers are ignored when matching threshold value rule sets to time series. When set to false it is no longer possible to generate threshold events using the ThresholdCrossingModule. The explorer and time series dialog will work with this option

standDownIndividualLocations

Since 2017.01: By default actions events for observation time series are only issued when all locations of the same ThresholdValueSet have been stood down. When it is required to issue all stand down events even though some locations are still in enhanced state the option 'standDownIndividualLocations' must be set to true. This functionality cannot be used in combination with SystemAlert triggers that are linked to enhanced forecasts. Use this functionality in combination with SystemAlert triggers that send email messages.

locationAttributeKeys

Since 2017.01: List of location attribute keys for which the values should be added to the action threshold log messages. If omitted no attributes will be added to the messages.

graceTime

Since 2017.01: To avoid numerous rate thresholds being issued during a period of heavy rain the graceTime period can be configured. After an initial threshold crossing for a given time series no further crossings will be issued for the duration of the graceTime period. Only if the time series crosses the next threshold level, an event will be issued.

stage

Threshold values and time series that are in units of stage.

discharge

Threshold values and time series that are in units of discharge.

forecastAvailableThresholdValue

If a threshold crossing event is measured for a given observed parameter, then the thresholdEventCrossing module logs whether or not there is a forecast run available for the corresponding forecast parameter, within a given relative time period. This information is used in the ThresholdSkillScoreDisplay.

timeSeriesSet

Define one or more time series sets for which the threshold values in this thresholdValueSet will be used.

ratingCurveLocationId

When specified the rating curve available at this location is used instead of the location from the time series set.

ratingCurveQualifierId

One or more qualifier ids the used rating curve should have all

ThresholdValueSetSequence

unit

Specify when the unit given for the values is not the same as the (internally stored) unit of the parameter it applies to. When specified it is required to also specify  configUnitConversionsId in Parameters.xml. In this unit conversions the conversion from the specified unit to the (internal) unit should be available

levelThresholdValue

Specific values for level thresholds.

aggregationLevelThresholdValues

Specific values for level thresholds. This contains one or more level threshold values for different aggregation periods.

rateThresholdValue

Specific values for rate thresholds.

maxThresholdValue

Specific values for max thresholds.

levelThresholdValue

levelThresholdId

ID of the level threshold type, as it is specified in the file thresholds

value

level threshold value

valueFunction

Location dependent threshold value by using a function with location attributes. eg. @FLOOD_WATCH@. When an attribute is missing for a location the threshold value is invisible/ignored for that location.

label

Optional label.

labelFunction

Optional location dependent label that is specified by a function, e.g. "@FLOOD_WATCH@", in which tags between "@" signs refer to location attributes that are defined in the locationSets config file. The tags are replaced by actual attribute values. These attribute values can be different for different locations. If an attribute is missing for a location, then the label is ignored for that location.

warningArea

One or more warnings that correspond to this threshold for the configured timeSeriesSet(s). These warningAreas are e.g. used in the ThresholdOverviewDisplay threshold crossing counts tabs.

upActionLogEventTypeId

ID of the action message (=additional log.message) that must be logged if a threshold value is crossed UP. This id has to contain a dot, because the action event is only visible to the master controller if this id contains a dot, e.g. "Threshold_Name.Up". The event code need not be unique.

downActionLogEventTypeId

ID of the action message (=additional log.message) that must be logged if a threshold value is crossed DOWN. This id has to contain a dot, because the action event is only visible to the master controller if this id contains a dot, e.g. "Threshold_Name.Down". The event code need not be unique.

aggregationLevelThresholdValues

levelThresholdId

ID of the level threshold type, as it is specified in the file thresholds

timeSpanValue

One or more level threshold values for different aggregation periods. If this threshold value set is used for a time series without an aggregation period, then the timeStep of that time series is used as aggregation period. If a value for the aggregation period of a time series is not configured here, then these threshold values are ignored for that time series.

When the aggregationLevelThresholdValues function is used in the threshold module using the ActionEvent logging, the threshold module will only use the timestep of the original time series, it will ignore the timeSpanValue element. For rainfall threshold checking with timespan larger than the timestep of the original time series, the rateThresholdValue function must be used instead.

timeSpanValueFunction

One or more level threshold values for different aggregation periods. If this threshold value set is used for a time series without an aggregation period, then the timeStep of that time series is used as aggregation period. If a value for the aggregation period of a time series is not configured here, then these threshold values are ignored for that time series. Here the value elements can be function expressions, e.g. "@N@ / 1000", in which tags between "@" signs refer to location attributes that are defined in the locationSets config file. The expression is evaluated when the configuration is read, by replacing the tags with actual values. These values can be different for different locations. When an attribute is missing for a location the threshold value is invisible/ignored for that location.

label

Optional label.

labelFunction

Optional location dependent label that is specified by a function, e.g. "@FLOOD_WATCH@", in which tags between "@" signs refer to location attributes that are defined in the locationSets config file. The tags are replaced by actual attribute values. These attribute values can be different for different locations. If an attribute is missing for a location, then the label is ignored for that location.

warningArea

One or more warnings that correspond to this threshold for the configured timeSeriesSet(s). These warningAreas are e.g. used in the ThresholdOverviewDisplay threshold crossing counts tabs.

upActionLogEventTypeId

ID of the action message (=additional log.message) that must be logged if a threshold value is crossed UP. This id has to contain a dot, because the action event is only visible to the master controller if this id contains a dot, e.g. "Threshold_Name.Up". The event code need not be unique.

downActionLogEventTypeId

ID of the action message (=additional log.message) that must be logged if a threshold value is crossed DOWN. This id has to contain a dot, because the action event is only visible to the master controller if this id contains a dot, e.g. "Threshold_Name.Down". The event code need not be unique.

rateThresholdValue

rateThresholdId

ID of the rate threshold type, as it is specified in the file thresholds

value

Value of the rate threshold. This is the change within timeSpan where timeSpan is the timeSpan configured below. Rate of change = (change within timeSpan) / timeSpan.

valueFunction

Location dependent threshold value by using a function with location attributes. eg. @FLOOD_WATCH@. When an attribute is missing for a location the threshold value is invisible/ignored for that location

label

Optional label.

labelFunction

Optional location dependent label that is specified by a function, e.g. "@FLOOD_WATCH@", in which tags between "@" signs refer to location attributes that are defined in the locationSets config file. The tags are replaced by actual attribute values. These attribute values can be different for different locations. If an attribute is missing for a location, then the label is ignored for that location.

upActionLogEventTypeId

ID of the action message (=additional log.message) that must be logged if threshold value is crossed UP. This id has to contain a dot, because the action event is only visible to the master controller if this id contains a dot, e.g. "Threshold_Name.Up". The event code need not be unique.

downEventTypeId

ID of the action message (=additional log.message) that must be logged if threshold value is crossed DOWN. This id has to contain a dot, because the action event is only visible to the master controller if this id contains a dot, e.g. "Threshold_Name.Down". The event code need not be unique.

timeSpan

This time span will be used in the computation of the rate of change. Rate of change = (change within timeSpan) / timeSpan.

rainRate

If rainRate is true, then the input time series is accumulated before the rate of change is computed.

maxThresholdValue

maxThresholdId

ID of the max threshold type, as it is specified in the file thresholds

value

max threshold value

valueFunction

Location dependent threshold value by using a function with location attributes. eg. @FLOOD_WATCH@. When an attribute is missing for a location the threshold value is invisible/ignored for that location

label

Optional label.

labelFunction

Optional location dependent label that is specified by a function, e.g. "@FLOOD_WATCH@", in which tags between "@" signs refer to location attributes that are defined in the locationSets config file. The tags are replaced by actual attribute values. These attribute values can be different for different locations. If an attribute is missing for a location, then the label is ignored for that location.

actionLogEventTypeId

ID of the action message (=additional log.message) that must be logged if threshold value is exceeded. This id has to contain a dot, because the action event is only visible to the master controller if this id contains a dot, e.g. "Threshold_Name.Max". The event code need not be unique.

timeSpan

If two peaks fall have a distance that is less then the defined timeSpan, only the first peak will be reported.

Event Threshold Value

An event threshold value is similar to a level threshold, only it has the possibility to be based on correlation events. See 07 Correlation Module 

The advantage to this is that for instance peak heights that occured through the years can be used in the correlation display as well as a threshold without storing the same information in different ways.

The <parameterIdFunction> specifies for which parameter the threshold is valid, the enabled function specifies whether the it should be included as threshold or not. This is mainly meant to filter out events needed for the correlation module that is not desired as threshold.

This functionality is specifically designed for multivalued attributes. An important requirement is that for all the functions: <valueFunction>, <labelFunction>, <parameterIdFunction> and <enabledFunction> the location should have the same amount of attribute values.

This can easily be accomplished by storing all attributes in the same multivalued attribute csv file like the example below.

<thresholdValueSet id="EventThresholds" name="EventThresholds">
   <eventThresholdValues>
      <eventThresholdId>LevelWarn</eventThresholdId>
      <valueFunction>@PEAK_HEIGHT_VALUE@</valueFunction>
      <labelFunction>@PEAK_HEIGHT_TIME@</labelFunction>
      <parameterIdFunction>@PEAK_HEIGHT_PAR@</parameterIdFunction>
      <enabledFunction>@PEAK_HEIGHT_ENABLED@</enabledFunction>
   </eventThresholdValues>
   <timeSeriesSet>
      <moduleInstanceId>ThresholdCheck</moduleInstanceId>
      <valueType>scalar</valueType>
      <parameterId>H.m</parameterId>
      <locationSetId>MultiValuedAttributeThresholds</locationSetId>
      <timeSeriesType>external historical</timeSeriesType>
      <timeStep unit="hour" divider="1" multiplier="1"/>
      <readWriteMode>add originals</readWriteMode>
   </timeSeriesSet>
</thresholdValueSet>

<?xml version="1.0" encoding="UTF-8"?>
<!-- edited with XMLSPY v5 rel. 4 U (http://www.xmlspy.com) by Juzer Dhondia (WL | Delft Hydraulics) -->
<!--Sample XML file generated by XMLSPY v5 rel. 4 U (http://www.xmlspy.com)-->
<locationSets xmlns="http://www.wldelft.nl/fews"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/locationSets.xsd" version="1.1">
  <locationSet id="MultiValuedAttributePeakHeights">
   <csvFile>
      <file>MultiValuedAttributePeakHeightsLocations.csv</file>
      <geoDatum>WGS 1984</geoDatum>
      <id>%ID%</id>
      <name>%NAME%</name>
      <x>%X%</x>
      <y>%Y%</y>
      <z>0</z>
      <attributeFile>
         <csvFile>MultiValuedAttributePeakHeightsLocationAttributes.csv</csvFile>
         <id>%ID%</id>
         <dateTimePattern>dd-MM-yyyy</dateTimePattern>
         <attribute id="PEAK_HEIGHT_PAR">
            <text>%PAR%</text>
         </attribute>
         <attribute id="PEAK_HEIGHT_TIME">
            <dateTime>%TIME%</dateTime>
         </attribute>
         <attribute id="PEAK_HEIGHT_VALUE">
            <number>%VALUE%</number>
         </attribute>
         <attribute id="PEAK_HEIGHT_ENABLED">
            <boolean>%ACTIVE%</boolean>
         </attribute>
      </attributeFile>
   </csvFile>
 </locationSet>
</locationSets>

ID,PAR,VALUE,TIME,ACTIVE
PH_locA,H.m,0.1,01-01-2001,true
PH_locA,H.m,0.2,02-02-2002,false
PH_locA,H.m,0.3,03-03-2003,true
PH_locA,H.sim,0.4,04-04-2004,false
PH_locA,H.sim,0.5,05-05-2005,true
PH_locA,H.sim,0.6,06-06-2006,true
PH_locB,H.m,1.1,11-10-2011,true
PH_locB,H.m,1.2,12-10-2012,false
PH_locB,H.m,1.3,13-10-2013,true
PH_locB,H.m,1.4,14-10-2014,false
PH_locB,H.sim,1.5,15-10-2015,true
PH_locB,H.sim,1.6,16-10-2016,false
PH_locB,H.sim,1.7,17-10-2017,false
PH_locB,H.sim,1.8,18-10-2018,true

forecastAvailableThresholdValue

is used to log the info that the related forecast run was available within a certain time window when the observed threshold has been crossed.

deprecatedConversionChoice

stageDischargeConversion

since 2010.01, for NEW style rating curve types. Used when stages are specified for the threshold levels and there are discharges in (some of) the specified time series sets.

dischargeParameterId

Discharge parameters for which the threshold values should be converted from stage to discharge.

dischargeStageConversion

since 2010.01, for NEW style rating curve types. Used when discharges are specified for the threshold levels and there are stages in (some of) the specified time series sets.

stageParameterId

Stage parameters for which the threshold values should be converted from discharge to stage

ratingCurve

For OLD style rating curve types. Convert the threshold levels from stage to discharge or vice versa using the rating curve defined here.

id

Rating curve id to use for threshold mapping.

ratingCurveType

Type of rating curve

parameterId

Parameter ids of the time series sets for which the conversion should take place.

Example Configuration Files

The following example shows the configuration of a thresholdValueSet where water level time series are checked against up and down crossing events. When a threshold is crossed up or down crossing events will be logged. The optional element standDownIndividualLocations is set to True, the threshold module will then treat the up and down events for the individual locations instead of catchments or regions. All threshold values and labels are configured using location attributes, from the metadata CSV/DBF files.  

Another example is showing the configuration for rainfall data. Different functionality is included in the thresholdValueSet shown below. The timeseriesSet is a 15 minute rainfall set for all raingauges in a country. The rateThresholdValue element will check 24 hour accumulated rainfall totals and check these total values against a location attribute value @IFD_10Y_24h@. When this total value is crossed for a location, a log messages and event action message is logged. This event action has a grace time of 1 day, this means that if more 24 hour total values are crossed within a 24 hour period, no new event crossing messages for this location will be logged.

The aggregationLevelThresholdValues are used in the Spatial Display. When the accumulation slider computes new aggregated values, the correct thresholdvalue will be used for the aggregated time period.