Time Dependent Locations
In the case that FEWS is used as a forecasting system the locations don't need to be time dependent in the majority of the cases. The time span of a forecasting system is usually several weeks.
For such a short time span the configuration of the locations can be static and if something changes in one of the forecasting locations this can be changed in the FEWS system without keeping track of how the situation was before the change.
However when FEWS is used for a Water Information System (WIS) it is important that FEWS also has knowledge of how the location configuration was in the past. In a WIS the time span is much larger and can be as large as 10 or 20 years.
To make it possible to reproduce the time series generated in the past it is necessary that FEWS still has knowledge about how the location configuration was in the past.
The chapter will describe how it is possible to configure time dependent locations. It is also very important to note that only the functionality which is described in this chapter can be considered time dependent.
Configuration of time dependent locations
What exactly is a time dependent location? A time dependent location has a start- and an endtime, attributes, but also the related locations of the location can vary in time.
A set of time dependent locations can be configured in the LocationSets.xml file. Below is an example file.
<locationSet id="example"> <csvFile> <file>myLocationFile</file> <geoDatum>Rijks Driehoekstelsel</geoDatum> <id>%ID%</id> <name>%NAME%</name> <dateTimePattern>yyyy-MM-dd HH:mm:ss</dateTimePattern> <startDateTime>%START%</startDateTime> <endDateTime>%EIND%</endDateTime> <x>%X%</x> <y>%Y%</y> <relation id="relation"> <relatedLocationId>%REL%</relatedLocationId> </relation> <attribute id="PARAMETERS"> <text>%PARAMETERS%</text> </attribute> <attributeFile> <csvFile>attribute.csv</csvFile> <id>%ID%</id> <timeZoneOffset>+00:00</timeZoneOffset> <dateTimePattern>yyyy-MM-dd HH:mm:ss</dateTimePattern> <startDateTime>%START%</startDateTime> <endDateTime>%EIND%</endDateTime> <attribute id="Q"> <number>%Q%</number> </attribute> </attributeFile> </csvFile> </locationSet>
The tags startDateTime and endDateTime relate to the columns in the file mapLayersFiles/myLocationFile.csv. This csv file defines which locations are in the locationSet example.
Besides defining the locations in a csv file it is also possible to define the locations in a dbf file.
The column ID defines the id of the location. The column NAME defines the name. The tag startDateTime refers to the column START.
The tag endDateTime refers to the column EIND. Together the startDateTime and endDateTime define for which period this location is valid.
The tag relation with id "relation" defines a related location. The value is stored in the column REL. The value should be the location id of the related location.
But, what if this relation changes over time? How should such a change be configured? This will be explained by an example.
If, for example, the related location of location A changes over time then this should be configured in the following way.
In this example we will assume that location A has a related location B until 1-1-2014.
After this date the related location will change to location C.
To configure this the following csv (or dbf) file should be used:
ID | NAME | X | Y | START | EINDE | PARAMETERS | REL |
1 | A | 12345 | 54321 | 1-1-2000 | 1-1-2014 | P1 | B |
1 | A | 12345 | 54321 | 1-1-2014 | P1 | C | |
2 | E | 11223 | 32211 | 1-1-2000 | P2 | M | |
3 | X | 12321 | 12321 | 1-1-2003 | 1-1-2010 | P2 | F |
First a row should exists in the csv (or dbf) file of this locationset with the value A in the column ID and the column which defines the relation (REL) should have the id
B of location B. The EIND column should indicate that this is valid until 1-1-2014. The START column should indicate the start date of this relation. A second row with the same value A for the ID column should also exist. The START column should indicate that this row is valid from 1-1-2014. The EIND column should indicate the end of this relation or should be left empty if there is no end date known for this relation.
The column REL should have the value C to indicate that the related location is C.
In the case that the related location of a location can vary over time but that a lot of attributes stay constant, it is useful to define the constant attributes in a separate attribute file. In the example above an attribute file (attribute.csv) is configured. This file should be located in the same directory (mapLayersFiles) as the file myLocationFile.csv.
However is some cases the attributes also vary over time but vary independent of the of location they belong to. The location might for example change its relation only once while some attributes are changed every year. If this is the case then one should also place these attributes in a seperate csv (or dbf) file. The way the start- and end time of the attributes are configured is comparable to how the start- and end time of the location itself is configured.
The element csv file contain an element startDateTime and endDateTime which define the columns which contain the start and endtime of a single row of attribute definitions.
The configuration of the start time and end time of a location and its attributes must be configured according to the following rules:
- The period in which the location exists in FEWS (visibilty period) is derived from the main configuration file in the locationSet configuration
- The periods defined in the attributes file should be consistent with the visibility period defined for the location. If all the periods defined in the attributefile for a certain location were combined they should span the exact period if all the periods defined in the main configuration file were combined to a period. If this is not the case an configuration error will be thrown during the startup of FEWS and the behaviour of the system with regard to time dependent locations is undefined.
Grid display
The grid display supports the usage of time dependent locations and polygons. The time displayed in the grid display can be varied by using the time slider at the top of the display. In a FEWS system were locations are not time
dependent, the locations and polygons shown will not change when another time is selected. However when locations are time dependent, the locations and polygons which are shown in the display will change when another time is selected.
The polygons of the locations of the time series shown in the display will vary automaticly over time when locations are configured to be time dependent. The map layers can be time dependent. An example configuration within the GridDisplay.xml is shown below.
<esriShapeLayer id="Peilgebieden"> <file>peilgebieden_example</file> <visible>true</visible> <selectByMapItemLocationRelation> <relationId>PEILGEBIED</relationId> <locationId>%GPGIDENT%</locationId> </selectByMapItemLocationRelation> <toolTip>%GPGNAAM%</toolTip> <timeZoneOffset>+01:00</timeZoneOffset> <dateTimePattern>dd-MM-yyyy</dateTimePattern> (optional) <startDateTimeAttributeName>STARTDATE</startDateTimeAttributeName> <endDateTimeAttributeName>ENDDATE</endDateTimeAttributeName> </esriShapeLayer>
When a user double clicks on a spot in a shape in the grid display, the plots dialog will automatically open and the time series of the selected polygon will be shown. But with the introduction of time dependent locations it is possible that the polygon which overlaps a selected geographical point varies over time. If this is the case and the user double clicks on a spot where the overlap in time occurs, the time series shown in the plots display will also vary over time, so multiple time series will be visible for the total period.” The MapToolTip (button "tooltip") also takes into account that locations and the related polygons can vary over time.
Filters and Map display
By default the map display and filters will not take into account that the locations shown are time dependent but show the situation as it is at the display time. In the explorer.xml it is possible to configure that the map display shows a time slider when locations are shown on the map which are time dependent. Activating the time slider should be considered as going into a sort of special mode which allows to view how locations were configured in the past.
<locationHistoryBarRelativePeriod unit="week" start="-100" end="100"/>
The screenshot below shows the slider button which appears when the locationHistoryBarRelativePeriod is configured in the explorer.xml
After pressing this button a time slider appears.
The time slider will show a time mark for each time there was a change for a location (in the view period). The default view period will be the period configured in the explorer.xml. The view period can be changed by selecting the button "Period history time slider" (button with the clock and arrows).
The t0 can be set to the selected time in the time slider by the button "Set t0 to the selected time". If a location is not visible at the selected time in the time slider, the location will still be shown in the map and the filter, but will be greyed out to indicate that it is not valid at the selected time. For performance reasons the icons in the filters will not be updated according to the selected time when the time slider is shown in the map display. The MapToolTip (button "tooltip") and MapItemSelectTool (button "select locations with lasso tool") take into account the selected time in time slider. If the time slider is not selected the system time is used.
In typical WIS systems users still want to have access to the data although the locations has been "stopped" by the provided stoptime. Therefore, the option <showLocationsOutsideVisibilityPeriod>true</showLocationsOutsideVisibilityPeriod> can be defined in Explorer.xml to show all locations no matter the start-/stoptime in de Filters. Locations outside of the defined start-/stoptime will be marked using black/gray crosses instead of the typical red/darkred ones for missing data.
Filter change events
Since 2016.02 a filter button has been added which lets the user choose which change events should be included. The change events are grouped by type of change events recognized in Delft-FEWS: Map Layers, Locations, Location Attributes, Location Relations;
Together with the context menu option for location filter "Show Config Diff for Selected Time" (see screenshot below) it will show a clear overview of the locations that have changed on that time step:
By default only the start and end time of the location are included, but this can be changed by (de)selecting any number of checkboxes.
The number of time steps in the slider will change accordingly:
Time dependent map layers
The map layers in the map display can be made time dependent in the same way as the map layers in the grid display (configuration in the SystemConfigFiles/Explorer.xml file). The example in the section Grid display shows how this can be done.
Transformations
The transformation module automatically takes into account the time dependency of the locations. No configuration is needed in the transformation module. If an end time is introduced for a location and time series are in the database for this location after the end time, then the part of the time series which is not valid anymore will not be read from the database. The benefit of this approach is that the configurator doesn't need to remove obsolete parts of time series for the location from the database.
Note: the usage of time dependent location relations and time dependent location set constraints is not supported in the transformation module before 2020.01 because this requires to switch the location itself during the transformation. This will be supported from 2020.01 onwards. This is not supported for all transformations. Transformations that do not support this will mainly be that when multiple time steps are necessary for the calculation in their input and / or output (like aggregation and accumulation). This is because the individual transformation can not handle a change in within their input or output period. Exceptions to this rule have been made after analysis of their functionality:
- <interpolationSerial><default>
- <merge><simple>
- <sample><equidistant>
Secondary validation
The secondary validation module also takes into account the time dependency of the locations. The secondary validation will only run if all of the used locations are valid, the values of location attributes used in the
expression can be time dependent and are always resolved for the time step for which the validation is being done.
<seriesComparisonCheck id="timeDependentSeriesComparisonCheck"> <expression>output - A1 >@A@</expression> <validatingVariableId>output</validatingVariableId> <outputFlag>unreliable</outputFlag> <logLevel>DEBUG</logLevel> <logEventCode>SecondaryValidation.locationSeriesComparisonCheckUnreliable</logEventCode> <logMessage>%AMOUNT_CHANGED_FLAGS% flags set to %OUTPUT_FLAG% by [%CHECK_ID%, %EXPRESSION%].</logMessage> </seriesComparisonCheck>
However it is important to explain how the time dependency of locations influences resolving location attributes.
In the example above, the expression contains one location attribute A. But which location should be used to resolve this location?
Both variable output and variable A1 provide a location which could be used to resolve the attribute. We chose to always use the value of the attribute of the output location in the expression.
Using the input is not very logical and in addition it would still not be clear which location to use in the case that the input consists of time series with more than one location.
What should be done in the case that more than one validating variable is configured? In that case the validation is being done for one validating variable at the time.
If 2 validating variables are being used, lets say for example A and B, then the secondary validation will first run for validating variable A and validate the time series defined by this variable.
In that case the location of variable A will be used to determine the value of the location attributes. After that, the secondary validation will run for validating variable B which will mean that the location of this variable will be used.
Time dependent location relations and time dependent location set constraints. (since 2022.01)
Since 2022.01 the Secondary Validation module has improved for handling time dependent location relations and location sets based on time dependent constraints.
The special thing about these time dependencies are that at different times a time series set contains different time series. This goes even further than location with start and or end time because here locations can still exist but actually at one point in time be part of a time series set and at another time not anymore.
For location sets based on a time dependent constraint and time dependent location relations it is needed to split the run period over all time dependent changes within that period and resolve all time series for each sub period individually.
Before 2022.01 the time series are only resolved for T0, this causes some validations to be executed for parts of time series that are actually not part of the time series set at a certain time, or parts of a time series not being validated while it is actually part of a time series set.
Chainage Location Sets (since 2020.02)
Since 2020.02 it is possible to use time dependent chainage attributes to determine a longitudinal profile.
The longitudinal profile will be visualized with all chainage locations that are part of the profile at any time in the period of the time slider.
When at a specific moment in time a location is not part of the profile a missing value will be shown in the table, but the graph will just connect the line to the next location that is part of the profile.
The