Page tree

 

 

 

 

 

 

 

 

 

 

 

 

DELFT-FEWS

 

 

 

Configuration Guide

 

Version 1.1

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

December 2006


 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

CLIENT :

 

 

 

 

TITLE :

 

 

Delft FEWS Configuration Guide

 

 

 

 

 

REFERENCES :

 

 

 

VER.

ORIGINATOR

DATE

REMARKS

REVIEW

APPROVED BY

0.1

Kappel

 

3 September 2004

FAT

Heynert

 

Ruijgh

 

0.2

Heynert

 

27 September 2004

FAT upd

 

 

 

 

0.3

Kappel

 

24 January 2005

quick upd

Heynert

 

 

 

0.4

Werner

 

4 February 2005

draft final

van Dijk

 

Ruijgh

 

0.5

Werner

 

20 February 2005

draft final

van Dijk

 

Ruijgh

 

0.6

van Dijk

 

25 February 2005

draft final

Werner

 

Ruijgh

 

1.0

van Dijk

 

September 2006

final

Werner

 

Ruijgh

 

1.1

Boot, Ververs

 

22 December 2006

update

Werner

 

Ruijgh

 

PROJECT IDENTIFICATION:

 

KEYWORDS:

, Flood Forecasting

NUMBER OF PAGES

286

CONFIDENTIAL:

YES, until (date)

NO

STATUS: PRELIMINARY DRAFT FINAL

 

 

 


Preface

The following documentation is provided for Delft FEWS:

 

  • General Documentation

NFFS Overview Manual

  • User Documentation…

Delft FEWS User Guide

  • System Configuration Documentation

Delft FEWS Configuration Guide

  • System Manager’s Documentation

System Installation Manual

System Support Manual

Admin Interface User Guide

 

The Delft FEWS Configuration Guide is presented in this document.

 

Guide to the reader

The DELFT-FEWS Configuration Guide provides the advanced user of DELFT-FEWS with the information required to set-up and maintain a configuration of DELFT-FEWS. The objective of the guide is to be used both as a reference manual during the development and maintenance of an implementation of DELFT-FEWS, as well as to provide some of the background philosophy on how to go about setting up a forecasting system. It is expected that the reader of this guide has a basic understanding of DELFT-FEWS and its structure.

 

New in this version

The following sections have been updated:

  • Error Correction Module configuration
  • Report Module configuration

 



Contents

wlL1 wlL2 wlL3 wlL4

1 Introduction ............................................................................................................................

2 Structure of DELFT-FEWS configuration .....................................................................

2.1 Introduction ................................................................................................................

2.2 Elements of the configuration ..................................................................................

2.3 Versions of configuration and XML file naming conventions ...........................

2.4 XML Schemas and schema validation ...................................................................

3 Data handling in DELFT-FEWS .......................................................................................

3.1 Introduction ................................................................................................................

3.2 Types of time series ..................................................................................................

3.3 Time Series Sets ........................................................................................................

4 System Configuration ...........................................................................................................

4.1 Introduction ................................................................................................................

4.2 FEWS Explorer ..........................................................................................................

4.2.1 System Information .....................................................................................

4.2.2 Explorer Options .........................................................................................

4.2.3 Zoom extents ................................................................................................

4.2.4 Explorer Tasks .............................................................................................

4.2.5 statusBar .......................................................................................................

4.2.6 restSettings ...................................................................................................

4.2.7 logPanelConfig ............................................................................................

4.3 Time Series Display Configuration ........................................................................

4.3.1 General Display Configuration .................................................................

4.3.2 Default view period ....................................................................................

4.3.3 Time Markers Display Configuration ......................................................

4.3.4 Parameters display configuration .............................................................

4.3.5 Module instance mapping ..........................................................................

4.4 Display Groups ..........................................................................................................

4.5 Location Icons ............................................................................................................

4.6 Module Descriptors ...................................................................................................

4.7 Display Descriptors ...................................................................................................

5 Regional configuration .........................................................................................................

5.1 Introduction ................................................................................................................

5.2 Locations .....................................................................................................................

5.3 Location Sets ..............................................................................................................

5.4 Parameters ..................................................................................................................

5.5 Branches ......................................................................................................................

5.6 Grids ............................................................................................................................

5.6.1 Regular grids ................................................................................................

5.6.2 Irregular grids ..............................................................................................

5.7 Filters ...........................................................................................................................

5.8 ValidationRuleSets ....................................................................................................

5.8.1 Validation on extreme values ....................................................................

5.8.2 Validation on rate of change ......................................................................

5.8.3 Validation on series of same readings .....................................................

5.8.4 Validation on Temporary Shifts ................................................................

5.9 Thresholds ..................................................................................................................

5.10 ThresholdValueSets ..................................................................................................

5.10.1 Defining level thresholds ...........................................................................

5.10.2 Defining rate thresholds .............................................................................

5.10.3 Defining peak event thresholds .................................................................

5.11 ColdModuleInstanceStateGroups ...........................................................................

5.12 ModuleInstanceDescriptors .....................................................................................

5.13 WorkflowDescriptors ................................................................................................

5.14 IdMapDescriptors ......................................................................................................

5.15 FlagConversionDescriptors ......................................................................................

5.16 UnitConversionsDescriptors ....................................................................................

5.17 CorrelationEventSetsDescriptors ............................................................................

5.18 TravelTimesDescriptors ...........................................................................................

5.19 TimeUnits ...................................................................................................................

5.20 Historical Events ........................................................................................................

5.21 Value Attribute Maps ................................................................................................

6 Module Instances ...................................................................................................................

6.1 Introduction ................................................................................................................

6.2 Interpolation Module Configuration .......................................................................

6.2.1 Serial interpolation ......................................................................................

6.2.2 Spatial interpolation ....................................................................................

6.3 Transformation Module Configuration ..................................................................

6.3.1 ArithmeticFunction & hydroMeteoFunction ..........................................

6.3.2 Stage-Discharge and Discharge-Stage transformation ..........................

6.3.3 Establishing catchment average precipitation .........................................

6.3.4 Aggregation, disaggregation and non-equidistant to equidistant .........

6.3.5 Rule based transformations .......................................................................

6.4 Import Module Configuration .................................................................................

6.4.1 Time Series Import Module .......................................................................

6.4.2 EA Import module ......................................................................................

6.5 Export Module Configuration .................................................................................

6.6 General Adapter Configuration ...............................................................................

6.6.1 General settings ...........................................................................................

6.6.2 Startup Activities .........................................................................................

6.6.3 Export Activities .........................................................................................

6.6.4 Execute Activities .......................................................................................

6.6.5 Import Activities .........................................................................................

6.6.6 Shutdown Activities ....................................................................................

6.7 Lookup Table module configuration ......................................................................

6.7.1 criticalConditionLookup ............................................................................

6.7.2 SimpleTableLookup ...................................................................................

6.7.3 MultipeDimensionLookup .........................................................................

6.8 Correlation Module Configuration .........................................................................

6.8.1 CorrelationEventSets ..................................................................................

6.8.2 TravelTimesSets ..........................................................................................

6.9 Error correction module configuration ...................................................................

6.10 Report Module Configuration .................................................................................

6.10.1 Configuring formatting of reports ............................................................

6.10.2 Configuring content of reports ..................................................................

6.10.3 Charts ............................................................................................................

6.10.4 Summary ......................................................................................................

6.10.5 Tables ............................................................................................................

6.10.5.1 htmlTable ......................................................................................................

6.10.5.2 thresholdsCrossingsTable ..........................................................................

6.10.5.3 maximumStatusTable .................................................................................

6.10.5.4 mergedPrecipitationTable ..........................................................................

6.10.6 SystemStatusTables ....................................................................................

6.10.6.1 liveSystemStatus .........................................................................................

6.10.6.2 exportStatus table ........................................................................................

6.10.6.3 importStatus table .......................................................................................

6.10.6.4 scheduledWorkflowStatus table ................................................................

6.10.6.5 completedWorkflowStatus table ...............................................................

6.10.6.6 currentForecastStatus table ........................................................................

6.10.6.7 logMessageListing table ............................................................................

6.10.6.8 forecastHistory table ...................................................................................

6.10.7 Report Export Module Configuration ......................................................

6.11 Performance Indicator module ................................................................................

6.11.1 Assessing performance of modules ..........................................................

6.11.2 Assessing performance of forecast values – lead time accuracy .........

6.11.3 Assessing performance of forecast values – timing of thresholds .......

6.11.4 Configuration of performance module .....................................................

6.12 Amalgamate Import Data module ...........................................................................

6.13 Archive Forecast Module .........................................................................................

6.14 Rolling Barrel Module ..............................................................................................

6.15 Support Locations Module Configuration .............................................................

6.16 Scenario Module Configuration ..............................................................................

7 Configuring Workflows ........................................................................................................

7.1 Introduction ................................................................................................................

7.2 Workflows ..................................................................................................................

8 Display configuration ............................................................................................................

8.1 Introduction ................................................................................................................

8.2 Grid display ................................................................................................................

8.2.1 Definition of class breaks ...........................................................................

8.2.2 Definition of background maps ................................................................

8.3 Longitudinal display .................................................................................................

8.4 What-If scenario display ...........................................................................................

8.5 Lookup table display .................................................................................................

8.6 Correlation display ....................................................................................................

9 Mapping ID’s, flags and units .............................................................................................

9.1 Introduction ................................................................................................................

9.2 IdMaps .........................................................................................................................

9.3 UnitConversions ........................................................................................................

9.4 Flag Conversions .......................................................................................................

10 Module Datasets and Module Parameters ......................................................................

10.1 Introduction ................................................................................................................

10.2 Module Datasets ........................................................................................................

10.3 Module Parameters ....................................................................................................

11 Setting up an operational system .......................................................................................

11.1 Introduction ................................................................................................................

11.2 Root configuration files ............................................................................................

11.2.1 clientConfig ..................................................................................................

11.2.2 LogConfig ....................................................................................................

11.2.3 synchConfig .................................................................................................

11.2.4 synchProfiles ................................................................................................

11.2.5 synchChannels .............................................................................................

11.3 Setting up scheduled forecasts .................................................................................

11.4 Setting up event-action configurations ...................................................................

12 Setting up a forecasting system ..........................................................................................

12.1 Introduction ................................................................................................................

12.2 What is required when setting up a DELFT-FEWS application? ......................

12.3 Designing the Forecasting System ..........................................................................

12.4 Creating a FEWS application directory .................................................................

12.5 Adding static data, configure the FEWS static configuration files ....................

12.5.1 Adding Workflows and Module Instances ..............................................

12.5.2 Configuring FEWS displays ......................................................................

13 Configuration management tool ........................................................................................

13.1 Introduction ................................................................................................................

13.2 Managing configurations ..........................................................................................

13.2.1 Download a configuration from a Master Controller ............................

13.2.2 Import configuration files ..........................................................................

13.2.3 Make a selected configuration file the active file ..................................

13.2.4 Delete a configuration file .........................................................................

13.2.5 Export a configuration file .........................................................................

13.2.6 Upload to Master Controller ......................................................................

13.2.7 Changing an existing configuration file ..................................................

13.3 Validation of a Configuration ..................................................................................

13.3.1 Primary Validation ......................................................................................

13.3.2 Secondary Validation (internal dependencies validation) ....................

13.4 Analysis of a Configuration .....................................................................................

13.4.1 Approach ......................................................................................................

13.4.2 Implementation ............................................................................................

13.4.3 Using the Configuration Manager Analysis Tool ..................................

14 Additional Modules ...............................................................................................................

14.1 Introduction ................................................................................................................

14.2 Flood Mapping Module ............................................................................................

14.2.1 Data requirements .......................................................................................

14.2.2 Data preparation ..........................................................................................

14.2.3 Configuring the Flood Mapping Module ................................................

14.2.4 Running the flood map module .................................................................

15 Tips and Tricks .......................................................................................................................


List of Appendices

 

A Colours available in DELFT-FEWS .................................................................................

B Enumerations ..........................................................................................................................

B.1 GeoDatum ...................................................................................................................

B.2 Time Zones .................................................................................................................

B.3 Units ............................................................................................................................

B.4 Data quality flags .......................................................................................................

B.5 Synchronisation Levels .............................................................................................

C Map component configuration ...........................................................................................

D Report configuration .............................................................................................................

D.1 Introduction ................................................................................................................

D.2 Types of reports .........................................................................................................

D.3 How is a report created .............................................................................................

D.4 Report Template ........................................................................................................

D.4.1 Tags ...............................................................................................................

D.5 Report Module Configuration .................................................................................

D.5.1 General ..........................................................................................................

15.1.2 Configuring the Declarations section .......................................................

D.1.1 Configuring the Report section .................................................................

D.1.2 Controlling the destination of a report .....................................................

E Glossary ....................................................................................................................................

  wlListFigs   wlListTabs wlListPhotos   wlListSymb   wlSummary

 

 

 

 



1                       Equation Section 1 wlStartOfText Introduction

The DELFT-FEWS configuration guide provides the advanced user of DELFT-FEWS with the information required to set-up and maintain a configuration of DELFT-FEWS. The objective of the guide is to be used both as a reference manual during the development and maintenance of an implementation of DELFT-FEWS, as well as to provide some of the background philosophy on how to go about setting up a forecasting system. It is expected that the reader of this guide has a basic understanding of DELFT-FEWS and its structure.

 

To understand how to configure DELFT-FEWS, a good understanding of the structure of the configuration is required. The first part of this guide therefore gives an introduction into the different parts of configuration. In the second part of the guide the concepts used in DELFT-FEWS in handling data are explained.

 

The third part of the guide is dedicated at the different configurations in DELFT-FEWS. Chapter 6 provides an overview of the available modules in DELFT-FEWS and how each of these can be configured to provide the required functionality. Chapter 7 explains how configured modules are linked into logical tasks through configuration of workflows. In chapter 8 the configuration of user displays is discussed. Chapter 9 discusses mapping information from external data sources. Chapter 10 discusses handling of static module data. In chapter 11 some elements of configuring DELFT-FEWS as an operational system are discussed.

 

In the fourth section a brief introduction is given on how to set-up a forecasting system. This is to give some idea on how to approach configuring a complex system such as DELFT-FEWS. A brief guide in the use of the configuration management module to support configuration is given in chapter 13. Chapter 14, finally, gives some tips and tricks on configuring DELFT-FEWS. Throughout the text, relevant details to be taken into account in configuration is given in text boxes.

 



2                       Structure of DELFT-FEWS configuration

2.1                Introduction

The configuration of DELFT-FEWS is defined in a set of XML files. In this section the different parts of the configuration are introduced. An understanding of these different parts of the configuration is required before attempting configuration of a DELFT-FEWS system.

 

When dealing with the configuration, it is important to note that the configuration may be retrieved from the local file system or from the local database.

 

  • In the former case the configuration is a set of directories, each containing different parts of the configuration. These directories are all contained under the Config directory.
  • In the latter case the configuration is contained entirely within the local database in a set of tables, each table containing different parts of the configuration. Each of the tables in the local database reflects one of the sub-directories in the file system.

 

When initiating DELFT-FEWS, it will first look for configuration stored on the local file system. If this is found, then the system will expect to use all configuration from the file system. If the Config directory is not available the system will expect to use all configuration from the database. If neither is found then an appropriate error message is issued and the system will stop.

 

The configuration stored in either the Config directory of the database is configuration that is common to all versions of an implementation of DELFT-FEWS for a particular forecasting system. In the live system situation the contents of the database will be synchronised between all operator clients and forecasting shell servers in the system, and is therefore expected to be identical in all parts of the system.

 

Besides this configuration that is synchronised, there is also a small set of XML files referred to as the root configuration files. These may be unique to each operator client and/or forecasting shell server. This root configuration is required to identify for example if the particular instance of DELFT-FEWS is operating in stand-alone mode or as an operator client, and for the latter cases information such as IP-addresses for the Master Controller the operator client should log on to. These root configuration files are always used from the file system. They have no effect on the hydrological configuration and are normally not changed during configuration of the forecasting system side of DELFT-FEWS.

2.2                Elements of the configuration

The two tables below provide an overview of the configuration elements of DELFT-FEWS. In the first table the configuration contained both in the database and on the file system is described. The second table describes the configuration that is only available on the file system.

 

Table 1 Overview of different configuration items contained either in the config directory or in the database

Configuration Item

Directory on File System

Table name in file system

Single/
Multiple

Definition of regional configuration, including all locations, parameters etc.

RegionConfigFiles

RegionConfigurations

Single

Definition of system configuration items, including the plug-ins available to the system, definition, icons etc.

SystemConfigFiles

SystemConfigurations

Single

Definition of modules for handling data and running forecasting models

ModuleConfigFiles

ModuleInstanceConfigs

Multiple

Definition of workflows for running sequences of modules

WorkflowFiles

WorkflowFiles

Multiple

Definition of mapping of ID’s and parameters between external sources (e.g. telemetry, modules) and ID’s and parameters defined in the DELFT-FEWS configuration

IdMapFiles

IdMaps

Multiple

Definition  of unit conversions between external sources (e.g. telemetry, modules) and units used in DELFT-FEWS

UnitConversionFiles

UnitConversions

Multiple

Definition of flag conversions between external sources (e.g. telemetry, modules) and flags used in DELFT-FEWS

FlagConversionFiles

FlagConversions

Multiple

Definition of layout of user displays, including What-if scenarios, Grid Display etc.)

DisplayConfigFiles

DisplayConfigurations

Multiple

Definition of module parameters stored in DELFT-FEWS

ModuleParameters

ModuleParameters

Multiple

Zipped files containing datasets for modules used by the forecasting system.

ModuleDataSetFiles

ModuleInstanceDatasets

Multiple

Definition of HTML template files used in creating HTML reports for use on the web server.

ReportTemplateFiles

ReportTemplates

Multiple

 


Table 2 Overview of different configuration items contained in the file system only

Configuration Item

Directory on File System

Root Configuration. Several XML files describing some of the settings specific to the Operator Client used (e.g. client configuration, IP addresses)

These files are contained in the root of the DELFT-FEWS configuration directory

2.3                Versions of configuration and XML file naming conventions

For each of the configurations managed by DELFT-FEWS in either the database or on the file system as described above, various versions of configuration may exist.

 

Configurations that are active and used as a default can be identified both in the file system and in the database.

 

On the file system a naming convention is introduced to identify which of the possible multiple versions are used as a default.

 

The naming convention for the default version:

<Name of XML configuration file>SPACE<Version number>SPACE<default>.xml

 

Other version of configuration will have a different version number. The <default> item is omitted.

 

Example:

 

The default version of the configuration settings for the FEWS Explorer could be:

Explorer 1.00 default.xml

 

A second version may exist. This should then not have the <default> item in the file name:

Explorer 2.00.xml

 

If the configuration does not include the “default” item it will not be used. This configuration may be reverted to by adding the “default” flag – and removing it from the other file.

 

In the database the default version for each configuration item is identified in an associated table. For each configuration item a default table is available. This is identified by a table with the same name, prefixed by “Default”. For example for the SystemConfigurations a table with the name DefaultSystemConfigurations identifies which of the available versions in the former table is to be used a default.

2.4                XML Schemas and schema validation

Each configuration item contained in an XML file must be formatted as specified in an appropriate XML schema (XSD file). Validating against the schemas is an important step in configuring DELFT-FEWS, as the primary validation makes sure the syntax of the configuration made is correct.

 

There are two types of configuration in DELFT-FEWS. In the first set, for each different schema type, only one default configuration file may be used and the name of the configuration file is unique. For the second set of configuration, multiple configuration types may be available for a specific schema. The names of these may be defined by the user. An XML file contained in the regional configuration element is then used to register these XML files with a user specified name to the system, and identify the type of configuration. This file is referred to as a descriptor file.

 

Table 1 identifies for which type of configuration a single files per type is allowed and for which multiple instances for each type of configuration may exist.

 


3                       Data handling in DELFT-FEWS

3.1                Introduction

One of the most important properties of DELFT FEWS as a forecasting system is its ability to efficiently deal with large volumes of dynamic data. Dynamic data covers mainly time series data in various formats (scalar- 0D, vector – 1D, grid – 2D, and polygon data – 2D). Dynamic data also includes the management of model states produced by the system.

 

A thorough understanding of how DELFT-FEWS handles dynamic data is fundamental in the correct configuration of an operational system. For each of the different types of dynamic data specific optimisations have been introduced.

 

To allow handling of time series data, the concept of a “Time Series Set” is introduced. A Time Series Set is used to retrieve and submit data to the database. In this chapter the concept of the time series set is explained.

3.2                Types of time series

External and Simulated time series

Time series are considered to be available from two sources. All time series sourced from external systems are considered as “External”. All time series produced by the forecasting system itself are considered as “Simulated”.

Forecast and Historical time series

Time series are considered to be of two categories in relation to time. Historical time series are continuous time series that describe a parameter at a location over a period of time. Forecast time series are different to historical time series in that for each location and parameter one forecast is independent of another forecast. A forecast is characterised by its start time and the period it covers. Generally when a new forecast is available for a given location and parameter it will supersede any previous forecast for that location parameter. Each forecast is therefore an independent entity.

 

On the basis of this, four categories of time series are identified;

  • External Historical
  • External Forecasting
  • Simulated Historical
  • Simulated Forecasting

 

There are significant differences in how each of these time series are handled.

External Historical time series

In an online system DELFT-FEWS will incrementally import observed data as it becomes available from external systems. This data should be imported as an External Historical time series. When data marked as external historical is presented to the system with exactly the same values and covering the same period as data for that location/parameter already available in the database then it will be ignored. Only new data is imported and stored. If data for a given period is already available but is changed (manual edit or update), then the new values will be added to the database. For each item of data added to the database, a time stamp is included to specify when the data was made available to the system.

 

When data of the external historical type is requested from the database, the most recently available data over that whole period is returned. If the data for that period was imported piecewise, then the individual pieces will be merged prior to the data being returned. An example is given in Figure 1 where data is imported sequentially. Each data imported/edited is indicated using a different line style. At the request for the complete series (a) the most recent data available over the complete period is merged and returned. The data imported at 12:00 partially overlaps that imported at 10:00. As the 12:00 data is the most recent, it will persist in the complete series. A manual edit may be done (or interpolation) to fill the gap in the data. This will be returned in a subsequent request for the complete series. Although a complete series is returned, the data is stored as it is imported, including a time stamp indicating when the import happened. If at a later stage the data available at directly preceding the manual edit is requested, then the additional data will not be included in the complete series.

 

Figure 1 Schematic representation of data imported as external historical

External Forecasting time series

External forecasts are imported by DELFT-FEWS as these are made available by the external forecasting systems. Again each forecast is imported and stored individually. External forecasts are referenced by the start time of that forecast. When retrieving an external forecast time series from the database, the most recent available forecast, as indicated by the forecast start time will be returned. The most recently available forecast is determined as the latest forecast with a start time earlier or equal to the start of the forecast to be made using DELFT-FEWS (forecast T0). It is thus not possible to see an external forecast time series on request, as the latest available is always returned.

 

With possible exceptions for modules considering multiple forecasts (e.g. performance module), only one external forecast is returned. Different external forecasts are not merged.

 

Simulated Historical time series

Simulated historical time series are similar to the external historical time series in that they are continuous in time. The difference is that the time series are referenced through the forecast (model) run they have been produced by. As a consequence the time series can be retrieved either by directly requesting it through opening the run and viewing, or if the run is approved.

 

Simulated historical time series are generally produced by model runs where a model initial state is used. Each time series has a history, i.e. the state used as its initial condition. Each state again has a history, i.e. the model run that produced the state. This history is used by the database in constructing a continuous time series.

Simulated Forecasting time series

Simulated forecast time series are again similar to external forecasting time series. Again the main differences is that they are referenced through the forecast (model) run they have been produced by. As a consequence the time series can be retrieved either by directly requesting it through opening the run and viewing, or if the run is approved. Simulated forecast time series are treated in the same way as the external forecast time series in that the last approved forecast (referred to as the current forecast) is seen as a default. All other runs can be seen on request only. Note that the last approved forecast which is shown by default may not be the last available forecast.

Figure 2 schematically shows how a sequence of runs producing simulated historical and simulated forecasting time series are stored. Each simulated historical run uses the module state saved at the end of the previous run. It can be seen that these simulated historical traces are treated as a continuous time series when requested later. For the forecasting time series, only the most recent (approved) time series is displayed.

Figure 2 Schematic overview of handling simulated forecasting and simulated historical time series. Three subsequent forecasts are shown, and the resulting complete time series returned when requested after 12:00. The historical time series is traced back using the state used to create the link to a previous run. For the forecast time series the most recent forecast supersedes previous forecasts.

3.3                Time Series Sets

Any module in DELFT-FEWS that requires data from the database, or produces data that must be stored in the database, does so through the use of a complex data type referred to as the Time Series Set. A time series set can be compared to a query that is run against the database. It contains all the keys to uniquely identify the set of data to be retrieved.

 

Time series sets form a large part of the configuration. Most modules have a standard structure, where the configuration starts with a request of specific set of data from the database using one or more input time series sets, a number of functional items which describe how the data is transformed, and one or more output time series sets which are used to store the data in the database under a unique combination of keys.

 

Figure 3 shows the elements of the Time Series Set complex type. a number of these elements are compulsory (solid borders), while other elements are optional (dashed borders). If any of the required elements is omitted then the primary validation of that configuration will fail.

 

Depending on the information required, each of the elements will be used differently. Some elements are simple flags, indicating specific properties of the time series set if they are present. For others a string or value must be given in the configuration. In code this will mean that value or string is assigned to a variable with the name of the element. Other elements may also contain properties. The example of the time series set below illustrates the different types of element of the time series set.

 

Optional items may also need to be required to fulfil the requirements of the module using the time series set. This will be indicated in this manual for those modules where appropriate.

Figure 3 Schema of the Time Series Set Complex type

description

This is an optional description for the time series set. It is only used a caption in configuration and is not stored with time series.

moduleInstanceId/moduleInstanceSetId

The module instance Id is the ID of the module that has written the data in the time series set to the database. This ID is one of the primary keys and is required to uniquely identify the data on retrieval.

 

In the time series set a single module instance Id may be referenced or multiple module instance Id’s. The latter is done either by including a list of module instance Id’s or by referencing a module instance set Id. This again resolves to a list of module instance set Id’s as defined in the ModuleInstanceSets.xml configuration file.

 

One or more moduleInstanceId may be defined, or a single ModuleInstanceSetId. These cannot be mixed

valueType

This specifies the dimension/data type of the time series. This element is an enumeration of four types;

  • scalar
  • longitudinalprofile
  • grid
  • polygon
parameterId

The parameterId describes the parameter of the data in the time series. This Id is a cross reference to the Parameters.xml configuration file in the regional configuration defining the parameters. The reference is not enforced through an enumeration in the XML schema. If a parameter not included in the parameter definition is referred to, an error will be generated and an appropriate message returned.

locationId/locationSetId

The locationId is a reference to the location for which the data series is valid. Each individual data series may belong to one location only. In the time series set a single location may be referenced or multiple locations may be referenced. The latter is done either by including a list of locationId’s or by referencing a locationSetId. This again resolves to a list of locationId’s as defined in the LocationSets.xml configuration file.

 

One or more locationId’s may be defined, or a single locationSetId. These cannot be mixed.

timeSeriesType

This specifies the type of time series (see discussion above). This is an enumeration of;

  • external historical
  • external forecasting
  • simulated historical
  • simulated forecasting
timeStep

This is the time step of the time series. The time step can be either equidistant or non-equidistant. The time step is defined in the parameters of the timeStep element;

 

Attributes;

  • unit (enumeration of: second, minute, hour, day, week, nonequidistant)
  • multiplier defines the number of units given above in a time step (not relevant for nonequidistant time steps).
  • divider same function as the multiplier, but defines fraction of units in time step.
  • timeZone defines the timeZone of the timeStep, this is only relevant for units of a day or larger.
relativeViewPeriod / relativeForecastPeriod

The relative view period defines the span of time for which data is to be retrieved. This span of time is referenced to the start time of the forecast run (T0) the time series set is used in. If the time series set is not used in a forecast run (e.g. in the displays), then the reference is to the DELFT-FEWS system time.

 

Parameters

  • unit identifies the time unit with which the time span is defined (enumeration of second, minute, hour, day, week).
  • start identifies the start time of the time span with reference to the T0 (in multiples of the unit defined).
  • end identifies the start time of the time span with reference to the T0 (in multiples of the unit defined).
  • startOverrulable Boolean flag to indicate if the start time given may be overruled by a user selection.
  • endOverrulable Boolean flag to indicate if the end time given may be overruled by a user selection.

 

For equidistant time series, all values at the time interval within the span defined will be returned by the database following a request. If no values are found, then missing values will be returned at the expected time step. For non-equidistant time series, all values found within the time span are returned. If none are found, then no values are returned.

 

The relativeForecastPeriod Period is relative to the T0 of the current/selected forecast or the external forecast time of an (the latest) external forecast. Use relativeViewPeriod instead for simulated series created by the current task run.

 

Figure 4 Schematic representation of the relative view period with reference to the T0. The start and end time defined may be overruled if the appropriate parameters are set to true.

externalForecastMaxAge

when the externalForecastMaxAge is not configured there is no maximum age for a forecast series to be used, so the returned external forcast can be very old when there is no recent forecast available. ALL external forecasts after the T0 are ALWAYS ignored. The age of an external forecast is defined as the time span between the external forecast time and T0.

 

Attributes;

  • unit (enumeration of: second, minute, hour, day, week)
  • multiplier defines the number of units given above
  • divider same function as the multiplier, but defines fraction of units.

 

externalForecastTimeCardinalTimeStep

When no external forecast exists in the data store younger that the specified age a new external forecast is returned with a minimum age that applies to the specified cardinal time step.

Attributes;

  • unit (enumeration of: second, minute, hour, day, week, nonequidistant)
  • multiplier defines the number of units given above (not relevant for nonequidistant time steps).
  • divider same function as the multiplier, but defines fraction of units.
  • timeZone defines the timeZone, this is only relevant for units of a day or larger.

 

readWriteMode

The readWriteModel definition is mainly used in the definition of filters to be applied in the time series display when used in edit mode. This element is an enumeration of;

  • read only implies the data cannot be edited.
  • add originals implies the data is new and is added to the database.
  • editing only visible to current task runs implies any changes made remain invisible to other tasks (used in What-If scenarios)
  • editing visible to all future task runs implies any changes made will be visible to other task
  • read originals only implies all edited, corrected or interpolated data should be ignored.

 

The only enumeration that can be used in timeseriessets in FEWS modules is:

  • read complete forecast reads the complete forecast series from the database. If this enumeration element is used no relative View Period has to be configured

 

It is a good convention to set this property to read only in all input blocks.

synchLevel

This is an integer value determining how the data is stored and synchronised through the distributed system. There is no enumeration as the synchLevel is used in the configuration of synchronisation, where optimisations can be defined for each synchLevel. The convention used is explained in the Live System configuration section.

expiryTime

This element allows the time series created to have a different expiry time to the default expiry time. This means it may be removed earlier, or later, by the rolling barrel function. For temporary series the value may be set to a very brief period. For other time series (e.g. Astronomical input series), the value should be set sufficiently high.

 

Attributes;

  • unit (enumeration of: second, minute, hour, day, week)
  • multiplier defines the number of units given above.
  • divider same function as the multiplier, but defines fraction of units.
delay

This element allows the time series retrieved to be lagged (positive or negative). The time stamps of the series will then be shifted by the period specified on retrieval. This is used only when retrieving time series from the database, and not inversely when submitting time series to the database.

 

Attributes;

  • unit (enumeration of: second, minute, hour, day, week)
  • multiplier defines the number of units given above.
  • divider same function as the multiplier, but defines fraction of units.
multiplier

This element allows the time series retrieved to be multiplied by the factor given. This is used only when retrieving time series from the database, and not inversely when submitting time series to the database.

divider

This element allows the time series retrieved to be divided by the factor given. This is used only when retrieving time series from the database, and not inversely when submitting time series to the database.

incrementer

This element allows the time series retrieved to be incremented by the factor given. This is used only when retrieving time series from the database, and not inversely when submitting time series to the database.

ensembleId

A time series set may be defined to retrieve all members of an ensemble at once (for example in evaluation of ensemble statistics). This is done by defining the optional ensembleId. The ensembleId should also be defined when writing new ensemble members. (e.g. on importing ensembles in the import module).

 


4                       System Configuration

4.1                Introduction

The system configuration items form a primary part of the configuration of DELFT-FEWS as a system. It includes definition of the functional elements DELFT-FEWS has available (both GUI plug-ins and Module plug-ins). The layout of the main GUI (the FEWS Explorer) and the Time Series display are also defined.

 

The system configuration items include;

  • Explorer configuration of the FEWS Explorer (the main GUI)
  • ModuleDescriptors Configurations of the plug-in modules available to DELFT-FEWS
  • DisplayDescriptors Configuration of the plug-in displays available to DELFT-FEWS
  • DisplayInstanceDescriptors Definition of the plug-in displays used.
  • TimeSeriesDisplayConfig Configuration of the time series display.
  • DisplayGroups Configuration of the shortcuts to display templates available in the time series display.
  • LocationIcons Definition of the icons used in the FEWS Explorer layout for locations.

 

For each of the configuration items listed above only one configuration is active (or default) at any one time. Each item is defined in an XML file with a unique name.

 


4.2                FEWS Explorer

The layout of the FEWS Explorer is configured in an XML file in the SystemConfigurations section. When available on the file system, the name of the XML file is for example:

 

Explorer 1.00 default.xml

 

Explorer Fixed file name for the explorer settings

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

Figure 5 shows the main elements of the Explorer configuration, These are divided into a number of sections, each of which will be discussed individually (these are indicated with a + sign).

 

description

An optional description of the configuration. This is for reference purposes only and will not be used elsewhere.

Figure 5 Elements in the Explorer configuration

4.2.1            System Information

This section contains some general information of the DELFT-FEWS configuration.

Figure 6 Elements in the System Information section of the Explorer configuration

description

An optional description of the configuration element. This is for reference purposes only and will not be used elsewhere.

systemCaption

The caption that will appear in the title bar of the FEWS Explorer window.

systemHelpFile

Reference to the file name and location of the help file

4.2.2            Explorer Options

Figure 7 Elements in the Explorer Options section of the Explorer configuration

 

The explorer options define whether a number of items are visible on the main display when started. Each of these may either have the value true” or “false”. The items listed in Figure 7 lists all the items. The names are self-explanatory.

 

4.2.3            Zoom extents

The zoom extents is used to define the pre-configured zoom levels that can be selected from the main display.

 

Figure 8 Elements in the Explorer Options section of the Explorer configuration

zoomExtent

Main element of each zoomExtent defined. Note that multiple zoom extents may exist, with the elements below to be defined for each.

 

Attributes:

  • title name of the zoom extent in the list of extents.
left, right, top, bottom

Coordinates of the corners of the zoom extent (specified in the geoDatum selected below)

mnemonic

Optional definition of a letter in the title to use as shortcut.

4.2.4            Explorer Tasks

The explorer tasks define the tasks that can be carried out from the explorer. These tasks are for example the initiation of plug-ins such as the time series display.

 

NOTE: These settings should be amended only by expert users.

 

Figure 9 Elements in the Explorer Tasks section of the Explorer configuration

explorerTask

Main element of each explorer Task. Note that multiple tasks may exist, with the elements below to be defined for each.

 

Attributes;

  • name name of the task
iconFile

Reference to an icon file to be used in the toolbar. If left empty the name will be used to identify the task in the toolbar

Mnemonic

Optional definition of a letter in the title to use as shortcut.

taskExe

Command line for executable to run on initiating the task (the task may be either a call to an executable or to a Java class)

taskClass

Java class to run on initiating the task (the task may be either a call to an executable or to a Java class)

arguments

Optional argument string to be passed to the task

workDir

Optional working directory to start the task in.

Description

Optional description of the task (for reference only)

toolbarTask

Boolean flag to indicate if the task is to appear as a part of the toolbar

menubarTask

Boolean flag to indicate if the task is to appear in the tools menu

allowMultipleInstances

Boolean flag to indicate if multiple instances of task can be initiated concurrently

 

4.2.5            statusBar

The status Bar settings define the format of the time string displayed in the status bar

 

Figure 10 Elements in the Status bar section of the Explorer configuration

dateTimeFormat

String defining the time format for time displayed in the status bar. For example HH:mm:ss will display time as 10:43:26.

 

4.2.6            restSettings

This section includes additional settings for the FEWS Explorer.

 

Figure 11 Elements in the Rest Settings section of the Explorer configuration

defaultSelectFilterId

Defines the default filter to be selected on starting the fewsExplorer

geoDatum

Default definition of the geographic datum. This is an enumeration of geographic datums supported. As further geographic datums are supported, the list will be expanded;

 

For the enumeration of geoDatums suppoted, see Appendix B

dateTimeFormat

Format definition for time strings in displays (e.g. yyyy-MM-dd HH:mm:ss is resolved to 2004-07-03 10:43:26)

 

cardinalTimeStep

Default cardinal time step for the system. The system time will be rounded down to the actual time to the closest cardinal time step.

 

Attributes;

  • unit (enumeration of: second, minute, hour, day, week)
  • multiplier defines the number of units given above in a time step.
  • divider same function as the multiplier, but defines fraction of units in time step.
timeZone

Defines the default time zone for the system. The default time zone is used for all times in user displays, unless locally overruled. This includes time series displays and the system time. The time zone used by the system should conform to a time zone that does not consider summer time. If this optional entry is not included then the timeZone is considered to be UTC+0:00 (or GMT). The time zone can be defined in two ways.

timeZoneOffset

The offset of the time zone with reference to UTC (equivalent to GMT). Entries should define the number of hours (or fraction of hours) offset. (e.g. +01:00)

timeZoneName

Enumeration of supported time zones. See appendix B for list of supported time zones.

4.2.7            logPanelConfig

Configuration of the log panel at bottom of the main display. This can be configured to show all messages (DEBUG level and up), or filtered from a defined level. Two types of log message can be displayed; those generated by the DEBUG logger and those by the EVENT logger. In normal use the latter is defined to show messages from a level of INFO or above. The former is not normally used except for configuration in the stand alone when additional information may be useful. Different settings are available for stand alone clients and for operator clients

 

Figure 12 Elements in the Log Panel section of the Explorer configuration

clientFilter

Root element of a definition of filters (multiple entries may exist).

clientId

Definition of log filters for Operator client or for Stand alone system (both may be included).

logFilter

Root element for log filter definition

eventType

Filter applicable to System logger or to debug logger. Enumeration of “system”or “event”.

Level

Level of log message below which messages are not displayed. Enumeration of DEBUG, INFO, WARN, ERROR, FATAL ERROR

 


4.3                Time Series Display Configuration

The layout of the time series display is configured in an XML file in the SystemConfigurations section. When available on the file system, the name of the XML file is for example:

 

TimeSeriesDisplayConfig 1.00 default.xml

 

TimeSeriesDisplayConfig Fixed file name for the explorer settings

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

Figure 5 shows the main elements of the TimeSeriesDisplay configuration, These are divided into a number of sections, each of which will be discussed individually (these are indicated with a + sign).

 

Figure 13 Elements in the TimeSeriesDisplay configuration

description

An optional description of the configuration. This is for reference purposes only and will not be used elsewhere.

4.3.1            General Display Configuration

This includes global setting for the time series display. Currently only one setting is available.

 

convertDatum

This optional Boolean setting can be used to start the time series display showing levels against the global reference level. The default is that levels are displayed against the local reference level. The difference between local and global reference is defined in the locations definition (see regional settings).

4.3.2            Default view period

The default view period can be used to define the time span of data displayed in the time series display (unless overruled by the user).

 

Parameters

  • unit identifies the time unit with which the time span is defined (enumeration of second, minute, hour, day, week).
  • start identifies the start time of the time span with reference to the T0 (in multiples of the unit defined).
  • end identifies the start time of the time span with reference to the T0 (in multiples of the unit defined).

 

4.3.3            Time Markers Display Configuration

Time series display markers are informative lines in the display. These may be defined to display vertical lines for times values of interest. The configuration of horizontal threshold lines is also included in this definition. Markers can be defined for three time values as well as for the thresholds;

  • systemTime
  • displayTime
  • timeZero
  • threshold

 

Figure 14 Elements in the TimeMarkersDisplay section of the TimeSeriesDisplay configuration

timeMarkerDisplayOptions

Root element of a definition time markers (multiple entries may exist).

 

Attributes;

  • marker enumeration of the possible markers (see list above).
color

Colour of the time series marker line (see enumeration of colours in appendix ??).

lineStyle

Line style of time series marker line. Enumeration of “solid”, “none”, “bar”, “dashdot”, “dashed”, “dotted”.

4.3.4            Parameters display configuration

In this section of the time series display, default attributes of lines plotted for parameters can be allocated. These defaults will be used when plotting lines for these parameters in the time series displays and in the reports. Note that for allocation of the colour only the preferred colour can be identified. When the line is plotted this colour will be used only if there is no other line with the same colour. If this is the case then the next colour in the colour palette will be used.

Figure 15 Elements in the ParameterDisplayConfig section of the TimeSeriesDisplay configuration

PreferredColor

The preferred colour for the line and markers. This colour will only be used if it is not yet available in the current graph. If it is, then the next colour in the template order will be selected.

lineStyle

Line style of time series line. Enumeration of “solid”, “none”, “bar”, “dashdot”, “dashed”, “dotted”.

markerStyle

Marker style for markers plotted on the vertices of the line. Enumeration of “none”, “+”, “x”, “circle”, “square”, “rectangle”, “diamond”, “triangleup” , “triangledown”.

markerSize

Size of the marker in points

Precision

Decimal precision with which values are given in the table.

min

Minimum of the y-scale to be used as a default for all displays of this parameter. The minimum is used, unless a value in the time series is lower than this minimum, in which case that is used. The minimum defined here can also be overruled in individual template definition in the DisplayGroups (see below).

max

Maximum of the y-scale to be used as a default for all displays of this parameter. This maximum is used, unless a value in the time series is higher than this maximum, in which case that is used. The maximum defined here can also be overruled in individual template definition in the DisplayGroups (see below).

4.3.5            Module instance mapping

The module instance mapping allows user defined strings to be defined which will display the moduleInstanceID in the legends of the time series display. This can help shorten legends, or add additional information.

 

Figure 16 Elements in the Log Panel section of the Explorer configuration

moduleInstanceMapping

Root element for each mapping to be defined.

 

Attributes;

  • id ModuleInstanceId to be replaced
description

String to replace the module instance id in the legends


4.4                Display Groups

A list of pre-configured displays can be configured in the Display groups. When available on the file system, the name of the XML file is for example:

 

DisplayGroups 1.00 default.xml

 

DisplayGroups Fixed file name for the explorer settings

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

The pre-configured displays are organised in a tree view in the time series display (see example in Figure 17 ). Each pre-configured display is identified by its name, and may include one or more subplots, each with one or more time series lines.

 

Figure 17 Example of time series display, showing two sub-plots and tree-view of pre-configured displays

 

Figure 18 Root element of the display groups definition

description

A description of the display groups configuration. Used for referencing onlu.

displayGroup

Root element for each displayGroup. A display group forms one of the main nodes in the tree view and may contain multiple displays. Multiple display groups may be defined.

 

Attributes;

  • name : name of the display group (used in the tree view)
display

Definition of a pre-configured display. Each display may contain multiple sub-plots. Multiple displays may be defined per display group.

 

Attributes;

  • name : name of the display (used in the tree view)

 

Figure 19 Elements in the Display section of the DisplayGroups configuration

subplot

Root element for each subplot. Multiple sub-plots may be defined per display.

timeSeriesSet

Definition of the time series set to be displayed. See section 3.3 for a description of the time series set definition.

 

 

4.5                Location Icons

Configuration of location icons can be used to help identify the different types of locations on the map display. This is an optional configuration item. If it is not available then the default location icon will be used to all locations. When available on the file system, the name of the XML file is for example:

 

LocationIcons 1.00 default.xml

 

LocationIcons Fixed file name for the location icon configuration

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

Figure 20 Elements in the LocationIcons configuration

rootDir

This is the directory where the icons referred to are stored. By convention this directory is the <REGION>\Icons directory. The directory can be given relative to the <REGION> directory. If the convention is followed then only “Icons” needs to be entered.

locationIcon

Root element of a location icon definition. Multiple entries may be defined.

description

Description of the group of locations for which an icon is defined (for reference in the configuration only).

iconID

ID of the icon to be used in the display for this group of locations. This id is the same as the name of the icon file, without the “.gif” file extension.

locationId/locationSetId

The locationId is a reference to the location for which icon is used. Either one or more locationId’s may be defined, or a single locationSetId.


4.6                Module Descriptors

The module descriptors is used to register module plug-ins that can be used in workflows. The module descriptors define the name of the module and the associated Java class to call. This class must implement the module plug-in interface for it to work within DELFT-FEWS. All modules that are included in the distribution of DELFT-FEWS are registered in the Module Descriptors. When available on the file system, the name of the XML file is for example:

 

ModuleDescrtiptors 1.00 default.xml

 

ModuleDescrtiptors Fixed file name for the module descriptors configuration

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

Figure 21 Elements in the ModuleDescriptors configuration

moduleDescriptor

Root element of the module descriptor configuration. One entry is required for each module defined.

 

Attributes;

  • Id : Id or Name of the module
description

Optional description of the module. This is used for reference only.

className

Java class called when running the module as referenced by its Id. NOTE; this class must implement the DELFT-FEWS module plug-in interface.

4.7                Display Descriptors

The display descriptors is used to register display plug-ins that can be called from the DELFT-FEWS GUI. The display descriptors define the name of the display and the associated Java class to call. This class must implement the display plug-in interface for it to work within DELFT-FEWS. All displays that are included in the distribution of DELFT-FEWS are registered in the Display Descriptors. When available on the file system, the name of the XML file is for example:

 

DisplayDescriptors 1.00 default.xml

 

DisplayDescriptors Fixed file name for the display descriptors configuration

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

Figure 22 Elements in the DisplayDescriptors configuration

displayDescriptor

Root element of the display descriptor configuration. One entry is required for each display defined.

 

Attributes;

  • Id : Id or Name of the display
description

Optional description of the display. This is used for reference only.

className

Java class called when running the display as referenced by its Id. NOTE; this class must implement the DELFT-FEWS display plug-in interface.

 



5                       Regional configuration

5.1                Introduction

The regional configuration items form the basis of the region specific configuration of DELFT FEWS as a forecasting system for a particular river system or administrative region. It includes definitions of the parameter, locations, units and flags used , which may vary per application of the system.

 

The region configuration items include;

  • Parameters definition of the parameters used for time series
  • Locations definition of all locations
  • LocationSets definition of sets of locations
  • Branches definition of branches (used for displaying longitudinal profiles)
  • Grids definition of grids used in the system.
  • TimeUnits Definition of time units supported by the system (used for mapping external time units to internal time units).
  • Filters Definition of filters in the main map display
  • ValidationRuleSets Definition of validation rule sets
  • Thresholds definition of threshold types
  • ThresholdValueSets definition of threshold values for all locations and data types.
  • ValueAttributeMaps attributes to be mapped from time series values for use in reports.
  • ColdModuleInstanceStateGroups Definition of groups of cold module states.
  • ModuleInstanceDescriptors Definition of instances of modules
  • WorkflowDescriptors Definition of workflows
  • IdMapDescriptors Definition of ID maps used for mapping external parameters and ID’s to DELFT-FEWS locations and parameters.
  • FlagConversionDescriptors Definition of Flag conversions used for mapping external data quality flags to DELFT-FEWS data quality flags.
  • UnitConversionsDescriptors Definition of unit conversions used for mapping external units to DELFT-FEWS units.
  • CorrelationEventSetsDescriptors Definition of sets of correlation events (used by correlation module only).
  • TravelTimesDescriptors Definition of sets of travel times for correlation events (used by correlation module only).
  • HistoricalEvents Definition of historical events to be plotted against real time forecast data for reference purposes.


5.2                Locations

DELFT-FEWS is a location oriented system. All time series data must be referenced to a (geographic) location. This location must be identified by its geographic coordinates within the coordinate system used. When available on the file system, the name of the XML file is for example:

 

Locations 1.00 default.xml

 

Locations Fixed file name for the locations configuration

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

Figure 23 Elements in the Locations configuration

geoDatum

Definition of the geoDatum used in defining the locations. This may be different than the geoDatum used in the displays. For enumeration of geoDatums, see Appendix B.

location

Root element for the definition of each individual location. Multiple entries may be defined.

 

Attributes;

  • id : id of the location. This must be unique
  • name : name of the location used in displays and reports

 

description

Optional description of the location. This description will appear as a tool-tip when hovering over the location in the map display.

shortName

Optional short name for the location. This string when available will replace the name in the time series display legend.

x

Geographic coordinate of the location (Easting)

y

Geographic coordinate of the location (Northing)

z

Optional elevation of the location above the global reference.


5.3                Location Sets

Location sets may be used to define logical groups of locations. Often an action may need to be taken on a whole set of locations (e.g. validation). By creating a LocationSet the action need only be defined once.

 

Any location may appear in more than one location sets. Internally a location set is simply evaluated as a list of locations.

 

When available on the file system, the name of the XML file is for example:

 

LocationSets 1.00 default.xml

 

LocationSets Fixed file name for the locationSets configuration

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

 

Figure 24 Elements in the LocationSets configuration

locationSet

Root element for the definition of a location set. Multiple entries may exist.

 

Attributes;

  • id : Id of the location set. This must be unique
description

Optional description of the location set. Used for reference purposes only.

locationID

Location ID configured to be a member of the locationSet. Multiple entries may exist.


5.4                Parameters

All time series data in DELFT-FEWS must be defined to be of one of the parameters supported. This configuration file defines the list of supported parameters, including the unit of the parameter.

 

Parameters are organised into ParameterGroups. All parameters within a group should have the same properties and the same units. Only parameters of the same group may be displayed in a single (sub) plot in the time series display, though this can be overruled if requested using a display template.

 

When available on the file system, the name of the XML file is for example:

 

Parameters 1.00 default.xml

 

Parameters Fixed file name for the Parameters configuration

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

Figure 25 Root element of the parameter definition

parameterGroup

Root element of each definition of a parameter group. Multiple entries may exist.

 

Attributes;

  • id : Id of the parameter group. The ID must be unique.
  • name : optional name for the parameter group. Used for reference only.

 

Figure 26 Elements of the ParameterGroup configuration in the parameter definition

description

Optional description of the parameter group. Used for reference purposes only.

parameterType

Definition if the parameters in the group if these are “instantaneous” parameters or “accumulative” parameter.

unit

Unit of the parameters defined in the group. The unit may be selected from a list of units supported by DELFT-FEWS. These are generally SI units. For an enumeration of supported units see Appendix B.

usesDatum

Indicates if the parameters in the group are to be converted when toggling between local and global datum. Value is either true or false. If the value is true, the elevation defined in the location is added to the time series in the database on conversion.

parameter

Definition of each parameter in a parameter group. Multiple parameters may be defined per group.

 

Attributes;

  • id : Id of the parameter. The ID must be unique.
  • name : optional name for the parameter. Used for reference only.
shortName

Short name for the parameter. This name will be used in the time series display and reports.

 


5.5                Branches

DELFT-FEWS is a location oriented system. All time series data must be referenced to a (geographic) location. Scalar time series need no additional information. For longitudinal time series data, each point in the vector must be referenced to a location in a branch structure. This location must be identified by its coordinate within the branch (chainage), and may also be defined by its geographic coordinates within the coordinate system used.

 

When available on the file system, the name of the XML file is for example:

 

Branches 1.00 default.xml

 

Branches Fixed file name for the Branches configuration

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

Figure 27 Elements of the Branches configuration

geoDatum

Definition of the geoDatum used in defining the locations of the branch. This may be different than the geoDatum used in the displays. For enumeration of geoDatums, see Appendix B.

branch

Root element of the definition of a branch. Multiple entries may exist to define multiple branches.

 

Attributes;

  • id : Id of the current branch. This ID must refer to a location ID in the Locations definition.
branchName

Name of the current branch. Used for reference purposes only

startChainage

Chainage of the start of the branch (only used in the longitudinal display)

endChainage

Chainage of the end of the branch (only used in the longitudinal display)

upNode, downNode

Optional item in branch to create branch linkage. This information is not used in DELFT-FEWS, but may be relevant to an external module when exported through the published interface.

zone

Optional item in branch that allows definition of a zone – this is a part of the branch that may be indicated in the longitudinal display with the name given (currently not used in DELFT-FEWS).

pt

Definition of the points belonging to the branch. At least two points must be defined per branch.

 

Attributes;

  • chainage ; coordinate of point as measured along the branch (should be greater than or equal to the start chainage and less than the end chainage).
  • label ; label used to identify the point
  • x ; optional geographic coordinate of the point (Easting)
  • y ; optional geographic coordinate of the point (Northing)
  • z ; optional elevation of the point. The elevation is an important attribute for plotting in the longitudinal profile display. This elevation is taken as the bed level.
  • description ; optional description string. When defined a vertical line will be drawn in the longitudinal display at the location of this point, and the description given displayed.
  • thresholdValueSetId ; optional reference to an ID of a threshold value set. When defined, the threshold values will be drawn as markers in the longitudinal display at the location of this point.
comment

Comment item. Used for reference only.

 


5.6                Grids

DELFT-FEWS is a location oriented system. All time series data must be referenced to a (geographic) location. Scalar time series need no additional information. For grid time series data, each point in the grid must be referenced to a location in a grid structure.

 

Grids may be regular or irregular. In regular grids each cell has the same width, height and area within the coordinate system it is specified in.

 

In irregular grids the grid has a fixed number of rows and columns, but the cell height and width is not equal in each row and column. For these grids additional information is required on the location of each individual cell in the grid to allow display in the grids display as well as for use in the spatial interpolation routine.

 

When available on the file system, the name of the XML file is for example:

 

Grids 1.00 default.xml

 

Grids Fixed file name for the Grids configuration

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

Figure 28 Elements of the grids configuration

regular

Definition of a regular grid. One or more regular grids may be defined.

 

Attributes;

  • locationId ; location Id of the grid. This locationId must be included in the locations definition.
irregular

Definition of a irregular grid. One or more irregular grids may be defined.

 

Attributes;

  • locationId ; location Id of the grid. This locationId must be included in the locations definition.

5.6.1            Regular grids

 

Figure 29 Elements of the Regular Grid in the Grids configuration

description

Optional description of the grid. Used only for reference purposes

rows, columns

Number of rows and columns in the grid

geoDatum

Coordinate system the grid is defined in. This may be a different coordinate system to that used in the main map display. The coordinate system may also differ per grid, as a grid may be regular in one coordinate system, but may be irregular in another. Defining the grid in the regular coordinate system is easier.

firstCellCenter

Coordinates of the center of the first grid cell. The convention in DELFT-FEWS is that this is the center point of the top left cell in the grid (Upper-Left).

firstCellCenter: x

Geographic coordinate of the first cell center point (Easting)

firstCellCenter: y

Geographic coordinate of the first cell center point (Northing)

firstCellCenter: z

Optional elevation of the first cell center point (Easting). If only this elevation is defined , then all cells in the grid are assumed to have the same elevation.

xCellSize

Cell width of each column in the grid. The cell width is given in the unit of the coordinate system referred to in the geoDatum. Generally this is metres, but in WGS 1984 this is decimal degrees.

yCellSize

Cell height of each row in the grid. The cell height is given in the unit of the coordinate system referred to in the geoDatum. Generally this is metres, but in WGS 1984 this is decimal degrees.

z

Optional definition of the elevation of each point in the grid. This definition is only necessary where a datum is required in for example 3-dimensional interpolation. This may be applied in for example interpolating temperature grids in high mountain areas.

 

5.6.2            Irregular grids

Figure 30 Elements of the irregular grid in the Grids configuration

description

Optional description of the grid. Used only for reference purposes

rows, columns

Number of rows and columns in the grid

geoDatum

Coordinate system the grid is defined in. This may be a different coordinate system to that used in the main map display. The coordinate system may also differ per grid. A grid may be regular in one coordinate system, but may be irregular in another. Defining the grid in the regular coordinate system is generally easier.

cellCentre

Definition of the cell centre points of all cells in the irregular grid. The number of cellCentre points defined must be the same as the number of cells in the grid (rows x columns).

cellCentre: x, cellCentre: y

Geographic coordinates of the ell centre point (x: Easting, y: Northing)

cellCentre: z

Elevation of the cell centre point.


5.7                Filters

Filters are used in DELFT-FEWS to define the locations that are displayed on the main map display, and that can be selected to display data. Filters are defined to arrange locations, with associated parameters in logical groups. Each filter is defined as a collection of time series sets.

 

Filters may be defined as a nested structure, allowing for the definition of a hierarchal set of filters.

 

When available on the file system, the name of the XML file is for example:

 

Filters 1.00 default.xml

 

Filters Fixed file name for the Filters configuration

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

Figure 31 Elements of the filters configuration

description

Optional description of the filter configuration. Used for reference purposes only.

filter

Definition of a filter. Multiple entries may exist where multiple filters are defined. Each filter may contain either a set of one or more time series set, or a child filter. The child is a reference to another filter definition that again contains either a child filter or a list of time series sets. This structure is used to construct a hierarchal tree view of filters.

 

Attributes;

  • id : Id of the filter. This ID is used in the tree view in the main display
  • name : optional name for the filter. For reference purposes only.
timeSeriesSet

Definition of a time series set belonging to a filter. Multiple time series sets may be defined.

child

Reference to another filter. The child element refers to the ID of the other filter as a foreign key.

 

Attributes;

                         foreignKey : Reference to ID of another filter

 

Figure 32 Example of filter configuration, as defined in the example XML configuration above.


5.8                ValidationRuleSets

Validation rules are defined in DELFT-FEWS to allow quality checking of all time series data (scalar time series only). Several validation criteria may be defined per time series. All validation rules for all time series are defined in this configuration. For each time series to be checked, a set of validation rules is defined. Defining validation rules to apply to a time series set using a locationSet rather than identifying series individually can simplify the configuration greatly. Most validation rules may be defined either as a constant value, or as a value valid per calendar month.

 

When available on the file system, the name of the XML file for configuring the Validation Rule Sets is for example:

 

ValidationRuleSets 1.00 default.xml

 

ValidationRuleSets Fixed file name for the Validation rules configuration

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

Figure 33 Elements of the ValidationRuleSets configuration.

validationRuleSet

Root element of the definition of a validation rule set. Multiple entries may exist.

 

Attributes;

  • validationRuleSetId: Optional reference ID for the validation rule set. Used only in messaging.
  • timeZone: Shift (in hours) of the time zone to be used in considering time values in validation.
timeSeriesSet

Definition of the time series to apply validation rule to.

extremeValues

Validation rules defined to check for extreme values (hard and soft limits)

rateOfChange

Validation rules defined to check rate of change.

sameReading

Validation rules defined to check for series of same readings.

temporaryShift

Validation rules defined to check for temporary shifts in time series.

5.8.1            Validation on extreme values

This group of validation rules checks that the values in the time series do not exceed minimum and maximum limits. These limits may be defined as soft limits or as hard limits. Values exceeding soft limits will be marked as doubtful but retained. Values exceeding hard limits will be marked as unreliable.

 

Figure 34 Elements of the Extreme values configuration of the ValidationRuleSets.

hardMax

Validation rule for checking hard maximum. Values exceeding this limit will be marked as unreliable.

 

Attributes;

  • constantValue: Value of hardMax limit, used irrespective of time of value.
hardMin

Validation rule for checking hard minimum. Values exceeding this limit will be marked as unreliable.

 

Attributes;

  • constantValue: Value of hardMin limit, used irrespective of time of value.
softMax

Validation rule for checking soft maximum. Values exceeding this limit will be marked as doubtful.

 

Attributes;

  • constantValue: Value of softMax limit, used irrespective of time of value.
softMin

Validation rule for checking soft minimum. Values exceeding this limit will be marked as doubtful.

 

Attributes;

  • constantValue: Value of softMin limit, used irrespective of time of value.
monthLimit

Element used when defining variable limits per calendar month. Twelve values must be defined. When defined the monthly limit will overrule the constant limit.

 

5.8.2            Validation on rate of change

This group of validation rules checks that the values in the time series do not exceed maximum rates of change. When the rate of change limit is exceeded, the values causing the limit to be exceeded will be marked as unreliable. Rate of change limits may be defined to be the same for the rate of rise as for the rate of fall. These may also be defined to be different.

 

Figure 35 Elements of the rate of change configuration of the ValidationRuleSets.

rateofRiseFallDifferent

Root element used if the rate of rise limit is defined different to the rate of fall.

rateOfRise

Validation rule defined for the rate of rise.

 

Attributes;

  • constantValue: Maximum rate of rise, used irrespective of date of the value.
rateOfFall

Validation rule defined for the rate of fall.

 

Attributes;

  • constantValue: Maximum rate of fall, used irrespective of date of the value.
monthLimit

Element used when defining variable limits per calendar month. Twelve values must be defined. When defined the monthly limit will overrule the constant limit.

5.8.3            Validation on series of same readings

Time series of data can be validated on series of same readings. This may be unlikely for field observations, and may indicate an instrumental error. In some cases a small variability may still be observed, despite instrumental error. The same readings check allows for defining a bandwidth within the value is considered to be the same.

 

Figure 36 Elements of the same reading configuration of the ValidationRuleSets.

sameReadingDeviation

Root element for definition of bandwidth the value may vary within if it is considered to the same reading. The bandwidth is twice the deviation.

 

Attributes;

  • constantValue: Value for deviation, used irrespective of date of the value.
sameReadingPeriod

Root element for definition of time span limit the value may remain the same to be considered realistic. If the reading remains the same for a longer period of time, ensuing values will be considered unreliable.

 

Attributes;

  • constantValue: Value for time span in seconds, used irrespective of date of the value.
monthLimit

Element used when defining variable limits per calendar month. Twelve values must be defined. When defined the monthly limit will overrule the constant limit.

5.8.4            Validation on Temporary Shifts

Time series of data can be validated on temporary shifts. These occur when instrunments are reset, and can be identified by the values rapidly falling to a constant value, remaining at that value for a short period of time and then returning to the original value range. A complex set of validation criteria include the rate of change as well as a maximum time the value remains the same.

Figure 37 Elements of the temporary shift configuration of the ValidationRuleSets.

rateOfTemporaryShift

Rate of change that must be exceeded both on change to shifted value and change back to original value range for validation rule to apply.

 

Attributes;

  • constantValue: Value for rate of change, used irrespective of date of the value.
temporaryShiftPeriod

Maximum time span constant shifted value is in time series for validation rule to apply.

 

Attributes;

  • constantValue: Value for time span in seconds, used irrespective of date of the value.
monthLimit

Element used when defining variable limits per calendar month. Twelve values must be defined. When defined the monthly limit will overrule the constant limit.

 


5.9                Thresholds

DELFT-FEWS supports checking of time series against thresholds. When thresholds are crossed, appropriate messages may be issued. Definition of thresholds is in two parts. In the first part of the configuration, the types of threshold used are defined. In the second, the values for threshold valid for a particular location and time series are defined. In this section the configuration for the definition of the thresholds is defined. DELFT-FEWS supports different types of threshold events. These include crossing of level and rate thresholds. The occurrence of a peak is also seen as a threshold event.

 

For each threshold defined, two additional items need to be configured. Internally DELFT-FEWS maintains threshold events as a non-equidistant time series, where the crossings are identified by an integer. For each threshold two unique integer Id’s need to be assigned. One ID is used to identify the  upcrossing of the threshold, the other Id’ is assigned to identify the downcrossing. The exception to this is the peak threshold where only a single Id needs to be assigned to identify the occurrence of the peak event.

 

Similar to the Id’s used for upcrossings and downcrossing, a warning level integer can be assigned to threshold crossings. This warning level is resolved to either an icon (for display in the main FEWS GUI), or a colour (for use in reports). Warning levels need not be unique. These levels are used only for level thresholds.

 

When available on the file system, the name of the XML file for configuring the types of thresholds is for example:

 

Thresholds 1.00 default.xml

 

Thresholds Fixed file name for the Thresholds configuration

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

levelThreshold

Root element for definition of a level threshold. Multiple entries may exist.

 

Attributes;

  • id: Unique Id of level threshold.
  • name: Name for the level threshold.
rateThreshold

Root element for definition of a rate threshold. Multiple entries may exist.

 

Attributes;

  • id: Unique Id of rate threshold.
  • name: Name for the rate threshold.
maxThreshold

Root element for definition of a peak event threshold. Multiple entries may exist.

 

Attributes;

  • id: Unique Id of rate threshold.
  • name: Name for the rate threshold.

 

Figure 38 Elements of the Threshold configuration

upWarningLevel

Integer level used in determining icon (through ValueAttributesMap) on up-crossing of threshold (level thresholds only).

downWarningLevel

Integer level used in determining icon (through ValueAttributesMap) on down-crossing of threshold (level thresholds only).

upIntId

Unique integer level defined in threshold crossing time series (internal) on up-crossing of threshold.

downIntId

Unique integer level defined in threshold crossing time series (internal) on down-crossing of threshold.

intId

Unique integer level defined in threshold crossing time series (internal) on occurrence of peak event.

5.10           ThresholdValueSets

Complementary to the definition of the types of thresholds identified, the values of the thresholds are defined in the ThresholdValueSets configuration. The configuration of this is similar to the validation rules. Several thresholds may be defined per time series. For each time series to be tested, a set of thresholds is defined.

 

Thresholds may be defined to initiate an action by the Master Controller when applied in a live forecasting system. Actions are taken in response to a log event code. To identify which threshold crossing for which locations will initiate an actions (e.g. enhanced forecasting), an event code can be defined in the ThresholdValueSet. When the threshold is crossed the event code is generated.

 

When available on the file system, the name of the XML file for configuring the ThresholdValueSets is for example:

 

ThresholdValueSets 1.00 default.xml

 

ThresholdValueSets Fixed file name for the ThresholdValueSets configuration

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

Figure 39 Elements of the ThresholdValueSets configuration.

thresholdValueSet

Root element for defining a set of thresholds. For each time series or time series set for which a threshold event is to be tested new element is required.

 

Attributes;

  • Id: Id of the thresholdValueSet defined.
  • Name: optional name, for reference purposes only
description

Optional description for the ThresholdValueSet. Used for reference purposes only

levelThresholdValue

Definition of values for level thresholds.

rateThresholdValue

Definition of values for rate thresholds.

maxThreshold

Definition of values fro peak event thresholds.

timeSeriesSet

Definition of the time series set for which the thresholds are to be tested.

5.10.1        Defining level thresholds

Figure 40 Elements of the Level Threshold configuration of the ThresholdValueSets configuration

levelThresholdId

Id of the level threshold. This Id must refer to a threshold type defined in the Thresholds definition (see previous paragraph).

value

Value of the threshold.

upActionLogEventTypeId

Event code to be generated on the up-crossing of the threshold. This event code can be used to initiate for example enhanced forecasting. The event code need not be unique. Multiple threshold crossings may generate the same event code. Note that event codes will only be generated for runs which have an a-priori approved status. This is normally the scheduled forecast run.

downActionLogEventTypeId

Event code to be generated on the down-crossing of the threshold. This event code can be used to initiate for example enhanced forecasting. The event code need not be unique. Multiple threshold crossings may generate the same event code. Note that event codes will only be generated for runs which have an a-priori approved status. This is normally the scheduled forecast run.

5.10.2        Defining rate thresholds

Figure 41 Elements of the Rate Threshold configuration of the ThresholdValueSets configuration

rateThresholdId

Id of the rate threshold. This Id must refer to a threshold type defined in the Thresholds definition (see previous paragraph).

value

Value of the rate threshold that must be exceeded in timeSpan.

timeSpan

Time span to use to establish the rate.

rainRate

Boolean indicator to identify thresholds in rain rates where the threshold is defined as the average rainRate over the timeSpan exceeding the threshold, and a rate in for example a level where the rate is determined as the value divided by the time span.

upActionLogEventTypeId

Event code to be generated on the up-crossing of the threshold. This event code can be used to initiate for example enhanced forecasting. The event code need not be unique. Multiple threshold crossings may generate the same event code. Note that event codes will only be generated for runs which have an a-priori approved status. This is normally the scheduled forecast run.

downActionLogEventTypeId

Event code to be generated on the down-crossing of the threshold. This event code can be used to initiate for example enhanced forecasting. The event code need not be unique. Multiple threshold crossings may generate the same event code. Note that event codes will only be generated for runs which have an a-priori approved status. This is normally the scheduled forecast run.

5.10.3        Defining peak event thresholds

Figure 42 Elements of the maxThreshold configuration of the ThresholdValueSets configuration

maxThresholdId

Id of the max threshold. This Id must refer to a threshold type defined in the Thresholds definition (see previous paragraph).

value

The value item is used here as a selection of peaks. The peak must exceed this value to be deemed significant (peaks over threshold)..

timeSpan

The timeSpan is used to establish independence of peaks. Peaks within timeSpan of each other are considered as being of the same event as a message will only be issued for the highest.

actionLogEventTypeId

Event code to be generated on the threshold occurring. This event code can be used to initiate for example enhanced forecasting. The event code need not be unique. Multiple threshold crossings may generate the same event code. Note that event codes will only be generated for runs which have an a-priori approved status. This is normally the scheduled forecast run.

 


5.11           ColdModuleInstanceStateGroups

Many forecasting models use an initial state as initial condition. When used in real time, DELFT-FEWS can be used to manage these states, such that models are run from a warm state. Long run times in initiating models is thus avoided.

 

When no warm state is available a cold state will be used. Additionally the user may explicitly select the cold state to be used as model initial condition.

 

A default initial condition must be available for models requiring state management. Additional groups of cold module states may also be defined. These can be selected in for example scenario runs. While a default state is required for every model,  additional states need only be defined where available. When the state indicated is not found for a particular , DELFT-FEWS will revert to the default state. Where it is found, it will be used as selected.

 

When available on the file system, the name of the XML file for configuring the ColdModuleInstanceStateGroups is for example:

 

ColdModuleInstanceStateGroups 1.00 default.xml

 

ColdModuleInstanceStateGroups
Fixed file name for the ColdModuleInstanceStateGroups configuration

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

Figure 43 Elements of the ColdModuleInstanceStateGroups configuration

defaultGroup

Definition of the default group of module states. This is a required item, and only a single definition is allowd.

 

Attributes;

  • id: Id of the state group (e.g. Default)
  • name: name of the state group.
additionalGroup

Definition of the additional group of module states. One or more items may exist.

 

Attributes;

  • id: id of the state group (e.g. Wet)
  • name: name of the state group.
description

Optional description of the state group. Used for reference purposes only.

 

 

 


5.12           ModuleInstanceDescriptors

Each module configured in DELFT-FEWS must be registered in the ModuleInstanceDescriptors configuration. This is required to identify the module to DELFT-FEWS (the name is free format), but is also required to define the type of module through reference to the moduleDescriptors defined (see system configuration).

 

When available on the file system, the name of the XML file for configuring the ModuleInstanceDescriptors is for example:

 

ModuleInstanceDescriptors 1.00 default.xml

 

ModuleInstanceDescriptors Fixed file name for the ModuleInstanceDescriptors configuration

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

Figure 44 Root element of the ModuleInstanceDescriptors configuration

ModuleInstanceDescriptorsId

Root element of the ModuleInstanceDescriptor element. For each module defined the element is repeated. Multiple instances may exist.

 

Attributes;

  • Id: Id of the Module Instance. This Id must be unique. Normally a string is used that gives some understanding of the role of the module ( e.g. SpatialInterpolationPrecipitation ).
  • name: Optional name for the module. Used for reference purposes only.
moduleId

Reference to the ModuleDescriptors defined in the SystemConfiguration to identify the type of module.

description

Optional description. Used for reference purposed only.


5.13           WorkflowDescriptors

Each workflow configured in DELFT-FEWS must be registered in the WorkflowDescriptors configuration. This is required to identify the workflow to DELFT-FEWS (the name is free format). The configuration also sets some properties of the workflow.

 

When available on the file system, the name of the XML file for configuring the WorkflowDescriptors is for example:

 

WorkflowDescriptors 1.00 default.xml

 

WorkflowDescriptors Fixed file name for the WorkflowDescriptors configuration

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

Figure 45 Elements of the workflowDescriptor configuration.

workflowDescriptor

Root element of the WorkflowDescriptor. New element is required for each workflow. Multiple instances may be defined.

 

Attributes;

  • Id: Id of the workflow. This Id must be unique. Normally a string is used that gives some understanding of the role of the module ( e.g. ImportExternal ).
  • name: Optional name for the module. Used for reference purposes only.
  • visible: Boolean toggle to indicate if workflow is visible for selection in the manual forecast display. Non-independent workflows (e.g. sub-workflows) should not be marked visible so that these cannot be run independently. Default is true.
  • Forecast: Boolean flag to indicated if workflow is identified as a forecast. This should be the case for workflows with simulated time series as a results. Import workflows of external data are not forecasts. Default is true.
  • allowApprove. Boolean flag to indicate if workflow may be approved a-priori through manual forecast display (stand-alone only). Default is true.
  • autoApprove. Boolean flag to indicate workflow should automatically be approved a-priori (stand-alone only). Default is false.

5.14           IdMapDescriptors

Each IdMap to support mapping external to internal location and parameter Id’s  configured in DELFT-FEWS must be registered in the IdMapDescriptors configuration. This is required to identify the IdMap to DELFT-FEWS (the name is free format).

 

When available on the file system, the name of the XML file for configuring the IdMapDescriptors is for example:

 

IdMapDescriptors 1.00 default.xml

 

IdMapDescriptors Fixed file name for the IdMapDescriptors configuration

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

Figure 46 Elements of the IdMapDescriptors configuration.

IdMapDescriptor

Root element of the IdMapDescriptor. New element is required for each IdMap. Multiple instances may be defined.

 

Attributes;

  • Id: Id of the idMap. This Id must be unique. Normally a string is used that gives some understanding of the role of the module ( e.g. ImportRTS ).
  • name: Optional name for the IdMap. Used for reference purposes only.

 

 

 


5.15           FlagConversionDescriptors

Each FlagConversion to support mapping external to internal data quality flags configured in DELFT-FEWS must be registered in the FlagConvesrionDescriptors configuration. This is required to identify the FlagConversion to DELFT-FEWS (the name is free format).

 

When available on the file system, the name of the XML file for configuring the FlagConversions is for example:

 

FlagConversions 1.00 default.xml

 

FlagConversions Fixed file name for the FlagConversions configuration

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

Figure 47 Elements of the FlagConversions configuration.

FlagConversion

Root element of the FlagConversion. New element is required for each FlagConversion. Multiple instances may be defined.

 

Attributes;

  • Id: Id of the FlagConversion. This Id must be unique. Normally a string is used that gives some understanding of the role of the module ( e.g. ImportRTS ).
  • name: Optional name for the FlagConversion. Used for reference purposes only.

 

 


5.16           UnitConversionsDescriptors

Each UnitConversion to support mapping external to internal units configured in DELFT-FEWS must be registered in the UnitConversionsDescriptors configuration. This is required to identify the UnitConversion to DELFT-FEWS (the name is free format).

 

When available on the file system, the name of the XML file for configuring the UnitConversionDescriptors is for example:

 

UnitConversionDescriptors 1.00 default.xml

 

UnitConversionDescriptors Fixed file name for the UnitConversionDescriptors configuration

1.00 Version number

default Unit to indicate the version is the default configuration (otherwise omitted).

 

Figure 48 Elements of the UnitConversions configuration.

UnitConversionDescriptor

Root element of the UnitConversionDescriptor. New element is required for each UnitConversion  identified. Multiple instances may be defined.

 

Attributes;

  • Id: Id of the UnitConversion. This Id must be unique. Normally a string is used that gives some understanding of the role of the module ( e.g. ImportRTS ).
  • name: Optional name for the UnitConversion. Used for reference purposes only.

 

 

 


5.17           CorrelationEventSetsDescriptors

The correlation module in DELFT-FEWS allows forecasts for a downstream location to be established using a correlation of peak events for the forecast site and one or more support sites. For each river multiple correlations between several sites on the river may be defined. Correlation sets can be defined to allow logical ordering these into logical groups. This configuration file defines the groups for which correlation event data will be later defined.

 

When available on the file system, the name of the XML file for configuring the CorrelationEventSetsDescriptors is for example:

 

CorrelationEventSetsDescriptors 1.00 default.xml

 

CorrelationEventSetsDescriptors
Fixed file name for the CorrelationEventSetsDescriptors configuration

1.00 Version number

default Unit to indicate the version is the default configuration (otherwise omitted).

 

Figure 49 Elements of the CorrelationEventSetsDescriptors configuration

CorrelationEventSetsDescriptor

Root element of the CorrelationEventSetsDescriptor. New element is required for each CorrelationEventSet identified. Multiple instances may be defined.

 

Attributes;

  • Id: Id of the CorrelationEventSet. This Id must be unique. Normally a string is used that gives some understanding of the group created ( e.g.Severn ).
  • name: Optional name for the CorrelationEventSet. Used for reference purposes only.

 

 

 

 

 


5.18           TravelTimesDescriptors

The correlation module in DELFT-FEWS allows forecasts for a downstream location to be established using a correlation of peak events for the forecast site and one or more support sites. For each river multiple correlations between several sites on the river may be defined. Together with the correlation establishing a forecast value, an estimate of travel time between the locations can be given. This is given either as a default travel time, or it is established through regression of the events considered. An estimate of the travel time is also used to establish which events in the upstream and downstream location are paired.

 

Correlation sets can be defined to allow logical ordering these into logical groups. This configuration file is similar to the CorrelationEventSets and defines the groups for which travel time data will be later defined.

 

When available on the file system, the name of the XML file for configuring the TravelTimesDescriptors is for example:

 

TravelTimesDescriptors 1.00 default.xml

 

TravelTimesDescriptors
Fixed file name for the TravelTimesDescriptors configuration

1.00 Version number

default Unit to indicate the version is the default configuration (otherwise omitted).

 

Figure 50 Elements of the TravelTimesDescriptors configuration

TravelTimesDescriptors

Root element of the TravelTimesDescriptor. New element is required for each TravelTimes set identified. Multiple instances may be defined.

 

Attributes;

  • Id: Id of the TravelTimes set. This Id must be unique. Normally a string is used that gives some understanding of the group created ( e.g.SevernTravelTimes ).
  • name: Optional name for the TravelTimes Set. Used for reference purposes only.

 

 

 


5.19           TimeUnits

External data sources to be imported in DELFT-FEWS may provide data at an equidistant time step. The time unit defined is often defined in a string, and must be resolved on import to a time unit recognised by DELFT-FEWS. The mapping of time units is defined in the TimeSteps configuration.

 

When available on the file system, the name of the XML file for configuring the TimeUnits for example:

 

TimeUnits 1.00 default.xml

 

TimeUnits Fixed file name for the TimeUnits configuration

1.00 Version number

default Unit to indicate the version is the default configuration (otherwise omitted).

 

Figure 51 Elements of the TimeUNits configuration

timeUnit

Root element for each external time unit identified. Multiple entries may exist.

Unit

String value for unit as identified in external data source

milliseconds

Equivalent of time unit in milliseconds (base unit in DELFT-FEWS). By convention 0 milliseconds is a non-equidistant time unit. -1 indicates that the unit is not supported. This is the case for time units such as months, years etc.


5.20           Historical Events

DELFT-FEWS allows a set of historical events to be defined that can be retrieved when looking at forecast data through the time series display. These events can then be displayed in the same plot as the real-time data for reference purposes.

 

Historical events are configured as a time series referenced to a location/parameter. When that location/parameter is displayed in the time series display, a drop down list of the events available for that specific combination is displayed. Selected events are displayed in the same sub-plot as the real time data for that location parameter.

 

This configuration is optional. If not available no historical events will be displayed.

 

When available on the file system, the name of the XML file for configuring the HistoricalEvents is:

 

HistoricalEvents 1.00 default.xml

 

HistoricalEvents Fixed file name for the HistoricalEvents configuration

1.00 Version number

default Unit to indicate the version is the default configuration (otherwise omitted).

 

 

Figure 52 Elements of the HistoricalEvents configuration.

historicalEvent

Root definition of an historical event. Multiple events may be defined.

 

Attributes;

  • locationId : Id of the location of the event (see definition of DELFT-FEWS locations).
  • parameterId : Id of the parameter in the event (see definition of DELFT-FEWS parameters).
  • name : name of the historical event. This name will be available as a list of historical events.
eventData

Time series data for the event. This follows the same defition of the inputVariable detailed in the Tranformation Module configuration. The typical profile option is used for defining an historical event).

 

Attributes;

  • variableId : ID of the variable (group).Later used in referencing the variable.
  • variableType : Optional type definition of variable (defaults to “any”)
  • convertDatum : Optional Boolean flag to indicate if datum is to be converted.
timeStep

Time step for typical profile if variable to be defined for the historical event.

 

Attributes;

  • unit (enumeration of: second, minute, hour, day, week, nonequidistant)
  • multiplier defines the number of units given above in a time step (not relevant for nonequidistant time steps)
  • divider same function as the multiplier, but defines fraction of units in time step.
relativeViewPeriod

Relative view period of the event. This is the time span of the event. The start and end information will be used when initially plotting the event to determine its position against the display time at the time of display

data

Data entered to define the event. Data is entered using the dateTime attribute only, the specific date and time values given for each data point. Other attributes available for defining typical profiles are not used.

 

Attributes;

  • dateTime : Attribute value indicating the value entered is valid for a specific date time combination. The string has the format “ <year>-<month>-<day>T<hour>:<minute>:<second> ”. For example the 23 rd of August is “ 1984-12-31T00:00:00 ”.
timeZone

Optional specification of the time zone for the data entered (see timeZone specification).

timeZone:timeZoneOffset

The offset of the time zone with reference to UTC (equivalent to GMT). Entries should define the number of hours (or fraction of hours) offset. (e.g. +01:00)

timeZone:timeZoneName

Enumeration of supported time zones. See appendix B for list of supported time zones.

 

Example of an Historic event


5.21           Value Attribute Maps

DELFT-FEWS allows attributes to be associated to values in a time series. This can be used to associate either a textual value or an icon for use in displays or in reports. Typically the use of value attribute maps is important in forecasts derived through application of the lookup table modules. Critical conditions are then defined which resolve a combination of inputs to a single “Lookup Index” output. This Lookup index is then resolved either to a textual message, an icon or a colour using the value attribute maps. The same principle is used in allocating colours/icons to thresholds, where the unique threshold index is used as an entry to the value attribute mapping.

 

When available on the file system, the name of the XML file for configuring ValueAttributeMaps is:

 

ValueAttributeMaps 1.00 default.xml

 

ValueAttributeMaps Fixed file name for the ValueAttributeMaps configuration

1.00 Version number

default Unit to indicate the version is the default configuration (otherwise omitted).

 

Figure 53 Elements of the value attribute maps configuration

valueAttributeMap

Root element for the definition of a set of attribute values. The Id used to identify this set is later referenced in for example the report module configuration to allow association of an attribute to a value. Multiple sets may be defined.

 

Attributes;

  • id : unique id of the set of value attributes
attibutes

Root element for associating an attribute to a value. Each value may be attributed a definition (text), a colour and/or an icon.

 

Attributes;

  • value : value for which the attributes defined must be associated. Note that an exact match is required to allow the mapping to be valid.
description

Text to be attributed where this value is given in the input series. This text may be used in a report.

image

Path and filename of the icon to be attributed where this value is given in the input series. This icon may be used in for example displays as well as in reports.

colour

Colour to be attributed where this value is given in the input series. This colour may be used in for example background colouring of a table in a report

 

 


6                       Module Instances

6.1                Introduction

All functionality used be DELFT-FEWS in processing dynamic data and running external forecasting modules is configured in a module instance. These are then executed in a logical sequence as defined in a workflow.

 

A variety of modules is available in DELFT-FEWS to provide specific functionality. Examples include interpolation, running external modules, data import etc. The modules available are defined in the ModuleDescriptors in the System configuration. This defines the Java classes used to run each module, and assigns a recognizable name for that module (e.g. Transformation). These Java classes implement the workflow plug-in interface to DELFT-FEWS. The list of available modules can be extended through adding classes implementing this plug-in interface.

 

To carry out a specific piece of data processing, and instance of a module is configured. This instance specifies the input data, the output data and the required steps in processing the data. Each module instance is given a unique name with which it is identified in the module instance section of the configuration. To link an instance of a module to the type of module available, the module instance is registered in the ModuleInstanceDescriptors in the Regional Configuration section.

 

The list of module instances in DELFT-FEWS includes;

  • Interpolation Module
  • Transformation Module
  • Import Module
  • Export Module
  • General Adapter Module
  • Lookup Table Module
  • Correlation Module
  • Error Correction Module
  • Report Module
  • Report Export Module
  • Performance Indicator Module

 

 


6.2                Interpolation Module Configuration

The Interpolation module generates data at desired locations or at desired points in time by means of either a serial or spatial interpolation technique. It is applied for the filling in of data gaps in measured on-line data, as well as to derive spatially distributed data for meteorological time series, such as precipitation and temperature, based on information available at neighbouring locations.

 

Two methods of interpolation are available;

Serial interpolation

In serial interpolation mode, interpolation is done to fill any gaps in a time series. The interpolation module will only consider the time series itself in filling these gaps. Interpolation methods that can be used are;

  • Filling of gaps with a default value
  • Filling of gaps by linear interpolation
  • Filling of gaps by block interpolation
  • Extrapolation of gaps at start or end of a time series

 

All these methods can be configured to only fill gaps that are not more than of a given duration. Essential to the understanding of the Interpolation module is that the module does not have the capability to identify gaps due to potentially unreliable data in a time series. It will only provide an alternative value for those data points of which the quality flag is set to Unreliable. The validation module can be configured to identify unreliable data and set quality flags as appropriate.

Spatial Interpolation

In spatial interpolation mode, the interpolation can be either applied to fill gaps in time series, or to create a new time series for a location using data from other (spatially distributed) locations. Spatial interpolation can also be applied for sampling scalar time series from grid time series, for re-sampling grids, or for creating grids from time series data. Different methods of spatial interpolation are available;

  • spatial interpolation using Kriging
  • spatial interpolation using Inverse Distance Weighting
  • spatial interpolation using bi-linear interpolation
  • Averaging of grid cells within a sub-basin boundary.

 

When available as configuration on the file system, the name of the XML file for configuring an instance of the interpolation module called for example InterpolateHBV_Forecast may be:

 

InterpolateHBV_Forecast 1.00 default.xml

 

InterpolateHBV_Forecast File name for the InterpolateHBV_ForecastData configuration.

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

Figure 54 Root element of the interpolation module configuration

interpolationSet

Root element for the definition of an interpolation step. Multiple entries may exist.

 

Attributes;

  • interploationId : Id of the interpolation defined. Used for reference purposes only. This Id will be included in log messages generated.
serialInterpolation

Root element for the definition of serial interpolation options.

spatialInterpolation

Root element for the definition of spatial interpolation options.

timeSeriesInputSet

Input time series set. Note that when the interpolation module is used to fill gaps in time series the input time series set is the same as the output time series set. The time series sets may include either a single location or a locationSet. Note that the latter may not always be possible when using the “default” interpolation option, as the default may be location specific.

outputSet

Output time series set. Note that when the interpolation module is used to fill gaps in time series the input time series set is the same as the output time series set. Identification is only required when the series generation option is used in spatial interpolation. The locations defined in this timeSeriesSet, and their geographical attributes, determine the locations of the series generated.

6.2.1            Serial interpolation

The serial interpolation option is used to define interpolation options for filling gaps in the time series identified. Multiple methods of interpolation may be identified. These will be executed in order of definition for the same time series (e,g, first linear interpolation, then an extrapolation and finally filling remaining gaps with default values).

 

Figure 55 Elements for defining serial interpolation options in the Interpolation module configuration.

serialInterpolationOption

Selection of type of serial interpolation. Enumeration of available options is;

  • linear ; for linear interpolation between available values
  • block ; for block interpolation (note: the last available value is then used until a new value available).
  • extrapolate ; for extrapolation at start or end of series. Extrapolation uses the last or first value to fill the gaps at the end or start of the series.
  • default ; for replacing unreliable values with a default.
gapLength

Maximum gap length of unreliable data in seconds which will be filled using the interpolation option defined. If the gap is longer, then none of the values will be replaced.

defaultValue

Default value to use to replace interpolation values.

6.2.2            Spatial interpolation

The serial interpolation option is used to define interpolation options for filling gaps in the time series identified using available data from other (spatially distributed) locations. This method can be used to either fill gaps, or to create a new time series.

 

Figure 56 Elements for defining spatial interpolation options in the Interpolation module configuration.

interpolationOption

Selection of type of spatial interpolation. Enumeration of available options is;

  • inversedistance ; for inverse distance weighted interpolation between available values at spatially distributed locations.
  • bilinear ; for bilinear interpolation between available values at spatially distributed locations.
  • kriging ; for interpolation using Kriging between available values at spatially distributed locations.
  • gridcellavaraging; for interpolation of time series based on averaging grid cells (used for example for establishing catchment averages where the catchment size is much larger than the grid cell size).
interpolationType

Specify if spatial interpolation is used for filling gaps in series or for generating a new series. Note in the latter case the output variable will need to be defined. This also defines if the output variable is a grid time series or a scalar time series Enumeration of available options is;

  • seriesfilling ; for filling gaps in time series (scalar time series only).
  • seriesgeneration: for creating a new time series.

 

valueOption

Option to determine how input values are used. Enumeration of available options is;

  • normal ; for using values as is.
  • residual: for applying linear regression first and applying spatial interpolation on residuals of values only.
  • splitwithelevation; for applying different linear regression parameters above and below an elevation split (required only when elevation is considered).
variogram

Root element for the semi-variogram to be used when Kriging is applied.

variogram:type

Type of variogram to be used. Enumeration of available options is;

  • exponential ;
     
  • Gaussian ;
     
  • Linear ;
     
  • Spherical ;
     
  • power
     

 

is the correlation coefficient, and the distance between parameter pairs.

variogram:nugget

Nugget of the variogram

variogram:slope

Slope of the variogram. Used for linear variogram types.

variogram:sill

Sill of the variogram.

variogram:range

Range of the variogram.

numberOfStations

Number of stations to consider in spatial interpolation. Used in Inverse distance when taking a limited number of stations into account. The nearest stations will be used in preference.

regressionElevation

Elevation level at which the regression split is applied.

minimumValue

Minimum value of the output data. For interpolation of rainfall data this should be set to zero. Numerically the interpolation may produce invalid (negative) data.

distanceParameters

Distance parameters for computing actual distances between locations when projection is geographical (WGS1984). Four parameters are required.

debug

Optional debug level. Spatial interpolation is implemented through a DLL. This can produce a log file, depending on level specified. A setting of 1 is the lowest level, a setting of 4 is highest (can produce very extensive log files).

coordinateFile

Coordinate file allocating grid cells to be considered per location. This coordinate file follows a specific format. Locations to be interpolated to are indicated through their spatial location. After each location a list of grid cells (m,n coordinates) to be considered is included.

 

coordinateSystem

Indicates if coordinate system is longitude-latitude this is defined as 1. If not 0 is used and distances are calculated in metres.

inverseDistancePower

Power applied to the inverse distance interpolation.

 

 

 


6.3                Transformation Module Configuration

The Transformation module is a general-purpose module that allows for generic transformation and manipulation of time series data. The module may be configured to provide for simple arithmetic manipulation, time interval transformation, shifting the series in time etc, as well as for applying specific hydro-meteorological transformation such as stage discharge relationships etc.

 

The Transformation module allows for the manipulation and transformation of one or more time series. The utility may be configured to provide for;

  • Manipulation of one or more series using a standard library of arithmetic operators/functions (enumerated);

Addition, subtraction, division, multiplication

Power function, exponential function

  • Hydro-meteorological functions like:

Deriving discharges from stages

Compute potential evaporation

Calculating weighted catchment average rainfall

  • Shifting series in time
  • Time interval conversion:
  • Aggregation
  • Dis-aggregation
  • Converting non-equidistant to equidistant series
  • Handling of typical profiles
  • Data hierarchy
  • Selection of (tidal) peaks

 

When available as configuration on the file system, the name of the XML file for configuring an instance of the interpolation module called for example TransformHBV_Inputs may be:

 

TransformHBV_Inputs 1.00 default.xml

 

TransformHBV_Inputs File name for the TransformHBV_Inputs configuration.

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

Figure 57 Root element of the Transformation module.

transformationSet

Root element for the definition of a transformation (processing an input to an output). . Multiple entries may exist.

 

Attributes;

  • transofrmationId : Id of the transformation defined. Used for reference purposes only. This Id will be included in log messages generated

 

Figure 58 Elements of the definition of an input variable.

inputVariable

Definition of the input variables to be used in transformation. This may either be a time series set or a typical profile. The InputVariable is assigned an ID. This ID is used later in the transformation functions as a reference to the data.

 

Attributes;

  • variableId : ID of the variable (group).Later used in referencing the variable.
  • variableType : Optional type definition of variable (defaults to “any”)
  • convertDatum : Optional Boolean flag to indicate if datum is to be converted.
timeSerieSet

Definition of an input variable as a time series set (see TimeSeriesSet definition).

timeStep

Time step for typical profile if variable to be defined as typical profile.

 

Attributes;

  • unit (enumeration of: second, minute, hour, day, week, nonequidistant)
  • multiplier defines the number of units given above in a time step (not relevant for nonequidistant time steps)
  • divider same function as the multiplier, but defines fraction of units in time step.
relativeViewPeriod

Relative view period of the typical profile to create. If this is defined and the time span indicated is longer than the typical profile data provided, then the profile data will be repeated until the required time span is filled. If the optional element is not provided then the typical profile data will be used only once.

data

Data entered to define the typical profile. Data can be entered in different ways. The typical profile can be defined as a series of values at the requested time step, inserted at the start of the series, or it can be mapped to specific time values (e.g. setting a profile value to hold at 03:15 of every day). Which of these is used depends on the attributes defined.

 

Attributes;

  • value : Required value for each step in the profile
  • monthDay : Attribute value indicating the value entered is valid for a month/day combination. The year value is added depending on the year value in which it is used. The string has the format “ --<month>-<day> ”. For example the 23 rd of August is “-- 08-23 ”.
  • dateTime : Attribute value indicating the value entered is valid for a specific date time combination. The string has the format “ <year>-<month>-<day>T<hour>:<minute>:<second> ”. For example the 23 rd of August is “ 1984-12-31T00:00:00 ”.
  • time : Attribute value indicating the value entered is valid for a specific time, irrespective of the date. The date value is added run time. The string has the format “ <hour>:<minute>:<second> ”. For example “ 01:15:00 ”.
timeZone

Optional specification of the time zone for the data entered (see timeZone specification).

timeZone:timeZoneOffset

The offset of the time zone with reference to UTC (equivalent to GMT). Entries should define the number of hours (or fraction of hours) offset. (e.g. +01:00)

timeZone:timeZoneName

Enumeration of supported time zones. See appendix B for list of supported time zones.

arithmeticFunction

Root element for defining a transformation as an arithmetic function (see next section for details).

ruleBasedTransformation

Root element for defining a rule based transformation (see next section for details on rules).

 

Attributes;

  • rule : definition of aggregation approach. Enumeration of;

selectpeakvalues

selectlowvalues

selectpeakvalueswithincertaingap

selectlowvalueswithincertaingap

equitononequidistant

equitononequidistantforinstantaneousseries

equitononequidistantforaccumulativeseries

datahierarchy

typicalprofiletotimeseries

zerodegreealtitudelevel

datatotimeseries

aggregate

Root element for defining a time aggregation transformation (rules are discussed below)

 

Attributes;

  • rule : definition of aggregation approach. Enumeration of;

instantaneous

accumulative

mean

constant

disaggregate

Root element for defining a time dis-aggregation transformation (rules are discussed below)

 

Attributes;

  • rule: definition of disaggregation approach. Enumeration of;

instantaneous

accumulative

disaggregateusingweights

constant

nonequidistantToEquidistant

Root element for defining transformation of an non-equidistant time series to an equidistant time series. (rules are discussed below)

 

Attributes;

  • rule: definition of approach. Enumeration of;

zero

missing

linearinterpolated

equaltolast

hydroMeteoFunction

Root element for defining one of the available hydro-meteorological transformations.

outputVariable

Definition of the output variables to be written following transformation. See the inputVariable for the attributes and structure. The output variable can only be a TimeSeriesSet (typical profiles are only used as inputs). The OutputVariable is assigned an ID. This ID must be defined as the result of the transformation.

 

6.3.1            ArithmeticFunction & hydroMeteoFunction

Through definition of an arithmetic function, a user defined equation can be applied in transforming a set of input data to a set of output data. Any number of inputs may be defined, and used in the user defined function. Each input variable is identified by its Id, as this is used configuring the function. The function is written using general mathematical operators. A function parser is used in evaluating the functions (per time step) and returning results. These are again assigned to variables which can be linked to output time series through the variableId.

 

Rather than use a usedDefinedFunction, a special function can also be selected from a list of predefined hydroMeteoFunctions. When selected this will pose requirements on other settings.

 

Transformations may be applied in segments, with different functions or different parameters used for each segment. A segment is defined as being valid for a range of values, identified in one of the input variables (see example below).

 

Figure 59 Example of applying segments to a time series

 

Figure 60 Elements of the Arithmetic section of the transformation module configuration

segments

Root element for defining segments. When used this must include the input variable Id used to determine segments as an attribute.

 

Attributes;

  • limitVariablId : Id of input variable used to test against segment limits.
segment

Root element for definition of a segment. At least one segment must be included.

limitLower

Lower limit of the segment. Function defined will be applied at a given time step only if value at that time step in the variable defined as limitVariable is above or equal to this value.

limitUpper

Upper limit of the segment. Function defined will be applied at a given time step only if value at that time step in the variable defined as limitVariable is below this value (below or equal only for the highest segment).

functionType

Element used only when defining a predefined hydroMeteoFunction. Depending on selected function, specific requirements will hold for defining input variables and parameters. If a special function is selected then the user defined function element is not defined; Enumeration of available options is (the most important are discussed below);

  • simpleratingcurve ; for applying a simple power law rating curve.
  • weigthtedaverage : special function for calculating weighted average of inputs. When a value in one of the inputs is missing, the remaining inputs will be used and the weights rescaled to unity.
  • penman: for calculating evaporation using Penman
  • penmannortheast: specific implementation of Penman formula
  • qhrelationtable : allows application of a rating curve using a table.
  • degreemanipulation
userDefinedFunction

Optional specification of a user defined function to be evaluated using the function parser. Only the function need be defined, without the equality sign. The function is defined as a string and may contain Id’s of inputSeries, names of variables and constants defined, and mathematical operators ( +; -; /; *; ^; SIN(); COS(); etc.)

constant

Allows definition of a constant to be used in the function.

coefficient

Optional element to allow coefficients for use in the function to be defined. These coefficients are allocated and Id for later use in the function defined. For user defined functions specific coefficients need to be defined. Multiple entries may be defined.

 

Attributes;

  • coefficientId : Id of the coefficient defined. This can be used in the function.
  • coefficientType : identification of the coefficient. Applied in rule based configurations.
  • value: value of the coefficient
tableColumnData

Definition of a table to use for transforming input variable to output variables.

 

Attributes;

  • nrOfColumns: number of columns in table (should equal 2).
  • variableIdCol1 Input variable associated with first column
  • variableIdCol2 Output variable associated with first column
tableColumnData:data

Element containing data for each row in the table

 

Attributes;

  • col1: value for column 1
  • col2: value for column 2
outputVariable

Id of the output variable from the function. This may be saved to the database by associating the Id to an outputVariable.

flag

Optional element to force saving the result data for the segment with a given flag. This may be used for example to force data from a segment as doubtful. Enumeration is either “unreliable” or “doubtful”. if data is reliable the element should not be included.

 

6.3.2            Stage-Discharge and Discharge-Stage transformation

Stage discharge transformations can be defined using the simpleratingcurve option of the hydroMeteoFunctions. To apply this certain properties must be defined in each segment.

 

For stage-discharge transformation the requirements are;

  • Coefficient values for coefficientId’s “a”, “b” and “c” must be defined.
  • Input variable Id must be “H”
  • Output variable Id must be “Q”.
  • limitVariableId must be “H”.

 

Example:

For stage-discharge transformation the requirements are;

  • Coefficient values for coefficientId’s “a”, “b” and “c” must be defined.
  • Input variable Id must be “Q”
  • Output variable Id must be “H”.
  • limitVariableId must be “Q”.

 

Example:

6.3.3            Establishing catchment average precipitation

Catchment average rainfall can be determined by weighting input precipitation time series. The weightedavarege option of the hydroMeteoFunctions can be applied to include the option of recalculation of weights if one of the input locations is missing. To apply this certain properties must be defined in each segment.

 

For establishing catchment average precipitation the requirements are;

  • functionType must be set to weightedavarege
  • Weights are given as coefficient values with coefficientId’s “a”, “b” and “c” etc.
  • Additional coefficients may be defined to allow for altitude correction.

 

Example:

6.3.4            Aggregation, disaggregation and non-equidistant to equidistant

This set of transformations allows temporal aggregation and disaggregation of time series. The time step defined in the input variable and the output variable determine the howthe time steps are migrated. The configuration need only define the rule followed in aggregation/disaggregation. Aggregation and disaggregation can only be used to transform between equidistant time steps. A nonequidistant series can be transformed to an equidistant series using the appropriate element (see above).

 

Aggregation rules;

  • Instantaneous: apply instantaneous resampling – ie value at cardinal time step in output series is same as in input time series at that time step.
  • accumulative : value in output time series is accumulated sum of values of time steps in input time series (use in for example aggregating rainfall in mm).
  • mean value in output time series is mean of values of time steps in input time series (use in for example aggregating rain rate in mm/hr).
  • constant

 

Disaggregation rules;

  • Instantaneous: apply linear interpolation – ie value at cardinal time step in output series is same as in input time series at that time step. Values in between are interpolated.
  • accumulative : value in output time series is derived as equal fraction of valuein input series. Fraction is determined using ration of time steps.
  • Disaggregateusingweights value in output time series weighted fraction of input value. Weights are defined as coefficients. These are sub-elements to the disaggregation element. The number of coefficients defined should be equal to the disaggregation ration (i.e. 24 when disaggregating from day to hour). The coefficient Id’s should be numbered 1 to n..
  • constant value in output time series at intermediate time steps is equal to the last available value in the input time series.

 

Rules for mapping non-equidistant time series to equidistant time series

  • zero value in output time series is zero if time values do not coincide
  • missing value in output time series is missing if time values do not coincide
  • linearinterpolated value in output time series is interpolated linearly between neighbouring values in input time series
  • equaltolast value in output time series is equal to last available value in input time series.

 

6.3.5            Rule based transformations

The set of rule based transformations is a library of specific data transformation functions. Configuration of the rule based transformation is the same as in the Arithmetic transformation. However, each rule may have specific requirements on the elements that need to be defined. Many parameters that affect the transformation will need to be defined as a coefficient, using the appropriate coefficientType definition.

 

The rule based transformations can be grouped into four main sections;

  • Selection of peak or low values from a time series.
  • Resampling a equidistant time series set using time values specified in a non-equidistant time series set.
  • Data hierarchy
  • Various transformations.

 

Selection of peak or low flow values

Selection of peaks and lows

Set of rules to allow selection of peaks and lows from an input time series.

 

Enumerations in the rule attribute of the ruleBasedTransformation element;

  • selectpeakvalues
  • selectlowvalues
  • selectpeakvalueswithincertaingap
  • selectlowvalueswithincertaingap

 

The first two enumerations will select all peaks or lows in the time series. The second two will select peaks only if there is a defined gap in time between peaks. If not they are considered to be of dependent and only the highest peak of the dependent sets will be returned.

 

Requirements for definitions of peak selections using gaps to define independence are;

  • A coefficientId “a” must be defined. The coefficientType must be set to “gaplengthinsec”. The value attributed defines the length of the minimum gap in seconds.

 

Example:

 

Sampling values from equidistant time series

This section of the rule based transformation can be applied to sample items from an equidistant time series at the time values in a non-equidistant time series. This may be required when applying transformations to a non-equidistant time series. The values to add will first need to be resampled to the right time value. An example is when wind and wave information is required at the time of the tidal peaks for entry in a lookup table.

 

Enumerations in the rule attribute of the ruleBasedTransformation element;

equitononequidistant

equitononequidistantforinstantaneousseries

equitononequidistantforaccumulativeseries

 

The first two elements are equivalent. The last will consider accumulations of the input variable up to the time value sampled.

 

Requirements for definitions of resampling equidistant time series are;

  • The limitVariableId attribute of the segements element must be the non-equidistant time series which determines the time values at which the equidistant series is to be sampled.
  • The userDefinedFunction must contain the equidistant time series to be sampled
  • The outputVariableId must resolve to a non-equidistant time series.

 

Example:

 

Data Hierarchy

This is a simple method to merge overlapping equidistant time series in a single equidistant series. Gaps in foremost (first) series will be filled with data of second series if a valid value is available at the current time step, otherwise the gap is filled with data from the third series and so on until no more time series are available. Only missing data values and unreliable values are filled. Doubtful values remain in the result series as doubtful.

Figure 61 Schematic example of merging series using data hierarchy.

 

In example above Series 1 is the most important time series, Series 2 has a lower hierarchy and series 3 has the lowest hierarchy. The resulting time series has values from all 3 series as shown in figure above.

 

Data hierarchy poses no specific requirements to variables defined. Only the Id of the output variable is of importance.

Creating time series from typical profiles

Typical profiles can be defined in the inputVariable as described above. To use a typical profile it must first be mapped to a dynamic time series. This can then be retrieved in a later configuration of a module for use.

 

Enumerations in the rule attribute of the ruleBasedTransformation element;

  • typicalprofiletotimeseries:
  • datatotimeseries

 

The first type of mapping is used when the typical profile has a concept of date/time (e.g. must be mapped to specific dates or time values). The second is used when only a series of data is given. The time series is then filled with the first data element given as the first time step of the relative view period to be created.

 

Typical profile mapping poses no specific requirements to variables defined. Only the Id of the output variable is of importance.

 


6.4                Import Module Configuration

The import module allows data from external source to be imported into DELFT-FEWS. Data may be provided to FEWS in a variety of formats. The approach taken in the import module is that a class is defined for each of the file formats that can be imported.

 

Data is imported from specified directories. An attempt is made to import all files in the directories configured. If a file conforms to the expected format then the data will be imported. If the file does not conform to the expected format, it will not be imported, but will be moved to a configurable directory with failed import files.

 

Two main groups of import can be defined;

  • Importing data in the XML format defined by the Environment Agency, UK.
  • Importing of various data formats (including ASCII formats, png files - e.g. meteosat images - grids and GRIB files).

 

On importing data, the approach to be used for converting flags, units, locations and parameters can be defined. These conversions are identified by referring to the appropriate configuration files (see Regional Configuration). When data is imported to an equidistant time series, a time tolerance may also be defined. If the time recorded is within this tolerance it will be snapped to the cardinal time step in the imported series.

 

When available as configuration on the file system, the name of the XML file for configuring an instance of the import module called for example ImportRTS may be:

 

ImportRTS 1.00 default.xml

 

ImportRTS File name for the ImportRTS configuration.

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

6.4.1            Time Series Import Module

The time series import class can be applied to import data from a variety of external formats. The formats are included in an enumeration of supported import types. Each of these enumerations is used for a specifically formatted file.

 

Figure 62 Elements of the TimeSeriesImport configuration

import

Root element for the definition of an import run task. Each task defined will import data in a specified format from a specified directory. For defining multiple formats, different import tasks reading from different directories must be defined.

general

Root element for general definitions used in the import runs.

description

Optional description for the import run. Used for reference purposes only.

importType

Specification of the format of the data to be imported. The enumeration of options includes;

  • MSW : Import of data provided by the MSW System (Rijkswaterstaat, the Netherlands).
  • KNMI : Import of synoptic data from KNMI (Dutch Meteorological Service).
  • WISKI : Import of time series data from the WISKI Database system (Kisters AG).
  • DWD-GME : Import of NWP data of the DWD Global Modell, (German Meteorological Service). This is a grid data format.
  • DWD-LM : Import of NWP data of the DWD Lokal Modell, (German Meteorological Service). This is a grid data format.
  • GRIB : Import of the GRIB data format. General format for exchange of meteorological data.
  • EVN: Import of data in the EVN format (Austrian Telemetry)
  • METEOSAT: Import of images form meteosat satellite

 

folder

Folder to import data from. This may be a UNC path (ie located on the network).

failedFolder

Folder to move badly formatted files to. This may be a UNC path (ie located on the network).

idMapId

ID of the IdMap used to convert external parameterId’s and locationId’s to internal parameter and location Id’s. Each of the formats specified will have a unique method of identifying the id in the external format. See section on configuration for Mapping Id’s units and flags.

unitConversionsId

ID of the UnitConversions used to convert external units to internal units. Each of the formats specified will have a unique method of identifying the unit in the external format. See section on configuration for Mapping Id’s units and flags.

flagConversionsId

ID of the FlagConversions used to convert external data quality flags to internal data quality flags. Each of the formats specified will have a unique method of identifying the flag in the external format. See section on configuration for Mapping Id’s units and flags.

missingValue

Optional specification of missing value identifier in external data format.

importTimeZone

Time zone the external data is provided in if this is not specified in the data format itself. This may be specified as a timeZoneOffset, or as a specific timeZoneName.

timeZoneOffset

The offset of the time zone with reference to UTC (equivalent to GMT). Entries should define the number of hours (or fraction of hours) offset. (e.g. +01:00)

timeZoneName

Enumeration of supported time zones. See appendix B for list of supported time zones.

gridStartPoint

Identification of the cell considered as the first cell of the grid. This may be in the upper left corner or in the lower left corner. Enumeration of options include;

  • NW : for upper left
  • SW : for lower left
tolerance

Definition of the tolerance for importing time values to cardinal time steps in the series to be imported to. Tolerance is defined per location/parameter combination. Multiple entries may exist.

 

Attributes;

  • locationId : Id of the location tolerance is to be considered for.
  • parameterId : Id for the parameter  tolerance is to be considered for.
  • timeUnit : Spefication of time units tolerances is defined in (enumeration).
  • unitCount : integer number of units defined for tolerance.
startTimeShift

Specification of a shift to apply to the start time of a data series to be imported as external forecasting. This is required when the time value of the first data point is not the same as the start time of the forecast. This may be the case in for example external precipitation values, where the first value given is the accumulative precipitation for the first time step. The start time of the forecast is then one time unit earlier than the first data point in the series. Multiple entries may exist.

startTimeShift:locationId

Id of the location to apply the startTimeShift to.

startTimeShift:parameterId

Id of the parameter to apply the startTimeShift to.

timeSeriesSet

TimeSeriesSet to import the data to. Multiple time series sets may be defined, and each may include either a (list of) locationId’s ar a locationSetId. Data imported is first read from the source data file in the format specifed. An attempt is then made to map the locationId’s and the parameterId’s as specified in the IdMap’s to one of the locations/parameters defined in the import time series sets. If a valid match is found, then the time values are mapped to those in the timeSeriesSet, taking into account the tolerance for time values. A new entry is made in the timeSeries for each valid match made.

 

For non-equidistant time series the time values imported will be taken as is. For equidistant time series values are only returned on the cardinal time steps. For cardinal time steps where no value is available, no data is returned.

externUnit

For some data formats an external unit is not defined in the file to be imported. This elements allows the unit to be specified explicitly. This unit is then used in possible unit conversions.

 

Attributes;

  • parameterId: Id of the parameter for which a unit is specified. This is the internal parameter Id.
  • unit: specification of unit. This unit must be available in the UnitConversions specified in the unitConversionsId element.
gridRecordTimeIgnore

Boolean flag to specify if the start of forecast is read from the GRIB file or if it is inferred from the data imported. In some GRIB files a start of forecast is specified, but the definition of this may differ from that used in DELFT-FEWS.

 

 

6.4.2            Example: Import of Meteosat images as time-series

Meteosat Images are generally imported as images in <filename>.png format. The Meteosat images constitute a time series of png images, that are geo-referenced by means of a specific world file. Each image needs its own world file, which in case of PNG carries the extension <filename>.pgw .

Import of images in another format, such as JPEG is also possible. The corresponding world file for a JPEG file has the extension <filename>.jpg .

The images are imported via a common time series import, for which a specific image parameter needs to be specified in a parameterGroup via the parameter id image .

 

<parameterGroup id="image">

  <parameterType>instantaneous</parameterType>

  <unit>-</unit>

  <valueResolution>8</valueResolution>

  <parameter id="image">

   <shortName>image</shortName>

  </parameter>

</parameterGroup>

 

The value resolution indicates the resolution of the values of the pixels (grey tones) in the Meteosat images. In this case 8 grey tones are resampled into a single grey tone for storage space reductions. In the module for the timemeseries import run for a Meteosat image the import is then configured as follows:

 

<import>

  <general>

   <importType>GrayScaleImage</importType>

   <folder>$REGION_HOME$/Import/MeteoSat</folder>

            <idMapId>IdImportMeteosat</idMapId>

   </general>

 

  <timeSeriesSet>

   <moduleInstanceId>ImportMeteosat</moduleInstanceId>

   <valueType>grid</valueType>

   <parameterId>image</parameterId>

   <locationId>meteosat</locationId>

   <timeSeriesType>external historical</timeSeriesType>

   <timeStep unit="minute" multiplier="15"/>

   <readWriteMode>add originals</readWriteMode>

   <synchLevel>4</synchLevel>

   <expiryTime unit="day" multiplier="750"/>

  </timeSeriesSet>

</import>

 

The goereferenced image can then be displayed in the grid display.

6.4.3            EA Import module

A specific import class is available for importing time series data from the XML format specified by the UK Environment Agency. The configuration items required are a sub-set of those required in the more generic time series import format. This is due to much of the required information being available in the XML file itself (ie file is self describing).

 

Figure 63 Elements of the EAImport configuration.

 


 

6.5                Export Module Configuration

The export module allows (forecast) data from DELFT-FEWS to be exported for use in external sources. Data is currently provided to these sources by FEWS in an XML format as defined by the UK environment agency. Where other (e.g. ASCII) formats, specific classes can be added to the export module to extend the list of formats supported.

 

On exporting data, the approach to be used for converting flags, units, locations and parameters can be defined. These conversions are identified by referring to the appropriate configuration files (see Regional Configuration).

 

File exported are written to the path specified. The file name of files to be exported. The filename is constructed as a time string (in milliseconds). An optional prefix can be applied to the time stamp string.

 

When available as configuration on the file system, the name of the XML file for configuring an instance of the import module called for example ExportForecast may be:

 

ExportForecast 1.00 default.xml

 

ExportForecast File name for the ExportForecast configuration.

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

Figure 64 Elements of the exportRun configuration.

folder

Folder to export data to. This may be a UNC path (ie located on the network).

idMapId

ID of the IdMap used to convert internal parameterId’s and locationId’s to external parameter and location Id’s. See section on configuration for Mapping Id’s units and flags.

unitConversionsId

ID of the UnitConversions used to convert internal units to external units. See section on configuration for Mapping Id’s units and flags.

flagConversionsId

ID of the FlagConversions used to convert internal data quality flags to external data quality flags. See section on configuration for Mapping Id’s units and flags.

exportFilePrefix

Optional string to use as prefix in the export file name.

 

exportMissingValue

Optional specification of missing value identifier in external data format.

temporaryFilePrefix

Optional prefix to the file name when writing the file. This can be used by systems reading the file to identify if the file is being written, thus avoiding concurrent reading/writing of a file. If not defined the prefix “tmp” is used. On completion of the file, an atomic replace of the filename is done.

exportTimeZone

Time zone the external data is exported to. This may be specified as a timeZoneOffset, or as a specific timeZoneName.

timeZoneOffset

The offset of the time zone with reference to UTC (equivalent to GMT). Entries should define the number of hours (or fraction of hours) offset. (e.g. +01:00)

timeZoneName

Enumeration of supported time zones. See appendix B for list of supported time zones.

timeSeriesSet

TimeSeriesSets defining the data to be exported. Multiple time series sets may be defined, and each may include either a (list of) locationId’s ar a locationSetId.


6.6                General Adapter Configuration

A key feature of DELFT-FEWS is its ability to run external modules to provide essential forecasting functionality. These modules may be developed by Delft Hydraulics as well as by other companies or institutions. The DELFT-FEWS system does not have any knowledge of the specific implementation of these modules. It is rather the central philosophy to have an open system, that is able to treat external modules as plug-ins that can be used if needed.

 

The General Adapter is the part of the DELFT-FEWS system that implements this feature. It is responsible for the data exchange with the modules and for executing the modules and their adapters. The central philosophy of the General Adapter is that it knows as little as possible of module specific details. Module specific intelligence is strictly separated from the DELFT-FEWS system. In this way an open system can be guaranteed. Module specific intelligence required by the module to run is vested in the module adapters.

 

Communication between the General Adapter and a module is established through the published interface (PI). The PI is an XML based data interchange format. The General Adapter is configured to provide the data required for a module to run in the PI format. A module adapter is then used to translate the data from the PI to the module native format. Vice versa, results will first be exported to the PI format by a module adapter before the General Adapter imports them back into DELFT-FEWS.

 

The General Adapter module can be configured to carry out a sequence of five types of tasks;

  • Startup Activities. These activities are run prior to a module run and any export import of data. The activities defined are generally used to remove files from previous runs that may implicate the current run.
  • Export Activities. These activities defined all items to be exported through the published interface XML formats to the external module, prior to the module or the module adapters being initialised.
  • Execute Activities. The execute activities define the external executables or Java classes to be run. Tracking of diagnostics from these external activities is included in this section.
  • Import Activities: These activities define all items to be imported following successful completion of the module run.
  • Shutdown Activities. These activities are run following completion of all other activities The activities defined are generally used to remove files no longer required.

 

Figure 65 Schematic interaction between the General Adapter and an external module

 

When available as configuration on the file system, the name of the XML file for configuring an instance of the general adapter module called for example HBV_Maas_Forecast may be:

 

HBV_Maas_Forecast 1.00 default.xml

 

HBV_Maas_Forecast File name for the HBV_Maas_Forecast configuration.

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

Figure 66 Elements of the General Adapter configuration

general

Root element of general settings.

activities

Root element for the activities to be defined. The activities are defined in a fixed order;

  • startUpActivities
  • exportActivities
  • executeActivities
  • importActivities
  • shutDownActivities

 

6.6.1            General settings

Figure 67 Elements of the general section of the general adapter configuration

description

Optional description of the configuration. Used for reference purposes only.

rootDir

Root directory for the external module. Other directories can be defined relative to this rootDir using predefined tags (see comment box below).

workDir

Working directory to be used by the external module. When started this directory will be the current directory.

exportDir

Directory to export data from DELFT-FEWS to the external module. All Published Interface files will be written to this directory (unless overruled in naming the specific export files).

exportDataSetDir

Directory to export module datasets from DELFT-FEWS to the external module. A module dataset is a ZIP file, which will be unzipped using this directory as the root directory. If the zip file contains full path information, this will be included as s tree of subdirectories under this directory.

exportIdMap

ID of the IdMap used to convert internal parameterId’s and locationId’s to external parameter and location Id’s. See section on configuration for Mapping Id’s units and flags.

importDir

Directory to import result data from the external module to DELFT-FEWS. All Published Interface files will be read from this directory (unless overruled in naming the specific export files).

importIdMap

ID of the IdMap used to convert external parameterId’s and locationId’s to interna l parameter and location Id’s. This may be defined to be the same as the import directory, but may also contain different mappings. See section on configuration for Mapping Id’s units and flags.

dumpFileDir

Directory for writing dump files to. Dump Files are created when one of the execute activities fails. A dump file is a ZIP file which includes all the dumpDir directories defined. The dump file is created immediately on failure, meaning that all data and files are available as they are at the time of failure and can be used for analysis purposes. The ZIP file name is time stamped to indicate when it was created.

 

dumpDir

Directory to be included in the dump file. All contents of the directory will be zipped. Multiple dumpDir’s may be defined.

 

NOTE: ensure that the dumpDir does not include the dumpFileDir. This creates a circular reference and may result in corrupted ZIP files.

diagnosticFile

File name and path of diagnostic files created in running modules. This file should be formatted using the Published Interface diagnostics file specification.

missVal

Optional specification of missing value identifier to be used in PI-XML exported to modules and imported from modules.

 

NOTE: it is assumed an external uses the same missing value identification for both import and export data.

convertDatum

Optional Boolean flag to indicate level data is used and produced by the module at a global rather than a local datum. The convention in DELFT-FEWS is that data is stored at a local datum. If set to true data in parameter groups supporting datum conversion will be converted on export to the global datum by adding the z coordinate of the location. (see definition of parameters and locations in Regional Configuration).

timeZone

The time zone with reference to UTC (equivalent to GMT) for all time dependent data communicated with the module. If not defined, UTC+0 (GMT) will be used.

 

timeZoneOffset

The offset of the time zone with reference to UTC (equivalent to GMT). Entries should define the number of hours (or fraction of hours) offset. (e.g. +01:00)

timeZoneName

Enumeration of supported time zones. See appendix B for list of supported time zones.

 

6.6.2            Startup Activities

Figure 68 Elements of the startUpActivities section of the General Adapter configuration.

purgeActivity

Root element of a purge activity used to delete files from previous runs. Multiple purge activities may be defined.

filter

Filter specifying files to be removed. Wildcards may be used.

 

Example (not the use of tags to define the directory names):

6.6.3            Export Activities

Figure 69 Elements of the ExportActivity section

 

Export activities are defined to allow exporting various data objects from DELFT-FEWS to the external modules. The list of objects that can be exported (see figure above) includes;

  • exportStateActivity to export module states
  • exportTimeSeriesActivity to export time series for scalar or polygon time series
  • exportMapStacksActivity to export time series for grid time series
  • exportProfilesActivity to export time series for longitudinal profile time series
  • exportDataSetActivity to export module datasets
  • exportParameterActivity to export module parameters

 

Note that for most types of exportActivity, multiple entries may exist.

Figure 70 Elements of the ExportStatesActivity section.

description

Optional description for the export states configuration. Used for reference purposes only.

moduleInstanceId

Id of the moduleInstance that has written the state to be exported. Generally this will be the same as the Id of the current instance of the General Adapter. This can also be the ID of another instance of the General Adapter. The latter is the case when using a state in a forecast run that has been written in an historical run.

stateExportDir

Directory to export the states to. This is the export location for the (binary) state files.

stateConfigFile

Name (and location) of the PI-XML file describing the states. If the directory location is not explicitly specified the file will be written in the exportDir defined in the general section.

stateLocations

Root element for the description of the state. Both a read location and a write location will need to be defined. This allows the name of the file(s)/directory to be different on read and write. Multiple locations may be defined, but these must all be of the same type.

 

Attributes;

  • type: indication of type of state to be imported. This may either be “directory” or “file”. Note that multiple locations are supported only if type is “file”.
stateLocation

Root element of a state location

readLocation

Location where the external module will read the state. This is the location (and name of file/directory) where the General Adapter writes the state.

writeLocation

Location where the external module is expected to write the state. This is the location (and name of file/directory) where the General Adapter expects to read the state.

stateSelection

Root element to specify how a state to be exported to the external module is to be selected. Two main groups are available, cold states and warm states. Only one of these types can be specified. Note that if a warm state selection is specified and an appropriate warm state cannot be found, a cold state will be exported by default.

coldState

Root element for defining the stateSelection method to always export a cold state.

coldState:groupId

Id of the group of cold states to be used. This must be a groupId as defined in the ColdModuleInstanceStateGroups configuration (see Regional Configuration).

coldState:startDate

Definition of the start date of the external module run when using the cold state. This startDate is specified relative to the start time of the forecast run. A positive startDate means it is before the start time of the forecast run.

 

Attributes;

  • unit (enumeration of: second, minute, hour, day, week)
  • multiplier defines the number of units given above.
  • divider same function as the multiplier, but defines fraction of units.

 

warmState

Root element for defining the stateSelection method to search for the most suitable warm state.

warmState:stateSearchPeriod

Definition of the search period to be used in selecting a warm state. The database will return the most recent suitable warm state found within this search period.

 

Attributes;

  • unit (enumeration of: second, minute, hour, day, week)
  • start defines the start time relative to the start of the forecast run in number of units given above (negative is before start of forecast).
  • end defines the end time relative to the start of the forecast run in number of units given above (negative is before start of forecast).
coldStateTime

Definition of the start time to use for a cold state if a suitable state is not found within the warm state search period.

 

Attributes;

  • unit (enumeration of: second, minute, hour, day, week)
  • value defines the start time of the cold state run relative to the start of the forecast run in number of units given above (negative is before start of forecast).

 

Figure 71 Elements of the exportTimeSeries section

description

Optional description of the timeSeries export configuration.

exportFile

Name (and location) of the PI-XML file with exported time series. If the directory location is not explicitly specified the file will be written in the exportDir defined in the general section.

timeSerieSets

Root element for defining timeSerieSets to be exported.

timeSerieSets: timeSerieSet

TimeSeriesSets to be exported. These may contain either a (list of) locations or a locationSet. Multiple entries may be defined.

Figure 72 Elements of the ExportMapStacksActivity.

description

Optional description of the timeSeries export configuration.

exportFile

Name (and location) of the PI-XML file describing the map stack of the exported grid time series. If the directory location is not explicitly specified the file will be written in the exportDir defined in the general section.

gridFile

Root element for defining file name of grid to be exported

locationId

LocationId for grid to be exported.

gridName

Name of the files for the grid to be exported. For grid files where each time slice is stored in a different file, this name is the prefix for the full file name. The final file name is created using an index of files exported (e.g the file name for the 4 th time step is grid00000.004).

gridFormat

Format of the exported grid. Enumeration of options include;

  • asc : for exporting to ARC-INFO ASCII grid format
  • pcrgrid : for exporting to PCRaster native grid file format
  • usgsdem : for exporting to USGS DEM format (BIL)
timeSerieSet

TimeSeriesSets to be exported. These should contain only one locationId. For exporting multiple grids, multiple exportMapStack activities should be defined.

 

exportProfilesActivity

Configuration of the exportProfiles activity is identical to the exportTimeSeries Activity.

Figure 73 Elements of the exportDataSets section

description

Optional description of the module dataset export configuration.

moduleInstanceId

Optional reference to the moduleInstanceId of the moduleDataSet to be exported. If not defined the moduleInstanceId of the current module instance is taken as a default (see section on Module Datasets and Parameters).

Figure 74 Elements of the exportParameter section

description

Optional description of the module parameter configuration.

moduleInstanceId

Optional reference to the moduleInstanceId of the moduleParameter to be exported. If not defined the moduleInstanceId of the current module instance is taken as a default (see section on Module Datasets and Parameters)

fileName

Name (and location) of the PI-XML file with exported parameters. If the directory location is not explicitly specified the file will be written in the exportDir defined in the general section.

6.6.4            Execute Activities

Figure 75 Elements of the ExecuteActivity configuration

executeActivity

Root element for the definition of an execute activity. For each external executable or Java class to run, an executeActivity must be defined. Multiple entries may exist.

description

Optional description for the activity. Used for reference purposes only.

command

Root element to define command to execute.

executable

File name and location of the executable to run the command is an executable. The file name may include environment variables, as well as tags defined in the general adapter or on the global.properties.

className

Name of Java Class to run if the command defined as a Java class. This class may be made available to DELFT-FEWS in a separate JAR file in the \Bin directory. It must implement the ModuleAdapterRunnable interface to run.

arguments

Root element for defining arguments to be passed to the executable/Java class

argument

Definition of an argument to be passed to the executable/Java Class

environmentVariables

Root element for defining environment variable prior to running the executable/Java class

environmentVariable

Definition of an environment variable prior to running the executable/Java class

environmentVariable.name

Name of environment variable

environmentVariable.value

Value of environment variable

timeOut

Optional timeout to be used when running module (in milliseconds). If run time exceeds timeout it will be terminated and the run considered as having failed.

6.6.5            Import Activities

Figure 76 Elements of the ImportActivities configuration

description

Optional description of import activity. Used for reference purposes only

importStateActivity

Root element for importing modules states resulting from the run of the external modules. Multiple elements may be defined. If no state is to be imported (for example in forecast run as opposed to state run), then the element should not be defined.

stateConfigFile

Name (and location) of the PI-XML file describing the states to be imported. If the directory location is not explicitly specified the file will be expected to be read from the importDir defined in the general section. This file contains all necessary information to define state type and location. The moduleInstanceId of the state imported is per definition the current module instance.

importTimeSeriesActivity

Root element for importing scalar and polygon time series resulting from the run of the external modules. Multiple elements may be defined.

importMapStacksActivity

Root element for importing grid time series resulting from the run of the external modules. Multiple elements may be defined.

importProfilesActivity

Root element for importing longitudinal profile time series resulting from the run of the external modules. Multiple elements may be defined.

importFile

PI-XML file describing the time series to be imported. The file contains all information on type of data to be imported (scalar, longitudinal, grid, polygon). For all data types except the grid the file also contains the time series data If the directory location is not explicitly specified the file will be expected to be read from the importDir defined in the general section.

6.6.6            Shutdown Activities

Figure 77 Elements of the Shutdown Activities configuration

This activity is the identical to the startUpActivities. The only difference is that these are carried out after the module run and import of data. See definition of StartUp activities for configuration.


6.7                Lookup Table module configuration

The Lookup table module is used to derive a simple value based on combining input values of different time series in the forecast database. These are then used to search in a multi-dimensional lookup table to derive the requested output. The module may also be employed to derive a decision based on a hierarchic set of rules (critical conditions table).

 

The lookup table utility is predominantly applied as the forecasting tool for coastal forecasting. Typically values such as predicted surge, wind force and direction, wave height, fluvial flow in an estuary are used to predict values at a number of points on the coast or in an estuary. These values are generally defined as a Lookup Index. This can then be resolved to a text string such as “Flood Warning” or “Severe Flood Warning” for use in for example reports using the ValueAttributeMaps (see Regional Configuration).

 

Three main types of lookup table may be defined;

  • simple table lookup. This is a two column (or row) table where the value at each time step in the input series is used to identify a relative position in the first column (or row). The result value is found in the second column (or row) at the same relative position.
  • Multi-dimensional lookup. This is a lookup in a matrix. Two input series are required. One is used to find the relative row position in the matrix at each tome step, while the other is used to find the relative column position in the matrix. The output value is found through resolving these relative positions in the matrix values using bi-linear interpolation.
  • Critical condition tables. These defined a set of heuristic rules. Multiple inputs can be combined and an output is found through evaluating the heuristic rules. A default output (also using rules can be defined).

 

 

When available as configuration on the file system, the name of the XML file for configuring an instance of the general adapter module called for example Coastal_Lookup_Forecast may be:

 

Coastal_Lookup_Forecast 1.00 default.xml

 

Coastal_Lookup_Forecast File name for the Coastal_Lookup_Forecast configuration.

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

 

Figure 78 Elements of the lookup table configuration.

LookupSet

Root element of the definition of a lookup table. Multiple entries may exist.

 

Attribute;

  • lookupSetId : Id of the lookup table. Used for reference purposes only (e.g. in log messages).
inputVariable

Definition of input variable to be used in the lookup table. For each entry in the lookup table an input variable will need to be identified. The variableId is used to refer to the time series. See Transformation Module for definition of inputVariable configuration.

outputVariable

Definition of output variable as a result of the lookup table. A single timeSeriesSet for one location output variable per lookup table is defined.

comment

Optional comment on lookup display configuration. Used for reference purposes only.

criticalConditionLookup

Root element for definition of a critical condition table. If no results can possibly be returned by any of the conditions specified, a defaultValue should be defined as a set of rules.

 

Attributes;

  • Id Id of the criticalConditionTable. Used for reference purposes only
simpleTableLookup

Root element for definition of a simple table lookup. Multiple entries may exist.

 

Attributes;

  • lookUpVariableId Id of the input variable to be used in the table.
  • outputVariableId Id of the output variable.
  • rows number of rows in lookup table (if lookup is defined per row then this is equal to 1).
  • cols number of columns in lookup table (if lookup table is defined per col then this is equal to 1).
  • type Optional indication of type of value in lookup table (same type will be returned). Enumeration of “float”or “int”.
multiDimensionalLookup

Root element for definition of a multidimensional lookup table. Multiple entries may exist.

 

Attributes;

  • lookUpRowVariableId Id of the input variable to be used in the table for finding relative row position.
  • lookUpColVariableId Id of the input variable to be used in the table for finding relative column position..
  • outputVariableId Id of the output variable.
  • rows number of rows in lookup table (matrix).
  • cols number of columns in lookup table (matrix).
  • type Optional indication of type of value in lookup table. Enumeration of “float”or “int”.

6.7.1            criticalConditionLookup

Figure 79 Elements of the criticalConditionLookup configuration

criticalCondition

Definition of a critical condition as a set of rules. Multiple entries may exist. When multiple entries do exist, then these will be resolved sequentially until a condition defined is met. The result is then written to the output time series. Each condition holds a set of rules. Each rule is resolved to a Boolean true or false. Rules can be combined in ruleGroups using Boolean operators. If a “true” value returned through combination of all rules and ruleGroups specified, then the conditions specified are met.

 

Attributes (only required attributes defined);

  • rule: string value for result to be returned if conditions specified are met (for reference purposes only).
  • ruleIndex: index value returned if conditions specified are met. This is the value returned in the output time series. Value given is either a numerical value enclosed in quotes (e.g. “4” or “Missing” to indicate a missing value should be returned).
ruleCriteria

Root element for definition of set of rules and ruleGroups. Multiple ruleCriteria can be defined. These are combined using the logical operator defined.

ruleCriteriaLogicalOperator

Operator for combining ruleCriteria to single Boolean value. Enumeration of “ and ” and “ or ”.

rule

Definition of a rule to resolve to a Boolean value.

 

Attributes;

  • variable: id of Input variable to use in evaluating rule
  • operator: Definition of operator to be used in comparison. Enumeration of options include;

lt : less than

le : less than or equal to

eq : equal to

ge : greater than or equal to

gt : greater than

ne : not equal to

  • value: Value to compare input variable to using operator defined.
  • logical: optional definition of logical operator to combine sequence of rules (for rules defined in a rule group only). Enumeration of “ and ” and “ or ”.
ruleGroup

Root element for defining a rule group. A rule group is a sequence of rules. Each rule is configured as defined above, and combined using the logical operator given in the rule. The logical operator need not be included in the last rule defined.

 

Example:

defaultValue

The default value element is identical to the specification of a criticalConditon as described above.

 

6.7.2            SimpleTableLookup

Figure 80 Elements of the simpleTableLookup configuration

LookUpData

Row vector of data used to find relative position of input variable.

 

Attributes;

  • number : optional definition of number of entries (otherwise inferred from data provided)
  • type : optional type indication of data. Enumeration of “float” and “int”.
  • separator : optional indication of separator string used between values. Default is space. Enumeration of;

space

data

Element containing data vector separated by separator character defined.

rowwise

Element to define vector of data to lookup data in. Data value at relative position is returned. Attributes are same as LookUpData element. Use this element if data is provided as a row.

columnwise

Element to define vector of data to lookup data in. Data value at relative position is returned. Use this element if data is provided as a one value per row (as a column)

 

Attributes;

  • number : optional definition of number of entries (otherwise inferred from data provided)
  • type : optional type indication of data. Enumeration of “float” and “int”.
  • separator : optional indication of separator string used between values. Default is space. Enumeration of;

lineseparator

info

Element containing information on how values are determined in lookup vector using the relative position determined.

info:extrapolation

Definition of how to extrapolate when relative position is above last or below first value in vector. Enumeration includes;

  • none : no extrapolation, missing value is returned
  • minmax : limit values returned to minimum/maximum of vector
  • linear : linear extrapolation using last or first two values in vector
info:interpolation

Definition of how to interpolate between values in vector. Enumeration includes;

  • class : returns closest value in vector.
  • linear : linear interpolation

 

Example:

6.7.3            MultipeDimensionLookup

Figure 81 Elements of the multiDimensionalLookup configuration

lookupColData

Row vector of data used to find relative position in matrix columns of input variable defined as lookUpColVariableId.

 

Attributes;

  • number : optional definition of number of entries (otherwise inferred from data provided)
  • type : optional type indication of data. Enumeration of “float” and “int”.
  • separator : optional indication of separator string used between values. Default is space. Enumeration of;

space

lookupRowData

Row vector of data used to find relative position in matrix rows of input variable defined as lookUpRowVariableId.

 

Attributes;

  • number : optional definition of number of entries (otherwise inferred from data provided)
  • type : optional type indication of data. Enumeration of “float” and “int”.
  • separator : optional indication of separator string used between values. Default is space. Enumeration of;

space

rowwise

Element for defining rows of marix as a vector of data on one line. For definition see simpleTableLookup. The number of rowwise elements provided must be equal to the number of columns defined in the multiDimensionalLookup element. Each rowwise vector must contain as many values as defined in cols in the multiDimensionalLookup element.

colwise

Element for defining rows of marix as a vector of data on one multiple lines. For definition see simpleTableLookup element. The number of colwise elements provided must be equal to the number of columns defined in the multiDimensionalLookup element. Each colwise vector must contain as many values as defined in cols in the multiDimensionalLookup element.

Info

See definition in simpleTableLookup element

 

Example:

 

 

 

6.8                Correlation Module Configuration

The Correlation Module is used to estimate a level at a downstream location through correlation of historical levels and discharges at specific upstream locations. This utility is provided through an interactive interface available on the operator client as well as for running automatically using a fixed correlation in the Forecasting Shell Server. The module may employ data from historical events at different locations as available in the central database

 

The module can be used in two ways:

  • As an automatic forecasting module. The module is then run through a preconfigured workflow with all required inputs being retrieved from those available in the database. Results are returned to the database. These results are then available for later viewing through for example a suitably configured report. In this mode a module Instance of the correlation module is defined as described below.
  • In interactive mode. In this mode the module is used through the correlation display available on the operator client. In this mode no results are returned to the database. A module instance does not need to be created in this mode, all required configuration settings are selected through appropriate options in the dialogue.

 

The correlation module uses two associated configuration itemsto establish correlations, these are the CorrelationEventSets and the TravelTimeSets.

 

When available as configuration on the file system, the name of the XML file for configuring an instance of the correlation module called for example Correlation_Severn_Forecast may be:

 

Correlation_Severn_Forecast 1.00 default.xml

 

Correlation_Severn_Forecast File name for the Correlation_Severn_Forecast configuration.

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

Figure 82 Elements of the correlationSets configuration

correlationSet

Root element for defining a correlation to be applied..

inputTimeSeriesSet

Time series set to be used as input for the correlation. This time series set can either be a complete hydrograph (e.g. equidistant time series). It may also be a non-equidistant series of peaks sampled using a transformation module defined previously in the workflow.

outputTimeSerieSet

Time series set to be used as input for the correlation. Values are returned for the same time step as the input series.

correlation

Root element for the definition of a correlation

forecastLocationId

LocationId for the forecast location. This is the location to be defined in the output time series set.

equationType

Definition of the equation type to be applied in determining the correlation.

 

Attributes;

  • equationType . Selection of equation type. Enumeration of options include

polynomial
 

simple_linear
 

exponential_divide
 

exponential_multiply
 

power
 

logarithmic
 

hyperbolic
 

  • polynomialOrder : Integer value for value of polynomial order. Applies only if polynomial equation is selected.
eventSetsDescriptorId

Id of the event sets to be used. This id is defined in the CorrelationEventSetsDescriptors (see Regional Configuration). A suitable CorrelationEventSets configuration must be available (see below).

travelTimesDescriptorId

Id of the event sets to be used. This id is defined in the CorrelationEventSetsDescriptors (see Regional Configuration). A suitable CorrelationEventSets configuration must be available (see below).

eventSelectionType

Method to be used in matching events. Events at the support and forecast location can be paired either on the basis of common EventId’s, or on the basis of a selection on travel time, where events at the upstream and downstream location are paired if these are found to belong to the same hydrological event as defined using travel time criteria defined in the TraveTimesConfiguration. Enumeration of options includes;

  • eventid
  • traveltime
Comment

Optional comment for correlation configuration. Used for reference purposes only.

selectionCriteria

Selection criteria used in defining events to be used in establishing correlations.

Figure 83 Elements of the selectionCriteria configuration

period

Root element for defining selection of events based on dates.

startDate

Start date of time span to be used in selection (yyyy-mm-dd).

endDate

End date of time span to be used in selection (yyyy-mm-dd).

startTime

Optional start time of time span to be used in selection (hh:mm:ss)

endTime

Optional end time of time span to be used in selection (hh:mm:ss)

selectInsidePeriod

Boolean to indicate if events are to be selected that fall in the time span defined, or that fall outside the time span defined.

thresholds

Root element for defining selection of events that fall above or below a threshold. The threshold is applied to events selected at the forecast location.

thresholdLimit

Value of the threshold to use in selecting events.

selectAboveLimit

Boolean to indicate if events are to be selected that fall above the threshold if true. Events are selected below the threshold if false.

tags

Root element for defining selection of events on tags defined in the eventSets

tag

Definition of tag to use in event selection. Multiple tags may be defined.

include

Boolean to define if events with given tag are to be included or excluded in selection.

6.8.1            CorrelationEventSets

This configuration file is related to the Correlation module, and is used to define the events used in establishing a correlation. The configuration file is in the CorrelationEventSets (table or directory). Each configuration defined is referenced using a CorrelationEventSetsId as defined in the Regional Configuration.

 

Figure 84 Elements for defining correlationEventSets

comment

Optional comment for correlation event sets configuration. Used for reference purposes only.

correlationEventSet

Root element for defining set of events at a location to be used in establishing correlations.

locationId

LocationId for which events are defined

parameterId

ParameterId for which events are defined.

event

Definition of events at the specified location for the specified parameter.

 

Attributes;

  • eventId : String ID for the event (may be used in matching events)
  • value : value of the event
  • date : event data (yyyy-MM-dd).
  • time : event time (hh:mm:ss).
  • tag : optional tag for event (may be used in event selection)

 

 

 

6.8.2            TravelTimesSets

This configuration file is related to the Correlation module, and is used to define the travel time between locations. These travel times may be used in matching events. The configuration file is in the TravelTimesSets (table or directory). Each configuration defined is referenced using a TravelTimesSetsId as defined in the Regional Configuration.

 

Figure 85 Elements of the TraveTimeSets configuration

travelTime

Root element for defining a set of travel times. Multiple entries may exist.

downstreamLocation

LocationId of the downstream (forecast) location

 

Attributes;

  • id : Id of the location
  • name : name of the location (for reference purposes only)
upstreamLocation

LocationId of the upstream (support) location. Multiple entries may exist.

 

Attributes;

  • id : Id of the location
  • name : name of the location (for reference purposes only)
travelTime

Definition of the travel time between the location (average)

 

Attributes;

  • unit unit of time (enumeration of: second, minute, hour, day, week)
  • multiplier defines the number of units given above in a time step.
  • divider same function as the multiplier, but defines fraction of units in time step.
validPeriod

Window around travel time for determining validity of event to be matched.

 

  • unit unit of time (enumeration of: second, minute, hour, day, week)
  • start : start of validity period
  • end : start of validity period

 


6.9                Error correction module configuration

The error modelling module is a generic forecasting module. The module is used to improve the reliability of forecast by attempting to identify the structure of the error a forecasting module makes during the modelling phase where both the simulated and observed values are available, and then applying this structure to the forecast values. This is under the assumption that the structure of the error remains unchanged. In defining the error model three time series will need to be defined;

 

  • Merged input time series of simulated model output for the historical period and of forecasted model output for the forecast period. The time series in the historical period will be used for establishing error model through comparison with the observed time series. The error forecast will be applied to the time series in the forecast period.
  • Input time series for the observed data.
  • Output time series for the updated simulated data for the historical period and the updated forecast data for the forecast period.

 

Two methods of establishing an error model are available. The first uses an AR (Auto Regressive) model only, but allows the order of the model to be determined automatically. The second method uses an ARMA model, but the order of both the AR and the MA (Moving Average) model must be defined. In both cases various transformations may be applied to normalise the residuals prior to establishing the error model.

 

When available as configuration on the file system, the name of the XML file for configuring an instance of the error module called for example GreatCorby_ErrorModel_Forecast may be:

 

GreatCorby_ErrorModel_Forecast 1.00 default.xml

 

GreatCorby_ErrorModel_Forecast File name for the GreatCorby_ErrorModel_Forecast configuration.

1.00 Version number

default Flag to indicate the version is the default configuration (otherwise omitted).

 

Figure 86 Elements of the error module configuration.

errorModelSet

Root element for definition of an error model set.

inputVariable

Definition of input variable to be used in the error correction model. At least two entries are required in the error model, one for observed time series and one for simulated time series For each entry an input variable will need to be identified. The variableId is used to refer to the time series. See Transformation Module for definition of inputVariable configuration.

autoOrderMethod

Root element for defining an error model using the AR structure.

Figure 87 Elements of the autoOrderMethod configuration.

orderSelection

Boolean to indicate if order of AR components should be established automatically or if the given order should be used.

order_ar

Order of the AR model. If the orderSelection is true, then this value is the maximum order (may not exceed 50). In literature mostly an value of the AR order up to 3 is chosen, higher values are possible, but will have a smaller contribution to the overall result of the error correction. 

order_ma

Not used in this method.

parameters

In the near future this optional setting will be used to exactly specify the values for all the parameters (multipliers, powers, dividers, etc) used in the error correction model. 

subtractMean

Boolean to indicate if mean of residuals should be subtracted prior to establishing error model.

boxcoxTransformation

Boolean to indicate if the residuals should be transformed using Box-Cox transformation prior to establishing error model.

lambda

Lambda parameter to use in Box-Cox transformation (note: value of 0 means the transformation is a natural logarithm). Values ranging from 0 to 0.5 are often used.

ObservedTimeSeriesId

Input time series set to be defined as the observed data to compare simulated model output to.

SimulatedTimeSeriesId

Input time series set to be defined as the simulated model output for both the historic and the forecast period. Multiple series will be combined into single series. Series with higher index will be overlayed by series with lower index.

OutputTimeSeriesId

Updated timeseries data generated by the error model. This serie can contain data for the historic and the forecast period.

 

fixedOrderMethod

Root element for defining an error model using the ARMA structure.

Figure 88 Elements of the fixedOrderMethod configuration.

correctionModel

Structure of the error model to be used. The model selection includes the selection of initial transformations. Enumeration of options included;

  • none
  • ARMA + systematic
  • systematic
  • ARMA
  • ARMA + log transformation
  • ARMA + systematic + log transformation

 

order_ar

Order of the AR part of the model. In literature mostly an value of the AR order up to 3 is chosen, higher values are possible, but will have a smaller contribution to the overall result of the error correction.

order_ma

Order of the MA part of the model. The order you specify determines the length of the period effected by the moving average function. The higher the order, the longer the effected period. The moving average model is not operational yet. 

ObservedTimeSeriesId

Input time series set to be defined as the observed data to compare simulated model output to.

SimulatedTimeSeriesId

Input time series set to be defined as the simulated model output for both the historic and the forecast period. Multiple series will be combined into single series. Series with higher index will be overlayed by series with lower index.

OutputTimeSeriesId

Updated timeseries data generated by the error model. This serie can contain data for the historic and the forecast period.

interpolationOptions
  • Interpolation options for filling the missing values of the observed time series. This parameter is optional.

 

 

interpolationType

You can make a selection of a type of interpolation. Enumeration of available options is;

  • linear ; for linear interpolation between available values
  • block ; for block interpolation (note: the last available value is then used until a new value available).
  • default ; for replacing unreliable values with a default.
gapLength

Maximum allowed gap size that can be filled using interpolation.

defaultValue

Default value required for ‘defaultvalue’ interpolation option.

 

maxObserved

Maximum value to be used by the error module. Higher values will be converted to NaN and not used as input for error correction. This parameter is optional. 

minObserved

Minimum value to be used by the error module. Lower values will be converted to NaN and not used as input for error correction. This parameter is optional. 

maxResult

Maximum value to be generated by the error module. This setting can be used to specify an upper limit of the generated output timeseries. This parameter is optional.

minResult

Minimum value to be generated by the error module. This setting can be used to specify a lower limit of the generated output timeseries. This parameter is optional.

ignoreDoubtful

Should the error module ignore doubtful input values. This parameter is optional.

outputVariable

Definition of output variable as a result of the error model. A single timeSeriesSet for one location output variable error model is defined.