You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 10 Next »

Issues related to the development of the Kisters Hydstra adapter (Deltares internal access only):

https://issuetracker.deltares.nl/browse/FEWS-16402

https://issuetracker.deltares.nl/browse/FEWS-20429

Contents

Introduction

The HYDSTRA modelling suite provides a flexible modelling framework for running operational hydrological forecasting models. Originally developed in Australia, the modelling suite has since been acquired and integrated into the WISKI system by Kisters AG (http://kisters.com.au/hydstra-modules.html). Due to support of the modelling suite being discontinued as a part of WISKI, the modelling suite can also be run from as a stand alone executable. The adapter that is described here is used to run the stand alone HYDSTRA modelling engine from the Delft FEWS system. This documentation focuses on the configuration and set-up of the adapter to provide the required inputs to run the model engine, to provide the required run parameters, and to read model results. 

Applications

The HYDSTRA modelling engine that this adapter describes is implemented in the following Delft FEWS configurations: 

  • Alberta River Forecasting System (ARFS), used by Alberta Environment and Parks (AEP), Edmonton, Canada (Deltares contacts: Dave Casson & Micha Werner). Note that AEP owns the IPR of the HYDSTRA Model engine code.

Download

Download of the model adapter is not available here: please e-mail to Delft-FEWS Product Management for more information.

Interface between FEWS and Hydstra

The adapter described here provides the following tasks in running Hydstra model engine from FEWS. The Hydstra adapter is developed in Java and is configured to run from the Delft FEWS configuration through a suitably configured General Adapter run. The tasks supported by the adapter include:

  • Reading of the input time series required for the Hydstra model run (e.g. precipitation, temperatur, discharge) as provided through from the general adapter through the FEWS Published Interface XML format (PI-XML), and transforming these to the input format required by the Hydsra model engine (Hysdstra Time Series Format, TSF).
  • Amending the Hydstra model definition file (TSO file) with the appropriate run-time parameters (e.g. model start time, run length etc). These data are provided from the FEWS General Adapter through the PI-XML formatted run file.
  • Amending the value of model parameters and states within the TSO file to allow influencing the behaviour of the Hydstra model. These parameters can be set either using the PI-XML parameters file exported from the FEWS General adapter (typically used for setting the value of model parameters, or through an PI-XML time series (typically used for setting the value of a model state at the start of the run).
  • Reading the output time series provided by the Hydstra model engine. These outputs are provided by the model engine using two formats; TSF formatted files and free format text files (format specified in the TSO file).

Hydstra model definition file

Each of the basin models that can be run through the Hydstra adapter is defined in the Hydstra Model Definition File (Model TSO file). This defines all model inputs and outputs, model structure and model parameters. The format of the file is JSON, which is loaded into the Hydstra adapter as a JSON object. While the Hydstra adapter does not interpret the full file, there are specific sections that are important. The model structure is made up of a series of nodes and links,, where each node is a modelling element of a defined type (this refers to a specific modelling algorithm such as the Synthetic Streamflow and Reservoir Routing - SSARR, or Unit Hydrograph - UH, algorithms). Links are important in this context to identify the inputs to the model as external time series are connected to the algorithms defined in the nodes though these links.

Identifying input time series to be provided from FEWS

When setting up the Hydstra model to run from FEWS, it is important to identify all expected input time series, where each time series is read from a TSF file. These are identified through the key word Source=TimeSeries. An example is provided below:

Source=TimeSeries(59.233,28.893,Text(C:\fews_systems\FEWS-Alberta\FEWS_ARFS\FEWS_ARFS\Modules\hydstra\Work\05DB006_QR.tsf),Agg)

To run the model from FEWS, the configuration must be set up that each named TSF file is exported. Note that the path to which the file is exported need not be set as the adapter when exporting the TSF file to be used by the model will set the path in the TSO file to ensure the file is found.  Note that the adapter will check that all names TimeSeries are available as a TSF file prior to the model being run. If there is a file missing then an error will be generated. Additionally the adapter also checks that all time series exported are linked to a TSF ile defined in the TSO file (with the exception of time series that are explicitly flagged as being time series pertaining to model states).

Identifying parameters and initial state values that can be changed

The Hydstra adapter allows the values of model parameters and initial state values to be changed by the user. These are changed within the node element of the Hydstra model definition file These values can be changed either through providing the value to be defined using a PI-XML parameter file, or through using a PI-XML time series of the values. To set this up, the nodes for which parameters are to be changed need to be identified, as well as the names of the parameter and/or initial state variables.  The snippet of a TSO file (in this case for a UH model) below shows an example of a node element definition.  To be able to change values in this element from FEWS, is it important to record the node name (in this example: "Uh_05DB006") as that is used by the adapter. Any of the parameters/initial state values that are defined in the Symbols section can be amended. To do so the new value must be provided in the PI-XML parameters file or in the PI-XML time series file, where the name of the symbol is used as an identier (e.g. "Baseflow"). It is not required to configure all symbols, as those not provided will not be amended. Note also that if a NaN is provided as a value, or the parameter/initial state value is missing in a time series, then the original value will not be amended.


Node
    {
    Name="Uh_05DB006"
    Description=
        {
        Dimensionless unit hydrograph node.
        }
    ObjectColour=RGB(0,255,0)
    Location=39.032000,46.747000
    Type="DimHgraphNode"
    Symbols=
        {
        float Theta = 0.2
        const int TimeToPeak = 45
        float Area = 2253
        float Baseflow = 13.0
        int FileOutput = 0
        String NodeName = "Uh_05DB006"
        String PrecipSourceName = ""
        float Outflow = 0
        int Outflow.Quality = 255
    }


Identifying output time series available for import to FEWS

After the model run has completed the adapter will read configured outputs provided by the Hydstra model engine.  Outputs can be provided by the engine to be read by the adapter through to formats. The first is through TSF formatted files, and the second is a free-format text file. The format of the latter is defined in the TSO file.

To identify those outputs provided as a TSF file, the sections in the TSO file with the keywords "Output=TimeSeries" must be identified. e.g.

Output=TimeSeries(47.637,37.956,Text(..\OutputFiles\Forecast_05CE020_QR.tsf),Qual(Mode(Overwrite)),Q1,T2)

Note that the name of the TSF file is later used to identify the location and parameter of the time series as the TSF format has no meta data internal to the file format.

The free-formatted text files are defined in the print sections in the TSO file. The example below shows a snippet. Note that great care must be taken if the format used here is amended as this may cause the adapter to be unable to read model outputs. The adapter will use the header names to identify the time series location (e.g. NodeName - which is a variable) as well as the parameter (e.g. "Outflow" and "Outflow.Quality"). 


hFile1=OPEN(PLOutFileName, Create, Exclusive)
Print(hFile1, "Node Type";comma;"DimGraph";comma;"NodeName";comma; NodeName;comma;"Run Time";comma;RunTime)
Print(hFile1, "Date";comma;"Time";comma;"Precip";comma;"Precip.Quality";comma;"Outflow";comma;"Outflow.Quality")
...
Print(hFile1,MonthString;"/";DateString;"/";YearNum;comma; HourString;":00";comma; Precip;comma;Precip.Quality;comma;Outflow;comma;Outflow.Quality)


Directory Structure

The Hydstra adapter and model use a specified directory structure to run from. While this can be configured differently the structure below is recommended. Note that this structure also reflects the relevant sections of the FEWS configuration. Descriptions of elements where relevant are provided in angle brackets. File names where provided below are given for the "Elbow" model (replace this with appropriate model name).


+---bin  <FEWS binaries>
|       
\---FEWS_sa  <FEWS REGION Home directory>

    |
    +---Config
    |   +---ColdStateFiles
    |   |       Elbow_Hydstra_Calib_Forecast default.zip  Default.zip           <cold state files (placeholder)>
    |   |
    |   +---IdMapFiles
    |   |       IdExportHydstra_Elbow.xml		<mapping of identifies for model input series exported from FEWS>
    |   |       IdImportHydstra_Elbow.xml		<mapping of identifies for model output series imported to FEWS>
    |   |
    |   +---ModuleConfigFiles
    |   |        Elbow_Hydstra_Calib_Template.xml      <main configuration file of the general adapter - note this may be a template)
    |	|	     Elbow_Hydstra_Forecast_Template.xml      <main configuration file of the general adapter - note this may be a template)
    |   |
    |   \---ModuleDataSetFiles
    |            Elbow_Hydstra_Historic.zip           <zipped TSO template (original)>
    |   |
    |   \---ModuleParFiles
    |            ParameterConfig_HydstraModels.xml.........<(optional) Hydsrta(template) parameter files>
    |
    \---Modules
        \---hydstra
            +---bin.................................... directory containing the Hydstra Adapter Java Class files 
            |
  	        +---HydstraBin..............................directory containing Hydstra executable and model libraries
            |

  	        +---InputFilesXML...........................directory to which model input time series are exported (exportDir in General Adapter Config)
            |
  	        +---OutputFiles.............................directory to which Hydstra TSF and TXT files are written
            |
  	        +---OutputFilesXML..........................directory from which output time series are read (importDir in General Adapter Config)
            |
  	        +---Template................................directory where (original) template TSO files are stored (exported from zipped modelDataSet)
            |
  	        +---Work....................................Working/Current directory during model execution








  • No labels

Disclaimer