Versions Compared

Old Version 51

changes.mady.by.user Erik Pelgrim

Saved on

compared with

New Version 52

changes.mady.by.user Boyan Domhof

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


The DIMR-software is used to run a SOBEK 3 model in Delft-FEWS. In the General Adapter of Delft-FEWS we refer to the different components of the dimr-tool. This wiki will describe what you need to do to convert a SOBEK3 model to a dimr-model and how to set-up the different components of your Delft-FEWS configuration. Read the steps in this guide to implement your SOBEK 3 model in Delft-FEWS using the dimr-software. When updating an existing model, some steps are not necessary.

Table of Contents

How to set-up a DIMR Modules folder?

...

Follow the steps below to set-up a DIMR model folder in your Delft-FEWS system.  Step 1 can be skipped when updating an existing model.

Info
titleStep 1: create the dimr model folder structure in Delft-FEWS
Create a rootDir folder for your SOBEK 3 model at $REGION_HOME$/Modules/<put_your_rootDir_here>. All sub-folders described in the table above can be put in this Modules folder.

The contents of the Modules folder can now be filled. We will start with the download of getting the dimr_bin files.

Info
titleStep 2: download getting the contents of the dimr_bin

The DIMR is part of DeltaShell and its binaries can be downloaded from our build server.

  • The latest version of the DIMR can be found by clicking on this link. The binaries for the DIMR-tool can be found in the folder plugins/DeltaShell.Dimr/. You need to make sure that the run_dimr is modified so that it points to the libraries in cli. Now you can unzip all files and put them into the dimr_bin folder.

Image Removed Write down the build number of the dimr-tool you will need this in step 3.

The actual model files must be created by converting your SOBEK 3 model. DeltaShell contains standard functionality to aid you in this step. 

There are two ways for getting the dimr_bin files:

  • The preferred method is getting the dimr_bin files from the used SOBEK 3 installation in which the model is created. Typically, the files are found in c:\Program Files (x86)\Deltares\SOBEK (xxx)\plugins\DeltaShell.Dimr\kernels\x64, where xxx is the SOBEK version. Copy the x64 directory to dimr_bin folder.
  • Another method is to getting the dimr_bin files from the Deltares Build server which can be found using the following link. This is not preferred as these are the latest development releases of DIMR Which version you will need depends on your used SOBEK 3 model version. Which DIMR version belongs to which SOBEK 3 version is therefore currently not known. If you are sure which version to use, lick the artifacts button Image Added, select either the Windows or Linux zip file to download and unpack in the dimr_bin directory.

It is good practice to add a README.txt file in the dimr_bin directory where you document to DIMR version and from where you got the DIMR files.

The actual model files must be created by converting your SOBEK 3 model. DeltaShell contains standard functionality to aid you in this step. 

Info
titleStep 3: generate the dimr_model files
  • In DeltaShell open your SOBEK 3 model.

Image Added

Info
titleStep 3: generate the dimr_model files

Take a look at the build numer of the dimr-tool from step 3. Open an instance of DeltaShell with a build number that corresponds to the build number of the dimr_bin.

  • In DeltaShell open your SOBEK 3 model.

Image Removed It is possible that a message will appear in DeltaShell stating that your project was made in an older version and needs to be migrated to the latest version of DeltaShell. Press "Yes" and the model will automatically migrate to this version. If this is not preferred, because of new model developments or stability, use the DeltaShell which belongs to the SOBEK 3 version you used to create the model.

  • Before you continue it is recommended to run the model by pressing the button in the DeltaShell GUI. This is an easy way to check if all functionality is up-to-date. Check the log messages and fix all issues that are mentioned.

When your model can finish awithout displaying errors it is possible to convert the SOBEK 3 model to a dimr-model.

  • In DeltaShell right-click on your Integrated Model (e.g. Image Added) and select the option "Export...". In the menu that opens you now select the option "DIMR configuration" and you follow the instructions.
  • Put the all model files that are generated in the folder dimr_model.

...

Info
titleStep 4: download the contents of the fews-dimr-adapter-bin folder

The DIMR and Delft-FEWS are communicating through the fews adapter. This adapter is part of Delft-FEWS and must be downloaded from the build.deltares.nl:

  • http://build.deltares.nl/project.html?projectId=FewsDevelopment&tab=projectOverview
  • Be sure the get the adapter from the environment corresponding to your FEWS version (click on Image Added to view other version)
  • Select the latest stable version (or the Delft-FEWS development version): FEWS – Development > install adapters > artifacts Image Added> fews-dimr-adapter-bin.XXXXX.zip.
  • Put all these unzipped files in the folder fews-dimr-adapter-bin

The folders Input and Logs will be filled when you run the model.  

How to

...

run the DIMR model?

Info
titleColdStateFiles

Delft-FEWS will need a state file to be able to run the SOBEK model. In order to generate the first state files of the SOBEK model it is recommended to do a stand-alone run of the DIMR/SOBEK from the command line. 

  • Go to FEWS_YourSystem / Modules / NewFolder / dimr_model / dflow1d and open the *.md1d file.
  • Go to option WriteRestart and make sure to set this setting on true. The model will now generate its state files. 

Image Removed

  • The SOBEK 3 model has default timeseries. We can thus do a stand-alone run of the model without providing the model with data.
  • Open your command line window from the dimr_model folder. This is your working directory. Run the run_dimr.bat and give the dimr_config file as an argument. This file was created in step 1B. In the example below this is the ovd_dv.xml file.
  • Run the model.

Image Removed or in newer versions of the software Image Removed

  • After the model has run succesfully it is possible to create a state-file for Delft-FEWS.
  • In the Config\ColdStateFiles\ create a zip for the cold state files. In our example this is the Walrus_Hydr_Sobek3_Update Default.zip
  • The zip file for the ColdState files contains the following information
    • Walrus_Hydr_Sobek3_Update Default \ dflow1d \ sobek.rda
    • Walrus_Hydr_Sobek3_Update Default \ dflow1d \ sobek.rdf
    • Walrus_Hydr_Sobek3_Update Default \ rtc \ state_import.xml
  • When you analyse the model results you will find a sobek.nda and sobek.ndf file in the dimr_model \ dflow1d folder and a state_export.xml in the dimr_model \ rtc folder. You can use these files. Make sure to rename the extension of these files. 

 

What information should I put in the ModuleDataSetFiles?

Info
titleModuleDataSetFiles

Delft-FEWS can distribute model files to FSS machines. It is recommended to zip the model files and put it in the Delft-FEWS configuration.

  • In the Config\ModuleDataSetFiles create a zip file for the (update) model run. In our example the name for this zip file is Walrus_Hydr_Sobek3_Update.zip
  • The zip file for the ModuleDataSetFile contains the following information:
    • .\dlfow1d\.*.
    • .\rtc\.*.
    • ovd_dv.xml
  • As you might have notice this is the content of the dimr_model file created previously. 

How do I get hold of the import and export timeseries of the SOBEK 3 / DIMR model?

Not yet completed

How to create the General Adapter Module?

An example of the general section of the general adapter can be found below.

Info
titleGeneral section of Module

Image Removed

In the example configuration above it can be seen that the working directory of the DIMR should always be the folder ./dimr_model. The Delft-FEWS exports timeseries to the folder .\Input. After the modelrun is completed the output of the model is expected in the folder .\dflow1d/output.

...

Run DIMR model

When the contents of every directory is set-up, we can run the DIMR model. The model can be executed using a batch file with an argument pointing to the DIMR configuration xml file.

  • To run the model, execute the following batch file: .\dimr_bin\x64\dimr\scripts\run_dimr.bat DIMR_CONFIG.xml (e.g. Rhine_LobRTK.xml or IJG.xml)

You could enter this command in a seperate batch file for easier starting the model run.

How to get the desired output from the DIMR model?

Info
titleRun DIMR model

In SOBEK 3 you can specify for which parameters you want to write the output. This is the same in the DIMR model and can be specified in the *.md1d file located in .\dimr_model\dflow1d.

Default, all results options are set to None. Apply the same output options as used in the SOBEK 3 model (e.g. Current, Average, ...)

How to generate cache files to read the SOBEK 3 DIMR model faster?

Info
titleSOBEK 3 DIMR cache files

DIMR has the option to make cache files of the SOBEK 3 model to read the model itself faster. This can be up to 100 times faster, because it does not have to processing large ASCII files. Therefore, it is good practice to generate the cache files.

To make the cache files, do the following:

  • Go to FEWS_YourSystem / Modules / NewFolder / dimr_model / dflow1d and open the *.md1d file.
  • In the file, enter under \[AdvancedOption] 'CacheMode = Write' and run the DIMR model

Image Added

  • In the dflow1d folder the .cache files are made.
  • Lastly, set CacheMode = Write to CacheMode = Read to ensure the DIMR does not generate new cache files as this is the same as not using cache files.

Image Added

How to generate ColdStateFiles?

Info
titleColdStateFiles

Delft-FEWS will need a state file to be able to run the SOBEK model. In order to generate the first state files of the SOBEK model it is recommended to do a stand-alone run of the DIMR from the command line (see How to run the DIMR model? how to do that). 

  • Go to FEWS_YourSystem / Modules / NewFolder / dimr_model / dflow1d and open the *.md1d file.
  • Set the correct restart options:
    • Older version:
      • Go to option WriteRestart and make sure to set this setting on true. The model will now generate its state files. 

Image Added

    • Newer versions:
      • Under \[Restart] there are 5 options available. These reflect the restart options in SOBEK 3.
      • To make a restart file, set UseRestart = 0 and WriteRestart = 1. The start and stop time should be set very wide like in the example below to ensure a restart will always be written. Intermediate restartfiles can be written using the RestartTimeStep option. Note: on the end of the simulation run, a restart will always be written if WriteRestart is set to 1 independent of the RestartTimeStep.

Image Added

  • The SOBEK 3 model has default timeseries. We can thus do a stand-alone run of the model without providing the model with data.
  • Run the DIMR model.

Image Added or in newer versions of the software Image Added

  • After the model has run succesfully it is possible to create a ColdState-file for Delft-FEWS.
  • In the Config\ColdStateFiles\ create a zip for the cold state files. In our example this is the Walrus_Hydr_Sobek3_Update Default.zip
  • The zip file for the ColdState files contains the following information
    • Walrus_Hydr_Sobek3_Update Default \ dflow1d \ sobek.rda
    • Walrus_Hydr_Sobek3_Update Default \ dflow1d \ sobek.rdf
    • Walrus_Hydr_Sobek3_Update Default \ rtc \ state_import.xml
  • When you analyse the model results you will find a sobek.nda and sobek.ndf file in the dimr_model \ dflow1d folder and a state_export.xml in the dimr_model \ rtc folder. You can use these files. Make sure to rename the extension of these files.
  • Set UseRestart to 1 in the *.md1d file to ensure the model will be using the restart.

What information should I put in the ModuleDataSetFiles?

Info
titleModuleDataSetFiles

Delft-FEWS can distribute model files to FSS machines. It is recommended to zip the model files and put it in the Delft-FEWS configuration.

  • In the Config\ModuleDataSetFiles create a zip file for the (update) model run. In our example the name for this zip file is Walrus_Hydr_Sobek3_Update.zip
  • The zip file for the ModuleDataSetFile contains the following information:
    • .\dlfow1d\.*.
    • .\rtc\.*.
    • DIMR_CONFIG_NAME.xml (e.g. Rhine_LobRTK.xml or IJG.xml)
    • Be sure to exclude .log, .rda, .rdf, .nda, .ndf, .\dlfow1d\ouput dir, state_import.xml and state_export.xml
  • As you might have notice this is the content of the dimr_model file created previously. 


How do I get hold of the import and export timeseries of the SOBEK 3 / DIMR model?

Not yet completed

How to create the General Adapter Module?

For more detailed information on the use of the General Adapter Module, see this link.

An example of the general section of the general adapter can be found below.

Info
titleGeneral section of Module

Image Added

In the example configuration above it can be seen that the working directory of the DIMR should always be the folder ./dimr_model. The Delft-FEWS exports timeseries to the folder .\Input. After the modelrun is completed the output of the model is expected in the folder .\dflow1d/output.


The  activities section is divided into four sub-sections: startUp-, export-, execute- and importActivities. The contents of all of these sections will be discussed in the info sections below. When you want to test your configuration during the building process then it is recommended to start with the startUpActivity, ExportActivities and ExecuteActivity sections. When no information (e.g. timeseries, states) are provided to the model DIMR will take its own default timeseries and states to perform a model run. Hence, this trick enables you to test if the model can be run from Delft-FEWS. The importDir can be checked to see if output has been generated.

Similarly, it is also possible to define a single input timeseries in the exportActivities. The DIMR will add default values to the missing timeseries in order to start the run. This trick allows you to check early on whether your ExportActivity and IdMapping is working properly. 

Info
titlestartUpActivities

 The startUpActivities are typically used to purge the model directory in the Modules folder to ensure a clean slate. See example below

Image Added


Info
titleexportActivities

The exportActivities are used to export all the necessary data and the model itself. These typically consists of (in order): exportStateActivity, exportTimeSeriesActivities, exportDataSetActivity and exportRunFileActivity. See examples below.

exportStateActivity

This activity exports the state of the model. This can be a cold state (the one generated under How to generate ColdStateFiles?) or a warm state saved in the Delft-FEWS database.

Image Added

exportTimeSeriesActivity

This activity exports all the necessary data from FEWS to be used in the model. There can be multiple activities.

Image Added

exportDataSetActivity

This activity exports the ModuleDataSet file to the working directory.

Image Added

exportRunFileActivity

This activity makes a run xml file which the pre- and post adapter uses to distribute the timeseries to the correct locations for the model to pick up.

Image Added


Info
titleexecuteActivities

Image Added

Similarly, it is also possible to define a single input timeseries in the exportActivities. The DIMR will add default values to the missing timeseries in order to start the run. This trick allows you to check early on whether your ExportActivity and IdMapping is working properly. 

Info
titlestartUpActivities

 Not yet completed

Info
titleexportActivities

 Not yet completed

Info
titleexecuteActivities

 Not yet completed


Info
titleimportActivities
In the ImportActivities the state files and the timeseries can be imported to Delft-FEWS.
It is important to convert the extension of the state files before you import it to Delft-FEWS, this can be done by adding the </relativeExportFile> to your configuration.
Image Modified
Image Added

Improved DIMR adapter

...