Introduction

This page describes how you can use OpenDA in combination with Delft-FEWS. OpenDA is an open interface standard for (and free implementation of) a set of tools to quickly implement data assimilation and calibration for arbitrary numerical models. If you want to make a workflow in Delft-FEWS to either use data assimilation or calibrate a model automatically, then you can use OpenDA for this. In order to use OpenDA in combination with Delft-FEWS, knowledge of both Delft-FEWS and OpenDA is required. For more information about OpenDA see the OpenDA website.

Setting-up the link between Delft-FEWS and OpenDA

The adapter between Delft-FEWS and OpenDA is called OpenDaInFews. OpenDaInFews is a separate Java program. The OpenDaInFews binaries consist of the following files:

• fews_openda.jar
• run_openda_in_fews.bat

OpenDaInFews depends on OpenDA and on the following additional files:

• log4j-1.2.14.jar
• Delft_PI_castor.jar
• Delft_PI.jar
• Delft_Util.jar
• csf.jar
• csf.dll

OpenDaInFews also needs the OpenDA binaries. In order to use OpenDaInFews all OpenDaInFews binary files and dependencies must be present in the OpenDA bin folder.

When running OpenDaInFews it may first do some preprocessing, e.g. converting data to a different format. After the preprocessing step, it will call OpenDA. At this point OpenDA will do a filtering or calibration run, depending on the configuration. After OpenDA has finished, some final postprocessing may take place before OpenDaInFews returns control to Delft-FEWS.

Currently OpenDaInFews can be used in three ways:

• run an algorithm in OpenDA ("algorithm run"). For this a full OpenDA configuration is needed (including an .oda file). In this context an algorithm means either a filtering algorithm for data assimilation (e.g. ensemble Kalman filter) or an optimization algorithm for calibration (e.g. simplex algorithm).
• run a model using the OpenDA black box stoch model wrapper ("single stoch model run"). For this only an OpenDA black box stoch model configuration is needed.
• run a model using the OpenDA black box model wrapper ("single model run"). For this only an OpenDA black box model configuration is needed.

Runtime arguments for OpenDaInFews:

The recommended way to run OpenDaInFews is to use a batch script to setup the Java classpath and invoke the OpenDaInFews main method. Such a batch script can be started from a command prompt or directly from a Delft-FEWS workflow. It is also possible to invoke the OpenDaInFews main method directly from another Java program. However, it is highly recommended to use the batch script approach, in order to avoid problems with loading of dependencies like e.g. jar and dll files from models. While it is possible to make such a batch script yourself, it is also possible to use the batch script called run_openda_in_fews.bat. This batch script is included in the OpenDaInFews binaries since 2012-11-15 (revision 3640), and will be part of future OpenDaInFews releases. This batch script passes its runtime arguments to OpenDaInFews, so the batch script needs the same runtime arguments that are listed in the table above.

Configuration

If you want to make a workflow in Delft-FEWS that uses OpenDA, then you need to configure the Delft-FEWS general adapter so that it runs OpenDaInFews and you also need to supply an OpenDA configuration to run. To do this, please use the following approach:

1. First create the OpenDA configuration. For documentation about OpenDA configuration files see http://www.openda.org/docu/openda_2.0/doc/xmlSchemasHTML/index.html. For more information about OpenDA and examples of OpenDA configuration files see http://www.openda.org/joomla/index.php?option=com_content&view=article&id=66&Itemid=82
2. Test your OpenDA configuration by running it with only OpenDA (without Delft-FEWS). For this you will probably also need to supply a configured model and some input data for the run. Only if that works, then you are ready for the next step.
3. Put your OpenDA configuration, including the configured model and the OpenDA binaries, in a place that Delft-FEWS can access. Usually this is somewhere in the ModuleDataSetFiles. Please note that the OpenDA binaries folder should also include the OpenDaInFews binaries and the run_openda_in_fews.bat file, which are not included in a standard OpenDA release.
4. Finally configure an executeActivity element in a Delft-FEWS general adapter configuration file that runs OpenDaInFews. Before this you also need to configure an exportRunFileActivity that exports the pi run info file that contains information that is needed to run OpenDaInFews. Note that if Delft-FEWS is expected to re-use the same OpenDA configuration for multiple runs, then you may also need to configure some purge activities to remove certain output files between runs.

Below are two different examples of configuration that can be added to a Delft-FEWS general adapter file to run OpenDaInFews.

Using a batch script (recommended):

<exportActivities>
	<exportRunFileActivity>
		<exportFile>%REGION_HOME%/Models/OpenDaConfig/run_info.xml</exportFile>
	</exportRunFileActivity>
</exportActivities>
<executeActivities>
	<executeActivity>
		<description>OpenDa in Fews adapter</description>
		<command>
			<executable>%REGION_HOME%/Models/OpenDA/bin/run_openda_in_fews.bat</executable>
		</command>
		<arguments>
			<argument>-f</argument>
			<argument>%REGION_HOME%/Models/OpenDaConfig/run_info.xml</argument>
			<argument>-a</argument>
			<argument>%REGION_HOME%/Models/OpenDaConfig/enkf_run.oda</argument>
		</arguments>
		<timeOut>100000000</timeOut>
		<overrulingDiagnosticFile>%REGION_HOME%/Models/OpenDaConfig/diagnostics/diagnostics.xml</overrulingDiagnosticFile>
	</executeActivity>
</executeActivities>

Calling OpenDaInFews directly from Delft-FEWS:

<exportActivities>
	<exportRunFileActivity>
		<exportFile>%REGION_HOME%/Models/OpenDaConfig/run_info.xml</exportFile>
	</exportRunFileActivity>
</exportActivities>
<executeActivities>
<executeActivity>
	<description>OpenDa in Fews adapter</description>
		<command>
			<className>nl.deltares.openda.fews.OpenDaInFews</className>
			<binDir>%REGION_HOME%/Models/OpenDA/bin/</binDir>
		</command>
		<arguments>
			<argument>-f</argument>
			<argument>%REGION_HOME%/Models/OpenDaConfig/run_info.xml</argument>
			<argument>-a</argument>
			<argument>%REGION_HOME%/Models/OpenDaConfig/enkf_run.oda</argument>
		</arguments>
		<timeOut>100000000</timeOut>
		<overrulingDiagnosticFile>%REGION_HOME%/Models/OpenDaConfig/diagnostics/diagnostics.xml</overrulingDiagnosticFile>
	</executeActivity>
</executeActivities>

Details of a run with OpenDA from Delft-FEWS using OpenDaInFews

Following is a short description of the steps that are usually involved in a workflow that runs OpenDA from Delft-FEWS using OpenDaInFews:

1. Delft-FEWS exports data that is input for the OpenDA run. This can be e.g. forcings for the model or observations to use for data assimilation. This step can be done either by the general adapter or by a separate timeSeriesExport run.
2. Delft-FEWS exports a state that is valid for the start of the run period.
3. Delft-FEWS exports a pi run info file that contains information that is needed to run OpenDaInFews. This contains a.o. the run period, a reference to the state file and a reference to the diagnostics file.
4. OpenDaInFews converts input data from Delft-FEWS to a format that the model can read. This only happens for a single (stoch) model run. For an algorithm run the data conversion should be done within OpenDA.
5. OpenDaInFews starts OpenDA using the run period from the pi run info file and the state exported from Delft-FEWS.
6. OpenDA runs using the supplied OpenDA configuration.
7. OpenDaInFews writes the results log messages produced by OpenDA in the diagnostics file.
8. OpenDaInFews converts output data from the model to a format that Delft-FEWS can import. This only happens for a single (stoch) model run. For an algorithm run the data conversion should be done within OpenDA.
9. Delft-FEWS transfers the log messages from the diagnostics file to its own log file/display.
10. Delft-FEWS imports the state that OpenDA created at the end of the run.
11. Delft-FEWS imports data that is output from the OpenDA run. This step can be done either by the general adapter or by a separate timeSeriesImport run.