1. Introduction
Although it may appear a huge challenge to turn a model engine into an OpenMI-compliant linkable component, it may not be as difficult as it seems. The OpenMI Software Development Kit provides a large number of software utilities that make migration easier. These utilities can be used by anyone migrating a model but are not required in order to comply with the OpenMI standard. The utilities can be used as a whole or you can select only a few of them; alternatively, you can use the utilities as the basis for your own implementations. This article assumes that you will use the OpenMI SDK to the full extent. Step-by-step instructions are given for the whole migration process, from defining the requirements for an OpenMI omponent, through design and implementation to testing. This section describes the requirements for OpenMI-compliance and introduces the Simple River model, which is used to illustrate the migration process.
1.1. OpenMI compliance
The official requirements for OpenMI compliance are:
...
However, when you use the software development kit provided by the OpenMI Association Technical Committee (OpenMI.OATC.Sdk) most of the requirements for compliance will automatically be taken care of.
1.2. The simple river example
A Simple River model engine was developed as an example of model migration. The model engine is programmed in Fortran and is a very simple conceptual river model. The Simple River consists of nodes and branches, as shown in Figure 1. For each timestep, the inflow to each node is obtained from a boundary-input file. These flow rates are multiplied by the timestep length and added to the storage in each node. Then, starting from the upstream end, the water is moved to downstream nodes and the flow rate in each branch is calculated.
...
Fig 2. Simple River input and output files
2. Planning the migration
Before you start migrating a model it is important that you have a precise idea about how your model is intended to be used when it is running as an OpenMI component. Think about any situation where it will be useful to run your model linked to other OpenMI components. Such components could be other models, data providers, optimization tools or calibration tools. You may even find it useful to run two instances of your model component in the same configuration.
This chapter suggests ways in which you can plan the migration of a model, including the development of use cases and the definition of exchange items.
2.1 Use cases
Use cases (examples of how software is used) have become very popular in software development. There are no formal requirements for defining a use case. However, what makes a use case different from an example is that a use case is more detailed and well defined. Most importantly, a use case must be formulated in such a way that, after completion of software development, you can unambiguously determine whether the use case is covered or not. The big advantage of use cases is that they are easily understood both by the software developer and the software user. At the beginning of the development process, a number of use cases should be defined. It is important that the repository of use cases at any time, in all areas of the software development, reflects the current target. If a particular use case cannot be fulfilled it should be modified or removed. Two use cases for the migrated Simple River model are given below. The use cases give a step-by-step description of how a user will use the models.
2.1.1 Use case 1: Connecting to other rivers
In the first use case, the Simple River model is connected to another OpenMI-compatible river
model (Figure 3).
...
- 5. The model user creates a unidirectional and ID-based link from the downstream branch in the other river model to an internal node in the Simple River model.
- 6. The model user selects input and output exchange items for the link (input quantity for the Simple River is 'Inflow').
- 7. The model user defines the simulation period.
- 8. The model user runs the simulation.
2.1.2 Use case 2: Inflow from geo-referenced catchment database
In the second use case, the inflow for the Simple River model comes from an OpenMIcompliant
runoff database (Figure 4).
...
Note that the runoff for a particular polygon is distributed on the river branches depending on how large a portion of a branch is included in each polygon. This type of boundary condition, where water is added to branches, was not possible in the original Simple River engine. The Simple River engine is (as a result of the migration) extended with this feature, simply because such a boundary condition becomes a possibility when running in combination the OpenMI.
2.2 Defining exchange items
Exchange items are combined information about what can be exchanged and where the exchanged item applies. An input exchange item could define that inflow can be accepted on nodes or river branches. An output exchange item could specify that flow can be provided on branches. The Quantity ID identifies what can be exchanged (e.g. 'Flow') and the ElementSet ID identifies where this quantity applies (e.g. 'Node:1').
The next step is to define input and output exchange items. The exchange items that are required in order to run the use cases are listed in Table 1.
...
Naturally, the exchange items should not be limited to a particular network, but for the purpose of planning the migration it is easier to start out with a specific case and then generalize this case when it comes to the more detailed design.
3. Wrapping
The OpenMI standard was designed to allow easy migration of existing model engines. The
standard is implemented in C# running under the .NET framework. Almost all existing model
engines are implemented in other programming languages, such as Fortran, Pascal, C and
C++. In order to bridge the gap between the different technologies and to minimize the
amount of changes needed to be made to the engine core a wrapping pattern will be the most
attractive choice in most cases.
This chapter describes the process of wrapping and the generic wrapper that is provided by
the OpenMI Software Development Kit.
3.1. A general wrapping pattern
Wrapping basically means that you create a C# class that implements the
ILinkableComponent interface. This wrapper will communicate internally with your engine
core. The wrapper will appear to the users as a 'black box', which means that all
communication will take place through the ILinkebleComponent interface (Figure 5).
Fig. 5 OpenMI wrapping pattern
One further advantage of using the wrapping pattern is that you can keep the OpenMIspecific implementations separated from your engine core. Typically, the engines will also be used as standalone applications where OpenMI is not used and it is naturally an advantage to be able to use the same engine in different contexts. This means that even in situations where new engines are built the wrapping pattern may still be the best choice.
3.2 The LinkableEngine
Model engines that are doing timestep-based computations have many things in common. It is therefore possible to develop a generic wrapper that can be used for these engines. This wrapper is called LinkableEngine and is located in the org.OpenMI.Utilities.Wrapper package. Basically, the LinkableEngine provides a default implementation of the ILinkableComponent interface. Naturally, the LinkableEngine cannot know the specific behaviour of your model engine; this information is obtained though the IEngine interface.
The recommended design pattern for model engine migration when using the LinkableEngine is shown in Figure 6. The design includes the following classes:
- The MyEngineDLL class is the compiled core engine code (e.g. Fortran).
- The MyEngineDLLAccess class is responsible for translating the Win32Api from MyEngineDLL to .NET (C#).
- Calling conventions and exception handling are different for .NET and Fortran. The MyEngineDotNetAccess class ensures that these operations follow the .NET conventions.
- The MyEngineWrapper class implements the IEngineAccess interface, which means that it can be accessed by the LinkableEngine class.
- The MyLinkableEngine class is responsible for the creation of the MyEngineWrapper class and for assigning a reference to this class to a protected field variable in the LinkableEngine class, thus enabling this class to access the MyEngineWrapper class.
More details of these classes are provided in the following sections.
The OpenMI standard puts a lot of responsibilities on the LinkableComponents. The main idea is that when the GetValues method is invoked the providing component must be able to deliver the requested values so that these apply to the requested time and the requested location. To be able to do this the LinkableComponent may have to interpolate, extrapolate or aggregate both in time and space. These and other things are handled by the LinkableEngine.
The LinkableEngine class includes the following features:
- Buffering: When a model is running as an OpenMI component it may be queried for values that correspond to a time that is before the current time of the model. Most models will only keep values for the current timestep and the previous timestep in memory. It is therefore necessary to store data associated with the OpenMI links in a buffer. The LinkableEngine handles the buffering for you.
- Temporal interpolation and extrapolation: Most models are only capable of delivering results at times that correspond to their internal timesteps. The LinkableEngine class handles all the temporal operations that are required for LinkableComponents.
- Spatial operations: The LinkableEngine provides a range of spatial data operations.
- Link book-keeping: The LinkableEngine handles book-keeping for links added to your component.
- Event handling: The LinkableEngine sends events that enable an event-listener to monitor the progress of the linked system when running.
More details about how the LinkableEngine works is given in OATC.OpenMI.SDK technical documentation.
4. Migration - step by step
The best strategy when migrating a model is to split the process into a number of steps; at the end of each step you can compile your code and run a small test.
The steps needed for migration are described in this chapter.
4.1 Step 1: Changing your engine core
The aim of the migration is to develop a class that implements the IEngine interface. As shown in Figure 6, the class that implements the IEngine interface is supported by other classes and the engine DLL.
Fig 6. Wrapping classes and engine core DLL
Model engines are typically compiled into an executable file (EXE). Such executable files are not accessible by other components and as such are not very suitable as a basis for OpenMI components. It is therefore necessary for your engine to be compiled into a dynamic link library file (DLL).
Ideally you should make modifications to your engines so that the same engine can be used both when running as an OpenMI component and when running as a standalone application. Having two versions of the same engine leads to unnecessary maintenance work. Therefore you could make a new application (EXE) that calls a function in the engine core DLL which, in turn, makes your engine perform a full simulation.
Figure 7 illustrates the software required to run an engine as a standalone application. The SimpleRiverApplication.EXE file is never used when running in an OpenMI setting.
Fig. 7. Running an engine as a standalone application
The following steps are required in the conversion of the engine core:
...
When your engine is running in the OpenMI Software Development Kit it must be able to initialize, perform single timesteps, finalize and be disposed as separate operations. This means that your engine core may need to be reorganized. You can do this in any way you like but one logical approach is to create four functions:
logical function Initialize()
(Open files and populate your engine with initial data)
logical function PerformTimeStep()
(Perform a single timestep)
logical function Finish()
(Close files)
logical function Dispose()
(De-allocate memory)
The RunSimulation function should now be changed so that it calls the Initialize function, then repeatedly calls the PerformTimeStep function until the simulation has completed, and finally
calls the Finish and Dispose functions.
At this point you should run your application again and check that the engine is still producing the correct results.
You have now completed the restructuring of the engine. The remaining changes that you need to make to the engine will be much smaller. The nature of the changes will be dependent on the particular engine. For now, you can move on to creating the wrapper code.
4.2 Step 2: Creating the .Net assemblies
The next step is to create the wrapper classes (Figure 8). For this stage, make sure that the OpenMI Software Development Kit is installed on your PC.
Fig 8. C# wrapping classes
...
After creating the assemblies and the classes, you can start working on the first class,
MyEngineDLLAccess. Details are given in the next section.
4.3 Step 3: Accessing the functions in th engine core
The third step is to implement the MyEngineDLLAccess class (Figure 9).
Fig 9. MyEngineDllAccess class
Because you are using a C# implementation of OpenMI, your engine needs to be accessible from .NET. In the pattern shown above this is handled in two wrappers, MyEngineDLLAccess and MyEngineDotNetAccess. The MyEngineDLLAccess class will make a one-to-one conversion of all exported functions in the engine core code to public .NET methods. The MyEngineDotNetAccess class will change some of the calling conventions.
The specific implementation of the MyEngineDLLAccess class depends on the compiler you are using. Start by implementing export methods for the Initialize, PerformTimeStep, Finish and Dispose functions.
The code listed below shows an example of such an implementation for the Simple River Fortran engine. Note that this implementation corresponds to a particular Fortran compiler; the syntax may vary between compilers.
| Code Block |
|---|
using System;
using System.Run-time.InteropServices;
using System.Text;
namespace MyOrganisation.OpenMI.MyModel
{
public class MyEngineDLLAccess
{
[DLLImport(@'C:\MyEngine\bin\MyEngine.DLL',
EntryPoint = 'INITIALIZE',
SetLastError=true,
ExactSpelling = true,
CallingConvention=CallingConvention.Cdecl)\]
public static extern bool Initialize(string filePath, uint length);
[DLLImport(@'C:\MyEngine\bin\MyEngine.DLL',
EntryPoint = 'PERFORMTIMESTEP',
SetLastError=true,
ExactSpelling = true,
CallingConvention=CallingConvention.Cdecl)\]
public static extern bool PerformTimeStep();
[DLLImport(@'C:\MyEngine\bin\MyEngine.DLL',
EntryPoint = 'FINISH',
SetLastError=true,
ExactSpelling = true,
CallingConvention=CallingConvention.Cdecl)\]
public static extern bool Finish();
}
}
|
4.4 Step 4: Implementing the MyEngineDotNetAccess
The fourth step is to implement the MyEngineDotNetAccess class (Figure 10).
...
| Code Block |
|---|
using System.Text;
namespace MyOrganisation.OpenMI.MyModel
{
public class MyEngineDotNetAccess
{
public void Initialize(string filePath)
{
if(!(MyModelDLL.Initialize(filePath, ((uint) filePath.Length))))
{
CreateAndThrowException();
}
}
public void PerformTimeStep()
{
if(!(MyModelDLL.PerformTimeStep()))
{
CreateAndThrowException();
}
}
public void Finish()
{
if(!(MyModel.Finish()))
{
CreateAndThrowException();
}
}
private void CreateAndThrowException()
{
int numberOfMessages = 0;
numberOfMessages = MyModelDLL.GetNumberOfMessages();
string message = 'Error Message from MyModel ';
for (int i = 0; i < numberOfMessages; i++)
{
int n = i;
StringBuilder messageFromCore = new StringBuilder(' ');
MyModelDLL.GetMessage(ref n, messageFromCore,(uint) messageFromCore.Length);
message +='; ';
message += messageFromCore.ToString().Trim();
}
throw new Exception(message);
}
}
|
4.5 Step 5: Implementing the MyEngineWrapper class
The fifth step is to implement the MyEngineWrapper class (Figure 11).
...
| Code Block |
|---|
using System;
using System.Collections
namespace MyOrganisation.OpenMI.MyModel
{
public class MyEngineWrapper : org.OpenMI.Utilities.Wrapper.IEngine
{
private MyEngineDotNetAccess _myEngine;
public void Initialize(Hashtable properties)
{
_myEngine = new MyEngineDotNetAccess();
_myEngine.Initialize((string)properties['FilePath']);
}
public void Finish()
{
_simpleRiverEngine.Finish();
}
}
}
|
4.6 Step 6: Implementing the MyModelLinkablComponent
The sixth step is to implement the MyModeLinkableComponent class (Figure 12).
...
This class inherits from the LinkableEngine class. The class creates the EngineWrapper and assigns it to the protected field variable _engineApiAccess.
4.7 Step 7: Implementation of the remaining IEngine methods
The basic structure of your engine and wrapper code is now in place. The task is now to go through the MyEngineWrapper class and complete the implementation of the methods that are currently auto-generated stub code. Some of these methods can be completed only by changing the code in the MyEngineWrapper; for others, changes also need to be made to the other classes and the engine core (MyEngineDLL). After completion of each method you should update the test classes and run the unit test.
...
| Code Block |
|---|
//== The org.OpenMI.Utilities.Wrapper.IEngine interface == // -- Execution control methods (Inherited from IRunEngine) -- void Initialize(Hashtable properties); bool PerformTimeStep(); void Finish(); //-- Time methods (Inherited from IRunEngine) -- ITime GetCurrentTime(); ITime GetInputTime(string QuantityID, string ElementSetID); ITimeStamp GetEarliestNeededTime(); //-- Data access methods (Inherited from IRunEngine) -- void SetValues(string QuantityID, string ElementSetID, IValueSet values); IValueSet GetValues(string QuantityID, string ElementSetID); //-- Component description methods (Inherited from IRunEngine) -- double GetMissingValueDefinition(); string GetComponentID(); string GetComponentDescription(); // -- Model description methods -- string GetModelID(); string GetModelDescription(); double GetTimeHorizon(); // -- Exchange items -- int GetInputExchangeItemCount(); int GetOutputExchangeItemCount(); org.OpenMI.Backbone GetInputExchangeItem(int exchangeItemIndex); org.OpenMI.Backbone GetOutputExchangeItem(int exchangeItemIndex); |
5. Migration of the Simple River
The previous chapter described the steps involved in migrating a model to the OpenMI. This
chapter shows how the migrated code is developed for the Simple River example.
5.1 The Simple River Wrapper
The Simple River model uses the migration pattern shown in Figure 4-9. Figure 4-21 gives a detailed explanation of how the Simple River wrapper works in terms of the wrapper classes.
...
Fig. 13 Simple River wrapper classes
4.5.2 Implementation of the Initialize method
The SimpleRiverEngineWrapper has two private field variables:
...
Note that no DataOperations are added to the OutputExchangeItems. The LinkableEngine class will complete the OutputExchangeItems for you by adding
spatial and temporal data operations to your OutputExchangeItems. You can still add your own data operations as well.
4.5 3 Implementing the SetValues method
The calling sequence for the SetValues method is shown on figure 15 below.
...
You can use any convention for naming these IDs. In the Simple River ElementSetID, the convention used was Branch:<branch number> (e.g. ´Branch:3' ) or 'AllBranches', and the Quantity IDs were 'Flow' and 'InFlow'.
4.5.4 Implementing the GetValues method
The source code for the IEngine GetValues implementation is shown below.
...
Fig. 16. Calling sequence for the GetValues method
4.5.5 Implementation of the remaining methods
Implementation of the remaining methods in the IEngine interface is not complicated. On the sequence diagram in Figure 4-27 you can see how each method is accessing the other engine wrapper classes.
...
| Code Block |
|---|
public OpenMI.Standard.ITime GetCurrentTime()
{
double time = _simulationStartTime + _simpleRiverEngine.GetCurrentTime() / ((double)(24*3600));
return new Oatc.OpenMI.Sdk.Backbone.TimeStamp(time);
}
|
6. Testing the component
It is important to test the component to check that it is working correctly. Traditionally, the procedure has been to complete implementation and then run the engine to see if it produces the correct results. However, in recent years new methodologies have been developed for testing. The dominant testing method for object oriented programs is unit testing. Unit testing is done in parallel with the implementation. This means that you will be able to find errors earlier and thus save time on debugging your code later.
This chapter discusses the testing of migrated components.
6.1 Unit testing
The testing procedure described here assumes you are using the NUnit test tool. You can download the NUnit user interface and libraries from http://www.NUnit.org. This web page also gives more information about NUnit. Basically, you create a test class for each of the wrapper classes; in the test classes you implement a test method for each public method in the class.
This chapter focuses on OpenMI-specific testing. The NUnit home page gives detailed information about Unit testing.
Section 4.2 describes how to create test assemblies. The test classes used for the Simple River example are shown in Figure 18.