How to turn an Ascii file reader into a LinkableComponent

Motivation

The input for a model is often contained in simple ASCII files. Wrapping these files in a Linkable Component allows you to make the data available to any OpenMI-compliant model.

You can also wrap the output file from one model so that it can serve as input for another component. Then you can run/test your model based on the output file, instead of actually having to link against to and run the model that made the file in parallel with you own model. Especially is this the only option, if the delivering model is not OpenMI-compliant.

Basic concept

Data in a Linkable Component is made available through the GetValues call of an Output item. Fig. 1 displays that exposing data from any Provider requires a GetValues call.

Figure 1: Data exchange using a GetValues call

What the Ascii file reader needs to do is to make an ILinkableComponent with an Output item that contains/have access to the data from the ascii file.

Implementation of an AsciiFileDataComponent

To be OpenMI-compliant, the AsciiFileDataComponent needs to implement the ILinkableComponent interface. The Oatc.OpenMI.Sdk.Backbone.LinkableComponent is an abstract class having basic implementation of parts of the ILinkableComponent interface. What remains to be implemented is:

  • InputItems : List of IInputs, not relevant, hence return an empty list
  • OutputItems: List of IOutputs, this is the list where to put the output item with the file data.
  • AdaptedOutputFactories: List of factories that can create variations of the Output items. Can return an empty list.
  • Initialise: This is where real work has to be done. We need to read the ascii file data and creates the corresponding Output items.
  • Validate: Well, if everything is done correctly in Initialize, there is probably not much to validate. You may return null here.
  • Prepare: Do nothing.
  • Update: We can not update any further. This is for time progressing components. Throw an exception.

The Initialise function is the key point. It needs to read the ascii file and create corresponding Output items. To create an output item, the following information is required:

  • IQuantity/IQuality/IValueDefinition: What does the output item contain
  • ITimeSet: The times where values are available
  • IElementSet: A spatial definition of where the values are situated.
  • IValueSet: The actual values.

To ease implementation, you can use the Oatc.OpenMI.Sdk.Buffer.TimeBufferer which is an output item that implements interpolation and extrapolation in time. For every ascii file, you create such a TimeBufferer and put the data from the ascii file into it.

Example implementation

Below is an example of an ASCII file; there are a number of comments, followed by a line specifying the quantity and another containing the element set definition, followed by lines of data, each with an associated date.

// Lines starting with // are comment
// - The first uncommented line defines the quantity
// - The second uncommented line defines the elements.
//   It should start with
//   - "IdBased", followed by a sequence of Id's: "Id1";"Id2",
//   - "Points", followed by a series of coordinates: "(x.xx1,y.yy1)";"(x.xx2,y.yy2)"
// - Remainder lines consist of a timestamp and values of the quantity for each element
"Flow"
"IdBased";Loc1";"Loc2";"Loc3"
"2005-01-01 00:00";"15.4";"18.2";"22.4"
"2005-01-01 03:00";"15.3";"18.5";"22.5"
"2005-01-01 06:00";"15.4";"18.9";"22.4"
"2005-01-01 09:00";"15.2";"19.0";"22.6"
"2005-01-01 12:00";"15.1";"18.8";"22.7"
"2005-01-01 15:00";"14.8";"18.6";"22.9"
"2005-01-01 18:00";"14.7";"18.4";"23.4"
"2005-01-01 21:00";"14.7";"18.2";"23.8"
"2005-01-02 00:00";"14.6";"18.2";"23.9"
"2005-01-02 03:00";"14.1";"18.1";"24.0"
"2005-01-02 06:00";"13.8";"18.2";"23.5"
"2005-01-02 09:00";"13.9";"18.0";"23.6"
"2005-01-02 12:00";"13.6";"17.8";"23.5"
"2005-01-02 15:00";"13.2";"17.3";"23.3"
"2005-01-02 18:00";"12.8";"17.4";"23.2"
"2005-01-02 21:00";"12.7";"17.2";"23.1"
"2005-01-03 00:00";"12.6";"17.2";"22.9"
"2005-01-03 03:00";"12.1";"16.8";"22.7"
"2005-01-03 06:00";"12.2";"16.6";"22.5"
"2005-01-03 09:00";"11.4";"16.5";"22.4"
"2005-01-03 12:00";"11.6";"16.4";"22.2"
"2005-01-03 15:00";"11.7";"16.3";"21.8"
"2005-01-03 18:00";"11.6";"16.0";"21.5"
"2005-01-03 21:00";"11.2";"15.8";"21.4"
"2005-01-04 00:00";"11.0";"15.6";"21.3"

We will let the component handle a number of ascii files, each filename given as one argument to Initialize. Prototype code for Initialize could look like:

public override void Initialize(IArgument[] arguments)
{
    foreach (IArgument argument in arguments)
    {
        if (argument != null)
            ReadFile(argument.ValueAsString);
    }
}

/// <summary>
/// Returns the next line from the file not being a comment line.
/// </summary>
private static string GetNextLine(StreamReader reader)
{
    string line;
    while ((line = reader.ReadLine()) != null)
        if (!line.StartsWith("//"))
            return (line);
    return (null);
}

private void ReadFile(string inputFile)
{
    TimeBufferer output = new TimeInterpolator();

    TimeSet timeSet = output.TTimeSet;
    ElementSet elementSet;

    StreamReader reader = new StreamReader(inputFile);

    // Read quantity and imply unit
    string line = GetNextLine(reader).Trim(' ', '"');
    Quantity quantity = new Quantity(line);
    if (line.Equals("Flow", StringComparison.InvariantCultureIgnoreCase))
        quantity.Unit = new Unit(PredefinedUnits.CubicMeterPerSecond);
    else if (line.Equals("WaterLevel", StringComparison.InvariantCultureIgnoreCase))
        quantity.Unit = new Unit(PredefinedUnits.Meter);
    else
        quantity.Unit = new Unit("Unspecified unit");
    output.ValueDefinition = quantity;

    // Read elementset
    line = GetNextLine(reader);
    string[] elements = line.Split(';');
    string elementTypeString = elements[0].Trim('"');
    if (elementTypeString.Equals("IdBased", StringComparison.InvariantCultureIgnoreCase))
    {
        elementSet = new ElementSet(inputFile + "-" + quantity.Caption, inputFile, ElementType.IdBased, "");
        for (int i = 1; i < elements.Length; i++)
        {
            elementSet.AddElement(new Element(elements[i].Trim('"')));
        }
    }

    else if (elementTypeString.Equals("Points", StringComparison.InvariantCultureIgnoreCase))
    {
        elementSet = new ElementSet(inputFile + "-" + quantity.Caption, inputFile, ElementType.Point, "");
        for (int i = 1; i < elements.Length; i++)
        {
            string[] coordinates = elements[i].Trim('"', '(', ')').Split(',');
            if (coordinates.Length < 2)
                throw new InvalidDataException("Invalid file format: only one coordinate for point: " + inputFile);
            Element element = new Element();
            double x = Double.Parse(coordinates[0], NumberFormatInfo.InvariantInfo);
            double y = Double.Parse(coordinates[1], NumberFormatInfo.InvariantInfo);
            Coordinate elmtCoor = new Coordinate(x, y);
            element.Vertices = new Coordinate[] { elmtCoor };
            elementSet.AddElement(element);
        }
    }
    else
    {
        throw new InvalidDataException("Invalid file format: Element type not reckognized: " + inputFile);
    }
    output.ElementSet = elementSet;


    // Read times and values. First element is time, following are element values.
    ITimeSpaceValueSet valueSet = output.Values;
    while ((line = GetNextLine(reader)) != null)
    {
        string[] fileValues = line.Split(';');
        if (fileValues.Length - 1 != elementSet.ElementCount)
            throw new InvalidDataException("Number of data does not match number of elements on line: \n" + line);

        DateTime timestamp = DateTime.ParseExact(fileValues[0].Trim('"'), "yyyy-MM-dd HH:mm", CultureInfo.InvariantCulture);
        Time time = new Time(timestamp);

        double[] locationValues = new double[fileValues.Length - 1];
        for (int i = 1; i < fileValues.Length; i++)
        {
            locationValues[i - 1] = Double.Parse(fileValues[i].Trim('"'), NumberFormatInfo.InvariantInfo);
        }
        timeSet.Times.Add(time);
        valueSet.Values2D.Add(locationValues);
    }

    reader.Close();

    timeSet.SetTimeHorizonFromTimes();
    
    _outputs.Add(output);
}

You can find the entire example in

http://openmi.svn.sourceforge.net/viewvc/openmi/trunk/src/csharp/Oatc.OpenMI/Examples/AsciiFileReader/

Difference to a numerical model

The difference between the wrapping of ASCII files and the approach used for numerical model components lies in the implementation of the OutputItems and the GetValues call.

In the case of a numerical model component, the ExchangeItems will usually be provided by a populated model. When wrapping an ASCII file, the information about possible ExchangeItems must be contained in the ASCII file itself.

Also, a numerical model will usually only provide instant values in its output items. The buffering is taken care of by putting an AdaptedOutput in front of the Output item that does the buffering. The ASCII file component provides directly the buffered data in an Output item