Blog from May, 2009

Why arrays are bigger on 64 bit windows:

http://blogs.msdn.com/joshwil/archive/2004/04/13/112598.aspx

extend PostSharp script to recognize *.snk file
don't forget to set Category(...): Delta Shell Wave public void InsertManyObjects() { ... }
change "channel = new Channel { Network = network }" ==> "channel = new Channel(); Network.Channels.Add(channel)"
the same for BranchFeatures, NodeFeatures and many other places
disabled setters for BranchFeature.Branch, NodeFeature.Node via interfaces
TODO: rename
from:
DelftTools.DataObjects
DelftTools.DataObjects.Functions
to:
DelftTools.Hydro
DelftTools.Functions
move FeatureType from FeatureCollectionProvider to IFeatureProvider
cleanup:
((FeatureCollection)hydroNetworkLayer.BranchLayer.DataSource).Add(GeometryFromWKT.Parse("LINESTRING (0 0, 100 100)"));
to
hydroNetworkLayer.BranchLayer.DataSource.Add(GeometryFromWKT.Parse("LINESTRING (0 0, 100 100)"));
make GeometryHelper static class instead requiring to create instance of it
enable logging in SQLite tests (debugging LazyPropertyChanged test)
2d necdf grid visualization is broken
model doesn't run good with IPC (switched to TCP/IP)
cross-sections are not drawn correctly (enable point-based cross-sections as points for now?)
fix bug in tree view node presenters (no presenter for Segments, etc.)
hide disabled layer features in move tool
HACK: update time series manually
TODO: listen to the changes in Function.Filters in the FunctionBindingList and update it so that all controls will update
Chart, bug: last / first point in line series disappears
Chart, bug: function (downsamplig) does not work anymore, see AreaSeries (disabled downsampling for now)
BUG: CrossSectionView, history tool - should be enabled only when ShowHistorCommand is used (fixed in CrossSectionView)
BUG: CrossSectionView band does not work anymore, added check for left/right/bottom/top
BUG: CrossSectionView, when dragging point to the same X - exception in Variable, TODO: make CrossSectionView add only unique X points (X+small_dx)
BUG: current water level on cross-section does not exist - added as a property
BUG: water level tool changed:
double top = (lineTool != null) ? lineTool.YValue : ((!level.HasValue) ? level.Value : 0);
to:
double top = waterLevel;

TODO: remove knowledge of the WATER from Swf.Controls, review if we can't make this water level tool easier,
TODO: CrossSectionView (MarkerTool), validation of LaftRight lines does not work as expected, lines can be anywhere
BUG: lines of the tools shouldn't be drawn when they exceed X, Y axis
BUG: crossSectionView, tooltip for bottom is shown at the end while for left/right it is shown during dragging, fit it in the bottom line
BUG: history tool does not work as expected when moving points
add current water area calculation in crs (only when tools are dragged) we will certainly need something like CrossSectionEditor as a service
TODO: make new feature map tools work via editors if they are available? Consistency for ADD/MOVE, now they work in 2 different ways
TODO: merge all topology rules, rules, etc. into editors?

    IHydroNetwork network = new Network();

    INode node1 = new Node();
    INode node2 = new Node();
    INode node3 = new Node();

    network.Nodes.Add(node1);
    network.Nodes.Add(node2);
    network.Nodes.Add(node3);

    IChannel channel1 = new Channel { Name = "channel2", Length = 100.0, Source = node1, Target = node2 };
    IChannel channel2 = new Channel { Name = "channel2", Length = 100.0, Source = node2, Target = node3 };

    network.Channels.Add(channel1);
    network.Channels.Add(channel2);

    // add first cross-section directly to branch
    ICrossSection crossSection1 = new CrossSection { Name = "cross-section1", Offset = 50.0 };
    branch1.CrossSections.Add(crossSection1);

    // add second cross-section via network (performance will be smaller!)
    ICrossSection crossSection2 = new CrossSection { Name = "cross-section2", Offset = 50.0, Branch = branch2 };
    network.CrossSections.Add(crossSection2);

Note, that IChannel is an interface extending IBranch from GeoAPI.Extensions.Networks.

You can also use var instead of IChannel, INode types in the code above, to make it a bit simpler:

    var network = new Network();

    var node1 = new Node();
    var node2 = new Node();
    var node3 = new Node();

    network.Nodes.Add(node1);
    network.Nodes.Add(node2);
    network.Nodes.Add(node3);

    var channel1 = new Channel { Name = "channel2", Length = 100.0, Source = node1, Target = node2 };
    var channel2 = new Channel { Name = "channel2", Length = 100.0, Source = node2, Target = node3 };

    network.Channels.Add(channel1);
    network.Channels.Add(channel2);

    // add first cross-section directly to branch
    var crossSection1 = new CrossSection { Name = "cross-section1", Offset = 50.0 };
    branch1.CrossSections.Add(crossSection1);

    // add second cross-section via network (performance will be smaller!)
    var crossSection2 = new CrossSection { Name = "cross-section2", Offset = 50.0, Branch = branch2 };
    network.CrossSections.Add(crossSection2);

All features which are created in the code must be added to the network either directly to the branch or via network.

// OLD STYLE:
        public static void ClearSegments(IChannel branch)
        {
            IList<BranchSegment> segments = branch.HydroNetwork.BranchSegments;
            IList<BranchSegmentBoundary> segmentBoundaries = branch.HydroNetwork.BranchSegmentBoundaries;
            IList<BranchSegmentCenter> segmentCenters = branch.HydroNetwork.BranchSegmentCenters;
            // First remove the existing segments
            var toBeRemoved = new List<BranchSegment>();
            foreach (BranchSegment branchSegment in branch.BranchSegments)
            {
                // update the branch features that are linked to this branch
                toBeRemoved.Add(branchSegment);
            }
            foreach (BranchSegment branchSegment in toBeRemoved)
            {
                segments.Remove(branchSegment);
                segmentBoundaries.Remove(branchSegment.BranchSegmentBoundaryStart);
                segmentCenters.Remove(branchSegment.BranchSegmentCenter);
            }
            if (toBeRemoved.Count > 0)
            {
                segmentBoundaries.Remove(toBeRemoved[toBeRemoved.Count - 1].BranchSegmentBoundaryEnd);
            }

            branch.BranchSegments.Clear();
        }


// NEW STYLE:
        public static void ClearSegments(IChannel branch)
        {
            branch.BranchSegments.Clear();
            branch.BranchSegmentBoundaries.Clear();
            branch.BranchSegmentCenters.Clear();
        }

A bit easier

With Design by Contract one defines contracts, which are assertion-like statements about functions' input and output parameters (pre- and postconditions), as well as things that never change throughout the execution of the function (invariants). See the demo from Peli de Halleux and Manuel Fahndrich on the interaction of Code Contracts and Pex!

This site lists add-ins for .NET Reflector. After downloading one of the add-ins copy the files to the same directory as 'Reflector.exe' and load them via the 'Add-Ins' command under the 'View' menu.

See: http://www.codeplex.com/reflectoraddins

Currently Network in DelftShell has specific properties of a hydrological network: cross section, structures, boundaries. We need to extract the hydrological specific properties and use a general purpose interface. This will enable other plugins in the DelfShell family to reuse the network but also to use freely available algorithm implementations.

Terminology (from wikipedia)

Graph:
In mathematics a graph is an abstract representation of a set of objects where some pairs of the objects are connected by links.

Network:
In graph theory, a network is a directed graph with weighted edges

There are several libraries in the public domain that can be used for graph support. Either directly or as reference:

QuickGraph

http://www.codeplex.com/quickgraph

QuickGraph provides generic directed/undirected graph datastructures and algorithms for .Net 2.0 and up. QuickGraph comes with algorithms such as depth first seach, breath first search. QuickGraph is bases on the Boost library (C++).
http://www.boost.org/doc/libs/1_39_0/libs/graph/doc/index.html

NTS - Net Topology Suite

NetTopologySuite is a C#/.NET port of JTS Topology Suite, a Java library for GIS operations, (OpenGIS compliant).
NTS is also used by SharpMap, it implements for example the OpenGiS geometry interfaces like LineString. Although NTS has some classes for graphs (see folder GeometriesGraph) these classes are used internally for the Topology Graph which is used to calculate the Intersection Matrix (relate algorithm; see also http://www.vividsolutions.com/jts/bin/JTS%20Technical 20Specs.pdf chapter 10).

The NTS site has some examples/tests for routing algorithms, see: http://code.google.com/p/nettopologysuite/source/browse/trunk/NetTopologySuite.Samples.Console/Tests/Various/PathFinderTest.cs and GraphBuilderTest.cs
These implementations use the QuickGraph library.

Jung

Java Universal Network/Graph Frameworkhttp://jung.sourceforge.net/index.html
This appears a very widely used general purpose graph library. I have found no port for .Net.

Some general purpose libraries with graph support

The C5 Generic Collection Library

\http://www.itu.dk/research/c5/
C5 is a library of generic collection classes. The core classes do not implement a graph but there is an implementation in the samples: Graph.cs

NGenerics

http://ngenerics.codeplex.com/
NGenerics is a more comprehensive general purpose collection library than the C5 library.
For an interface description see:http://www.codeproject.com/KB/recipes/DotNet2Datastructures.aspx

QuickGraph seems a logical choice to use as library for DelftShell. It extensively uses interfaces which enables us to reuse our current classes in DelftShell and offers a large number of graph algorithm implementations.

As proof of concept a simple test has been written that uses the Dijkstra algorithm for the shortest path but only uses interfaces from QuickGraph.

INetwork network = new Network();

INode nodeA = new Node {Name = "A"};
INode nodeB1 = new Node { Name = "B1" };
INode nodeB2 = new Node { Name = "B2" };
INode nodeC = new Node { Name = "C" };

IBranch edgeAB1 = new Branch(nodeA, nodeB1);
IBranch edgeAB2 = new Branch(nodeA, nodeB2);
IBranch edgeB1C = new Branch(nodeB1, nodeC);
IBranch edgeB2C = new Branch(nodeB2, nodeC);

network.Nodes.Add(nodeA);
network.Nodes.Add(nodeB1);
network.Nodes.Add(nodeB2);
network.Nodes.Add(nodeC);

network.Branches.Add(edgeAB1);
network.Branches.Add(edgeAB2);
network.Branches.Add(edgeB1C);
network.Branches.Add(edgeB2C);

IDictionary<IBranch, double> branchCost = new Dictionary<IBranch, double>();

branchCost.Add(edgeAB1, 100);
branchCost.Add(edgeAB2, 200);
branchCost.Add(edgeB1C, 50);
branchCost.Add(edgeB2C, 200);

var dijkstra = new UndirectedDijkstraShortestPathAlgorithm<INode, IBranch>(network, branchCost, new ShortestDistanceRelaxer());

VertexPredecessorRecorderObserver<INode, IBranch> predecessorObserver = new VertexPredecessorRecorderObserver<INode, IBranch>();

using (ObserverScope.Create(dijkstra1, predecessorObserver))
{
// Run the algorithm with A set to be the source
  dijkstra1.Compute(nodeA);
}

...

double distance = AlgoUtility.ComputePredecessorCost(predecessorObserver.VertexPredecessors, branchCost, nodeC);
Debug.WriteLine(string.Format("A -> {0}: {1}", nodeC, distance));

result is :
A -> Node C: 150

The interface of INode, IBranch and INetwork are defined outside the QuickGraph library.

public interface INode
public interface IBranch : IEdge<INode>
public interface INetwork : IUndirectedGraph<INode, IBranch>

IEdge and IUndirectedGraph are defined in the QuickGraph library.

DelftShell now uses a custom build file (DelftShell.Taget) to copy necessary dlls and invoke postsharp when necessary. This means that when you create a new Assembly you have to set up some additional properties in the .csproj file.

Setting up a new project file

It is important to include these steps when you a add a new project to DelftShell (maybe we can make a template of this??). You have to make some modifications to the .csproj file by editing it in a text editor.

1 Add the following line to the project file directly below the import of Microsoft.CSharp.targets

<Import Project="..\..\..\..\..\\build\DelftShell.targets" />
This makes sure the new project is build with the DelftShell custom build logic

2
Add the following xml just before the closing </Project> tag of the project file.

<PropertyGroup>
  <IsPluginComponent>true</IsPluginComponent>
  <PluginName>$(ProjectName)</PluginName>
  <UsePostSharp>true</UsePostSharp>
</PropertyGroup>

IsPluginComponent makes sure the output of the assembly is copied to the 'loader'.
PluginName is the name of plugin project.
UsePostSharp determines if PostSharp is invoked on the dll.

What does DelftShell.Targets file do?

1 Invoke PostSharp on the assembly if necessary. This is done by calling PostSharpPostBuild.cmd for the project.
2 If the project is a PluginComponent it copies the output of the project (the dll) to \loader\bin\plugins\$PluginName\
3 If the project is a PluginComponent it copies the referenced libararies of the project to \loader\bin\plugins\$PluginName\. The location of these files is relative to the project folder. For example:

\DelftShellPlugins\DelftModels\modules\DelftShell.Plugins.DelftModels.FlowModel
\DelftShellPlugins\DelftModels\lib\DelftShell.Plugins.DelftModels.FlowModel

The subdirectory under \lib is determined by the PluginName.

The last two steps are done by calling PluginPostBuild.cmd

Location of the build files

\DelftShell.Targets : $SolutionDir\build
\PluginPostBuild.cmd: $SolutionDir\bin
\PostSharpPostBuild.cmd: $SolutionDir\bin

I tried to optimize performance of the MD arrays a bit. It still works very slow for big arrays in my opinion and we will need some solution which will perform good for a real projects where arrays will be a 100x Mln values. Obviously NetCDF will be one of the solutions in those cases with implementation of NetCdfMultiDimensionalArray : ICachedMultiDimensionalArray {}.

Currently there are far too many checks +events are fired on every change. Actions to take in the nearest future:

• group events
• implement NetCdfMultiDimensionalArray : ICachedMultiDimensionalArray {} to allow working with huge arrays which don't fit into memory
• test NetCdfFunctionStore on huge 2d arrays (100Mb on file system)

Here are also a few MultiDimensionalArray / Function test results from TeamCity: