Page History

Testing Conventions

Where data access or integration tests can write their output data?

Delta Shell uses the followin following directory layout, not locations of the test/ and test-data/ directories used for test code and data:

build/
doc/
lib/
setup/
src/ ..................................... program sources
  Common/
  DeltaShell/
  Plugins/
    DelftModels/
       DeltaShell.Sobek.Readers/
          DeltaShell.Sobek.Readers.csproj
          ...
test/ .................................... tests sources
  Common/
  DeltaShell/
  Plugins/
    DelftModels/
       DeltaShell.Sobek.Readers.Tests/
          DeltaShell.Sobek.Readers.Tests.csproj
          ...
test-data/ ............................... data files used by some tests
  Common/
  DeltaShell/
  Plugins/
    DelftModels/
       DeltaShell.Sobek.Readers.Tests/
          RD-02X.bui
          ...
DeltaShell.proj .......................... MSBuild project
DeltaShell.sln ........................... Visual Studio Solution

...

• Projects of the tests are organized in the same way as projects of the sources (in most cases), see above listing.
• Tests should never write their output data into test-data/ directory. That directory is used for input data files used the tests. Instead they should copy required data into the test output directory (usually bin/Debug or bin/Release, it is very easy to do using FileUtils.Copy() method).
• Wiki MarkupMake sure that output files from the previous test run are deleted somethere somewhere in test \ [DS10:SetUp\].

Where can I find data files which can be used in data access or integration tests?

...

Why?
To make it easy to find the tests for a class.

Unit test categories.

...

Make sure that you put \ [DS10:Category(TestCategory.<name of the category>)\] attribute above your test if it is not a unit test. This is *very important* since it makes sure that all unit tests run very fast (3000 tests in <1 min).

Currently we use the following categories:

• <no caterody category is specified> - unit test. Fast test to test a class in isolation.
• Integration - integration test where multiple classes / components are used
  • DataAccess - specific integration test where file is accessed (read / write)
  • WindowsForms - specific integration where form is shown
• Pefromance Performance - unit or integration test where speed of the code is measured.
• Jira - (optional) test describing a bug in Jira. The test should specify for which issue it was created.
• WorkInProgress - used to mark test which is incomplete (probably failing). In this case it will not run on build server during automated testing.
• Slow - very slow integration tests (>1m) need to run separately, less frequently, in order to keep the rest of the test suite run fast. Mark your test Slow in addition to the other categories.

...

Example of the unit test

[Test]
public void AddDependentVariableValues()
{
   var x = new Variable<double>();
   var y = new Variable<double> { Arguments = {x} };

   x.Values.Add(0.0);
   x.Values.Add(0.1);
   x.Values.Add(0.2);

   Assert.AreEqual(3, x.Values.Count, "number of values in x (independent variable)");
   Assert.AreEqual(3, y.Values.Count, "number of values in y (dependent variable)");
   Assert.AreEqual(y.DefaultValue, y.Values[0], , "first value in dependent variable is set to default");
}

Example of the unfinished performance test.

[Test]
[Category(TestCategory.Performance)]
[Category(TestCategory.WorkInProgress)]
public void PerformanceAddValuesToArgument_WithEvents()
{
    IVariable<double> x = new Variable<double> { GenerateUniqueValueForDefaultValue = true };
    IVariable<double> y = new Variable<double> { Arguments = {x} };


    const int valuesToAdd = 20000;

    // check if code runs fast enough
    TestHelper.AssertIsFasterThan(50, () => x.Values.InsertAt(0, 0, valuesToAdd));
}

...

Every developer is strongly adviced advised to use TeamCity pre-tested commit feature if commit was not tested locally! Visual Studio plugin for TeamCity can be downloaded from My Settings & Tools menu on build server.

...

Each configuration has maximum duration, typically 30, 60 or 90 minutes. The test run is killed after that time. Identify why it's taking longer than before:
– TestRunner hanging on .NET exception window? Logon build server with remote desktop to note exception and close window.
– Infinite loop / recursion going on? Check the build log for the running test / stacktrace stack trace on process kill to identify test.
– Was a very 'heavy' test added? Contact author, fix test duration, or (worst case) adjust configuration allowed time

...

...

The how-to for tortoise is described

.

Fix Drag&Drop problem

...