| Info |
|---|
The purpose of this document is to provide guildelines guidelines on how to write and maintain tests in Delta Shell. It also provides list of typical issues occuring occurring during automatic testing and guidelines on how to resolve them. The issues are discussed in order of severity. |
| Table of Contents |
|---|
Testing Conventions
Where data access or integration tests can write their output data?
Delta Shell uses the followin following directory layout, not locations of the test/ and test-data/ directories used for test code and data:
| No Format |
|---|
build/
doc/
lib/
setup/
src/ ..................................... program sources
Common/
DeltaShell/
Plugins/
DelftModels/
DeltaShell.Sobek.Readers/
DeltaShell.Sobek.Readers.csproj
...
test/ .................................... tests sources
Common/
DeltaShell/
Plugins/
DelftModels/
DeltaShell.Sobek.Readers.Tests/
DeltaShell.Sobek.Readers.Tests.csproj
...
test-data/ ............................... data files used by some tests
Common/
DeltaShell/
Plugins/
DelftModels/
DeltaShell.Sobek.Readers.Tests/
RD-02X.bui
...
DeltaShell.proj .......................... MSBuild project
DeltaShell.sln ........................... Visual Studio Solution
|
...
- Projects of the tests are organized in the same way as projects of the sources (in most cases), see above listing.
- Tests should never write their output data into test-data/ directory. That directory is used for input data files used the tests. Instead they should copy required data into the test output directory (usually bin/Debug or bin/Release, it is very easy to do using FileUtils.Copy() method).
- Make sure that output files from the previous test run are deleted somethere somewhere in test [DS10:SetUp].
Where can I find data files which can be used in data access or integration tests?
...
Currently we use the following categories:
- <no caterody category is specified> - unit test. Fast test to test a class in isolation.
- Integration - integration test where multiple classes / components are used
- DataAccess - specific integration test where file is accessed (read / write)
- WindowsForms - specific integration where form is shown
- Pefromance Performance - unit or integration test where speed of the code is measured.
- Jira - (optional) test describing a bug in Jira. The test should specify for which issue it was created.
- WorkInProgress - used to mark test which is incomplete (probably failing). In this case it will not run on build server during automated testing.
- Slow - very slow integration tests (>1m) need to run separately, less frequently, in order to keep the rest of the test suite run fast. Mark your test Slow in addition to the other categories.
...
Example of the unit test
| Code Block | ||||
|---|---|---|---|---|
| ||||
[Test]
public void AddDependentVariableValues()
{
var x = new Variable<double>();
var y = new Variable<double> { Arguments = {x} };
x.Values.Add(0.0);
x.Values.Add(0.1);
x.Values.Add(0.2);
Assert.AreEqual(3, x.Values.Count, "number of values in x (independent variable)");
Assert.AreEqual(3, y.Values.Count, "number of values in y (dependent variable)");
Assert.AreEqual(y.DefaultValue, y.Values[0], , "first value in dependent variable is set to default");
}
|
Example of the unfinished performance test.
| Code Block | ||||
|---|---|---|---|---|
| ||||
[Test]
[Category(TestCategory.Performance)]
[Category(TestCategory.WorkInProgress)]
public void PerformanceAddValuesToArgument_WithEvents()
{
IVariable<double> x = new Variable<double> { GenerateUniqueValueForDefaultValue = true };
IVariable<double> y = new Variable<double> { Arguments = {x} };
const int valuesToAdd = 20000;
// check if code runs fast enough
TestHelper.AssertIsFasterThan(50, () => x.Values.InsertAt(0, 0, valuesToAdd));
}
|
...
Every developer is strongly adviced advised to use TeamCity pre-tested commit feature if commit was not tested locally! Visual Studio plugin for TeamCity can be downloaded from My Settings & Tools menu on build server.
...
Each configuration has maximum duration, typically 30, 60 or 90 minutes. The test run is killed after that time. Identify why it's taking longer than before:
– TestRunner hanging on .NET exception window? Logon build server with remote desktop to note exception and close window.
– Infinite loop / recursion going on? Check the build log for the running test / stacktrace stack trace on process kill to identify test.
– Was a very 'heavy' test added? Contact author, fix test duration, or (worst case) adjust configuration allowed time
...
The how-to for tortoise is described
| Wiki Markup |
|---|
{clickable:http://tortoisesvn.net/docs/nightly/TortoiseSVN_en/tsvn-howto-rollback.html}{color:#0000ff}here{color}{clickable} |
.
| Section | ||||
|---|---|---|---|---|
|
...