Test Plan
Deltares Software commitment to quality control and quality assurance has leaded them to develop a formal and extensive procedure to verify the correct working of all of their geotechnical engineering tools, including a test plan.
The Test plan document can be as varied as the products and organizations to which they apply.
This section describes how these tests will take place at three different levels:
- Code level - unit test: Program code must be tested alongside the code. Such a test checks if the code does what it is supposed to do with a little test. The tests on code level are the unit tests. For each relevant function, a unit test is defined within the C# solution. A relevant function is a function that actually performs part of the calculation, validation or I/O of the core. Properties and purely administrational functions do not have unit tests.
These tests are considered to be ok when the unit tests pass and when the code coverage of those tests is more than 75%. For these tests the following information must be provided in the Test Report:
Number of unit tests
Code coverage of the unit tests (not done)
Specify if all unit tests succeed or not
- Functional level - integration test: The tests on functional level are the integration tests. These types of tests combine multiple functions in D-Flow Slide to prove that high level functionality works. For this, a unit test is defined within the C# solution for each method with high level functionality. These tests are considered to be ok when the unit tests pass. The following information must be provided in the Test Report:
- Number of integration tests (= Benchmarks)
- Code coverage of the unit tests (not done)
- Specify if all integration tests succeed or not
3. Application level (System tests) : The tests on this level are to provide proof of the fact that the D-Flow Slide meets its acceptation criteria. The criteria are:
- Alle main functions must provide the correct answers (detailed results of these tests can be found in the Test Report) to confirm its performance according to the functional design.
- All possible errors must be handled and reported properly.
Based on the formulae in the Functional Design for D-Flow Slide benchmarks were created to test the kernel and the User Interface.
Test Report
The following chapters describe the tests in detail. If all tests are run with a satisfying result, the kernel is said to be good
The Userinterface has been tested manually by students using the Checklist for the program, based on the usermanual with technical requirements.
Number of Unit tests
- Deltares.FlowSlide.Data.Tests : 291 tests inclusive benchmarks (test the calculation)
- Deltares.FlowSlide.Forms.Tests : 40 tests (test the UI of D-Flow Slide)
Benchmarks - Integration tests
These benchmark (integration test, 29) checks are provided in the following sections, to allow the users to overview the checking procedure and verify for themselves the correct functioning of D-FLOW SLIDE.
The benchmarks for Deltares Systems are subdivided into five separate groups as described below:
- Group 1 – Benchmarks from literature (exact solution)
Simple benchmarks for which an exact analytical result is available from literature. - Group 2 – Benchmarks from literature (approximate solution)
More complex benchmarks described in literature for which an approximate solution is known. - Group 3 – Benchmarks from spreadsheets
Benchmarks which test program features using Excel spreadsheets. - Group 4 – Benchmarks generated by the program itself
Benchmarks for which the reference results are generated using D-FLOW SLIDE. - Group 5 – Benchmarks compared with other programs
Benchmarks for which the results of D-FLOW SLIDE are compared with the results of other programs.
As much as software developers would wish they could, it is impossible to prove the correctness of any non-trivial program. Re-calculating all the benchmarks and making sure the results are as they should be will prove to some degree that the program works as it should. Nevertheless there will always be combinations of input values that will cause the program to crash or produce wrong results. Hopefully by using the verification procedure the number of times this occurs will be limited.
The benchmarks will all be described to such detail that reproduction is possible at any time. In some cases, when the geometry is too complex to describe, the input file of the benchmark is needed. The results are presented in text format with each benchmark description.
The input files (*.fsx, *.mbr, *.slq) of all benchmarks can be downloaded here.
The test document used to check to program manually has been attached to this page.
The spreadsheets used for benchmarks in group 3 can be downloaded here.
Overview of the benchmarks
Legend:
= Results of D-Flow Slide and results of the Benchmark are identical.
= Results of D-Flow Slide and results of the Benchmark differ.
Group | Input file (*.fsx) | Title | Global | Detailed | Overall | Advanced liquefaction(SLIQ2D) | Advanced breaching |
|---|---|---|---|---|---|---|---|
1 | bm1-1 | Study Case described in Technisch Rapport Voorland Zettingsvloeiing |
|
| |
|
|
| bm1-2a | Determination of the steepest possible breaching profile (step 7 of the Global check) - Case A | | |||||
| bm1-2b | Determination of the steepest possible breaching profile (step 7 of the Global check) - Case B | | |||||
| bm1-2c | Determination of the steepest possible breaching profile (step 7 of the Global check) - Case C | | |||||
| bm1-2d | Determination of the steepest possible breaching profile (step 7 of the Global check) - Case D | | |||||
| bm1-2e | Determination of the steepest possible breaching profile (step 7 of the Global check) - Case E | | |||||
| bm1-2f | Determination of the steepest possible breaching profile (step 7 of the Global check) - Case F | | |||||
| bm1-2g | Determination of the steepest possible breaching profile (step 7 of the Global check) - Case G | | |||||
| bm1-2h | Determination of the steepest possible breaching profile (step 7 of the Global check) - Case H | | |||||
| bm1-2i | Determination of the steepest possible breaching profile (step 7 of the Global check) - Case I | | |||||
2
| bm2-1 | Spui dike - hmp 63.9 (location Nieuw Beijerland) |
|
| |
|
|
bm2-2 | Spui dike - hmp 65.0 (between locations Oud Beijerland and Nieuw Beijerland) |
|
| (a) |
|
| |
bm2-3 | Spui dike - hmp 67.8 (location Oud Beijerland) |
|
| |
|
| |
3
| bm3-1a | Global check with traject: step 1 = No (Global passes) |
|
| |
|
|
| bm3-1b | Global check with traject: step 1 = Yes, step 3 = Yes (Global fails) | | | (b) | |||
bm3-1c | Global check with traject: step 1 = Yes, step 3 = No, step 4 = No (Global passes) |
|
|
|
| ||
bm3-1d | Global check with traject: step 1 = Yes, step 2 = No, step 4 = Yes, step 5 = Yes (Global fails) |
|
| |
|
| |
bm3-1e | Global check with traject: step 1 = Yes, step 2 = No, step 4 = Yes, step 5 = No, step 6 = Yes (Global fails) |
|
| |
|
| |
bm3-1f | Global check with traject: step 1 = Yes, step 2 = No, step 4 = Yes, step 5 = No, step 6 = No, step 7 = Yes (Global fails) |
|
| |
|
| |
| bm3-1g | Global check with traject: step 1 = Yes, step 2 = No, step 4 = Yes, step 5 = No, step 6 = No, step 7 = No (Global passes) | | | | |||
5
| bm5-1a | Comparison with SLIQ2D-Windows - 1 saturated layer (Case LGZM1) |
|
|
|
| |
bm5-1b | Comparison with SLIQ2D-Windows - 1 saturated layer (Case LGZM2) |
|
|
|
| ||
bm5-1c | Comparison with SLIQ2D-Windows - 1 saturated layer (Case LGZM3) |
|
|
|
| ||
bm5-1d | Comparison with SLIQ2D-Windows - 1 saturated layer (Case LGZM4) |
|
|
|
| ||
bm5-1e | Comparison with SLIQ2D-Windows - 1 saturated layer (Case SIMPLETA) |
|
|
|
| ||
bm5-1f | Comparison with SLIQ2D-Windows - 1 saturated layer (Case LG1D5N5H) |
|
|
|
| ||
bm5-1g | Comparison with SLIQ2D-Windows - 1 saturated layer (Case HBPZBUI3) |
|
|
|
| ||
bm5-2 | Comparison with SLIQ2D-Windows - 2 layers partially saturated |
|
|
|
| ||
bm5-3 | Comparison with HMBreach - 1 layer |
|
|
| |
(a) The Global and Detailed checks with D-FLOW SLIDE fail whereas according to the by hand calculation they should succeed. However, by lack of information, the input value of several parameters (state parameter, critical retrogression length and migration velocity foreshore) is set arbitrary. Therefore, this comparison is not completely relevant, and it can't be concluded that the program gives incorrect results.
(b) The overall result is not correctly determined in D-Flow Slide if the foreshore is artificial: D-Flow Slide uses the Detailed check result but it should directly conclude that the Overall check fails.
