Test Plan
...
Deltares Systems Software commitment to quality control and quality assurance has leaded them to develop a formal and extensive procedure to verify the correct working of all of their geotechnical engineering tools, including a test plan.
The Test plan document can be as varied as the products and organizations to which they apply.
This section describes how these tests will take place at three different levels:
- Code level - unit test: Program code must be tested alongside the code. Such a test checks if the code does what it is supposed to do with a little test. The tests on code level are the unit tests. For each relevant function, a unit test is defined within the C# solution. A relevant function is a function that actually performs part of the calculation, validation or I/O of the core. Properties and purely administrational functions do not have unit tests.
These tests are considered to be ok when the unit tests pass and when the code coverage of those tests is more than 75%. For these tests the following information must be provided in the Test Report:
Number of unit tests
Code coverage of the unit tests (not done)
Specify if all unit tests succeed or not
- Functional level - integration test: The tests on functional level are the integration tests. These types of tests combine multiple functions in D-Flow Slide to prove that high level functionality works. For this, a unit test is defined within the C# solution for each method with high level functionality. These tests are considered to be ok when the unit tests pass. The following information must be provided in the Test Report:
- Number of integration tests (= Benchmarks)
- Code coverage of the unit tests (not done)
- Specify if all integration tests succeed or not
...
3. Application level (System tests) : The tests on this level are to provide proof of the fact that the D-Flow Slide meets its acceptation criteria. The criteria are:
- Alle main functions must provide the correct answers (detailed results of these tests can be found in the Test Report) to confirm its performance according to the functional design.
- All possible errors must be handled and reported properly.
Based on the formulae in the Functional Design for D-Flow Slide benchmarks were created to test the kernel and the User Interface.
Test Report
The following chapters describe the tests in detail. If all tests are run with a satisfying result, the kernel is said to be good
The Userinterface has been tested manually by students using the Checklist for the program, based on the usermanual with technical requirements.
Number of Unit tests
- Deltares.FlowSlide.Data.Tests : 291 tests inclusive benchmarks (test the calculation)
- Deltares.FlowSlide.Forms.Tests : 40 tests (test the UI of D-Flow Slide)
Benchmarks - Integration tests
These benchmark (integration test, 29) . An extensive range of benchmark checks have been developed to check the correct functioning of each tool. During product development these checks are run on a regular basis to verify the improved product. These benchmark checks are provided in the following sections, to allow the users to overview the checking procedure and verify for themselves the correct functioning of D-FLOW SLIDE.
The benchmarks for Deltares Systems are subdivided into five separate groups as described below:
- Group 1 – Benchmarks from literature (exact solution)
Simple benchmarks for which an exact analytical result is available from literature. - Group 2 – Benchmarks from literature (approximate solution)
More complex benchmarks described in literature for which an approximate solution is known. - Group 3 – Benchmarks from spreadsheets
Benchmarks which test program features using Excel spreadsheets. - Group 4 – Benchmarks generated by the program itself
Benchmarks for which the reference results are generated using D-FLOW SLIDE. - Group 5 – Benchmarks compared with other programs
Benchmarks for which the results of D-FLOW SLIDE are compared with the results of other programs.
As much as software developers would wish they could, it is impossible to prove the correctness of any non-trivial program. Re-calculating all the benchmarks and making sure the results are as they should be will prove to some degree that the program works as it should. Nevertheless there will always be combinations of input values that will cause the program to crash or produce wrong results. Hopefully by using the verification procedure the number of times this occurs will be limited.
The benchmarks will all be described to such detail that reproduction is possible at any time. In some cases, when the geometry is too complex to describe, the input file of the benchmark is needed. The results are presented in text format with each benchmark description.
The input files (*.fsx, *.mbr, *.slq) of all benchmarks can be downloaded here.
The test document used to check to program manually has been attached to this page.
The spreadsheets used for benchmarks in group 3 can be downloaded here belonging to the benchmarks can be downloaded from those pages.
Overview of the benchmarks
Legend:
= Results of D-Flow Slide and results of the Benchmark are identical.
= Results of D-Flow Slide and results of the Benchmark differ.
Group | Input file (*.fsx) | Group | File name | Title | Global | Simple CUR-113 | Detailled (TR) | Detailed | Overall | Advanced liquefaction(SLIQ2D) | Advanced breaching | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | bm1-1 | Study Case described in "in Technisch Rapport Voorland Zettingsvloeiing" | Failed |
| Passed |
| 2 | bm2-1 | Spui dike - hmp 63.9 (location Nieuw Beijerland) | Failed |
|
| | Failed
| bm2 | |
| bm1- | 2Spui dike - hmp 65.0 (between locations Oud Beijerland and Nieuw Beijerland) | Passed |
| Passed 2a | Determination of the steepest possible breaching profile (step 7 of the Global check) - Case A | | ||||||||||
| bm1- | 3Spui dike - hmp 67.8 (locationoud Beijerland) | Failed |
| Passed |
| 3 | bm3-1 | Global check with traject: step 1 = no, step 3 = yes 2b | Determination of the steepest possible breaching profile (step 7 of the Global check) - Case B | |
| bm3|||||
| bm1- | 22c | Determination of the steepest possible breaching profile (step 7 of the Global check) - Case C | | Global check with traject: step 1 = no, step 3 = no | bm3 | |||||||||||
| bm1- | 3Global check with traject: |
|
|
|
| 4 | bm4-1 | Test on the level indicator 2d | Determination of the steepest possible breaching profile (step 7 of the Global check) - Case D | | 5 | |||||
| bm1- | 1a Comparison with SLIQ2D - Case LGZM12e | Determination of the steepest possible breaching profile (step 7 of the Global check) - Case E | | Failed | bm5 | |||||||||||
| bm1- | 1b Comparison with SLIQ2D - Case LGZM22f | Determination of the steepest possible breaching profile (step 7 of the Global check) - Case F | | Failed | ||||||||||||
| bm1- | 1c Comparison with SLIQ2D - Case LGZM32g | Determination of the steepest possible breaching profile (step 7 of the Global check) - Case G | | Failed | bm5 | |||||||||||
| bm1- | 1d2h | Determination of the steepest possible breaching profile (step 7 of the Global check) - Case H | | Comparison with SLIQ2D - Case LGZM4 | Failed | bm5-1e | Comparison with SLIQ2D - Case SIMPLETA |
|
|
| Failed | |||||
| bm5-1f | Comparison with SLIQ2D - Case LG1D5N5H |
|
|
| Failed | ||||||||||
| bm1-2i | Determination of the steepest possible breaching profile (step 7 of the Global check) - Case I | |
| bm5-1g | Comparison with SLIQ2D - Case HBPZBUI3 Failed | bm5- | ||||||||||
2 | Comparison with DZettingsVloeiing - Study Case described in "Technisch Rapport Voorland Zettingsvloeiing"
| Passed |
|
According to hand-calculation, the Advanced check passes but according to D-Flow Slide it fails.
Group 1: Benchmarks from literature (exact solution)
This section describes a number of benchmarks for which an exact analytical solution can be found in the literature.
Study Case described in "Technisch Rapport Voorland Zettingsvloeiing".
Description
The example given in "Annex A - Case Study" of the Deltares report 1200503-001-GEO-0004 "Concept Technisch Rapport Voorland Zettingsvloeiing" of G.A. van den Ham & Co is used.
In this example, both global and detailed checks are completed the basis of a (fictitious) dike section, which with regard to geometry and soil structure is typical of the Southwest Delta.
The dike has a height of NAP + 5 m and a crest width of 3 m.
The foreshore begins to imaginary toe of the dike at an elevation of NAP and is 60m wide. The toe of the trench is NAP-15, 0 m and has a slope of 1:6. The dike section is 800m long.
The soil profile is as follows:
- from NAP +3.5 m to NAP +1 m: peat
- from NAP +1 m to NAP -5 m: silty clay
- from NAP -5 m to NAP -18 m: moderately to loosely compacted sand (Calais)
*from NAP -18 m to NAP -30 m: densely compacted sand
Benchmarks results
details of the analytical calculation can be found
The global check
Description
Group 2: Benchmarks from literature (approximate solution)
This section uses the results of a by-hand calculation performed for three profiles of the Spui dike project (see figure below):
...
bm2-1 | Spui dike - hmp 63.9 (location Nieuw Beijerland) |
|
| |
|
|
bm2-2 | Spui dike - hmp 65.0 (between locations Oud Beijerland and Nieuw Beijerland) |
...
|
| (a) |
|
| ||
bm2-3 | Spui dike - hmp 67.8 (location Oud Beijerland) |
|
| |
|
|
XXX Insert figure
As the details of the calculation performed by hand are not available, those benchmarks are set in group 2.
hmp 63.9 (Nieuw Beijerland)
Description
The dike profile at hmp 63.9 is given in the figure below.
XXX Insert figure
The soil profile and the material properties are given in the following table:
Level top layer (m NAP) | Material | Formation | Material sensitive to liquefaction | D r (%) | n (%) | n min (%) | n max (%) | Gamma unsat (kN/m 3) | Gamma sat (kN/m 3) | Gamma grains (kN/m 3) | D 50 (mm) | D 15 (mm) | Phi (deg) | c (kPa) | Eps voldm0 | s max | s 2 | k so (kN/m 2) | m | r | u | v | |||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Grond level | Clay | Dunkirk / anthropogenic | No |
|
|
|
| 18 | 18 | 26.5 |
|
| 25 | 0 3 |
|
|
|
|
| ||||||
-0.5 | Clay/Sand | Dunkirk | No |
|
|
|
| 18 | 18 | 26.5 |
|
| 25 | 0 |
|
|
|
|
|
|
|
| |||
bm3-1a | Global check with traject: step 1 = No (Global passes) |
|
| |
|
| |||||||||||||||||||
| bm3-1b | Global check with traject: step 1 = Yes, step 3 = Yes (Global fails) | | | (b) | |||||||||||||||||||||
bm3-1c | Global check with traject: step 1 = Yes, step 3 = No, step 4 = No (Global passes) |
| -6.5 | Peat | Holland | No |
|
|
| 10 | 10 | 26.5 |
|
| 25 | 0 |
|
|
|
|
|
|
|
| |
-7.5 | Clay | Calais | No |
|
|
|
| 18 | 18 | 26.5 |
|
| 25 | 0 |
|
|
|
|
|
|
|
| |||
-8.5 | Sand clayey/Sand | Calais | Yes | 35 | 45 | 36.5 | 49.5 | 19 | 19 | 26.5 | 160 | 80 | 25 | 0 | 0.000444 | 1.575 | 1.2 | 43.2 | 3 | 7 | 1.2 | 1 | |||
-9.5 | Sand | Calais | Yes | 30 | 45.6 | 36.5 | 49.5 | 20 | 20 | 26.5 | 160 | 80 | 30 | 0 | 0.000325 | 1.55 | 1.2 | 41.6 | 3 | 7 | 1.2 | 1 | |||
-12.5 | Sand | Calais | Yes | 25 | 46.3 | 36.5 | 49.5 | 20 | 20 | 26.5 | 160 | 80 | 30 | 0 | 0.000238 | 1.525 | 1.2 | 40 | 3 | 7 | 1.2 | 1 | |||
-18 | Sand | Pleistocene | Yes | 60 | 41.7 | 36.5 | 49.5 | 20 | 20 | 26.5 | 160 | 80 | 35 | 0 | 0.00212 | 1.7 | 1.2 | 51.2 | 3 | 7 | 1.2 | 1 |
Benchmark results
D-FLOW SLIDE results
hmp 65.0 (between Oud Beijerland and Nieuw Beijerland)
hmp 67.8 (oud Beijerland)
Group 3: Benchmarks from spreadsheets
This section contains tests that are missing in the other groups, for example for the CUR-113 method.
Group 5: Benchmarks compared with other programs
These benchmarks are intended to verify the advanced method by comparing D-FLOW SLIDE results with those from the older program SLIQ-2D, using both DOS and Windows version of this program.
...
Benchmark name
...
Original file name (SLIQ 2D DOS)
...
Soil type
...
Slope height (m)
...
Slope angle
...
n (%)
...
n min (%)
...
n max (%)
...
Eps voldm0
...
m
...
r
...
s 2
...
s max
...
v
...
k so (kN/m 2)
...
u
...
Unit weight (kN/m 3)
...
D r (%)
...
bm5-1a
...
LGZM1
...
Sand clay
...
21.9
...
1:4.5
...
45.5
...
50
...
35
...
0.0025
...
1.7
...
7
...
1.28
...
1.7
...
1
...
50000
...
1
...
8.856
...
30
...
bm5-1b
...
LGZM2
...
Sand clay
...
21.9
...
1:5.5
...
45.5
...
50
...
35
...
0.0035
...
1.7
...
7
...
1.28
...
1.7
...
1
...
50000
...
1
...
8.856
...
30
...
bm5-1c
...
LGZM3
...
Sand clay
...
21.9
...
1:7
...
45.5
...
50
...
35
...
0.005
...
1.7
...
7
...
1.28
...
1.7
...
1
...
50000
...
1
...
8.856
...
30
...
bm5-1d
...
LGZM4
...
Sand clay
...
21.9
...
1:7.5
...
45.5
...
50
...
35
...
0.006
...
1.7
...
7
...
1.28
...
1.7
...
1
...
50000
...
1
...
8.856
...
30
...
bm5-1e
...
SIMPLETA
...
Sea sand
...
10
...
1:3.25
...
47.4
...
50
...
37
...
0.0092
...
3
...
7
...
1.23
...
1.35
...
1
...
39460
...
1.33
...
8.547
...
20
...
bm5-1f
...
LG1D5N5H
...
Sand clay
...
20.9
...
1:6.5
...
40.2
...
50
...
35
...
0.0022
...
1.7
...
7
...
1.18
...
1.4
...
1
...
85000
...
1
...
9.717
...
65.333
| |||||||
bm3-1d | Global check with traject: step 1 = Yes, step 2 = No, step 4 = Yes, step 5 = Yes (Global fails) |
|
| |
|
| |
bm3-1e | Global check with traject: step 1 = Yes, step 2 = No, step 4 = Yes, step 5 = No, step 6 = Yes (Global fails) |
|
| |
|
| |
bm3-1f | Global check with traject: step 1 = Yes, step 2 = No, step 4 = Yes, step 5 = No, step 6 = No, step 7 = Yes (Global fails) |
|
| |
|
| |
| bm3-1g | Global check with traject: step 1 = Yes, step 2 = No, step 4 = Yes, step 5 = No, step 6 = No, step 7 = No (Global passes) | | | | |||
5
| bm5-1a | Comparison with SLIQ2D-Windows - 1 saturated layer (Case LGZM1) |
|
|
|
| |
bm5-1b | Comparison with SLIQ2D-Windows - 1 saturated layer (Case LGZM2) |
|
|
|
| ||
bm5-1c | Comparison with SLIQ2D-Windows - 1 saturated layer (Case LGZM3) |
|
|
|
| ||
bm5-1d | Comparison with SLIQ2D-Windows - 1 saturated layer (Case LGZM4) |
|
|
|
| ||
bm5-1e | Comparison with SLIQ2D-Windows - 1 saturated layer (Case SIMPLETA) |
|
|
|
| ||
bm5-1f | Comparison with SLIQ2D-Windows - 1 saturated layer (Case LG1D5N5H) |
|
|
|
| ||
bm5-1g | Comparison with SLIQ2D-Windows - 1 saturated layer (Case HBPZBUI3) |
|
|
|
| ||
bm5-2 | Comparison with SLIQ2D-Windows - 2 layers partially saturated |
|
|
|
| ||
bm5-3 | Comparison with HMBreach - 1 layer |
|
|
| |
(a) The Global and Detailed checks with D-FLOW SLIDE fail whereas according to the by hand calculation they should succeed. However, by lack of information, the input value of several parameters (state parameter, critical retrogression length and migration velocity foreshore) is set arbitrary. Therefore, this comparison is not completely relevant, and it can't be concluded that the program gives incorrect results.
(b) The overall result is not correctly determined in D-Flow Slide if the foreshore is artificial: D-Flow Slide uses the Detailed check result but it should directly conclude that the Overall check fails.
...
bm5-1g
...
HBPZBUI3
...
Sea sand
...
22
...
1:6.5
...
45.5
...
50
...
35
...
0.0054
...
2
...
7
...
1.25
...
1.4
...
1.25
...
50000
...
1
...
8.856
...