Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Test Plan

 

Deltares Software

Table of Contents

Table of Contents
maxLevel3

Introduction

Deltares Systems commitment to quality control and quality assurance has leaded them to develop a formal and extensive procedure to verify the correct working of all of their geotechnical engineering tools, including a test plan.

The Test plan document can be as varied as the products and organizations to which they apply.

 

 

This section describes how these tests will take place at three different levels:

  1. Code level - unit test: Program code must be tested alongside the code. Such a test checks if the code does what it is supposed to do with a little test. The tests on code level are the unit tests. For each relevant function, a unit test is defined within the C# solution. A relevant function is a function that actually performs part of the calculation, validation or I/O of the core. Properties and purely administrational functions do not have unit tests.

    These tests are considered to be ok when the unit tests pass and when the code coverage of those tests is more than 75%. For these tests the following information must be provided in the Test Report:

     

    • Number of unit tests

    • Code coverage of the unit tests (not done)

    • Specify if all unit tests succeed or not 

  2. Functional level - integration test: The tests on functional level are the integration tests. These types of tests combine multiple functions in D-Flow Slide to prove that high level functionality works. For this, a unit test is defined within the C# solution for each method with high level functionality. These tests are considered to be ok when the unit tests pass. The following information must be provided in the Test Report:
    • Number of integration tests (= Benchmarks)
    • Code coverage of the unit tests (not done)
    • Specify if all integration tests succeed or not

...


3. Application level (System tests) :  The tests on this level are to provide proof of the fact that the D-Flow Slide meets its acceptation criteria. The criteria are:

 

  • Alle main functions must provide the correct answers (detailed results of these tests can be found in the Test Report) to confirm its performance according to the functional design.
  • All possible errors must be handled and reported properly.

    Based on the formulae in the Functional Design for D-Flow Slide benchmarks were created to test the kernel and the User Interface.

Test Report

 

The following chapters describe the tests in detail. If all tests are run with a satisfying result, the kernel is said to be good

The Userinterface has been tested manually by students using the Checklist for the program, based on the usermanual with technical requirements.


Number of Unit tests

 

  • Deltares.FlowSlide.Data.Tests : 291 tests inclusive benchmarks (test the calculation)
  • Deltares.FlowSlide.Forms.Tests : 40 tests (test the UI of D-Flow Slide)

Benchmarks - Integration tests

These benchmark (integration test, 29) . An extensive range of benchmark checks have been developed to check the correct functioning of each tool. During product development these checks are run on a regular basis to verify the improved product. These benchmark checks are provided in the following sections, to allow the users to overview the checking procedure and verify for themselves the correct functioning of D-FLOW SLIDE.
The benchmarks for Deltares Systems are subdivided into five separate groups as described below:

  • Group 1 Benchmarks from literature (exact solution)
    Simple benchmarks for which an exact analytical result is available from literature.
  • Group 2 Benchmarks from literature (approximate solution)
    More complex benchmarks described in literature for which an approximate solution is known.
  • Group 3 Benchmarks from spreadsheets
    Benchmarks which test program features using Excel spreadsheets.
  • Group 4 Benchmarks generated by the program itself
    Benchmarks for which the reference results are generated using D-FLOW SLIDE.
  • Group 5 Benchmarks compared with other programs
    Benchmarks for which the results of D-FLOW SLIDE are compared with the results of other programs.
     

As much as software developers would wish they could, it is impossible to prove the correctness of any non-trivial program. Re-calculating all the benchmarks and making sure the results are as they should be will prove to some degree that the program works as it should. Nevertheless there will always be combinations of input values that will cause the program to crash or produce wrong results. Hopefully by using the verification procedure the number of times this occurs will be limited.
The benchmarks will all be described to such detail that reproduction is possible at any time. In some cases, when the geometry is too complex to describe, the input file of the benchmark is needed. The results are presented in text format with each benchmark description.

 

The input files (*.fsx, *.mbr, *.slq) of all benchmarks can be downloaded here.

The test document used to check to program manually has been attached to this page.

The spreadsheets used for benchmarks in group 3 can be downloaded here belonging to the benchmarks can be downloaded from those pages.

Overview of the benchmarks

...

...

Legend:
(tick) = Results of The check result (i.e. Sufficient or Insufficient) is given in blue for D-Flow Slide and in purple for results of the Benchmark are identical.

...

Group

...

File name

(error) = Results of D-Flow Slide and results of the Benchmark differ.

(*.fsx) Insufficient / Insufficient => 2 Sufficient / Sufficient => (tick) bm23 Insufficient / Insufficient => bm3-1   bm32   bm33   Test on the level indicator Insufficient / Insufficient => bm5-1f bm5-2 Comparison with DZettingsVloeiing - Study Case described in "Technisch Rapport Voorland Zettingsvloeiing"

Group

Input file (*.fsx

Input file
SLIQ2D 1.1.3.1 (*.slq)

Input file
SLIQ2D-Dos (*.sli)

Title

Global (VTV)

Simple (CUR-113)

Detailed

 Overall

Advanced

liquefaction(SLIQ2D Detailed (TR)

Advanced breaching
(SLIQ2DHMTurb)

1

         

bm1-1

 

 

 

Study Case described in "in Technisch Rapport Voorland Zettingsvloeiing"

Insufficient / Insufficient => (tick)

 

Sufficient / Sufficient => (tick)

 

2

bm2-1

 

 

 

Spui dike - hmp 63.9 (location Nieuw Beijerland)

Insufficient / Insufficient => (tick)

 (tick)

 

  bm2

bm1-

 

 

 

Spui dike - hmp 65.0 (between locations Oud Beijerland and Nieuw Beijerland)

Sufficient / Sufficient => (tick)

 

2aDetermination of the steepest possible breaching profile (step 7 of the Global check) - Case A(tick)    
bm1-

 

 

 

Spui dike - hmp 67.8 (location Oud Beijerland)

2bDetermination of the steepest possible breaching profile (step 7 of the Global check) - Case B(tick) 

Insufficient / Sufficient => (error)

 

3

  

 

Global check with traject: step 1 = no, step 3 = yes

bm1-2cDetermination of the steepest possible breaching profile (step 7 of the Global check) - Case C(tick)    
bm1-

 

 

 

Global check with traject: step 1 = no, step 3 = no

2dDetermination of the steepest possible breaching profile (step 7 of the Global check) - Case D(tick)    
bm1-2eDetermination of the steepest possible breaching profile (step 7 of the Global check) - Case E(tick)

 

 

 

Global check with traject:

 

 

 

 

4

bm4-1

    

 

 

5

bm5-1a

 

 

 

Comparison with SLIQ2D - Case LGZM1

 

 

 

bm1-2fDetermination of the steepest possible breaching profile (step 7 of the Global check) - Case F(tick) 

bm5-1b

   

Comparison with SLIQ2D - Case LGZM2

 

 

 

Insufficient / Insufficient => (tick)

 

bm5-1c

 

 

 

Comparison with SLIQ2D - Case LGZM3

 

 

 

Insufficient / Insufficient => (tick)

bm1-2gDetermination of the steepest possible breaching profile (step 7 of the Global check) - Case G(tick) 

 

bm5-1d

   

Comparison with SLIQ2D - Case LGZM4

 

 

 

Insufficient / Insufficient => (tick)

 

bm5-1e

 

 

 

Comparison with SLIQ2D - Case SIMPLETA

 

 

 

Insufficient / Insufficient => (tick)

bm1-2hDetermination of the steepest possible breaching profile (step 7 of the Global check) - Case H(tick) 

 

   

Comparison with SLIQ2D - Case LG1D5N5H

 

 

 

Insufficient / Insufficient => (tick)

 

bm5-1g

 

 

 

Comparison with SLIQ2D - Case HBPZBUI3

 

 

 

Insufficient / Insufficient => (tick)

bm1-2iDetermination of the steepest possible breaching profile (step 7 of the Global check) - Case I(tick) 

 

   

2

 

 

Sufficient / Sufficient => (tick)

 

Group 1: Benchmarks from literature (exact solution)

This section describes a number of benchmarks for which an exact analytical solution can be found in the literature.

1.1. Study Case described in "Technisch Rapport Voorland Zettingsvloeiing"

Description

The example given in "Annex A - Case Study" of the Deltares report 1200503-001-GEO-0004 "Concept Technisch Rapport Voorland Zettingsvloeiing" of G.A. van den Ham & Co is used.
In this example, both global and detailed checks are completed on the basis of a (fictitious) dike section, which with regard to geometry and soil structure is typical of the Southwest Delta.
The dike has a height of NAP+5 m, a crest width of 3 m and a slope of 1:3.
The foreshore begins to imaginary toe of the dike at an elevation of NAP and is 60 m wide. The toe of the trench is NAP-15 m and has a slope of 1:6. The dike section is 800 m long. The phreatic level is at NAP-2 m.
The soil profile is as follows:

  • from NAP+3.5 m to NAP+1 m: peat
  • from NAP+1 m to NAP-5 m: silty clay
  • from NAP-5 m to NAP-18 m: moderately to loosely compacted sand (Calais)
  • from NAP-18 m to NAP-30 m: densely compacted sand
Benchmarks results

The details of the calculation can be found in annex A of the report. The main results are given in the table below. The global check fails but the detailed check passes.

D-Flow Slide results

Results of benchmark 1-1 for the Global Check

Results

Benchmark

D-Flow Slide

Relative error %

Step 1

xxx

xxx

xxx

Group 2: Benchmarks from literature (approximate solution)

This section uses the results of a by-hand calculation performed for three profiles of the Spui dike project (see figure below):

...

bm2-1

Spui dike - hmp 63.9 (location Nieuw Beijerland)

(tick)

(tick)

(tick)

 

 

bm2-2

Spui dike - hmp 65.0 (between locations

...

Oud Beijerland and Nieuw Beijerland)

...

(error)(a)

(error)(a)

(error)(a)

 

 

bm2-3

Spui dike - hmp 67.8 (location Oud Beijerland)

(tick)

(tick)

(tick)

 

 

3

 

Image Removed

As the details of the calculation performed by hand are not available, those benchmarks are set in group 2.

2.1. Spui dike at km 63.9 (Nieuw Beijerland)

Description

The dike profile at hmp 63.9 is given in the figure below.
Image Removed

The soil profile and the material properties are given in the following table:

Level top layer (m NAP)

Material

Formation

Material sensitive to liquefaction

D r (%)

n (%)

n min (%)

n max (%)

Gamma unsat (kN/m 3)

Gamma sat (kN/m 3)

Gamma grains (kN/m 3)

D 50 (mm)

D 15 (mm)

Phi (deg)

c (kPa)

Eps voldm0

s max

s 2

k so (kN/m 2)

m

r

u

v

Grond level

Clay

Dunkirk / anthropogenic

No

 

 

 

 

18

18

26.5

 

 

25

0

 

 

 

 

 

 

 

 

-0.5

Clay/Sand

Dunkirk

No

 

 

 

 

18

18

26.5

 

 

25

0

 

 

 

 

 

 

 

 

-6.5

Peat

Holland

No

 

 

 

 

10

10

26.5

 

 

25

0

 

 

 

 

 

 

 

 

-7.5

Clay

Calais

No

 

 

 

 

18

18

26.5

 

 

25

0

 

 

 

 

 

 

 

 

-8.5

Sand clayey/Sand

Calais

Yes

35

45

36.5

49.5

19

19

26.5

0.160

0.080

25

0

0.000444

1.575

1.2

43.2

3

7

1.2

1

-9.5

Sand

Calais

Yes

30

45.6

36.5

49.5

20

20

26.5

0.160

0.080

30

0

0.000325

1.55

1.2

41.6

3

7

1.2

1

-12.5

Sand

Calais

Yes

25

46.3

36.5

49.5

20

20

26.5

0.160

0.080

30

0

0.000238

1.525

1.2

40

3

7

1.2

1

-18

Sand

Pleistocene

Yes

60

41.7

36.5

49.5

20

20

26.5

0.160

0.080

35

0

0.00212

1.7

1.2

51.2

3

7

1.2

1

The other calculation parameters used are:

  • Water level: 0.9 m
  • Required probability of failure of the dike: 1:4000
  • Percentage probability of failure byliquefaction: 1 %
  • Model factor: 1
  • Area ratio (c = A2/A1): 1.4
  • Standard deviation cot(gamma): 4.6
  • Considered dike length: 100 m
Benchmark results

According to the by hand calculation, both Global and Advanced checks fail.

D-FLOW SLIDE results

D-FLOW SLIDE results are in accordance with the results by hand as show in the table below.

Method

Benchmark

D-FLOW SLIDE

Relative error

Global check

Failed

Failed

(plus)

Detailed check

Failed

Failed

(plus)

2.2. Spui dike at km 65.0 (between Oud Beijerland and Nieuw Beijerland)

Description

The dike profile at km 65.0 is given in the figure below.
Image Removed
The soil profile and the material properties are the same as for the profile at km 63.9 (see paragraph 2.1).

Benchmark results

According to the by hand calculation, both Global and Advanced checks pass.

D-FLOW SLIDE results

D-FLOW SLIDE results are in accordance with the results by hand as show in the table below.

Method

Benchmark

D-FLOW SLIDE

Relative error

Global check

Passed

Passed

(plus)

Detailed check

Passed

Passed

(plus)

2.3. Spui dike at km 67.8 (oud Beijerland)

Description

The dike profile at km 67.8 is given in the figure below.
Image Removed
The soil profile and the material properties are given in the following table:

bm3-1a

Global check with traject: step 1 = No (Global passes)

(tick)

(tick)

(tick)

 

 

bm3-1b

Global check with traject: step 1 = Yes, step 3 = Yes (Global fails)

(tick)(tick)(error)(b)  

bm3-1c

Global check with traject: step 1 = Yes, step 3 = No, step 4 = No (Global passes)

(tick)

 

 

 

 

bm3-1d

Global check with traject: step 1 = Yes, step 2 = No, step 4 = Yes, step 5 = Yes (Global fails)

(tick)

(tick)

(tick)

 

 

bm3-1e

Global check with traject: step 1 = Yes, step 2 = No, step 4 = Yes, step 5 = No, step 6 = Yes (Global fails)

(tick)

(tick)

(tick)

 

 

bm3-1f

Global check with traject: step 1 = Yes, step 2 = No, step 4 = Yes, step 5 = No, step 6 = No, step 7 = Yes (Global fails)

(tick)

(tick)

(tick)

 

 

bm3-1gGlobal check with traject: step 1 = Yes, step 2 = No, step 4 = Yes, step 5 = No, step 6 = No, step 7 = No (Global passes)(tick)(tick)(tick)  

Level top layer (m NAP)

Material

Formation

Material sensitive to liquefaction

D r (%)

n (%)

n min (%)

n max (%)

Gamma unsat (kN/m 3)

Gamma sat (kN/m 3)

Gamma grains (kN/m 3)

D 50 (mm)

D 15 (mm)

Phi (deg)

c (kPa)

Eps voldm0

s max

s 2

k so (kN/m 2)

m

r

u

v

Grond level

Clay

Dunkirk / anthropogenic

No

 

 

 

 

18

18

26.5

 

 

25

0

5

 

 

 

 

 

 

 

 

 

 

bm5-0.5

Clay/Sand

Dunkirk

No

1a

Comparison with SLIQ2D-Windows - 1 saturated layer (Case LGZM1)

 

 

 

 (tick)

 

18

18

bm5-1b

Comparison with SLIQ2D-Windows - 1 saturated layer (Case LGZM2)

  26.5

 

 

25 (tick)

0

 

 

bm5-1c

Comparison with SLIQ2D-Windows - 1 saturated layer (Case LGZM3) 

 

 

 

 (tick)

 

bm5-4

Peat

Holland

1d

Comparison with SLIQ2D-Windows - 1 saturated layer (Case LGZM4)

  No

 

 

 (tick)

 

10

10

bm5-1e

Comparison with SLIQ2D-Windows - 1 saturated layer (Case SIMPLETA)

  26.5

 

 

25 (tick)

0

 

 

bm5-1f

Comparison with SLIQ2D-Windows - 1 saturated layer (Case LG1D5N5H) 

 

 

 

 (tick)

 

bm5-5

Clay

Calais

1g

Comparison with SLIQ2D-Windows - 1 saturated layer (Case HBPZBUI3) No

 

 

 

 

18

18

26.5 (tick)

 

 

25

0

 

bm5-2

Comparison with SLIQ2D-Windows - 2 layers partially saturated

 

(tick) 

 

 

 

 

 

 

-6

Sand

Calais

Yes

20

46.9

36.5

49.5

20

20

26.5

0.160

0.080

30

0

0.000174

1.5

1.2

38.4

3

7

1.2

1

-20

Sand

Pleistocene

Yes

60

41.7

36.5

49.5

20

20

26.5

0.160

0.080

35

0

0.00212

1.7

1.2

51.2

3

7

1.2

1

Benchmark results

According to the by hand calculation, the Global check fails and the Advanced check passes.

D-FLOW SLIDE results

D-FLOW SLIDE results differ from the results by hand for the Detailed check, as show in the table below.

Method

Benchmark

D-FLOW SLIDE

Relative error

Global check

Failed

Failed

(plus)

Detailed check

Passed

Failed

(minus)

Group 3: Benchmarks from spreadsheets

This section contains tests that are missing in the other groups, for example for the CUR-113 method.

Group 5: Benchmarks compared with other programs

Comparison with SLIQ2D (Dos and Windows) - One fully saturated layer with variable slope angle

Description

The benchmarks in this paragraph are intended to verify the advanced method by comparing D-FLOW SLIDE results with those from the older program SLIQ-2D, using both DOS and Windows versions of this program.
Because of the limitations in the DOS version of SLIQ2D, only one fully saturated layer is inputted. The geometry and the material properties for each cases are given in the table below.
The original slope angle is 1:1.25 and is set to "variable" so that the program will search (for each point) for the most unfavorable slope.

Benchmark name

Original file name (SLIQ2D-DOS)

Soil type

Slope height (m)

Slope angle

n (%)

n min (%)

n max (%)

Eps voldm0

m

r

s 2

s max

v

k so (kN/m 2)

u

Unit weight (kN/m 3)

D r (%)

bm5-1a

LGZM1

Sand clay

21.9

1:1.25

45.5

50

35

0.0025

1.7

7

1.28

1.7

1

50000

1

8.856

30

bm5-1b

LGZM2

Sand clay

21.9

1:1.25

45.5

50

35

0.0035

1.7

7

1.28

1.7

1

50000

1

8.856

30

bm5-1c

LGZM3

Sand clay

21.9

1:1.25

45.5

50

35

0.005

1.7

7

1.28

1.7

1

50000

1

8.856

30

bm5-1d

LGZM4

Sand clay

21.9

1:1.25

45.5

50

35

0.006

1.7

7

1.28

1.7

1

50000

1

8.856

30

bm5-1e

SIMPLETA

Sea sand

10

1:1.25

47.4

50

37

0.0092

3

7

1.23

1.35

1

39460

1.33

8.547

20

bm5-1f

LG1D5N5H

Sand clay

20.9

1:1.25

40.2

50

35

0.0022

1.7

7

1.18

1.4

1

85000

1

9.717

65.333

bm5-1g

HBPZBUI3

Sea sand

22

1:1.25

45.5

50

35

0.0054

2

7

1.25

1.4

1.25

50000

1

8.856

30

Benchmark results

...

Number of unstable points per slope angle

...

In SLIQ2D-Windows only the value of the tangent angle is available in the output. The other parameters (such as Lambda, Ko, the normal stress p and the deviatoric stress q) are not saved and can only be red on the screen, making the comparison difficult.
The automatical generation of the variable values of the slope angle used by SLIQ2D-DOS is not exactly the same as in SLIQ2D-Windows or D-FLOW SLIDE (which use 2 extra slopes compared to SLIQ2D-DOS: 1:1.4 and 1:1.3).

Benchmark 5-1a - Number of unstable points per slope angle, for the 3 programs

Slope angle
(Tangent Alpha)

Slope inclinaison
(1:xxx)

Nb. unstable points
SLIQ2D-DOS

Nb. unstable points
SLIQ2D-Windows

Nb. unstable points
D-FLOW SLIDE

Relative error between
D-FLOW SLIDE and SLIQ2D-DOS

Relative error between
D-FLOW SLIDE and SLIQ2D-Windows

( - )

( - )

( - )

( - )

( - )

( % )

( % )

<0.143

<7

0

0

0

0.00

0.00

0.143

7

1

1

1

0.00

0.00

0.154

6.5

2

2

2

0.00

0.00

0.167

6

3

3

3

0.00

0.00

0.182

5.5

7

7

7

0.00

0.00

0.2

5

15

15

15

0.00

0.00

0.211

4.75

6

6

6

0.00

0.00

0.222

4.5

15

15

15

0.00

0.00

0.235

4.25

17

17

17

0.00

0.00

0.25

4

19

19

19

0.00

0.00

0.267

3.75

23

23

23

0.00

0.00

0.286

3.5

25

25

25

0.00

0.00

0.308

3.25

24

24

24

0.00

0.00

0.333

3

40

40

40

0.00

0.00

0.364

2.75

24

24

24

0.00

0.00

0.4

2.5

26

26

26

0.00

0.00

0.444

2.25

23

23

23

0.00

0.00

0.5

2

14

14

14

0.00

0.00

0.571

1.75

9

9

9

0.00

0.00

0.667

1.5

3

3

3

0.00

0.00

0.714

1.4

0

1

1

 

0.00

...

Comparison between SLIQ2D-Windows and D-FLOW SLIDE

...

Results between SLIQ2D-Windows and D-FLOW SLIDE are exactly the same for the tangent angle, as show in the table below.

Table 5-2: Maximum relative error between SLIQ2D-Windows and D-FLOW SLIDE for the tangent angle

Case

Tan Alpha

bm5-1a

0.00 %

bm5-1b

0.00 %

bm5-1c

0.00 %

bm5-1d

0.00 %

bm5-1e

0.00 %

bm5-1f

0.00 %

bm5-1g

0.00 %

...

Comparison between SLIQ2D-DOS and D-FLOW SLIDE

...

Table 5-2: Maximum relative error between SLIQ2D-DOS and D-FLOW SLIDE for different parameters

Case

Tan Alpha

bm5-1a

0.00 %

bm5-1b

0.00 %

bm5-1c

0.00 %

bm5-1d

0.00 %

bm5-1e

0.00 %

bm5-1f

0.00 %

bm5-1g

0.00 %

bm5-3

Comparison with HMBreach - 1 layer

 

 

 

 

 (tick)

(a) The Global and Detailed checks with D-FLOW SLIDE fail whereas according to the by hand calculation they should succeed. However, by lack of information, the input value of several parameters (state parameter, critical retrogression length and migration velocity foreshore) is set arbitrary. Therefore, this comparison is not completely relevant, and it can't be concluded that the program gives incorrect results.

(b) The overall result is not correctly determined in D-Flow Slide if the foreshore is artificial: D-Flow Slide uses the Detailed check result but it should directly conclude that the Overall check fails.

 

...