Install Python(x,y) to install a full set of python tools including easy_install. Easy_install can be used to install the Mako templates. Do not use Python 3.x or up!

info = struct( ...
    'revision', 1234, ...                                                       // revision number of current binary
    'binary',   'trunk', ...                                                    // name of current binary
    'type',     'custom',                                                       // mode of current run (custom or default)
    'test',     'CarrierGreenspan',                                             // name of current test
    'run',      'default' ...                                                   // name of current run
);

dirs = struct( ...
    'output',  'C:\SKILLBED\analysis\trunk\custom\CarrierGreenspan\default', ... // Path where output should be stored
    'data',    '', ...                                      

{include:ContentHeader}

{tip}Within the Deltares network, a public testbed client is available through the following URL: http://d00753:5000/interface/index{tip}

h1. Contents

{toc}

h1. Introduction

The XBeach Testbed is an automated testing environment for the [XBeach model|XBEACH:Home]. The testbed monitors the quality of the model developments. In order to do so, a collection of model setups is collected in the [Subversion repository|https://repos.deltares.nl/repos/XBeach/testbed2011/] of Deltares together with the latest model source code, possibly arranged in different branches. When the testbed is run, both the source code revisions as well as the model setups are downloaded from the repository, binaries are built and tests are run.

Tests can be run in different configurations, with different paramerter files and different versions of the XBeach executable. After all tests are finished, a collection of Matlab scripts performs the post-processing of the model results. The use of Matlab scripts makes management of the post-processing easy accessible for everyone. The scripts generate figures, tables and statistical performance scores. The result of the post-processing is then captured in several PDF reports. The reports are automatically e-mailed to the developers of XBeach.

The XBeach Testbed runs on a daily basis and contains over 50 model setups and 4 different executables. The testbed is written in Python and can be run over different computers using XML-RPC. The testbed is therfore splitted into a server and client part, which are both automatically updated from the subversion repository as well.

h1. Installation

h2. Requirements

The XBeach Testbed requires the following software to be installed:

{table-plus:width=800|enhableHighlighting=true}
|| Name || URL || Remarks ||
| Python 2.6 | [http://www.python.org/] | |
| PySVN 1.7 | [http://pysvn.tigris.org/] | |
| Pylons 1.0 | [http://pylonshq.com/] | Client only |
| Mako 0.4.0 | [http://www.makotemplates.org/] | Client only |
| Miktex 2.8 | [http://miktex.org/] | Windows client only |
| pdflatex | | Linux client only |
| Matlab | [http://www.mathworks.nl/products/matlab/] | Client only |
| MPICH2 | [http://www.mcs.anl.gov/research/projects/mpich2/] | Server only |
| Microsoft Visual Studio 2008 | [http://www.microsoft.com/visualstudio/en-us/products/professional/default.mspx] | Windows server only |
| Intel Fortran Compiler | [http://software.intel.com/en-us/intel-compilers/] | Windows server only |
| GNU Fortran Compiler | [http://gcc.gnu.org/fortran/] | Linux server only |
{table-plus}

The XBeach Testbed requires the following environment variables to be set:

{table-plus:width=800|enhableHighlighting=true}
|| Name || Value || Remarks ||
| PYTHON_PATH | Path to python executable | |
| PDFLATEX_PATH | Path to pdflatex executable | Client only |
| MATLAB_PATH | Path to Matlab executable | Client only |
| MPIEXEC_PATH | Path to mpiexec executable | Server only |
| VS90COMNTOOLS | Microsoft Visual Studio 2008 installation directory | Windows server only |
| IFORT_COMPILER11 | Intel Fortran Compiler installation directory | Windows server only |
| INTEL_LICENSE_FILE | Path to Intel Fortran Compiler license file | Windows server only |
| GFORTRAN_PATH | Path to gfortran executable | Linux server only |
{table-plus}

h2. Download

The XBeach Testbed can be installed by running the following python script: [https://repos.deltares.nl/repos/XBeach/testbed2011/tools/python/install.py]. The script downloads the testbed from the subversion repository.

h2. Run

The server can be started by running the tools/python/testbed/server.py script.

The client can be started by running the tools/python/testbed/client.py script.

The tools/testbed.cfg file on the client machine should be adapted to hold the hostnames and ports of the servers to be used. Adding -h to the script call will display help information.

A public client is available via the webpage [http://d00753:5000/interface/index]. Keep in mind that this client is also a server, preforming too many requests will slow down it's performance. Consider installing a dedicated client by that time.

h1. Overview

!testbed.png|width=800!

h1. Adding a test

* Create a checkout of the testbed repository: [https://repos.deltares.nl/repos/XBeach/testbed2011/]
* Create a test directory with the name of the test in the _input_ directory
* Create a working model in the test directory
* Create a Matlab script that analyses the model output in the test directory

The Matlab script takes two arguments: _info_ and _dirs_. Both arguments are structs that have the following contents:

{code}
info = struct( ...
    'revision', 1234, ...                    // Path where static data can                              // revision number of current binarybe found
    'binarynetwork',   'trunk',P:\STORAGE\DATA\CarrierGreenspan' ...                           // Network storage that can be used
);

[general]                   // name of current binary
    'type',     'custom',                                       // SECTION: General information
enable          = 1  // mode of current run (custom or default)
    'test',     'CarrierGreenspan',                                             // nameEnable oftest currentin test
    'run',skillbed
runs      'default' ...     = default,  stationary                                           // nameDefinition of currenttest run
);

dirs = struct( ...
    'output',  'C:\TESTBED\analysis\trunk\custom\CarrierGreenspan\default', ... // Path where output should be stored
    'data',    '', ...runs
responsible     = Bas Hoonhout <bas.hoonhout@deltares.nl>                       // Name and e-mail of maintainer

[run_default]                                              // Path where static data can be found
    'network', 'P:\STORAGE\DATA\CarrierGreenspan' ...        // SECTION: Configuration of test run "default"
enable          = 1  // Network storage that can be used
);
{code}

* Create a configuration file called _test.cfg_ in the test directory

The configuration file has the following or similar contents:

{code}
[general]                                   // Enable run in skillbed
params          = params_original.txt                     // SECTION: General information
enable          = 1        // Parameter file to use in run
binaries        = all                                      // Enable test in testbed
runs            = default, stationary   // Binaries that should use this run
types           = all                        // Definition of test runs
responsible     = Bas Hoonhout <bas.hoonhout@deltares.nl>                       // NameModes andin e-mailwhich ofthis maintainer

[run_default] should be used (custom/default/all)
platforms       = all                                                           // SECTION:Platforms Configurationon ofwhich testthis run "default"
enable should be used (win32/unix/all)
nodes           = 1                                                             // EnableNumber runof innodes testbed
paramsthat should be used during this run (> 1 for = params_original.txtMPI)
analysis        = analyze_this                                   // Parameter file to use in run
binaries        = all// Matlab function to be used for the postprocessing of the results

[run_stationary]                                                // Binaries that should use this run
types          // =SECTION: allConfiguration of test run "stationary", see above
enable          = 1
params          = params_stationary.txt
binaries        = trunk, speedup
types           = custom
platforms       = win32
nodes   // Modes in which this run should be used (custom/default/all)
platforms= 1
analysis        = allanalyze_this

[categories]                                                           // Platforms on which this run should be used (win32/unix/all)
nodes        // SECTION: Category definitions used for classification of test
configuration   = 1 1D                                                            // NumberGrid of nodes that should be used during this run (> 1 for MPI)
analysis        = analyze_thisconfiguration (1D/2D)
waves           = stationary                                                    // MatlabWave function to be used for the postprocessing of the results

[run_stationary]configuration (stationary/non-stationary)
waterlevels     = constant                                                      // Water level configuration (constant/varying)
morphology      = no    // SECTION: Configuration of test run "stationary", see above
enable          = 1
params          = params_stationary.txt
binaries        = trunk, speedup
types           = custom
platforms    // Morphology configuration (yes/no)
fractions = win32
nodes     = 1     = 1
analysis        = analyze_this

[categories]                                              // Number of sediment fractions
hardlayers      = no           // SECTION: Category definitions used for classification of test
configuration   = 1D                                                            // Hard Gridlayer configuration (1Dyes/2Dno)
wavesgroundwaterflow = no         = stationary                                                  // Ground //water Waveflow configuration (stationaryyes/non-stationary)
waterlevels     = constant                                                      // Water level configuration (constant/varying)
morphology      = no                                                            // Morphology configuration (yes/no)
fractions       = 1                                                             // Number of sediment fractions
hardlayers      = no                                                            // Hard layer configuration (yes/no)
groundwaterflow = no                                                            // Ground water flow configuration (yes/no)
{code}

* Create a TEX file in the _latex/___tests_ directory with a descriptive text for the added test and references to the postprocessing output

Images are referenced to as shown below. The first argument is the path containing the test and run name of the output. The second argument is the file name to be referenced. The third argument is an optional caption of the image.

{code}
${tools.image('CarrierGreenspan/default/', 'fig1.eps', 'Model output')}
{code}

Tables are referenced in a similar manner:

{code}
${tools.table('Assateague_Island/profA/', 'conditions.tex', 'Hydrodynamic boundary conditions XBeach simulations')}
{code}

* Create one or more references to the test description TEX file in one or more documents

References to descriptive texts are made as follows:

{code}
<%include file="carriergreenspan.tex"/>
{code}

* Commit the additions to the Subversion repositoryno)

${tools.image('CarrierGreenspan/default/', 'fig1.eps', 'Model output')}

${tools.table('Assateague_Island/profA/', 'conditions.tex', 'Hydrodynamic boundary conditions XBeach simulations')}

<%include file="carriergreenspan.tex"/>

>> python client.py --test-report trunk_custom --skip-clean

d = xb_read_dims;

xbi = xb_read_input;

xbo = xb_read_output(pwd, 'vars', { ... }, ...);
xbo = xb_peel(xb_read_output(pwd, 'vars', { ... }, ...));

s = xb_stagger(d.x,d.y);

path = fullfile( ... );
k = disper(w,h);
S = xb_spectrum(zs);
[r2 sci relbias bss] = xb_skill( ... );

xb_plot_profiles(xbo, ...);
xb_plot_profiles(xbo, 'measured', [x y], 'durosta', [x y], ...);

xb_plot_wavetrans(xbo, ...);
xb_plot_wavetrans(xbo, 'Hrms_lf', zs, 'Hrms_hf', H);

xb_write_skilltable(xbo, [x y]);