Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 4.0
Include Page
ContentHeader
ContentHeader

Contents

Table of Contents

Introduction

The XBeach Skillbed is an automated testing environment for the XBeach model. The skillbed monitors the quality of the model developments. In order to do so, a collection of model setups is collected in the Subversion repository of Deltares together with the latest model source code, possibly arranged in different branches. When the skillbed is run, both the source code revisions as well as the model setups are downloaded from the repository, binaries are built and tests are run.

Tests can be run in different configurations, with different parameter files and different versions of the XBeach executable. After all tests are finished, a collection of Matlab scripts performs the post-processing of the model results. The use of Matlab scripts makes management of the post-processing easy accessible for everyone. The scripts generate figures, tables and statistical performance scores. The result of the post-processing is then captured in several PDF reports. The reports are automatically e-mailed to the developers of XBeach.

The XBeach Skillbed runs on a daily basis and contains over 50 model setups and 4 different executables. The skillbed is written in Python and can be run over different computers using XML-RPC. The skillbed is therfore splitted into a server and client part, which are both automatically updated from the subversion repository as well.

Installation

Requirements

The XBeach Skillbed requires the following software to be installed:

Info
titleTips and Tricks

Install Python(x,y) to install a full set of python tools including easy_install. Easy_install can be used to install the Mako templates. Do not use Python 3.x or up!

Wiki Markup
{

...

table-plus:width=800|enhableHighlighting=true}



|| Name || URL || Remarks ||
| Python 2.6/2.7 | [http://www.python.org/] | |
| PySVN 1.7 | [http://pysvn.tigris.org/] | |
| Pylons 1.0 | [http://pylonshq.com/] | Client only |
| Mako 0.4.0 | [http://www.makotemplates.org/] | Client only |
| Miktex 2.8 | [http://miktex.org/] | Windows client only |
| pdflatex | | Linux client only |
| Matlab | [http://www.mathworks.nl/products/matlab/] | Client only |
| MPICH2 | [http://www.mcs.anl.gov/research/projects/mpich2/] | Server only |
| Microsoft Visual Studio 2008 | [http://www.microsoft.com/visualstudio/en-us/products/professional/default.mspx] | Windows server only |
| Intel Fortran Compiler | [http://software.intel.com/en-us/intel-compilers/] | Windows server only |
| GNU Fortran Compiler | [http://gcc.gnu.org/fortran/] | Linux server only |
{table-plus}

...

The

...

XBeach

...

Skillbed

...

requires

...

the

...

following

...

environment

...

variables

...

to

...

be

...

set:

...

Wiki Markup
{table-plus:width=800|enhableHighlighting=true}



|| Name || Value || Remarks ||
| PYTHON_PATH | Path to python executable | |
| PDFLATEX_PATH | Path to pdflatex executable | Client only |
| MATLAB_PATH | Path to Matlab executable | Client only |
| MPIEXEC_PATH | Path to mpiexec executable | Server only |
| VS90COMNTOOLS | Microsoft Visual Studio 2008 installation directory | Windows server only |
| IFORT_COMPILER11 | Intel Fortran Compiler installation directory | Windows server only |
| INTEL_LICENSE_FILE | Path to Intel Fortran Compiler license file | Windows server only |
| GFORTRAN_PATH | Path to gfortran executable | Linux server only |
{table-plus}

...

These

...

environment

...

variables

...

can

...

also

...

be

...

set

...

through

...

the

...

skillbed.inst

...

file

...

in

...

the

...

tools

...

directory.

...

Download

The skillbed can be installed by running the following python script: https://repos.deltares.nl/repos/openearthtools/trunk/python/applications/skillbed/tools/python/install.py

...

.

...

The

...

script

...

downloads

...

the

...

skillbed

...

from

...

the

...

OpenEarthTools

...

Subversion

...

repository.

...

Run

The server can be started by running the tools/python/skillbed/server.py

...

script.

...

The

...

client

...

can

...

be

...

started

...

by

...

running

...

the

...

tools/python/skillbed/client.py

...

script.

...

The

...

tools/skillbed.cfg

...

file

...

on

...

the

...

client

...

machine

...

should

...

be

...

adapted

...

to

...

hold

...

the

...

hostnames

...

and

...

ports

...

of

...

the

...

servers

...

to

...

be

...

used.

...

Adding

...

-h

...

to

...

the

...

script

...

call

...

will

...

display

...

help information.

Overview

Image Added

Adding a test

...

  • Create

...

  • a

...

  • test

...

  • directory

...

  • with

...

  • the

...

  • name

...

  • of

...

  • the

...

  • test

...

  • in

...

  • the

...

  • input

...

  • directory

...

  • Create

...

  • a

...

  • working

...

  • model

...

  • in

...

  • the

...

  • test

...

  • directory

...

It

...

is

...

possible

...

to

...

group

...

related

...

tests

...

in

...

a

...

single

...

directory.

...

Simply

...

use

...

subdirectories

...

for

...

this

...

purpose.

...

In

...

de

...

test

...

configuration

...

file,

...

the

...

subdirectories

...

should

...

be

...

mentioned

...

(see

...

further

...

down).

...

  • Create

...

  • a

...

  • Matlab

...

  • script

...

  • that

...

  • analyses

...

  • the

...

  • model

...

  • output

...

  • in

...

  • the

...

  • test

...

  • directory

...

The

...

Matlab

...

script

...

takes

...

two

...

arguments:

...

info

...

and

...

dirs

...

.

...

Both

...

arguments

...

are

...

structs

...

that

...

have

...

the

...

following

...

contents:

{
Code Block
}
info = struct( ...
    'revision', 1234, ...                                                       // revision number of current binary
    'binary',   'trunk', ...                                                    // name of current binary
    'type',     'custom',                                                       // mode of current run (custom or default)
    'test',     'CarrierGreenspan',                                             // name of current test
    'run',      'default' ...                                                   // name of current run
);

dirs = struct( ...
    'output',  'C:\SKILLBED\analysis\trunk\custom\CarrierGreenspan\default', ... // Path where output should be stored
    'data',    '', ...                                                          // Path where static data can be found
    'network', 'P:\STORAGE\DATA\CarrierGreenspan' ...                           // Network storage that can be used
);
  • Create a configuration file called .config in the test directory

The configuration file has the following or similar contents:

Code Block
{code}

* Create a configuration file called _.config_ in the test directory

The configuration file has the following or similar contents:

{code}
[general]                                                                       // SECTION: General information
enable          = 1                                                             // Enable test in skillbed
runs            = default, stationary                                           // Definition of test runs
responsible     = Bas Hoonhout <bas.hoonhout@deltares.nl>                       // Name and e-mail of maintainer

[run_default]                                                                   // SECTION: Configuration of test run "default"
enable          = 1                                                             // Enable run in skillbed
params          = params_original.txt                                           // Parameter file to use in run
binaries        = all                                                           // Binaries that should use this run
types           = all                                                           // Modes in which this run should be used (custom/default/all)
platforms       = all                                                           // Platforms on which this run should be used (win32/unix/all)
nodes           = 1                                                             // Number of nodes that should be used during this run (> 1 for MPI)
analysis        = analyze_this                                                  // Matlab function to be used for the postprocessing of the results

[run_stationary]                                                                // SECTION: Configuration of test run "stationary", see above
enable          = 1
params          = params_stationary.txt
binaries        = trunk, speedup
types           = custom
platforms       = win32
nodes           = 1
analysis        = analyze_this

[categories]                                                                    // SECTION: Category definitions used for classification of test
configuration   = 1D                                                            // Grid configuration (1D/2D)
waves           = stationary                                                    // Wave configuration (stationary/non-stationary)
waterlevels     = constant                                                      // Water level configuration (constant/varying)
morphology      = no                                                            // Morphology configuration (yes/no)
fractions       = 1                                                             // Number of sediment fractions
hardlayers      = no                                                            // Hard layer configuration (yes/no)
groundwaterflow = no                                                            // Ground water flow configuration (yes/no)
{code}

If

...

you

...

use

...

subdirectories

...

to

...

group

...

related

...

tests,

...

the

...

configuration

...

file(s)

...

can

...

be

...

stored

...

in

...

the

...

root

...

directory

...

or

...

in

...

any

...

subdirectory.

...

The

...

location

...

of

...

the

...

configuration

...

file

...

determines

...

where

...

the

...

test

...

definition

...

end

...

and

...

the

...

run

...

definition

...

starts.

...

In

...

case

...

the

...

configuration

...

file

...

is

...

located

...

in

...

one

...

or

...

more

...

subdirectories,

...

the

...

names

...

of

...

the

...

subdirectories

...

are

...

concatenated

...

to

...

the

...

test

...

name,

...

separated

...

by

...

an

...

underscore

...

(_).

...

Any

...

other

...

subdirectories

...

should

...

be

...

mentioned

...

in

...

the

...

run

...

definition

...

of

...

the

...

configuration

...

file

...

itself.

...

In

...

the

...

path

...

of

...

directories

...

to

...

a

...

configuration

...

file,

...

no

...

other

...

configuration

...

files

...

may

...

be

...

found,

...

otherwise

...

this

...

configuration

...

file

...

is

...

used

...

instead.

...

A

...

simple

...

script

...

to

...

generate

...

a

...

default

...

configuration

...

file

...

can

...

be

...

found

...

in

...

the

...

root

...

of

...

the

...

input

...

directory.

...

It

...

searches

...

for

...

subdirectories

...

to

...

be

...

interpreted

...

as

...

runs

...

or

...

otherwise

...

generates

...

a

...

single

...

run

...

for

...

the

...

root

...

directory.

...

Please

...

at

...

least

...

alter

...

responsibilities

...

and

...

test

...

characteristics

...

manually.

...

  • Create

...

  • a

...

  • TEX

...

  • file

...

  • in

...

  • the

...

  • latex/_

...

  • tests

...

  • directory

...

  • with

...

  • a

...

  • descriptive

...

  • text

...

  • for

...

  • the

...

  • added

...

  • test

...

  • and

...

  • references

...

  • to

...

  • the

...

  • postprocessing

...

  • output

...

Images

...

are

...

referenced

...

to

...

as

...

shown

...

below.

...

The

...

first

...

argument

...

is

...

the

...

path

...

containing

...

the

...

test

...

and

...

run

...

name

...

of

...

the

...

output.

...

The

...

second

...

argument

...

is

...

the

...

file

...

name

...

to

...

be

...

referenced.

...

The

...

third

...

argument

...

is

...

an

...

optional

...

caption

...

of

...

the

...

image.

{
Code Block
}
${tools.image('CarrierGreenspan/default/', 'fig1.eps', 'Model output')}
{code}

Tables

...

are

...

referenced

...

in

...

a

...

similar

...

manner:

{
Code Block
}
${tools.table('Assateague_Island/profA/', 'conditions.tex', 'Hydrodynamic boundary conditions XBeach simulations')}
{code}

* Create one or more references to the test
  • Create one or more references to the test description TEX file in one or more documents

References to descriptive texts are made as follows:

Code Block
 description TEX file in one or more documents

References to descriptive texts are made as follows:

{code}
<%include file="carriergreenspan.tex"/>
{code}

* Commit the additions to the Subversion repository

h3. Troubleshoot

Sometimes the resulting report is not what you expected. The following information might be relevant to get the report just as you want.

* References turn into question marks (??)

The reference is not valid. Please make sure you use the full reference identifier: <test name>_<run name>_<filename> _without_ extension for generated files. For static files that are not generated by the analysis scripts, you simply use the filename _without_ extension.

* My formulas are not aligned as they should

Often formulas are aligned using table-like alignment commands. These end in line endings (two back-slashes). This line ending is also the command for Python to concatenate lines to a single line. In order to use the former functionality only, simply add a space after the line ending command.

* I have to wait a day before I see the results of the changes I made in the report

Using the skillbed client, you can test the report generation. Since no real data is available, the test report will be crowded with giraffes. Nevertheless, it enables you to check whether you texts and formulas come out well and your references are properly configured. From the _tools/python/skillbed_ directory, you need to run the following command to test the generation of the trunk_custom report:

{
  • Commit the additions to the Subversion repository

Troubleshoot

Sometimes the resulting report is not what you expected. The following information might be relevant to get the report just as you want.

  • References turn into question marks (??)

The reference is not valid. Please make sure you use the full reference identifier: <test name><run name><filename> without extension for generated files. For static files that are not generated by the analysis scripts, you simply use the filename without extension.

  • My formulas are not aligned as they should

Often formulas are aligned using table-like alignment commands. These end in line endings (two back-slashes). This line ending is also the command for Python to concatenate lines to a single line. In order to use the former functionality only, simply add a space after the line ending command.

  • I have to wait a day before I see the results of the changes I made in the report

Using the skillbed client, you can test the report generation. Since no real data is available, the test report will be crowded with giraffes. Nevertheless, it enables you to check whether you texts and formulas come out well and your references are properly configured. From the tools/python/skillbed directory, you need to run the following command to test the generation of the trunk_custom report:

Code Block
code}
>> python client.py --test-report trunk_custom --skip-clean
{code}

You

...

will

...

find

...

the

...

result

...

in

...

the

...

run

...

directory.

...

Coding

...

guidelines

...

The

...

Matlab

...

XBeach

...

Toolbox

...

is

...

available

...

within

...

the

...

skillbed

...

environment.

...

The

...

following

...

commands

...

from

...

this

...

toolbox

...

are

...

preferred

...

to

...

use,

...

including

...

the

...

names

...

of

...

the

...

variables.

...

Please

...

see

...

the

...

corresponding

...

documentation

...

for

...

the

...

exact

...

syntax:

...

  • Read

...

  • model

...

  • dimensions
{
Code Block
}d = xb_read_dims;{code}

* Read model
  • Read model input
Code Block
 input

{code}xbi = xb_read_input;
  • Read model output
Code Block
{code}

* Read model output

{code}
xbo = xb_read_output(pwd, 'vars', { ... }, ...);
xbo = xb_peel(xb_read_output(pwd, 'vars', { ... }, ...));
{code}

* Generate grid information

{code}
  • Generate grid information
Code Block
s = xb_stagger(d.x,d.y);{code}

* Miscellaneous functions

  • Miscellaneous functions
Code Block
{code}
path = fullfile( ... );
k = disper(w,h);
S = xb_spectrum(zs);
[r2 sci relbias bss] = xb_skill( ... );
  • Create figures and tables
Code Block
{code}

* Create figures and tables

{code}
xb_plot_profiles(xbo, ...);
xb_plot_profiles(xbo, 'measured', [x y], 'durosta', [x y], ...);

xb_plot_wavetrans(xbo, ...);
xb_plot_wavetrans(xbo, 'Hrms_lf', zs, 'Hrms_hf', H);

xb_write_skilltable(xbo, [x y]);
{code}