20 Transformation Module (New Version)

What	nameofinstance.xml
Description	Configuration for the new version of the transformation module
schema location	http://fews.wldelft.nl/schemas/version1.0/transformationModule.xsd
Entry in ModuleDescriptors	<moduleDescriptor id="TransformationModule"> <description>Transformation Module</description> <className>nl.wldelft.fews.system.plugin.transformationmodule.TransformationModule</className> </moduleDescriptor>

Transformation Module Configuration (New Version)

The Transformation module is a general-purpose module that allows for generic transformation and manipulation of time series data. The module may be configured to provide for simple arithmetic manipulation, time interval transformation, shifting the series in time etc, as well as for applying specific hydro-meteorological transformations such as stage discharge relationships etc.

A new version of the FEWS Transformation Module is currently under construction. The new version is much more easy to configure than the old version. The new version uses a new schema for configuration, which is explained below (TODO). Currently only a few of the old transformations are available in the new transformation module. All of the old transformations will be made available in the new module, as well as several new transformations.

List of transformations

This list will be continuously updated to show the current status of the transformations available in the new transformation module.

Group	Transformation	Comment
adjust	stage	New feature
adjust	q	New feature
adjust	qUsingObservedInstantaneousDischarge	New feature
adjust	qUsingMeanDailyDischarge	New feature
adjust	tide	New feature
aggregation	instantaneous
aggregation	accumulative
aggregation	instantaneousToMean
aggregation	meanToMean
aggregation	seasonal
altitude	zero degree altitude
altitude	elevation lapse
conditional	simple	This allows definition of a single rule group (from the current lookup module)
conditional	multiple	This allows definition of multiple rule groups, combined with logical operators (from the current lookup module)
conversion	datum	This includes an enumeration option called direction local/global
disaggregation	instantaneous
disaggregation	accumulative
disaggregation	meanToInstantaneous
disaggregation	meanToMean
disaggregation	weights
discharge stage	power	Uses the same coefficient sets as stage discharge power.
discharge stage	table	Uses the same coefficient sets (for table entries) as stage discharge table.
evaporation	penmanNorthEast	should be renamed to the specific type of penman equation
filter	moving window	contains options average/maximum/minimum
filter	block	contains options average/maximum/minimum
interpolation serial	linear
interpolation serial	block
interpolation serial	default
interpolation serial	extrapolate constant	Extrapolates by filling the gap between the start of the input time series and the first data value or by filling the gap between the last data value and the end of the input time series or by filling both of these gaps.
interpolation serial	extrapolate base	Extrapolates by adding future values that start at the last data value and linearly approach a given base value.
interpolation serial	extrapolate exponential	New feature: Extrapolates by adding future values that start at the last data value and exponentially approach a given base value.
interpolation serial	extrapolate window	Should contain option on how to extrapolate: persitence/trend/mean trend/persistence trend/accumulation/moving average.
interpolation spatial	average
interpolation spatial	closest distance
interpolation spatial	inverse distance
interpolation spatial	sum
interpolation spatial	weighted	This is the old weighted average approach used for rainfall.
interpolation spatial	weightedTimesOutputArea
interpolation spatial	bilinear
interpolation spatial	triangulation	Used to be called Renka & Cline
interpolation spatial	kriging
lookup	simple	from the current lookup module
lookup	multi-dimensional	from the current lookup module
merge	simple	merge data hierarchy
merge	toggle
merge	mean	New feature?
merge	synoptic	This needs a true false element to indicate if the smaller or the larger quantities prevail (new feature)
merge	forecasts	equivalent to sampleForecasts - this is a new feature.
moisture	soil moisture	unit test not yet available
moisture	optical depth	unit test not yet available
precipitation	rainfallEvent	writes median value of event (Enumeration of 2 to indicate where to write the data)
precipitation	rainfallEventLength	writes length of event (Enumeration of 2 to indicate where to write the data)
precipitation	tipping bucket
profile	timeSeries	Converts a profile to a time series.
profile	harmonics	Converts an input profile consisting of harmonic components (a Fourier series) to a time series.
review	stage	New feature
review	tidal balance	New feature
sample	historical
sample	forecasts	This samples the first n values from a forecast - somewhat similar to the merge-forecast function
sample	equidistant	converts an equidistant series to another equidistant series or to a non-equidistant series
sample	non-equidistant	converts a non-equidistant series to another non-equidistant series or to an equidistant series
selection	peaks
selection	independent peaks	previously called selectWithinHorizontalgaps
selection	lows
selection	independent lows
selection	maximum
selection	minimum
stage discharge	power	Uses the same coefficient sets as discharge stage power.
stage discharge	power backwater	Backwater correction (new feature).
stage discharge	power unsteady	Unsteady flow (new feature).
stage discharge	parabolic	New feature
stage discharge	table	Uses the same coefficient sets (for table entries) as discharge stage table.
statistical	summary	Summary statistics act on the ensemble. Options include max/min/mean/quartile/(percentile + value)
statistical	window	These are statisicts over a time window - e.g. to define monthly mean temp - may be seasonal
statistical	climatology	These are the seasonal stats - used to define e.g. mean monthly temp
structure	general weir fixed height
structure	general weir variable height
structure	flatV weir	Enumeration to indicate simple or crest tapping
structure	flatV weir backwater	New feature
structure	crump weir	Enumeration to indicate simple or crest tapping
structure	crump weir backwater	New feature
structure	pumpFixedDischarge
structure	pumpHeadDischargeTable
structure	pumpSpeedDischargeTable
structure	pumpSpeedHeadDischargeTable
time shift	constant
time shift	variable
time shift	length
user	simple	free format expression
user	periodic	This can be used for periodic output - the range can be defined, e.g 0-1, 0-360 or 0-100

Configuration details (TODO)

When available as configuration on the file system, the name of an XML file for configuring an instance of the transformation module called for example TransformHBV_Inputs may be:

TransformHBV_Inputs 1.00 default.xml

TransformHBV_Inputs	File name for the TransformHBV_Inputs configuration.
1.00	Version number
default	Flag to indicate the version is the default configuration (otherwise omitted).

The configuration for the transformation module consists of two parts: transformation configuration files in the Config/ModuleConfigFiles directory and coefficient set configuration files in the Config/CoefficientSetsFiles directory.

In a transformation configuration file one or more transformations can be configured. See List of transformations for the available types of transformation. Some transformations require coefficient sets in which given coefficients are defined. For a given transformation that requires a coefficient set there are different ways of defining the coefficient set in the configuration. One way is to specify an embedded coefficient set in the transformation configuration itself. Another way is to put a reference in the transformation configuration. This reference consists of the name of a separate coefficient set configuration file and the id of a coefficient set in that file.

Both the transformations and coefficient sets can be configured to be time dependent. This can be used for instance to define a given coefficient value to be 3 from 1 January 2008 to 1 January 2009, and to be 4 from 1 January 2009 onwards. This can be done by defining multiple periodCoefficientSets, each one with a different period, as in the following xml example.

<periodCoefficientSet>
    <period>
        <startDate>2008-01-01T00:00:00Z</startDate>
        <endDate>2009-01-01T00:00:00Z</endDate>
    </period>
    <structure>
        <pumpFixedDischarge>
            <discharge>3</discharge>
        </pumpFixedDischarge>
    </structure>
</periodCoefficientSet>
<periodCoefficientSet>
    <period>
        <validAfterDate>2009-01-01T00:00:00Z</validAfterDate>
    </period>
    <structure>
        <pumpFixedDischarge>
            <discharge>4</discharge>
        </pumpFixedDischarge>
    </structure>
</periodCoefficientSet>

The Z in 2008-01-01T00:00:00Z in the above example means that the time is specified in GMT. To specify the time in a different time zone use e.g. 2008-06-20T11:33:00+05:00 for time zone GMT+05:00. Note: 2008-06-20T11:33:00+05:00 is physically the same time as 2008-06-20T06:33:00Z.

Page tree

20 Transformation Module (New Version)

Transformation Module Configuration (New Version)

List of transformations

Configuration details (TODO)