Amalgamate Import Data module
Deprecated
Please note that this module is replaced by 24 ImportAmalgamate
Import data is stored in DELFT-FEWS for a default length of time before it is removed from the database by the rolling barrel. To keep track on how long data is to be kept in the system, an expiry time is set on each piece of data imported. When the expiry time has passed the data is removed. To keep track of when data became available to the system a creation time is also administered. This time is used when identifying what data was made available to DELFT-FEWS. This allows tracing the data available at the time of the forecast. A disadvantage of tracking when data was imported is that each piece of data is stored individually in a database record (as a BLOB). The amount of these records with small BLOB's will have implications on the size of the database.
If observed data is to be kept in the system longer than forecast data without having severe implications on the size of the database, the amalgamate module can be configured to amalgamate multiple small lengths of data to a single BLOB in a single record. These can be stored with a much longer expiry time than the default. On a scheduled system this module can be run on a daily basis, amalgamating for example import data that is a number of weeks old and is about to expire into single blocks with a much later expiry time.
When available as configuration on the file system, the name of the XML file for configuring an instance of the amalgamate module called for example Amalgamate_Import may be:
Amalgamate_Import 1.00 default.xml
Amalgamate_Import | File name for the Amalgamate_Import configuration. |
1.00 | Version number |
default | Flag to indicate the version is the default configuration (otherwise omitted). |
Figure 138 Elements of the Amalgamate Module configuration
task
Root element for definition of an amalgamate task. Multiple entries may exist.
maximumCombinedBlobLength
Definition of the maximum length of the amalgamated BLOB.
Attributes;
- unit unit of time (enumeration of: second, minute, hour, day, week)
- multiplier defines the number of units given above in a time step.**
- divider same function as the multiplier, but defines fraction of units in time step.**
timeSeriesSet
Time series set to amalgamate. The input and output time series sets are identical. Set a new expiry time in the time series set to ensure it is kept in the database for the required period.
