Page History

...

Workflows may produce inefficient blobs that only span a few time steps. These blobs will be amalgamated. After the amalgamate is finished, the original import runs with all its imported external data is scheduled for deletion. Large grids, external forecasts and samples should be imported in a separate workflow, that are not handled through this module.

...

Import runs younger than the specified age are skipped. After the amalgamate has run it is no longer possible to create an archive with the exact original data available during the run.

amalgamateOrphans

For systems that did not have an amalgamate running it maybe required to amalgamate the complete database. Of course this should only be done once.
It is possible do this with the amalgamateOrphans=true. As this task may take too long (regular FSS tasks have a time out of 3 hours) this should be handled with care.
In that case it maybe useful to have the task run at an FSS that has no localdatastore, but has direct access to the central database. Only in that particular case the amalgamate runs (hard coded in the software) for only 1 hour. Then then task is aborted and the remaining imports are not handled in that taskrun anymore. This is done on purpose, as it is now possible to schedule this task with an interval of e.g. 3 hours (making it possible to the MC to do all the required cleaning stuff like rollingBarrel and markedRecordManager). In a few hours (to even days or in some rare situations evens weeks) all the taksruns are handled and the complete database has been amalgamated. Then the scheduled task can be stopped.

Example

<moduleDescriptor id="ImportAmalgamate">
  <className>nl.wldelft.fews.system.plugin.amalgamate.ImportAmalgamate</className>
</moduleDescriptor>

...