...
Copying comments, flags and flag sources from input to output
<TODO>
Preventing previously calculated values to be overwritten with missings
Delft-FEWS processes data in moving windows compared to the timezero of the workflow. These workflows can be run several times per day. In certain conditions this could lead a transformation to calculate a missing value for a datetime for which earlier a correct value was calculated. This can lead to warnings such as:
Existing value overwritten by missing
Suspicious write action. Long time series written with only changes at the start and at the end. If this happens often this will explode the database.
This mechanism is illustrated with the image below, which shows a water level for which an average per day is calculated. The box shows the moving window (relativeViewPeriod). The image shows the daily values (red dots) calculated in the first run.
In a later run (illustrated below), there are not sufficient values to calculate a daily value for 24 December. So this run will return a missing value for 24 December (while calculating a new value for 27 December.
If these transformations would directly write to the same output timeserie (in this case with timeseriesType “external historical”), the later run would cause the average water level for December 24 to be overwritten with a missing value. The log would include warnings about this!
To prevent this, the output timeserie should be of timeserieType “temporary”, and next be merged (merge / simple transformation) with the final timeserie. This way, missing values will not overwrite previously calculated values.
Run transformations for a set of selected locations
...