...
| Warning | ||
|---|---|---|
| ||
When multiple Secondary Validation checks for a that could change data for the same time series are necessary within 1 workflow, all checks should be present within 1 and the same module config file. This is because at the beginning of the Secondary Validation module all previous secondary validations are being undone for consistent behaviour and for significant database performance reasons. |
| Warning | ||
|---|---|---|
| ||
Since 2019.02 #92895 there have been fixes in the writing of output time series in the Secondary Validation module. These fixes improved consistency in writing behaviour making sure repeated runs will result in the same outcome. This, however, resulted in significant different outcome for some very specific use cases involving simulated time series. In case simulated time series were being validated as output time series in a workflow that did not create them, it results in the original data being hidden since the new write action creates a newer module run instance for the data. Changing simulated data in a different workflow than it was created should not have been allowed, but because of the previous inconsistencies in writing behaviour it was possible to use this data as output for the secondary validation module. In order to still support validation of simulated data in a different workflow than it was created, a "logs only" mode has been introduced. This mode can be considered as "read only" and will be described below. |
Logs only (read only) mode
This mode has been introduced to enable validation of simulated time series that have been created in a previous workflow.
When simulated time series created in a previous workflow are being validated outside of the "logs only" mode it results in the original data being hidden since the new write action creates a newer module run instance for the data resulting in hiding of the original data.
It lets the Secondary Validation module know that there will be no data changes and therefore previous Secondary Validation does not need to be undone which normally is required for consistent and repeatable behaviour.
The Secondary Validation module will at the start check whether there in any check configured that could possibly change the data and therefore requires a write action.
As long as if it is certain there will not be any write actions then the "logs only" mode will be enabled.
- minNumberOfValuesCheck: never required write actions
- minNonMissingValuesCheck: never required write actions
- minReliableOrDoubtfulValuesCheck: never required write actions
- minReliableValuesCheck: never required write actions
- seriesComparisonCheck: requires write actions when there is no <outputMode> defined or <outputMode> is flags_and_logs (default). Does not require write action when <outputMode>logs_only</outputMode> is configured
- flagsComparisonCheck: requires write actions when there is no <outputMode> defined or <outputMode> is flags_and_logs (default). Does not require write action when <outputMode>logs_only</outputMode> is configured
- spatialHomogeneityCheck: requires write actions when in ANY of the thresholds there is no <outputMode> defined or <outputMode> is flags_and_logs (default). Does not require write action when in ALL of the thresholds <outputMode>logs_only</outputMode> is configured
- mannKendallCheck: requires write actions when in ANY of the thresholds there is no <outputMode> defined or <outputMode> is flags_and_logs (default). Does not require write action when in ALL of the thresholds <outputMode>logs_only</outputMode> is configured
- flagPersistencyCheck: ALWAYS requires write actions, when this check is present in the Secondary Validation module, it will NEVER go into logs_only mode