Versions Compared

Old Version 4

changes.mady.by.user Simone De Kleermaeker

Saved on

compared with

New Version Current

changes.mady.by.user Simone De Kleermaeker

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Quality flags describe the quality and origin of data.
The Flags available in Delft-FEWS are a based on a combination of two components: the origin and the quality of the data. 

  • Origin of the data

...

  • Original
    • The data value is the original value. It has not been changed.
  • Completed
    • The original value was missing and has been replaced by a non-missing value.
    • CorrectedThe original value has been replaced with another (non-missing) value.
  • : Original / Completed / Corrected
  • Quality of the data

...

  • Data is reliable and valid

...

  • Data validity is uncertain

...

  • : Reliable / Doubtful / Unreliable

...

Note
  • Unreliable Data will not be
  • used for calculations like transformations or external models.

...


  • It can still be viewed and will be exported with a regular exportModule, as long as the unreliable flag is shown with it.

...

note
  • No difference is made between historic and forecast data when it comes to Quality Flags.
    In the data model of Delft-FEWS this difference is captured in the time series type definition.
  • External sources may either be an actual external source, a forecasting module or a transformation. 

Additional Quality flags

In addition, there are a few other quality flags (or more broadly, time series flags) defined in Delft-FEWS, these are listed and described in the table below.

You can define custom quality flags. This can be useful to store external quality flag information that can't be mapped to the flags known to Delft-FEWS.
Custom Flags have no meaning, other than to store this extra bit of metadata.
The <name> and <description> are not used anywhere in Delft-FEWS (e.g. in the GUI) and only serve as additional information to the Flag mappings defined in the file itself. 

ValidationRuleSet labels

The legend of the quality labels also includes optional labels that can be configured in validationRuleSet.xml.
These labels are unrelated to the QualityFlags described on this page. 

Enumeration of quality flags

This leads to the The following quality flags available in are known to Delft-FEWS.Table 1                   Enumeration of quality flags

Enumeration

Description

0

Original/Reliable
The data value is the original value retrieved from an external source and it successfully passes all validation criteria set.

1

Corrected/Reliable
The original value was removed and corrected. Correction may be through interpolation or manual editing.

2

Completed/Reliable
Original value was missing. Value has been filled in through interpolation, transformation (e.g. stage discharge) or a model.

3

Original/Doubtful
Observed value retrieved from external data source. Value is valid, but marked as suspect due to soft validation limits being exceeded.

4

Corrected/Doubtful
The original value was removed and corrected. However, the corrected value is doubtful due to validation limits.

5

Completed/Doubtful
Original value was missing. Value has been filled in as above, but resulting value is doubtful due to limits in transformation/interpolation or input value used for transformation being doubtful.

6

Original/Unreliable
Observed value retrieved from external data source. Value is invalid due to validation limits set. Value is marked as unreliable and will not be used in transformations and model calculations.

7

Corrected/Unreliable
The original value was removed and corrected. However, corrected value is unreliable

8

Completed/Unreliable
Original value was missing. Value has been filled in as above, but resulting value is unreliable,

9

Missing
Missing value in originally observed series. Note: this is a special form of both Original/Unreliable and Original/Reliable.

10

Deleted
Only used for non-equidistant series when timestep is deleted

11, 12, 13

Not used

14

Accumulation Reset
Used for rainfall data processing when the storage is reset (canister processing of accumulating rainfall timeseries (e.g. when storage is reset), see see https://publicwiki.deltares.nl/spaces/FEWSDOC/pages/120819119/Precipitation

FlagSource
Anchor
flagSource
flagSource

Next to the flag it is also possible to get or set information why the flag is as it is through the so-called flagSource. 
Delft-FEWS stores not only the quality flags, but also the source of the flag, the so-called flagSource (Since version flagSource (since 2012.01). 
The user is able to see why a certain value is validated as unreliable, eg. due to exceeding of the hard max .Table 1                   List of Flag Sourcesvalidation rule.

flagSourceDescription

CA

Conditional aggregation

FC

secondary validation, flag comparison

HN

hard min
HX

hard max

IMP

flag is imported

MAN

manual edit

MK

secondary validation, Mann-Kendall test

OSC

oscillation

ROF

rate of fall

ROR

rate of rise

SC

secondary validation, series comparison

SFP

start flag persistency 

SN

soft min

SR

same reading

SVP

secondary validation, Flag persistency (FlagPersistencyCheck)

SX

soft max

TS

temporary shift

...