Versions Compared

Old Version 1

changes.mady.by.user Camiel van Breugel

Saved on

compared with

New Version Current

changes.mady.by.user Simone De Kleermaeker

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Quality flags

Quality flags are constructed on a philosophy of two qualifiers. The first describes the origin of the data and the second the quality.

Possible origins of data are:

+ Original: This entails the data value is the original value. It has not been amended by DELFT-FEWS
+ Completed: This entails the original value was missing and was replaced by a non-missing value.
+ Corrected: This entails the original value was replaced with another non-missing value.

Possible qualifiers are:

+ Reliable: Data is reliable and valid
+ Doubtful: The validity of the data value is uncertain
+ Unreliable: The data value is unreliable and cannot be used.

Following this specification, the table below gives an overview of quality flag enumerations

describe the quality and origin of data. The Flags available in Delft-FEWS are a based on a combination of two components: 

  • Origin of the data: Original / Completed / Corrected
  • Quality of the data: Reliable / Doubtful / Unreliable

Note
  • Unreliable Data will not be used for calculations like transformations or external models.
    It can still be viewed and will be exported with a regular exportModule, as long as the unreliable flag is shown with it.
  • No difference is made between historic and forecast data when it comes to Quality Flags.
    In the data model of Delft-FEWS this difference is captured in the time series type definition.
  • External sources may either be an actual external source, a forecasting module or a transformation. 

Additional Quality flags

In addition, there are a few other quality flags (or more broadly, time series flags) defined in Delft-FEWS, these are listed and described in the table below.

You can define custom quality flags. This can be useful to store external quality flag information that can't be mapped to the flags known to Delft-FEWS.
Custom Flags have no meaning, other than to store this extra bit of metadata. The <name> and <description> are not used anywhere in Delft-FEWS (e.g. in the GUI) and only serve as additional information to the Flag mappings defined in the file itself. 

ValidationRuleSet labels

The legend of the quality labels also includes optional labels that can be configured in validationRuleSet.xml.
These labels are unrelated to the QualityFlags described on this page. 

Enumeration of quality flags

The following quality flags are known to Delft-FEWS.Table 1                   Enumeration of quality flags

Enumeration

Description

0

Original/Reliable
The data value is the original value retrieved from an external source and it successfully passes all validation criteria set.

1

Corrected/Reliable
The original value was removed and corrected. Correction may be through interpolation or manual editing.

2

Completed/Reliable
Original value was missing. Value has been filled in through interpolation, transformation (e.g. stage discharge) or a model.

3

Original/Doubtful
Observed value retrieved from external data source. Value is valid, but marked as suspect due to soft validation limits being exceeded.

4

Corrected/Doubtful
The original value was removed and corrected. However, the corrected value is doubtful due to validation limits.

5

Completed/Doubtful
Original value was missing. Value has been filled in as above, but resulting value is doubtful due to limits in transformation/interpolation or input value used for transformation being doubtful.

6

Original/Unreliable
Observed value retrieved from external data source. Value is invalid due to validation limits set. Value is marked as unreliable and will not be used in transformations and model calculations.

7

Corrected/Unreliable
The original value was removed and corrected. However, corrected value is unreliable

8

Completed/Unreliable
Original value was missing. Value has been filled in as above, but resulting value is unreliable,

9

Missing
Missing value in originally observed series. Note: this is a special form

of 

of both Original/Unreliable and Original/Reliable.

Notes:

...

10

Deleted
Only used for non-equidistant series when timestep is deleted

11, 12, 13

Not used

14

Accumulation Reset
Used for processing of accumulating rainfall timeseries (e.g. when storage is reset), see https://publicwiki.deltares.nl/spaces/FEWSDOC/pages/120819119/Precipitation

FlagSource
Anchor
flagSource
flagSource

Next to the flag it is also possible to get or set information why the flag is as it is through the so-called flagSource (since 2012.01). 
The user is able to see why a certain value is validated as unreliable, eg. due to exceeding of the hard max validation rule.

flagSourceDescription

CA

Conditional aggregation

FC

secondary validation, flag comparison

HN

hard min
HX

hard max

IMP

flag is imported

MAN

manual edit

MK

secondary validation, Mann-Kendall test

OSC

oscillation

ROF

rate of fall

ROR

rate of rise

SC

secondary validation, series comparison

SFP

start flag persistency 

SN

soft min

SR

same reading

SVP

secondary validation, Flag persistency (FlagPersistencyCheck)

SX

soft max

TS

temporary shift