04 Data Display and Data Editor

• Editing Data
• Changing display properties
• Interactively modifying display colours
• The F12 debug menu (Timeseries Display)
• Statistical functions
• Filtering data

Data Display and Data Editor

The Data Display and Data Editor offer graphical displays for presenting and editing data. Both have nearly the same functionality, but only the Data Editor allows you to edit data. In addition, the Data Editor can only be opened once, while multiple instances of the Data Display can be opened.

This display offers the following features:

• Display of time series in:

  • Graphs (scalar/longitudinal profiles)
  • Tables (scalar/longitudinal profiles)
• Tabular data editor
• Display of actual data with:
  • Historical events
  • Threshold levels

Data is displayed in a table and a graph simultaneously. The splitter between the table and the graph can be moved to modify the display. Select the data series that should be displayed using the List Boxes and/or the Map Display (see also Filters).

Decimals for the reference values (ticks) in the chart autorange between 0 and 6, always using the lowest amount necessary to differentiate between ticks.

When the display is started through the FEWS Tools menu item Data Editor, data can be edited and the edited data can be stored in the FEWS database. When the display is started through the FEWS Tools menu item Data Display or through the FEWS toolbar, the data can only be viewed.

1.1 Data Display

Press the [Data Display] button on the toolbar to present graphs displaying the data. The splitter is normally positioned on the left of the display.

When data series of different data types have been selected, the data will be presented in different subplots, with the data series of a single data type merged into one subplot.

The data in the graph is updated automatically when new data becomes available. Below the graph, the workflows that generated the forecast time series are listed. Their numbers are used in the graph legend and table headers as well.

Time series and graph order

There are specific rules for time series ordering. For time series from the data viewer, the order of filters is used.

Time series are first ordered by their filter and then according to their time series set within the filter.

There are also rules for the ordering of the graphs when the time series come from the data viewer/filters.

The graphs follow the order of the parameter groups defined in Filters.xml.

Some Things to Keep in Mind:

• Since the parameter group ordering may not exactly match the filter and time series set ordering, it can happen that the letters associated with the time series in the graphs will not be ascending.
• When parameters and/or locations are present in multiple filters, it can happen that for the same parameter-location selection, the order of time series is different depending on which filters are selected. The first selected filter a time series is part of will determine its order.

The following buttons are used in the Time Series Display:

Table

By clicking this button, the time series in the graph is also displayed in tabular format. Please note that if you want to edit data, you must select the Data Editor from the Tools menu (see section 4.3).

Selecting the small arrow at the right of the icon shows the next menu:

Shortcuts

Display templates are available for rapid assessment of data. The templates can be activated by clicking the aforementioned button. The left side of the display shows the available shortcuts in a pre-configured grouping.

The data shown via the Shortcuts belong to the forecasts selected in the Forecasts panel of the FEWS Explorer. Please note that the current forecast is always displayed in the graphs.

Graph

This button can be used to hide or show the graph. If the table was not yet selected, it will appear.

Selecting the small arrow to the right of the icon shows the following menu:

 Toggle graph

When multiple data series are selected for a single data type (parameter), the graphs will be placed in one subplot of the graph. This sometimes complicates the analysis of time series with different data ranges. By clicking the above-mentioned button, you can change the layout of the chart to either:

• Merged into a single graph
• Stacked in different subplots

Scale to show unreliable data

This button can be used to focus on time intervals with unreliable data within the selected time series. Data is marked as unreliable if values exceed hard limits.

Show data labels

Clicking the 'Show data labels' button allows you to display quality flags for the data you are presenting. The information is shown at the bottom of the screen.

The following color codes have been used:

• The upper line indicates the origin of the data:

  • Original data: no line is displayed
  • Automatically corrected data (original data flagged as unreliable and automatically interpolated): blue line
  • Automatically completed data (original data is missing and interpolated by system): magenta line
  • Manual corrected data (original data edited by a user): dark blue line
  • Manual completed data (filled up by a user): dark magenta line

• The middle line indicates the quality of the data:

  • Reliable data: no line is displayed
  • Doubtful data: orange
  • Unreliable data: yellow

• The lower line indicates where comments have been added to the data:

  • No comment: no line is displayed
  • Comment available: dark red

The lines are displayed under the data points, and the colors correspond with the coloring used in the tables.

Show threshold colors in background

Toggle Filter and Shortcuts

  Hide unreliables

Stack plot

Show legend at vertical axis

Show legend inside graph

Show legend at horizontal axis

Hide legend

Hide footer

Show selected values in legend

Show on/off checkboxes in legend 

It is possible to show checkboxes in the legend to quickly hide/unhide time series in the time series dialog. You can find the option to show legend checkboxes in the chart dropdown button in the toolbar.

Toggling off a checkbox will only hide the individual time series. If you toggle off all time series in a chart or subplot, the plot will still show an empty chart. This is by design so that the checkboxes still indicate which time series can be toggled back on. To hide an entire chart or subplot, you should use the TimeSeries Visibility Dialog with the legend checkboxes disabled.

When statistical series are active, it is possible to also hide them individually. The same applies to their source series, they can be hidden without the resulting statistical series being hidden.

When checkboxes are not shown in the legend the hidden series will not be in the legend.

But when checkboxes are shown in the legend, their will be an empty checkbox for the series just like in the visibility dialog.

When "hide original series" is selected for the statistical functions, the sources series for the statistics will not be shown in the graph, table, visibility dialog and legend (with or without checkboxes)

For resampling, one can use the following button:

Show column indicator in legend (since 2024.01)

If toggled, the indicator of the table column corresponding to the time series is added to the legend. For example, the following legend   

will then be shown as

corresponding to the columns in the table

No threshold scaling

By clicking this button, the scaling will not adjust to show thresholds that are out of the data range.

Scale to show relevant thresholds

With this option, the graph is scaled so that relevant thresholds (the closest threshold values) are displayed together with the selected time series in the graph.

Scale to show all thresholds

This button can be used to show all pre-configured thresholds of a certain parameter at the selected location(s). This means that all thresholds levels (e.g. Standby, Flood Watch, Flood Warning and Severe Flood Warning) are displayed in the graph.

Validation

Users

Comments

Timeseries Lister

This button will add a database viewer below the display for the time series that are shown. The database records are visualized as individual rows, which can be selected. When selected, they can be browsed through, displaying all columns. All meta information and statistics are calculated on-the-fly and are not stored. Within selections or filters (e.g., on start/end time), the content of the visible columns is updated immediately.

Features:

• Columns Available: General columns about module instance, location, parameter, disk space, and all kinds of relevant meta information about the selected record(s).
• Basic Statistics: Sum, average, maximum, minimum, etc.
• Data Quality: Information on the number of unreliable values and periods of unreliable data.
• Validation Results: Number of records and periods for primary and secondary validation results, such as hard maximums, flag comparisons, or series comparisons.

This functionality helps users inspect the number of records and periods for different quality and validation criteria.

The Time Series Lister has similar functionality to the Database Viewer, including filtering, ordering, and right-mouse context menu options. These are described in detail in the Time Series Lister section.

Descriptive statistics

This button will add a table to the display with descriptive statistical information. The statistics only appear if a column in the table is selected. Use the right mouse button to copy this table to the clipboard for pasting into Excel.

Statistical Functionality Available (if configured):

• Mean
• Min (with date)
• Max (with date)
• Sum
• Standard deviation
• Percentiles (exceedance and non-exceedance)

Time Series Information:

• Location ID
• Location name
• Parameter ID
• Parameter name
• Module instance
• Start time
• End time

Data Quality Information:

• Number of values
• Number of completed values
• Number of corrected values
• Number of doubtful values
• Number of missing values
• Number of reliable values
• Number of unreliable values

Timeseries Information (hideable panel)

This button is used to show (or hide) a panel that contains four subpanels (toggleable). These subpanels display the Time Series Information, Attributes, Validation Criteria, and available Thresholds for the selected time series at the selected time step.

Opening this panel will automatically hide the descriptive statistics panel when it's open.

Thresholds

The thresholds table will show: Type, Id, Value, Label, Comment, Description, Time Span, Season, Up Action, Down Action

Since this is a lot of columns and not all columns apply to all thresholds, columns can be hidden via right mouse "Show / hide columns":

The hidden columns will be stored in the user settings.

Toggle splitter Orientation

This button allows you to place the splitter between the table and graph in a horizontal position. Clicking the same button again returns the splitter to its default vertical position.

Search and Select Forecast

Displays in FEWS by default only show current approved forecasts. Previous forecasts can be shown if you approve them in the Forecast Management Display or by using the "search and select forecasts" option.

There are two ways to use this button. The small arrow on the right opens up a drop-down menu where you can quickly select the number of recent forecasts you want to view.

When you click on the button itself, it opens a separate display with additional options and information about the available time series.

Hide whole forecasts (since 2020.02)

Since 2020.02 it is possible to hide all time series belonging to the same forecast.

When time series are selected, either with the dropdown button or by using the shortcut ctrl+alt+shift+X, all time series sharing a forecast with one of the selected forecasts will be hidden.

In the image, forecast [1] has been hidden.

In the "search and select forecasts," there will be a postfix stating "(hidden)" behind their forecast time.

All hidden forecasts can be made visible again by selecting the "Show all hidden forecasts" dropdown menu item or by using the shortcut ctrl+alt+shift+Z.

Historical Events

With this button historical events can be activated (provided that they are pre-configured). They will be displayed in the same graph to enable a comparison with the actual timeseries. A special option in the Data Display is the option to add historic events to the chart window. These events can be predefined and included in the configuration. Alternatively, dependent on the set-up of the system (existence of an Deltares Open Archive), they can be defined in the Archive Catalogue Display.

To show historic events for a specific location, follow these steps:

1. Select the location from the location list box in the FEWS explorer.
2. Select the parameter from the parameter list box.
3. Open the chart display by pressing the chart button.
4. Press the button in the Chart Display and select one of the historical events from the list.

Display Time

The "Display Time" button opens a dialog where the view period in the graph can be changed, saved, or reset.

If the period entered in this dialog has been applied, then this view period is restored when a restore is requested. The restore request can be triggered by the chart popup menu (Auto Range-> Both Axes), or by the shortcut Ctrl+0, or by left swipe in the chart. To restore the default view period as configured in TimeSeriesDisplayConfig, the "Reset" button in this dialog should be used.

With the "Save" button, the current relative view period is stored in the user_settings.ini file and is applied again after restarting the application.

Navigating in Graphs

Zooming in and out can be done with the help of the scroll wheel of the mouse. Scrolling away from the user results in zooming in, while scrolling towards the user results in zooming out.

Zooming is also possible by selecting a zoom range. Drag the mouse while holding down the right button to form a rectangular area to zoom in to.

When clicking the right mouse button, the dropdown menu presented below appears. The menu provides you with some additional options.

Save As

You can save the graph to file with this option.

Print

This option allows you to print the graph.

Zoom in / out

You can select to zoom in and out along one or both axis with this option.

Auto Range

This feature allows you the set automatic scaling along one or both axis.

Zoom to range...

In the popup dialog “Zoom to range…” it is possible to set the minimum and maximum of the left vertical axis. 

This manually entered range will stay that way until the user starts zooming again. This functionality is activated when from the zoom tool the drop-down option “Allow Vertical Axis Zoom” is selected. 

Changing the view period

Since 2020.01, it is possible to move the view period to the first or to the last data point available. The buttons are in the following order:

1. Move to the first data point available of the selected time series (the length of the period will remain the same).
2. Move one full view period back.
3. Move a half view period back.
4. Move a half view period forward.
5. Move a whole view period forward.
6. Move to the last data point available of the selected time series (the length of the period will remain the same).
7. Set view period to include all data of the selected time series.

Data Editor

The Graph and Editor display has a table and an edit mode. 

The leftmost column displays the date and time of the data. The period displayed spans the overall period defined in the filters for all time series.

The colors in the headings of the columns correspond with the line color in the graph.

In the date/time column, different colors are used to provide a logical meaning to certain important moments:

• Light blue: historical data, either observed or hindcast (data before T₀)
• Dark blue: forecast data (data after T₀₎

For the data columns, the background color is used to indicate the quality of the data, identified by the data flags set of the Validation utility. Further, an indication of the editability of the data is given by the background color. The meaning of the background colors in the data fields is:

• White: reliable, editable
• Light yellow: doubtful
• Bright yellow: unreliable
• Gray: not editable

The font color in the data fields indicates the data origin. The following colors are used:

• Black: original data
• Blue: corrected (original data flagged "Unreliable" and interpolated)
• Magenta: completed (original data missing and interpolated)

The edit mode can be enabled from the toolbar with the "Enable Edit mode" button. Data (only those with a white background) can be edited by double-clicking a field in the column. Editing may be done in the table directly. The result of an edit action is directly displayed in the corresponding graph. Manually corrected data will be saved to the database by disabling the Edit mode again from the toolbar. A pop-up will ask to save the edited data to the database. With [OK], the data is saved, [NO], the data is not saved and with [Cancel], changes will be undone.

Through manual editing of data, the source of the data changes from original to corrected. The font color used for edited data is blue to indicate that it is completed data. The background color remains white, as the data has not yet been validated. If, however, after pressing [Apply], validation indicates that the value exceeds certain validation limits, the background color will be set accordingly.

ReadOnlyPeriod

Since 2020.01, it has been possible to "freeze" part of a time series and make that part uneditable. This can be useful if the values need to be evaluated and verified, then used in different systems. This ensures that they all contain the same data and cannot change. This can be done by configuring a ReadOnlyPeriod. This contains a relative time and a time series filter. All time series that match the filter become uneditable before the given relative time. Example configuration:

<!-- This item is not fully configured, only displayed to show the position of readOnlyPeriod in the XML file-->
<description>...</description>
<!-- This item is not fully configured, only displayed to show the position of readOnlyPeriod in the XML file-->
<generalDisplayConfig>...</generalDisplayConfig>
<!-- This item is not fully configured, only displayed to show the position of readOnlyPeriod in the XML file-->
<defaultViewPeriod unit="day" start="-5" end="5"/>
<!-- This item is not fully configured, only displayed to show the position of readOnlyPeriod in the XML file-->
<globalDatumLocationSetId>...</globalDatumLocationSetId>
<scrollerDefaultViewPeriod start="-2" end="2" unit="hour"/>
<readOnlyPeriod>
   <relativeTime value="-2" unit="day"></relativeTime>
   <timeSeries>
      <locationId>LocA</locationId>
   </timeSeries>
</readOnlyPeriod>

In this case, data older than 2 days before the system time, in a time series where the location ID is LocA, becomes uneditable.

You can configure as many readOnlyPeriods as you wish. They should be configured in the TimeSeriesDisplayConfig, following scrollerDefaultViewPeriod. "Frozen" time series can be edited if the "enable editing read-only period" option is selected. Time series that are entirely uneditable remain so.

Copy and Paste Timeseries

A selected timeseries can be copied to the clipboard with the Copy selected timeseries button. Up to 10 series can be copied and kept on the clipboard. Duplicate entries to the clipboard are ignored.

The latest copied series can be pasted with the Paste selected time series button. Any of the other items on the clipboard can be selected and pasted using the dropdown to open Paste from previous selection.

Seamless Archive integration  

Seamless integration automatically augments the data record through on-the-fly downloads from the OpenArchive. This feature only works for external historical datatypes!

To begin, press the button "Connect to the external data source." FEWS will then start looking for the data relevant for the plot in the OpenArchive.

When any relevant data is found (i.e., at least a matching locationId and parameterId), the button "Read data from the external data source" becomes enabled. Press the button “Read data from the external data source”, and the data will be read and shown in the plot.

Longitudinal profile

Since release 2007.02, the display offers also the opportunity to show longitudinal profiles. In such graph, the parameter values (Y-axis) are plotted against the river chainage (X-axis). The graph can progress over time, eitehr step by step or animated, via the time control toolbar, described in the Spatial display. both the graph and table will change over time as they shown data along the profile.

Longitudinal profiles can be simply added to the display by configuration of timeSeriesSet’s with value type longitudinalprofile in the config file  DisplayGroups.xml

Rating curve

The  display offers also the opportunity to show rating curves.  In such graph, the rating curve stage values  are plotted against discharge values.  By default stage is shown on the Y-axis and discharge on the X-axis.  Configure  stageAxisOrientation ‘horizontal’ in TimeSeriesDisplay config if the axes should be switched.  Rating curve display shows the rating curve that belongs to the location  selected in the table or in the chart legend. If this location has multiple rating curves with different qualifiers,  rating display shows a list where the specific rating curve can be selected

Displaying gaugings along with the rating curves

Since release 2019.02 it is possible to display gaugings (observations of level and discharge (velocity) combinations) along with the rating curve. To show the observations in rating curve display, follow these steps:

1. open TimeSeriesDialog with the time series you want to see along with the rating curve,
2. open rating curve display for required location,
3. select option “Show selected stage and discharge as scatter plot” from the drop down menu of the button “ Rating Curves” (on the TSD toolbar),
4. select one stage series and one discharge series from the time series table or from the chart legend. The selected series will be shown as scatter plot along the rating curve. All values in the chart zoom period will be displayed. The number of values shown can be changed by changing of the zoom period.
   
   Note that the selected series must be stage and discharge, it means the series parameter must be from the same parameter group as the rating curve parameters configured in Parameters.xml. Other series selection will be ignored.

Category plot

In Category plot the parameter values (X-axis) are plotted against the locations (Y-axis).
The graph can progress over time, either step by step or animated, via the time control toolbar. Both the graph and table will change over time
The detail description of the time toolbar can be found in the paragraph Spatial display.

If there are more subplots in a plot, a separate table wil be created for each subplot.

F

Configuration

To show the data in the category plot, the 'category' plot viewer type should be specified in DisplayGroups.xml with element <plotViewerId> :

<plot id="MODISCategory1Plot">
    <subplot>
        <timeSeriesSet>
            <moduleInstanceId>ImportModis</moduleInstanceId>
            <valueType>scalar</valueType>
            <parameterId>OWA</parameterId>
            <qualifierId>Aspect-ID   0</qualifierId>
            <locationSetId>ImportModis</locationSetId>  
            <timeSeriesType>external historical</timeSeriesType>
            <timeStep unit="hour" multiplier="1"/>
            <relativeViewPeriod unit="hour" start="-24" end="4"/>
            <readWriteMode>read only</readWriteMode>
        </timeSeriesSet>         
        <timeSeriesSet>
            <moduleInstanceId>ImportModis</moduleInstanceId>
            <valueType>scalar</valueType>
            <parameterId>SFR</parameterId>
            <qualifierId>Aspect-ID   0</qualifierId>
            <locationSetId>ImportModis</locationSetId>          
            <timeSeriesType>external historical</timeSeriesType>
            <timeStep unit="hour" multiplier="1"/>
            <relativeViewPeriod unit="hour" start="-24" end="4"/>
            <readWriteMode>read only</readWriteMode>
        </timeSeriesSet>
    </subplot>
    <plotViewerId>category</plotViewerId>
</plot>   

Restrictions

The category plot can be customized in TimeSeriesDisplay.xml and in DisplayGroups.xml, similar to the other plot types. However not all configuration elements are applicable to the category plot. If these elements are still configured, they wil be ignored.
The category plot is only applicable to scalar timeseries.

Storing 'favorites'

The data selection made in the Data Viewer can be stored through its toolbar. This allows for storing favorite data selections / graphs in the user settings. More information on this functionality can be found in the location parameter selection section of the FEWS Explorer page.

<readOnlyPeriod>
   <relativeTime value="-2" unit="day"></relativeTime>
   <timeSeries>
      <locationId>LocA</locationId>
   </timeSeries>
</readOnlyPeriod>

Long Term Scroller

For viewing a longer period than is shown in specific graphs, the long term scoller can be openen via the button highlighted in red below.

It will open a zoomed out graph were a smaller period can be selected via the black vertical lines. The red vertical line shows the display time and the green vertical line the system time.

History popup

When clicking ctrl+H on a value, a popup will appear which shows the history of data on that time step.