Time Series Lister

Introduction

The Time Series Lister is a display to verify and inspect time series which are written to the database in any workflow. 

Figure 1a: the GUI of the Time Series Lister (with a workflow selected)

When a filter - location - parameter selection is made via the Data Viewer, the Time Series Lister results will be filtered accordingly.

Figure 1b: the GUI of the Time Series Lister (with a workflow selected AND a Data Viewer selection)

The Time Series Lister is a pre-configured display and a very handy tool for inspecting the contents of the FEWS database (localDataStore). It provides an overview in tabular form of statistics and other metadata (columns) of the time series (rows) in the database. To aid the inspection of data, the user can add or remove columns, and select and filter rows. From the Time Series Lister, it is also possible to access the content (values) of selected time series by opening the time series display or the spatial (grid) display.

Configuration

To enable the Time Series Lister, it must be added as <explorerTask> in Explorer.xml:

<explorerTask name="Timeseries Lister">
   <iconFile>tableInfo.jpg</iconFile>
   <taskClass>nl.wldelft.fews.gui.plugin.timeseriestableviewer.TimeSeriesTableViewer</taskClass>
   <toolbarTask>true</toolbarTask>
   <menubarTask>true</menubarTask>
   <allowMultipleInstances>false</allowMultipleInstances>
   <accelerator>ctrl T</accelerator>
</explorerTask>

Usage

The Time Series Lister enables users to inspect the database. Database records are visualized as individual rows, which can be selected. Once selected, records can be browsed through (all columns) or their values can be inspected via a graph (scalar series) or the spatial display (grid data).

In the top window, workflows are available. When a workflow is selected, the lower part of the screen shows the records that have been written to the database in this selected workflow. Individual rows in this lower part can also be selected. All meta-information and statistics are calculated on-the-fly and are not stored. Within selections or filters (e.g., on start/end time), the content of the visible columns is updated immediately.

There are many columns that can be viewed: general columns about module instance, location, parameter, disk space, and all kinds of relevant meta-information about the selected record(s). Besides the general metadata, the basic statistics of that row can be inspected. These basic statistics include sum, average, maximum, minimum, etc.

The remaining columns describe the quality of the data. For example, if the user wants to know how many values are 'unreliable' and how many 'periods' of unreliable data are available, this information can be retrieved. The same is true for all kinds of combinations of doubtful, completed, and corrected data. This same functionality (number of records, periods) is available for primary and secondary validation results. This means that the user can inspect the number of records and periods of, e.g., hard maximums, flag comparison, or series comparison.

When many records (rows) are available, the user can filter the results. Double-clicking a cell will automatically filter for that value. If you want to filter on, e.g., a location, select the location (name or ID) column and press F8 or double-click it. The background color of the cell will turn blue (indicating there's a filter on it). This filter mechanism is applicable to all available columns, including Start and End Time. When the user changes the start and/or end time, the visible columns will be updated accordingly.

Context Menu Options (lower table)

Figure 2: context menu of the Time Series Lister

The Graphical User Interface contains a number of context menu options for selection/filtering and inspection of data. Right-click the mouse on one of the cells to access the context menu.

Special Filter option: Start time / End time

Two columns have special features: the Start and End Time columns can be used to increase or limit the view period. If you adjust these, all statistical columns or columns showing information about quality flags and (secondary) validation will be adjusted automatically.

Double-clicking in these time columns will provide the user with additional options, as shown in the figure below.

Figure 3: Special filter in the Start/End time columns

Meta Data Elements / Select Columns (F8)

Figure 4: Selecting different columns (elements) in the Time Series Lister

The meta data elements are grouped as follows.

• General (e.g. moduleinstanceId, location, parameter, timestep, blobsize etc.)
• Basis Statistics (Sum, Mean, Median, Standard Deviation, Percentile-5, Percentile-95)

The below mentioned groups give an overview of number of values and periods based on quality flags, primary and secondary validation.

• Missing
• Original Reliable
• Corrected Reliable
• Completed Reliable
• Original Doubtful
• Corrected Doubtful
• Completed Doubtful
• Original Unreliable
• Corrected Unreliable
• Completed Unreliable
• Soft Min
• Soft Max
• Hard Min
• Hard Max
• Same Reading
• Rate of Change
• Temporary shift
• Series Comparison
• Flag Comparison
• Spatial Homogeneity

The list of available columns can be adjusted by (un)selecting the individual elements. Only the selected ones will be visible in Time Series Lister after confirming by clicking <OK>.

Selections can be copied to the clipboard (<Copy Selection> button) and adjusted in a text editor (figure 5). These lists can be shared amongst users (e.g. e-mail). From a text editor you can adjust the list by removing rows, select the remaining rows and then copy it the clipboard (<CTRL+C>) again. After that use the <Paste Selection> button to update your selected columns (figure 6). See below:

Figure 5: Copying the selection of columns to clipboard

Figure 6: Pasting the selection of columns from the clipboard

Delete module instance from Time Series Lister

It is possible to remove module instances from your localDataStore via the Time Series Lister. This function only exists in a FEWS stand alone mode. The procedure is as follows:    

1. In Time Series Lister, select one workflow, press F12, a menu will pop out (figure 7).

figure 7: <F12> menu in Time Series Lister

2. Select '9 Delete module instance for selection', the module instances in this workflow will be displayed for selection. One can select the module instances that should be removed from the database.

figure 8: select module instances to be removed from database

3. Press 'OK', the selected module instances will be removed. If all module instances in one workflow are all removed, the workflow will become empty, but the workflow still appears in the Time Series Lister.

To completely remove this workflow from the list, go to FEWS F12 menu (in log panel, press F12), select 'P database', select 'validate, repair, and compact datastore indices and cache files', then restart FEWS, the workflow will be completely removed. To completely remove the data from the localDataStore (i.e. reduce file size), press F12, select 'P database,' and select 'repair and defrag localdatastore'.

Figure 9: Remove workflow and data using F12 - P - database menu