Table of Contents |
---|
Overview
The Tabular Config Files Display (available since 2015.01 build 57292 ) can show a list of configuration files (csv and dbf) that are used to configure the selected locations, parameters and qualifiers. The configuration files can be can be displayed using a table viewer with sorting, filtering and column hiding functionality. This display is also accessible using accessible using the F12 debug menu. There have been several improvements to this display since the first introduction in 2015:
- Since 2019.02 the display is also connected to the forecast tree.
- Since 2020.02 the Tabular Config Files Display is connected to shapefiles that are displayed in the Map Display or the Spatial Display.
- Since 2024.01 it is also possible to edit CSV config files with the Tabular Config Files Display. In this way, it is a Metadata Manager Display.
Configuration
To use the Location Attributes Display the TabularConfigFilesDisplay viewer, configure it as as an <explorerTask> in Explorer.xml. For example:
Code Block | ||||
---|---|---|---|---|
| ||||
<explorerTask name="LocationTabular AttributesConfigFiles Display"> <iconFile>csvfile.png</iconFile> <mnemonic>L</mnemonic> <predefinedDisplay>tabular config files display</predefinedDisplay> <toolbarTask>true</toolbarTask> <menubarTask>true</menubarTask> <accelerator>ctrl L</accelerator> <toolWindow>false</toolWindow> <loadAtStartup>true</loadAtStartup> </explorerTask> |
Displaying Content
...
NOTE: Since 2024.01 adding the display to the explorerTasks it will by default allow editing of metadata in Stand Alone applications. These changes are local. It is never possible to edit metadata in the Operator Client. Edit permissions can be limited by setting the editing permissions in the TabularConfigFilesDisplay.xml. See: Section Edit metadata
Display functionality, link to Data Viewer (Filters)
When the Tabular Config Files Display is started, there will be a toolbar at the top of the display, and the two panels will probably be empty as is shown in the example below
Top panel (list view)
The top panel lists meta data files in the \MapLayers folder that are used in the Delft-FEWS application.
- Only .csv or .dbf files that are used in the configuration can be displayed.
If locations are configured in *.xml, *.Json or database, these file will not be shown. - Files are sorted alphabetically.
- Content of the first file is shown by default in the bottom panel. Selecting another file will display its content.
...
The display is linked to the locations list in the Data Viewer (Filters.xml).
- After a location in the locations list is selected, the meta data files that are used by this location will be listed.
- If parameters and/or qualifiers are also configured in csv files, the display will be linked to parameters and qualifiers in a similar fashion.
Bottom panel (table view)
The bottom panel shows the content of a meta data file selected in the top panel, by default the content of the first file in the list is shown.
Light blue columns on the left are "virtual columns", which show the Location ID and Name, not the content of a meta data file column.
- LOCATION_NAME : visible for all location attribute files
- LOCTION_ID: visible for files that are used as main files
- PARAMETER_NAME: visible for all parameter attribute file
Sorting of column(s)
The table with metadata
Only configuration files in .csv or .dbf format can be displayed. If the locations are configured in .xml, the file will not be shown in the display.
Sorting
The table can be sorted by clicking on a column header, rotating through the following options:
- sort alphabetically ascending
- sort alphabetically descending
- remove sort
When multiple column headers have been sorted on, indices indicate which is the order in which the columns are sorted.
In the right mouse popup menu the Undo sort option quickly deselects all selected sort columns.
Filtering in columns
Tables can be filtered by double clicking on a cell entry.
- Only records with exactly the same entry for the selected column will be displayed.
- The selected entry will be highlighted (blue).
- Double clicking another entry in a different column will expand the filter.
- To undo
...
- filtering
...
Column hiding
...
- , double click a highlighted cell again.
- In the right mouse popup menu the Remove all filters option quickly remove the set filters.
Hiding of columns
Columns can be hidden.
- In the right mouse popup menu the Set column visibility options opens a list of all columns.
- This allows a user to select or deselect columns that should be visible (at least one column has to remain visible).
- These settings
...
- are stored in the user settings and will
...
- be applied when Delft-FEWS is restarted.
Toolbar
The toolbar contains 6 buttons with the following functionality
Lock: To lock the tabular config file display. When locked, selecting another location in the locations list, won't change the selected location row in the display. | |
List all Files: To list all meta data files used in the configuration (not only the ones linked to a selected location) | |
Show modifiers: To show modified values in the tabular config files. These are location attributes where modifiers have been applied. All values that have been changed by modifiers will get a blue background color. A tooltip will display the unmodified value. | |
Show virtual columns: LOCATION_ID, LOCATION_NAME, PARAMETER_NAME are based on meta data in the CSV/dbf files. These virtual columns, together with special edit columns can be made visible or hidden in the bottom panel | |
Copy: To copy the content of selected cells to the clipboard | |
Export to CSV: To save selected meta data file as CSV file, e.g. to export modified values to a CSV file. |
Link to Forecast Tree (Topology)
The Tabular Config File Display can also be linked to the forecast tree (Topology.xml, since 2019.02).
When locations are configured in the Topology.xml configuration with the tabularLocationId element, the related tabular config files for those locations can be displayed.
Code Block | ||||
---|---|---|---|---|
| ||||
<nodes id="NodesWithLocations" name="Nodes with Location Ids">
<node id="NodeWithLocationId" name="Node with location">
<tabularLocationId>GLB</tabularLocationId>
</node>
... |
Since 2021.01 it is no longer possible to use the <locationId> element was for this linking. Please update your configuration to use <tabularLocationId> instead.
Link to Layer panel in Map or Spatial display (shapefiles)
Attributes from map layer shape files configured in the Map or Spatial Display, can be viewed with the Tabular Config Files display. No additional configuration is required.
- There are two ways select a shapefile attribute:
- In the Map Display or Spatial Display, open the layer panel and select a layer in the top panel, and a layer attribute in the bottom panel.
- In the Map display, enable the tooltip icon and select a layer attribute:
- When hovering over the layers on the map or Grid, the layer attributes will be highlighted.
- When they are highlighted, using a left mouse click to select the map element, this will show the corresponding entry in the Tabular Config Files Display.
- The Tabular Config Files Display will show the associated Shape Layer DBF file and its entries.
- The selected layer attribute will highlight the corresponding row in the table.
- When selecting a row in the table, the corresponding layer attribute will be highlighted on the Map or Spatial Display.
Configure Metadata Column Headers
By default the column headers of the meta data table use the column name in the CSV or DBF file.
They can be overruled in LocationSets.xml with two name elements:
- <columnDisplayName> for standard location elements (like id, name, x, y)
- <name> for location attribute names (like HYFS_STATIONID in the example below)
Note: Do not use the same <name> twice within a single meta data file. This will throw a Config ERROR when Delft-FEWS is started.
Code Block | ||||
---|---|---|---|---|
| ||||
<csvFile>
<file>AUS_stations</file>
<readOnly>true</readOnly>
<geoDatum>WGS 1984</geoDatum>
<id columnDisplayName="HyFS Station ID">H%bom_stn_num#6%</id>
<name columnDisplayName="HyFS Station Name">%bom_stn_name% - H%bom_stn_num#6%</name>
<x columnDisplayName="Longitude">%long_dec_deg%</x>
<y columnDisplayName="Latitude">%lat_dec_deg%</y>
<attribute id="HYFS_STATIONID" name="HyFS Station ID">
<description>HYFS station location identifier</description>
<text>H%bom_stn_num#6%</text>
</attribute>
..... |
Configure Metadata Rules
Meta data rules to check the meta data <text> for errors can be configured in LocationSets.xml.
There are rules available to check the text value on format, against enumerationValues and against a regularExpression.
The Meta Data Rules are checked by Delft-FEWS when the meta data configuration is loaded,
Errors will be logged when meta data is not according to the configured rules.
In The Tabular Config File Display, the rules are used to find and flag errors.
format
In the attribute element, a format can be added to the <text> element. The example below forces the attribute to contain 6 tokens.
- If the meta data file contains a value that breaks with this rule, a leading zero is added to the attribute.
In the example if the meta data contains the value "12345", Delft-FEWS will change this to "012345".
Code Block | ||||
---|---|---|---|---|
| ||||
<attribute id="HYFS_SENSLOCID" name="BoM Sensor Location Id">
<description>Associated HYFS station location identifier</description>
<text>%BOM_STATIONID#6%</text>
<required>true</required>
</attribute> |
<enumerationValue>
In the attribute element, you can add <enumerationValue> elements.
- The attribute <text> value will be checked to match one of the enumeration values.
- In the Tabular Config Files editor, a drop-down element will allow the user to select one of the enumerations.
Code Block | ||||
---|---|---|---|---|
| ||||
<attribute id="ACCEPTANCE">
<description>Only accepted sensors are included in the location set</description>
<text>%ACCEPTANCE%</text>
<enumerationValue>ACCEPTED</enumerationValue>
<enumerationValue>REJECTED</enumerationValue>
<enumerationValue>DISABLED</enumerationValue>
<required>true</required>
</attribute> |
<regularExpression>
When an attribute can contain multiple values, a regularExpression can be used.
The example below allows for example an attribute value "NSW;VIC".
See regular expression function examples on the internet for more examples.
Code Block | ||||
---|---|---|---|---|
| ||||
<attribute id="REGION" name="Region">
<description>Regional office(s) responsible for station. Enumeration: WA, NT, SA, QLD, NSW, VIC, TAS. Reflects time zone information</description>
<text>%REGION%</text>
<regularExpression>[WA|NT|SA|QLD|NSW|VIC|TAS|;]*</regularExpression>
<required>true</required>
</attribute> |
Edit Metadata
Anchor | ||||
---|---|---|---|---|
|
You can edit meta data from CSV meta data files (since 2024.01).
For this, a TabularConfigFilesDisplay.xml configuration file is needed in the DisplayConfigFiles folder (see example below):
- editPermission: restricts the permission to edit meta data files to specific user group(s).
With this display file added, the toolbar of the Tabular Config Files display is extended with meta data edit functionality.
Code Block | ||||
---|---|---|---|---|
| ||||
<?xml version="1.0" encoding="UTF-8"?>
<tabularConfigFilesDisplay xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/tabularConfigFilesDisplay.xsd">
<editPermission>Configuration</editPermission>
</tabularConfigFilesDisplay> |
Toolbar (extended)
The toolbar contains 11 additional buttons with the following functionality:
Only Show Errors: When Errors are found in the meta data, enabling this button will only show the meta data files and rows that have errors. Very much linked to the configurable meta data rules. | |
Only Show Edits: When this button is pressed, the non-committed edits will be filtered on. Only the meta data files and rows that have edits will be shown. | |
Edit Mode: With the correct permissions it is possible to edit meta data. This button activates the edit mode. | |
Show Originals: Wen data is edited and not yet committed, the original values can be shown. | |
Revert: Edited values that are not yet committed can be reverted. The user will be asked to revert the selected row, the selected meta data file, or all meta data changes. | |
Refresh Config: When edits are made, the Delft-FEWS system can be updated to use this edited configuration | |
Commit changes to config dir: The edited meta data files will be copied from temp folder to the config folder. | |
Delete Row: Delete the selected row from the meta data table | |
Add Row: Add a new row to the selected meta data table | |
Cut: Cut the selected meta data table cells to the Clipboard | |
Paste: Paste the clipboard to the selected meta data table cells |
Edit process
The meta data edit process is split in several editing and commit steps. This allows the user to first test metadata changes, before updating the Delft-FEWS configuration.
- When metadata values are edited, the changes are stored in the \temp\editedMapLayers\ folder. All files that are edited will be temporarily stored in this folder.
In the Tabular Config Files Display GUI, all files that are edited will be shown in bold with (draft) as post-fix. - After pressing the Refresh Config button, the meta data files in this \temp folder will be used by the Delft-FEWS application.
The user can then check the changes made (and revert if needed). - After pressing the Commit changes to config dir button, the edited files from the temp folder will overwrite the original files in the config folder.
At this point is not possible to revert changes anymore or check the original values.
The updated meta data files will not only contain the updated meta data values.
Two additional columns (with blue background) are added with information on the user that made the changes and when the changes were committed.
- LAST_EDIT_USER
- LAST_EDIT_TIME
Add Custom Activities (toolbar buttons)
The Tabular Config File Display also allows for adding additional task buttons to the toolbar.
These task buttons can start executables or scripts to check configuration consistency, link to GIT or SVN or run Python or Powershell scripts.
They allow a user to update other parts of the configuration, like XML files, by using (Python) scripts that create or update Delft-FEWS XML configuration files based on the meta data.
In the example below, a taskButton is added that starts several Python scripts through Powershell.
Code Block | ||||
---|---|---|---|---|
| ||||
<?xml version="1.0" encoding="UTF-8"?>
<tabularConfigFilesDisplay xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/tabularConfigFilesDisplay.xsd">
<editPermission>Configuration</editPermission>
<taskButton name="Update Config and Metadata for Public Web Products">
<description>Update Config and Metadata for Public Web Products</description>
<permission>Configuration</permission>
<iconFile>update_config.svg</iconFile>
<workDir>%REGION_HOME%\Modules\HYMM_python_scripts</workDir>
<executable>$WINPOWERSHELL$</executable>
<arguments>
<argument>$PYTHON$ create_metadata.py %REGION_HOME%\Config\MapLayerFiles\Australia;</argument>
<argument>$PYTHON$ create_maps.py %REGION_HOME% %REGION_HOME%\Modules\HYMM_python_scripts;</argument>
<argument>$PYTHON$ create_products.py %REGION_HOME% %REGION_HOME%\Modules\HYMM_python_scripts;</argument>
</arguments>
</taskButton>
</tabularConfigFilesDisplay>
Global.properties
WINPOWERSHELL=C://Windows/System32/WindowsPowerShell/v1.0/powershell.exe
PYTHON=%REGION_HOME%/Modules/Python3/python.exe
|
Automated completion of attribute files
Often, locations will be defined in a single main location file, while the meta data is spread over several attributeFiles.
To make it easier to add new locations (and their required meta data), you can configure LocationSets.xml to automatically complete attributeFiles (since 2023.01).
Auto-complete based on parent file
- <requireAllIdsInParentFile>: when true, a new location added to the location file will automatically be added to the attributeFile
In the example below there is a main meta data file (AUS_sensors.csv) listing all locations and some key attributes, with an additional location attribute file (AUS_sensors_acceptance.csv).
Since requireAllIdsInParentFile = True, a new location row will automatically be added to AUS_sensors_acceptance.csv (the attributeFile) when a new location is added to AUS_sensors.csv (the main csvFile).
This way Delft-FEWS will make sure that the AUS_sensors_acceptance.csv file contains the same records as in the master AUS_sensors.csv file.
Auto-complete based on locationSet
Sometimes there's not such a one-on-one relation between a location file and attribute file, but instead you want the attribute file to contain all locations in a locationSet.
- <requireAllIdsInLocationSet>: when true, a new location added to the locationSet will automatically be added to the attributeFile
The example below shows how the locationSet from before, which is based on a location file (AUS_sensors.csv), is filtered using a constraint based on an attribute (AUS_sensors_acceptance.csv).
For the remaining attributeFiles, we want to ensure they contain all locations in the (filtered/constraint) locationSet, not all locations listed in the original locationFile.
Since requireAllIdsInLocationSet= true, a new location row will automatically be added to AUS_sensor_idmap.csv (the attributeFile) when a new location is added to the locationSet (e.g. by changing the attribute to satisfy the constraint).
Note: When you configure the <required> element for an attribute, Delft-FEWS will log an ERROR if they are not filled.
Using this option can help you identify missing required meta data more easily.
Code Block |
---|
<constraints>
<attributeTextEquals id="ACCEPTANCE" equals="ACCEPTED"/>
</constraints> |
Automatically accept or reject locations in locationSet
<To be documented>