| Table of Contents |
|---|
Introduction
Currently, the only graphical user interface to access the Deltares Open Archive is via the Delft-FEWS GUI. In addition, data can be approached via the Geonetwork web client of via THREDDS web server.
...
This user guide is derived from the HyFS application as used at the Bureau of Meteorology.
Archived data
In the Delft-FEWS application, scheduled and manual archive workflows can take care of archiving the data (sets) from the database. The scheduled archive workflow is generally run once a day and will generally archive the following type of data:
...
Note: External data which is archived elsewhere should in general not be archived within the context of the Delft-FEWS application.
Events
Within Forecasting organisations, events are situations that can be distinguished from business as ususal as something special or interesting is happening, e.g. a flood, drought or calamity. Generally the operations during an event is hectic and no one wants to spent time on data archiving. however, after the event the archived data may be crucial for legal inquiries, performance assessment, or model calibration or to use as input for training. Therefore, the concept of Event has been introduced to label a period of interest for a certain area, such that one easily can find this data back.
...
Finally, events can play a crucial role in the data management of the archive. Data labelled for a specific type of event may be help longer in the archive compared to data not labelled for an event. E.g. Numerical Weather Predictions may be removed quickly, except for those external forecast that are part of a Flood Watch or training event.
Archive display
All data that is archived can be accessed and downloaded through the Delft-FEWS archive display. The archive display can be started from the Delft-FEWS Explorer from the Tools menu.
...
In the following paragraphs the functionality of the different tabs is explained.
Create an archive event
Events are created for Areas and are valid for a period; specified by the start and end time.
...
4 Press the search button on the right part of the display to check if there is any data in the archive for the selected area and period.
| Info | ||
|---|---|---|
| ||
The location and parameter list buttons below the search button allow you to select locations and parameters. This selection is only relevant when defining a historical event. The dataset that is created will always contain all timeserieSets. But in the case of a historical event, a list with selected parameters/locations will be included. When the dataset is imported as historical event, the locations/parameters will be used to filter the data |
5 Press the save event button to save the event file on the local computer. The event will be added as a new line in the event list table in red.
6 Press the upload event button to upload the event file from the local computer to the archive server.
Figure Delft-FEWS Archive display; create new event
...
3 Press the upload event button to upload the event file from the local computer to the archive server.
Search and Download archive events
The Deltares archive will contain many events for all catchments. The events can be searched and downloaded in the Search for Events tab.
...
Note: By double clicking one of the cells in the event list, the event list will be filtered and show only those events that have the same content of the selected cell. To filter for example on historic events, double click in the event type cell where the content is historic event. All cells in that event table column will now have a blue background color indicating there is a filter active. To remove the filter, double click again on one of the event type cells.
Figure Delft-FEWS Archive display; search for events (filtered by flood event using double click)
Search and Download archive data
The Deltares archive will contain data that is not always included in an archive event file. If the data is not included in an event file it can be downloaded in the Search in archive tab.
...
- Select the area
- Set the search start and end times.
- Select the type of dataset
- Select the source of the data (if available, from 2015.01 onwards only)
- Press the search button
- Select one or multiple data sets from the list for download
- Press the download and import data button to download the data from the archive server and import into the local datastore. If do not want the data imported, you can also press the download (only) button.
- Whatif template name is only active if dataset type is simulated forecasting. This is a dropdown window will all whatif templates configured. You can filter your search results by choosing an id.
- Whatif senarion name is only active if dataset type is simulated forecasting. You can filter your search results by typing the name of the scenario you are searching for.
- Data search property windows are only available if they are configured (see example below.) The dopdowns are only active if dataset type is simulated forecasting. You can configure no more than two search properties.
Figure Delft-FEWS Archive display; search in archive and download data
Note: After selecting an area and period make sure to press the search in archive data before downloading the files from the archive server. Especially the National area can contain many external forecast data files and the size of the files can be considerable.
Importing archived data into Delft-FEWS stand alone application
If you did not import the data directly as part of the download, you can import the data in the Delft-FEWS Stand Alone system with use of the Manual Forecast display. To import downloaded data; take the following steps:
...
If you want to use the data within a different context, you may also copy the data to the normal Import folders, and setup a normal timeSeriesImport run module with idmapping to bring the data into the Delft-FEWS stand alone application registered under your timeSeriesImport run module.
...
.
| Info | ||
|---|---|---|
| ||
When multiple historical simulations are imported from the archive, it is not possible to switch between these simulations in a similar manner as can be done in the original system, by making a previous simulation “Current". When importing multiple historical simulations, each newly imported simulation will overwrite the results of a previous simulation, similar to normal observations. Therefore, only import the historical simulations that are really needed. When the import is needed to retrieve the warm state that was used to create a forecast, only import that historical simulation. If the intention is to rerun a historical simulation, first import the historical simulation that provided the warm state for the historical simulation of interest. Then import that historical simulation itself. |
Using archived configurations
These are the 3 ways you can make use of an archived configuration:
- ArchiveDisplay “download and import” button can be used from version 2020.02. It downloads and launches the selected config as a new SA application.
- Menu item "Load latest archive configuration" has two steps : it downloads automatically the latest archived config and then replaces the current config with that downloaded latest config.
- You can use a config downloaded through the ArchiveDisplay in your current SA config, by closing that SA and replacing the config manually. There is currently no way to do this automatically like with the latest config (see previous item)
Archive Database
A mongo database plugin can be added to the archives to store scalar data. This tab is only available if you added a <archiveDatabase> to your config. To search for or download data from the database, different filters can (or must be used). The list of areas available is loaded from the configuration.
When the user selects both an area and a time series type, the list of sources available will be populated. When a source is selected, the available time steps, parameters and module instance ids will be loaded from the database.
To search for data sets, area, time series type and source need to be selected. You can also filter the results by time steps, parameters and module instance ids, but these fields aren’t mandatory: if nothing is selected, data for all of the available options will be included in the search result. To search for data, klick the magnifying glass on the left side of the panel.
The summary of the available data sets will be displayed in the text field at the bottom of the panel.
You can download the selected data by clicking the download button (under the search button). When the download is complete, the summary will be logged and also displayed in the panel.
When you are downloading data from the search and download datasets panel, if part or all of the time series are stored in the mongo database, it will be automatically downloaded along with the data stored in the open archive.
Config example:
| Code Block | ||
|---|---|---|
| ||
<?xml version="1.0" encoding="UTF-8"?>
<archives xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/archives.xsd">
<catalogueUrl>url</catalogueUrl>
<elasticSearchUrl>url</elasticSearchUrl>
<archiveDatabase>
<archiveDatabaseUrl>url</archiveDatabaseUrl>
<user>user</user>
<password>pass</password>
<pluginClass>nl.fews.archivedatabase.mongodb.MongoDbArchiveDatabase</pluginClass>
<pluginFactoryMethod>create</pluginFactoryMethod>
<openDatabasePluginPanelBinDir>d:\mongdb-plugin\</openDatabasePluginPanelBinDir>
</archiveDatabase>
</archives> |