It is possible to edit archived data in Delft-FEWS and subsequently upload it back to the Open Archive, with a few simple steps. This page describes the configuration required for this.

  • The data is edited in the Time Series Dialog ( / ).
  • The edited data can be uploaded to the Open Archive manually or automatically, depending on the configuration and whether you are using a Stand Alone or Operator client. The User guide on how to upload data to the Open Archive can be found here: 21 Archive display#21Archivedisplay-Uploadtoarchive. Data edits are uploaded in pi-xml format to the webservice of the Open Archive. The webservice will save the uploaded pi-xml files in the folder upload/edit. 
  • The files are processed using a dedicated element in the archive export module.

Table of contents

Prerequisites

We assume the following is available, before you start to add the configuration to allow uploads of edited data to the Archive.

  • Delft-FEWS Open Archive installed
    • Server is running and accessible
    • Basic archive configuration is in place
    • Some data has been archived
  • Stand alone or Operator Client application
  • Time series is editable (timeSeriesType = external historical, and readWriteMode = add originals)
  • Time series is available in a filter
  • User has the right permissions to edit the data
  • Locations defined in a CSV/Shape file in MapLayerFiles (so not in locations.xml)

Config files requiring updates

Once the prerequisites are met, the following files require an update, for each an example will be provided below, both of the file itself and the change required.

Note that names could be different based on your configuration.

The following files should be added:

Admin Interface updates

The new workflow should be scheduled on regular basis so that edited data can be processed. 
Note: The uploaded pi-xml files are processed in 'First in First out' order. 

Note: The procedure of merging data edits into the Open Archive is based on reading the time series of a NetCDF-files into a Delft-FEWS workflow, replacing the edited values and exporting a new NetCDF-file to the archive (version 2025.02). This means that custom attributes which were added in the original NetCDF file will not be exported.

An enhancement to this procedure is scheduled for a later version (tbd).

Config file update examples

MapLayerFiles - Location attributes

Note that the attribute should be available for the location for which you want to edit the data, as well as properly configured in the Locations.csv and LocationsSets.xml

LocationSets.xml changes required
	<locationSet id="DummyLocations">
		<csvFile>
			<file>DummyLocations.csv</file>
			<geoDatum>WGS 1984</geoDatum>
			<id>%FewsId%</id>
			<parentLocationId>%ParentId%</parentLocationId>
			<x>%X%</x>
			<y>%Y%</y>
			<attribute id="ArchiveArea">
				<text>%ArchiveArea%</text>
			</attribute>
		</csvFile>
	</locationSet>

ArchiveModuleDisplay.xml

This is a DisplayConfigFiles/

ArchiveModuleDisplay.xml example
<?xml version="1.0" encoding="UTF-8"?>
<archiveModuleDisplay xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews https://fewsdocs.deltares.nl/schemas/version1.0/archiveModuleDisplay.xsd">
	<workFolder>$ArchiveDownloadFolder$/workdir</workFolder>
	<downloadFolders>
		<downloadFolderObserved>$ArchiveDownloadFolder$/observed</downloadFolderObserved>
		<downloadFolderSimulated>$ArchiveDownloadFolder$/simulated</downloadFolderSimulated>
		<downloadFolderSimulatedHistorical>$ArchiveDownloadFolder$/simulatedhistorical</downloadFolderSimulatedHistorical>
		<downloadFolderExternalForecasts>$ArchiveDownloadFolder$/externalforecasts</downloadFolderExternalForecasts>
		<downloadFolderRatingCurves>$ArchiveDownloadFolder$/ratingcurves</downloadFolderRatingCurves>
		<downloadFolderHistoricalEvents>$ArchiveDownloadFolder$/historicalevents</downloadFolderHistoricalEvents>
		<downloadFolderForecasterNotes>$ArchiveDownloadFolder$/messages</downloadFolderForecasterNotes>
		<downloadFolderConfiguration>$ArchiveDownloadFolder$/configuration</downloadFolderConfiguration>
		<downloadFolderEventAttachments>$ArchiveDownloadFolder$/attachments</downloadFolderEventAttachments>
	</downloadFolders>
	<archiveImportWorkflowId>Archive_Import_Data</archiveImportWorkflowId>
	<hideArchiveUploadTab>false</hideArchiveUploadTab>
	<archiveUploadIdMapId>IdMapToArchive</archiveUploadIdMapId>
</archiveModuleDisplay>

In the ArchiveModuleDisplay the following elements should be added:

ArchiveModuleDisplay.xml changes required
 	<hideArchiveUploadTab>false</hideArchiveUploadTab>
	<archiveUploadIdMapId>IdMapToArchive</archiveUploadIdMapId>

Archives.xml

This is a SystemConfigFiles/

Archives.xml example
<?xml version="1.0" encoding="UTF-8"?>
<archives xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews https://fewsdocs.deltares.nl/schemas/version1.0/archives.xsd">
	<enabled>$ArchiveEnabled$</enabled>
	<catalogueUrl>$ArchiveUrl$/deltares-archive-server/catalogue</catalogueUrl>
	<fileServerUrl>$ArchiveUrl$/thredds/fileServer/data</fileServerUrl>
	<elasticSearchUrl>$ArchiveUrl$/deltares-archive-server/elastic</elasticSearchUrl>
	<areaLocationAttributeFunction>@ArchiveArea@</areaLocationAttributeFunction>
	<uploadEditedDataToOpenArchive>true</uploadEditedDataToOpenArchive>
</archives>

In the Archives.xml the following configuration needs to be added:

Archives.xml changes required
	<areaLocationAttributeFunction>@ArchiveArea@</areaLocationAttributeFunction>

Optionally the UploadEditedData element can be added to automatically update the data when editing in an OC.

Archives.xml changes optional
	<uploadEditedDataToOpenArchive>true</uploadEditedDataToOpenArchive>

ModuleConfigFiles/MergeEdits.xml

A configuration example for the ModuleConfigFile to merge the edited data can be found below:

New ModuleConfigFile MergeEditx.xml required
<?xml version="1.0" encoding="UTF-8"?>
<exportArchiveModule xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews https://fewsdocs.deltares.nl/schemas/version1.0//exportArchiveModule.xsd">
	<mergeEditedObservedData>
		<archiveFolder>$ArchiveFolder$</archiveFolder>
	</mergeEditedObservedData>
</exportArchiveModule>

The <archiveFolder> should contain the location of the ArchiveFolder at the server which runs the FSS, we strongly recommend using global.properties to set the ArchiveFolder. 

Don't forget to list the moduleInstance in the ModuleInstanceDescriptors (we recommend using a CSV for this).

WorkflowFiles/Merge_Edits.xml

A configuration example for the WorkflowFile is shown below:

New WorkflowFile Merge_Edits.xml required
<?xml version="1.0" encoding="UTF-8"?>
<workflow xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews https://fewsdocs.deltares.nl/schemas/version1.0/workflow.xsd" version="1.1">
	<activity>
		<runIndependent>true</runIndependent>
		<moduleInstanceId>MergeEdits</moduleInstanceId>
	</activity>
</workflow>

RegionConfigFiles

List the new moduleInstance and workflow in respectively ModuleInstanceDescriptors.xml and WorkflowDescriptors.xml:

ModuleInstanceDescriptors changes required
<moduleInstanceDescriptor id="MergeEdits"/>
WorkflowFileDescriptors changes required
<workflowDescriptor id="Merge_Edits" name="Merge uploaded edits of archived data" visible="true"/>


  • No labels

Disclaimer