Summary
Importing data in DELFT-FEWS is different from running a wizard in e.g. a much used office software package. To be able to import data into the system the following requirements should be met:
- The file-type you want to import should be readable by one of the imports available in Delft-Fews
- You must configure an instance of this import module
- You must add the moduleInstance to an workflow (usually tne ImportExternal Workflow)
- In case the locationId's and parameterId's in your Delft-Fews configuration are not identical to those in the files you need to import a IdMap file should be set up and referred to in the import module
- You will probably need to adjust one or more locationsets, the filters.xml file and the DisplayGroups.xml file if you need to display or process the newly imported data types.
Please note thet Delft-Fews DELETES all files after importing them. As such, you should ALWAYS copy files from another directory to the location from which Delft-Fews imports files. If you do not do this you might loose you files.
Setting up the import module
The first step is to configure an instance of the import module for the type of data you need to import. In this example we will import data in PI XML format for 4 locations. The file we want to import is shown below:
<?xml version="1.0" encoding="UTF-8"?> <TimeSeries xmlns="http://www.wldelft.nl/fews/PI" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews/PI http://fews.wldelft.nl/schemas/version1.0/pi-schemas/pi_timeseries.xsd" version="1.2"> <series> <header> <type>instantaneous</type> <locationId>locA</locationId> <parameterId>WaterLevel</parameterId> <timeStep unit="second" multiplier="3600"/> <startDate date="2006-08-23" time="17:00:00"/> <endDate date="2006-08-24" time="00:00:00"/> <missVal>-8888.0</missVal> <longName>locA</longName> <units>m</units> </header> <event date="2006-08-23" time="17:00:00" value="8.66"/> <event date="2006-08-23" time="18:00:00" value="9.66"/> <event date="2006-08-23" time="19:00:00" value="8.66"/> <event date="2006-08-23" time="20:00:00" value="-8888.0"/> <event date="2006-08-23" time="21:00:00" value="3"/> <event date="2006-08-23" time="22:00:00" value="3"/> <event date="2006-08-23" time="23:00:00" value="3"/> <event date="2006-08-24" time="00:00:00" value="-8888.0"/> </series> <series> <header> <type>instantaneous</type> <locationId>locB</locationId> <parameterId>WaterLevel</parameterId> <timeStep unit="second" multiplier="3600"/> <startDate date="2006-08-23" time="13:00:00"/> <endDate date="2006-08-24" time="00:00:00"/> <missVal>-999.0</missVal> <longName>locB</longName> <units>m</units> </header> <event date="2006-08-23" time="13:00:00" value="0.63"/> <event date="2006-08-23" time="14:00:00" value="0.61"/> <event date="2006-08-23" time="15:00:00" value="0.59"/> <event date="2006-08-23" time="16:00:00" value="0.59"/> <event date="2006-08-23" time="17:00:00" value="0.59"/> <event date="2006-08-23" time="18:00:00" value="0.59"/> <event date="2006-08-23" time="19:00:00" value="0.60"/> <event date="2006-08-23" time="20:00:00" value="0.57"/> <event date="2006-08-23" time="21:00:00" value="0.59"/> <event date="2006-08-23" time="22:00:00" value="0.59"/> <event date="2006-08-23" time="23:00:00" value="-999.0"/> <event date="2006-08-24" time="00:00:00" value="-999.0"/> </series> <series> <header> <type>instantaneous</type> <locationId>LocC</locationId> <parameterId>WaterLevel</parameterId> <timeStep unit="second" multiplier="3600"/> <startDate date="2006-08-23" time="13:00:00"/> <endDate date="2006-08-24" time="00:00:00"/> <missVal>-999.0</missVal> <longName>LocC</longName> <units>m</units> </header> <event date="2006-08-23" time="13:00:00" value="-1.42"/> <event date="2006-08-23" time="14:00:00" value="-1.42"/> <event date="2006-08-23" time="15:00:00" value="-1.44"/> <event date="2006-08-23" time="16:00:00" value="-1.46"/> <event date="2006-08-23" time="17:00:00" value="-1.46"/> <event date="2006-08-23" time="18:00:00" value="-1.46"/> <event date="2006-08-23" time="19:00:00" value="-1.48"/> <event date="2006-08-23" time="20:00:00" value="-1.48"/> <event date="2006-08-23" time="21:00:00" value="-1.48"/> <event date="2006-08-23" time="22:00:00" value="-1.49"/> <event date="2006-08-23" time="23:00:00" value="-999.0"/> <event date="2006-08-24" time="00:00:00" value="-999.0"/> </series> <series> <header> <type>instantaneous</type> <locationId>LocD</locationId> <parameterId>WaterLevel</parameterId> <timeStep unit="second" multiplier="3600"/> <startDate date="2006-08-23" time="13:00:00"/> <endDate date="2006-08-24" time="00:00:00"/> <missVal>-999.0</missVal> <longName>LocD</longName> <units>m</units> </header> <event date="2006-08-23" time="13:00:00" value="-1.42"/> <event date="2006-08-23" time="14:00:00" value="-1.42"/> <event date="2006-08-23" time="15:00:00" value="-1.44"/> <event date="2006-08-23" time="16:00:00" value="-1.46"/> <event date="2006-08-23" time="17:00:00" value="-1.46"/> <event date="2006-08-23" time="18:00:00" value="-1.46"/> <event date="2006-08-23" time="19:00:00" value="-1.48"/> <event date="2006-08-23" time="20:00:00" value="-1.48"/> <event date="2006-08-23" time="21:00:00" value="-1.48"/> <event date="2006-08-23" time="22:00:00" value="-1.49"/> <event date="2006-08-23" time="23:00:00" value="-999.0"/> <event date="2006-08-24" time="00:00:00" value="-999.0"/> </series> </TimeSeries>
When configuring the import module we will need to extract the following information from the file:
- the locationId's in the XML file
- the parameterID's in the XML file
- the timestep in the XML file
This information is needed to configure the TimeseriesSet in the import module. A quick look at the files shows that there are four locations: LocA, LocB, LocC and LocD. Only one parameter is present (WaterLevel) and the data comes in hourly timesteps.
Another type of information that we will need is the location at which Delft-Fews can find to files to be imported. For this example we assume the files will be located in c:/Import/PI .
Below the configuration XML file for the import module that will read the files is shown:
<?xml version="1.0" encoding="UTF-8"?>
<!-- edited with XMLSpy v2006 rel. 3 sp1 (http://www.altova.com) by Jaap S (WL | Delft Hydraulics) -->
<timeSeriesImportRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews
http://fews.wldelft.nl/schemas/version1.0/timeSeriesImportRun.xsd">
<import>
<general>
<importType>PI</importType> <!-- Indicated we will read PI Files -->
<folder>c:/Import/PI</folder> <!-- read the files from here -->
<!-- Files WILL BE DELETED after import -->
<failedFolder>c:/Backup/PI</failedFolder>
<!-- If specified the imported files will be copied to this location -->
<idMapId>IdImport</idMapId> <!-- The IdMap for this import -->
<importTimeZone>
<timeZoneOffset>+00:00</timeZoneOffset>
</importTimeZone>
</general>
<timeSeriesSet>
<moduleInstanceId>Import/moduleInstanceId> <!-- The name of this moduleinstance -->
<valueType>scalar</valueType>
<parameterId>H.obs</parameterId>
<locationSetId>LevelGauges</locationSetId>
<timeSeriesType>external historical</timeSeriesType>
<timeStep unit="hour" multiplier="1"/>
<readWriteMode>add originals</readWriteMode>
<synchLevel>1</synchLevel>
</timeSeriesSet>
</import>
</timeSeriesImportRun>
