Versions Compared

Old Version 61

changes.mady.by.user Unknown User (schelle@wldelft.nl)

Saved on

compared with

New Version 62

changes.mady.by.user Unknown User (schelle@wldelft.nl)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Contents

Table of Contents

Summary

This document contains a step by step guide how to import data in PI XML format into Delft-Fews. This document does NOT describe all the possible options of the Import module; this information can be found in the Delft-Fews configuration guide.

...

  1. The file-type you want to import should be readable by one of the imports available in Delft-Fews
  2. You must configure an instance of this import module
  3. You must add the moduleInstance to a workflow (usually the ImportExternal Workflow)
  4. In case the locationId's and parameterId's in your Delft-Fews configuration are not identical to those in the files you need to import a IdMap file should be set up and referred to in the import module
  5. You will probably need to adjust one or more locationsets, the filters.xml file and the DisplayGroups.xml file if you need to display or process the newly imported data.
    Info

    Please note that Delft-Fews DELETES all files after importing them. As such, you should ALWAYS copy files from another directory to the location from which Delft-Fews imports files. If you do not do this you might loose you files.

Setting up the import module

Setting up the Import module to import data in PI XML format involves the following steps: 

  1. make the Configuration file for the import module and store in the ModuleConfigFiles directory. In this example it should be named Import 1.00 default.xml
  2. register this new module in the ModuleInstanceDescriptors 1.00 default.xml file in the RegionConfigFiles directory
  3. Add the new module at a workflow in order to run it.

Step 1: Configure the import module (create the Import 1.00 default.xml configuration File)

The first step is to configure an instance of the import module for the type of data you need to import. In this example we will import data in PI XML format for 4 locations. The file we want to import is shown below:

...

  • moduleInstanceId - Name of this configuration instance, reflects the XML file name
  • valueType - scalar, this is scalar data
  • parameterId - H.obs - This is the parameterId in Delft-Fews. The parameter Id in the XML file is WaterLevel. The Id mapping (shown below) will map Waterlevel to H.obs
  • locationSetId - In order to simplify configuration we have defined a list of locations (a locationset) in the file LocationSets.xml (in the RegionConfigFiles directory). This set holds the following four locations: one, two, three, fours. As these are different from the locations in the XML file to be imported id mapping will be set up.
  • timeStep - the timestep defined here should map the timestep in the XML files to be imported.

Step 2: Register the new module - Add to the ModuleInstanceDescriptors 1.00 default.xml file

All moduleInstances must be registered in Delft-Fews. This is done by adding en entry in the moduleInstanceDescriptors file in the RegionConfigFiles section of the configuration. The moduleInstancedescriptors file shown below includes this appropriate entry:

...

No Format
<?xml version="1.0" encoding="UTF-8"?>
<moduleInstanceDescriptors xmlns="http://www.wldelft.nl/fews"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews
http://fews.wldelft.nl/schemas/version1.0/moduleInstanceDescriptors.xsd"
version="1.0">
	<!--Import Module Instances-->
	<moduleInstanceDescriptor id="Import">
		<description>Import Our new data</description>
		<moduleId>TimeSeriesImportRun</moduleId>
	</moduleInstanceDescriptor>
</moduleInstanceDescriptors>

Step 3: Add the module to a workflow (ImportExternal 1.00 Default.xml)

In order to use the module (to be able to run it from the manual forecast dialog) you will need to list the module as an activity in a workflow. Assuming this workflow is already registered (see the Delft-Fews configuration manual on how to register workflow) the only thing to do is to add in to the workflow. In the following example the new module is the only activity in the workflow:

No Format
<?xml version="1.0" encoding="UTF-8"?>
<workflow xmlns="http://www.wldelft.nl/fews"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.wldelft.nl/fews
http://fews.wldelft.nl/schemas/version1.0/workflow.xsd" version="1.1">
	<activity>
		<runIndependent>true</runIndependent>
		<moduleInstanceId>Import</moduleInstanceId>
	</activity>
</workflow>

Step 4: Make the locationSet 

In order to refer to all four locations in one go (Instead of having to create four different timeseriesset) a locationset is defined in the file LocationSets 1.00 Default.xml in the RegionConfigFiles directory:

No Format
<?xml version="1.0" encoding="UTF-8"?>
<locationSets version="1.1"
xmlns="http://www.wldelft.nl/fews"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.wldelft.nl/fews
http://fews.wldelft.nl/schemas/version1.0/locationSets.xsd">
	<locationSet id="LevelGauges" name="LevelGauges">
		<locationId>one</locationId>
		<locationId>two</locationId>
		<locationId>three</locationId>
		<locationId>four</locationId>
	</locationSet>

</locationSets>

Set up Id Mapping

Step 5: Make the idmap file IdImport 1.00.xml

 The format of the IdMap files is described in the DELFT-FEWS configuration guid and also in the .xsd scheme. All idMaps are stored in the IdMapFiles directory.  The Idmap needed for this example (IdImport 1.00 Default.xml) should match the parameter WaterLevel to H.obs and should also map the four locations. The files is shown below:

No Format
<?xml version="1.0" encoding="UTF-8"?>
<idMap xmlns="http://www.wldelft.nl/fews"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.wldelft.nl/fews
http://fews.wldelft.nl/schemas/version1.0/idMap.xsd" version="1.1">
	<parameter internal="H.obs" external="WaterLevel"/>
	<location internal="one" external="LocA"/>
	<location internal="two" external="LocB"/>
	<location internal="three" external="LocC"/>
	<location internal="four" external="LocD"/>
</idMap>

Step 6: register the idmap file

Similar to ModuleInstances, IdMaps should also be registered. In this case we need to register the IdImport file in the file IdMapDescriptors in the RegionConfigFiles directory:

No Format
<?xml version="1.0" encoding="UTF-8"?>
<idMapDescriptors xmlns="http://www.wldelft.nl/fews"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.wldelft.nl/fews
http://fews.wldelft.nl/schemas/version1.0/idMapDescriptors.xsd" version="1.0">
	<idMapDescriptor id="IdImport"/>
</idMapDescriptors>

Setting up a (temporary) display to view the imported data

Delft-Fews only shows data that is configured to be visible to the user. All other data is invisible to the user. The file DisplayGroups.xml (located in the SystemConfigFiles directory) defines pre-defined graphs. In the example below five graphs have been defined to view the newly imported data:

No Format
<?xml version="1.0" encoding="UTF-8"?>
<displayGroups version="1.0"
xmlns="http://www.wldelft.nl/fews"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.wldelft.nl/fews
http://fews.wldelft.nl/schemas/version1.0/displayGroups.xsd">
	<displayGroup name="Test">
		<display name="Gauge One">
			<relativeViewPeriod unit="hour" start="-12" end="36"/>
			<subplot>
				<timeSeriesSet>
					<moduleInstanceId>Import</moduleInstanceId>
					<valueType>scalar</valueType>
					<parameterId>H.obs</parameterId>
					<locationId>one</locationId>
					<timeSeriesType>external historical</timeSeriesType>
					<timeStep unit="hour" multiplier="1"/>
					<relativeViewPeriod unit="hour" end="48" start="-48"/>
					<readWriteMode>add originals</readWriteMode>
				</timeSeriesSet>
			</subplot>
		</display>
		<display name="Gauge Two">
			<relativeViewPeriod unit="hour" start="-12" end="36"/>
			<subplot>
				<timeSeriesSet>
					<moduleInstanceId>Import</moduleInstanceId>
					<valueType>scalar</valueType>
					<parameterId>H.obs</parameterId>
					<locationId>two</locationId>
					<timeSeriesType>external historical</timeSeriesType>
					<timeStep unit="hour" multiplier="1"/>
					<relativeViewPeriod unit="hour" end="48" start="-48"/>
					<readWriteMode>add originals</readWriteMode>
				</timeSeriesSet>
			</subplot>
		</display>
		<display name="Gauge Three">
			<relativeViewPeriod unit="hour" start="-12" end="36"/>
			<subplot>
				<timeSeriesSet>
					<moduleInstanceId>Import</moduleInstanceId>
					<valueType>scalar</valueType>
					<parameterId>H.obs</parameterId>
					<locationId>three</locationId>
					<timeSeriesType>external historical</timeSeriesType>
					<timeStep unit="hour" multiplier="1"/>
					<relativeViewPeriod unit="hour" end="48" start="-48"/>
					<readWriteMode>add originals</readWriteMode>
				</timeSeriesSet>
			</subplot>
		</display>
		<display name="Gauge Four">
			<relativeViewPeriod unit="hour" start="-12" end="36"/>
			<subplot>
				<timeSeriesSet>
					<moduleInstanceId>Import</moduleInstanceId>
					<valueType>scalar</valueType>
					<parameterId>H.obs</parameterId>
					<locationId>four</locationId>
					<timeSeriesType>external historical</timeSeriesType>
					<timeStep unit="hour" multiplier="1"/>
					<relativeViewPeriod unit="hour" end="48" start="-48"/>
					<readWriteMode>add originals</readWriteMode>
				</timeSeriesSet>
			</subplot>
		</display>
		<display name="All together">
			<relativeViewPeriod unit="hour" start="-12" end="36"/>
			<subplot>
				<timeSeriesSet>
					<moduleInstanceId>Import</moduleInstanceId>
					<valueType>scalar</valueType>
					<parameterId>H.obs</parameterId>
					<locationSetId>LevelGauges</locationSetId>
					<timeSeriesType>external historical</timeSeriesType>
					<timeStep unit="hour" multiplier="1"/>
					<relativeViewPeriod unit="hour" end="48" start="-48"/>
					<readWriteMode>add originals</readWriteMode>
				</timeSeriesSet>
			</subplot>
		</display>
	</displayGroup>
</displayGroups>

Add to the appropriate filters(s)

To be able to use the main map display and the filters to access the data you should set up an entry in the Filters.xml file in the RegionConfigFiles directory. See the example below:

No Format
<?xml version="1.0" encoding="UTF-8"?>
<filters version="1.1"
xmlns="http://www.wldelft.nl/fews"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.wldelft.nl/fews
http://fews.wldelft.nl/schemas/version1.0/filters.xsd">
	<filter name="Gauges" id="Gauges">
		<timeSeriesSet>
			<moduleInstanceId>Import</moduleInstanceId>
			<valueType>scalar</valueType>
			<parameterId>H.obs</parameterId>
			<locationSetId>LevelGauges</locationSetId>
			<timeSeriesType>external historical</timeSeriesType>
			<timeStep unit="hour" multiplier="1"/>
			<relativeViewPeriod unit="hour" end="48" start="-48"/>
			<readWriteMode>add originals</readWriteMode>
		</timeSeriesSet>
	</filter>
</filters>

Test the import

To test the import run the ImportExternal Workflow. Check to log panel for error messages. If you want to view the data please make sure to first set the system time to a time/date you have data for. In this example August 23 2006.