Overview

The DataWare import function (<importType>DataWare</importType>) imports data from REST services that return XML content.

The REST Service URLs look as follows http://servername:portnumber/rest/read/data/parameter/startdate/enddate/location

Configuring the Import

To make the importer known to FEWS, the import module has to be declared in the moduleInstanceDescriptors.xml in the RegionConfigFiles directory of the FEWS configuration:

<?xml version="1.0" encoding="UTF-8"?>
<moduleInstanceDescriptors xmlns="http://www.wldelft.nl/fews"
		xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
		xsi:schemaLocation="http://www.wldelft.nl/fews
		http://fews.wldelft.nl/schemas/version1.0/moduleInstanceDescriptors.xsd" version="1.0">
...
	<moduleInstanceDescriptor id="ImportDataWare">
		<moduleId>TimeSeriesImportRun</moduleId>
	</moduleInstanceDescriptor>
...
</moduleInstanceDescriptors>

 

An example of the DataWare import configuration will be given here. The importType is named DataWare which should be configured in the general section of the import. De URL to the rest service should be configured here as well. The actual data to be retrieved can be tuned by adding setting the relativeViewPeriod and the timeSeries to specify the locations and parameters. N.B.: the used parameters and locations have to be mapped with an idMap (IdImportDataWare in this example). Since the DataWare parser specifies some custom quality flags, a flag conversion map can be specified in the flagConversionId to map flags to FEWS flags.
TimeZone is assumed to be configured in the general section, otherwise GMT is assumed.

If no relativeViewPeriod is configured, all data is requested for a parameter and location.

<?xml version="1.0" encoding="UTF-8"?>
<timeSeriesImportRun xmlns="http://www.wldelft.nl/fews" 
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
	xsi:schemaLocation="http://www.wldelft.nl/fews 
	http://fews.wldelft.nl/schemas/version1.0/timeSeriesImportRun.xsd">
	<import>
		<general>
			<importType>DataWare</importType>
			<serverUrl>http://server:port/rest/read/data/</serverUrl>
			<connectionTimeOutMillis>20000</connectionTimeOutMillis>
			<relativeViewPeriod unit="day" start="-1" end="0" startOverrulable="true" endOverrulable="true"/>
			<idMapId>IdImportDataWare</idMapId>
			<flagConversionsId>ImportFlagConversions</flagConversionsId>			
			<importTimeZone>
				<timeZoneName>GMT-5</timeZoneName>
			</importTimeZone>
		</general>
		<timeSeriesSet>
			<moduleInstanceId>ImportDataWare</moduleInstanceId>
			<valueType>scalar</valueType>
			<parameterId>H.obs</parameterId>
			<locationId>Amsterdam</locationId>
			<timeSeriesType>external historical</timeSeriesType>
			<timeStep unit="nonequidistant" />
			<readWriteMode>add originals</readWriteMode>
		</timeSeriesSet>
		<timeSeriesSet>
			<moduleInstanceId>ImportDataWare</moduleInstanceId>
			<valueType>scalar</valueType>
			<parameterId>H.obs</parameterId>
			<locationId>Delft</locationId>
			<timeSeriesType>external historical</timeSeriesType>
			<timeStep unit="nonequidistant" />
			<readWriteMode>add originals</readWriteMode>
		</timeSeriesSet>
		<timeSeriesSet>
			<moduleInstanceId>ImportDataWare</moduleInstanceId>
			<valueType>scalar</valueType>
			<parameterId>Q.obs</parameterId>
			<locationId>Amsterdam</locationId>
			<timeSeriesType>external historical</timeSeriesType>
			<timeStep unit="nonequidistant" />
			<readWriteMode>add originals</readWriteMode>
		</timeSeriesSet>
		<timeSeriesSet>
			<moduleInstanceId>ImportDataWare</moduleInstanceId>
			<valueType>scalar</valueType>
			<parameterId>Q.obs</parameterId>
			<locationId>Delft</locationId>
			<timeSeriesType>external historical</timeSeriesType>
			<timeStep unit="nonequidistant" />
			<readWriteMode>add originals</readWriteMode>
		</timeSeriesSet>
	</import>
</timeSeriesImportRun>

The previous timeseries will be mapped using theIdImportDataWare id mapping:

 
	<parameter external="19839" internal="H.obs" />
	<parameter external="19840" internal="Q.obs" />
	<location external="1ha" internal="Amsterdam" />
	<location external="2ha" internal="Delft" />
 

The previous configuration will result in the following 4 REST urls being used:

 
http://server:port/rest/read/data/19839/2015-01-04/2015-01-05/1ha
http://server:port/rest/read/data/19839/2015-01-04/2015-01-05/1ha
http://server:port/rest/read/data/19840/2015-01-04/2015-01-05/2ha
http://server:port/rest/read/data/19840/2015-01-04/2015-01-05/2ha
 
 

In case no relative view period was configured, a typical URL will look as follows:

 
http://server:port/rest/read/data/19839/*/*/1ha
 

Example XML response from DataWare service

 

<ArrayOfData xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<Data><Key>19839</Key><Time>2014-11-01 00:00:00.000</Time><Value>58.2457</Value><Quality>29</Quality></Data>
...

 

Java source code

DataWareTimeSeriesParser.java

  • No labels

Disclaimer