Overview

The Open Geospatial consortium has developed the concept of Sensor Web Enablement to make sensor data available for the web. A number of services have been specified including associated XML-formats. The most important XML-specification for FEWS seems to be the Observations and Measurements model (http://portal.opengeospatial.org/files/?artifact_id=17038&version=3 ).

The following actions were taken:

  • Investigation of the xml model and interface to determine which xml files/objects should be supported to implement the observation and measurement model
  • Implementation of a SweTimeSeriesParser to test whether import from a URL fits the current import strategy of FEWS.

XML model investigation
Object/Model mapping
The SWE model depends on other models like Geographic Markup Language (GML), Geographic MetaData (GMD), OGC Web Services (OWS) and the Open Geospatial Consortium (OGC) basic model itself. The number of xml files needed, to support a generic object/model mapping for SWE, will involve hundreds of xml files. In FEWS, Castor is used to generate java classes for object/model mapping. A test was performed to see whether Castor could generate code for a single SWE file; observation.xml. Castor did not succeed due to the embedded references to external XML files.

O & M Interface investigation
During investigation of the xml model it was found that the model supports a range of requests and responses to obtain actually the same data. Although quite readable for human, an automated process like a time series parser will have a hard time investigating responses and determining which data it received.

Before use!

During tests we managed to get servers to hang because the amount of XML data, that had to be returned by the server, caused OutOfMemory exceptions. This occurred even for small queries (short period of just a few days and only one location and parameter) . To avoid these problems, please ensure that you've tested your queries on the target service, and adjust your relative view period appropriately.

To avoid these memory exceptions, we've adjusted the import module so that it will send a request to the SWE service for every single location, parameter combination in the IdMap file. A drawback of this approach is that the performance of the import has slowed down.

Configuration (Example)

In order to import data from a swe enabled service one has to configure the following items:

  • The url at which the swe service is listening
  • The mapping between internal locations and parameters to swe known offerings, procedures and observerved properties

The url can be configured in the moduleConfigFile for the desires SWE import. Use the serverUrl tag in the general section as described in the following example:

ImportSwe.xml
<?xml version="1.0" encoding="UTF-8"?>
<timeSeriesImportRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/timeSeriesImportRun.xsd">
  <import>
    <general>
     <importType>SWE</importType>
      <serverUrl>http://swe.knmi.nl:8080/52nSOSv2kmds/sos</serverUrl>
      <relativeViewPeriod unit="week" start="-100" end="0" />
      <idMapId>IdImportKNMI_SWE</idMapId>
      <unitConversionsId>ImportUnitConversions</unitConversionsId>
      <importTimeZone>
        <timeZoneOffset>+01:00</timeZoneOffset>
      </importTimeZone>
      <dataFeedId>tnonitg</dataFeedId>
    </general>
    <timeSeriesSet>
      <moduleInstanceId>ImportKNMI_SWE</moduleInstanceId>
      <valueType>scalar</valueType>
      <parameterId>P.meting</parameterId>
      <locationSetId>KNMI-sensors(SWE)</locationSetId>
      <timeSeriesType>external historical</timeSeriesType>
      <timeStep unit="nonequidistant" />
      <relativeViewPeriod unit="day" start="-5" end="0" startOverrulable="true"/>
      <readWriteMode>add originals</readWriteMode>
      <synchLevel>1</synchLevel>
    </timeSeriesSet>
    <timeSeriesSet>
      <moduleInstanceId>ImportKNMI_SWE</moduleInstanceId>
      <valueType>scalar</valueType>
      <parameterId>LV.meting</parameterId>
      <locationSetId>KNMI-sensors(SWE)</locationSetId>
      <timeSeriesType>external historical</timeSeriesType>
      <timeStep unit="nonequidistant" />
      <relativeViewPeriod unit="day" start="-5" end="0" startOverrulable="true"/>
      <readWriteMode>add originals</readWriteMode>
      <synchLevel>1</synchLevel>
    </timeSeriesSet>
    <timeSeriesSet>
      <moduleInstanceId>ImportKNMI_SWE</moduleInstanceId>
      <valueType>scalar</valueType>
      <parameterId>WS.meting</parameterId>
      <locationSetId>KNMI-sensors(SWE)</locationSetId>
      <timeSeriesType>external historical</timeSeriesType>
      <timeStep unit="nonequidistant" />
      <relativeViewPeriod unit="day" start="-5" end="0" startOverrulable="true"/>
      <readWriteMode>add originals</readWriteMode>
      <synchLevel>1</synchLevel>
    </timeSeriesSet>
    <timeSeriesSet>
      <moduleInstanceId>ImportKNMI_SWE</moduleInstanceId>
      <valueType>scalar</valueType>
      <parameterId>T.meting</parameterId>
      <locationSetId>KNMI-sensors(SWE)</locationSetId>
      <timeSeriesType>external historical</timeSeriesType>
      <timeStep unit="nonequidistant" />
      <relativeViewPeriod unit="day" start="-5" end="0" startOverrulable="true"/>
      <readWriteMode>add originals</readWriteMode>
      <synchLevel>1</synchLevel>
    </timeSeriesSet>
  </import>
</timeSeriesImportRun>

The offerings, procedures and observedProperties are mapped in the idMap file for the swe import. See next example:

IdImportSwe
<?xml version="1.0" encoding="UTF-8"?>
<idMap version="1.1" xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews [http://fews.wldelft.nl/schemas/version1.0/idMap.xsd]">
<parameter internal="P.meting" external="urn:ogc:def:phenomenon:precipitationIntensity" externalQualifier="weatherNL" />
<parameter internal="WS.meting" external="urn:ogc:def:phenomenon:windSpeed" externalQualifier="weatherNL"/>
<parameter internal="T.meting" external="urn:ogc:def:phenomenon:airTemperature" externalQualifier="weatherNL"/>
<parameter internal="LV.meting" external="urn:ogc:def:phenomenon:relativeHumidity" externalQualifier="weatherNL"/>

<\!\--locaties voor KNMI gegevens: level-\->
<location internal="KNMI-6235" external="urn:ogc:object:sensor:KNMI:procedure06235"/>
<location internal="KNMI-6239" external="urn:ogc:object:sensor:KNMI:procedure06239"/>
<location internal="KNMI-6240" external="urn:ogc:object:sensor:KNMI:procedure06240"/>
<location internal="KNMI-6251" external="urn:ogc:object:sensor:KNMI:procedure06251"/>
<location internal="KNMI-6260" external="urn:ogc:object:sensor:KNMI:procedure06260"/>
<location internal="KNMI-6269" external="urn:ogc:object:sensor:KNMI:procedure06269"/>
<location internal="KNMI-6270" external="urn:ogc:object:sensor:KNMI:procedure06270"/>
<location internal="KNMI-6275" external="urn:ogc:object:sensor:KNMI:procedure06275"/>
<location internal="KNMI-6280" external="urn:ogc:object:sensor:KNMI:procedure06280"/>
<location internal="KNMI-6283" external="urn:ogc:object:sensor:KNMI:procedure06283"/>
<location internal="KNMI-6290" external="urn:ogc:object:sensor:KNMI:procedure06290"/>
<location internal="KNMI-6310" external="urn:ogc:object:sensor:KNMI:procedure06310"/>
<location internal="KNMI-6344" external="urn:ogc:object:sensor:KNMI:procedure06344"/>
<location internal="KNMI-6356" external="urn:ogc:object:sensor:KNMI:procedure06356"/>
<location internal="KNMI-6370" external="urn:ogc:object:sensor:KNMI:procedure06370"/>
<location internal="KNMI-6375" external="urn:ogc:object:sensor:KNMI:procedure06375"/>
<location internal="KNMI-6380" external="urn:ogc:object:sensor:KNMI:procedure06380"/>
<location internal="KNMI-6391" external="urn:ogc:object:sensor:KNMI:procedure06391"/>
</idMap>

Note the parameter mappings in the above example.  Four different parameters are described:

  • The externalQualifier is required and describes which offering is used to get the data from the service.
  • The internal parameter name is mapped to the external 'observed property'
  • The internal location is mapped to the external 'procedure'
  • No labels

Disclaimer