Versions Compared

Old Version 2

changes.mady.by.user Camiel van Breugel

Saved on

compared with

New Version Current

changes.mady.by.user Marc van Dijk

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Introduction

Export scalar timeseries to PI type format (example config). This The Delft-FEWS Published Interface (PI) xml format is widely used to get scalar data exported from the Delft-FEWS database. This Delft-FEWS PI xml format is described in detail in the Delft-Fews published interface documentation. The format is also used in the Delft-FEWS PI Rest Webservice, not all functionality from the PI Webservice is supported in this PI Export function.

Configuration of the the PI Export is following the same rules as other Delft-FEWS Export functions. The export type to configure in the general section of the Export Module in <exportTypeStandard>pi</exportTypeStandard>. Because there are many versions of the PI Export function it is best to use the latest version <exportTypeStandard>pi 1.31</exportTypeStandard>.

Example PI Export Module Instance Config file

Code Block
xml
xml

<?xml version="1.0" encoding="UTF-8"?>
<TimeSeries
    <timeSeriesExportRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews/PI http://fews.wldelft.nl/schemas/version1.0/pi-schemas/pi_timeseries.xsd"
    version="1.2" /timeSeriesExportRun.xsd">
	<export>
		<general>
			<exportTypeStandard>pi 1.31</exportTypeStandard>
			<folder>$REPORT_ROOT_DIR$\Observations</folder>
			<exportFileName>
				<name>Lake_Levels_Remote_Observed_.xml</name>
				<suffix>
					<timeZeroFormattingString>yyyyMMddHHmm</timeZeroFormattingString>
				</suffix>
			</exportFileName>
			<exportMissingValue>-999</exportMissingValue>
			<exportTimeZone>
				<timeZoneName>GMT</timeZoneName>
			</exportTimeZone>
		</general>
		<timeSeriesSet>
			<moduleInstanceId>Import_LakeLevel</moduleInstanceId>
			<valueType>scalar</valueType>
			<parameterId>H.obs</parameterId>
			<locationSetId>hydro_lakes</locationSetId>
			<timeSeriesType>external historical</timeSeriesType>
			<timeStep unit="nonequidistant"/>
			<relativeViewPeriod unit="day" start="-365" end="0"/>
			<readWriteMode>read only</readWriteMode>
		</timeSeriesSet>
	</export>
</timeSeriesExportRun>

This export function will export water level time series in the PI XML format, see below

Code Block
xml
xml
   <series>
        <header>
            <type>instantaneous</type>
            <moduleInstanceId>Import_LakeLevel</moduleInstanceId>
            <locationId>LakeVictoria</locationId>
            <parameterId>H.obs</parameterId>
            <timeStep unit="nonequidistant"/>
            <startDate date="2022-05-10" time="07:00:00"/>
            <endDate date="2023-05-10" time="07:00:00"/>
            <missVal>-999</missVal>
            <stationName>Lake Victoria</stationName>
            <lat>-0.69</lat>
            <lon>33.56</lon>
            <x>33.56</x>
            <y>-0.69</y>
            <units>m</units>
        </header>
        <event date="2022-05-11" time="18:49:00" value="1136.81" flag="0"/>
        <event date="2022-05-21" time="16:48:00" value="1136.83" flag="0"/>
        <event date="2022-05-31" time="14:46:00" value="1136.84" flag="0"/>
        <event date="2022-06-10" time="12:45:00" value="1136.78" flag="0"/>
        <event date="2022-06-20" time="10:43:00" value="1136.75" flag="0"/>
        <event date="2022-06-30" time="08:42:00" value="1136.71" flag="0"/>
        <event date="2022-07-10" time="06:40:00" value="1136.65" flag="0"/>
    </series>
</TimeSeries>


Different versions of the PI XML export support additional functionality, compared with the first basic version.  Support of the validation status, it is possible since PI timeseries version 1.19 (<exportTypeStandard>pi 1.19</exportTypeStandard>). It is then possible to export flagSourceColumns and (custom)flagSources in the PI-XML timeseries format. For example a flag source column named 'V3' that has a custom flag source value 'OK' will result in an attribute  fs:V3="OK". This does not require additional configuration of the export module.


Code Block
xml
xml
<TimeSeries 
	xmlns="http://www.wldelft.nl/fews/PI"

...

 
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
	xsi:schemaLocation="http://www.wldelft.nl/fews/PI http://fews.wldelft.nl/schemas/version1.0/pi-schemas/pi_timeseries.xsd" 
	version="1.19" xmlns:fs="http://www.wldelft.nl/fews/fs">

...

<timeZone>1.0</timeZone>
    <series>
        <header>

...

<type>instantaneous</type>
            <moduleInstanceId>WerkFilter</moduleInstanceId>

...

<locationId>140-w_Leidsche Rijn</locationId>

...

<parameterId>HGTE_METING_15M</parameterId>
            <timeStep unit="second" multiplier="900"/>
            <startDate date="

...

2012-

...

06-

...

25" time="

...

17:

...

45:00"/>
            <endDate date="

...

2012-

...

06-

...

25" time="

...

21:

...

45:00"/>

...

<missVal>NaN</missVal>
            <stationName>AANVOERDER_140-w_Leidsche Rijn</stationName>
            <lat>52.082416488309036</lat>
            <lon>5.075337740730613</lon>
            <x>133623.0</x>
            <y>454951.0</y>

...

<z>-9999.0</z>

...

<units>mNAP</units>
        </header>
        <event date="

...

2012-

...

06-

...

25" time="

...

19:

...

30:00" value="-

...

0.414" flag="0"

...

fs:V3="OK" fs:V4="

...

OK"/>
        <event date="

...

2012-

...

06-

...

25" time="

...

21:

...

45:00" value="-0.

...

385" flag="0" fs:V2="OK" fs:V3="OK"/>
    </series>
</TimeSeries>

Since PI version 1.23, it is possible to add time and value ranges in the PI-XML timeseries format, as shown in the example below:

Code Block
xml
xml
<TimeSeries xmlns="http://www.wldelft.nl/fews/PI" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="44"http://www.wldelft.nl/fews/PI http://fews.wldelft.nl/schemas/version1.0/pi-schemas/pi_timeseries.xsd" version="1.23" xmlns:fs="http://www.wldelft.nl/fews/fs">
    <timeZone>0.0</timeZone>
    <series>
        <header>
            <type>instantaneous</type>
            <moduleInstanceId>moduleInstanceId</moduleInstanceId>
            <locationId>loc</locationId>
            <parameterId>par</parameterId>
            <qualifierId>A</qualifierId>
            <qualifierId>B</qualifierId>
            <timeStep unit="nonequidistant"/>
           <event <startDate date="20031000-0301-01" time="00:3000:00" value="0.0020" flag="44/>
            <endDate date="1000-01-01" time="00:00:00"/>
           <event <forecastDate date="20032000-0301-01" time="00:45:00" value="0.0030" flag="44"/>00:00:00"/>
            <missVal>NaN</missVal>
            <longName>a long name</longName>
            <lat>1.0</lat>
            <lon>2.0</lon>
            <x>2.0</x>
            <y>1.0</y>
            <z>3.0</z>
            <units>m</units>
            <sourceOrganisation>a source organisation</sourceOrganisation>
            <sourceSystem>a source system</sourceSystem>
            <fileDescription>a description</fileDescription>
            <creationDate>2100-01-01</creationDate>
            <creationTime>00:00:00</creationTime>
        </header>
        <properties>
            <string key="organisation" value="Deltares"/>
            <dateTime key="request time" date="2000-01-01" time="12:00:00"/>
        </properties>
        <event date="20031000-0301-01" time="0100:00:00" startDate="0999-12-31" endDate="1000-01-02" value="4.0" minValue="3.5" maxValue="5.5"/>
    </series>
</TimeSeries>


Since PI version 1.24, it is possible to add minutes attribute to time steps. Below is an example which shows how to add a minutes attribute to time steps:

Code Block
xml
xml
<TimeSeries xmlns="http://www.wldelft.nl/fews/PI" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews/PI http://fews.wldelft.nl/schemas/version1.0/pi-schemas/pi_timeseries.xsd" version="1.24" xmlns:fs="http://www.wldelft.nl/fews/fs">
    <timeZone>0.0</timeZone>
    <series>
        <header>
            <type>instantaneous</type>
            <locationId>loc</locationId>
            <parameterId>par</parameterId>
            <timeStep minutes="15 50"/>
    .0040" flag="44"/>
        <event<startDate date="20032018-0302-0112" time="0109:15:00" value/>
            <endDate date="-999.02018-02-12" flagtime="8810:50:00"/>
            <missVal>NaN</missVal>
            <units>m</units>
        </header>
        <event date="20032018-0302-0112" time="0109:3015:00" value="0.0060186" flag="442"/>
        <event date="20032018-0302-0112" time="0109:4550:00" value="0.0070" flag183" flag="2"/>
    </series>
</TimeSeries>

PI version 1.25 does not contain any changes relevant to export.

Since PI version 1.26, product information can be exported in the time series header. This will be automatically exported when forecast products are linked to the exported time series. Below is an example which shows how this is exported.

Code Block
xml
xml
<TimeSeries xmlns="http://www.wldelft.nl/fews/PI" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews/PI http://fews.wldelft.nl/schemas/version1.0/pi-schemas/pi_timeseries.xsd" version="1.26" xmlns:fs="http://www.wldelft.nl/fews/fs">
    <timeZone>0.0</timeZone>
    <series>
        <header>
            <type>instantaneous</type>
            <moduleInstanceId>Forecast</moduleInstanceId>
            <locationId>amerongen_beneden</locationId>
            <parameterId>H.voorspeld.daily</parameterId>
            <timeStep unit="second" multiplier="86400"/>
   ="44"/>
        <event <startDate date="2003-03-012012-12-31" time="00:00:00"/>
            <endDate date="2013-01-05" time="0200:00:00" value="0.0080" flag="44"/>/>
            <forecastDate date="2013-07-09" time="17:05:00"/>
            <missVal>-999.0</missVal>
            <stationName>amerongen beneden</stationName>
            <lat>51.98</lat>
            <lon>5.4</lon>
            <x>5.4</x>
        <event    <y>51.98</y>
            <z>0.0</z>
            <units>m</units>
            <firstValueTime date="20032013-0301-01" time="0200:1500:00" value="0.009000001" flag="44"//>
            <lastValueTime date="2013-01-04" time="00:00:00"/>
            <maxValue>4</maxValue>
            <minValue>1</minValue>
            <valueCount>4</valueCount>
            <product id="ANiceProduct" name="ANiceProduct">
        <event        <productDate date="20032019-0306-0125" time="0211:3031:00" value="0.010000001" flag="44"/>59"/>
                <category id="Meteo" name="Meteo"/>
                <productInfo>
                    <user>Erik Pelgrim</user>
                    <confidence>MEDIUM</confidence>
                    <classification>INTERNAL</classification>
                    <comment>Awww</comment>
                </productInfo>
                <productInfo>
                    <user>Anne Hommelberg</user>
                    <confidence>MEDIUM</confidence>
                    <classification>CONFIDENTIAL</classification>
                    <comment>It is a very nice product!</comment>
                </productInfo>
                <productInfo>
                    <user>Onno van den Akker</user>
                    <confidence>LOW</confidence>
                    <classification>PUBLIC</classification>
                    <comment>Theoretically impossible!</comment>
                </productInfo>
                <productInfo>
                    <user>Erik Pelgrim</user>
                    <confidence>MEDIUM</confidence>
                    <classification>INTERNAL</classification>
                    <comment>Awww</comment>
                </productInfo>
            </product>
        </header>
        <event date="20032012-0312-0131" time="0200:4500:00" value="-999.0.011000001" flag="448"/>
        <event date="20032013-0301-01" time="0300:00:00" value="0.0123" flag="440"/>
        <event date="20032013-0301-0102" time="0300:1500:00" value="0.0131" flag="440"/>
        <event date="20032013-01-03-01" time="0300:3000:00" value="0.0144" flag="440"/>
        <event date="20032013-0301-0104" time="0300:4500:00" value="0.0150000011" flag="440"/>
        <event date="20032013-0301-0105" time="0400:00:00" value="-999.0.016" flag="448"/>
    </series>
</TimeSeries>


Since PI version 1.27, it is possible to add threshold label, description and comments to time steps. For exporting thresholds the element <exportThresholds>true</exportThresholds> need to be added to the general section of the Export configuration.

Below is an example which shows how this can be done:

Code Block
xml
xml
<general>
	<exportTypeStandard>pi 1.31</exportTypeStandard>
	<folder>$REPORT_ROOT_DIR$\Observations</folder>
	<exportFileName>
		<name>Lake_Levels_Remote_Observed_.xml</name>
		<suffix>
			<timeZeroFormattingString>yyyyMMddHHmm</timeZeroFormattingString>
		</suffix>
	</exportFileName>
	<exportMissingValue>-999</exportMissingValue>
	<exportTimeZone>
		<timeZoneName>GMT</timeZoneName>
	</exportTimeZone>
	<exportThresholds>true</exportThresholds>
</general>

will export the  <event date="2003-03-01" time="04:15:00" value="0.017" flag="44thresholds.

<TimeSeries xmlns="http://www.wldelft.nl/fews/PI" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews/PI http://fews.wldelft.nl/schemas/version1.0/pi-schemas/pi_timeseries.xsd" version="1.26" xmlns:fs="http://www.wldelft.nl/fews/fs">
    <timeZone>0.0</timeZone>
    <series>
        <header>
            <type>instantaneous</type>
            <locationId>loc</locationId>
            <parameterId>par</parameterId>
            <timeStep unit="second" multiplier="3600"/>
            <startDate date="2014-10-19" time="00:00:00"/>
        <event    <endDate date="20032014-0310-0119" time="0403:3000:00"/>
            <missVal>NaN</missVal>
            <units>m</units>
            <thresholds>
                <highLevelThreshold id="FaseA" name="Fase A: Verhoogde waakzaamheid" label="label for threshold" description="description" comment="comment" value="0.0180000011.15" groupId="Alarm" flaggroupName="44"/>Alarm"></highLevelThreshold>
            </thresholds>
        </header>
        <event date="20032014-0310-0119" time="0400:4500:00" value="0.019000001186" flag="442"/>
        <event date="20032014-0310-0119" time="0501:00:00" value="0.020000001183" flag="442"/>
    </series>
</TimeSeries>