Page History

...

TimeSeries XML file format

The XML export import format is described in the document SpecsImporterenCSV.doc (which is in Dutch, as are the Domain tables)by the schema files that can be found here. Here follows an example CSV XML export file:

Code Block


Monster.identificatie;MeetPunt.identificatie;Grootheid.code;Typering.code;Parameter.groep;Parameter.code;Parameter.omschrijving;Eenheid.code;Hoedanigheid.code;Compartiment.code;Begindatum;Begintijd;Tijd_UTCoffset;Limietsymbool;Numeriekewaarde;Alfanumeriekewaarde;Kwaliteitsoordeel.code# Exporting timeseries with header:#528002 CONCTTE.Cl (mg/l) unknown.NVT.OW nonequidistant never main0#;528002;CONCTTE;;ChemischeStof;Cl;;mg/l;NVT;OW;2012-02-11;11:00:00;0;;9.0;;00;528002;CONCTTE;;ChemischeStof;Cl;;mg/l;NVT;OW;2012-02-11;12:00:00;0;;8.0;;00;528002;CONCTTE;;ChemischeStof;Cl;;mg/l;NVT;OW;2012-02-11;13:00:00;0;;8.0;;00;528002;CONCTTE;;ChemischeStof;Cl;;mg/l;NVT;OW;2012-02-11;14:00:00;0;;8.0;;00;528002;CONCTTE;;ChemischeStof;Cl;;mg/l;NVT;OW;2012-02-11;15:00:00;0;;-999.0;;99;528002;CONCTTE;;ChemischeStof;Cl;;mg/l;NVT;OW;2012-02-11;16:00:00;0;;-999.0;;99
&nbsp;# Exporting timeseries with header:#528002 CONCTTE.P (mg/l) unknown.NVT.OW nonequidistant never main0#;528002;CONCTTE;;ChemischeStof;P;;mg/l;NVT;OW;2012-02-11;11:00:00;0;;9.0;;00;528002;CONCTTE;;ChemischeStof;P;;mg/l;NVT;OW;2012-02-11;12:00:00;0;;9.0;;00;528002;CONCTTE;;ChemischeStof;P;;mg/l;NVT;OW;2012-02-11;13:00:00;0;;9.0;;00;528002;CONCTTE;;ChemischeStof;P;;mg/l;NVT;OW;2012-02-11;14:00:00;0;;9.0;;00;528002;CONCTTE;;ChemischeStof;P;;mg/l;NVT;OW;2012-02-11;15:00:00;0;;9.0;;00;528002;CONCTTE;;ChemischeStof;P;;mg/l;NVT;OW;2012-02-11;16:00:00;0;;9.0;;00
&nbsp;# Exporting timeseries with header:#528001 CONCTTE.N (mg/l) unknown.NVT.OW nonequidistant never main0#;528001;CONCTTE;;ChemischeStof;N;;mg/l;NVT;OW;2012-02-11;11:00:00;0;;9.0;;00;528001;CONCTTE;;ChemischeStof;N;;mg/l;NVT;OW;2012-02-11;12:00:00;0;;9.0;;00;528001;CONCTTE;;ChemischeStof;N;;mg/l;NVT;OW;2012-02-11;13:00:00;0;;9.0;;00;528001;CONCTTE;;ChemischeStof;N;;mg/l;NVT;OW;2012-02-11;14:00:00;0;;9.0;;00;528001;CONCTTE;;ChemischeStof;N;;mg/l;NVT;OW;2012-02-11;15:00:00;0;;9.0;;00;528001;CONCTTE;;ChemischeStof;N;;mg/l;NVT;OW;2012-02-11;16:00:00;0;;9.0;;00
&nbsp;# Exporting timeseries with header:#528001 WATDTE (m) unknown.NVT.OW nonequidistant never main0#;528001;WATDTE;;;;;m;NVT;OW;2012-02-11;11:00:00;0;;1.8;;00;528001;WATDTE;;;;;m;NVT;OW;2012-02-11;12:00:00;0;;2;;00;528001;WATDTE;;;;;m;NVT;OW;2012-02-11;13:00:00;0;;2.1;;00;528001;WATDTE;;;;;m;NVT;OW;2012-02-11;14:00:00;0;;1.9;;00;528001;WATDTE;;;;;m;NVT;OW;2012-02-11;15:00:00;0;;1.9;;00;528001;WATDTE;;;;;m;NVT;OW;2012-02-11;16:00:00;0;;1.8;;00
&nbsp;# Exporting timeseries with header:#528005 WATDTE (m) unknown.NVT.OW nonequidistant never main0#;528005;WATDTE;;;;;m;NVT;OW;2012-02-11;11:00:00;0;;1.8;;00;528005;WATDTE;;;;;m;NVT;OW;2012-02-11;12:00:00;0;;2;;00;528005;WATDTE;;;;;m;NVT;OW;2012-02-11;13:00:00;0;;2.1;;00;528005;WATDTE;;;;;m;NVT;OW;2012-02-11;14:00:00;0;;1.9;;00;528005;WATDTE;;;;;m;NVT;OW;2012-02-11;15:00:00;0;;1.9;;00;528005;WATDTE;;;;;m;NVT;OW;2012-02-11;16:00:00;0;;1.8;;00
&nbsp;# Exporting timeseries with header:#528001 T (°C) unknown.NVT.OW nonequidistant never main0#;528001;T;;;;;oC;NVT;OW;2012-02-11;11:00:00;0;;1.8;;00;528001;T;;;;;oC;NVT;OW;2012-02-11;12:00:00;0;;2;;00;528001;T;;;;;oC;NVT;OW;2012-02-11;13:00:00;0;;2.1;;00;528001;T;;;;;oC;NVT;OW;2012-02-11;14:00:00;0;;1.9;;00;528001;T;;;;;oC;NVT;OW;2012-02-11;15:00:00;0;;1.9;;00;528001;T;;;;;oC;NVT;OW;2012-02-11;16:00:00;0;;1.8;;00

Output CSV timeseries headers
MeetPunt.code: contains location id
Grootheid.code: value always "DUMMY"
Parameter.code: contains parameter id.
Eenheid.code: contains unit code
Hoedanigheid.code: contains qualifier 1
Compartiment.code: contains qualifier 2
Begindatum: contains date part of timestamp
Begintijd: contains time part of timestamp
Limietsymbool: contains OutOfDetectionRangeFlag value
Numeriekewaarde: contains timestep value
Alfanumeriekewaarde: empty
Kwaliteitsoordeel.code: contains timestep flag value
Comment: contains timestep comment value (not part of UmAquo standard)

Date and time formats
Format Begindatum: yyyyMMdd
Format Begintijd: HH:mm:ss

201202131600umaquo.xml

Expected XML elements:

MeetObject: contains location information

MonsterObject: Forms the link between the MeetObject and the timeseries

WaardeReeksTijd: Elements containing timeseries dataSyntax
Commentline prefix: '#'
Column separator: ','
Decimal separator: '.'

Validation

Validation of the values in the UmAquo import files is done against the Aquo schemas . Instead of validating over the web, validation is done locally by validating against the schemas present in the resource file UmAquo_schemas.jar . This archive contains a subset of schemas from the Aquo schemas site. If the Aquo schemas are updated then a new version of the schemas jar must be distributed.

...

Code Block

<?xml version="1.0" encoding="UTF-8"?>
<timeSeriesExportRun xmlns="http://www.wldelft.nl/fews"
                     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                     xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/timeSeriesExportRun.xsd">
	<!-- This is an example export configuration file for exporting data from as UmAquo CSVXMLfile file 	-->
	<export>
		<general>			
			<serializerClassName>nl.wldelft.deltareswebservice.umaquo.timeseriesserializers.UmAquoCsvTimeSeriesSerializer<UmAquoXmlTimeSeriesSerializer</serializerClassName>
			<binDir>%REGION_HOME%/Modules/umaquo-bin</binDir>
			<folder>$EXPORT_FOLDER$/UmAquo/CSV</folder>
            <exportFileName>
                <name>umaquo.csv<xml</name>
                <prefix>
			<timeZeroFormattingString>yyyyMMddHHmm</timeZeroFormattingString>
		</prefix>
            </exportFileName>
			<idMapId>IdExportUmAquo</idMapId>
            <flagConversionsId>ExportFlagConversion</flagConversionsId>
			<exportMissingValue>-999</exportMissingValue>
			<omitMissingValues>true</omitMissingValues>
		</general>    
                <properties>
                   <!-- Optional: comment line prefix character. Default = # -->
                   <string key="COMMENT_PREFIX" value="#"/>
                   <!-- Optional: column separator character. Default = ; -->
                   <string key="COLUMN_SEPARATOR" value=";"/>
                   <!-- Optional: decimal character. Default = . -->
                   <string key="DECIMAL_SEPARATOR" value="."/>
                   <!-- Optional: Regular expression for date value. Default = yyyy-MM-dd -->
                   <string key="CSV_DATEPATTERN" value=";"/>
                   <!-- Optional: Regular expression for time value. Default = HH:mm:ss -->
                   <string key="CSV_TIMEPATTERN" value="HH:mm:ss"/>
                   <!-- Optional: select UmAquo schema version 2009 or 2011. Default = 2009 -->
                   <int key="SCHEMA_VERSION" value="2009"/>          > 
                   <!-- Optional: setting to export location info. Default = false -->
                   <bool key="WRITELOCATIONINFOLENIENT" value ="true" />>        
                </properties>
		<timeSeriesSet>
			<moduleInstanceId>ImportUmAquo</moduleInstanceId>
			<valueType>scalar</valueType>
			<parameterId>MyPar</parameterId>
			<locationSetId>MyLocSet</locationSetId>
			<timeSeriesType>external historical</timeSeriesType>
			<timeStep unit="nonequidistant"/>
            <relativeViewPeriod unit="hour" start="-5" end="0"/>
			<readWriteMode>add originals</readWriteMode>
		</timeSeriesSet>
	</export>
</timeSeriesExportRun>

Here is an example id-map file:

For the Umaquo XML export it is important that the external location ids are mapped to a valid Umaquo format:

Code Block
NL\.umam\.\d{2}\..{1,49}

This can be done using the locationIdPattern option provided by FEWS id mapping. It is also possible to export location in a different format by setting the export property LENIENT to true.

Code Block


<?xml version="1.0" encoding="UTF-8"?>
<idMap version="1.1" xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/idMap.xsd" version="1.1">
    	<!--    internalLocationexternal: FEWS location
            internalParameter: FEWS parameter
            internalLocation: UM Aquo location
            externalParameter: UM Aquo parameter
            externalQualifier : eenheid
            externalQualifier1 : hoedanigheid
            externalQualifier2 : compartiment
    -->
    <map internalLocation="LOC-001" internalParameter="Q.obs" externalLocation="LOC-001.external"
         externalParameter="Q.obs.external"
         externalQualifier="m3/s;kubieke meter per seconde" externalQualifier1="NVT;Niet van toepassing"          externalQualifier2="NT;Niet van toepassing"/>
</idMap>

Id-Mapping goes as follows:

Internal timeseries location and parameter combinations are mapped to external location, parameter and qualifier sets as follows:

externalLocation values are written to the Meetpunt column
externalParameter values are written to the Parameter column
1st externalQualifier values are written to the Eenheid column
2nd externalQualifier values are written to the Hoedanigheid column
3rd externalQualifier values are written to the Compartiement column

Here is an example flag conversion file:

Code Block


<?xml version="1.0" encoding="UTF-8"?>
<flagConversions xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/flagConversions.xsd">
  <flagConversion>
    <inputFlag>     <value>0</value></inputFlag>
    <outputFlag>    <value>0</value></outputFlag>
  </flagConversion>
  <flagConversion>
    <inputFlag>     <value>1</value></inputFlag>
    <outputFlag>    <value>0</value></outputFlag>
  </flagConversion>
  <flagConversion>
    <inputFlag>     <value>2</value></inputFlag>
    <outputFlag>    <value>0</value></outputFlag>
  </flagConversion>
  <flagConversion>
    <inputFlag>     <value>3</value></inputFlag>
    <outputFlag>    <value>50</value></outputFlag>
  </flagConversion>
  <flagConversion>
    <inputFlag>     <value>4</value></inputFlag>
    <outputFlag>    <value>50</value></outputFlag>
  </flagConversion>
  <flagConversion>
    <inputFlag>     <value>5</value></inputFlag>
    <outputFlag>    <value>50</value></outputFlag>
  </flagConversion>
  <flagConversion>
    <inputFlag>     <value>6</value></inputFlag>
    <outputFlag>    <value>99</value></outputFlag>
  </flagConversion>
  <flagConversion>
    <inputFlag>     <value>7</value></inputFlag>
    <outputFlag>    <value>50</value></outputFlag>
  </flagConversion>
  <flagConversion>
    <inputFlag>     <value>8</value></inputFlag>
    <outputFlag>    <value>50</value></outputFlag>
  </flagConversion>
  <flagConversion>
    <inputFlag>     <value>9</value></inputFlag>
    <outputFlag>    <value>99</value></outputFlag>
  </flagConversion>
  <defaultOuputFlag><value>0</value></defaultOuputFlag>
  <missingValueFlag><value>99</value></missingValueFlag>
</flagConversions>

Exporting UmAquo CSV Location info

By configuring the property writeLocationInfo in the timeSeriesExportRun file, it is also possible to export an additional CSV file containing location information.

Grootheden-->
	<!--externalQualifier: eenheid-->
	<!--externalQualifier1: Hoedanigheid-->
	<!--externalQualifier2: Compartiment-->
	<!--externalQualifier3: Waardebepalingsmethode-->
	<parameter internal="NEERSG" external="NEERSG" externalQualifier="mm" externalQualifier1="NVT" externalQualifier2="LT"/>
	<parameter internal="VERDPG" external="VERDPG" externalQualifier="mm" externalQualifier1="NVT" externalQualifier2="LT"/>
	<parameter internal="WATHTE" external="WATHTE" externalQualifier="m" externalQualifier1="NVT" externalQualifier2="OW"/>
	<parameter internal="WATDTE" external="WATDTE" externalQualifier="m" externalQualifier1="NVT" externalQualifier2="OW"/>
	<parameter internal="Q" external="Q" externalQualifier="m3/d" externalQualifier1="NVT" externalQualifier2="OW"/>
	<parameter internal="Ntot" external="CONCTTE.Ntot" externalQualifier="mg/l" externalQualifier1="NVT" externalQualifier2="OW"/>
	<parameter internal="Ptot" external="CONCTTE.Ptot" externalQualifier="mg/l" externalQualifier1="NVT" externalQualifier2="OW"/>
	<parameter internal="Ptot" external="CONCTTE.Ptot" externalQualifier="mg/l" externalQualifier1="NVT" externalQualifier2="OW" externalQualifier3="I10048.91"/>
	<locationIdPattern internalLocationSet="Meetnet" internalLocationPattern="*" externalLocationPattern="NL.umam.00.*" />
</idMap>

Here is an example flag conversion file:

please see UmAquoCsvExportThe contents of this file is also described in the document SpecsImporterenCSV.doc

Page tree

Versions Compared

Old Version 1

New Version Current

Key

TimeSeries XML file format

Validation

Exporting UmAquo CSV Location info