Page History

...

The XML import format is described in the document SpecsImporterenCSV.doc (which is in Dutch, as are the Domain tables)by the schema files that can be found here. Here follows an a few example XML import filefiles:

Code Block


Monster.identificatie;MeetPunt.identificatie;Grootheid.code;Typering.code;Parameter.groep;Parameter.code;Parameter.omschrijving;Eenheid.code;Hoedanigheid.code;Compartiment.code;Begindatum;Begintijd;Tijd_UTCoffset;Limietsymbool;Numeriekewaarde;Alfanumeriekewaarde;Kwaliteitsoordeel.code
# Exporting timeseries with header:
#528002 CONCTTE.Cl (mg/l) unknown.NVT.OW nonequidistant never main 0
#
;528002;CONCTTE;;ChemischeStof;Cl;;mg/l;NVT;OW;2012-02-11;11:00:00;0;;9.0;;00
;528002;CONCTTE;;ChemischeStof;Cl;;mg/l;NVT;OW;2012-02-11;12:00:00;0;;8.0;;00
;528002;CONCTTE;;ChemischeStof;Cl;;mg/l;NVT;OW;2012-02-11;13:00:00;0;;8.0;;00
;528002;CONCTTE;;ChemischeStof;Cl;;mg/l;NVT;OW;2012-02-11;14:00:00;0;;8.0;;00
;528002;CONCTTE;;ChemischeStof;Cl;;mg/l;NVT;OW;2012-02-11;15:00:00;0;;-999.0;;99
;528002;CONCTTE;;ChemischeStof;Cl;;mg/l;NVT;OW;2012-02-11;16:00:00;0;;-999.0;;99

# Exporting timeseries with header:
#528002 CONCTTE.P (mg/l) unknown.NVT.OW nonequidistant never main 0
#
;528002;CONCTTE;;ChemischeStof;P;;mg/l;NVT;OW;2012-02-11;11:00:00;0;;9.0;;00
;528002;CONCTTE;;ChemischeStof;P;;mg/l;NVT;OW;2012-02-11;12:00:00;0;;9.0;;00
;528002;CONCTTE;;ChemischeStof;P;;mg/l;NVT;OW;2012-02-11;13:00:00;0;;9.0;;00
;528002;CONCTTE;;ChemischeStof;P;;mg/l;NVT;OW;2012-02-11;14:00:00;0;;9.0;;00
;528002;CONCTTE;;ChemischeStof;P;;mg/l;NVT;OW;2012-02-11;15:00:00;0;;9.0;;00
;528002;CONCTTE;;ChemischeStof;P;;mg/l;NVT;OW;2012-02-11;16:00:00;0;;9.0;;00

# Exporting timeseries with header:
#528001 CONCTTE.N (mg/l) unknown.NVT.OW nonequidistant never main 0
#
;528001;CONCTTE;;ChemischeStof;N;;mg/l;NVT;OW;2012-02-11;11:00:00;0;;9.0;;00
;528001;CONCTTE;;ChemischeStof;N;;mg/l;NVT;OW;2012-02-11;12:00:00;0;;9.0;;00
;528001;CONCTTE;;ChemischeStof;N;;mg/l;NVT;OW;2012-02-11;13:00:00;0;;9.0;;00
;528001;CONCTTE;;ChemischeStof;N;;mg/l;NVT;OW;2012-02-11;14:00:00;0;;9.0;;00
;528001;CONCTTE;;ChemischeStof;N;;mg/l;NVT;OW;2012-02-11;15:00:00;0;;9.0;;00
;528001;CONCTTE;;ChemischeStof;N;;mg/l;NVT;OW;2012-02-11;16:00:00;0;;9.0;;00

# Exporting timeseries with header:
#528001 WATDTE (m) unknown.NVT.OW nonequidistant never main 0
#
;528001;WATDTE;;;;;m;NVT;OW;2012-02-11;11:00:00;0;;1.8;;00
;528001;WATDTE;;;;;m;NVT;OW;2012-02-11;12:00:00;0;;2;;00
;528001;WATDTE;;;;;m;NVT;OW;2012-02-11;13:00:00;0;;2.1;;00
;528001;WATDTE;;;;;m;NVT;OW;2012-02-11;14:00:00;0;;1.9;;00
;528001;WATDTE;;;;;m;NVT;OW;2012-02-11;15:00:00;0;;1.9;;00
;528001;WATDTE;;;;;m;NVT;OW;2012-02-11;16:00:00;0;;1.8;;00

# Exporting timeseries with header:
#528005 WATDTE (m) unknown.NVT.OW nonequidistant never main 0
#
;528005;WATDTE;;;;;m;NVT;OW;2012-02-11;11:00:00;0;;1.8;;00
;528005;WATDTE;;;;;m;NVT;OW;2012-02-11;12:00:00;0;;2;;00
;528005;WATDTE;;;;;m;NVT;OW;2012-02-11;13:00:00;0;;2.1;;00
;528005;WATDTE;;;;;m;NVT;OW;2012-02-11;14:00:00;0;;1.9;;00
;528005;WATDTE;;;;;m;NVT;OW;2012-02-11;15:00:00;0;;1.9;;00
;528005;WATDTE;;;;;m;NVT;OW;2012-02-11;16:00:00;0;;1.8;;00

# Exporting timeseries with header:
#528001 T (°C) unknown.NVT.OW nonequidistant never main 0
#
;528001;T;;;;;oC;NVT;OW;2012-02-11;11:00:00;0;;1.8;;00
;528001;T;;;;;oC;NVT;OW;2012-02-11;12:00:00;0;;2;;00
;528001;T;;;;;oC;NVT;OW;2012-02-11;13:00:00;0;;2.1;;00
;528001;T;;;;;oC;NVT;OW;2012-02-11;14:00:00;0;;1.9;;00
;528001;T;;;;;oC;NVT;OW;2012-02-11;15:00:00;0;;1.9;;00
;528001;T;;;;;oC;NVT;OW;2012-02-11;16:00:00;0;;1.8;;00

Expected CSV headers
MeetPunt.code: mapped to location id (required)
Grootheid.code: if Parameter.code not present then this column is mapped to parameter id (optional)
Parameter.code: mapped to parameter.id. If not present then Grootheid.code is used.
Eenheid.code: mapped to timeseries unit and to qualifier 0 (required)
Hoedanigheid.code: mapped to qualifier 1 (required)
Compartiment.code: mapped to qualifier 2 (required)
Begindatum: mapped to timestamp together with BeginTijd (required)
Begintijd: mapped to timestamp. If not present only Begindatum is used. (optional)
Limietsymbool: mapped to OutOfDetectionRangeFlag value (optional)
Numeriekewaarde: contains numeric value. If empty then Alfanumeriekewaarde field is parsed to numeric value.
Alfanumeriekewaarde: only used if Numeriekewaarde field is empty
Kwaliteitsoordeel.code: mapped to flag value

Date and time formats
Format Begindatum: yyyyMMdd
Format Begintijd: HH:mm:ss

Syntax
Commentline prefix: '#'
Column separator: ','
Decimal separator: '.'

VEGMWATERNET.XML

VISWATERNET.XML

Expected XML elements:

MeetObject: contains location information

MonsterObject: Forms the link between the MeetObject and the timeseries

WaardeReeksTijd: Elements containing timeseries data

Validation

Validation of the values in the UmAquo import files is done against the Aquo schemas . Instead of validating over the web, validation is done locally by validating against the schemas present in the resource file ^UmAquoUmAquo_schemas.jar. This archive contains a subset of schemas from the Aquo schemas site. If the Aquo schemas are updated then a new version of the schemas jar must be distributed.

...

Code Block

<?xml version="1.0" encoding="UTF-8"?>
<timeSeriesImportRun xmlns="http://www.wldelft.nl/fews"
                     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                     xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/timeSeriesImportRun.xsd">
    <!-- This is an example import configuration file for importing UmAquo CSV data from an import directory    -->
    <import>
        <general>
           <!-- Class name of UmAquo CSV parser -->
            <parserClassName>nl.wldelft.deltareswebservice.umaquo.timeseriesparsers.UmAquoCsvTimeSeriesParser<UmAquoXmlTimeSeriesParser</parserClassName>

            <!-- Path to directory containing UmAquo libraries and schemas. If omitted then the content of the umaquo-bin
                 can be placed in the FEWS-bin dir -->
            <binDir>%REGION_HOME%/Modules/umaquo-bin</binDir>
            
            <!-- Directory from which CSV files are to be imported -->
            <folder>$IMPORT_FOLDER$/UmAquo/CSV<XML</folder>
            <failedFolder>$IMPORT_FAILED_FOLDER$/UmAquo/CSV<XML</failedFolder>
            <backupFolder>$IMPORT_BACKUP_FOLDER$/UmAquo/CSV<XML</backupFolder>
            <idMapId>IdImportUmAquo</idMapId>
            <importTimeZone>
                <timeZoneOffset>+00:00</timeZoneOffset>
            </importTimeZone>
        </general>
                <properties>
                   <!-- Optional: commentselect lineUmAquo prefixschema character.version Default2009 = # -->
                   <string key="COMMENT_PREFIX" value="#"/>
                   <!-- Optional: column separator characteror 2011. Default = ;2009 -->
                   <string<int key="COLUMNSCHEMA_SEPARATORVERSION" value=";2009"/>
                   <!-- Optional: decimalallow character. Default = . -->
                   <string key="DECIMAL_SEPARATOR" value="."/>
                   <!-- Optional: Regular expression for date valuelenient validation. Default = yyyy-MM-ddfalse -->
                   <string<bool key="CSV_DATEPATTERNLENIENT" value=";"/>
                   <!-- Optional: Regular expression for time value. Default = HH:mm:ss -->
                   <string key="CSV_TIMEPATTERN" value="HH:mm:ss"/>
                   <!-- Optional: select UmAquo schema version 2009 or 2011. Default = 2009 -->
                   <int key="SCHEMA_VERSION" value="2009true"/>
                </properties>
        <timeSeriesSet>
            <moduleInstanceId>ImportUmAquo</moduleInstanceId>
            <valueType>scalar</valueType>
            <parameterId>MyPar</parameterId>
            <locationSetId>MyLocSet</locationSetId>
            <timeSeriesType>external historical</timeSeriesType>
            <timeStep unit="nonequidistant"/>
            <readWriteMode>add originals</readWriteMode>
            <synchLevel>1</synchLevel>
        </timeSeriesSet>
    </import>
</timeSeriesImportRun>

Here is an example id-map file:

Code Block


<?xml version="1.0" encoding="UTF-8"?>
<idMap xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/idMap.xsd" version="1.1">
    <!--    internalLocation: FEWS location
            internalParameter: FEWS parameter
            internalLocation: UM Aquo location
            externalParameter: UM Aquo parameter
            externalQualifier : eenheid
            externalQualifier1 : hoedanigheid
            externalQualifier2 : compartiment
    -->
    <map internalLocation="LOC-001" internalParameter="Q.obs" externalLocation="LOC-001.external"
         externalParameter="Q.obs.external"
         externalQualifier="m3/s;kubieke meter per seconde" externalQualifier1="NVT;Niet van toepassing"          externalQualifier2="NT;Niet van toepassing"/>
</idMap>

Id-Mapping goes as follows:

Internal timeseries location and parameter combinations are mapped to external location, parameter and qualifier sets as follows:

externalLocation values are read from the Meetpunt column
externalParameter values are read from the Parameter column
1st externalQualifier values are read from the Eenheid column
2nd externalQualifier values are read from the Hoedanigheid column
3rd externalQualifier values are read from the Compartiement column

please see UmAquoCsvImport

Here is an example flag conversion file:

...

please see UmAquoCsvImport

Page tree

Versions Compared

Old Version 1

New Version Current

Key

Validation