Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The WQ CSV import format differs slightly from the UmAquo CSV import format is as described in the document SpecsImporterenCSV.doc. This format is not so strictly defined and it is also lacking the header information required by the UmAquoCsvTimeSeriesParser class. Here follows an example CSV import file:

...

Here is an example import module configuration file:

Code Block

<?xml version="1.0" encoding="UTF-8"?>

...


<timeSeriesImportRun xmlns="http://www.wldelft.nl/fews"

...


 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

...


 xsi:schemaLocation="http://www.wldelft.nl/fews [http://fews.wldelft.nl/schemas/version1.0/timeSeriesImportRun.xsd]">

...


<!-- This is an example import configuration file for importing WQ CSV data from an

...

 import directory&nbsp; -->
    <import>
        <general>
        <!-- Class name of WQ CSV parser is same as UmAquo CSV parser -->
            <parserClassName>nl.wldelft.webservice.umaquo.timeseriesparsers.UmAquoCsvTimeSeriesParser</parserClassName>

...



            <!-- Path to directory containing UmAquo libraries and schemas. If omitted then the content of the umaquo-bin

...


can be placed in the FEWS-bin dir \-->

...


            <binDir>%REGION_HOME%/Modules/umaquo-bin</binDir>

...



            <!-- Directory from which CSV files are to be imported -->
            <folder>$IMPORT_FOLDER$/WQ/CSV</folder>

...


            <failedFolder>$IMPORT_FAILED_FOLDER$/WQ/CSV</failedFolder>

...


            <backupFolder>$IMPORT_BACKUP_FOLDER$/WQ/CSV</backupFolder>

...


            <idMapId>IdImportWqCsv</idMapId>
            <importTimeZone>
                <timeZoneOffset>+00:00</timeZoneOffset>

...


            </importTimeZone>
        </general>
        <properties>
            <!-- Optional: decimal character. Default = '.'. By configuring a value '*' it becomes possible to import both
            ',' and '.' separated values in the same file -->
            <string key="DECIMAL_SEPARATOR" value="*"/>
            <!-- Optional: column separator character. Default = , -->
            <string key="

...

COLUMN_SEPARATOR" value=";"/>
            <!-- Optional: Regular expression for date value. Default = yyyy-MM-dd -

...

->
            <string key="CSV_DATEPATTERN" value="dd-MM-yyyy"/>

...


            <!-- Optional: select UmAquo schema version 2009 or 2011. Default = 2009 -->

...


            <int key="SCHEMA_VERSION" value="2011"/>

...


            <!-- Optional: allow lenient validation. Default = false -

...

->
            <bool key="LENIENT" value="true"/>

...


            <!-- Required for WQCsv: Provide a header line describing the columns. It is important that the column separator
           matches the value for the COLUMN_SEPARATOR field -

...

->
            <string key="HEADER_LINE" value=";MeetPunt.identificatie;;;Waardebepalingsmethode.code;Begindatum;Begintijd;Typering.code;;Limietsymbool;Numeriekewaarde;Hoedanigheid.code;Eenheid.code;Compartiment.code;"/>

...


            <!-- Optional: allow import to skip the first number of lines. This can be required when the import file contains a head that does not match the above header -->
            <int key="FIRST_LINE" value="1"/>
        </properties>
        <timeSeriesSet>
            <moduleInstanceId>ImportUmAquo</moduleInstanceId>
            <valueType>scalar</valueType>
            <parameterId>MyPar</parameterId>
            <locationSetId>MyLocSet</locationSetId>
            <timeSeriesType>external historical</timeSeriesType>
            <timeStep unit="nonequidistant"/>
            <readWriteMode>add originals</readWriteMode>
            <synchLevel>1</synchLevel>
        </timeSeriesSet>
    </import>
</timeSeriesImportRun>

Expected CSV header names

The header names that can be provided to describe the content of the WQ-CSV must be selected from the list below.

Monster.identificatie: mapped to sampling id (optional). If MeetPunt.identificatie not specified then the monster identifier is mapped to location id.

MeetPunt.identificatieMeetPunt.code: mapped to location id (required)

Typering.code: Umaquo timeseries require either Typering.code or Grootheid.code. If defined it will be used as parameter id (either Typering.code or Grootheid.code)
Grootheid.code: Umaquo timeseries require either Typering.code or Grootheid.code. If defined it will be used as parameter id (either Typering.code or Grootheid.code). Can used in combination with Parameter.code or Parameter.omschrijving

Parameter.code: mapped to parameter.id in combination with Grootheid.code. Always used in combination with Grootheid.code. : if Parameter.code not present then this column is mapped to parameter id (optional)

Parameter.codeomschrijving: mapped to parameter.id . If not present then in combination with Grootheid.code. Always used in combination with Grootheid.code is used. (optional)
Eenheid.code: mapped to timeseries unit and to qualifier 0 (required)
Hoedanigheid.code: mapped to qualifier 1 (required)
Compartiment.code: mapped to qualifier 2 (required)
Waardebepalingsmethode.code: mapped to qualifier 3 (optional)
Begindatum: mapped to timestamp together with BeginTijd (required)
Begintijd: mapped to timestamp. If not present only Begindatum is used. (optional)
Limietsymbool: mapped to OutOfDetectionRangeFlag value (optional)
Numeriekewaarde: contains numeric value. If empty then Alfanumeriekewaarde field is parsed to numeric value.
Alfanumeriekewaarde: only used if Numeriekewaarde field is empty
Kwaliteitsoordeel.code: mapped to flag value

For more information on how to configure IdMapping and FlagConversions check section UmAquoCsvImport