This is an import that makes use of the custom import mechanism described in Custom import formats. The WQ CSV is an extension of the UmAquo CSV import and makes use of the same text parser, UmAquoCsvTimeSeriesParser.java, and a bin directory containing all dependant library files UmAquo.zip.
For more information on the UmAquoCsvTimeSeriesParser.java please see the section UmAquoCsvImport.
The WQ CSV import format differs slightly from the UmAquo CSV import format as described in the document SpecsImporterenCSV.doc. This format is not so strictly defined and it is also lacking the header information required by the UmAquoCsvTimeSeriesParser class. Here follows an example CSV import file:
2003925AM_55;OEEUW600;Eeuwselscheloop Grens;201.32;305.22;25-09-2003;13:15:00;Ca;Ca;;5.48;nf;mg/l;OW;Caopg 60181;OBOSS100;Uit de Bosschen Pottevenweg;192.83;389.12;23-08-2004;11:55:00;Calcium opgelost;;;0.5229;;mmol/l;OW;Caopg 2003925AM_55;OEEUW600;Eeuwselscheloop Grens;201.32;305.22;25-09-2003;13:15:00;Aldrin;;;0.24;;mg/l;OW;Aldrin 2003925AM_55;OEEUW600;Eeuwselscheloop Grens;201.32;305.22;25-09-2003;13:15:00;Aldrin;;;1.35;;mg/kg;BS;Aldrin |
Because the WQ CSV import format is lacking header information, it is necessary to add this information to the timeseries import configuration file as shown below.
Here is an example import module configuration file:
<?xml version="1.0" encoding="UTF-8"?>
<timeSeriesImportRun xmlns="http://www.wldelft.nl/fews"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.wldelft.nl/fews [http://fews.wldelft.nl/schemas/version1.0/timeSeriesImportRun.xsd]">
<!-- This is an example import configuration file for importing WQ CSV data from an import directory -->
<import>
<general>
<!-- Class name of WQ CSV parser is same as UmAquo CSV parser -->
<parserClassName>nl.wldelft.webservice.umaquo.timeseriesparsers.UmAquoCsvTimeSeriesParser</parserClassName>
<!-- Path to directory containing UmAquo libraries and schemas. If omitted then the content of the umaquo-bin
can be placed in the FEWS-bin dir \-->
<binDir>%REGION_HOME%/Modules/umaquo-bin</binDir>
<!-- Directory from which CSV files are to be imported -->
<folder>$IMPORT_FOLDER$/WQ/CSV</folder>
<failedFolder>$IMPORT_FAILED_FOLDER$/WQ/CSV</failedFolder>
<backupFolder>$IMPORT_BACKUP_FOLDER$/WQ/CSV</backupFolder>
<idMapId>IdImportWqCsv</idMapId>
<importTimeZone>
<timeZoneOffset>+00:00</timeZoneOffset>
</importTimeZone>
</general>
<properties>
<!-- Optional: decimal character. Default = '.'. By configuring a value '*' it becomes possible to import both
',' and '.' separated values in the same file -->
<string key="DECIMAL_SEPARATOR" value="*"/>
<!-- Optional: column separator character. Default = , -->
<string key="COLUMN_SEPARATOR" value=";"/>
<!-- Optional: Regular expression for date value. Default = yyyy-MM-dd -->
<string key="CSV_DATEPATTERN" value="dd-MM-yyyy"/>
<!-- Optional: select UmAquo schema version 2009 or 2011. Default = 2009 -->
<int key="SCHEMA_VERSION" value="2011"/>
<!-- Optional: allow lenient validation. Default = false -->
<bool key="LENIENT" value="true"/>
<!-- Required for WQCsv: Provide a header line describing the columns. It is important that the column separator
matches the value for the COLUMN_SEPARATOR field -->
<string key="HEADER_LINE" value=";MeetPunt.identificatie;;;Waardebepalingsmethode.code;Begindatum;Begintijd;Typering.code;;Limietsymbool;Numeriekewaarde;Hoedanigheid.code;Eenheid.code;Compartiment.code;"/>
<!-- Optional: allow import to skip the first number of lines. This can be required when the import file contains a head that does not match the above header -->
<int key="FIRST_LINE" value="1"/>
</properties>
<timeSeriesSet>
<moduleInstanceId>ImportUmAquo</moduleInstanceId>
<valueType>scalar</valueType>
<parameterId>MyPar</parameterId>
<locationSetId>MyLocSet</locationSetId>
<timeSeriesType>external historical</timeSeriesType>
<timeStep unit="nonequidistant"/>
<readWriteMode>add originals</readWriteMode>
<synchLevel>1</synchLevel>
</timeSeriesSet>
</import>
</timeSeriesImportRun>
|
Expected CSV header names
The header names that can be provided to describe the content of the WQ-CSV must be selected from the list below.
Monster.identificatie: mapped to sampling id (optional). If MeetPunt.identificatie not specified then the monster identifier is mapped to location id.
MeetPunt.identificatie: mapped to location id (required)
Typering.code: Umaquo timeseries require either Typering.code or Grootheid.code. If defined it will be used as parameter id (either Typering.code or Grootheid.code)
Grootheid.code: Umaquo timeseries require either Typering.code or Grootheid.code. If defined it will be used as parameter id (either Typering.code or Grootheid.code). Can used in combination with Parameter.code or Parameter.omschrijving
Parameter.code: mapped to parameter.id in combination with Grootheid.code. Always used in combination with Grootheid.code. (optional)
Parameter.omschrijving: mapped to parameter.id in combination with Grootheid.code. Always used in combination with Grootheid.code. (optional)
Eenheid.code: mapped to timeseries unit and to qualifier 0 (required)
Hoedanigheid.code: mapped to qualifier 1 (required)
Compartiment.code: mapped to qualifier 2 (required)
Waardebepalingsmethode.code: mapped to qualifier 3 (optional)
Begindatum: mapped to timestamp together with BeginTijd (required)
Begintijd: mapped to timestamp. If not present only Begindatum is used. (optional)
Limietsymbool: mapped to OutOfDetectionRangeFlag value (optional)
Numeriekewaarde: contains numeric value. If empty then Alfanumeriekewaarde field is parsed to numeric value.
Alfanumeriekewaarde: only used if Numeriekewaarde field is empty
Kwaliteitsoordeel.code: mapped to flag value
For more information on how to configure IdMapping and FlagConversions check section UmAquoCsvImport