Overview

Imports time series data with observed daily rainfall from the KNMI that is delivered to the Dutch waterboards. The files are in CSV format with file extension (*.dat) the next definition in the file:
<location ID>, <location name>, <X in km>, <Y in km>, <date in YYYYMMDD>, <value in 0.1 mm>. See the example file and the KNMI site.

Notice that the rainfall is measured at 08:00 UT (=GMT), but this time is not written in the file. Therefore the time will be read bij the FEWS import reader as 00:00 hours. The rainfall is supplied as an accumulative time series over the last 24 hours. This requires the time step in FEWS to be configured as

<timeStep unit="day" multiplier="1" timeZone="GMT-8"/>

More information on the KNMI rainfall data sets can be found on the KNMI website

Configuration (Example)

A complete import module configuration consists of an ID Mapping file and a Import Module Instance file. To convert the rainfall in a proper unit (from 0.1 mm/day to mm/day for example) it is also required to configure a Unit Conversion file.

ModuleConfigFiles

The following example of an Import Module Instance will import the time series as equidistant series for timezone GMT with a time step of 24 hours.

ImportKNMI.xml
<?xml version="1.0" encoding="UTF-8"?>
<timeSeriesImportRun ......">
  <import>
    <!--IRIS (24h)-->
    <general>
      <importType>KNMIIRIS</importType>
      <folder>$IMPORT_FOLDER_KNMI_IRIS$</folder>
      <failedFolder>$IMPORT_FAILED_FOLDER_KNMI_IRIS$</failedFolder>
      <backupFolder>$IMPORT_BACKUP_FOLDER_KNMI_IRIS$</backupFolder>
      <idMapId>IdImportIRIS</idMapId>
      <unitConversionsId>ImportKNMIUnits</unitConversionsId>
      <!--data is supplied at 08:00 GMT, but in the file this time is not mentioned, so read as 00:00 hrs.
          so the time zone offset (to GMT) should be -8 hrs-->
      <importTimeZone>
        <timeZoneOffset>-08:00</timeZoneOffset>
      </importTimeZone>
      <dataFeedId>KNMI-IRIS</dataFeedId>
    </general>
    <timeSeriesSet>
      <moduleInstanceId>ImportKNMI</moduleInstanceId>
      <valueType>scalar</valueType>
      <parameterId>P.meting</parameterId>
      <locationSetId>KNMI-IRIS</locationSetId>
      <timeSeriesType>external historical</timeSeriesType>
      <timeStep unit="day" multiplier="1" timeZone="GMT-8"/>
      <readWriteMode>add originals</readWriteMode>
      <synchLevel>1</synchLevel>
    </timeSeriesSet>

    <!--to let the import module know that the KNMI rainfall is an accumulative timeseries in 0.1 mm/d 
      that should be converted to for example mm/d-->
    <externUnit parameterId="P.meting" unit="0.1 mm/d"/>

  </import>
</timeSeriesImportRun>

IdMapFiles

Defines mappings between KNMI and FEWS parameters and locations.

sample of IdImportEPS.xml
<idMap version="1.1"  ..............>
  <map internalParameter="P.meting" internalLocation="KNMI_827" externalParameter="827" externalLocation="827"/>
  <map internalParameter="P.meting" internalLocation="KNMI_831" externalParameter="831" externalLocation="831"/>
  <map internalParameter="P.meting" internalLocation="KNMI_896" externalParameter="896" externalLocation="896"/>
  <map internalParameter="P.meting" internalLocation="KNMI_902" externalParameter="902" externalLocation="902"/>
  .....
</idMap>

Important in this configuration is the externalParameter and the externalLocation have the same identifier.

UnitConversionFile

Defines the conversion of the units that should be applied.

sample of ImportKNMIUnits.xml
<?xml version="1.0" encoding="UTF-8"?>
<unitConversions ...................>
  <unitConversion>
    <inputUnitType>0.1 mm/d</inputUnitType>
    <outputUnitType>mm/d</outputUnitType>
    <multiplier>0.1</multiplier>
    <incrementer>0</incrementer>
  </unitConversion>  ........
  ........
</unitConversions>

Example file

An example of a csv-file from IRIS to be imported using the KNMI-IRIS import Module.

sample of irisgegevens_20071025.dat
   10,HOLLUM              , 172, 605,20071028,     0
   11,WEST TERSCHELLING   , 144, 598,20071028,     0
   16,PETTEN              , 106, 531,20071028,     0
   17,DEN BURG            , 116, 563,20071028,     1
   18,NES (AMELAND)       , 181, 607,20071028,     0
   19,DE COCKSDORP        , 121, 575,20071028,     2
   21,CALLANTSOOG         , 109, 541,20071028,     0
   26,FORMERUM            , 149, 601,20071028,     1
   64,SNEEK               , 172, 561,20071028,     0
   65,MAKKUM              , 156, 564,20071028,     1
   67,DOKKUM              , 195, 593,20071028,     0
   69,APPELSCHA           , 219, 553,20071028,     0
   73,DRACHTEN            , 203, 570,20071028,     0

Example Files

See attached files

Java source code

KnmiIrisTimeSeriesParser.java

  • No labels

Disclaimer