View Source

Overview

Imports time series data from Diver loggers. The files have a sort of Windows ini file format with file extension (*.mon). The format of the MON ini files is not well defined. Many programs interpret the structure differently and have various names for the ini file sections and parameters.

Sections: Section declarations start with '[' and end with ']'; i.e. '[Logger settings]' or '[Instrument info]'.
Parameters or item: this is the content of a section with an '=' sign between the key and the value; i.e. "location = abc"

The Date format used is: "yyyy/MM/dd HH:mm:ss"

The files can contain a flexible number of channels, where a channel will contain information on one parameter. Example files with 2 and 3 channels will be shown, when multiple channels/parameters need to be imported a <timeSeriesSet> needs to be configured for each channel/parameter. To illustrate this an example config for an import with 3 channels is given.

Configuration (Example)

A complete import module configuration consists of an ID Mapping file and a Import Module Instance file.

ModuleConfigFiles

The following example of an Import Module Instance will import the time series as equidistant series for timezone GMT+1 with a time step of 1 hour. Many times the MON files do not save the data at rounded hourly tims, therefore a tolerance has been added to map the imported data to correct hourly interval time series.

<?xml version="1.0" encoding="UTF-8"?>
<timeSeriesImportRun ......">
  <import>
    <general>
      <importType>DIVERMON</importType>
      <folder>$IMPORT_FOLDER_MON$</folder>
      <failedFolder>$IMPORT_FAILED_FOLDER_MON$</failedFolder>
      <backupFolder>$IMPORT_BACKUP_FOLDER_MON$</backupFolder>
      <idMapId>IdImportMON</idMapId>
      <unitConversionsId>ImportUnitConversions</unitConversionsId>
      <importTimeZone>
        <timeZoneOffset>+01:00</timeZoneOffset>
      </importTimeZone>
      <dataFeedId>MON</dataFeedId>
    </general>
    <tolerance locationSetId="ImportMON_H.meting.cm_uur" parameterId="H.meting.cm" timeUnit="minute" unitCount="30"/>
    <timeSeriesSet>
      <moduleInstanceId>ImportMON</moduleInstanceId>
      <valueType>scalar</valueType>
      <parameterId>H.meting.cm</parameterId>
      <locationSetId>ImportMON_H.meting.cm_uur</locationSetId>
      <timeSeriesType>external historical</timeSeriesType>
      <timeStep unit="hour" multiplier="1"/>
      <readWriteMode>add originals</readWriteMode>
      <synchLevel>1</synchLevel>
    </timeSeriesSet>
  </import>
</timeSeriesImportRun>

Here is an example config file for an import file with three channels:

<?xml version="1.0" encoding="UTF-8"?>
<timeSeriesImportRun ......">
  <import>
		<general>
			<importType>DiverMon</importType>
			<folder>$IMPORT_FOLDER$/DiverMon</folder>
			<idMapId>IdMap</idMapId>
			<missingValue>-999.0</missingValue>
			<expiryTime unit="day" multiplier="366"/>
		</general>
		<timeSeriesSet>
			<moduleInstanceId>ImportMON</moduleInstanceId>
			<valueType>scalar</valueType>
			<parameterId>X.yy.zz</parameterId>
			<locationId>Delft</locationId>
			<timeSeriesType>external historical</timeSeriesType>
			<timeStep unit="minute" multiplier="10"/>
			<readWriteMode>add originals</readWriteMode>
		</timeSeriesSet>
		<timeSeriesSet>
			<moduleInstanceId>ImportMON</moduleInstanceId>
			<valueType>scalar</valueType>
			<parameterId>A.bb.cc</parameterId>
			<locationId>Delft</locationId>
			<timeSeriesType>external historical</timeSeriesType>
			<timeStep unit="minute" multiplier="10"/>
			<readWriteMode>add originals</readWriteMode>
		</timeSeriesSet>
		<timeSeriesSet>
			<moduleInstanceId>ImportMON</moduleInstanceId>
			<valueType>scalar</valueType>
			<parameterId>D.ee.ff</parameterId>
			<locationId>Delft</locationId>
			<timeSeriesType>external historical</timeSeriesType>
			<timeStep unit="minute" multiplier="10"/>
			<readWriteMode>add originals</readWriteMode>
		</timeSeriesSet>
	</import>
</timeSeriesImportRun>

IdMapFiles

ID mapping defines mappings between Diver MON and FEWS parameters and locations. Remember that ID mapping is case sensitive.

<idMap version="1.1"  ..............>
    <parameter internal="G.hh.ii" external="LEVEL"/>
	<parameter internal="J.kk.ll" external="TEMPERATURE"/>

    <parameter internal="X.yy.zz" external="PCH"/>
	<parameter internal="A.bb.cc" external="HG"/>
	<parameter internal="D.ee.ff" external="2: SPEC.COND."/>

	<location internal="Utrecht" external="13PB1"/>
	<location internal="Delft" external="ro954"/>
</idMap>

Example file

There is a wide range of MON file types, here are examples of files with 2 and 3 channels respectively.

Data file for DataLogger.
==============================================================================
COMPANY    : Your Company Name
LICENSE    :
DATE       : 08/24/07
TIME       : 11:20:44
FILENAME   : H:\Grondwater\DLG-Mariapeel NIEUW\24-8-2007\13PB175646_03_16_0708_23_07.mon
CREATED BY : EnviroMon version 1.9
==========================    BEGINNING OF DATA     ==========================
[Instrument info]
  Instrument type         =Diver =3
  Instrument state        =Stopped =2
  Serial number           =.N00-75646  210.
  Instrument number       =13PB1
                          =0.00000000000000E-0000
  Location                =13PB1
  Sample Rate             =00 01:00:00 0
  Sample Mode             =T
  Channel                 =2
[Channel 1]
  Identification          =LEVEL
  Reference               =0.0       cm
  Range                   =500.0     cm
  Master Level            =0.0       cm
  Altitude                =0         m
[Channel 2]
  Identification          =TEMPERATURE
  Reference               =-20.00    deg C
  Range                   =100.00    deg C
  Master Level            =100.00    deg C
  Altitude                =-20.00    deg C
[Instrument info from data header]
  Serial number           =.N00-75646  210.
  Instrument number       =13PB1
  Location                =13PB1
  Sample Rate             =00 01:00:00 0
  Sample Mode             =T
  Tb                      =22:47:10 16/03/07
  Te                      =22:47:10 23/08/07
[Channel 1 from data header]
  Identification          =LEVEL
  Reference               =0.0       cm
  Range                   =500.0     cm
  Master Level            =0.0       cm
  Altitude                =0         m
[Channel 2 from data header]
  Identification          =TEMPERATURE
  Reference               =-20.00    deg C
  Range                   =100.00    deg C
  Master Level            =100.00    deg C
  Altitude                =-20.00    deg C
[Data]
3841
2007/03/16 10:47:22.0        93.2       11.61
2007/03/16 11:47:22.0       209.7        8.96
2007/03/16 12:47:22.0       209.4        8.93
2007/03/16 13:47:22.0       208.9        8.94
2007/03/16 14:47:22.0       208.5        8.95
2007/03/16 15:47:22.0       207.9        8.95
2007/03/16 16:47:22.0       207.7        8.96

Data file for DataLogger.
==============================================================================
COMPANY    : <Company name>
COMP.STATUS: Do
DATE       : 22/10/2013
TIME       : 07:22:29
FILENAME   : D:\diver offive\vistrap\MON\ro954_131022072229_M0026.MON
CREATED BY : SWS Diver-Office 5.0.25.0
==========================    BEGINNING OF DATA     ==========================
[Logger settings]
  Instrument type         =CTD-Diver=17
  Status                  =Started =0
  Serial number           =.N10-M0026  317.
  Instrument number       =
                          =0
  Location                =ro954
  Sample period           =M10
  Sample method           =T
  Number of channels      =3
[Channel 1]
  Identification          =PRESSURE
  Reference level         =400       cm
  Range                   =5750      cm
  Master level            =5750      CMH2O
  Altitude                =0         m
[Channel 2]
  Identification          =TEMPERATURE
  Reference level         =-20.00    °C
  Range                   =100.00    °C


[Channel 3]
  Identification          =2: SPEC.COND.
  Reference level         =0.000     mS/cm
  Range                   =120.000   mS/cm


[Series settings]
  Serial number           =.N10-M0026  317.
  Instrument number       =
  Location                =ro954
  Sample period           =00 00:10:00 0
  Sample method           =T
  Start date / time       =00:00:08 12/09/13
  End date / time         =00:20:06 22/10/13
[Channel 1 from data header]
  Identification          =PRESSURE
  Reference level         =400       cm
  Range                   =5750      cm
  Master level            =5750      CMH2O
  Altitude                =0         m
[Channel 2 from data header]
  Identification          =TEMPERATURE
  Reference level         =-20.00    °C
  Range                   =100.00    °C


[Channel 3 from data header]
  Identification          =2: SPEC.COND.
  Reference level         =0.000     mS/cm
  Range                   =120.000   mS/cm


[Data]
5751
2013/09/12 08:00:00.0         1332       18.18       0.824
2013/09/12 08:10:00.0         1326       18.18       0.828
2013/09/12 08:20:00.0         1328       18.17       0.824
2013/09/12 08:30:00.0         1329       18.16       0.828
2013/09/12 08:40:00.0         1329       18.17       0.824
2013/09/12 08:50:00.0         1328       18.21       0.824
2013/09/12 09:00:00.0         1329       18.23       0.820

Accepted Mon file Sections and Parameters

The Mon Import module in Delft-FEWS does not parse all data in the MON file. The important sections and parameters are the following:

[Logger settings] or [Instrument info]; Read location Id from this section
- Location or Locatie ; location Id
[Channel X] or [Kanaal X] ; Not used
[Series settings] or [Instrument info from data header] ; Not used
[Channel X from data header] or [Kanaal X from data header]; Read Parameter Id
- Identification or Identificatie = Parameter Id
[Data]; Data values with the different channels in columns. The datavalues may have a "."or a ","as decimal seperator, both options are accepter by the import function.

When the MON file is not in the correct format a warning message is returned. Known problems are missing location ID's or parameter ID's in the MON files.

Java source code

DiverMonTimeSeriesParser.java