...
Imports time series data from Diver loggers. The files have a sort of Windows ini file format with file extension (*.mon). The format of the MON ini files is not well defined. Many programs interpret the structure differently and have various names for the ini file sections and parameters.
Sections: Section declarations start with '\[' and end with '\]'; i.e. '\[Logger settings\]' or '\[Instrument info\]'.Wiki Markup - Parameters or item: this is the content of a section with an '=' sign between the key and the value; i.e. "location = abc"
The Date format used is: "yyyy/MM/dd HH:mm:ss"
Configuration (Example)
A complete import module configuration consists of an ID Mapping file and a Import Module Instance file.
ModuleConfigFiles
The files can contain a flexible number of channels, where a channel will contain information on one parameter. Example files with 2 and 3 channels will be shown, when multiple channels/parameters need to be imported a <timeSeriesSet> needs to be configured for each channel/parameter. To illustrate this an example config for an import with 3 channels is given.
Configuration (Example)
A complete import module configuration consists of an ID Mapping file and a Import Module Instance file.
ModuleConfigFiles
The following example of an Import Module Instance will import the time series as equidistant The following example of an Import Module Instance will import the time series as equidistant series for timezone GMT+1 with a time step of 1 hour. Many times the MON files do not save the data at rounded hourly tims, therefore a tolerance has been added to map the imported data to correct hourly interval time series.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
<?xml version="1.0" encoding="UTF-8"?>
<timeSeriesImportRun ......">
<import>
<general>
<importType>DIVERMON</importType>
<folder>$IMPORT_FOLDER_MON$</folder>
<failedFolder>$IMPORT_FAILED_FOLDER_MON$</failedFolder>
<backupFolder>$IMPORT_BACKUP_FOLDER_MON$</backupFolder>
<idMapId>IdImportMON</idMapId>
<unitConversionsId>ImportUnitConversions</unitConversionsId>
<importTimeZone>
<timeZoneOffset>+01:00</timeZoneOffset>
</importTimeZone>
<dataFeedId>MON</dataFeedId>
</general>
<tolerance locationSetId="ImportMON_H.meting.cm_uur" parameterId="H.meting.cm" timeUnit="minute" unitCount="30"/>
<timeSeriesSet>
<moduleInstanceId>ImportMON</moduleInstanceId>
<valueType>scalar</valueType>
<parameterId>H.meting.cm</parameterId>
<locationSetId>ImportMON_H.meting.cm_uur</locationSetId>
<timeSeriesType>external historical</timeSeriesType>
<timeStep unit="hour" multiplier="1"/>
<readWriteMode>add originals</readWriteMode>
<synchLevel>1</synchLevel>
</timeSeriesSet>
</import>
</timeSeriesImportRun>
|
IdMapFiles
ID mapping fefines mappings between Diver MON and FEWS parameters and locations. Remember that ID mapping is case sensitive.
...
<idMap version="1.1" ..............>
<map internalParameter="H.meting.cm" internalLocation="Dorperdijk_beneden" externalParameter="niveau" externalLocation="Dorperdijk beneden"/>
</idMap>
" encoding="UTF-8"?>
<timeSeriesImportRun ......">
<import>
<general>
<importType>DIVERMON</importType>
<folder>$IMPORT_FOLDER_MON$</folder>
<failedFolder>$IMPORT_FAILED_FOLDER_MON$</failedFolder>
<backupFolder>$IMPORT_BACKUP_FOLDER_MON$</backupFolder>
<idMapId>IdImportMON</idMapId>
<unitConversionsId>ImportUnitConversions</unitConversionsId>
<importTimeZone>
<timeZoneOffset>+01:00</timeZoneOffset>
</importTimeZone>
<dataFeedId>MON</dataFeedId>
</general>
<tolerance locationSetId="ImportMON_H.meting.cm_uur" parameterId="H.meting.cm" timeUnit="minute" unitCount="30"/>
<timeSeriesSet>
<moduleInstanceId>ImportMON</moduleInstanceId>
<valueType>scalar</valueType>
<parameterId>H.meting.cm</parameterId>
<locationSetId>ImportMON_H.meting.cm_uur</locationSetId>
<timeSeriesType>external historical</timeSeriesType>
<timeStep unit="hour" multiplier="1"/>
<readWriteMode>add originals</readWriteMode>
<synchLevel>1</synchLevel>
</timeSeriesSet>
</import>
</timeSeriesImportRun>
|
Here is an example config file for an import file with three channels:
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
<?xml version="1.0" encoding="UTF-8"?>
<timeSeriesImportRun ......">
<import>
<general>
<importType>DiverMon</importType>
<folder>$IMPORT_FOLDER$/DiverMon</folder>
<idMapId>IdMap</idMapId>
<missingValue>-999.0</missingValue>
<expiryTime unit="day" multiplier="366"/>
</general>
<timeSeriesSet>
<moduleInstanceId>ImportMON</moduleInstanceId>
<valueType>scalar</valueType>
<parameterId>X.yy.zz</parameterId>
<locationId>Delft</locationId>
<timeSeriesType>external historical</timeSeriesType>
<timeStep unit="minute" multiplier="10"/>
<readWriteMode>add originals</readWriteMode>
</timeSeriesSet>
<timeSeriesSet>
<moduleInstanceId>ImportMON</moduleInstanceId>
<valueType>scalar</valueType>
<parameterId>A.bb.cc</parameterId>
<locationId>Delft</locationId>
<timeSeriesType>external historical</timeSeriesType>
<timeStep unit="minute" multiplier="10"/>
<readWriteMode>add originals</readWriteMode>
</timeSeriesSet>
<timeSeriesSet>
<moduleInstanceId>ImportMON</moduleInstanceId>
<valueType>scalar</valueType>
<parameterId>D.ee.ff</parameterId>
<locationId>Delft</locationId>
<timeSeriesType>external historical</timeSeriesType>
<timeStep unit="minute" multiplier="10"/>
<readWriteMode>add originals</readWriteMode>
</timeSeriesSet>
</import>
</timeSeriesImportRun>
|
IdMapFiles
ID mapping defines mappings between Diver MON and FEWS parameters and locations. Remember that ID mapping is case sensitive.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
<idMap version="1.1" ..............>
<parameter internal="G.hh.ii" external="LEVEL"/>
<parameter internal="J.kk.ll" external="TEMPERATURE"/>
<parameter internal="X.yy.zz" external="PCH"/>
<parameter internal="A.bb.cc" external="HG"/>
<parameter internal="D.ee.ff" external="2: SPEC.COND."/>
<location internal="Utrecht" external="13PB1"/>
<location internal="Delft" external="ro954"/>
</idMap>
|
Example file
There is a wide range of MON file types, here are examples of files with 2 and 3 channels respectively.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
Data file for DataLogger.
==============================================================================
COMPANY : Your Company Name
LICENSE :
DATE : 08/24/07
TIME : 11:20:44
FILENAME : H:\Grondwater\DLG-Mariapeel NIEUW\24-8-2007\13PB175646_03_16_0708_23_07.mon
CREATED BY : EnviroMon version 1.9
========================== BEGINNING OF DATA ==========================
[Instrument info]
Instrument type =Diver =3
Instrument state =Stopped =2
Serial number =.N00-75646 210.
Instrument number =13PB1
=0.00000000000000E-0000
Location =13PB1
Sample Rate =00 01:00:00 0
Sample Mode =T
Channel =2
[Channel 1]
Identification =LEVEL
Reference =0.0 cm
Range =500.0 cm
Master Level =0.0 cm
Altitude =0 m
[Channel 2]
Identification =TEMPERATURE
Reference =-20.00 deg C
Range =100.00 deg C
Master Level =100.00 deg C
Altitude =-20.00 deg C
[Instrument info from data header]
Serial number =.N00-75646 210.
Instrument number =13PB1
Location =13PB1
Sample Rate =00 01:00:00 0
Sample Mode =T
Tb =22:47:10 16/03/07
Te =22:47:10 23/08/07
[Channel 1 from data header]
Identification =LEVEL
Reference =0.0 cm
Range =500.0 cm
Master Level =0.0 cm
Altitude =0 m
[Channel 2 from data header]
Identification =TEMPERATURE
Reference =-20.00 deg C
Range =100.00 deg C
Master Level =100.00 deg C
Altitude =-20.00 deg C
[Data]
3841
2007/03/16 10:47:22.0 93.2 11.61
2007/03/16 11:47:22.0 209.7 8.96
2007/03/16 12:47:22.0 209.4 8.93
2007/03/16 13:47:22.0 208.9 8.94
2007/03/16 14:47:22.0 208.5 8.95
2007/03/16 15:47:22.0 207.9 8.95
2007/03/16 16:47:22.0 207.7 8.96
|
Example file
...
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
Data file for DataLogger. ============================================================================== COMPANY : Your<Company Company Name LICENSE name> COMP.STATUS: Do DATE : 0822/2410/072013 TIME : 1107:2022:4429 FILENAME : HD:\Grondwater\DLG-Mariapeel NIEUW\24-8-2007\13PB175646_03_16_0708_23_07.mondiver offive\vistrap\MON\ro954_131022072229_M0026.MON CREATED BY : EnviroMonSWS version 1.9Diver-Office 5.0.25.0 ========================== BEGINNING OF DATA ========================== [InstrumentLogger infosettings] Instrument type =CTD-Diver=17 Status =Diver =3 Instrument state =StoppedStarted =20 Serial number =.N00N10-75646M0026 210317. Instrument number =13PB1 =0.00000000000000E-0000 Location =13PB1 Sample Rate =00 01:00:00 0 =ro954 Sample Modeperiod =M10 =TSample method Channel =T Number of channels =23 [Channel 1] Identification =LEVEL PRESSURE Reference level =0.0 =400 cm cm Range =500.05750 cm Master Levellevel =0.05750 cm CMH2O Altitude =0 m [Channel 2] Identification =TEMPERATURE Reference level Reference =-20.00 °C Range =-20.00 deg C Range =100.00 °C [Channel 3] Identification =100.00 deg C =2: SPEC.COND. MasterReference Levellevel =1000.00000 deg C mS/cm AltitudeRange =-20.00 =120.000 deg C [Instrument info from data headermS/cm [Series settings] Serial number =.N00N10-75646M0026 210317. Instrument number =13PB1 Location =13PB1ro954 Sample Rate period =00 0100:0010:00 0 Sample Modemethod =T Tb Start date / time =2200:4700:1008 1612/0309/0713 TeEnd date / time =22:47:10 23/08/07=00:20:06 22/10/13 [Channel 1 from data header] Identification =LEVEL Identification =PRESSURE Reference level =0.0400 cm Range =500.05750 cm Master Levellevel =0.05750 cm CMH2O Altitude =0 m [Channel 2 from data header] Identification =TEMPERATURE Reference Reference level =-20.00 deg C °C Range =100.00 °C [Channel 3 degfrom Cdata header] Master Level Identification =2: SPEC.COND. Reference level =1000.00000 deg C mS/cm AltitudeRange =-20.00 =120.000 deg C mS/cm [Data] 3841 2007/03/16 10:47:22.0 5751 2013/09/12 08:00:00.0 1332 93 18.218 110.61824 20072013/0309/1612 1108:4710:2200.0 1326 20918.718 80.96828 20072013/0309/1612 1208:4720:2200.0 209.4 1328 18.17 80.93824 20072013/0309/1612 1308:4730:2200.0 208.9 1329 8.94 2007/03/16 14:47:22.018.16 208.50.828 2013/09/12 08:40:00.0 8.95 2007/03/16 15:47:22.0 1329 20718.917 80.95824 20072013/0309/1612 1608:4750:2200.0 207.7 1328 8.96 |
Accepted Mon file Sections and Parameters
The Mon Import module in Delft-FEWS does not parse all data in the MON file. The important sections and parameters are the following:
18.21 0.824
2013/09/12 09:00:00.0 1329 18.23 0.820
|
Accepted Mon file Sections and Parameters
The Mon Import module in Delft-FEWS does not parse all data in the MON file. The important sections and parameters are the following:
- [Logger settings] or [Instrument info]; Read location Id from this section
- Location or Locatie ; location Id
- [Channel X] or [Kanaal X] ; Not used
- [Series settings] or [Instrument info from data header] ; Not used
- [Channel X from data header] or [Kanaal X from data header]; Read Parameter
Wiki Markup \[Logger settings\] or \[Instrument info\]; Read location Id from this section
- Location or Locatie ; location Id
Wiki Markup \[Channel 1\] or \[Kanaal 1\] ; Not used
Wiki Markup \[Channel 2\] or \[Kanaal 2\] ; Not used
Wiki Markup \[Series settings\] or \[Instrument info from data header\] ; Not used
Wiki Markup \[Channel 1 from data header\] or \[Kanaal 1 from data header\]; Read Parameter Id
- Identification or Identificatie = Parameter Id
\[Channel 2 from data header\] or \[Kanaal 2 from data header\]; Read Parameter IdWiki Markup - Identification or Identificatie = Parameter Id
\[Data\]; Data values with the different channels in columns. The datavalues may have a "."or a ","as decimal seperator, both options are accepter by the import function.Wiki Markup
When the MON file is not in the correct format a warning message is returned. Known problems are missing location ID's or parameter ID's in the MON files.
...