Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • parameterId: determines which parameter this value belongs to, only used when there is no parameterColumn
  • locationId: determines which location this value belongs to, only used when there is no locationColumn
  • ignoreNumericalParameters: when set to true, this column will not be read for numerical parameters. Useful when the value is provided in 2 columns, 1 for numerical and 1 for enumeration (i.e. alphanumeric). It prevents values beings overwritten with an undesired value from the wrong column.
  • ignoreEnumerationParameters: when set to true, this column will not be read for enumeration parameters. Useful when the value is provided in 2 columns, 1 for numerical and 1 for enumeration. It prevents values beings overwritten with an undesired value from the wrong column.
  • requireNumericalParameters: when set to true, the import will fail when this column is filled for non-numerical parameters (enumerations). Useful when the value is provided in 2 columns, 1 for numerical and 1 for enumeration and when the wrong column is filled the import file will be rejected so it can be corrected before the data will be imported.
  • requireEnumerationParameters: when set to true, the import will fail when this column is filled for non-enumeration parameters (numerical). Useful when the value is provided in 2 columns, 1 for numerical and 1 for enumeration and when the wrong column is filled the import file will be rejected so it can be corrected before the data will be imported.

Values column - dynamic

Dynamic value columns are only supported for GeneralCSV Parser and designed to have multiple value columns without specifying the numbers of columns or which parameterId is related to the column.
They have to be the last columns in the csv. Only the location of the first of the columns is specified in the TableMetaData.
An IdMap needs to be used to resolve the internal parameter id of the column name.

Code Block
<table>
        <dateColumn name="Date" pattern="yyyy-MM-dd"/>
         <locationColumn name="STATION"/>
         <valueColumns/>
</table>

 

Flag column

A flag column <flagColumn name="Y"/> can be configured to store the quality flag with the value.

...


Code Block
Year Day
1980 360 
1980 361 
1981 362

user Column (since 2016.02)

Since 2016.02 there is a column that allows for specifying a user with each data point. 

Code Block
languagexml
titleExample configuration for usage of userColumn
<general>
   <importType>generalCSV</importType>
   <folder>importfolder</folder>
   <table>
      <dateTimeColumn name="DATE" pattern="dd-MM-yy HH:mm"/>
      <locationColumn name="LOC"/>
      <unitColumn name="UNIT"/>
      <parameterColumn name="PARAM"/>
      <valueColumn name="VALUE"/>
      <userColumn name="USER"/>
   </table>
</general>

limitSymbol Column

Since 2014.02 there is a column that allows for specifying a limit symbol with each data point. 

Code Block
languagexml
titleExample configuration for usage of userColumn
<general>
   <importType>generalCSV</importType>
   <folder>importfolder</folder>
   <table>
      <dateTimeColumn name="DATE" pattern="dd-MM-yy HH:mm"/>
      <locationColumn name="LOC"/>
      <unitColumn name="UNIT"/>
      <parameterColumn name="PARAM"/>
      <valueColumn name="VALUE"/>
      <limitSymbolColumn name="SYMBOL"/>
   </table>
</general>

This column can recognise < as below detection range, or > as above detection range.

Column(s) not available for this import type

attributeColumn (only available for export since attribute values can only be configured, not imported)

ensembleColumn (only available for export, not supported for import yet)

ensembleMemberColumn (only available for export, not supported for import yet)

Other general configuration options

column separator and decimal separator

Since 2016.01 it is possible to choose from multiple column separators: comma ","  or semi-colon ";" or pipe "|" or tab "&#009;" or space "&#x20;"

When specifying a column separator it is compulsory to also specify the decimal separator as comma ","  or point "."

 

Code Block
languagexml
titleExample configuration for usage of colum separator and decimal separator
<general>
   <importType>generalCSV</importType>
   <folder>importfolder</folder>
   <table>
      <dateTimeColumn name="DATE" pattern="dd-MM-yy HH:mm"/>
      <locationColumn name="LOC"/>
      <unitColumn name="UNIT"/>
      <parameterColumn name="PARAM"/>
      <valueColumn name="VALUE"/>
   </table>
   <missingValue>-999</missingValue>
   <importTimeZone>
      <timeZoneOffset>+01:00</timeZoneOffset>
   </importTimeZone>
   <columnSeparator>|</columnSeparator>
   <decimalSeparator>.</decimalSeparator>
</general>

This enables importing for example the format below:

Code Block
none
none
DATE|LOC|VALUE|UNIT|PARAM
01-01-81 00:00|H-2001|2.3|m|P.m

locale-specific CSV files

If the first line contains a comma, the decimal separator is taken to be a period (.), otherwise it is supposed to be a semicolon (;) and the decimal separator is taken to be a comma. This way locale-specific CSV files are supported.


DATELOCVALUEUNITPARAMFS_AFS_BFS_CFS_DFS_E
01-01-81 00:00H-20012.3mP.mOKOKOKOKOK


Samples Import

For the import of ecological data based on samples, the general csv import is used extensively because there is a lot of information per timeseries.

...