Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

No Format
<general>
      <importTypeStandard>database</importTypeStandard>
      <jdbcDriverClass>com.mysql.jdbc.Driver</jdbcDriverClass>
      <jdbcConnectionString>jdbc:mysql://192.168.100.206/cwb_ac</jdbcConnectionString>
      <user>sobek</user>
      <password>sobek>cwb</password>
      <relativeViewPeriod startOverrulable="true" endOverrulable="true" start="-1" end="1" unit="day"/>
      <table name="qpe_sums_obs">
        <dateTimeColumn name="recdate"/>
        <valueColumn name="rad_gz" unit="mm/hr" locationId="Qpesums" parameterId="P.radar.actual" parser="Mosaic"/>
      </table>
      <table name="qpe_sums_foc">
        <forecastDateTimeColumn name="createdate"/>
        <dateTimeColumn name="raddate"/>
        <valueColumn name="rad_gz" unit="mm/hr" locationId="Qpesums" parameterId="P.radar.forecast" parser="Mosaic"/>
      </table>
      <unitConversionsId>ImportUnitConversions</unitConversionsId>
      <importTimeZone>
        <timeZoneOffset>+00:00</timeZoneOffset>
      </importTimeZone>
      <dataFeedId>QPE_Sums</dataFeedId>
    </general>
[noformat}



h5. failedFolder

Folder to move badly formatted files to. This may be a UNC path (ie located on the network).

h5. idMapId

ID of the IdMap used to convert external parameterId's and locationId's to internal parameter and location Id's. Each of the formats specified will have a unique method of identifying the id in the external format. See section on configuration for Mapping Id's units and flags.

h5. unitConversionsId

ID of the UnitConversions used to convert external units to internal units. Each of the formats specified will have a unique method of identifying the unit in the external format. See section on configuration for Mapping Id's units and flags.

h5. flagConversionsId

ID of the FlagConversions used to convert external data quality flags to internal data quality flags. Each of the formats specified will have a unique method of identifying the flag in the external format. See section on configuration for Mapping Id's units and flags.

h5. missingValue

Optional specification of missing value identifier in external data format.

h5. importTimeZone

Time zone the external data is provided in if this is not specified in the data format itself. This may be specified as a timeZoneOffset, or as a specific timeZoneName.

h5. timeZoneOffset

The offset of the time zone with reference to UTC (equivalent to GMT). Entries should define the number of hours (or fraction of hours) offset. (e.g. \+01:00)

h5. timeZoneName

Enumeration of supported time zones. See appendix B for list of supported time zones.

h5. gridStartPoint

Identification of the cell considered as the first cell of the grid. This may be in the upper left corner or in the lower left corner. Enumeration of options include;
* _NW_ : for upper left
* _SW_ : for lower left

h5. dataFeedId

Optional id for data feed. If not provided then the folder name will be used. This is is used in the [SystemMonitorDisplay in the importstatus tab|06 System Monitor Display].

h5. tolerance

Definition of the tolerance for importing time values to cardinal time steps in the series to be imported to. Tolerance is defined per location/parameter combination. Multiple entries may exist.

Attributes;
* _locationId_ : Id of the location tolerance is to be considered for.
* _parameterId_ : Id for the parameter tolerance is to be considered for.
* _timeUnit_ : Spefication of time units tolerances is defined in (enumeration).
* _unitCount_ : integer number of units defined for tolerance.

h5. startTimeShift

Specification of a shift to apply to the start time of a data series to be imported as external forecasting. This is required when the time value of the first data point is not the same as the start time of the forecast. This may be the case in for example external precipitation values, where the first value given is the accumulative precipitation for the first time step. The start time of the forecast is then one time unit earlier than the first data point in the series. Multiple entries may exist.

h5. startTimeShift:locationId

Id of the location to apply the startTimeShift to.

h5. startTimeShift:parameterId

Id of the parameter to apply the startTimeShift to.

h5. timeSeriesSet

TimeSeriesSet to import the data to. Multiple time series sets may be defined, and each may include either a (list of) locationId's ar a locationSetId. Data imported is first read from the source data file in the format specifed. An attempt is then made to map the locationId's and the parameterId's as specified in the IdMap's to one of the locations/parameters defined in the import time series sets. If a valid match is found, then the time values are mapped to those in the timeSeriesSet, taking into account the tolerance for time values. A new entry is made in the timeSeries for each valid match made.

For non-equidistant time series the time values imported will be taken as is. For equidistant time series values are only returned on the cardinal time steps. For cardinal time steps where no value is available, no data is returned.

h5. externUnit

For some data formats an external unit is not defined in the file to be imported. This elements allows the unit to be specified explicitly. This unit is then used in possible unit conversions.

Attributes;
* parameterId: Id of the parameter for which a unit is specified. This is the internal parameter Id.
* unit: specification of unit. This unit must be available in the UnitConversions specified in the unitConversionsId element.

h5. gridRecordTimeIgnore

Boolean flag to specify if the start of forecast is read from the GRIB file or if it is inferred from the data imported. In some GRIB files a start of forecast is specified, but the definition of this may differ from that used in DELFT-FEWS.{info}
When importing grid data from file formats where the attributes of the grid is not specified in the file being imported (ie the file is not self-describing), a definition of the grid should be included in the Grids configuration (see Regional Configuration).

It is also advisable to define the Grid attributes fro self describing Grids such as those imported from GRIB files. If no GRIB data is available, then DELFT-FEWS will require a specification of the grid to allow a Missing values grid to be created.{info}

h3. Example: Import of Meteosat images as time-series

failedFolder

Folder to move badly formatted files to. This may be a UNC path (ie located on the network).

idMapId

ID of the IdMap used to convert external parameterId's and locationId's to internal parameter and location Id's. Each of the formats specified will have a unique method of identifying the id in the external format. See section on configuration for Mapping Id's units and flags.

unitConversionsId

ID of the UnitConversions used to convert external units to internal units. Each of the formats specified will have a unique method of identifying the unit in the external format. See section on configuration for Mapping Id's units and flags.

flagConversionsId

ID of the FlagConversions used to convert external data quality flags to internal data quality flags. Each of the formats specified will have a unique method of identifying the flag in the external format. See section on configuration for Mapping Id's units and flags.

missingValue

Optional specification of missing value identifier in external data format.

importTimeZone

Time zone the external data is provided in if this is not specified in the data format itself. This may be specified as a timeZoneOffset, or as a specific timeZoneName.

timeZoneOffset

The offset of the time zone with reference to UTC (equivalent to GMT). Entries should define the number of hours (or fraction of hours) offset. (e.g. +01:00)

timeZoneName

Enumeration of supported time zones. See appendix B for list of supported time zones.

gridStartPoint

Identification of the cell considered as the first cell of the grid. This may be in the upper left corner or in the lower left corner. Enumeration of options include;

  • NW : for upper left
  • SW : for lower left
dataFeedId

Optional id for data feed. If not provided then the folder name will be used. This is is used in the SystemMonitorDisplay in the importstatus tab.

tolerance

Definition of the tolerance for importing time values to cardinal time steps in the series to be imported to. Tolerance is defined per location/parameter combination. Multiple entries may exist.

Attributes;

  • locationId : Id of the location tolerance is to be considered for.
  • parameterId : Id for the parameter tolerance is to be considered for.
  • timeUnit : Spefication of time units tolerances is defined in (enumeration).
  • unitCount : integer number of units defined for tolerance.
startTimeShift

Specification of a shift to apply to the start time of a data series to be imported as external forecasting. This is required when the time value of the first data point is not the same as the start time of the forecast. This may be the case in for example external precipitation values, where the first value given is the accumulative precipitation for the first time step. The start time of the forecast is then one time unit earlier than the first data point in the series. Multiple entries may exist.

startTimeShift:locationId

Id of the location to apply the startTimeShift to.

startTimeShift:parameterId

Id of the parameter to apply the startTimeShift to.

timeSeriesSet

TimeSeriesSet to import the data to. Multiple time series sets may be defined, and each may include either a (list of) locationId's ar a locationSetId. Data imported is first read from the source data file in the format specifed. An attempt is then made to map the locationId's and the parameterId's as specified in the IdMap's to one of the locations/parameters defined in the import time series sets. If a valid match is found, then the time values are mapped to those in the timeSeriesSet, taking into account the tolerance for time values. A new entry is made in the timeSeries for each valid match made.

For non-equidistant time series the time values imported will be taken as is. For equidistant time series values are only returned on the cardinal time steps. For cardinal time steps where no value is available, no data is returned.

externUnit

For some data formats an external unit is not defined in the file to be imported. This elements allows the unit to be specified explicitly. This unit is then used in possible unit conversions.

Attributes;

  • parameterId: Id of the parameter for which a unit is specified. This is the internal parameter Id.
  • unit: specification of unit. This unit must be available in the UnitConversions specified in the unitConversionsId element.
gridRecordTimeIgnore

Boolean flag to specify if the start of forecast is read from the GRIB file or if it is inferred from the data imported. In some GRIB files a start of forecast is specified, but the definition of this may differ from that used in DELFT-FEWS.

Info

When importing grid data from file formats where the attributes of the grid is not specified in the file being imported (ie the file is not self-describing), a definition of the grid should be included in the Grids configuration (see Regional Configuration).

It is also advisable to define the Grid attributes fro self describing Grids such as those imported from GRIB files. If no GRIB data is available, then DELFT-FEWS will require a specification of the grid to allow a Missing values grid to be created.

Example: Import of Meteosat images as time-series

Wiki Markup
Meteosat Images are generally imported as images in _\[filename\].png_ format. The Meteosat images constitute a time series of png images, that are geo-referenced by means of a specific world file. Each image needs its own world file, which in case of PNG carries the extension _\[filename\].pgw_ .
Import of images in another format, such as JPEG is also possible. The corresponding world file for a JPEG file has the extension _\[filename\].jpg_ .
The images are imported via a common time series import, for which a specific image parameter needs to be specified in a _parameterGroup_ via the parameter id _image_ .

...

unmigrated-wiki-markup

No Format

_<parameterGroup id="image">_
       _<parameterType\]instantaneous\[/parameterType>_
       _<unit>-</unit>_
       _<valueResolution>8</valueResolution>_
       _<parameter id="image">_
       _<shortName>image</shortName>_
       _</parameter>_
_</parameterGroup>_

...

noformat


The

...

value

...

resolution

...

indicates

...

the

...

resolution

...

of

...

the

...

values

...

of

...

the

...

pixels

...

(grey

...

tones)

...

in

...

the

...

Meteosat

...

images.

...

In

...

this

...

case

...

8

...

grey

...

tones

...

are

...

resampled

...

into

...

a

...

single

...

grey

...

tone

...

for

...

storage

...

space

...

reductions.

...

In

...

the

...

module

...

for

...

the

...

timemeseries

...

import

...

run

...

for

...

a

...

Meteosat

...

image

...

the

...

import

...

is

...

then

...

configured

...

as

...

follows:

No Format
<import>

...


    <general>
        <importType>GrayScaleImage</importType>

...


        <folder>$REGIONHOME$/Import/MeteoSat</folder>

...


             <idMapId>IdImportMeteosat</idMapId>

...


    </general>

...



    <timeSeriesSet>
        <moduleInstanceId>ImportMeteosat</moduleInstanceId>

...


        <valueType>grid</valueType>

...


        <parameterId>image</parameterId>

...


        <locationId>meteosat</locationId>

...


        <timeSeriesType>external historical</timeSeriesType>

...


        <timeStep unit="minute" multiplier="15"/>

...


        <readWriteMode>add originals</readWriteMode>

...


        <synchLevel>4</synchLevel>

...


        <expiryTime unit="day" multiplier="750"/>

...


   </timeSeriesSet>

...


</import>

...

noformat

The goereferenced image can then be displayed in the grid display.

...