Custom time series
...
import formats using java
SINCE FEWS 2009.01
Fews allows you to write your own time series import format with javain Java.
In the import module you one can specify the fully qualified class name and the bin directory that contains a jar file with you compiled java code and optimally other third party libaries. A jar file is just a zip file that contains you compiled java files.
e.g. classname = com.yourcompany.yourparsers.YourParser
and binDir = $REGION_HOME$/parsers/bin
...
The import module will also open and close the files for you.
The import files can retain on the file system, in a zip file, tar file or gz file, ftp server, sftp server. The programmer will not notice the differenceh2
Time Series Content handler
The interface is described in: TimeSeriesContentHandler.java
Types of parsers
Text parsers
Most import files are text based.
Binary parser
Grid coverage files are often binary
File parsers
This kind of parsers will use a third party library that not accepts streams
Database parsers
This parsers will read from a database.
Msaccess (mdb) of firebird (fdb) files are automatically recognized
It also possible to explicit configure a database connection with an external database in the import module
Server parsers
This kind of parsers will read data from a (internet) url
A url, username and password is provided to the parser
Additional consumer interfaces
PeriodConsumer
Database and server parsers often needs a period in their query to the database of server.
When this interface implemented the import module will provide an absolute period.
TimeSeriesHeadersConsumer
Database and server parsers often needs the location and parameter ids in their queries
When this interface implemented the import module will convert the FEWS headers with specified id map and provide them to the parser. The mapping is used in the opposite direction compared to normal mapping. This can result in different mapping when the id map is not one internal to one external and visa versa.
VirtualDirConsumer
With a virtual dir the parser can open meta files with additional information required for to parse file that retain in the same directory as the imported file. For example some grid coverage formats need an additional file for the geo referencing.
PropertiesConsumer
SINCE FEWS 2011.01
Ideally a parser does not need any additional information and is dedicated to parse one strict specified file format.
In some cases the parser needs additional configuration. Don't use a separate additional configuration file in this case because this file can not be managed by FEWS.
You can configure additional properties in the time series import module. By implementing the PropertiesConsumer interface the properties are injected in your parser.
Code Block | ||||
---|---|---|---|---|
| ||||
<import>
<general>
<importType>PI</importType>
<folder>$IMPORT_FOLDER$/meteo/tabel/tk</folder>
<failedFolder>$IMPORT_FAILED_FOLDER$/meteo/tabel/tk</failedFolder>
<backupFolder>$IMPORT_BACKUP_FOLDER$/meteo/tabel/tk</backupFolder>
<idMapId>IdImportMeteo_1d</idMapId>
<unitConversionsId>ImportUnitConversions</unitConversionsId>
<dataFeedId>Neerslag_24uur</dataFeedId>
</general>
<properties>
<string key="decimalSeparator" value="."/>
/properties>
<timeSeriesSet>
|
Code Block |
---|
public class CsvTimeSeriesParser implements TextParser<TimeSeriesContentHandler>, PropertiesConsumer { @Override public void setProperties(Properties properties) { decimalSeparator = properties.getString("decimalSeparator", "."); } |
Examples
TextParsers
LocationIdsHeaderCsvParser.java
...
NtuQuarterTimeSeriesParser.java
XML parsers
PiTimeSeriesParser.java
PiMapStackParser.java
Binary parsers
GrayscaleImageTimeSeriesParser.java (with header is separate file)
MosaicRadarTimeSeriesParser.java (Little endian IEEE floats and integers)
NimrodGridTimeSeriesParser.java
Server parsers
RmoTimeSeriesServerParser.java