Introduction



The import module allows data from external source to be imported into DELFT-FEWS. Data may be provided to FEWS in a variety of formats. The approach taken in the import module is that a class is defined for each of the file formats that can be imported.



Data is imported from specified directories. An attempt is made to import all files in the directories and subdirectories configured. If a file conforms to the expected format then the data will be imported. If the file does not conform to the expected format, it will not be imported, but will be moved to a configurable directory with failed import files.





Note that Delft-FEWS can only import the specific data formats that are listed here. Delft-FEWS assumes data types for a configured import to remain the same over time as Delft-FEWS is usually part of an operational system. This means that it will not have the flexibility in importing data that for example programs like Matlab and Excel have. Instead, for each new filetype a dedicated import must be written. However, the list of supported filetypes is ever increasing and adding new imports is fairly simple. 



You can select the files to be imported via the directory and its subdirectories where the files live and by means of a file mask, which is then used to match the file names against.



Two main groups of import can be defined;


    
	
  • Importing data in the XML format defined by the Environment Agency, UK.
    • 
	
  • Importing of various data formats (including ASCII formats, png files- e.g. meteosat images- grids and GRIB files).
    • 





On importing data, the approach to be used for converting flags, units, locations and parameters can be defined. These conversions are identified by referring to the appropriate configuration files (see Regional Configuration). When data is imported to an equidistant time series, a time tolerance may also be defined. If the time recorded is within this tolerance it will be snapped to the cardinal time step in the imported series.



When available as configuration on the file system, the name of the XML file for configuring an instance of the import module called for example ImportRTS may be:



ImportRTS 1.00 default.xml







 ImportRTS 


 File name for the ImportRTS configuration.






 1.00 


 Version number






 default 


 Flag to indicate the version is the default configuration (otherwise omitted).












Import Module Configuration




See also How to Import data





There are similarities between the import module and the General Adapter module as both allow import of data into DELFT-FEWS from an external database. The philosophy of the two modules is, however, different. In the import module there is no prior expectance on the availability of data to be imported. Data that is available is imported and the module will not fail if insufficient data is available. In the General Adapter there are stricter controls on the availability of data, and errors may occur if insufficient data is available.





Note that two main classes are defined for the import module. One for the specific EA XML time series import and one for the general time series import (including GRIB imports).



These are defined in the moduleDescriptors in SystemConfiguration. The first is normally referred to as "EAImport", the second as "TimeSeriesImport"