Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

scrollbar

What

nameofinstance.xml

Description

Configuration for the general adapter module

schema location

http://fews.wldelft.nl/schemas/version1.0/generalAdapterRun.xsd

Entry in ModuleDescriptors

<moduleDescriptor id="GeneralAdapter">
<description>General Adaptor to launch published interface compliant modules</description>
<className>nl.wldelft.fews.system.plugin.generaladapter.GeneralAdapter</className>
</moduleDescriptor>

Table of Contents

 

General Adapter Configuration

...


Figure 66 Elements of the General Adapter configuration

general

Root element of general settings.

burnInProfile

Burn-in period and initial value for cold state starts.

activities

Root element for the activities to be defined. The activities are defined in a fixed order;

  • startUpActivities
  • exportActivities
  • executeActivities
  • importActivities
  • shutDownActivities

 

General settings


Figure 67 Elements of the general section of the general adapter configuration

description

Optional description of the configuration. Used for reference purposes only.

piVersion

Version of the PI specification that is supported by the pre and post adapter.

rootDir

Root directory for the external module. Other directories can be defined relative to this rootDir using predefined tags (see comment box below).

workDir

Working directory to be used by the external module. When started this directory will be the current directory.

exportDir

Directory to export data from DELFT-FEWS to the external module. All Published Interface files will be written to this directory (unless overruled in naming the specific export files).

exportDataSetDir

Directory to export module datasets from DELFT-FEWS to the external module. A module dataset is a ZIP file, which will be unzipped using this directory as the root directory. If the zip file contains full path information, this will be included as a tree of subdirectories under this directory.

exportIdMap

ID of the IdMap used to convert internal parameterId's and locationId's to external parameter and location Id's. See section on configuration for Mapping Id's units and flags.

exportUnitConversionsId

Id of UnitConversions to be used for export unit mapping

importDir

Directory to import result data from the external module to DELFT-FEWS. All Published Interface files will be read from this directory (unless overruled in naming the specific export files).

importIdMap

ID of the IdMap used to convert external parameterId's and locationId's to internal parameter and location Id's. This may be defined to be the same as the import directory, but may also contain different mappings. See section on configuration for Mapping Id's units and flags.

importUnitConversionsId

Id of UnitConversions to be used for import unit mapping

...

Directory for writing dump files to. Dump Files are created when one of the execute activities fails. A dump file is a ZIP file which includes all the dumpDir directories defined. The dump file is created immediately on failure, meaning that all data and files are available as they are at the time of failure and can be used for analysis purposes. The ZIP file name is time stamped to indicate when it was created.

dumpDir

Directory to be included in the dump file. All contents of the directory will be zipped. Multiple dumpDir's may be defined.

NOTE: ensure that the dumpDir does not include the dumpFileDir. This creates a circular reference and may result in corrupted ZIP files.

diagnosticFile

File name and path of diagnostic files created in running modules. This file should be formatted using the Published Interface diagnostics file specification.

missVal

Optional specification of missing value identifier to be used in PI-XML exported to modules and imported from modules.

NOTE: it is assumed an external module uses the same missing value identification for both import and export data.

convertDatum

Optional Boolean flag to indicate level data is used and produced by the module at a global rather than a local datum. The convention in DELFT-FEWS is that data is stored at a local datum. If set to true data in parameter groups supporting datum conversion will be converted on export to the global datum by adding the z coordinate of the location. (see definition of parameters and locations in Regional Configuration).

timeZone

Time zone with reference to UTC (equivalent to GMT) for all time dependent data communicated with the module. If not defined, UTC+0 (GMT) will be used. This time zone is used when importing pi files and the time zone is not available in the pi file.

timeZoneOffset

The offset of the time zone with reference to UTC (equivalent to GMT). Entries should define the number of hours (or fraction of hours) offset. (e.g. +01:00). This time zone is used when importing pi files and the time zone is not available in the pi file.

timeZoneName

Enumeration of supported time zones. See appendix B for list of supported time zones. This time zone is used when importing pi files and the time zone is not available in the pi file.

time0Format

The date time format of the %TIME0% variable
yyyy Year
M Month in year
d Day in month
H Hour in day (0-23)
m Minute in hour
s Second in minute

startDateTimeFormat

The date time format of the %START_DATE_TIME% variable. yyyy = Year, MM = Month in year, dd = Day in month, HH = Hour in day (0-23), mm = Minute in hour, ss = Second in minute. The %START_DATE_TIME% variable only works in an importStateActivity (since 2014.02) and inside a template file for an exportCustomFormatRunFileActivity (since 2013.02).

endDateTimeFormat

The date time format of the %END_DATE_TIME% variable. yyyy = Year, MM = Month in year, dd = Day in month, HH = Hour in day (0-23), mm = Minute in hour, ss = Second in minute. The %END_DATE_TIME% variable only works in an importStateActivity (since 2014.02) and inside a template file for an exportCustomFormatRunFileActivity (since 2013.02).

ensembleMemberCount

Defines if ensembles are read from or written to a number of sub directories.

Burn-In Profile

Burn-in profile for cold state starts. Used to replace first part of a timeseries.

...

Length of time series beginning that is to be replaced.

timeSeries

Initial value (which should match cold state), location and parameter should be specified.

Startup Activities


Figure 68 Elements of the startUpActivities section of the General Adapter configuration.

...

Root element of a purge activity used to delete files from previous runs. Multiple purge activities may be defined.

filter

Filter specifying files to be removed. Wildcards may be used.

...

Example (not the use of tags to define the directory names):

unzipActivity

Root element of an unzip activity used to unpack a zip file and put the contained files in the directory of choice. Multiple unzip activities may be defined.

...

  • description - optional description of the activity (for documentation only)
  • sourceZipFile - the name of the zip file to be unzipped
  • destinationDir - the name of the directory where the files will be put

 

zipActivity

Root element of a zip activity used to pack all files and subdirectories of an indicated directory to a zip file for later use/inspection. Multiple zip activities may be defined. The file name may include environment variables, as well as tags defined in the general adapter or on the global.properties. See the environment variables section for a list of internal variables.

...

  • description - optional description of the activity (for documentation only)
  • sourceDir - the name of the directory containing the files to be zipped
  • destinationZipFile - the name of the zip file to be created

 

Example:

No Format

<startupActivities>
    <unzipActivity>
        <sourceZipFile>extra_files.zip</sourceZipFile>
        <destinationDir>%ROOT_DIR%/work</destinationDir>
    </unzipActivity>
</startupActivities>
...
<shutdownActivities>
    <zipActivity>
        <sourceDir>%ROOT_DIR%/work</sourceDir>
        <destinationZipFile>%ROOT_DIR%/inspection/%TIME0%_saved.zip</destinationZipFile>
    </zipActivity>
</shutdownActivities>

...


Figure 70 Elements of the ExportStateActivity section.

description

Optional description for the export states configuration. Used for reference purposes only.

moduleInstanceId

Id of the moduleInstance that has written the state to be exported. Generally this will be the same as the Id of the current instance of the General Adapter. This can also be the ID of another instance of the General Adapter. The latter is the case when using a state in a forecast run that has been written in an historical run.

ensembleId

Export state exported by this ensemble.

...

Directory to which the module instance state is to be exported. This should be the path where the model expects to find the state file(s). This should be either an absolute path or a path relative to the exportDir defined in the general section. All contents of the stored state are exported to this directory.

stateConfigFile

Optional. Deprecated, do not use this if a pi state description xml file is not needed. Name (and location) of the PI-XML file describing the states. If the directory location is not explicitly specified the file will be written in the exportDir defined in the general section. This should be either an absolute path or a path relative to the exportDir defined in the general section. This option writes the input state file paths to a pi state description xml file.

stateLocations

Optional. Deprecated, do not use this if a pi state description xml file is not needed. Root element for the description of the state. Both a read location and a write location will need to be defined. This allows the name of the file(s)/directory to be different on read and write. Multiple locations may be defined, but these must all be of the same type. This option writes the input state file paths to a pi state description xml file.

  • Attributes type: indication of type of state to be imported. This may either be "directory" or "file". Note that multiple locations are supported only if type is "file".
  • stateLocation - Root element of a state location
  • readLocation - Location where the external module will read the state. This is the location (and name of file/directory) where the General Adapter writes the state. This should be either an absolute path or a path relative to the stateExportDir defined in this activity.
  • writeLocation - Location where the external module is expected to write the state. This is the location (and name of file/directory) where the General Adapter expects to read the state. This should be an absolute path.
Code Block
xml
xml

<stateLocations type="file">
	<stateLocation>
		<readLocation>state.inp</readLocation>
		<writeLocation>state.out</writeLocation>
	</stateLocation>
</stateLocations>

...

Root element to specify how a state to be exported to the external module is to be selected. Two main groups are available, cold states and warm states. Only one of these types can be specified. Note that if a warm state selection is specified and an appropriate warm state cannot be found, a cold state will be exported by default.

  • coldState - Root element for defining the stateSelection method to always export a cold state.
  • groupId - Id of the group of cold states to be used. This must be a groupId as defined in the ColdModuleInstanceStateGroups configuration (see Regional Configuration).
  • coldState:startDate - Definition of the start date of the external module run when using the cold state. This startDate is specified relative to the start time of the forecast run. A positive startDate means it is before the start time of the forecast run.
  • coldState:fixedStartTime - (Since 2012_02) the start date can be configured as an fixed time
  • warmState - Root element for defining the stateSelection method to search for the most suitable warm state.
  • stateSearchPeriod - Definition of the search period to be used in selecting a warm state. The database will return the most recent suitable warm state found within this search period.
  • coldStateTime - Definition of the start time to use for a cold state if a suitable state is not found within the warm state search period.
  • insertColdState - When you set insertColdState to true, the defaultColdState is inserted into the WarmStates when no WarmState is found inside the stateSearchPeriod. By default the cold state is not inserted as warm state
Code Block
xml
xml

<stateSelection>
        <warmState>
          <stateSearchPeriod unit="hour" start="-48" end="0"/>
	  <coldStateTime unit="hour" value="-48"/>
	  <insertColdState>true</insertColdState>"
	</warmState>
</stateSelection>

adjustTimeSeriesStartTime

...

New example that does not use a pi state description xml file. In this case, if the model adapter needs to know the input state file paths, then these can be read from the netcdf version of the run info file (which can be exported using the new ExportNetcdfRunFileActivity described below):

Code Block
xml
xml

<exportStateActivity>
	<moduleInstanceId>HBV_AareBrugg_Hist</moduleInstanceId>
	<stateExportDir>%ROOT_DIR%/FEWS/states</stateExportDir>
	<stateSelection>
		<warmState>
			<stateSearchPeriod unit="hour" start="-240" end="-96"/>
		</warmState>
	</stateSelection>
	<writeIntermediateState>true</writeIntermediateState>
	<minimumRunLength unit="day" multiplier="4"/>
</exportStateActivity>

Old example that writes the input state file paths to a pi state description xml file. Do not use this if a pi state description xml file is not needed:

Code Block
xml
xml

<exportStateActivity>
	<moduleInstanceId>HBV_AareBrugg_Hist</moduleInstanceId>
	<stateExportDir>%ROOT_DIR%/FEWS/states</stateExportDir>
	<stateConfigFile>%ROOT_DIR%/FEWS/states/states.xml</stateConfigFile>
	<stateLocations type="file">
		<stateLocation>
			<readLocation>HBV_States.zip</readLocation>
			<writeLocation>HBV_States.zip</writeLocation>
		</stateLocation>
	</stateLocations>
	<stateSelection>
		<warmState>
			<stateSearchPeriod unit="hour" start="-240" end="-96"/>
		</warmState>
	</stateSelection>
	<writeIntermediateState>true</writeIntermediateState>
	<minimumRunLength unit="day" multiplier="4"/>
</exportStateActivity>

...


Figure 71 Elements of the exportTimeSeries section

description

Optional description of the timeSeries export configuration.

exportFile

Name (and location) of the PI-XML file with exported time series. If the directory location is not explicitly specified the file will be written in the exportDir defined in the general section.

exportBinFile

When true the events in the PI time series file are written to a binary file instead of the xml file. The written xml file will only contain the time series headers and optionally a time zone. The binary file has the same name as the xml file only the extension is "bin" instead of "xml". During PI time series import the bin file is automatically read when available. The byte order in the bin file is always Intel x86.

...

Root element for defining timeSerieSets to be exported.

timeSerieSets: timeSerieSet

...

Can be used to select all approved forecasts with a forecast start time lying within this period

exportMapStacksActivity

 


Figure 72 Elements of the ExportMapStacksActivity.

description

Optional description of the timeSeries export configuration.

exportFile

Name (and location) of the PI-XML file describing the map stack of the exported grid time series. If the directory location is not explicitly specified the file will be written in the exportDir defined in the general section.

gridFile

Root element for defining file name of grid to be exported

locationId

LocationId for grid to be exported.

gridName

Name of the files for the grid to be exported. For grid files where each time slice is stored in a different file, this name is the prefix for the full file name. The final file name is created using an index of files exported (e.g the file name for the 4^th^ time step is grid00000.004).

gridFormat

Format of the exported grid. Enumeration of options include;

  • asc : for exporting to ARC-INFO ASCII grid format
  • pcrgrid : for exporting to PCRaster native grid file format
  • usgs : for exporting to USGS DEM format (BIL)
timeSerieSet

TimeSeriesSets to be exported. These should contain only one locationId. For exporting multiple grids, multiple exportMapStack activities should be defined.

 

exportProfilesActivity

Configuration of the exportProfiles activity is identical to the exportTimeSeries Activity.

exportDataSetActvity


Figure 73 Elements of the exportDataSets section

description

Optional description of the module dataset export configuration.

moduleInstanceId

Optional reference to the moduleInstanceId of the moduleDataSet to be exported. If not defined the moduleInstanceId of the current module instance is taken as a default (see section on Module Datasets and Parameters).

exportParameterActivity


Figure 74 Elements of the exportParameter section

description

Optional description of the module parameter configuration.

moduleInstanceId

Optional reference to the moduleInstanceId of the moduleParameter to be exported. If not defined the moduleInstanceId of the current module instance is taken as a default (see section on Module Datasets and Parameters) . The parameterfile should be stored in the folder config\ModuleParFiles.

fileName

Name (and location) of the PI-XML file with exported parameters. If the directory location is not explicitly specified the file will be written in the exportDir defined in the general section.

...

Optional description of the ExportTableActivity configuration.

exportFile

File to which the table will be exported. This file is always placed in exportDir.

tableType

Type of table to be exported. Currently enumeration must be "ratingCurve"

operation

ID of the table to be exported.

parameters

Parameters for the convertEquation operation. Must include minimumLevel, maximumLevel and stepSize

locationId/locationSetId

location id to select the rating curve

exportNetcdfActivity

Exports scalar, grid or 1d/2d spectra time series to a NetCDF file. All time series specified inside one exportNetcdfActivity must have the same value type (grid, scalar, 1d spectra or 2d spectra). For this export it is required that all ensembles have exactly the same ensemble member indices. The usage of ensemble member id's (strings) is not supported yet.

...

Optional description of the ExportNetcdfActivity configuration.

exportFile

File to which the data will be exported. This file is always placed in exportDir.

timeSeriesSets

TimeSeriesSet that defines what data is to be exported.

omitMissingValues

Include missing values in the export file or leave them out.

omitEmptyTimeSeries

The time series is not exported when the time series is empty or when; omitMissingValues = true and the time series is empty after removing the missing values.

exportRunFileActivity


Figure 3 Elements of the ExportRunFileActivity configuration

...

Optional description of the ExportRunFileActivity configuration.

exportFile

File to which the data will be exported. This file is always placed in exportDir.

properties

Kind of environment variables for the pre and post adapters. These properties are copied to the run file. This is also a convenient way to pass global properties to a pre or post adapter. An adapter is not allowed to access the FEWS global.properties directly. Global properties (between $) are replace by there literal values before copied to the run file. These extra options makes an additional pre or post adapter configuration file unnecessary.

...

  • string
  • int
  • float
  • double (since stable build 2014.01)
  • bool

 

example of exported run file

Code Block
xml
xml

<?xml version="1.0" encoding="UTF-8"?>
<Run xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.wldelft.nl/fews/PI" xsi:schemaLocation="http://www.wldelft.nl/fews/PI http://fews.wldelft.nl/schemas/version1.0/pi-schemas/pi_run.xsd" version="1.5">
	<logLevel>info</logLevel>
    <timeZone>0.0</timeZone>
    <startDateTime date="1900-01-01" time="00:00:00"/>
    <endDateTime date="2100-01-01" time="00:00:00"/>
    <time0 date="2000-01-01" time="00:00:00"/>
    <workDir>workdir</workDir>
    <outputDiagnosticFile>diagnostic</outputDiagnosticFile>
</Run>

 

exportNetcdfRunFileActivity (since stable build 2014.01)

...

Configuration example
Code Block
xml
xml

<exportNetcdfRunFileActivity>
	<description>This run file is passed as argument to XBeachPreAdapter</description>
	<exportFile>run_info.nc</exportFile>
	<properties>
		<bool key="use_friction" value="true"/>
	</properties>
</exportNetcdfRunFileActivity>

...

An example of a netcdf run file can be downloaded here: run_info_example.nc. Below is the same example netcdf run file, converted to text format:

Code Block

netcdf run_info_example {
dimensions:
	path_length = 255 ;
	input_netcdf_file_count = 2 ;
	input_state_file_count = 1 ;
	output_netcdf_file_count = 2 ;
variables:
	double start_time ;
		start_time:long_name = "start_time" ;
		start_time:standard_name = "time" ;
		start_time:units = "minutes since 2002-11-26 00:00:00.0 +0000" ;
	double end_time ;
		end_time:long_name = "end_time" ;
		end_time:standard_name = "time" ;
		end_time:units = "minutes since 2002-11-26 00:00:00.0 +0000" ;
	char work_dir(path_length) ;
	char input_netcdf_files(input_netcdf_file_count, path_length) ;
	char input_state_files(input_state_file_count, path_length) ;
	char output_netcdf_files(output_netcdf_file_count, path_length) ;
	char properties ;
		properties:string_property = "zs_0 with spaces" ;
		properties:INT_property = 3 ;
		properties:float_PROPERTY = 3.5f ;
		properties:DOUBLE_PROPERTY_321 = 0.123456789 ;
		properties:logical_property_1 = "true" ;
		properties:logical_property_2 = "false" ;

// global attributes:
		:title = "Run file" ;
		:institution = "Deltares" ;
		:source = "Run file from Delft-FEWS" ;
		:history = "2014-03-18 12:22:55 GMT: exported from Delft-FEWS to D:\\fews\\workspace\\fews\\junit_test_output\\nl\\wldelft\\fews\\system\\plugin\\generaladapter\\exportDir\\run_info.nc" ;
		:references = "http://www.delft-fews.com" ;
data:

 start_time = 0 ;

 end_time = 5880 ;

 work_dir = "..\\work" ;

 input_netcdf_files =
  "..\\work\\boundary_data.nc",
  "..\\work\\more_boundary_data.nc" ;

 input_state_files =
  "..\\input_state\\state.inp" ;

 output_netcdf_files =
  "..\\work\\model_output_data1.nc",
  "..\\work\\model_output_data2.nc" ;

 properties = " " ;
}

...

Optional description for the activity.

templateFile

The pathname of the template file relative to the rootDir (or absolute path). This file can contain location attribute tags between '@' signs, general adapter tags between '%' signs and global properties between '$' signs.

exportFile

The pathname of the export file relative to the exportDir (or absolute path). This file is created during the workflow run and will be a copy of the template file in which the tags are replaced with actual (modified) values. If this file already exists, then it will be overwritten.

locationId

Optional. Location attribute tags in the template file will be replaced with the corresponding (modified) attribute values of the location with this locationId. If this is not specified, then location attribute tags will not be replaced.

configuration example
Code Block
xml
xml

<?xml version="1.0" encoding="UTF-8"?>
<generalAdapterRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/generalAdapterRun.xsd">
	<general>
		...
		<startDateTimeFormat>yyyy MM dd HH mm</startDateTimeFormat>
		<endDateTimeFormat>yyyy MM dd HH mm</endDateTimeFormat>
	</general>
	<activities>
		<exportActivities>
			<exportCustomFormatRunFileActivity>
				<templateFile>%ROOT_DIR%/templatefiles/event_tox2_template.inp</templateFile>
				<exportFile>event_tox2.inp</exportFile>
				<locationId>locationWithAttributes</locationId>
			</exportCustomFormatRunFileActivity>
		</exportActivities>
	</activities>
</generalAdapterRun>

Example of template file:

Code Block

%START_DATE_TIME%                     ! MODEL START         : ISYEAR,ISMONTH,ISDATE,ISHR,ISMN [year,month,day,hour,minute]
%END_DATE_TIME%                       ! MODEL STOP          : ISYEAR,ISMONTH,ISDATE,ISHR,ISMN [year,month,day,hour,minute]
2                                     ! TIME STEP           :[sec]
@TOXIC_ID@                            ! TOXIC ID            : CAS NO.-TOXIC, 444#-OI
@ACCIDENT_LOC_X@     @ACCIDENT_LOC_Y@ ! ACCIDENT LOC.       : X   Y      TM coordinate (referred in LXLY.INP)
@ACCIDENT_TIME@                       ! ACCIDENT TIME       : IEVYEAR,IEVMONTH,IEVDATE,IEVHR,IEVMN [year,month,day,hour,minute]
@spill_duration@                      ! spill duration      : [MIN]
@spill_material_mass@                 ! spill material mass : [G]
10.                                   ! CHLA                : CHLA CONC. [UG/L] FOR TOXIC CALCULATION
5.                                    ! TEM                 : WATER TEMPERATURE [DEGREE C] FOR TOXIC CALCULATION
5.                                    ! SSC                 : Suspended Solid Conc.  [MG/L] FOR TOXIC CALCULATION
0.1                                   ! DOC                 : DOC CONC.  [MG/L] FOR TOXIC CALCULATION
100.                                  ! I                   : IRADIATION [LY/DAY]
5.                                    ! WSPD                : WINDSPEED  [M/S]
$SIMULATION_NUMBER$                   ! simulation number
%TIME0%$Identifier$%TIME0%   $Identifier$ ! test

Example of exported file:

Code Block

2002 11 30 09 00                     ! MODEL START         : ISYEAR,ISMONTH,ISDATE,ISHR,ISMN [year,month,day,hour,minute]
2002 11 30 11 00                       ! MODEL STOP          : ISYEAR,ISMONTH,ISDATE,ISHR,ISMN [year,month,day,hour,minute]
2                                     ! TIME STEP           :[sec]
131-52-2                            ! TOXIC ID            : CAS NO.-TOXIC, 444#-OI
235875.9     329956.4 ! ACCIDENT LOC.       : X   Y      TM coordinate (referred in LXLY.INP)
2012 01 10 09 00                       ! ACCIDENT TIME       : IEVYEAR,IEVMONTH,IEVDATE,IEVHR,IEVMN [year,month,day,hour,minute]
10.0                      ! spill duration      : [MIN]
500000.0                 ! spill material mass : [G]
10.                                   ! CHLA                : CHLA CONC. [UG/L] FOR TOXIC CALCULATION
5.                                    ! TEM                 : WATER TEMPERATURE [DEGREE C] FOR TOXIC CALCULATION
5.                                    ! SSC                 : Suspended Solid Conc.  [MG/L] FOR TOXIC CALCULATION
0.1                                   ! DOC                 : DOC CONC.  [MG/L] FOR TOXIC CALCULATION
100.                                  ! I                   : IRADIATION [LY/DAY]
5.                                    ! WSPD                : WINDSPEED  [M/S]
73                   ! simulation number
200211300900FSS200211300900   FSS ! test

 

exportAreaSelectionActivity (currently NGMS only)

...

Optional description for the activity. Used for reference purposes only.

exportFile

File name of the file to be exported. Always placed into the export dir.

...

Identifier for locationset for which to generate grid files. The locations should have polygons associated with them.

Execute Activities


Elements of the ExecuteActivity configuration

...

Configuration example:

Code Block
xml
xml

<logFile>
	<file>XBerror.txt</file>
	<!-- Import every line as a separate FEWS error log message. -->
	<errorLinePattern>*</errorLinePattern>
</logFile>
<logFile>
	<file>XBwarning.txt</file>
	<!-- Import every line that contains "warning" as a separate FEWS info log message. -->
	<infoLinePattern>*warning*</infoLinePattern>
</logFile>
<logFile>
	<file>XBlog.txt</file>
	<!-- Import every line that contains "ERROR" as a separate FEWS debug log message. -->
	<debugLinePattern>*ERROR*</debugLinePattern>
</logFile>

...


Figure 76 Elements of the ImportActivities configuration

exportPlaceholderFile
Note

Option exportPlaceholderFile is not supported for spectra data.

...

The intention of this exportPlaceholderFile functionality is that the model cq modeladapter reads the placeholders to see which timeseries are required by FEWS.
After simulation, the model cq modeladapter overwrites these files with its own data over the placeholder files ready to be imported by the import activity.

Code Block
xml
xml

<importTimeSeriesActivity>
	<exportPlaceholderFile>true</exportPlaceholderFile>
	<importFile>output.xml</importFile>
	<timeSeriesSets>
        ....
	</timeSeriesSets>
</importTimeSeriesActivity>

...

Optional description of import activity. Used for reference purposes only

importStateActivity


Elements of the ImportStateActivity section.

Root element for importing modules states resulting from the run of the external modules. Multiple elements may be defined. If no state is to be imported (for example in forecast run as opposed to state run), then the element should not be defined.

  • description - Optional description for the activity.
  • stateConfigFile - Deprecated, do not use this if a pi state description xml file is not needed. Name (and location) of the PI-XML file describing the states to be imported. This file contains all necessary information to define state type and location. The moduleInstanceId of the state imported is per definition the current module instance. This should be either an absolute path or a path relative to the importDir defined in the general section. This option reads the output state file paths from a pi state description xml file.
  • stateImportDir - Optional state import directory. This should be either an absolute path or a path relative to the importDir defined in the general section. If this stateImportDir is specified, then the importFile paths in this activity can be paths relative to this stateImportDir instead of absolute paths. This stateImportDir is only needed when you want to use relative importFile paths. If all importFile paths in this activity are absolute, then this stateImportDir is not needed and will not be used. This option does not use a pi state description xml file.
  • stateFile - One or more output state files from the model. The specified files will be imported. This option does not use a pi state description xml file.
  • importFile - Path and name of an output state file from the model. This file will be imported. This should be either an absolute path or a path relative to the stateImportDir defined in this activity. The %END_DATE_TIME% tag can be used here (since 2014.02), in case the state file name contains a timestamp of the end time of the run.
  • relativeExportFile - This relative path and name are used to store the imported file in FEWS. This path should be relative to the stateExportDir in the exportStateActivity that will be used to export this state file again for a future model run. If the imported output state file needs to be renamed before it can be used as input state file for a future model run, then the name of the relativeExportFile can be different from the name of the importFile. The %END_DATE_TIME% tag can be used here (since 2014.02), in case the state file name contains a timestamp of the end time of the run.
  • compressedStateLocation - Optional. By default the warm state is zipped and stored as a blob in the database. For large states (>50MB) it is recommended to store the data outside the database in a directory. This directory is configured in the clientConfig.xml (optional element warmStatesDirectory). When using a stand alone system or using oracle and a single MC system it is possible to store larger grids inside the database. Expired states are removed automatically from this directory by the CompactCacheFiles workflow. The state description is still written to the database, the blob field will be empty.
  • expiryTime - Optional. When the state is an intermediate result in a forecast run you can make the state expire. By default the expiry time is the same as for the module instance run.
  • synchLevel - Optional synch level for state. Defaults to 0 is not specified (i.e. same as data generated by the forecast run)

Configuration example:

New example that does not use a pi state description xml file. In this case, the output state file paths are configured directly in the importStateActivity as absolute paths:

Code Block
xml
xml

<importStateActivity>
	<stateFile>
		<importFile>%WORK_DIR%/state.out</importFile>
		<!-- Rename imported state file so that it can be used as input state for a future model run. -->
		<relativeExportFile>state.inp</relativeExportFile>
	</stateFile>
	<stateFile>
		<importFile>%WORK_DIR%/state2.out</importFile>
		<!-- Rename imported state file so that it can be used as input state for a future model run. -->
		<relativeExportFile>state2.inp</relativeExportFile>
	</stateFile>
</importStateActivity>

New example that does not use a pi state description xml file. In this case, the output state file paths are configured directly in the importStateActivity as paths relative to a stateImportDir:

Code Block
xml
xml

<importStateActivity>
	<stateImportDir>%WORK_DIR%</stateImportDir>
	<stateFile>
		<importFile>state.out</importFile>
		<!-- Rename imported state file so that it can be used as input state for a future model run. -->
		<relativeExportFile>state.inp</relativeExportFile>
	</stateFile>
	<stateFile>
		<importFile>state2.out</importFile>
		<!-- Rename imported state file so that it can be used as input state for a future model run. -->
		<relativeExportFile>state2.inp</relativeExportFile>
	</stateFile>
</importStateActivity>

Old example that reads the input state file paths from a pi state description xml file. Do not use this if a pi state description xml file is not needed:

Code Block
xml
xml

<importStateActivity>
	<stateConfigFile>pi_output_state_description.xml</stateConfigFile>
</importStateActivity>

...

  • importFile - PI-XML file describing the time series to be imported. The file contains all information on type of data to be imported (scalar, longitudinal, grid, polygon). For all data types except the grid the file also contains the time series data If the directory location is not explicitly specified the file will be expected to be read from the importDir defined in the general section.

 

importMapStacksActivity

Root element for importing grid time series resulting from the run of the external modules. Multiple elements may be defined. importFile and timeSeriesSet should be defined.

importNetcdfActivity

Imports scalar, grid or 1d/2d spectra time series from a NetCDF file resulting from the run of the external modules. All time series specified inside one importNetcdfActivity must have the same value type (grid, scalar, 1d spectra or 2d spectra). For this activity the exportPlaceholderFile option only works for scalar and grid data, not for spectra data. Note: this activity is exactly the same as importPiNetcdfActivity, only the name is better, since the NetCDF format has nothing to do with the PI format.

  • maximumSnapDistance (since stable build Since 2014.01) . FEWS-10771. Optional maximum horizontal snap distance in meters. When the parser provides horizontal location coordinates (x,y) and no locationIds, then the location mapping can will be done by matching the horizontal coordinates. The horizontal snap distance is the tolerance used to detect which internal and external horizontal coordinates are the same. This only works when the input format provides the coordinate system for the coordinates of the locations. When the parses does not provide the coordinates for a time series an error is logged. Note: this option has no effect for grid data.

importPiNetcdfActivity

Deprecated, do not use. Please use importNetcdfActivity instead. Imports scalar, grid or 1d/2d spectra time series from a NetCDF file resulting from the run of the external modules. All time series specified inside one importPiNetcdfActivity must have the same value type (grid, scalar, 1d spectra or 2d spectra). For this activity the exportPlaceholderFile option only works for scalar and grid data, not for spectra data.

  • maximumSnapDistance (since stable build 2014.01) - Optional maximum snap distance in meters. When the parser provides location coordinates the location mapping can maximumVerticalSnapDistance Since 2014.02. Optional maximum vertical snap distance in meters. When the parser provides vertical location coordinates (z) and no locationIds, then the location mapping will be done by matching the vertical coordinates. The vertical snap distance is the tolerance used to detect which internal and external vertical coordinates are the same. This only works when the input format provides the coordinate system for the coordinates of the locations. When the parses does not provide the vertical coordinates for a time series an error is logged. Note: this option has no effect for grid datacurrently only works for netcdf grid data.
  • startWhileRunningExecuteActivities Default is false. If this is true, then this importActivity will run continuously during the configured execute activities. Additionally this importActivity will also run as part of the configured import activities as normal. This way it is possible to import data that is produced by an execute activity, while it is being produced. For instance if a model run writes new output data to an existing file after each timeStep, then the continuously running importActivity will immediately import the file, including the new data. This way the new data can be viewed in FEWS as soon as it becomes available, i.e. already during the model run. Currently the data that is imported during the execute activities can only be viewed after selecting "open most recent running forecast and adjust system time" from the debug menu in the FEWS Explorer window. If the running forecast is opened and selected in the data viewer, then the displays are updated each time when new data becomes available during the run. This feature only has effect for stand alone FEWS systems and for FEWS systems that use direct database access.

importPiNetcdfActivity

Deprecated, do not use. Please use importNetcdfActivity instead. Imports scalar, grid or 1d/2d spectra time series from a NetCDF file resulting from the run of the external modules. All time series specified inside one importPiNetcdfActivity must have the same value type (grid, scalar, 1d spectra or 2d spectra). For this activity the exportPlaceholderFile option only works for scalar and grid data, not for spectra data. For more documentation see importNetcdfActivity.

importProfilesActivity

Root element for importing longitudinal profile time series resulting from the run of the external modules. Multiple elements may be defined. importFile and timeSeriesSet should be defined.

Shutdown Activities


Elements of the Shutdown Activities configuration
This activity is the identical to the startUpActivities. The only difference is that these are carried out after the module run and import of data. See definition of StartUp activities for configuration.