EXTERNAL_FORECAST_TIME
What | nameofinstance.xml |
---|---|
Required | no |
Description | Export data (timeseries) from Delft-Fews to several file formats |
schema location |
Entry in ModuleDescriptors
<className>nl.wldelft.fews.system.plugin.dataExport.TimeSeriesExport</className>
</moduleDescriptor>
Excerpt | ||
---|---|---|
| ||
Exports data to several file formats |
...
This elements describes how to construct the filename(s) of the exported file(s).
If only the name element is given a fixed name is used for each export. The prefix and suffix elements describe how to create a filename prefix and/or suffix. The temporaryPrefix is used to generate a prefix for the temporary file as it is being written. After that the file is renamed.
validate
The option useExternalLocationIdAsName (instead of the regular file name) forces the use of the location ID or Name as filename. Note that this option does not work in combination with an idMap that filters the locations to be exported.
validate
Optional Optional element. Only applicable if the data are exported to the xml-file. This option activates the validation of the exported file against a XML schema.
...
Id of IdMap to be used for parameterId and locationId mapping
unitConversionsId
externalLocationNameFunction (since Delft-FEWS 2023.2)
For export functions that can write location names along side location ID's, one can choose the location attribute that should be parsed to the location name field. It requires mentioning the attribute between @ signs; e.g.: <externalLocationNameFunction>@externalLocationName@</externalLocationNameFunction>
unitConversionsId
Id of UnitConversions to be used Id of UnitConversions to be used for unit mapping
flagConversionsId
...
NetcdfScalarTimeSeriesSerializer
NetcdfMatroosScalarTimeSeriesSerializer
NetcdfScalarNonEquidistantTimeSeriesSerializer
NetcdfVerticalProfileSerializer
NetcdfGridTimeSeriesSerializer
NetcdfZLayersTimeSeriesSerializer
NetcdfDomainTimeSeriesSerializer
omitMissingValues
If set to true records with missing values are not exported
precision
Available since 2018.02. Optional element to set the number of decimals all values should be displayed with. If set, additional zeros will be appended and/or values will be rounded when necessary.
It is possible to configure a valueResolution for parameters via the parameters.xml. The configured precision for time series with parameters which have a value resolution should never exceed the maximum number of decimals needed to display values with this resolution. If the precision does exceed this, a warning will be given and the configured precision will be ignored.
The precision can never exceed 8 decimals due to limitations on the resolution with which values can be stored in the FEWS database (floating point errors).
exportTimeZone
TimeZone in which to export the data. Can either be a string (timeZoneName) or an offset (timeZoneOffset).
convertDatum
Convert (vertical) datum to local datum during export. The conversion will be done for all parameters which use datum (as configured in Parameters.xml) The local datum is defined in the z element in the locations.xml file.
geoDatum
Convert the geographical coordinate system (horizontal datum and projection) to specified geoDatum during export. Not all serializers support this parameter so please check the documentation for a particular serializer to see if it is supported.
ensembleMemberFormat
Available since 2019.02. Can either have value 'name' or 'index'. If 'name' is configured, the ensemble member Id is written. Otherwise the ensemble member index is written.
forecastSelectionPeriod
If configured all forecasts with a forecast time within the configured period will be exported.
When also configuring a <timeZeroFormattingString> in the <prefix> of the <exportFileName>, each forecast will be exported to a separate file to easily differentiate between the different forecasts.
When no <timeZeroFormattingString> is configured in the <exportFileName> all forecasts will be exported to the same file.
Example configuration:
Code Block | ||
---|---|---|
| ||
<export>
<general>
<exportType>SomeValidExportType</exportType>
<folder>MyExportFolder</folder>
<exportFileName>
<name>_MyExportedFile.txt</name>
<prefix>
<timeZeroFormattingString>yyyyMMddHHmm</timeZeroFormattingString>
</prefix>
</exportFileName>
<idMapId>MyIdMap</idMapId>
<forecastSelectionPeriod start="-2" end="0" unit="day"/>
</general>
...
</export> |
exportManualChanges
If used, only manual changed to the data will be exported. Unless the manualDBChangeViewPeriod is used, the relativeViewPeriod from the associated timeSeriesSet is used.
exportLocationAttributeAsNetCDFVariable
Since 2021.01. Adds a variable to the NetCdf file. Name of the variable is the value of ncVariable, the value is the configured location attribute of attributeId.
config example:
Code Block | ||||
---|---|---|---|---|
| ||||
<exportLocationAttributeAsNetCDFVariable>
<ncVariable>LocationAlias</ncVariable>
<attributeId>niceName</attributeId>
</exportLocationAttributeAsNetCDFVariable> |
Example result (in txt format) of using exportLocationAttributeAsNetCDFVariable:
Code Block | ||
---|---|---|
| ||
netcdf {
dimensions:
time = 3;
stations = 6;
char_leng_id = 64;
char_leng_name = 255;
char_leng_attribute = 255;
variables:
double time(time);
time:standard_name = "time";
time:long_name = "time";
time:units = "minutes since 1970-01-01 00:00:00.0 +0000";
time:axis = "T";
double lat(stations);
lat:standard_name = "latitude";
lat:long_name = "Station coordinates, latitude";
lat:units = "degrees_north";
lat:axis = "Y";
lat:_FillValue = 9.96921E36;
double lon(stations);
lon:standard_name = "longitude";
lon:long_name = "Station coordinates, longitude";
lon:units = "degrees_east";
lon:axis = "X";
lon:_FillValue = 9.96921E36;
double y(stations);
y:standard_name = "latitude";
y:long_name = "y coordinate according to WGS 1984";
y:units = "degrees_north";
y:axis = "Y";
y:_FillValue = 9.96921E36;
double x(stations);
x:standard_name = "longitude";
x:long_name = "x coordinate according to WGS 1984";
x:units = "degrees_east";
x:axis = "X";
x:_FillValue = 9.96921E36;
double z(stations);
z:long_name = "height above mean sea level";
z:units = "meters";
z:_FillValue = 9.96921E36;
char station_id(stations, char_leng_id);
station_id:long_name = "station identification code";
station_id:cf_role = "timeseries_id";
char station_names(stations, char_leng_name);
station_names:long_name = "station name";
char LocationAlias(stations, char_leng_attribute);
LocationAlias:niceName = "location attribute value";
float waterlevel(time, stations);
waterlevel:standard_name = "water_surface_height_above_reference_datum detection_minimum";
waterlevel:long_name = "waterlevel";
waterlevel:units = "m";
waterlevel:_FillValue = -9999.0f;
waterlevel:coordinates = "lat lon";
waterlevel:cell_methods = "time: maximum";
// global attributes:
:Conventions = "CF-1.6";
:title = "Data";
:institution = "Deltares";
:source = "Export NETCDF-CF_TIMESERIES from Delft-FEWS";
:history = "actual history attribute text replaced by dummy text in unit test";
:references = "http://www.delft-fews.com";
:Metadata_Conventions = "Unidata Dataset Discovery v1.0";
:summary = "Data exported from Delft-FEWS";
:date_created = "actual date_created attribute text replaced by dummy text in unit test";
:fews_implementation_version = "0.0";
:fews_build_number = "development";
:coordinate_system = "WGS 1984";
:featureType = "timeSeries";
:time_coverage_start = "2003-03-01T00:00:00+0000";
:time_coverage_end = "2003-03-01T00:30:00+0000";
:geospatial_lon_min = "-3.5458";
:geospatial_lon_max = "-2.15939";
:geospatial_lat_min = "51.74003";
:geospatial_lat_max = "52.76901";
data:
time =
{1.744128E7, 1.7441295E7, 1.744131E7}
lat =
{51.93591, 51.74003, 52.76901, 52.55523, 52.19899, 52.43011}
lon =
{-2.15939, -2.2469, -3.1088, -2.37608, -2.38931, -3.5458}
y =
{51.93591, 51.74003, 52.76901, 52.55523, 52.19899, 52.43011}
x =
{-2.15939, -2.2469, -3.1088, -2.37608, -2.38931, -3.5458}
z =
{1.1, 2.2, 3.3, 9.96921E36, 5.5, 6.6}
station_id ="26", "27", "28", "24", "29", "25"
station_names ="Slate Mill", "Ebley Mill", "Llanymynech", "Burcote", "Knightsford Bridge", "Rhos-Y-Pentref"
LocationAlias ="A-attribute-2026", "A-attribute-2027", "A-attribute-2028", "A-attribute-2024", "A-attribute-2029", "A-attribute-2025"
waterlevel =
{
{-9999.0, 1.21, 1.31, 1.41, 1.51, 1.61},
{-9999.0, 2.21, 2.31, 2.41, 2.51, 2.6100001},
{-9999.0, 3.21, 3.31, 3.41, 3.51, 3.6100001}
}
} |
omitMissingValues
If set to true records with missing values are not exported
exportLastValue
If set to true only the last value will be exported
precision
Available since 2018.02. Optional element to set the number of decimals all values should be displayed with. If set, additional zeros will be appended and/or values will be rounded when necessary.
It is possible to configure a valueResolution for parameters via the parameters.xml. The configured precision for time series with parameters which have a value resolution should never exceed the maximum number of decimals needed to display values with this resolution. If the precision does exceed this, a warning will be given and the configured precision will be ignored.
The precision can never exceed 8 decimals due to limitations on the resolution with which values can be stored in the FEWS database (floating point errors).
exportTimeZone
TimeZone in which to export the data. Can either be a string (timeZoneName) or an offset (timeZoneOffset).
convertDatum
Convert (vertical) datum to local datum during export. The conversion will be done for all parameters which use datum (as configured in Parameters.xml) The local datum is defined in the z element in the locations.xml file.
geoDatum
Convert the geographical coordinate system (horizontal datum and projection) to specified geoDatum during export. Not all serializers support this parameter so please check the documentation for a particular serializer to see if it is supported.
ensembleMemberFormat
Available since 2019.02. Can either have value 'name' or 'index'. If 'name' is configured, the ensemble member Id is written. Otherwise the ensemble member index is written.
forecastSelectionPeriod
If configured all forecasts with a forecast time within the configured period will be exported. Since 2020.01 all forecasts will be exported to the same file.
When also configuring a <timeZeroFormattingString> in the <prefix> of the <exportFileName>, each forecast will be exported to a separate file to easily differentiate between the different forecasts.
Note: earlier versions export a separate file for each forecast by default. The forecast time is used to create a file extension (format yyyyMMddHH), unless <timeZeroFormattingString> is configured
Example configuration:
Code Block | ||
---|---|---|
| ||
<export>
<general>
<exportType>SomeValidExportType</exportType>
<folder>MyExportFolder</folder>
<exportFileName>
<name>_MyExportedFile.txt</name>
<prefix>
<timeZeroFormattingString>yyyyMMddHHmm</timeZeroFormattingString>
</prefix>
</exportFileName>
<idMapId>MyIdMap</idMapId>
<forecastSelectionPeriod start="-2" end="0" unit="day"/>
</general>
...
</export> |
exportManualChanges
If used, only manual changed to the data will be exported. Unless the manualDBChangeViewPeriod is used, the relativeViewPeriod from the associated timeSeriesSet is used. Note: the view period is calculated relative to the dispatch time and not T0
exportChanges
If configured, any changes to the data in the configured period will be exported. Unless the dbChangeViewPeriod is configured, the relativeViewPeriod from the associated timeSeriesSet is used . Note: the view period is calculated relative to the dispatch time and not T0
An example:
Code Block |
---|
<general>
<exportType>PI</exportType>
<folder>$EXPORT_FOLDER$</folder>
<exportFileName>
<name>exportedTimeSeries.xml</name>
</exportFileName>
<exportChanges>
<dbChangeViewPeriod unit="day" multiplier="2"/>
</exportChanges>
</general> |
columnSeparator and decimalSeparator
...
netCDF4DeflateLevel
This property only works with netcdf4.
With this property the deflate level for writing compressed netcdf files can be set from 0 to 9. 0 meaning no compression and 9 maximum compression. Default will be 5, this level gives best compression without losing too much time when reading or writing.
metadata
(Meta data export has only been implemented for a limited set of export types. Currrently the NetCDF, LILA and HHRR types export meta data)
only works with netcdf4.
With this property the deflate level for writing compressed netcdf files can be set from 0 to 9. 0 meaning no compression and 9 maximum compression. Default will be 5, this level gives best compression without losing too much time when reading or writing.
metadata
(Meta data export has only been implemented for a limited set of export types. Currrently the NetCDF, grid2shp, LILA and HHRR types export meta data)
Optional metadata that is written in the exported file. The options netcdfMapDPhase and alertMapDPhase are deprecated (do not use these). For the other options it is possible to use the following tags:
%TIME_ZERO% the T0 of this time series export run.
%CURRENT_TIME% the current time.
%COLD_STATE_TIME% the cold state time.
%FORECAST_END_TIME% the forecast time set by the forecast length estimator or the forecast length option in the manual forecast dialog
%MODULE_INSTANCE_ID% the id of this module instance.
%MODULE_INSTANCE_NAME% the name of this module instance.
%MODULE_INSTANCE_DESCRIPTION% the configured description of this module instance.
%MODULE_INSTANCE_ATTRIBUTE(attributeId, moduleInstanceId)% moduleInstanceId is optional. When not specified the module instance of the module itself is used, like for the %MODULE_INSTANCE_ID% tagOptional metadata that is written in the exported file. The options netcdfMapDPhase and alertMapDPhase are deprecated (do not use these). For the other options it is possible to use the following tags:
%TIME_ZERO% the T0 of this time series export run.
%CURRENT_TIME% the current time.
%MODULE_INSTANCE_ID% the id of this module instance.
%MODULE_INSTANCE_NAME% the name of this module instance.
%MODULE_INSTANCE_DESCRIPTION% the configured description of this module instance.
%WORKFLOW_ID% the id of the workflow in which this export runs.
%WORKFLOW_NAME% the name of the workflow in which this export runs.
%WORKFLOW_DESCRIPTION% the configured description of the workflow in which this export runs.
...
%EXTERNAL_FORECAST_TIME% - external analysis time. When configuring it, it needs two parameters: the first is the external forecast isid, the second should be the time format. Neither the id not nor the time format should contain "," (a comma). The two arguments should be separated bz by a comma.
Example: %EXTERNAL_ANALYSISFORECAST_TIME(thisIsTheId),(yyyy/MM/dd HH:mm:ss z)%
%COLD_STATE_TIME(yyyy/MM/dd HH:mm:ss z)% - the cold state start time. If data is unavailable it will be filled as "Unknown".
Since 2022.02 location/parameter/qualifier/moduleInstance attributes between @ are also recognized.
Configuration example of metadata:
Code Block |
---|
<metadata> <title>title</title> <institution> institution </institution> <source>source</source> <history>Exported at time zero = %TIME_ZERO(yyyy/MM/dd HH:mm:ss z)% in module instance %MODULE_INSTANCE_ID% as part of workflow %WORKFLOW_NAME% by user %USER_ID%.</history> <references>references</references> <comment>The actual time of writing was %CURRENT_TIME(yyyy-MM-dd HH:mm:ss z)%</comment> <summary>A summary of the data for data<@ATTRIBUTEID@</summary> <keyword>keyword1</keyword> <keyword> keyword with lots of spaces </keyword> <keyword>keyword 3</keyword> <customAttributes> <string key="emptyAttribute" value=" "/> <int key=" custom2 " value="123456"/> <string key="custom_3" value="This is a custom attribute with 'quotes' in it."/> <string key=" " value="attribute with empty key specified is not written"/> <float key="just_another_float" value="3.5"/> <bool key="truth" value="true"/> </customAttributes> </metadata> |
...