Introduction on Expiry time

One of the most important concepts of Delft-FEWS is that always the latest forecast data is available to be used. In a typical operational Delft-FEWS forecasting system, new data is constantly imported, while older data is no longer relevant and can be removed. For a lot of database tables in the central Delft-FEWS database, the rows (records) in the database have an expiryTime column specifying when the data can be removed. This expiryTime is set at creationTime. The removal is done by a continous running process in the Master Controller called the McRollingBarrel. The McRollingBarrel - in 10 minute intervals - removes data for which the expiry time is exceeded and no dependent rows are available. This is often critical for maintaining an optimal application performance.


The global property DEFAULT_EXPIRY_DAYS (default 10 days) is a global property that controls the default expiration time for all forecast data that is stored in the Delft-FEWS database.


This will override the expiryTime in days when log entries created by the client will expire.


This will override the expiryTime in days when log entries created by the client will expire.

Expiry Time for TimeSeries

Delft-FEWS allows for fine grained control of expiry times for timeseries by specifying the default in a global propery and overriding the expiry time on a deeper level for instance in the configuration of the import. Once configured, it is good practive to verify the expiry times of imported data using Time Series Lister.

  1. The System default expiry time for forecast data in Delft-FEWS is 10 days. The system default can be altered by specifying a global property, e.g. DEFAULT_EXPIRY_DAYS=30

  2. Import workflows can have expiry time in general section, will be used for all imported time series

    <expiryTime unit="day" multiplier="365"/>
  3. Import, Transformation, General Adapter, .. modules can have expiry times in timeSeriesSet element, e.g.

    <expiryTime unit="day" multiplier="2"/>

The expiryTime value in the TimeSeriesSet configuration of a TimeSeriesImport overrides DEFAULT_EXPIRY_DAYS setting for the expiry time of TimeSeries and ImportStatus. If an expiryTime is defined at the TimeSeriesSet level, the time series blobs will get this expiry time, no matter what the workflow says.
If the timeseries is of an external type, the timeseries will also stay accessible. If a time series is of the simulated type, data access has to go through a query where the taskrun record plays a major role. This taskrun record expirytime is defined by the workflow.

Use globalProperties for controlling expiryTime timeSeriesSets...

When expiryTime of many TimeSeriesSets are conceptually related, it is recommended to explicitely set these using a globalProperty variable using tags.

Guidelines for setting expiry times

Set the expiry times in the configuration as brief as possible in order to keep the database tables as small as possible and performing well. If needed use the Open Archive to store data for longer. When the Delft-FEWS is used as a forecasting system it is common to use 2, 7 or 10 days whereas Water Information System uses much longer periods. Expiry time should be what is required for the daily work, e.g.

    1. Observed and processed (external historical) time series, 2-6 months (>1 year for Astro)
    2. Historic simulations (simulated historical) time series and model states, 10 days to 6 months
    3. Imported forecasts (external forecasts) time series, 2 hours to 7 days
    4. Forecast simulations (simulated forecasts) time series, 2 to 10 days

NB. Another means of For intermediate data processing steps - that do not need to be displayed - the data does not even need to used in displays, use temporary series (synchlevel 9)

WorkflowDescriptors runExpiryTime

This setting overrides the expiry time of the taskrun and everything attached to it. The ‘runExpiryTime’ applies only to forecast runs, so only to time series created in a forecast run. Imported time series (external historicals or external forecasts) are not connected to a particular workflow run, so ‘runExpiryTime’ will not work for imported timeseries.

Expiry time for thresholdEvents

The expiryTime value of thresholdEvents is the maximum of 365 days and the DEFAULT_EXPIRY_DAYS setting. There is also a eventExpiryTime in the ThresholdGroups.xml

Expiry time for Forecaster Notes

Forecaster notes have a default expiry time of 5 days. This can be overruled per event code in the ForecasterNotesDisplay.xml file. 

<eventCode id="Comms"><expiryTime unit="day" multiplier="10"/></eventCode>