21 Timeseries Availability Checker

<moduleDescriptor id="TimeSeriesAvailabilityChecker">
	<description>TimeSeriesAvailabilityChecker</description>
	<className>nl.wldelft.fews.system.plugin.timeseriesavailabilitychecker.TimeSeriesAvailabilityChecker</className>
</moduleDescriptor>

Timeseries Availability Checker (since build 24501)

The timeseries availability checker module can be used to perform certain checks on time series data. A check can generate log messages, which can trigger actions in the master controller, like e.g. sending warning e-mails.

Configuration

When available as configuration on the file system, the name of an XML file for configuring an instance of the timeseries availability checker module called for example CheckImportedData may be:

CheckImportedData 1.00 default.xml

In a timeseries availability checker configuration file one or more checks can be configured. The configured checks will be performed one by one. Each check will check the configured time series one by one on the configured properties. If a time series does not pass the check, then the configured log message is logged. See below for more details about each part of the configuration.

Time Series Definitions

The configuration contains definitions of one or more time series that can be used as input for checks. Each timeSeries contains a timeSeriesId and a timeSeriesSet. The timeSeriesId can be used to reference the time series in a check. Alternatively time series definitions can be embedded in the checks.

Checks

The configuration contains one or more checks. Four different types of checks are available:

• minNumberOfValuesCheck: Checks whether there are enough values within a configured period. If not, then a log message with the configured event code and level is logged. The log event code can be used to trigger a certain action in the master controller, e.g. sending warning emails.
• minNonMissingValuesCheck: Checks whether there are enough non-missing values within a configured period. If not, then a log message with the configured event code and level is logged. The log event code can be used to trigger a certain action in the master controller, e.g. sending warning emails. A non-missing value is a value that is reliable, doubtful or unreliable.
• minReliableOrDoubtfulValuesCheck: Checks whether there are enough values that are reliable or doubtful within a configured period. If not, then a log message with the configured event code and level is logged. The log event code can be used to trigger a certain action in the master controller, e.g. sending warning emails.
• minReliableValuesCheck: Checks whether there are enough reliable values within a configured period. If not, then a log message with the configured event code and level is logged. The log event code can be used to trigger a certain action in the master controller, e.g. sending warning emails.

Contents of Checks

The minNumberOfValuesCheck, minNonMissingValuesCheck, minReliableOrDoubtfulValuesCheck and minReliableValuesCheck all consist of the following elements:

• id: Identifier of the check. This is only used in log messages and exception messages.
• timeSeries: One or more time series that need to be checked. This can be either an embedded timeSeriesSet or a reference to a time series defined at the start of the configuration file. If this contains multiple time series (e.g. for multiple locations), then each time series is checked individually.
• checkRelativePeriod: The check will only consider data in this time period. This time period is relative to the timeZero of the taskrun in which the module instance runs. The start and end of the period are included. This period overrules any relativeViewPeriods specified in the timeSeriesSets of the time series.
• minNumberOfValues: The minimum required number of values in the time series to pass the check.
• logLevel: Log level for the log message that is logged if a time series does not pass the check. Can be DEBUG, INFO, WARN, ERROR or FATAL. If level is error or fatal, then the module will stop running after logging the first log message.
• logEventCode: Event code for the log message that is logged if a time series does not pass the check. This event code has to contain a dot, e.g. "TimeSeries.Check", because the log message is only visible to the master controller if the event code contains a dot.
• logMessage: Log message that is logged if a time series does not pass the check. It is possible to use the tag %HEADER% in the logMessage. The %HEADER% tag will be replaced with the header of the time series.

Configuration Example

A configuration example with four different checks configured is given below:

<timeSeriesAvailabilityChecker xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/timeSeriesAvailabilityChecker.xsd">
	<timeSeries>
		<timeSeriesId>input1</timeSeriesId>
		<timeSeriesSet>
			<moduleInstanceId>CheckImportedData</moduleInstanceId>
			<valueType>scalar</valueType>
			<parameterId>H.obs</parameterId>
			<locationId>location1</locationId>
			<timeSeriesType>external historical</timeSeriesType>
			<timeStep unit="minute" multiplier="15"/>
			<!-- any relativeViewPeriod here will always be overruled by checkRelativePeriod in each check -->
			<readWriteMode>read only</readWriteMode>
		</timeSeriesSet>
	</timeSeries>
	<timeSeries>
		<timeSeriesId>input2</timeSeriesId>
		<timeSeriesSet>
			<moduleInstanceId>CheckImportedData</moduleInstanceId>
			<valueType>scalar</valueType>
			<parameterId>H.obs</parameterId>
			<locationId>location2</locationId>
			<timeSeriesType>external historical</timeSeriesType>
			<timeStep unit="minute" multiplier="15"/>
			<!-- any relativeViewPeriod here will always be overruled by checkRelativePeriod in each check -->
			<readWriteMode>read only</readWriteMode>
		</timeSeriesSet>
	</timeSeries>
	<minReliableValuesCheck id="MinReliableValuesCheck">
		<timeSeries>
			<timeSeriesId>input1</timeSeriesId>
		</timeSeries>
		<timeSeries>
			<timeSeriesId>input2</timeSeriesId>
		</timeSeries>
		<checkRelativePeriod unit="hour" start="-12" end="0"/>
		<minNumberOfValues>6</minNumberOfValues>
		<logLevel>WARN</logLevel>
		<logEventCode>TimeSeries.Check</logEventCode>
		<logMessage>Not enough reliable values available for time series %header%</logMessage>
	</minReliableValuesCheck>
	<minReliableOrDoubtfulValuesCheck id="MinReliableOrDoubtfulValuesCheck">
		<timeSeries>
			<timeSeriesId>input1</timeSeriesId>
		</timeSeries>
		<timeSeries>
			<timeSeriesId>input2</timeSeriesId>
		</timeSeries>
		<checkRelativePeriod unit="hour" start="-12" end="0"/>
		<minNumberOfValues>12</minNumberOfValues>
		<logLevel>WARN</logLevel>
		<logEventCode>TimeSeries.Check</logEventCode>
		<logMessage>Not enough reliable or doubtful values available for time series %header%</logMessage>
	</minReliableOrDoubtfulValuesCheck>
	<minNonMissingValuesCheck id="MinNonMissingValuesCheck">
		<timeSeries>
			<timeSeriesId>input1</timeSeriesId>
		</timeSeries>
		<timeSeries>
			<timeSeriesId>input2</timeSeriesId>
		</timeSeries>
		<checkRelativePeriod unit="hour" start="-12" end="0"/>
		<minNumberOfValues>18</minNumberOfValues>
		<logLevel>INFO</logLevel>
		<logEventCode>TimeSeries.Check</logEventCode>
		<logMessage>Not enough non-missing values available for time series %header%</logMessage>
	</minNonMissingValuesCheck>
	<minNumberOfValuesCheck id="MinNumberOfValuesCheck">
		<timeSeries>
			<timeSeriesSet>
				<moduleInstanceId>CheckImportedData</moduleInstanceId>
				<valueType>scalar</valueType>
				<parameterId>P.obs</parameterId>
				<locationSetId>locationSet1</locationSetId>
				<timeSeriesType>external historical</timeSeriesType>
				<timeStep unit="nonequidistant"/>
				<!-- any relativeViewPeriod here will always be overruled by checkRelativePeriod in each check -->
				<readWriteMode>read only</readWriteMode>
			</timeSeriesSet>
		</timeSeries>
		<checkRelativePeriod unit="hour" start="-12" end="0"/>
		<minNumberOfValues>24</minNumberOfValues>
		<logLevel>DEBUG</logLevel>
		<logEventCode>TimeSeries.Check</logEventCode>
		<logMessage>Not enough values available for time series %header%</logMessage>
	</minNumberOfValuesCheck>
</timeSeriesAvailabilityChecker>