This import is not yet available in DELFT-FEWS version 2018.02 as it has been developed specifically for the FFFS project.
Imports pollution incident ans suspension of monitoring data from a JSON web service provided by the UK Environmental Agency (EA) to meet UK reporting obligations under the EU Bathing Water Directive.
The documentation for the BWQ web service API is provided here: https://environment.data.gov.uk/bwq/doc/api-reference-v0.6.html
The data items of interest here are only the "pollution incidents" and "suspension of monitoring" described in this document.
The parser for this data has the ID "UkEaBwq" and can parse both the data for "pollution incidents" and "suspension of monitoring" as the JSON data retrieved for both data items has a similar structure, the names of data fields have been made configurable as properties, shown in the configuration examples below.
<?xml version="1.0" encoding="UTF-8"?> <timeSeriesImportRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/timeSeriesImportRun.xsd"> <import> <general> <importType>UkEaBwq</importType> <serverUrl>https://environment.data.gov.uk/doc/bathing-water-quality/pollution-incident</serverUrl> <relativeViewPeriod unit="day" start="-10" end="0"/> <importTimeZone> <timeZoneOffset>+00:00</timeZoneOffset> </importTimeZone> </general> <properties> <string key="idProperty" value="incidentNotation"/> <string key="valueProperty" value="incidentType.notation"/> <string key="locationProperty" value="samplingPoint.notation"/> <string key="startDateTimeProperty" value="startOfIncident"/> <string key="recordDateTimeProperty" value="som_recordDateTime"/> <string key="endDateTimeProperty" value="endOfIncident"/> <string key="importProperties" value="incidentNotation, som_recordDateTime, startOfIncident, expectedEndOfIncident, endOfIncident, bathingWater.notation, samplingPoint.notation, nirsRef"/> <bool key="importHistory" value="true"/> <int key="pageSize" value="10"/> </properties> <timeSeriesSet> <moduleInstanceId>ImportBWQIncidents</moduleInstanceId> <valueType>scalar</valueType> <parameterId>Incident.type</parameterId> <locationSetId>BWQFS.sites</locationSetId> <timeSeriesType>external historical</timeSeriesType> <timeStep unit="nonequidistant"/> <readWriteMode>add originals</readWriteMode> </timeSeriesSet> </import> </timeSeriesImportRun>
Note: the properties reflect the JSON field names to be used for incident ID, the time series value and location to be written to the FEWS database, followed by the fields that indicate the start and end of an event and the time it was recorded. Additional fields that are to be imported to the FEWS database as properties recorded with each time series value are given in the "importProperties" field.
The "importHistory" property when true will cause the importer to not just import the "most recent" record for each pollution incident, in addition to this a separate query will be posted for each incident to retrieve the complete change history for each pollution incident. Typically the first record for an incident gives the begin date/time, pollution type and estimated end date/time for the incident, the last (most recent) record will add the exact end date/time of the incident and optionally, additional records in between may be present where the content of the previous record is corrected for some specific reason.
The page size property controls how many incidents to retrieve at once, the parser will loop over the pages until all records within the specified viewperiod have been retrieved, a larger page size means a lower number of http requests but may retrieve more data than required, so if the parser is scheduled to be run on a daily schedule a page size as small as "1" may be more practical.
For each incident, at the start date/time a parameter value is recorded that reflects the incident type (see table "Pollution Incident Types" in the API document) and at the end of the incident a zero value is recorded to indicate the end of the pollution incident.
<?xml version="1.0" encoding="UTF-8"?> <timeSeriesImportRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/timeSeriesImportRun.xsd"> <import> <general> <importType>UkEaBwq</importType> <serverUrl>https://environment.data.gov.uk/doc/bathing-water-quality/suspension</serverUrl> <relativeViewPeriod unit="day" start="-10" end="0"/> <importTimeZone> <timeZoneOffset>+00:00</timeZoneOffset> </importTimeZone> </general> <properties> <string key="locationProperty" value="samplingPoint"/> <string key="startDateTimeProperty" value="startOfSuspension"/> <string key="recordDateTimeProperty" value="som_recordDateTime"/> <string key="endDateTimeProperty" value="endOfSuspension"/> <string key="importProperties" value="som_notation, som_recordDateTime, startOfSuspension, expectedEndOfSuspension, endOfSuspension, bathingWater, samplingPoint, nirsRef, pollutionIncident"/> <int key="pageSize" value="10"/> </properties> <timeSeriesSet> <moduleInstanceId>ImportBWQSuspensions</moduleInstanceId> <valueType>scalar</valueType> <parameterId>Suspension.monitoring</parameterId> <locationSetId>BWQFS.sites</locationSetId> <timeSeriesType>external historical</timeSeriesType> <timeStep unit="nonequidistant"/> <readWriteMode>add originals</readWriteMode> </timeSeriesSet> </import> </timeSeriesImportRun>
The most important differences with the configuration for pollution incidents are the url and field names, additionally the option "importHistory" is not available for suspension of monitoring data, also there is no incident type or similar field to record as time series value in the FEWS database so in this case the "valueProperty" definition is not included, as time series value, "1" is recorded at the start of a suspension of monitoring event and "0" is recorded when the suspension of monitoring event ends.
