Page tree
Skip to end of metadata
Go to start of metadata

Overview

The WIWB Import connects to the Hydronet WIWB API and has support for downloading observations and forecasts for both scalar and grid data. WIWB has a concept of DataSources which have to be configured as a property of the import. Using the meta data api of  the WIWB API the type of data is determined and processed accordingly. The import feature is available since December 2018, from Delft-FEWS version 2016.02 onwards.

 

More about WIWB

WIWB is the Dutch national database for storage and dissemination of weather related data, specifically for the Dutch water sector. It has been developed in 2017 by HydroLogic, HKV and HydroConsult for Het Waterschapshuis, a national body that represents all Dutch water boards. The WIWB database is based on the HydroNET platform (www.hydronet.com) and offers a REST API for retrieving data and metadata. WIWB contains weather observations and forecasts, based on weather stations, meteorological radar data and numerical weather models.

WIWB is intended for professional use only by Dutch water boards. It is not intended for public or commercial use. Only officials from Dutch water boards can request access to the WIWB API. More information on specifications and use of the WIWB API can be requested from the WWIB helpdesk (helpdesk@hydrologic.com).

Access to WIWB API

Access to the WIWIB API is limited to the regional water authorities in the Netherlands and can be requested from the Waterschapshuis Water (HWH) by sending an e-mail to mr. Willem Aberson. Please list the following items in your e-mail

  • Organisation name
  • Contact person and contact details (e-mail and telephone number).
  • List with IP addresses that require access to the API.  
  • Short explanation of intended use:
    • Systems to be connected to the API
    • Required data sources
    • Expected data request per time interval.

HWH will forward the required information to the WIWB back-office operated by HydroLogic. Your account will be configured within 5 days after the request was received by the WIWB back office.

More info about WIWB at HWH can be found here.

HTTPS and client.truststore

Access to the WIWIB API is done using a secure HTTPS connection. If a FEWS configuration is used with a client.truststore it may be required to import the latest public certificate of the WIWB service into the existing client.truststore. For more information on configuring secure HTTPS connection with a client.truststore, see How to configure secure https connection to Matroos

A detailed presentation highlighting various WIWB configuration options in Delft-FEWS can be found in the attached document.

Note that the various improved configuration options related to the WIWB import in Delft FEWS (as described in the detailed presentation) are made available for Delft-FEWS versions 2016.01 and higher, with a patch from later then December 2018.

A template configuration for WIWB imports are also available in the attached ZIP file

Import Configuration

To configure the WIWB import the dataSource property has to be specified. Se the list of supported dataSource values in the next section.

For ensembles the ensemble id can be configured using the ensembleId property. In case no property is configured, the default value "EPS" will be used.

DataSource values

The following data sources are currently provided through WIWB. New data sources might become available in future.

More information on the data sources can be obtained form the links provided in the table. All times are in UTC. The parameter code - between [brackets] - is the parameterID which must be used in the FEWS mapping configuration as 'parameter external'. 

An overview of the grid definitions of the complete grid extent are defined in the attached WIWB grid definitions file.

An extended table, offering more information about the WIWB parameters can be found the attached overview Excel file (version November 2018). 

 

Datasource nameDatasource code (ID)TypeOriginal IntervalAvailable Parameter(s) [parameter code]Remarks
HarmonieKnmi.HarmonieModel Grid, forecast1 hour

Air Pressure [AirPressure]
Global Radiation Flux [GlobalRadFlux]
High Cloud Cover [HighCloudCover]
Large Scale Precipitation [LargeScaleP]
Latent Heat Flux [LatHeatFlux]
Low Cloud Cover [LowCloudCover]
Medium Cloud Cover [MediumCloudCover]
Net Longwave Radiation Flux [NetLwRadFlux]
Net Shortwave Radiation Flux [NetSwRadFlux]
Relative Humidity [RH]
Sensible Heat Flux [SensHeatFlux]
Snow Depth [SnowDepth]
Soil temperature level 2 [Temperature.Soil.Level2]
Temperature [TMP]
Total Cloud Cover [TCDC]
Total Precipitation [APCP]
U-Component of Wind [UGRD:10m]
V-Component of Wind [VGRD:10m]
U-Component of max wind gust [U.Max.Wind.Gust]
V-Component of max wind gust [V.Max.Wind.Gust]

 
HiRLAMKnmi.HirlamModel Grid, forecast1 hour

Air Pressure [AirPressure]
Dewpoint Temperature [DPT]
Global Radiation Flux [GlobalRadFlux]
High Cloud Cover [HighCloudCover]
Large Scale Precipitation [LargeScaleP]
Latent Heat Flux [LatHeatFlux]
Low Cloud Cover [LowCloudCover]
Medium Cloud Cover [MediumCloudCover]
Net Longwave Radiation Flux [NetLwRadFlux]
Net Shortwave Radiation Flux [NetSwRadFlux]
Sensible Heat Flux [SensHeatFlux]
Snow Depth [SnowDepth]
Temperature [TMP]
Total Cloud Cover [TCDC]
Total Precipitation [APCP]
U-Component of Wind [UGRD:10m]
V-Component of Wind [VGRD:10m]
Water Equivalent of Accumulated Snow Depth [WaterEqSnowDepth]

 
KNMI AWS 10 MinutesKnmi.AwsTenMinutesScalar, observed10 minute

Air Pressure [AirPressure]
Dewpoint Temperature [DPT]
Horizontal Visibility [HorizontalVisibility]
Maximum Wind Speed [MaximumWindSpeed]
Precipitation [P]
Precipitation Duration [Precipitation Duration]
Relative Humidity [RH]
Solar radiation [Radiation.Solar]
Temperature [TMP]
Total Cloud Cover [TCDC]
Wind Direction [WindDirection]
Wind Speed [WindSpeed]

 
KNMI Evaporation (From Meteobase)Knmi.FromMeteobase.EvaporationScalar, observed1 dayEvaporation [Evaporation] 
KNMI Reference EvaporationKnmi.EvaporationScalar, observed1 dayEvaporation [Evaporation] 
KNMI Iris Stations

Knmi.IrisValidated
Knmi.IrisUnvalidated

Scalar, observed1 day

Precipitation [P]
Snow Depth [SnowDepth]

 
KNMI Naval Reports

Knmi.Naval.Forecasts
Knmi.Naval.Warnings

Report, observed1 dayNaval reports for warnings and forecasts [Naval.Reports] 
KNMI Radar Corrected B (Near-Real time)Knmi.Radar.CorrectedBGrid, radar5 minutePrecipitation [P]Grid definition required in grids.xml.
See below for the configuration example.
KNMI Radar Corrected C2 (After)Knmi.Radar.CorrectedC2Grid, radar5 minutePrecipitation [P]Grid definition required in grids.xml.
See below for the configuration example.
KNMI Radar Corrected D2 (Re-analysis)Knmi.Radar.CorrectedD2Grid, radar5 minutePrecipitation [P]Grid definition required in grids.xml.
See below for the configuration example.
KNMI Radar Uncorrected (Real time)Knmi.Radar.UncorrectedGrid, radar5 minutePrecipitation [P]Grid definition required in grids.xml.
See below for the configuration example.
KNMI Regional EPSKnmi.RegionalEpsEnsemble Scalar, forecast6 hour

Dewpoint Temperature [DPT]
Maximum Temperature [TMAX]
Minimum Temperature [TMIN]
Precipitation [P]
Snow Depth [SnowDepth]
Temperature [TMP]
Total Cloud Cover [TCDC]
U-Component of Wind [UGRD:10m]
V-Component of Wind [VGRD:10m]
Wind Direction [WindDirection]
Wind Speed [WindSpeed]

Note that the ensemble members 1-50 are
the real ensemble,
while member 51 and 52
are the control and deterministic runs.
So it is advised to store the ensemble in a
temporary ensembleId and to
postprocess
the members 1-50 to the real ensemble.
KNMI SynopsKnmi.SynopsScalar, observed1 hour

Air Pressure [AirPressure]
Dewpoint Temperature [DPT]
Horizontal Visibility [HorizontalVisibility]
Maximum Temperature [TMAX]
Maximum Wind Speed [MaximumWindSpeed]
Minimum Temperature [TMIN]
Precipitation [P]
Precipitation Duration [Precipitation Duration]
Relative Humidity [RH]
Solar radiation [Radiation.Solar]
Temperature [TMP]
Total Cloud Cover [TCDC]
Total Precipitation [APCP]
Wind Direction [WindDirection]
Wind Speed [WindSpeed]

 
KNMI Synops (From Meteobase)Knmi.FromMeteobase.SynopsScalar, observed1 hourPrecipitation [P] 
KNMI Waqua TSKnmi.WaquaTsModel Scalar, forecast10 minute

Astronomical Tide [AstronomicalTide]
Observations from WAQC_ODC [WaquaTs.Obs]
Total Water Level [TotalWaterLevel]
Water Surge [WaterSurge]

 
KNMI Weather WarningsKnmi.WarningsWarning, observed1 hour

Heat warning [Warning.Heat]
Rain warning [Warning.Rain]
Slipperiness and snow warning [Warning.Slipperiness.Snow]
Storm warning [Warning.Storm]
Vision warning [Warning.Vision]
Whirlwind and waterspout warning [Warning.Whirlwind.Waterspout]
Wind warning [Warning.Wind]

 
KNMI Water Setup EPSKnmi.WaterSetupEpsEnsemble Scalar, forecast6 hourWater Surge [WaterSurge] 
Meteobase Makkink EvaporationMeteobase.Evaporation.MakkinkGrid, observed1 dayEvaporation [Evaporation] 
Meteobase Pennman-Monteith EvaporationMeteobase.Evaporation.PennmanMonteithGrid, observed1 dayEvaporation [Evaporation] 
Meteobase PrecipitationMeteobase.PrecipitationGrid, observed1 dayPrecipitation [P] 

 

Aggregated products

All data sources as in the table above have their original interval. If you want to import an aggregated product, the WIWB API supports that. You can aggregate every product with a multiplier of its original interval. The <timestep> configured in the Delft-FEWS import moduleInstance of that datasource will be interpreted whether you want to import the original or an aggregated product. E.g. the KNMI Radar products have an original interval of 5 minutes, so aggregation options are 10 minutes (2 x 5 minutes), 15 minutes (3 x 5 minutes) etc. As mentioned, the <timestep> definition in Delft-FEWS will be leading. Disaggregating is not supported.

The WIWB API has the default option that aggregation to larger timesteps is valid when at least one timestep is available. For example: a 5 minute timestep is aggregated to hourlies when at least one 5 minute timestep is available. For Delft-FEWS applications this is usually not the best solution. Therefore a so-called "minimumAggregationAvailability" option has been added. This value (of type double) has a valid range from 0 to 1 and indicates to the WIWB API the fraction of how much original timesteps should be available in the new timestep. Usually the value will be set to 1, as also can be seen from the below config example "aggregated radar products".

Missing values

If the WIWB specifies the missing value for a datasource, this will be used by FEWS. The missing value can also be specified in the import configuration as well. The default value for missing value is -9999 for the WIWB API.

Grid definitions

The WIWB API can provide the grid definitions of the models. The WIWB Import will log the Grid Definition if debug is enabled during the import.

It is possible to download a subextent of a grid by defining properties in the Import file. The methodology is described in detail in this presentation. Note that defining a subgrid for the KNMI radar product is a bit more complicated due to the projection of the frid. The methodology to get the correct configuration values in the import file and the grids.xml can be found in this document.

View period

Where appropriate a viewPeriod can be configured. The view period can be no longer than one year, since the WIWP api doesn't support periods longer than that.

For model forecasts only the latest forecast will be downloaded.

Examples

This example is configured to import scalar observations from the Knmi.AwsTenMinutes dataSource:

Example AwsTenMinutes
<import>
	<general>
		<importType>WIWB</importType>
		<serverUrl>https://wiwb.hydronet.com/api</serverUrl>
		<user>$WIWB_USER$</user>
		<password>$WIWB_PW$</password>
		<relativeViewPeriod unit="hour" start="-120" end="0" startOverrulable="true"/>
		<idMapId>IdImportWIWB_obs</idMapId>
		<dataFeedId>WIWB (Knmi.AwsTenMinutes)</dataFeedId>
	</general>
	<properties>
		<string key="dataSource" value="Knmi.AwsTenMinutes"/>
	</properties>
	<timeSeriesSet>
		<moduleInstanceId>ImportKNMI</moduleInstanceId>
		<valueType>scalar</valueType>
		<parameterId>P.meting</parameterId>
		<locationSetId>KNMI-EVP</locationSetId>
		<timeSeriesType>external historical</timeSeriesType>
		<timeStep unit="minute" multiplier="10"/>
		<readWriteMode>add originals</readWriteMode>
	</timeSeriesSet>
</import>


The following example is configured for importing grids from the Knmi.Radar.Uncorrected dataSource:

<import>
	<general>
		<importType>WIWB</importType>
		<serverUrl>https://wiwb.hydronet.com/api</serverUrl>
		<user>$WIWB_USER$</user>
		<password>$WIWB_PW$</password>
		<relativeViewPeriod unit="hour" start="-12" end="-10" startOverrulable="false"/>
		<idMapId>IdImportWIWB_obs</idMapId>
		<dataFeedId>WIWB (Knmi.Radar.Uncorrected)</dataFeedId>
	</general>
	<properties>
		<string key="dataSource" value="Knmi.Radar.Uncorrected"/>
	</properties>
	<timeSeriesSet>
		<moduleInstanceId>ImportKNMI</moduleInstanceId>
		<valueType>grid</valueType>
		<parameterId>P.radar</parameterId>
		<qualifierId>uncorrected</qualifierId>
		<locationId>Hydronet_radar</locationId>
		<timeSeriesType>external historical</timeSeriesType>
		<timeStep unit="minute" multiplier="5"/>
		<readWriteMode>add originals</readWriteMode>
		<synchLevel>6</synchLevel>
	</timeSeriesSet>
</import>

Example of idMapping to some of the WIWB parameters in the IdImportWIWB_obs.xml file:

<idMap version="1.1" xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/idMap.xsd">
	<parameter external="Evaporation" internal="E.meting"/>
	<parameter external="P" internal="P.meting"/>
	<parameter external="P" internal="P.radar" internalQualifier="realtime"/>
	<parameter external="WindDirection" internal="Wind.meting.richting"/>
	<parameter external="WindSpeed" internal="Wind.meting.snelheid"/>
	<parameter external="TMP" internal="T.meting"/>
	<location external="Hydronet_radar" internal="Hydronet_radar"/>
	<locationIdPattern internalLocationSet="KNMI-EVP" internalLocationPattern="KNMI_*" externalLocationPattern="06*"/>
	<enableOneToOneMapping/>
</idMap>

Example of the grids.xml file for the KNMI Radar grid products

<?xml version="1.0" encoding="UTF-8"?>
<grids xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/grids.xsd">
	<regular locationId="knmi_radar_1km">
		<rows>765</rows>
		<columns>700</columns>
		<polarStereographic>
			<originLatitude>90</originLatitude>
			<originLongitude>0</originLongitude>
			<trueScalingLatitude>60</trueScalingLatitude>
			<equatorRadius>6378137</equatorRadius>
			<poleRadius>6356752</poleRadius>
		</polarStereographic>
		<firstCellCenter>
			<x>500</x>
			<y>-3650500</y>
			<z>0</z>
		</firstCellCenter>
		<xCellSize>1000</xCellSize>
		<yCellSize>1000</yCellSize>
	</regular>
</grids>

Example of a aggregated radar product

<import>
	<general>
		<importType>WIWB</importType>
		<serverUrl>https://wiwb.hydronet.com/api</serverUrl>
		<connectionTimeOutMillis>900000</connectionTimeOutMillis>
		<user>$WIWB_USER$</user>
		<password>$WIWB_PW$</password>
		<relativeViewPeriod unit="hour" start="-24" end="0" startOverrulable="true"/>
		<idMapId>IdImportWIWB_radar</idMapId>
		<dataFeedId>WIWB (Knmi.Radar.CorrectedC2, hour)</dataFeedId>
	</general>
	<properties>
		<string key="dataSource" value="Knmi.Radar.CorrectedC2"/>
		<double key="minimumAggregationAvailability" value="1.00"/>
    </properties>
	<timeSeriesSet>
		<moduleInstanceId>ImportRadar</moduleInstanceId>
		<valueType>grid</valueType>
		<parameterId>P.radar</parameterId>
		<qualifierId>CorrectedC2</qualifierId>
		<locationId>Hydronet_radar</locationId>
		<timeSeriesType>external historical</timeSeriesType>
		<timeStep unit="hour" multiplier="1"/>
		<readWriteMode>add originals</readWriteMode>
		<synchLevel>6</synchLevel>
	</timeSeriesSet>
</import>
  • No labels