Digitale Delta API 3.0

Configuration example

Data provided via services providing scalar observation data according to the digitale delta API 3.0 can be importen into FEWS via the DigitalDelta3.0 import type.

It should support authentication via OAUTH 2, with a token url and client id and client secret.

The client id can be configured as user and the client secret can be configured as password.

The token url must be configured as a property.

An example configuration can be seen below.

DD API 3.0 import

<?xml version="1.0" encoding="UTF-8"?>
<timeSeriesImportRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews ../../../../Schemas/timeSeriesImportRun.xsd">
	<import>
		<general>
			<importTypeStandard>DigitalDelta3.0</importTypeStandard>
			<serverUrl>https://dd.api3.nl/observation</serverUrl>
			<connectionTimeOutMillis>300000</connectionTimeOutMillis>
			<user>$CLIENT_ID$</user>
			<password>$CLIENT_SECRET$</password>
			<relativeViewPeriod unit="hour" start="-3" end="0"/>
		</general>
		<properties>
			<string key="tokenUrl" value="https://dd.api3.nl/oauth2/token"/>
			<!--<string key="audience" value="https://dd.api3.nl"/>-->
		</properties>
 		<timeSeriesSet>
 			<moduleInstanceId>ImportDD</moduleInstanceId>
 			<valueType>scalar</valueType>
 			<parameterId>MyPar</parameterId>
 			<locationSetId>MyLocSet</locationSetId>
 			<timeSeriesType>external historical</timeSeriesType>
 			<timeStep unit="nonequidistant"/>
 			<readWriteMode>add originals</readWriteMode>
 			<synchLevel>1</synchLevel>
  		</timeSeriesSet>
    </import>
</timeSeriesImportRun>

Service requirements

This section is a guideline for what a DD API 3.0 service should conform to in order to be able to let Delft FEWS clients connect to it.

It should support authentication via OAUTH 2, with a token url and client id and client secret.

Delft-FEWS will construct request url's as follows:

<serverUrl>?$filter=Parameter/ObservationType eq '<parameterId>' and Foi/Code eq '<locationId>' and PhenomenonTime/BeginPosition ge <startTime> and PhenomenonTime/EndPosition le <endTime>

Where

<serverUrl> will be replaced by the server url configured in the FEWS import configuration
<parameterId> will be replaced by external parameter id, this will always be supplied
<locationId> will be replaced by external location id, this will always be supplied
<startTime> will be replaced by the start of the requested period according to the format: yyyy-MM-ddTHH:mm:ssZ this will always be supplied
<endTime> will be replaced by the end of the requested period according to the format: yyyy-MM-ddTHH:mm:ssZ this will always be supplied

There will not be a part in the url request to specify the desired time step of the data. In case the data returned will not be according to a consistent time step, a non-equidistant time step should be used within FEWS from which later on equidistant time series can be constructed if desired.

For example:

https://ddapi3.water.nl/observation?$filter=Parameter/ObservationType eq 'WNS15028' and Foi/Code eq 'AMSRK_0001' and PhenomenonTime/BeginPosition ge 2001-12-07T00:00:00Z and PhenomenonTime/EndPosition le 2002-01-02T00:00:00Z

or with escaped characters:

https://ddapi3.water.nl/observation?$filter=Parameter%2FObservationType%20eq%20%27WNS15028%27%20and%20Foi%2FCode%20eq%20%27AMSRK_0001%27%20and%20PhenomenonTime%2FBeginPosition%20ge%202001-12-07T00%3A00%3A00Z%20and%20PhenomenonTime%2FEndPosition%20le%202002-01-02T00%3A00%3A00Z

For service providers it is import to know they will always get Parameter/ObservationType eq '<parameterId>' defined. The <parameterId> will always be a WNS code, for more information on this see the aquo standard. It will be the task of administrators of the FEWS system to make the external parameter id match the WNS code.

In the response the ObservationType should come back in the parameter block and also have the WNS code as value:

"Parameter": {
"Quantity": "MONSDTE",
"MeasurementDevice": "Druksensor",
"ObservationType": "WNS15028",
"Capacity": "WATKLM",
"Compartment": "GW"
},

The FEWS external location id will be AMSRK_0001, which must match the value of Code within the Foi block:

"Foi": {
"Id": "GWLICHAAM_ZAND_MS_01",
"Code": "AMSRK_0001",

The times will be 2013-01-01 to 2013-01-06, taken from:

"Result": {
"id" : "a4",
"type" : "Coverage",
"domain" : {
"type" : "Domain",
"domainType" : "PointSeries",
"axes": {
"x" : { "values": [-10.1] },
"y" : { "values": [ -40.2] },
"t" : { "values": ["2013-01-01","2013-01-02","2013-01-03",
"2013-01-04","2013-01-05","2013-01-06"] }
},

The values will be taken from the block under ranges that matches the ObservationType :

"ranges" : {
"WNS15028" : {
"type" : "NdArray",
"dataType": "float",
"axisNames": ["t"],
"shape": [6],
"values" : [ 43.9599, 43.9599, 43.9640, 43.9640, 43.9679, 43.9879 ]
}

The quality flags will be taken from the block called quality :

"quality": {
"type": "NdArray",
"dataType": "string",
"values": [
"VALID", "VALID", "VALID", "VALID", "VALID", "VALID",
"VALID", "VALID", "VALID", "VALID", "VALID", "VALID",
"VALID", "VALID", "VALID", "VALID", "VALID", "VALID",
"VALID", "VALID", "VALID", "VALID", "VALID", "VALID"
]

It must be taken into account that according to the JSON rules key-value pairs within the same object are allowed to be in any order.

So in the example below "resultTime": "2024-03-21T00:05:00Z", is also allowed to be before "phenomenonTime": {....},

Full example:

{
  "@odata.context": "https://ddapi.water.nl/v3/odata/$metadata#observations",
  "value": [
    {
      "Id": "a4f6af36-6300-49b4-a579-b1aba70e3288",
      "Type": "CoverageJSON",
      "ResultTime": "2013-01-01T12:30:00Z",
      "PhenomenonTime": {
        "BeginPosition": "2013-01-01T12:30:00Z",
        "EndPosition": "2013-01-06T12:30:00Z"
      },
      "ValidTime": {
        "BeginPosition": "2013-01-01T12:30:00Z"
      },
      "Parameter": {
        "Quantity": "WATHTE",
        "MeasurementDevice": "Druksensor",
        "ObservationType": "WNS9040",
        "Capacity": "NAP",
        "Compartment": "GW"
      },
      "Metadata": {
        "WellProperties": {
          "SurfaceLevel": 2.45,
          "WellDepth": 15.0,
          "FilterTop": -12.0,
          "FilterBottom": -13.0,
          "TubeType": "HDPE",
          "TubeDiameter": 0.032
        }
      },
      "Foi": {
        "Id": "GWLICHAAM_ZAND_MS_01",
        "Code": "AMSRK_0001",
        "Description": "DIEMZDK",
        "Geography@odata.type": "#GeographyPoint",
        "Geography": {
          "Type": "Point",
          "Coordinates": [
            5.00166,
            52.34331
          ]
        }
      },
      "Result": {
        "id": "a4",
        "type": "Coverage",
        "domain": {
          "type": "Domain",
          "domainType": "PointSeries",
          "axes": {
            "x": {
              "values": [
                -10.1
              ]
            },
            "y": {
              "values": [
                -40.2
              ]
            },
            "t": {
              "values": [
                "2013-01-01",
                "2013-01-02",
                "2013-01-03",
                "2013-01-04",
                "2013-01-05",
                "2013-01-06"
              ]
            }
          },
          "referencing": [
            {
              "coordinates": [
                "x",
                "y"
              ],
              "system": {
                "type": "GeographicCRS",
                "id": "http://www.opengis.net/def/crs/OGC/1.3/CRS84"
              }
            },
            {
              "coordinates": [
                "t"
              ],
              "system": {
                "type": "TemporalRS",
                "calendar": "Gregorian"
              }
            }
          ]
        },
        "parameters": {
          "WNS9040": {
            "type": "Parameter",
            "description": {
              "en": "[] [WATHTE] [] [m] [NAP] [GW]"
            },
            "unit": {
              "symbol": "m"
            }
          },
          "quality": {
            "type": "Parameter",
            "description": {
              "en": "Judgement of quality of datapoint"
            },
            "observedProperty": {
              "id": "oajsfgiujhsagffpoiuahgfpih",
              "label": {
                "en": "Judgement of quality of datapoint"
              }
            }
          }
        },
        "ranges": {
          "WNS9040": {
            "type": "NdArray",
            "dataType": "float",
            "axisNames": [
              "t"
            ],
            "shape": [
              6
            ],
            "values": [
              43.9599,
              43.9599,
              43.9640,
              43.9640,
              43.9679,
              43.9879
            ]
          },
          "quality": {
            "type": "NdArray",
            "dataType": "string",
            "axisNames": [
              "t"
            ],
            "shape": [
              6
            ],
            "values": [
              "VALID",
              "INVALID",
              "VALID",
              "UNKNOWN",
              "UNKNOWN",
              "UNKNOWN"
            ]
          }
        }
      }
    }
  ],
  "@odata.nextLink": "https://ddapi.water.nl/v3/odata/observations"
}

Qualifiers and addition Parameter/XXX filtering

Since build patches with number 609110736 and higher, (external) qualifiers are also used for extra url request filtering options.

To support this, FEWS will use qualifiers to extend the URL request with extra Parameter/XXX filters. This way any other Parameter/XXX filter will be supported as well.

For each (external) qualifier id of the FEWS time series a prefix will define the exact Parameter/XXX and the rest of the qualifier defines the value.

The (external) qualifier id must match the next format: <prefix>_<value>

For each (external) qualfier id the url request is extended with

and Parameter/<prefix> eq '<value>'

Example water quantity

A FEWS time series with external FEWS-parameter “WNS9040” and only one external FEWS-qualifer: “MeasurementDevice_Handmeting” will lead to

Parameter/ObservationType eq ‘WNS9040’ and Parameter/MeasurementDevice eq ‘Handmeting’

While a FEWS time series with external FEWS-parameter “WNS9040” and only one external FEWS-qualifer: “MeasurementDevice_Peilbuis” will lead to

Parameter/ObservationType eq ‘WNS9040’ and Parameter/MeasurementDevice eq ‘Peilbuis’

It can well be that for this use case within FEWS two internal parameters are being used without an internal qualifier. This can be solved in id mapping as follows:

Example water quality

Cases with mutliple qualifiers are supported as well:

For example, a FEWS time series with external FEWS-parameter “WNS12345” and external FEWS-qualifers: “Gender_Male” & “LengthClass_Small” will lead to the next part of the request:

Parameter/ObservationType eq ‘WNS12345’ and Parameter/Gender eq ‘Male’ and Parameter/LengthClass eq ‘Small’

This can be solved via an externalQualfierFunction in the id mapping:

Where in Qualfiers.xml the externalQualifierId can be defined as follows:

    <file>LenghtClass.csv</file>
    <id>LC_%fewsid%</id>
    <name>%name%</name>
    <group>LengthClass</group>
    <attribute id="ExternalQualifierId">
       <text>LengthClass_%Codes%</text>
    </attribute>

</csvFile>

<csvFile>
    <file>Gender.csv</file>
    <id>GD_%fewsid%</id>
    <name>%name%</name>
    <group>Gender</group>
    <attribute id="ExternalQualifierId">
       <text>Gender_%Codes%</text>
    </attribute>

</csvFile>

Page tree

Digitale Delta API 3.0

Configuration example

Service requirements

Example water quantity