Page History

day for the validity of the location attributesdbServerType

scrollbar

Excerpt

hidden	true

Functionality to define Locations, LocationSets, IdMaps, DisplayGroups, ThresholdValueSets and ValidationRuleSets from a DBF file

Function:	_Functionality to define locations and to generate locationSets from a CSV file /Shape-DBF file / DB Table
Where to Use?	Locations, LocationSets, IdMaps, DisplayGroups, ThresholdValueSets and ValidationRuleSets
Why to Use?	To have only one file or a set of files where all region specific information is stored.
Description:	Based on the CSV, DBF-Shape file or DB table you can easily manage the configuration
Available since:	DelftFEWS200803

Configuration

...

When there is no ID defined in the MapLayerFile (either csv, shape, dbf etc), the location is ignored. For example in the config csv example below, if on row 20 in the csv in column ID no value is defined, the location defined on row 20 will be ignored as a location for FEWS.

locationSets from CSV file

...

Info
Since 2021.02 you can configure ignoreIdsMissingInLocationSet=true when you expect the csv/dbf/table contains more locations than the location set. Don't use this function to attach the same csv file to multiple locations sets. This will increase the startup time of FEWS.

locationSets from esriShapeFilelocationSets from esriShapeFile

...

The most useful way is first to read all locations from the Shape/DBF file into one locationSet, where all attributes are assigned.
See for example:

Code Block

	xml
	xml

<locationSet id="gegevensdump" editable="false">
    <esriShapeFile>
      <file>gegevens</file>
      <geoDatum>Rijks Driehoekstelsel</geoDatum>
      <id>%ID%</id>
      <name>%NAAM%</name>
      <description>%TYPE%</description>
      <iconName>%ICONFILE%</iconName>
      <toolTip><![CDATA[<html></html>]]></toolTip>
      <parentLocationId>%PARENT%</parentLocationId>
      <timeZoneOffset>+05:00</timeZoneOffset>
      <dateTimePattern>yyyyMMdd HH:mm:ss</dateTimePattern>
      <visibilityStartTime>%START%</visibilityStartTime>
      <visibilityEndTime>%EIND%</visibilityEndTime>
      <x>%X%</x>
      <y>%Y%</y>
      <z>0</z>
      <attribute id="PARENT">
        <text>%PARENT%</text>
      </attribute>
      <attribute id="TYPE">
        <text>%TYPE%</text>
      </attribute>
      <attribute id="CITECTLOC">
        <text>%CITECTLOC%</text>
      </attribute>
      <attribute id="IDMAP_Q">
        <text>%DEBIET%</text>
      </attribute>
      <attribute id="HMIN_Q">
        <number>%HMIN_Q%</number>
      </attribute>
      <attribute id="HMAX_Q">
        <number>%HMAX_Q%</number>
      </attribute>
      <attribute id="ROC_Q">
        <number>%ROC_Q%</number>
      </attribute>
    </esriShapeFile>
  </locationSet>

locationSets from

...

geoJSON file

Code Block

	xml
	xml

<locationSet id="Provinces">
	<geoJsonFile>
		<file>provinces.geojson</file>
		<shapeType>polygon</shapeType>
		<id>%name%</id>
		<name>%name%</name>
		<attribute id="level">
			<number>%level%</number>
		</attribute>
	</geoJsonFile>
</locationSet>

locationSets from database table

...

It is also possible to read locations directly from a database table. The contents of the database table It is also possible to read locations directly from a database table. The contents of the database table are on the fly read and converted to a DBZ file. This DBZ file will be used by FEWS. This is for backup purpose in case the database is not available any more, like in stand-alone test environments.

Code Block

	xml
	xml

<locationSet id="grondwater">
  <table>
	<databaseServer>
		<url>jdbc:oracle:thin:@oraclex02@dummy_hostname:1521:FEWS<dummy_databasename</url>
		<user>fews<<user>dummy_username</user>
		<encryptedPassword>***<<encryptedPassword>dummy_password_encrypted</encryptedPassword>
	</databaseServer>
    <name>PEILBUIZEN</name>
    <geoDatum>Rijks Driehoekstelsel</geoDatum>
    <id>ult_%FEWS_ID%</id>
    <name>%NAAM%</name>
    <description>Gebiedsnaam: %GEBIEDSNAA% </description>
    <x>%X_COORDINA%</x>
    <y>%Y_COORDINA%</y>
    <attribute id="DOMMEL_ID">
      <text>%MEETPUNTCO%</text>
    </attribute>
    <attribute id="TMX_ID">
      <text>%TMX%</text>
    </attribute>
    <attribute id="HARD_MAX">
      <number>%HARD_MAX%</number>
    </attribute>
    <attribute id="HARD_MIN">
      <number>%HARD_MIN%</number>
    </attribute>
  </table>
</locationSet>

In this example the above database table PEILBUIZEN is read from the database and completely converted to a zipped DBase file, named PEILBUIZEN.dbz. This file is used by FEWS to read all the required data.
To use a Firebird/Derby database file you should use the element <databaseFile> instead of <dbServerName>. Other connection strings:

jdbc:postgresql://hostdummy_hostname:port/databasedummy_databasename
jdbc:oracle:thin:@//host@dummy_hostname:port/servicedummy_nameservicename
jdbc:oracle:thin:@host@dummy_hostname:port:SID
jdbc:oracle:thin:@TNSName
jdbc:sqlserver://hostdummy_hostname:port/databasedummy_databasename

To encrypt the password, use F12 menu (clipboard - > encrypt password, available since 2016.01)

using attributes, constraints, relations etc.

In the above example the visibilityStartTime and visibilityEndTime tags are used to define the columns in the CSV/DBF file that contain the start and end dateTimes of the visibilityPeriod for each location. The (optional) visibilityPeriod is the period for which a location is visible in the user interface. The start and the end of the period are inclusive. Currently the visibility period is used in the map (explorer) window, the time series display and the spatial display. If startDateTime is not defined, then the location is visible for all times before endDateTime. If endDateTime is not defined, then the location is visible for all times after startDateTime. If startDateTime and endDateTime are both not defined, then the location is visible for all times. Furthermore the (optional) dateTimePattern tag is used to define the pattern for the dateTimes defined in the CSV/DBF file. If dateTimePattern is not specified, then the default pattern "yyyyMMdd" is used, which is the internal format that a CSV/DBF file uses for columns of type 'D' (date columns). The (optional) timeZoneOffset is the offset of the times in the CSV/DBF file, relative to GMT. For example "+02:00" means GMT+02:00. If no offset is specified, then time zone GMT is used by default.

Next you can derive the required locationSets from this dump by using constraints.
You can use constraints like:

attributeTextEquals
attributeTextContains
attributeTextStartsWith
idContains
attributeExists
etc (see schema or the schema diagram)

For example:

...

<locationSet id="ST_K.meting" editable="false">
    <locationSetId>gegevensdump</locationSetId>
    <constraints>
      <not>
        <attributeTextEquals equals="" id="IDMAP_KLEP"/>
      </not>
      <attributeTextEquals equals="Stuwen" id="TYPE"/>
    </constraints>
</locationSet>

location relations

...

Relations between location relations can be used in attribute functions, transformations and time series sets. Currently the one to many relations can only be used for time series sets in the filters.xml.

Code Block

	xml
	xml

<locationSet id="Stations">
	<esriShapeFile>
		<file>Stations</file>
		<geoDatum>WGS 1984</geoDatum>
		<id>%ID%</id>
		<name>%ID%</name>
		<x>%X%</x>
		<y>%Y%</y>
		<z>0</z>
		<attributeFile>
			<csvFile>upstream</csvFile>
			<id>%ID%</id>
			<oneToManyRelation id="UPSTREAM">
				<relatedLocationId>%RELATION%</relatedLocationId>
			</oneToManyRelation>
			<relation id="CATCHMENT">
				<relatedLocationId>%CATCHMENT%</relatedLocationId>
			</relation>   
		</attributeFile>
	</esriShapeFile>

time dependent z (since 2022.01)

...

When using a separate attribute file the z can be time dependent. The z is used for converting a local datum to a global datum (water level).

Code Block

	xml
	xml

<locationSet id="LocationSetWithTimeDependentAttributes">
	<csvFile>
		<file>TimeDependentLocationSet.csv</file>
		<id>%id%</id>
		<x>0</x>
		<y>0</y>
		<attributeFile>
			<csvFile>TimeDependentLocationSetAttributes.csv</csvFile>
			<id>%id%</id>
			<startDateTime>%START%</startDateTime>
			<endDateTime>%END%</endDateTime>
			<z>%z%</z>
			<attribute id="timeDependentAttribute">
				<text>%timeDependentAttribute%</text>
			</attribute>
		</attributeFile>
	</csvFile>
</locationSet>

Code Block

ID;START;END;timeDependentAttribute;z
TD_Loc1;19000101;21000101;A;1
TD_Loc2;19000101;19850103;A;2
TD_Loc2;19850103;21000101;B;3
TD_Loc3;19000101;19850103;B;4
TD_Loc3;19850103;21000101;A;5
TD_Loc4;19000101;19850103;A;6
TD_Loc4;19850103;21000101;B;7
TD_Loc5;19000101;19850103;A;8
TD_Loc5;19850103;21000101;B;9

using attributes, constraints, relations etc.

...

In the above example the visibilityStartTime and visibilityEndTime tags are used to define the columns in the CSV/DBF file that contain the start and end dateTimes of the visibilityPeriod for each location. The (optional) visibilityPeriod is the period for which a location is visible in the user interface. The start and the end of the period are inclusive. Currently the visibility period is used in the map (explorer) window, the time series display and the spatial display. If startDateTime is not defined, then the location is visible for all times before endDateTime. If endDateTime is not defined, then the location is visible for all times after startDateTime. If startDateTime and endDateTime are both not defined, then the location is visible for all times. Furthermore the (optional) dateTimePattern tag is used to define the pattern for the dateTimes defined in the CSV/DBF file. If dateTimePattern is not specified, then the default pattern "yyyyMMdd" is used, which is the internal format that a CSV/DBF file uses for columns of type 'D' (date columns). The (optional) timeZoneOffset is the offset of the times in the CSV/DBF file, relative to GMT. For example "+02:00" means GMT+02:00. If no offset is specified, then time zone GMT is used by default.

Next you can derive the required locationSets from this dump by using constraints.
You can use constraints like:

attributeTextEquals
attributeTextContains
attributeTextStartsWith
idContains
attributeExists
etc (see schema or the schema diagram)

For example:

Code Block

	xml
	xml

<locationSet id="ST_K.meting" editable="false">
    <locationSetId>gegevensdump</locationSetId>
    <constraints>
      <not>
        <attributeTextEquals equals="" id="IDMAP_KLEP"/>
      </not>
      <attributeTextEquals equals="Stuwen" id="TYPE"/>
    </constraints>
</locationSet>

It is also possible in a locationSet to link to time-dependent attributes. Time-dependent attributes need to be defined in a separate CSV/DBF file. In the locationSet use the attributeFile tag to make a reference to such a file. The following xml example has a reference to the file PumpStationsAttributes.dbf, which contains attributes that have different values for different periods in time, as well as different values for different locations. In this case the startDateTime and endDateTime tags are used to define the columns in the CSV/DBF file that contain the start and end dateTimes for each row. A given row in the CSV/DBF file contains values that are only valid between the time period for that row. This period is defined by the optional startDateTime and endDateTime for that row. If a row has no startDateTime, then it is valid always before the endDateTime. If a row has no endDateTime, then it is valid always after the startDateTime. If a row has no startDateTime and no endDateTime, then it is always valid. When Time-dependent attributes are change in time, make sure the startDateTime of one row is the same as the endDateTime of the previous row. If there is a gap of one day between the startDeteTime and the endDateTime, then there is a gap of one day for the validity of the location attributes. Since the 2021.01 the <attributeFile> can also be used to add additional attributes when the locations itself are read from a database table <table> .

Code Block

	xml
	xml

<locationSet id="PumpStations">
  <esriShapeFile>
    <file>PumpStations</file>
    <geoDatum>WGS 1984</geoDatum>
    <id>%ID%</id>
    <name>%ID%</name>
    <x>%X%</x>
    <y>%Y%</y>
    <z>0</z>
    <attributeFile>
      <csvFile>PumpStationsAttributes.csv</csvFile>
      <id>%ID%</id>
      <timeZoneOffset>+05:00</timeZoneOffset>
      <dateTimePattern>dd-MM-yyyy HH:mm</dateTimePattern>
      <startDateTime>%START%</startDateTime>
      <endDateTime>%EIND%</endDateTime>
      <attribute id="speed">
        <number>%FREQ%</number>
      </attribute>
      <attribute id="discharge">
        <number>%POMPCAP%</number>
      </attribute>
    </attributeFile>
  </esriShapeFile>
</locationSet>

link shape file or geoJSON file to csv file

Code Block

	xml
	xml

<locationSet id="Meteo Stations">
	<csvFile>
		<file>Meteo_Stations</file>
		<geoDatum>WGS 1984</geoDatum>
		<id>%ID%</id>
		<name>%ID%</name>
		<x>%X%</x>
		<y>%Y%</y>
		<z>0</z>
		<attribute id="region">
			<description>Catchment</description>
			<text>%REGION%</text>
		</attribute>
		<attributeFile>
			<esriShapeFile>Meteo_Stations_Polygons</esriShapeFile>
			<id>%ID%</id>
			<attribute id="freq">
				<number>%FREQ%</number>
			</attribute>
		</attributeFile>
	</csvFile>
</locationSet>

It is also possible in a locationSet to link to time-dependent attributes. Time-dependent attributes need to be defined in a separate CSV/DBF file. In the locationSet use the attributeFile tag to make a reference to such a file. The following xml example has a reference to the file PumpStationsAttributes.dbf, which contains attributes that have different values for different periods in time, as well as different values for different locations. In this case the startDateTime and endDateTime tags are used to define the columns in the CSV/DBF file that contain the start and end dateTimes for each row. A given row in the CSV/DBF file contains values that are only valid between the time period for that row. This period is defined by the optional startDateTime and endDateTime for that row. If a row has no startDateTime, then it is valid always before the endDateTime. If a row has no endDateTime, then it is valid always after the startDateTime. If a row has no startDateTime and no endDateTime, then it is always valid. When Time-dependent attributes are change in time, make sure the startDateTime of one row is the same as the endDateTime of the previous row. If there is a gap of one day between the startDeteTime and the endDateTime, then there is a gap of one day for the validity of the location attributes. Since the 2021.01 the <attributeFile> can also be used to add additional attributes when the locations itself are read from a database table <table> .

Code Block

xml

<locationSet id="PumpStations">
  <esriShapeFile>
    <file>PumpStations</file>
    <geoDatum>WGS 1984</geoDatum>
    <id>%ID%</id>
    <name>%ID%</name>
    <x>%X%</x>
    <y>%Y%</y>
    <z>0</z>
    <attributeFile>
      <csvFile>PumpStationsAttributes.csv</csvFile>
      <id>%ID%</id>
      <timeZoneOffset>+05:00</timeZoneOffset>
      <dateTimePattern>dd-MM-yyyy HH:mm</dateTimePattern>
      <startDateTime>%START%</startDateTime>
      <endDateTime>%EIND%</endDateTime>
      <attribute id="speed">
        <number>%FREQ%</number>
      </attribute>
      <attribute id="discharge">
        <number>%POMPCAP%</number>
      </attribute>
    </attributeFile>
  </esriShapeFile>
</locationSet>

locationIcons

...

It is possible to define the location icon with a new option in the locationSets derived from CSV/Shape-DBF files. You can define the location icon with the element iconName. The icon files should be defined as complete file name and this file should be available in the Config\IconFiles directory. If you want to refer to Config\IconFiles\Waterlevel.gif, you should define the iconName as

...

It is possible to reference an attribute that is available at a related location by prefixing an attribute id with a location relation id.

@relationId:attributeId@

examples

@CITECTLOC@_@IDMAP_DEBIET@

...

Code Block

	xml
	xml

<transformation id="pump with head-discharge table with coefficient set functions">
  <structure>
    <pumpHeadDischargeTable>
      ...
      <coefficientSetFunctions>
        <headDischargeTableRecord head="@head@" discharge="@discharge@ * 1000"/>
      </coefficientSetFunctions>
      ...
    </pumpHeadDischargeTable>
  </structure>
</transformation>

...

Sample input and output

...

Code Block

language	xml
title	Example of attribute file

<locationSet id="locations">
  <csvFile>
    <file>locations.csv</file>
    <id>%ID%</id>
    <x>%X%</x>
    <y>%Y%</y>
    <attribute .....// first here your normal attributes
    </attribute>
    <attributeFile>
      <csvFile>pumpcurves.csv</csvFile>
      <id>%LOCID%</id>
      <attribute id="head">
        <number>%dH%</number>
      </attribute>
      <attribute id="discharge">
        <number>%Q%</number>
      </attribute>
    </attributeFile>
  </csvFile>
</locationSet>

...

Code Block

title	example csv file

LOCID,dH,Q
0001,0,0
0001,0.1,1
0001,0.2,1.9
0001,0.3,2.5
0001,0.5,3.5
0001,1,5

Error and warning messages

...

Description of errors and warnings that may be generatedgenerated

Error:

exceeds 255 characters *.csv Illegal argument Column name

Error can appear when using standard encoding (ISO-8859-1) in the CSV file.

Action:

Reduce amount of characters in cell. Error will also appear when the column is not used in the definition of the LocationSet and/or attributes.

Error:

Error message

Action:

Action to fix

...

Known issues

...

Describe all known issues and unexpected behaviour

...

Since 2016.01 an error is logged when values contain decimal comma instead of decimal point. e.g. 453,73 should be 453.73
When using standard encoding (ISO-8859-1) Delft-FEWS will return an Illegal argument error when a Column name exceeds 255 characters.

Technical reference

...

Entry in moduleDescriptors:	Specification of: ENTRY and DESCRIPTION in the SystemConfigFiles\ModuleDescriptors.xml

No Format
<moduleDescriptor id="ENTRY"> <description>DESCRIPTION</description>

...

Save

Page tree

Versions Compared

Old Version 51

New Version Current

Key

Contents

Configuration

locationSets from CSV file

locationSets from esriShapeFilelocationSets from esriShapeFile

locationSets from

geoJSON file

locationSets from database table

using attributes, constraints, relations etc.

location relations

time dependent z (since 2022.01)

using attributes, constraints, relations etc.

link shape file or geoJSON file to csv file

locationIcons

Sample input and output

Error and warning messages

Known issues

Related modules and documentation

Technical reference