Overview

Exports time series data in a tabular ASCII format from the Australian Bureau of Meteorology; the HCS format stands for (Hydrological Comma Separated. The files consist of a set of header lines and then lines with a fixed number of fields. The fields are separated by a comma and the order is fixed. Multiple locations and parameters can be put in a single file.

The file format

The file format is described in the BOM document "External Agency Hydrological data Transfer - Client Requirements, Version 2.20".

An example of a file is shown below:

# HEADER: Agency Id: BoM
# HEADER: File Generation Date: 2008-08-01T14:30:45z
# HEADER: File Format: BOM-HCS
# HEADER: File Format Version: 2.0
# HEADER: Generated by (system): TimeStudio
# HEADER: Number of Records: 11
# HEADER: Local ObsTime Offset: 0
# HEADER: Data Fields: IndexNo, SensorType, SensorDataType, SiteIdType, SiteId, ObservationTimestamp, RealValue, Unit, SensorTypeParam1, SensorTypeParam2, Quality, Comment
1,"WL",1,"SLSR","44198-01-01","2008-02-01T01:07:06z",1.150000,"metres","LGH",,1,""
2,"WL",1,"SLSR","44198-01-01","2008-02-01T01:08:06z",1.200000,"metres","LGH",,1,""
3,"WL",1,"SLSR","44198-01-01","2008-02-01T01:43:06z",1.150000,"metres","LGH",,1,""
4,"WL",1,"SLSR","44198-01-01","2008-02-01T01:46:06z",1.200000,"metres","LGH",,1,""
5,"WL",1,"SLSR","44198-01-01","2008-02-01T02:04:06z",1.150000,"metres","LGH",,1,""
6,"WL",1,"SLSR","44198-01-01","2008-02-01T02:11:06z",1.200000,"metres","LGH",,1,""
7,"WL",1,"SLSR","44198-01-01","2008-02-01T02:16:06z",1.150000,"metres","LGH",,1,""
8,"WL",1,"SLSR","44198-01-01","2008-02-01T02:22:06z",1.200000,"metres","LGH",,1,""
9,"WL",1,"SLSR","44198-01-01","2008-02-01T02:23:06z",1.150000,"metres","LGH",,1,""
10,"WL",1,"SLSR","44198-01-01","2008-02-01T02:25:06z",1.200000,"metres","LGH",,1,""
11,"WL",1,"SLSR","44198-01-01","2008-02-01T02:28:06z",1.150000,"metres","LGH",,1,""

Example config file

An example Export configuration for hte HCS format is provided below.

<?xml version="1.0" encoding="UTF-8"?>
<timeSeriesExportRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/timeSeriesExportRun.xsd">
	<export>
		<general>
			<exportType>hcs</exportType>
			<folder>$EXPORT_FOLDER$/ProdStore</folder>
			<exportFileName>
				<name>$PRODUCTID$.hcs</name>
				<suffix>
					<timeZeroFormattingString>_yyyyMMddHHmm</timeZeroFormattingString>
				</suffix>
			</exportFileName>
			<idMapId>IdExportHCS</idMapId>
			<flagConversionsId>HcsExportFlagConversions_Obs</flagConversionsId>
			<exportMissingValueString>-10000000</exportMissingValueString>
			<omitMissingValues>true</omitMissingValues>
			<exportTimeZone>
				<timeZoneName>GMT</timeZoneName>
			</exportTimeZone>
			<dataFeedId>Export.hcs</dataFeedId>
			<convertDatum>false</convertDatum>
			<exportLastValue>true</exportLastValue>
		</general>
		<properties>
			<string key="systemName" value="HyFS"/>
		</properties>
		<exportAttribute internalAttributeId="Z" externalAttributeId="GAUGE_DATUM"/>
		<timeSeriesSet>
			<moduleInstanceId>Normalize_RiverTelemetry</moduleInstanceId>
			<valueType>scalar</valueType>
			<parameterId>H.obs.proc</parameterId>
			<locationSetId>$REGION$_Product_HCS_WaterHeight</locationSetId>
			<timeSeriesType>external historical</timeSeriesType>
			<timeStep unit="nonequidistant"/>
			<relativeViewPeriod unit="hour" start="$START$" end="0"/>
			<readWriteMode>read only</readWriteMode>
		</timeSeriesSet>
	</export>
</timeSeriesExportRun>

In this example water level series are exported. For water levels the ID mapping will normally contain the following information.

    <parameter internal="H.obs" external="WL" externalQualifier="1" externalQualifier2="metres"/>

  • Parameter external = WL (this is the  SensorType column)
  • externalQualifier = 1 (this is the SensorDataType column)
  • externalQualifier2 = meters (this is the Unit column)

For rainfall (or other) parameters it is also possible to add the timestep of the series, this need to be added explicitly to the ID mapping.

    <parameter internal="P.obs.period" internalQualifier1="Accum" internalQualifier2="15m" external="RN" externalQualifier="6" externalQualifier2="mm" externalQualifier3="900"/>

  • externalQualifier3 = 900 (this is the  SensorParam2 column)

Whe nsome of hte qualifers are not configured in hte ID mapping, default values are used.

  • String sensorDataType configurable as a property for use in the data rows. Default: "6".
  • String unit is configurable as a property for use in the data rows. Default: "mm".

Properties

String systemName is configurable as a property for use in header. Default: "HYDS"

#HEADER: Generated by (system): 'systemName'


Export Last Value

In the general element the <exportLastValue> element can be set to true. This will only export the last value of the configured Relative View Period.


Export Gauge datum

When water levels are exported in local datum or global datum, the datum can be exported to the  SensorParam1 column. To enable this functionality the location attribute need to be linked to the GAUGE_DATUM colmn.

        <exportAttribute internalAttributeId="Z" externalAttributeId="GAUGE_DATUM"/>

In the above example the location attribute "Z" will be linked to externalk attribute GAUGE_DATUM, this is the SensorParam1 column. This is only enabled when the external parameter is water level "WL".




  • No labels

Disclaimer