• Created by Unknown User (schelle@wldelft.nl), last modified on 07-12-2007

Overview

This import is available in DELFT-FEWS versions after 06-12-2007

Imports time series data in csv format from the Woodleigh System Control Centre in Singapore. The first line is a header for each column indicating the location. The filename encodes the parameter. Eg., in the file Aname_RF.txt is parameter is RF (rainfall). If the column after a data columns has a header named "Qf" it is interpreted as a column holding quality flags for the data column. The flags are converted to DELFT-FEWS data flags using the flagconversions mapping.

Configuring the Import

The reader is named WSCCCsv which should be configured in the general section of the import. An example import configuration is shown below:

<?xml version="1.0" encoding="UTF-8"?>
<timeSeriesImportRun xmlns="http://www.wldelft.nl/fews"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.wldelft.nl/fews
http://fews.wldelft.nl/schemas/version1.0/timeSeriesImportRun.xsd">
	<import>
		<general>
			<importType>WSCCCsv</importType>
			<folder>$IMPORT_FOLDER$/WSCC</folder>
			<failedFolder>$IMPORT_FAILED_FOLDER$</failedFolder>
			<idMapId>IdImportWSCC</idMapId>
			<flagConversionsId>ImportFlagConversions</flagConversionsId>
			<importTimeZone>
				<timeZoneOffset>+08:00</timeZoneOffset>
			</importTimeZone>
		</general>
		<timeSeriesSet>
			<moduleInstanceId>ImportWSCC</moduleInstanceId>
			<valueType>scalar</valueType>
			<parameterId>H.obs</parameterId>
			<locationSetId>WSCC_Level</locationSetId>
			<timeSeriesType>external historical</timeSeriesType>
			<timeStep unit="minute" multiplier="10"/>
			<readWriteMode>add originals</readWriteMode>
			<synchLevel>1</synchLevel>
			<expiryTime unit="day" multiplier="3650"/>
		</timeSeriesSet>
	</import>
</timeSeriesImportRun>

An example IdMapping file (that maps the first column of the also attached example input file) is shown below:

<?xml version="1.0" encoding="UTF-8"?>
<idMap version="1.1" xmlns="http://www.wldelft.nl/fews"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.wldelft.nl/fews
http://fews.wldelft.nl/schemas/version1.0/idMap.xsd">
	<parameter internal="H.obs" external="LEV"/>
	<parameter internal="Q.obs" external="FLW"/>
	<parameter internal="P.obs" external="RF"/>
	<parameter internal="Q.rated" external="VOL"/>
	<!-- vlume -->
	<location internal="LowerPierce" external="S23-USR-LEV-RES-1"/>
</idMap>

An example flag conversions file for the WSCC data is shown below:

<?xml version="1.0" encoding="UTF-8"?>
<!--WSCC flag conversion file for Import-->
<flagConversions xmlns="http://www.wldelft.nl/fews" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:schemaLocation="http://www.wldelft.nl/fews 
http://fews.wldelft.nl/schemas/version1.0/flagConversions.xsd">
<!-- F -  Telemetry Failed
 N -  Telemetry Normal
 M -   ManuallySet ( value set by the opeartor)
 B -   Blocked ( alarm disabled by the operator )
 C -   Calculation Failure
-->
	<flagConversion>
		<inputFlag>
			<name>Telemetry Failed</name>
			<value>F</value>
		</inputFlag>
		<outputFlag>
			<name>ORIGINAL_MISSING</name>
			<value>9</value>
			<description>Missing value in originally observed series.
Note this is a special form of both ORIGINAL/UNRELIABLE
and ORIGINAL/RELIABLE.</description>
		</outputFlag>
	</flagConversion>
	<flagConversion>
		<inputFlag>
			<name>Telemetry Normal</name>
			<value>N</value>
		</inputFlag>
		<outputFlag>
			<name>ORIGINAL_RELIABLE</name>
			<value>0</value>
			<description>Observed value retrieved from external data source.
Value is valid, marked as original reliable as validation is yet to be done</description>
		</outputFlag>
	</flagConversion>
	<flagConversion>
		<inputFlag>
			<name>ManuallySet ( value set by the opeartor)</name>
			<value>M</value>
		</inputFlag>
		<outputFlag>
			<name>CORRECTED_RELIABLE</name>
			<value>1</value>
			<description>The original value was removed and corrected.
Correction may be through byteerpolation or manual editing</description>
		</outputFlag>
	</flagConversion>
	<flagConversion>
		<inputFlag>
			<name>Blocked ( alarm disabled by the operator )</name>
			<value>B</value>
		</inputFlag>
		<outputFlag>
			<name>ORIGINAL_UNRELIABLE</name>
			<value>6</value>
			<description>Observed value retrieved from external data source.
Value is invalid due to validation limits set. Value is removed.</description>
		</outputFlag>
	</flagConversion>
	<flagConversion>
		<inputFlag>
			<name>Calculation Failure</name>
			<value>C</value>
		</inputFlag>
		<outputFlag>
			<name>ORIGINAL_DOUBTFUL</name>
			<value>3</value>
			<description>Observed value retrieved from external data source.
Value is valid, but marked as suspect due to soft validation limits being exceeded</description>
		</outputFlag>
	</flagConversion>
	<defaultOuputFlag>
		<name>ORIGINAL_RELIABLE</name>
		<value>0</value>
		<description>The data value is the original value retrieved from an
external source and it successfully passes all validation criteria set.</description>
	</defaultOuputFlag>
	<missingValueFlag>
		<name>ORIGINAL_MISSING</name>
		<value>9</value>
	</missingValueFlag>
</flagConversions>

The file format

Date	Time	TN-1	QF-1	TN-2	QF-2	TN-3	QF-3	TN-N	QF-N
DataType	date	time	float	Char(1)	float	Char(1)	float	Char(1)	float	Char(1)
DataFormat	ddmmyyyy	hh:mm	xxxx.xxxxxx xxxx.xxxxxx	xxxx.xxxxxx		xxxx.xxxxxx

NOTE :

  1. All the columns in text file will be separated by comma(,) character
  1. Value of all the Columns contains the Rain Fall value for that particular time interval.
    This rain fall value is derive from current (raw accumulated pulse) value - previous (raw accumulated pulse) value,
    in other words the rain fall (in mm) within that time interval (i.e. 10 mins).
  1. Definition of the Data quality flags(QF) as listed below;

    F

    Telemetry Failed

    N

    Telemetry Normal

    M

    ManuallySet ( value set by the opeartor)

    B

    Blocked ( alarm disabled by the operator )

    C

    Calculation Failure

  2. Name of the text file will be 'yyyymmddhhmm_PARAMETER'
    Eg. 200703121500_RF
  3. TN stands for TagName (parameter name) which defined in the SCADA System. Also user can customise the TN which may different from the SCADA System

Example:

Date,Time,S23-USR-LEV-RES-1,Qf,S48(51)-UPP-LEV-RES-1,Qf,S46(50)-LPP-LEV-RES-1,Qf,S46(24)-MCP-LEV-RES-1,Qf
06052007,02:10,2.232906,N,8.376068,N,0.271062,F,0.039377,N
06052007,02:20,2.232906,B,8.376068,N,0.271062,F,0.039377,N
06052007,02:30,2.232906,F,8.376068,N,0.271062,F,0.039377,N
06052007,02:40,2.232906,M,8.376068,N,0.271062,F,0.039377,N
06052007,02:50,2.232906,C,8.376068,N,0.271062,F,0.039377,N
06052007,03:00,2.232906,N,8.376068,N,0.271062,F,0.039377,N
  • No labels

Disclaimer