Excerpt | ||
---|---|---|
| ||
Imports csv type time series data from the Woodleigh System Control Centre in Singapore |
Overview
Note |
---|
This import is available in DELFT-FEWS versions after 06-12-2007 |
Imports time series data in csv format from the Woodleigh System Control Centre in Singpore. These file ar in a csv formatSingapore. The first line is a header for each column indicating the location. The filename encodes the parameter. egEg., in the file Aname_RF.txt is parameter is RF (rainfall). If the column after a data columns has a header named "Qf" it is interpreted as a column holding quality flags for the data column. The flags are converted to DELFT-FEWS data flags using the flagconversions mapping.
Configuring the Import
The reader is named WSCCCsv which should be configured in the general section of the import. An example import configuration is shown below:
No Format |
---|
<?xml version="1.0" encoding="UTF-8"?> <timeSeriesImportRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/timeSeriesImportRun.xsd"> <import> <general> <importType>WSCCCsv</importType> <folder>$IMPORT_FOLDER$/WSCC</folder> <failedFolder>$IMPORT_FAILED_FOLDER$</failedFolder> <idMapId>IdImportWSCC</idMapId> <flagConversionsId>ImportFlagConversions</flagConversionsId> <importTimeZone> <timeZoneOffset>+08:00</timeZoneOffset> </importTimeZone> </general> <timeSeriesSet> <moduleInstanceId>ImportWSCC</moduleInstanceId> <valueType>scalar</valueType> <parameterId>H.obs</parameterId> <locationSetId>WSCC_Level</locationSetId> <timeSeriesType>external historical</timeSeriesType> <timeStep unit="minute" multiplier="10"/> <readWriteMode>add originals</readWriteMode> <synchLevel>1</synchLevel> <expiryTime unit="day" multiplier="3650"/> </timeSeriesSet> </import> </timeSeriesImportRun> |
...
No Format |
---|
<?xml version="1.0" encoding="UTF-8"?> <idMap version="1.1" xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/idMap.xsd"> <parameter internal="H.obs" external="LEV"/> <parameter internal="Q.obs" external="FLW"/> <parameter internal="P.obs" external="RF"/> <parameter internal="Q.rated" external="VOL"/> <!-- vlume --> <location internal="LowerPierce" external="S23-USR-LEV-RES-1"/> </idMap> |
An example flag conversions file for the WSCC data is shown below:
No Format |
---|
<?xml version="1.0" encoding="UTF-8"?>
<!--WSCC flag conversion file for Import-->
<flagConversions xmlns="http://www.wldelft.nl/fews"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.wldelft.nl/fews
http://fews.wldelft.nl/schemas/version1.0/flagConversions.xsd">
<!-- F - Telemetry Failed
N - Telemetry Normal
M - ManuallySet ( value set by the opeartor)
B - Blocked ( alarm disabled by the operator )
C - Calculation Failure
-->
<flagConversion>
<inputFlag>
<name>Telemetry Failed</name>
<value>F</value>
</inputFlag>
<outputFlag>
<name>ORIGINAL_MISSING</name>
<value>9</value>
<description>Missing value in originally observed series.
Note this is a special form of both ORIGINAL/UNRELIABLE
and ORIGINAL/RELIABLE.</description>
</outputFlag>
</flagConversion>
<flagConversion>
<inputFlag>
<name>Telemetry Normal</name>
<value>N</value>
</inputFlag>
<outputFlag>
<name>ORIGINAL_RELIABLE</name>
<value>0</value>
<description>Observed value retrieved from external data source.
Value is valid, marked as original reliable as validation is yet to be done</description>
</outputFlag>
</flagConversion>
<flagConversion>
<inputFlag>
<name>ManuallySet ( value set by the opeartor)</name>
<value>M</value>
</inputFlag>
<outputFlag>
<name>CORRECTED_RELIABLE</name>
<value>1</value>
<description>The original value was removed and corrected.
Correction may be through byteerpolation or manual editing</description>
</outputFlag>
</flagConversion>
<flagConversion>
<inputFlag>
<name>Blocked ( alarm disabled by the operator )</name>
<value>B</value>
</inputFlag>
<outputFlag>
<name>ORIGINAL_UNRELIABLE</name>
<value>6</value>
<description>Observed value retrieved from external data source.
Value is invalid due to validation limits set. Value is removed.</description>
</outputFlag>
</flagConversion>
<flagConversion>
<inputFlag>
<name>Calculation Failure</name>
<value>C</value>
</inputFlag>
<outputFlag>
<name>ORIGINAL_DOUBTFUL</name>
<value>3</value>
<description>Observed value retrieved from external data source.
Value is valid, but marked as suspect due to soft validation limits being exceeded</description>
</outputFlag>
</flagConversion>
<defaultOuputFlag>
<name>ORIGINAL_RELIABLE</name>
<value>0</value>
<description>The data value is the original value retrieved from an
external source and it successfully passes all validation criteria set.</description>
</defaultOuputFlag>
<missingValueFlag>
<name>ORIGINAL_MISSING</name>
<value>9</value>
</missingValueFlag>
</flagConversions>
|
The file format
No Format |
---|
Date Time TN-1 QF-1 TN-2 QF-2 TN-3 QF-3 TN-N QF-N DataType date time float Char(1) float Char(1) float Char(1) float Char(1) DataFormat ddmmyyyy hh:mm xxxx.xxxxxx xxxx.xxxxxx xxxx.xxxxxx xxxx.xxxxxx |
...
- Definition of the Data quality flags(QF) as listed below;
F
Telemetry Failed
N
Telemetry Normal
M
ManuallySet ( value set by the opeartor)
B
Blocked ( alarm disabled by the operator )
C
Calculation Failure
- Name of the text file will be 'yyyymmddhhmm_PARAMETER'
Note |
---|
The current import ignores the quality flags. |
- Eg. 200703121500_RF
- TN stands for TagName (parameter name) which defined in the SCADA System. Also user can customise the TN which may different from the SCADA System
...
No Format |
---|
Date,Time,S23-USR-LEV-RES-1,Qf,S48(51)-UPP-LEV-RES-1,Qf,S46(50)-LPP-LEV-RES-1,Qf,S46(24)-MCP-LEV-RES-1,Qf
06052007,02:10,2.232906,N,8.376068,N,0.271062,F,0.039377,N
06052007,02:20,2.232906,B,8.376068,N,0.271062,F,0.039377,N
06052007,02:30,2.232906,F,8.376068,N,0.271062,F,0.039377,N
06052007,02:40,2.232906,M,8.376068,N,0.271062,F,0.039377,N
06052007,02:50,2.232906,C,8.376068,N,0.271062,F,0.039377,N
06052007,03:00,2.232906,N,8.376068,N,0.271062,F,0.039377,N
|