Imports csv type time series data from the Woodleigh System Control Centre in Singapore |
This import is available in DELFT-FEWS versions after 06-12-2007 |
Imports time series data in csv format from the Woodleigh System Control Centre in Singapore. The first line is a header for each column indicating the location. The filename encodes the parameter. Eg., in the file Aname_RF.txt is parameter is RF (rainfall). If the column after a data columns has a header named "Qf" it is interpreted as a column holding quality flags for the data column. The flags are converted to DELFT-FEWS data flags using the flagconversions mapping.
The reader is named WSCCCsv which should be configured in the general section of the import. An example import configuration is shown below:
<?xml version="1.0" encoding="UTF-8"?> <timeSeriesImportRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/timeSeriesImportRun.xsd"> <import> <general> <importType>WSCCCsv</importType> <folder>$IMPORT_FOLDER$/WSCC</folder> <failedFolder>$IMPORT_FAILED_FOLDER$</failedFolder> <idMapId>IdImportWSCC</idMapId> <flagConversionsId>ImportFlagConversions</flagConversionsId> <importTimeZone> <timeZoneOffset>+08:00</timeZoneOffset> </importTimeZone> </general> <timeSeriesSet> <moduleInstanceId>ImportWSCC</moduleInstanceId> <valueType>scalar</valueType> <parameterId>H.obs</parameterId> <locationSetId>WSCC_Level</locationSetId> <timeSeriesType>external historical</timeSeriesType> <timeStep unit="minute" multiplier="10"/> <readWriteMode>add originals</readWriteMode> <synchLevel>1</synchLevel> <expiryTime unit="day" multiplier="3650"/> </timeSeriesSet> </import> </timeSeriesImportRun> |
An example IdMapping file (that maps the first column of the also attached example input file) is shown below:
<?xml version="1.0" encoding="UTF-8"?> <idMap version="1.1" xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/idMap.xsd"> <parameter internal="H.obs" external="LEV"/> <parameter internal="Q.obs" external="FLW"/> <parameter internal="P.obs" external="RF"/> <parameter internal="Q.rated" external="VOL"/> <!-- vlume --> <location internal="LowerPierce" external="S23-USR-LEV-RES-1"/> </idMap> |
An example flag conversions file for the WSCC data is shown below:
<?xml version="1.0" encoding="UTF-8"?> <!--WSCC flag conversion file for Import--> <flagConversions xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/flagConversions.xsd"> <!-- F - Telemetry Failed N - Telemetry Normal M - ManuallySet ( value set by the opeartor) B - Blocked ( alarm disabled by the operator ) C - Calculation Failure --> <flagConversion> <inputFlag> <name>Telemetry Failed</name> <value>F</value> </inputFlag> <outputFlag> <name>ORIGINAL_MISSING</name> <value>9</value> <description>Missing value in originally observed series. Note this is a special form of both ORIGINAL/UNRELIABLE and ORIGINAL/RELIABLE.</description> </outputFlag> </flagConversion> <flagConversion> <inputFlag> <name>Telemetry Normal</name> <value>N</value> </inputFlag> <outputFlag> <name>ORIGINAL_RELIABLE</name> <value>0</value> <description>Observed value retrieved from external data source. Value is valid, marked as original reliable as validation is yet to be done</description> </outputFlag> </flagConversion> <flagConversion> <inputFlag> <name>ManuallySet ( value set by the opeartor)</name> <value>M</value> </inputFlag> <outputFlag> <name>CORRECTED_RELIABLE</name> <value>1</value> <description>The original value was removed and corrected. Correction may be through byteerpolation or manual editing</description> </outputFlag> </flagConversion> <flagConversion> <inputFlag> <name>Blocked ( alarm disabled by the operator )</name> <value>B</value> </inputFlag> <outputFlag> <name>ORIGINAL_UNRELIABLE</name> <value>6</value> <description>Observed value retrieved from external data source. Value is invalid due to validation limits set. Value is removed.</description> </outputFlag> </flagConversion> <flagConversion> <inputFlag> <name>Calculation Failure</name> <value>C</value> </inputFlag> <outputFlag> <name>ORIGINAL_DOUBTFUL</name> <value>3</value> <description>Observed value retrieved from external data source. Value is valid, but marked as suspect due to soft validation limits being exceeded</description> </outputFlag> </flagConversion> <defaultOuputFlag> <name>ORIGINAL_RELIABLE</name> <value>0</value> <description>The data value is the original value retrieved from an external source and it successfully passes all validation criteria set.</description> </defaultOuputFlag> <missingValueFlag> <name>ORIGINAL_MISSING</name> <value>9</value> </missingValueFlag> </flagConversions> |
Date Time TN-1 QF-1 TN-2 QF-2 TN-3 QF-3 TN-N QF-N DataType date time float Char(1) float Char(1) float Char(1) float Char(1) DataFormat ddmmyyyy hh:mm xxxx.xxxxxx xxxx.xxxxxx xxxx.xxxxxx xxxx.xxxxxx |
NOTE :
F |
Telemetry Failed |
N |
Telemetry Normal |
M |
ManuallySet ( value set by the opeartor) |
B |
Blocked ( alarm disabled by the operator ) |
C |
Calculation Failure |
Example:
Date,Time,S23-USR-LEV-RES-1,Qf,S48(51)-UPP-LEV-RES-1,Qf,S46(50)-LPP-LEV-RES-1,Qf,S46(24)-MCP-LEV-RES-1,Qf 06052007,02:10,2.232906,N,8.376068,N,0.271062,F,0.039377,N 06052007,02:20,2.232906,B,8.376068,N,0.271062,F,0.039377,N 06052007,02:30,2.232906,F,8.376068,N,0.271062,F,0.039377,N 06052007,02:40,2.232906,M,8.376068,N,0.271062,F,0.039377,N 06052007,02:50,2.232906,C,8.376068,N,0.271062,F,0.039377,N 06052007,03:00,2.232906,N,8.376068,N,0.271062,F,0.039377,N |