Overview
Imports time series data from a URL in generalCsv format with one header line containing a column headers of the time series:
- The first line contains the column names (fields) in the csv file
- All other lines contain the values for the columns.
Import type
The import type is CyMons. It determines the correct URL to read the data from and passes it to the General Csv parser.
Use the same table data as needed when using General Csv directly.
URL setup
The base URL must be given in the configuration. The next is used in testing:
http://wispweb.waterinsight.nl/api?service=cymons&version=1.0.0&request=GetData&format=csv
The parser extends the URL for this service are as follows:
-bbox: Put here the spatial extent box in lat/lon. Four numbers are expected, separated with commas, and are in the order minx, miny, maxx, maxy. This exact string will be taken from the property "extent" in the import configuration and with a '&' put after the base URL.
Configuration for extending URL with &bbox=5.0,40.0,10.0,60.0
<properties> <string key="extent" value="bbox=5.0,40.0,10.0,60.0"/> </properties>
-time: Specify here the temporal interval, using two ISO timestamps separated by a comma. The timestamps will be the startTime and endTime of the period for the import.
For example: &time=2012-06-06T00:00:00,2012-06-07T00:00:00
The total URL will then be:
This returns the following data:
Data returned from URL
region,station,date,latitude,longitude,chl,tsm,cpc,kd IJsselmeer,mussel station,2012-06-06T12:26:59+02:00,5.31635,52.6991333333333,29.3505278308,18.4597481301,57.9296245881,1.14733568134 IJsselmeer,mussel station,2012-06-06T12:26:11+02:00,5.31635,52.6991333333333,28.651262766,15.5006520974,57.6792727273,0.605416725609 IJsselmeer,mussel station,2012-06-06T12:25:53+02:00,5.31635,52.6991333333333,29.3137135645,16.3497957084,57.4957619108,0.745017947332 IJsselmeer,mussel station,2012-06-06T12:25:40+02:00,5.31635,52.6991333333333,0.0,15.5725644785,58.9776510609,0.469809131707 IJsselmeer,mussel station,2012-06-06T12:25:05+02:00,5.31635,52.6991333333333,30.2566391293,17.2406431075,57.3583605658,0.945756308022 IJsselmeer,mussel station,2012-06-06T12:24:49+02:00,5.31635,52.6991333333333,30.5027829311,19.5533337059,56.916185112,1.35591351096 IJsselmeer,mussel station,2012-06-06T12:24:32+02:00,5.31635,52.6991333333333,31.6586633907,17.7698561703,59.3494926334,1.04303262978 IJsselmeer,mussel station,2012-06-06T12:24:18+02:00,5.31635,52.6991333333333,31.5253240584,17.3288079801,59.8415629811,0.917639816894 IJsselmeer,FL47,2012-06-06T10:18:18+02:00,5.2238,52.9124666666667,60.5626101647,36.0802119284,113.562462874,2.86612140427 IJsselmeer,FL47,2012-06-06T10:17:59+02:00,5.2238,52.9124666666667,60.6012048609,35.0840735276,112.923119892,2.45612283233 IJsselmeer,FL47,2012-06-06T10:17:41+02:00,5.2238,52.9124666666667,59.639546751,33.8951103188,111.570965818,2.62868840577 IJsselmeer,FL47,2012-06-06T10:17:19+02:00,5.2238,52.9124666666667,58.4780792048,35.0013366563,111.283689416,3.00098071488 IJsselmeer,FL09,2012-06-06T08:14:46+02:00,5.48843333333333,52.8102666666667,34.19905689,22.8225272192,57.6093737272,0.897942981739 IJsselmeer,FL09,2012-06-06T08:14:29+02:00,5.48843333333333,52.8102666666667,0.0,21.2177865596,58.908051498,0.614010565726 IJsselmeer,FL09,2012-06-06T08:14:13+02:00,5.48843333333333,52.8102666666667,35.4195921068,24.0490306661,58.5208567651,1.04311419186 IJsselmeer,FL09,2012-06-06T08:13:54+02:00,5.48843333333333,52.8102666666667,34.8168585571,23.6797125806,57.9979266736,0.934142681444
The URL can be protected by authentication, the parser can supply username and password configured in the import together with the base URL (escape & in xml by using & ):
<importType>CyMons</importType> <serverUrl>http://wispweb.waterinsight.nl/api?service=cymons&version=1.0.0&request=GetData&format=csv</serverUrl> <user>dummy_username</user> <password>dummy_password</password>
Config Example
Configuration for CyMons import
<?xml version="1.0" encoding="UTF-8"?> <timeSeriesImportRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/timeSeriesImportRun.xsd"> <import> <general> <importType>CyMons</importType> <serverUrl>http://wispweb.waterinsight.nl/api?service=cymons&version=1.0.0&request=GetData&format=csv</serverUrl> <user>dummy_username</user> <password>dummy_password</password> <relativeViewPeriod unit="day" start="-1000" end="0" startOverrulable="true" endOverrulable="true"/> <table name="WISPWEB_csv"> <dateTimeColumn name="date" pattern="yyyy-MM-dd'T'HH:mm:ssX"/> <locationColumn name="station"/> <valueColumn name="chl" unit="mg/l" parameterId="CHLFa.m"/> <valueColumn name="tsm" unit="mg/l" parameterId="TSM.m"/> <valueColumn name="cpc" unit="mg/l" parameterId="CPC.m"/> <valueColumn name="kd" unit="mg/l" parameterId="KD.m"/> </table> <idMapId>IdImportCymons</idMapId> <unitConversionsId>ImportUnitConversions</unitConversionsId> <importTimeZone> <timeZoneOffset>+00:00</timeZoneOffset> </importTimeZone> <dataFeedId>Cymons</dataFeedId> </general> <properties> <string key="extent" value="bbox=5.0,40.0,10.0,60.0"/> </properties> <timeSeriesSet> <moduleInstanceId>ImportCymons</moduleInstanceId> <valueType>scalar</valueType> <parameterId>TSM.m</parameterId> <locationSetId>WispLocations</locationSetId> <timeSeriesType>external historical</timeSeriesType> <timeStep unit="nonequidistant"/> <readWriteMode>add originals</readWriteMode> <synchLevel>1</synchLevel> <expiryTime unit="day" multiplier="2"/> </timeSeriesSet> <timeSeriesSet> <moduleInstanceId>ImportCymons</moduleInstanceId> <valueType>scalar</valueType> <parameterId>CHLFa.m</parameterId> <locationSetId>WispLocations</locationSetId> <timeSeriesType>external historical</timeSeriesType> <timeStep unit="nonequidistant"/> <readWriteMode>add originals</readWriteMode> <synchLevel>1</synchLevel> <expiryTime unit="day" multiplier="2"/> </timeSeriesSet> <timeSeriesSet> <moduleInstanceId>ImportCymons</moduleInstanceId> <valueType>scalar</valueType> <parameterId>CPC.m</parameterId> <locationSetId>WispLocations</locationSetId> <timeSeriesType>external historical</timeSeriesType> <timeStep unit="nonequidistant"/> <readWriteMode>add originals</readWriteMode> <synchLevel>1</synchLevel> <expiryTime unit="day" multiplier="2"/> </timeSeriesSet> <timeSeriesSet> <moduleInstanceId>ImportCymons</moduleInstanceId> <valueType>scalar</valueType> <parameterId>KD.m</parameterId> <locationSetId>WispLocations</locationSetId> <timeSeriesType>external historical</timeSeriesType> <timeStep unit="nonequidistant"/> <readWriteMode>add originals</readWriteMode> <synchLevel>1</synchLevel> <expiryTime unit="day" multiplier="2"/> </timeSeriesSet> </import> </timeSeriesImportRun>
