This CanadaMeteo data import is used for targeted access of Observation Summary Data Observations summary data in XML format.

The files that are downloaded and imported can be found at http://dd.meteo.gc.ca/observations/xml/ . 

The parser is included in Delft-FEWS versions 2017.02 and higher.


This documentation contains:

1) Data Provider Documentation

2) Sample Configuration


This data import is available since Delft-FEWS 2018.02 and a patch more recent then January, 2020.

Please comment if anything further is needed or requires updating.

1) Data Provider Documentation

This service is hosted by the Meteorological Service of Canada:

For overview documentation https://eccc-msc.github.io/open-data/msc-data/obs_station/readme_obs_insitu_xmldatamart_en/

The license of the data provider (Environment and Climate Change Canada) can be found here  https://dd.weather.gc.ca/doc/LICENCE_GENERAL.txt

2) Sample Configuration

Example of the Delft-FEWS import module

<?xml version="1.0" encoding="UTF-8"?>
<!-- edited with XMLSpy v2006 rel. 3 sp1 (http://www.altova.com) by Computer Services (WL | Delft Hydraulics) -->
<timeSeriesImportRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/timeSeriesImportRun.xsd">
   <import>
      <general>
         <importType>CanadaMeteo</importType>
         <serverUrl>http://dd.meteo.gc.ca/observations/xml</serverUrl>
         <user>dummy_username</user>
         <password>dummy_password</password>
		 <!-- Time period has to be configured for the importer to work -->
         <relativeViewPeriod unit="hour" start="-5" end="0" startOverrulable="true" endOverrulable="true"/>
      </general>
      <properties>
         <!--Optional configuration field. Value is a comma separated string of the region folder to be included.-->
         <!--If it is not configured, data will be fetched from all folders for the specified time. Case sensitive.-->
         <!--Possible choices: AB, BC, MB, NB, NL, NS, NT, NU, ON, PE, QC, SK, YT-->
         <string key="RegionDirectory" value="AB, BC"/>
         <!--Optional configuration field. Value is a comma separated string of the time folders to be included.-->
         <!--If it is not configured, data will be fetched from all folders for the specified time.-->
         <!--Possible choices: "hourly", "today", "yesterday"-->
         <string key="TimeDirectory" value="hourly, today"/>
      </properties>
      <timeSeriesSet>
         <moduleInstanceId>CanadaMeteo</moduleInstanceId>
         <valueType>scalar</valueType>
         <parameterId>mean_sea_level</parameterId>
         <locationId>3025481</locationId>
         <timeSeriesType>external historical</timeSeriesType>
         <timeStep unit="nonequidistant"/>
         <readWriteMode>add originals</readWriteMode>
      </timeSeriesSet>
   </import>
</timeSeriesImportRun>


Configuration Notes:

  • The first set of data (member) contains information that is not specific to one location,  it is not imported.
  • Data concerning record temperatures, record rain, record snow, since when temperature and precipitation data is available, present weather and tendency characteristic are ignored. (Thus fields that do not contain data measured at sampling time).
  • Measurements with no "value" are ignored.
  • Wind direction is calculated from code (for example, NW for NorthWest) to degrees. For location ID climate station number is used.
  • If, in the import module, a RegionDirectory is configured (for the purpose of importing data for, say, a single Province only), special attention should be paid to make sure that the locationSet that data is imported to, includes locations that are available in the directory specified. 
  • If data is not available for the chosen times, en empty time series set is returned.
  • It is practical to configure either a short time period, or other restrictions to prevent running out of local memory.

Locations and id mapping

Two files used for Locations and IdMapping are linked here.

  • Link: A list of Environment Canada locations from which data can be imported. 
    • Note that the 'external location IDs' are in the column "# MSC"; these correspond to the XML data file's climate_station_number.
    • This means you will either need to set these as your Delft-FEWS locationID
    • Alternatively, you need to import the MSC column as an attribute and use an id-mapping function to reference that, for example <locationIdFunction internalLocationSet="EC_stations" externalLocationFunction="@climate_station_number@"/> where, in the locationSets, climate_station_number is an attributeID defined from the associated column in the .csv file that stores the locations: <attribute id="climate_station_number"><text>%# MSC%</text></attribute>.
  • Link: IdMappingFile

Note that both files were valid at the time of upload to the WIKI. Any changes made by Environment Canada, to either the locations or the naming of parameters, will not be automatically reflected in these attachments. 


These files adhere to a schema: xmlns="http://dms.ec.gc.ca/schema/point-observation/2.1" An example of such an xml data file:

om:ObservationCollection xmlns:om="http://www.opengis.net/om/1.0" xmlns="http://dms.ec.gc.ca/schema/point-observation/2.1" xmlns:gml="http://www.opengis.net/gml" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <om:member>
      <om:Observation>
         <om:metadata>
            <set>
               <general>
                  <author build="build.4083" name="MSC-DMS-PG-WXO-Summary" version="3.1"/>
                  <dataset name="mscobservation/atmospheric/surface_weather/wxo_dd_today_summary-1.0-ascii/"/>
                  <phase name="product-wxo_xml-1.0/"/>
                  <id xlink:href="/data/msc/observation/atmospheric/surface_weather/wxo_dd_today_summary-1.0-ascii/product-wxo_xml-1.0/20181012234500000/sk/intermediate/fr"/>
                  <parent xlink:href="/data/msc/observation/atmospheric/surface_weather/wxo_dd_today_summary-1.0-ascii/product-wxo_xml-1.0/20181012234500000/sk/intermediate/fr"/>
               </general>
               <identification-elements>
                  <element name="province" uom="unitless" value="SK"/>
                  <element name="creation_date_utc" uom="unitless" value="2018-10-12T23:50:00.000Z"/>
                  <element name="observation_date_utc" uom="unitless" value="2018-10-12T23:45:00.000Z"/>
                  <element name="license" uom="unitless" value="http://dd.weatheroffice.ec.gc.ca/doc/LICENCE_GENERAL.txt"/>
                  <element name="language" uom="unitless" value="FR"/>
               </identification-elements>
            </set>
         </om:metadata>
         <om:samplingTime>
            <gml:TimeInstant>
               <gml:timePosition>2018-10-12T23:00:00.000Z</gml:timePosition>
            </gml:TimeInstant>
         </om:samplingTime>
         <om:resultTime>
            <gml:TimeInstant>
               <gml:timePosition>2018-10-12T23:00:00.000Z</gml:timePosition>
            </gml:TimeInstant>
         </om:resultTime>
         <om:procedure xlink:href="msc/observation/atmospheric/surface_weather/wxo_dd_today_summary-1.0-ascii/product-wxo_xml-1.0/20181012234500000/sk/intermediate/fr"/>
         <om:observedProperty gml:remoteSchema="/schema/point-observation/2.0.xsd"/>
         <om:featureOfInterest>
            <gml:FeatureCollection>
               <gml:location>
                  <gml:Point>
                     <gml:pos/>
                  </gml:Point>
               </gml:location>
            </gml:FeatureCollection>
         </om:featureOfInterest>
         <om:result>
            <elements>
               <element name="hot_spot_location_canada" uom="unitless" value="Rolling Hills">
                  <qualifier name="hot_spot_temperature_canada" uom="Celsius" value="20.0"/>
                  <qualifier name="province" uom="AB" value="unitless"/>
               </element>
               <element name="cold_spot_location_canada" uom="unitless" value="Eureka">
                  <qualifier name="cold_spot_temperature_canada" uom="Celsius" value="-21.8"/>
                  <qualifier name="province" uom="NU" value="unitless"/>
               </element>
               <element name="hot_spot_location_province" uom="unitless" value="Maple Creek">
                  <qualifier name="hot_spot_temperature_province" uom="Celsius" value="18.4"/>
               </element>
               <element name="cold_spot_location_province" uom="unitless" value="Broadview">
                  <qualifier name="cold_spot_temperature_province" uom="Celsius" value="-8.9"/>
               </element>
            </elements>
         </om:result>
      </om:Observation>
   </om:member>
   <om:member>
      <om:Observation>
         <om:metadata>
            <set>
               <general>
                  <author build="build.4083" name="MSC-DMS-PG-WXO-Summary" version="3.1"/>
                  <dataset name="mscobservation/atmospheric/surface_weather/wxo_dd_today_summary-1.0-ascii/"/>
                  <phase name="product-wxo_xml-1.0/"/>
                  <id xlink:href="/data/msc/observation/atmospheric/surface_weather/wxo_dd_today_summary-1.0-ascii/product-wxo_xml-1.0/20181012234500000/sk/intermediate/fr"/>
                  <parent xlink:href="/data/msc/observation/atmospheric/surface_weather/wxo_dd_today_summary-1.0-ascii/product-wxo_xml-1.0/20181012234500000/sk/intermediate/fr"/>
               </general>
               <identification-elements>
                  <element name="station_name" uom="unitless" value="Aéroport d'Assiniboia"/>
                  <element name="latitude" uom="degree" value="49.729192"/>
                  <element name="longitude" uom="degree" value="-105.946114"/>
                  <element name="transport_canada_id" uom="unitless" value="WJI"/>
                  <element name="observation_date_utc" uom="unitless" value="2018-10-12T23:45:00.000Z"/>
                  <element name="observation_date_local_time" uom="unitless" value="2018-10-12T17:45:00.000 CST"/>
                  <element name="climate_station_number" uom="unitless" value="4020286"/>
                  <element name="wmo_station_number" uom="unitless" value="71487"/>
               </identification-elements>
            </set>
         </om:metadata>
         <om:samplingTime>
            <gml:TimeInstant>
               <gml:timePosition>2018-10-12T23:00:00.000Z</gml:timePosition>
            </gml:TimeInstant>
         </om:samplingTime>
         <om:resultTime>
            <gml:TimeInstant>
               <gml:timePosition>2018-10-12T23:00:00.000Z</gml:timePosition>
            </gml:TimeInstant>
         </om:resultTime>
         <om:procedure xlink:href="msc/observation/atmospheric/surface_weather/wxo_dd_today_summary-1.0-ascii/product-wxo_xml-1.0/20181012234500000/sk/intermediate/fr"/>
         <om:observedProperty gml:remoteSchema="/schema/point-observation/2.0.xsd"/>
         <om:featureOfInterest>
            <gml:FeatureCollection>
               <gml:location>
                  <gml:Point>
                     <gml:pos>49.729192 -105.946114</gml:pos>
                  </gml:Point>
               </gml:location>
            </gml:FeatureCollection>
         </om:featureOfInterest>
         <om:result>
            <elements>
               <element name="air_temperature_today_high" uom="Celsius" value="11.9"/>
               <element name="air_temperature_today_low" uom="Celsius" value="-5.5"/>
               <element name="wind_gust_speed" uom="km/h" value=""/>
               <element name="wind_direction" uom="code" value="WSW"/>
               <element name="record_high_temperature" uom="Celsius" value="17.4">
                  <qualifier name="record_high_temperature_year" uom="unitless" value="2012"/>
               </element>
               <element name="record_low_temperature" uom="Celsius" value="-9.2">
                  <qualifier name="record_low_temperature_year" uom="unitless" value="2006"/>
               </element>
               <element name="record_rain_amount" uom="mm" value="6.8">
                  <qualifier name="record_rain_year" uom="unitless" value="1997"/>
               </element>
               <element name="record_snow_amount" uom="cm" value="0.0">
                  <qualifier name="record_snow_year" uom="unitless" value="1996"/>
               </element>
               <element name="temperature_data_since_year" uom="unitless" value="1993"/>
               <element name="precipitation_data_since_year" uom="unitless" value="1993"/>
            </elements>
         </om:result>
      </om:Observation>
   </om:member>
<om:member>
   <om:Observation>
      <om:metadata>
         <set>
            <general>
               <author build="build.4083" name="MSC-DMS-PG-WXO-Summary" version="3.1"/>
               <dataset name="mscobservation/atmospheric/surface_weather/wxo_dd_hour_summary-1.0-ascii/"/>
               <phase name="product-wxo_xml-1.0/"/>
               <id xlink:href="/data/msc/observation/atmospheric/surface_weather/wxo_dd_hour_summary-1.0-ascii/product-wxo_xml-1.0/20180921014500000/ns/intermediate/en"/>
               <parent xlink:href="/data/msc/observation/atmospheric/surface_weather/wxo_dd_hour_summary-1.0-ascii/product-wxo_xml-1.0/20180921014500000/ns/intermediate/en"/>
            </general>
            <identification-elements>
               <element name="station_name" uom="unitless" value="Bedford Range"/>
               <element name="latitude" uom="degree" value="44.745858"/>
               <element name="longitude" uom="degree" value="-63.661689"/>
               <element name="transport_canada_id" uom="unitless" value="ABR"/>
               <element name="observation_date_utc" uom="unitless" value="2018-09-21T01:30:00.000Z"/>
               <element name="observation_date_local_time" uom="unitless" value="2018-09-20T22:30:00.000 ADT"/>
               <element name="climate_station_number" uom="unitless" value="8200574"/>
               <element name="wmo_station_number" uom="unitless" value="71325"/>
            </identification-elements>
         </set>
      </om:metadata>
      <om:samplingTime>
         <gml:TimeInstant>
            <gml:timePosition>2018-09-21T01:00:00.000Z</gml:timePosition>
         </gml:TimeInstant>
      </om:samplingTime>
      <om:resultTime>
         <gml:TimeInstant>
            <gml:timePosition>2018-09-21T01:00:00.000Z</gml:timePosition>
         </gml:TimeInstant>
      </om:resultTime>
      <om:procedure xlink:href="msc/observation/atmospheric/surface_weather/wxo_dd_hour_summary-1.0-ascii/product-wxo_xml-1.0/20180921014500000/ns/intermediate/en"/>
      <om:observedProperty gml:remoteSchema="/schema/point-observation/2.0.xsd"/>
      <om:featureOfInterest>
         <gml:FeatureCollection>
            <gml:location>
               <gml:Point>
                  <gml:pos>44.745858 -63.661689</gml:pos>
               </gml:Point>
            </gml:location>
         </gml:FeatureCollection>
      </om:featureOfInterest>
      <om:result>
         <elements>
            <element name="present_weather" uom="code" value="Partly Cloudy"/>
            <element name="mean_sea_level" uom="kPa" value="102.6"/>
            <element name="tendency_amount" uom="kPa" value="0.20"/>
            <element name="tendency_characteristic" uom="code" value="rising"/>
            <element name="horizontal_visibility" uom="km" value="24.1"/>
            <element name="air_temperature" uom="Celsius" value="10.7"/>
            <element name="dew_point" uom="Celsius" value="8.7"/>
            <element name="relative_humidity" uom="percent" value="87"/>
            <element name="wind_speed" uom="km/h" value="6"/>
            <element name="wind_direction" uom="code" value="NW"/>
            <element name="wind_gust_speed" uom="km/h" value="15"/>
            <element name="total_cloud_cover" uom="code" value="5"/>
            <element name="wind_chill" uom="unitless" value=""/>
            <element name="humidex" uom="unitless" value=""/>
         </elements>
      </om:result>
   </om:Observation>
</om:member>
</om:ObservationCollection> 



  • No labels

Disclaimer