You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Next »

Overview

Imports grid time series data from CMEMS NetCDF files. Data is imported per z-layer, cut into smaller time steps to avoid getting "data file too big" error.

Import Configuration

An example of the CMEMS import will be given here. 

The type of import should be configured using the DATA_TYPE property:

  • DATA_TYPE should be set to "HOURLY" for hourly data.
  • DATA_TYPE should be set to "DAILY" for daily data.
  • DATA_TYPE should be set to "MONTHLY" for monthly data.

Properties "product" and "service" are compulsory fields. Please fill them in exactly the same as it should b in the request url. (See the examples bellow.)

FEWS will automatically add x_lo, x_hi, y_lo, y_hi, z_lo, z_hi values to the url. The data will be extracted from the grid configuration.  Please make sure that the configured grid layer depth (z-layer) matches exactly the z_lo layer values the library provides. If it is not an exact match the data request will be refuesd.  The z_hi values will be calculated by fews (z_lo+0.01). Do not use more than two decimals to configure the layers.

Z-layer configuration example
LAYER,Z
1,-0.49
8,-9.57
18,-47.37
23,-109.72
32,-541.08
36,-1062.43
46,-3992.48

The parameters needed will be added to the request url by FEWS. It uses the external paramter ID for this, so please make sure it matches the parameter that should be in the url exactly.

 

This example is configured to import 2D Hourly grids for the last 24 hours:

ImportNetcdf_Grid 1.00 default.xml
<?xml version="1.0" encoding="UTF-8"?>
<timeSeriesImportRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/timeSeriesImportRun.xsd">

	<import>
		<general>
			<importType>Cmems</importType>		
			<serverUrl>http://nrtcmems.mercator-ocean.fr/mis-gateway-servlet/Motu?action=productdownload</serverUrl>
			<user>USERNAME</user>
			<password>PASSWORD</password>
			<relativeViewPeriod unit="hour" start="-24" end="0" startOverrulable="true"/>
			<idMapId>IdImportCmems</idMapId>
			<unitConversionsId>ImportUnitConversions</unitConversionsId>
			<expiryTime unit="week" multiplier="500"/>
		</general>
		<properties>
			<string key="DATA_TYPE" value="HOURLY"></string>
			<string key="TICKET_URL" value="https://cmems-cas.cls.fr/cas/v1/tickets"></string>
			<string key="product" value="global-analysis-forecast-phy-001-024-hourly-t-u-v-ssh"/>
			<string key="service" value="http://purl.org/myocean/ontology/service/database%23GLOBAL_ANALYSIS_FORECAST_PHY_001_024-TDS"/>
		</properties>
		<timeSeriesSet>
			<moduleInstanceId>Import_E2O_Server</moduleInstanceId>
			<valueType>grid</valueType>
			<parameterId>Wind.u</parameterId>
			<locationId>$LOOP_LOCATION_ID$</locationId>
			<timeSeriesType>external historical</timeSeriesType>
		    <timeStep unit="nonequidistant"/>
			<readWriteMode>add originals</readWriteMode>
		</timeSeriesSet>		
	</import>
</timeSeriesImportRun></timeSer

 
The following example is configured for importing daily 3D grids:
<import>
		<general>
			<importType>CMEMS</importType>
			<serverUrl>http://nrt.cmems-du.eu/motu-web/Motu?action=productdownload</serverUrl>
			<connectionTimeOutMillis>300000</connectionTimeOutMillis>
			<user>$CMEMS_USER$</user>
			<password>$CMEMS_PASSWORD$</password>
			<relativeViewPeriod unit="day" start="-2" end="10" startOverrulable="true"/>
			<idMapId>IdImport_CMEMS</idMapId>
			<maximumVerticalSnapDistance>0.1</maximumVerticalSnapDistance>
			<unitConversionsId>ImportUnitConversions</unitConversionsId>
			<dataFeedId>CMEMS_3D</dataFeedId>
		</general>
		<properties>
			<string key="DATA_TYPE" value="DAILY"/>
			<string key="TICKET_URL" value="https://cmems-cas.cls.fr/cas/v1/tickets"/>
			<string key="product" value="global-analysis-forecast-phy-001-024"/>
			<string key="service" value="GLOBAL_ANALYSIS_FORECAST_PHY_001_024-TDS"/>
		</properties>
		<timeSeriesSet>
			<moduleInstanceId>Import_CMEMS_3D</moduleInstanceId>
			<valueType>grid</valueType>
			<parameterId>T.simulated</parameterId>
			<locationId>$LOOP_LOCATION_ID$</locationId>
			<timeSeriesType>external historical</timeSeriesType>
			<timeStep unit="nonequidistant"/>
			<readWriteMode>add originals</readWriteMode>
			<expiryTime unit="day" multiplier="$EXPIRY_DAYS_EXTERNAL_FORECAST_GRID$"/>
		</timeSeriesSet>
	</import>


 
The following example is configured for importing monthly 3D grids:
	<import>
		<general>
			<importType>Cmems</importType>
			<serverUrl>http://nrtcmems.mercator-ocean.fr/mis-gateway-servlet/Motu?action=productdownload</serverUrl>
            <connectionTimeOutMillis>300000</connectionTimeOutMillis>
			<user>$CMEMS_USER$</user>
			<password>$CMEMS_PASSWORD$</password>
			<relativeViewPeriod unit="day" start="-1" end="1" startOverrulable="true"/>
			<idMapId>IdImport_CMEMS</idMapId>
			<unitConversionsId>ImportUnitConversions</unitConversionsId>
			<expiryTime unit="day" multiplier="356"/>
		</general>
		<properties>
			<string key="TICKET_URL" value="https://cmems-cas.cls.fr/cas/v1/tickets"/>
			<string key="DATA_TYPE" value="MONTHLY"/>
			<string key="product" value="global-analysis-forecast-phy-001-024-monthly"/>
			<string key="service" value="http://purl.org/myocean/ontology/service/database%23GLOBAL_ANALYSIS_FORECAST_PHY_001_024-TDS"/>
		</properties>
		<timeSeriesSet>
			<moduleInstanceId>Import_CMEMS</moduleInstanceId>
			<valueType>grid</valueType>
			<parameterId>T.simulated.CMEMS</parameterId>
			<locationId>$LOOP_LOCATION_ID$</locationId>
			<timeSeriesType>external forecasting</timeSeriesType>
			<timeStep unit="nonequidistant"/>
			<readWriteMode>add originals</readWriteMode>
		</timeSeriesSet>
	</import>

 

Id Map Configuration

An example of the IdMapping used for the NETCDF-CF_GRID import is shown below.

IdImportNetCDF 1.00 default.xml
<idMap version="1.1" xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/idMap.xsd">
	<parameter internal="Wind.u" external="uo"/>
	<parameter internal="Wind.v" external="uv"/>
</idMap>

Grids Configuration

When importing grids in the FEWS database it is required to configure the grid characteristics in the grids.XML file. The grid characteristics must be similar to the grid imported from the NetCDF file.

Grids 1.00 default.xml
	<regular locationId="CMEMS_2D">
		<rows>241</rows>
		<columns>265</columns>
		<geoDatum>WGS 1984</geoDatum>
		<firstCellCenter>
			<x>43</x>
			<y>28</y>
		</firstCellCenter>
		<xCellSize>0.0833</xCellSize>
		<yCellSize>0.0833</yCellSize>
	</regular>
  • No labels

Disclaimer