NETCDF-CF_GRID

Overview

This import is available in DELFT-FEWS versions after 28-10-2009 (FEWS version 2009.02)

Imports grid time series data from NetCDF files which comply to the CF standard. More information about the cf standards can be found at http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html. In DELFT-FEWS versions 2015.02 and later this import type can also be used to import data from NetCDF files that comply to the UGRID 0.9 conventions (see https://github.com/ugrid-conventions/ugrid-conventions). For details about the exact formats of NetCDF files that can be imported in FEWS, see NetCDF formats that can be imported in Delft-FEWS.

See also the following two other types of NetCDF-CF imports that are available:

Time series (NETCDF-CF_TIMESERIES)
Profiles (NETCDF-CF_PROFILE)

In DELFT-FEWS versions 2011.02 and later this import type can also be used to import data using OPeNDAP, see Import data using OPeNDAP.

Note that for the import to correctly import (irregular) grids, the grid must be configured in grids.xml. There is a tool available to obtain the grid configuration from the NetCDF file in the F12 debug menu (located under "Convert"). Since 2017.02, the import will automatically detect the irregular grid for temporary time series (<timeSeriesType> set to "temporary" or "temporary external forecast"). The imported grid is then temporarily stored along with the data, and can be used within the same workflow, which could for example use a spatial interpolation to map the imported data to a configured grid. Note that this functionality is only available for temporary time series since storing an irregular grid definition can easily take up just as much disk space as the data itself, and is therefore undesirable for data that is to be stored for a longer time period. Also note that this functionality is only available for grids that do not have z-values / z-layers.

Import Configuration

An example of the NETCDF-CF_GRID import will be given here.

From 2021.01 it is possible to import NETCDF attributes as timeSeriesProperties into FEWS. This can be configured in the properties (see example below). Using the example config, the value of the NETCDF attribute ‘systemid’ will be imported as a timeSeriesProperty ‘rivieren2’.

Since 2021.02 it is also possible to export timeSeriesProperties as NETCDF attributes. You can find a config example here: https://publicwiki.deltares.nl/display/FEWSDOC/NETCDF-CF_GRID+Export

ImportNetcdf_Grid 1.00 default.xml

<?xml version="1.0" encoding="UTF-8"?>
<timeSeriesImportRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/timeSeriesImportRun.xsd">
	<import>
		<general>
			<importType>NETCDF-CF_GRID</importType>
			<folder>$IMPORT_FOLDER$/NETCDF</folder>
			<failedFolder>$IMPORT_FAILED_FOLDER$</failedFolder>
			<backupFolder>$IMPORT_BACKUP_FOLDER$</backupFolder>
			<idMapId>IdImportNetCDF</idMapId>
			<unitConversionsId>ImportUnitConversions</unitConversionsId>
		</general>
		<properties>
			<string key="netcdfattribute:taskrunid" value='timeseriesproperty:rivieren1'/>
			<string key="netcdfattribute:systemid" value='timeseriesproperty:rivieren2'/>
		</properties>   
		<timeSeriesSet>
			<moduleInstanceId>ImportNetcdf_Grid</moduleInstanceId>
			<valueType>grid</valueType>
			<parameterId>Snelheid.u.F0</parameterId>
			<locationId>hmcn_zeedelta</locationId>
			<timeSeriesType>external forecasting</timeSeriesType>
			<timeStep unit="minute" multiplier="30"/>
			<readWriteMode>add originals</readWriteMode>
			<synchLevel>6</synchLevel>
		</timeSeriesSet>
	</import>
</timeSeriesImportRun>

Import of metadata and (OpenDAP) grid references (since 2019.02)

It is possible to import gigabytes of grids in a fraction of a second. To achieve this only references to grid data in NetCDF files, OPeNDAP are imported. The grids are streamed from the OPenDAP service or NetCDF file to the wms service, oc, fss. The central database can be released of big gridded model results by exporting the model results directly to OPeNDAP and import the references back in the same workflow. This works even without any configuration changes in other areas as long the OPenDAP service is reachable. When a netcdf file is created in a workflow an external process is often moving this file to the final opendap location later on. This future file url can be configured (since 2021.01). The data becomes visible in the OC as soon the netcdf file is moved to the final location.

ImportNetcdf_Grid 1.00 default.xml

<?xml version="1.0" encoding="UTF-8"?>
<timeSeriesImportRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/timeSeriesImportRun.xsd">
   <import>
      <general>
			<importType>NETCDF-CF_GRID_REFERENCES</importType>
			<folder>$IMPORT_FOLDER$/NETCDF</folder>
			<idMapId>IdImportNetCDF</idMapId>
			<unitConversionsId>ImportUnitConversions</unitConversionsId>
      </general>
      <properties>
         <string key="futureFolderUrl" value="http..."/>
      </properties>
      <timeSeriesSet>
         <moduleInstanceId>GridImportNetcdf</moduleInstanceId>
         <valueType>grid</valueType>
         <parameterId>H.sim</parameterId>
         <locationId>locA</locationId>
         <timeSeriesType>simulated forecasting</timeSeriesType>
         <timeStep unit="nonequidistant"/>
                  <relativeViewPeriod unit="day" start="0" end="36524"/>
         <readWriteMode>add originals</readWriteMode>
      </timeSeriesSet>
   </import>
</timeSeriesImportRun>

Id Map Configuration

An example of the IdMapping used for the NETCDF-CF_GRID import is shown below.

IdImportNetCDF 1.00 default.xml

<idMap version="1.1" xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/idMap.xsd">
	<map internalParameter="Snelheid.u.F0" internalLocation="hmcn_zeedelta" externalParameter="velocity" externalLocation="hmcn_zeedelta"/>
</idMap>

Grids Configuration

When importing grids in the FEWS database it may be required to configure the grid characteristics in the grids.XML file. The grid characteristics must be similar to the grid imported from the NetCDF file.

Grids 1.00 default.xml

	<regular locationId="hmcn_zeedelta">
		<description>HMCN Zeedelta Model</description>
		<rows>101</rows>
		<columns>101</columns>
		<geoDatum>Rijks Driehoekstelsel</geoDatum>
		<firstCellCenter>
			<x>54000</x>
			<y>449000</y>
		</firstCellCenter>
		<xCellSize>130</xCellSize>
		<yCellSize>70</yCellSize>
	</regular>

Import of Waterwatch NetCDF data

For the import of Waterwatch NetCDF data a special NetCDF import type can be used "NETCDF-CF_GRID-NW". This import type has been added in July 2011 to the FEWS 2010.01 and 2011.01 builds, and will be available in the 2011.02 build. Waterwatch NetCDF data for Dutch waterboards requires the Transverse Mercator projection to be used. This regular grid projection has been added to the FEWS code in October 2011.