Page History

...

A first check to determine whether a given NetCDF file can be imported in FEWS is to use an online CF compliance checker, see e.g. http://cfconventions.org/compliance-checker.html or https://compliance.ioos.us/index.html. If the NetCDF file is not valid according to the CF compliance checker, then the NetCDF file can probably not be imported in FEWS. In this case please contact the data supplier that created the NetCDF file and ask them to make the NetCDF file CF-1.6 compliant. If the file is not CF-1.6 compliant then there is no reason to contact the FEWS support team.

Scalar Time Series Example

Netcdf file: scalar_time_series.nc

Text version: scalar_time_series.txt

Regular Grid Time Series Example

Netcdf file:  regular_grid_time_series.nc

Text version: regular_grid_time_series.txt

Also note that there can be slight differences between importing a grid from a netCDF file into Delft-FEWS and into other viewers. For example a netCDF file with a rotated pole grid that contains both one-dimensional coordinate variables in the rotated coordinate system and two-dimensional coordinate variables in the WGS1984 system. Panoply will read from this file the two-dimensional coordinate variables. Delft-FEWS can read from this file either the two-dimensional coordinate variables (irregular grid definition) or the one-dimensional coordinate variables (regular grid definition, i.e. more efficient, better performance). If the one-dimensional coordinate variables or grid mapping in this netCDF file are not CF compliant, then this might result in problems in Delft-FEWS, which are not visible in Panoply.

Scalar Time Series Curvilinear Grid Time Series Example

Netcdf file: curvilinear_gridscalar_time_series.nc

Text version: curvilinear_gridscalar_time_series.txt

Unstructured Regular Grid (UGRID) Time Series

For an example of data in UGRID format, see http://ugrid-conventions.github.io/ugrid-conventions/

To import unstructured grid data in Delft-FEWS you need to add an irregular grid definition to the grids.xml configuration file for your data. For more information see 06 Grids.

Track Time Series (single trajectory per file)

For featureType "point" all points in the file are imported together to form a single track. For featureType "trajectory" the parser supports only a single track per file, i.e. variables in the NetCDF file must be one dimensional and only have a time dimension. For an example of the track data format that can be imported, see http://cfconventions.org/cf-conventions/cf-conventions.html#_single_trajectory

NetCDF kickstarter tool

Example

Netcdf file:  regular_grid_time_series.nc

Text version: regular_grid_time_series.txt

Curvilinear Grid Time Series Example

Netcdf file: curvilinear_grid_time_series.nc

Text version: curvilinear_grid_time_series.txt

Unstructured Grid (UGRID) Time Series

For an example of data in UGRID format, see http://ugrid-conventions.github.io/ugrid-conventions/

To import unstructured grid data in Delft-FEWS you need to add an irregular grid definition to the grids.xml configuration file for your data. For more information see 06 Grids#06Grids-UsingaUGRIDNetCDFfileasunstructuredgriddefinition

Track Time Series (single trajectory per file)

For featureType "point" all points in the file are imported together to form a single track. For featureType "trajectory" the parser supports only a single track per file, i.e. variables in the NetCDF file must be one dimensional and only have a time dimension. For an example of the track data format that can be imported, see http://cfconventions.org/cf-conventions/cf-conventions.html#_single_trajectory

NetCDF kickstarter tool

The netCDF kickstarter is a very useful tool for creating template netCDF files and is available on http://zandmotor.citg.tudelft.nl/The netCDF kickstarter is a very useful tool for creating template netCDF files and is available on http://zandmotor.citg.tudelft.nl/netcdfkickstarter/ This tool can be used to create template netCDF files or to create scripts that generate template netCDF files. In this tool the user can specify what data variables the netCDF file should contain and the netCDF kickstarter will create a template netCDF file that contains the corresponding metadata variables and empty data variables, fully compliant according to the CF conventions. Then the user only needs to add the actual data values to this template file in order to create a CF compliant netCDF file. The kickstarter can also generate a script (in one of several programming languages) to create a template netCDF file that contains the metadata variables and empty data variables specified by the user. Then the user only needs to extend this script with commands to write the actual data values to the netCDF file.

...

The global attribute "Conventions" must be specified in the netcdf file (e.g. Conventions="CF-1.6"). If this attribute is not present or not valid, then the NetCDF Java library assumes a default convention (different from CF), which can cause unexpected behaviour in the NetCDF import in Delft-FEWS. Also see http://cfconventions.org/Data/cf-conventions/cf-conventions-1.6/build/cf-conventions.html#identification-of-conventions.

Grid data

Gridded data

See for references to the cf-conventions website below.

1. all variables (data and coordinate) all data and coordinate variables must have a unique variable name and the following attributes:
   1. units (required, see units link below)
   2. standard_name (required, see standard_name link below) - should comply with standard names list
   3. long_name (recommended, see long_name links below)
   4. _FillValue and/or missing_value (required if
   there are
   1. data contains missing values
   in the data
   1. )
   2. scale_factor (optional to compact data)
   3. add_offset (optional to compact data)
   Also see http://cfconventions.org/Data/cf-conventions/cf-conventions-1.6/build/cf-conventions.html#units, http://cfconventions.org/Data/cf-conventions/cf-conventions-1.6/build/cf-conventions.html#long-name and http://cfconventions.org/Data/cf-conventions/cf-conventions-1.6/build/cf-conventions.html#standard-name.
2. the standard_name attributes should comply to the list with standard names, see http://cfconventions.org/26.html.
3. a data variable for a 2D grid must depend on three dimensions, representing time and two spatial dimensions (in the order time, y, x). The names of the dimensions can be anything. For the time dimension there must be a corresponding 1D time coordinate variable with the same name. If the grid is regular or rectangular, then for each spatial dimension there must be a corresponding 1D coordinate variable with the same name. See http://cfconventions.org/Data/cf-conventions/cf-conventions-1.6/build/cf-conventions.html#idp5553648. If the grid is curvilinear, then the y and x coordinate variables are both 2D and depend on both spatial dimensions. In this case the data variable must have an attribute "coordinates" that refers to the y and x coordinate variables (in that order), e.g. coordinates="y x". See http://cfconventions.org/Data/cf-conventions/cf-conventions-1.6/build/cf-conventions.html#idp5559280. Each coordinate variable must have an attribute "axis" that indicates which axis is defined by that coordinate variable. For the time coordinate variable the axis must be "T", for the x coordinate variable the axis must be "X", for the y coordinate variable the axis must be "Y".
if the data has a forecast reference time/analysis time/base time, then this time must be stored in a separate coordinate variable that contains only one value, which is the forecast reference time. This variable must have standard_name="forecast_reference_time". The name of this variable must be present in the coordinates attribute of the data variable, e.g. coordinates="analysis_time" or coordinates="y x analysis_time". If needed, different data variables can point to different forecast reference time variables. See
4. Data variable for a 2D grid must depend on three dimensions, representing time and two spatial dimensions (in either the order time, y, x or time, x, y).  The names of the dimensions can be anything.
   1. Time dimension must have a corresponding 1D time coordinate variable with the same name.
   2. Spatial dimensions, for
      1. regular or rectangular grids - each spatial dimension must have a corresponding 1D coordinate variable with the same name (see coordinates link below).
      2. curvilinear grids -  both spatial dimensions are 2D and depend on both spatial dimensions.
         1. The data variable must have an attribute "coordinates" that refers to the y and x coordinate variables (in that order), e.g. coordinates="y x"
         2. Each coordinate variable must have an attribute "axis" that indicates which axis is defined by that coordinate variable.
            1. Time coordinate variable requires axis "T"
            2. x coordinate variable requires axis "X"
            3. y coordinate variable requires axis "Y"
            4. (optional) analysis_time coordinate, see below
5. Time coordinate variable must have the attributes
   1. "units" with a valid unit of time - e.g. units="minutes since 1970-01-01 00:00:00.0 +0000" (time coordinate link below)
   2. "axis" with value "T"
6. Time and spatial coordinate variables must have values either in ascending or in descending order.
7. Forecast reference time/analysis time/base time (optional), must be stored in a separate coordinate variable that
   1. contains only one value: the forecast reference time, and
   2. standard_name="forecast_reference_time"

The name of this variable must be present in the coordinates attribute of the data variable, e.g. coordinates="analysis_time" or coordinates="y x analysis_time".
If needed, different data variables can point to different forecast reference time variables (see scalar coordinate link below).

...

...

...

...

...

Compacted data

Netcf data variables with attributes scale_factor and add_offset will be treated as compacted data and the values of these attributes will be used to uncompact the values of this variable.

How to import a netCDF file into Delft-FEWS

To import data from a netCDF file into Delft-FEWS you must first create the relevant configuration.

Import configuration

To import data from a netCDF file into Delft-FEWS, either create a time series import run (see 03 Import Module) or an importNetcdfActivity in a general adapter (see 05 General Adapter Module#05GeneralAdapterModule-importNetcdfActivity). For an import run, use one of the following importTypes, depending on the type of data that you want to import:

NETCDF-CF_TIMESERIES, NETCDF-CF_GRID, NETCDF-CF_PROFILE or NETCDF-CF_TRAJECTORY.

Do not use the obsolete importType "NetcdfGridDataset". Only use importType "NetcdfGridDataset" for grid NetCDF files that contain ensembles or qualifiers

Compacted data

Netcf data variables with attributes scale_factor and add_offset will be treated as compacted data and the values of these attributes will be used to uncompact the values of this variable.

How to import a netCDF file into Delft-FEWS

To import data from a netCDF file into Delft-FEWS you must first create the relevant configuration.

Import configuration

To import data from a netCDF file into Delft-FEWS, either create an import run (see 03 Import Module) or a netcdfImportActivity in a general adapter (see 05 General Adapter Module#05GeneralAdapterModule-importNetcdfActivity).

To import scalar time series you also need to configure the corresponding internal locations (see 01 Locations) and an id mapping (see 01 ID Mapping).

...

1. In the grids.xml config file create an empty regular grid definition.
2. Open the netCDF file in your favorite viewer and find the details of the coordinate system in which the grid is a regular grid. This coordinate system is usually described in a grid mapping variable, which has an attribute called "grid_mapping_name". In your grid definition choose the geoDatum or grid projection that corresponds to this coordinate system.
3. In the netCDF file find the variables that contain the coordinates of the grid cells in the coordinate system in which the grid is regular. These are usually the variables with one-dimensional variables that usually have standard names like "projection_x_coordinate" and "projection_y_coordinate" or "grid_longitude" and "grid_latitude". According to the CF conventions these coordinates are cell centers. Find the lowest x coordinate and use that as firstCellCenter x in your grid definition. Find the highest y coordinate and use that as firstCellCenter y in your grid definition.
4. In the netCDF file look at the spacings between the x and y coordinates. If these spacings are constant, then use them as xCellSize and yCellSize coordinates. If these spacings are constant, then use them as xCellSize and yCellSize in your grid definition.
5. In the netCDF file find the sizes of the dimensions of the coordinate variables. Use these as the number of rows and columns in your grid definition.

If, after following these instructions carefully, the netCDF file is still not imported correctly into Delft-FEWS, then the file is probably not valid according to the NetCDF-CF conventions. In that case try to make the netCDF file

...

CF compliant. As a last resort, you can try to use the obsolete option gridStartPoint (to swap the grid in the north-south direction) or the obsolete importType "NetcdfGridDataset", but only as a last resort.

Error messages from the Delft-FEWS NetCDF import

...