You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »

Contents

NetCDF formats in general

The NetCDF file format is very flexible in the ways that it allows the data to be stored in the file. Unfortunately this means that NetCDF files come in many different formats and the way the variables and dimensions (data structure) are organised in the file can be completely different for different NetCDF files. Therefore it cannot be expected that the FEWS NetCDF import will be able to import every single netcdf file that's out there. Because it is just not possible to anticipate all the possible data structures that people can come up with.

For this reason there exist several conventions that describe standard ways of organising the data in NetCDF files. The FEWS NetCDF import is based on the CF-1.6 conventions. This means that if a NetCDF file is in a format that is fully compatible with the CF-1.6 conventions, then there is a good chance that FEWS will be able to import the data from the NetCDF file. However there are a few exceptions, which are listed on this page. For instance FEWS can only import data that depends on a time dimension (time series), while the CF-1.6 conventions also describe formats for data that does not depend on time.

Furthermore there are some exceptions where FEWS can import NetCDF data, even though it does not comply with the CF-1.6 conventions. This can be due to historic reasons. Or this can be due to the fact that some data formats are not (yet) part of the CF-1.6 conventions (e.g. spectral data), in which case these exceptions are also described on this page.

NetCDF formats that can be imported in Delft-FEWS

TODO work in progress

A first check to determine whether a given NetCDF file can be imported in FEWS is to use an online CF compliance checker, see e.g. http://cfconventions.org/compliance-checker.html. If the NetCDF file is not valid according to the CF compliance checker, then the NetCDF file can probably not be imported in FEWS.

Checklist to create NetCDF data in the format of the Netcdf CF-1.6 conventions

TODO work in progress

This checklist focusses on the points that are important to be able to import the netcdf data into Delft-FEWS. The full NetCDF CF-1.6 conventions can be found on http://cfconventions.org/1.6.html.

Before creating/adapting any code/script to produce NetCDF files, it can be very useful to read the following webpages:

Furthermore some links with reference documentation about NetCDF files:

Grid data

  1. all data and coordinate variables should have a unique variable name and the following attributes:
    units (required)
    standard_name (required)
    long_name (recommended)
    _FillValue and/or missing_value (recommended)
    scale_factor (optional)
    add_offset (optional)
    Also see http://cfconventions.org/1.6.html#units, http://cfconventions.org/1.6.html#long-name and http://cfconventions.org/1.6.html#standard-name.
  2. the standard_name attributes should comply to the list with standard names, see http://cfconventions.org/26.html.
  3. a data variable for a 2D grid must depend on three dimensions, representing time and two spatial dimensions (in the order time, y, x). The dimension names can be anything. For the time dimension there must be a corresponding time coordinate variable with the same name. If the grid is regular or rectangular, then for each spatial dimension there must be a corresponding coordinate variable with the same name. See http://cfconventions.org/1.6.html#idp5553648. If the grid is curvilinear, then the y and x coordinate variables are both two-dimensional and depend on both spatial dimensions. In this case the data variable must have an attribute "coordinates" that refers to the y and x coordinate variables (in that order), e.g. coordinates="y x". See http://cfconventions.org/1.6.html#idp5559280.
  4. if the data has a forecast reference time/analysis time/base time, then this time must be stored in a separate coordinate variable that contains only one value, which is the forecast reference time. This variable must have standard_name="forecast_reference_time". The name of this variable must be present in the coordinates attribute of the data variable, e.g. coordinates="analysis_time" or coordinates="y x analysis_time". If needed, different data variables can point to different forecast reference time variables. See http://cfconventions.org/1.6.html#scalar-coordinate-variables
  5. the time coordinate variable must either have an attribute "units" with a valid unit of time and/or an attribute "axis" with value "T".
  6. the time and spatial coordinate variables must have values either in ascending or in descending order.

Error messages from the Delft-FEWS NetCDF import

If you get the error "invalid URL" then either the URL is invalid, or the OpenDAP server does not return any data. The latter either means that the OpenDAP server itself does not work correctly, or that the user does not have access rights to download the file on the OpenDAP server. This can be easily verified by following these steps:

  1. copy the URL of the netcdf file or catalog from the FEWS configuration or from the error message.
  2. if the URL ends with .nc, then append .html to it, so that it ends with .nc.html
  3. browse to the resulting URL in a browser, e.g. Mozilla Firefox.
  4. if the browser gives an error, then there is a problem with the URL, the OpenDAP server or user access rights to the server. In this case please refer to the administrator of the OpenDAP server.

Related modules and documentation

Internal

External

  • No labels

Disclaimer