Function: |
Import NetCDF data from an OPeNDAP server directly into Delft-FEWS |
|---|---|
Where to Use? |
This can be used for importing NetCDF data into the Delft-FEWS system. |
Why to Use? |
The advantage of importing NetCDF data directly from an OPeNDAP server, as opposed to importing local NetCDF files, is that the files do not have to be stored locally. Furthermore if only part of a file is needed, then only that part will be downloaded instead of the entire file. This can save a lot of network bandwidth (i.e. time) for large data files. |
Preconditions: |
The data to import needs to be available on an OPeNDAP server that is accessible by the Delft-FEWS system. |
Outcome(s): |
The imported data will be stored in the Delft-FEWS dataStore. |
Available since: |
DelftFEWS201102 |
Contents
Overview
OPeNDAP (Open-source Project for a Network Data Access Protocol) can be used to import NetCDF data from an OPeNDAP server directly into Delft-FEWS. For more information on OPeNDAP see http://opendap.org/. Currently only import of (zipped) NetCDF files from an OPeNDAP server is supported. Three types of NetCDF data can be imported: grid time series, scalar time series and profile time series. For more information on these specific import types see their individual pages: NETCDF-CF_GRID, NETCDF-CF_TIMESERIES and NETCDF-CF_PROFILE.
How to import data from an OPeNDAP server
Import data from a single file
Import data from a catalog
Import data for a given period of time
To import only data for a given period of time specify either a relative period or an absolute period in the import configuration file. See relativeViewPeriod, startDateTime and endDateTime for more information. The import will first search the metadata of each file that needs to be imported from the OPeNDAP server. Then for each file that contains data within the specified period, only the data within the specified period will be imported.
This can be used to import only the relevant data if only data for a given period is needed, which can save a lot of time. However, for this to work the import still needs to search through all the metadata of the file(s) to be imported. So for large catalogs that contain a lot of files, it can still take a lot of time for the import to download all the required metadata from the OPeNDAP server.
Import data for a given sub grid
Configuration
Config Example |
|
|---|
description of the config example.
Known issues
Export of data
It is not possible to export data directly using the OPeNDAP protocol, since the OPeNDAP protocol only supports reading data from the server. If it is required to export data from Delft-FEWS and make it available on an OPeNDAP server, then this can be done in two steps:
1. setup a separate OPeNDAP server that points to a given storage location. For instance a THREDDS server, which is relatively easy to install. The OPeNDAP server picks up any (NetCDF) files that are stored in the storage location and makes these available for download using OPeNDAP.
2. export the data to a NetCDF file using a Delft-FEWS export run. Export of grid time series, scalar time series and profile time series is supported (respectively export types NETCDF-CF_GRID, NETCDF-CF_TIMESERIES and NETCDF-CF_PROFILE). Set the output folder for the export run to the given storage location. That way the exported data will automatically be picked up by the OPeNDAP server.
Related modules and documentation
Import Module configuration options, NETCDF-CF_GRID, NETCDF-CF_TIMESERIES, NETCDF-CF_PROFILE.
