Accessing netCDF/OPeNDAP data with Python
Get PythonXY, add the netCDF4 package (PythonXY installer). Not all netCDF4 packages have been compiled with opendap yet. Therefore, manually also install the pydap package with easy_install.
python # start python in DOS shell to install easy_install >>> from ez_setup import use_setuptools >>> use_setuptools() >>> exit() # now easy_install works outside python easy_install Paste # sometimes Paste is problematic, so let's install itmanually easy_install Pydap
OpenEarthTools provides a module opendap.py
that makes pydap quack like netCDF4 (repos, manual download) so you can talk directly to opendap data via the web. Now execute the following Python lines, or download the full example code (repos,manual download):
#!/usr/bin/env python # Read data from an opendap server import netCDF4, pydap, urllib import pylab, matplotlib import numpy as np from opendap import opendap # OpenEarthTools module, see above that makes pypdap quack like netCDF4
- Go to an OPeNDAP server (e.g. http://opendap.deltares.nl) and pick a netCDF file by copying the contents of the Data URL box.
- Define the associated url you just copied.
url_grid = r'http://opendap.deltares.nl/thredds/dodsC/opendap/rijkswaterstaat/vaklodingen/vaklodingenKB116_4544.nc' url_time = r'http://opendap.deltares.nl/thredds/dodsC/opendap/rijkswaterstaat/waterbase/concentration_of_suspended_matter_in_water/id410-DELFZBTHVN.nc'
- Extract the data.
# Get grid data grid = opendap(url_grid) G_x = grid.variables['x'] G_y = grid.variables['y'] G_z = grid.variables['z'] G = {} # dictionary ~ Matlab struct G['x'] = G_x[:].squeeze() G['y'] = G_y[:].squeeze() G['z'] = G_z[1,:,:].squeeze() # download only one temporal slice # represent fillValue from data as Masked Array # the next release of netcdf4 will return a masked array by default, handling NaNs # correctly too (http://code.google.com/p/netcdf4-python/issues/detail?id=168) G['z'] = np.ma.masked_invalid(G['z'])
# Get time series data time = opendap(url_time) T_t = time.variables['time'] T_z = time.variables['concentration_of_suspended_matter_in_water'] T = {} # dictionary ~ Matlab struct T['t'] = netCDF4.num2date(T_t[:], units=T_t.units) T['z'] = T_z[:].squeeze()
- plot ...
# plot grid data matplotlib.pyplot.pcolormesh(G['x'],G['y'],G['z']) pylab.xlabel('x [m]') pylab.ylabel('y [m]') matplotlib.pyplot.colorbar() matplotlib.pyplot.axis('tight') matplotlib.pyplot.axis('equal') matplotlib.pyplot.title(url_grid) pylab.savefig('vaklodingenKB116_4544') # plot time series data pylab.clf() matplotlib.pyplot.plot_date(T['t'], T['z'], fmt='b-', xdate=True, ydate=False) pylab.ylabel('SPM [kg/m3]') matplotlib.pyplot.title(url_time) pylab.savefig('DELFZBTHVN')
- et voila
Download the code of this python example (repos,manual download) + opendap.py pydap2netCDF4 module (repos, manual download).
See also: OPeNDAP subsetting with python,Accessing netCDF/OPeNDAP unstructured grids with python, Accessing netCDF/OPeNDAP data with R, Accessing netCDF/OPeNDAP data with Matlab, curvilinear model data subsetting with python
4 Comments
Rogier Westerhoff
Hi,
It says:
"Go to an OPeNDAP server (e.g. http://opendap.deltares.nl) and pick a netCDF file by copying the contents of the Data URL box.
Define the associated url you just copied."
But after that in the box it says:
"# specify an url, netCDF4 version 0.9.1 has no OPeNDAP support yet, so we use a local netCDF file."
Is netCDF4 now ready or should we still use a local file (i.e. download first?)
Gerben de Boer
This only applies to OPeNDAP support for python-netcdf4 on Windows.
NetCDF4 itself works fine on all platforms (win + mac + linux).
OPeNDAP support inside netCDF4 currently only works for mac and linux. Unidata - the maker of netcdf - will work in windows support for netcdf4, and will hopefully solve for OPeNDAP support at the same time. See:
Fedor Baart
You can also use the opendap function in OpenEarthTools/python/io . That one uses netCDF4 (if it works) or pydap.client and adapts the interface of the pydap.client to look pretty similar to netCDF4 (uses dataset.variables for example).
Gerben de Boer
The example has now been succefully tested for windows (PythonXY 2.6.6.1 + netCDF4 0.9.4 + pydap 3.0.rc.13-py2.6).