OPeNDAP is a web service for netCDF-CF files. Web services offer you live access without the need to download a local copy, that would only occupy disk space and become outdated. However, for performance issues, having a local up-to-date copy is very useful. This applies even more when you often work in places with low-bandwidth internet connection, or even without wifi, such as public transport. A local copy for temporary use is known as a cache. Remove the cache when you do not need it any more.
For many OGC WxS webservices special software exist to make those caches, e.g. geocache, geowebcache. For OPeNDAP servers you can simply download all data files simply with an ftp server. By default an OPeNDAP end-note offers both OPeNDAP and HTTPServer as a service, the latter is simply an http/ftp download that can be bathces (e.g. this link). For linux command line batch download can be done with wget which also has a dos/windows version. For windows wget is not available, Windows wget BITSadmin (which will be phased out after Windows 7). This example show how to 2 vaklodingen netCDF files. The remote and local path are turned into variables, to facilitate use creation of this batch file using copy-and-paste from the html catalog (e.g. this link). You can do that with the column mode in many regular text editors, for instance Textpad.
| Code Block |
|---|
rem source using web / slashes set from=http://opendap.deltares.nl/thredds/fileServer/opendap/rijkswaterstaat/vaklodingen/ rem local destination using windows \ slashes set into=d:\opendap.deltares.nl\thredds\dodsC\opendap\rijkswaterstaat\vaklodingen\ bitsadmin /transfer Job001 /download /priority normal %from%vaklodingenKB114_4544.nc %into%vaklodingenKB114_4544.nc bitsadmin /transfer Job001 /download /priority normal %from%vaklodingenKB115_4544.nc %into%vaklodingenKB115_4544.nc bitsadmin /transfer Job001 /download /priority normal %from%vaklodingenKB115_4544.nc %into%vaklodingenKB116_4544.nc |
On XP machines Bitadmin needs to be installed seperately, and will end in a different folder, e.g. c:\program Files\Support tools, for which you will need to adapt the bitsadmin bat file lines to
| Code Block |
|---|
"c:\program Files\Support tools" /transfer Job001 /download /priority normal %from%vaklodingenKB115_4544.nc %into%vaklodingenKB116_4544.nc |
I myself have a local cache of regularly used datasets in a folder called d:\opendap.deltares.nl\thredds\dodsC\opendap. The directory tree structure is the very same as the base url for OPeNDAP urls on our own OpeNDAP server, such that in my processing scripts I only need to replace http:// with d: to make script work with a local cache. I configured my local data cache not to be back-up by the institutes automatic back-up scheme, since it is only a cache from web-data after all.