|
The OpenEarth infrastructure supports a bottom-up approach for long-term project-transcending collaboration adhering to four basic criteria:
To make the OpenEarth infrastructure work, the following ICT components are required in practice(see figure):
The remainder of this tutorial addresses the steps needed to set up this infrastructure.
Register at VMWARE and download "VMWARE Server" at https://www.vmware.com/. Make sure you write down the Windows code as it is needed to install VMWare Server. Once VMWare Server is installed (login with administrator rights if needed) open it and create a virtual machine:
Download ubuntu Desktop version 9.04 from http://www.ubuntu.com/GetUbuntu/download and install it (the download is an ISO-file). Make a map ISO in "d:\Virtual Machines" and copy the UBUNTU ISO image there. In VM Ware select the virtual server you created and select in the hardware list the CDROM/DVD option. Choose edit. Under the section "ISO Image" choose browse and browse to the UBUNTU ISO-image ("standard" then "ISO" then select "Ubuntu") you just copied. Now go to the console and start up the virtual machine (click the screen as indicated, start the machine by pressing the big play button that appears). Now VMWare will open with the virtual machine with the Ubuntu desktop. Select the Install icon. TIP: Your computer may require installation of a remote console plugin first.
Now the Ubuntu operating system will install (select all settings to your liking: keyboardtype, timezone etc.). After it is finished you need to reboot the virtual machine. TIP: UBUNTU will request to do this automatically, otherwise use the powerbutton in the upper right corner of your virtual desktop. If your virtual machine does not start up properly a possible cause could be that the ISO image is still selected in the CDROM/DVD drive (see previous steps). Unselecting it is likely to fix this problem. The virtual machine will indicate that the virtual CDRom drive is locked (?!?). You can select to override. You will be requested to create a useraccount and a password, select something useful like 'admin-oet' or so. TIP: write down your username and password, you will need them later!
Once the virtual machine is created select it:
cd Desktop/vmware-tools-distrib/ sudo ./vmware-install.pl |
TIP: Select for all options the indicated default value (NB: some steps take some time to complete ... be patient!). When prompted for display size option [12] is recommended (1024x768).
Ubuntu will suggest to install security updates. You can install them if you want to (if your internet connection is slow maybe opt to do this later).
To setup the subversion service on your virtual machine, first install subversion using the synaptic package manager (see the section Overview of useful commands for tips on locating the synaptic package manager). You may need to reload for subversion to appear in the package list. Select "subversion" and "subversion-tools" and press 'Apply' to actually install these packages. Once subversion package is installed the instructions on
https://help.ubuntu.com/community/Subversion
can be followed to set up subversion repositories.
A user 'subversion' needs to be added by typing the following command in a terminal window:
sudo adduser subversion |
A directory can be created where new repositories can be put:
sudo mkdir /home/svn cd /home/svn |
Tip: you will probably want to select a more appropriate directory for actual applications (something like /var/data/repos/). The appropriate location depends on a companies local preferences.
A new repository is created by issuing the following commands:
sudo mkdir myproject sudo chown -R www-data:subversion myproject sudo chmod -R g+rws myproject sudo svnadmin create /home/svn/myproject sudo chmod -R g+rws myproject |
In each repository directory uncomment the password keyword in the svn config file "svnserve.conf" and add usernames and passwords to the passwordfile "passwd". Set the authentication settings to your preference: e.g. anon-access = none. Tip: Use 'sudo gedit' to start an editor program. You need to run gedit as sudo to allow filesaving.
To start the svnservice and make it remotely accessible start the following svnserve with the following command:
sudo svnserve -d --foreground -r /home/svn |
or
sudo -u www-data svnserve -d --foreground -r /home/svn |
NB: running svnserve in the foreground the cursor will not reappear (until the service is stopped with crtl-z) so don't wait for it to reappear. If you want to execute another command start a new terminal window. If no error messages appear subversion is running and you can now remotely access the repositories, e.g. with a program like TortoiseSVN.
Besides starting svnserve manually and running it on the foreground, it is also possible to make svnserve start automatically on bootup running it in the background. Follow the instructions on:
http://odyniec.net/articles/ubuntu-subversion-server/
under the section "Svnserve Initialisation Script". It basically comes down to copying the info from:
http://odyniec.net/articles/ubuntu-subversion-server/svnserve
to a file (e.g. using gedit) and saving it as "/etc/init.d/svnserve". Make sure the info in svnserve is edited to include the directory you selected for the location of repositories. The script is installed by running:
sudo update-rc.d svnserve defaults |
Indicate your repository locations (e.g. /var/data/repos/) in the svnserve script:
DAEMON_ARGS="-d -r /var/data/repos" |
To make sure the svnserve script is executed on bootup it needs to be made executable by:
sudo chmod +x /etc/init.d/svnserve |
Subversion should now work and you should be able to commit your first files. For tips see the five easy steps to becoming an OpenEarth developer. The location of your repository will be svn://[ip-address][repositoryname]. You can find the IP address of your virtual machine under the summary tab of VMWare. The repository name you selected yourself in one of the previous steps.
With the Synaptic package manager install Tomcat6 (identify all Tomcat6 packages by inserting 'Tomcat6*' in the search box). Select them and press Apply to install. Once the packages are installed you can start the Tomcat6 server by issuing the following command:
sudo /etc/init.d/tomcat6 start |
or
sudo /etc/init.d/tomcat6 restart |
Under "/var/lib/tomcat6/" create a directory "content" and set the proper groups and rights:
cd /var/lib/tomcat6/ sudo mkdir content sudo chown tomcat6:tomcat6 content sudo /etc/init.d/tomcat6 restart |
Now that your Tomcat6 service is running you can install Thredds via a webinterface. However, Thredds will want to write logfiles. The standard security settings on your virtual machine are too strict. An adjusted .policy file should be added in "/etc/tomcat6/policy.d/" with the following content:
grant codeBase "file:${catalina.base}/-" { permission java.security.AllPermission; }; |
Name it either 00openpolicy.policy or 51openpolicy.policy so that it is run either first or last (this is indicated by the two digits in the name). Apparently this works. NB: A number that is not first or last still results in problems.
Now download Thredds and mount the file 'thredds.war' file (NB: file should have this exact name) by accessing http://\[ip-number\]:8080/manager/html, where [ip-number] is the ip-number of your virtual server. Once you deployed the war file a new option will appear in the Tomcat6 management webpage. Select 'start' to start the Thredds service.
Edit in "/var/lib/tomcat6/content/thredds" the file "tomcat-users.xml". Create a role admin and manager in the config.xml and create a user (e.g. admin) and assign it both roles. For more detail see:
https://help.ubuntu.com/8.10/serverguide/C/tomcat.html
Now you can configure the data to be served with THREDDS. There are some
manuals on the web ([Signell (USGS)|http://rsignell.tiddlyspot.com/#[A%20method%20for%20setting%20up%20TDS%20catalogs%20for%20the%20first%20time]], Unidata). Edit in "/var/lib/tomcat6/content/thredds" the file "catalog.xml". Place data at:
/var/data/ |
Set datasetRoot, datasetScan to the proper data location and indicate the names you wish to appear on the Thredds server. NB: no spaces are allowed in directory names. Change the wildcard to "*.nc".
For restricted datasets (make internal data available for remote access for your employees only) configure tomcat accordingly: http://www.unidata.ucar.edu/projects/THREDDS/tech/reference/RestrictedAccess.html. We advice to restrict access based on the http rather than based on for instance using ownership properties of the data on the underlying (linux) file server. Note that for remote access the netCDF java library cannot handle http authentication/encryption itself ([http://user:password@opendap.institution.nl]...).
Every now and then THREDDS needs to be updated. Check the Unidata THREDDS page for posts. Future releases can be anticipated in the THREDDS issue tracking system https://www.unidata.ucar.edu/jira/browse/TDS. To deploy a new THREDDS with only a few minutes downtime, do the following. Download the new war thredds.war
(release 4.2.10 in this example) and prepare it ready for instant copying to the Tomcat webapps directory /var/lib/tomcat6/content/thredds/
. Please read this check-list for upgrading.
sudo rename thredds.war thredds.war.4.2.9 sudo cp thredds.war.4.2.10 /var/lib/tomcat6/webapps/ cd /var/lib/tomcat6/content/webapps/ cp thredds.war thredds.war.old |
NB You are not allowed to call it thredds.old.war
as any file with trailing *.war
will be unpacked by Tomcat. We advocate the labeling thredds.war.old
and thredds.war.4.2.10
to prevent mistakes with releases. After this preparation do the actual re-deployment.
cd /var/lib/tomcat6/content/webapps/ sudo /etc/init.d/tomcat6 stop sudo rm -rf thredds cp thredds.war.4.2.10 thredds.war sudo /etc/init.d/tomcat6 restart |
In synaptic package manager select:
Create a dir, e.g. kml, in var/data/
. Open /etc/apache2/sites-enabled/000-default
in a text editor and make sure you point to it as /var/data/kml
. This is described by Apache's documentroot documentation. You can also point to files in another directory than /var/data
. This is described by Apache's Files Outside the DocumentRoot documentation. On our test server dtvirt5 we made the kml folder a subfolder of the data directory for THREDDS, such that this data folder is the only folder where we have to give write access to the group of data-administrators.
... DocumentRoot /var/www RewriteEngine on # /kml Alias /kml /data/kml <Directory /data/kml> Order allow,deny Allow from all </Directory> ... |
In both cases make sure the DocumentRoot
contains a representative index.html
file that contains links to the SubVersion, OPeNDAP, and kml services. You can optionally lay-out the index.html
with some logos, e.g. http://http://dtvirt5.deltares.nl/. You can also delete the default "It works!" index.html page, such that the subfolder kml wil appear (unless you relocated the kml folder as described above).
Restart apache service to apply settings
sudo /etc/init.d/apache2 restart |
Now open /etc/apache2/sites-enabled/000-default and add (just after documentroot):
AddType application/vnd.google-earth.kml+xml .kml AddType application/vnd.google-earth.kmz .kmz |
To remotely manage the virtual server install WinSCP. For WinSCP to work you must install OpenSSH-server with the synaptic package manager. For the data directory to be accessible by WinSCP make sure to edit the rights:
sudo chmod -R og+w /var/data sudo chmod +s /var/data |
ifconfig -a |
sudo nautilus |
sudo gedit |
mount |
ls /dev/sd* |
mount /dev/sdb1 /var/data -t auto |
man mount |
sudo /etc/init.d/tomcat6 restart sudo /etc/init.d/svnserve start sudo /etc/init.d/svnserve stop sudo /etc/init.d/apache2 restart |
http://httpd.apache.org/docs/2.2/howto/auth.html Apache authentication and authorization documentation
http://httpd.apache.org/docs/2.2/mod/mod_dav.html Webdav
http://tomcat.apache.org/tomcat-6.0-doc/index.html Tomcat documentation
http://www.drbd.org/ Drbd
http://www.unidata.ucar.edu/software/netcdf/docs/netcdf-install/ NetCDF downloads
http://mexcdf.sourceforge.net/ Matlab netcdf library
http://www.scipy.org/ Python numerical package
http://code.google.com/p/netcdf4-python/ Netcdf python software
http://www.inspire-geoportal.eu/InspireEditor/ Inspire editor
http://www.gfdl.noaa.gov/~vb/grids/gridspec-tools.html Gridspec
https://help.ubuntu.com/8.10/serverguide/C/tomcat.html Install tomcat under ubuntu
http://opendap.org/download/hyrax.html Hyrax server download
http://docs.opendap.org/index.php/Hyrax Hyrax documentation
http://www.unidata.ucar.edu/projects/THREDDS/ Thredds server
http://en.wikipedia.org/wiki/DMZ_%28computing%29 DMZ information
http://en.wikipedia.org/wiki/Secure_copy Secure copy
http://www.epic.noaa.gov/java/ncBrowse/ NcBrowse
http://www.mpimet.mpg.de/fileadmin/software/cdo/ CDO
http://nco.sourceforge.net/ NCO
http://www.unidata.ucar.edu/software/idv/ IDV
http://www.unidata.ucar.edu/downloads/netcdf/netcdf-java-4/index.jsp NetCDF-UI
http://publicwiki.deltares.nl/display/OET/OpenEarth OpenEarthTools
http://subversion.tigris.org/ Subversion
https://help.ubuntu.com/community/Subversion Subversion setup on ubuntu
|