IMPORTANT NOTE: As of January 2012, the GENI API is being upgraded in some aggregates (such as PlanetLab). As a result, the Gush helper scripts may no longer work. We are working on an updated version. Please email us for more info.
This page describes the setup process required to use Gush. Gush currently supports execution on PlanetLab and ProtoGENI. Support for ORCA is underway.
Important: Before using Gush with PlanetLab or ProtoGENI, you must create a slice and sliver on the desired control framework(s). In addition, Gush requires the ability to SSH to all hosts without typing a passphrase. You should set up an ssh-agent to provide password-free ssh access to hosts as needed. To handle the case where connecting to a new node brings up a fingerprint verification question, add "StrictHostKeyChecking no" to your ~/.ssh/config. We have found that PlanetLab keys change periodically, and by not requiring StrictHostKeyChecking we encounter less errors. Additional information about creating ProtoGENI slices and slivers, and using ProtoGENI with Gush can be found on the ProtoGeniExample and ProtoGeniExampleTwo pages.
Gush is written in a mix of C++ and Python, and we intend for it to be runnable on any UNIX-ish OS. That said, we have been using it primarily with Linux and Mac OS. Compiling Gush can be challenging, so please contact us if you experience too many problems. Gush requires the following libraries:
- xmlrpc-c: libxmlrpc_client++, libxmlrpc_client, libxmlrpc_server_abyss++, libxmlrpc_server++, libxmlrpc_server_abyss, libxmlrpc_server, libxmlrpc_abyss, libxmlrpc++, libxmlrpc, libxmlrpc_util, libxmlrpc_xmlparse, libxmlrpc_xmltok
Note: lbxmlrpc-c3 will not work. xmlrpc-c must be installed, and to our knowledge, there is no debian package available.
- curl: libcurl
- xml2: libxml2
- zlib compression/decompression: libz
- math: libm
- openssl: libssl, libcrypto
- dynamic linking: libdl
- readline: libreadline
- curses: libcurses
- intl domain names: libidn
- pthreads: libpthread
The boost software package is also required, as well as lex and yacc (we use flex and bison).
Gush supports the latest version of the GENI API. It uses a modified version of the Omni client for communication with GENI aggregate managers. More information about Omni can be found on the OmniOverview page and on the QuickStart guide. Since Omni is written in Python, you must install a few Python libraries in addition to the C++ libraries mentioned above. More details about configuring Python and Omni are explained below.
For reference, the following commands were used to install the prerequisite software for Gush (including Omni) on a new machine with a clean installation of Ubuntu 10.04 Desktop 64bit.
$ sudo apt-get install openssh-server subversion keychain g++-4.1 autoconf2.59 makedepend emacs23 $ sudo ln -s /usr/bin/g++-4.1 /usr/bin/g++ $ sudo apt-get install curl libcurl3 libcurl4-openssl-dev libboost-dev xutils-dev flex bison libreadline-dev $ sudo apt-get install python-m2crypto python-dateutil python-pyopenssl libxmlsec1 xmlsec1 libxmlsec1-openssl libxmlsec1-dev $ wget http://gush.cs.williams.edu/xmlrpc-c-1.16.34.tgz $ tar xzf xmlrpc-c-1.16.34.tgz $ cd xmlrpc-c-1.16.34 $ ./configure $ make $ sudo make install
The following commands were used to install the prerequisite software on a clean install of Mac OS X 10.6.8 (using fink and macports).
$ fink install wget svn autoconf2.6 curl libcurl3-unified boost1.35.python26 boost-jam flex bison readline $ sudo port install py26-m2crypto py26-dateutil py26-openssl xmlsec boost $ ln -s /opt/local/include/boost /usr/local/include $ wget http://gush.cs.williams.edu/xmlrpc-c-1.16.34.tgz $ tar xzf xmlrpc-c-1.16.34.tgz $ cd xmlrpc-c-1.16.34 $ ./configure $ make $ sudo make install
Setting Up Gush
After installing the required libraries mentioned above, the first step to using Gush is to download and compile the code. The code can be obtained from the SVN repository linked to this site. You must request an account before obtaining the code. Please email jeannie AT cs dot williams dot edu for access to the repository.
After obtaining the password, check out Gush using the following command:
$ svn co http://gush.cs.williams.edu/svn/gush/trunk --username guest
Once the code has been downloaded, follow the instructions found in README.txt to compile and install Gush. These instructions are included here for your reference.
NOTE: We have statically compiled versions of both the Gush and Gush client binaries available. If you would like to skip the compilation step (which can be tedious!), most users can simply use the statically compiled binaries. We currently have binaries for 64 bit Ubuntu Linux and an older 32 bit Ubuntu Linux. Other Linux distributions may be able to run these binaries, but this has not been tested fully. To use these binaries, download the appropriate tarball, untar it, and skip ahead to the "Configuring Python for Omni" section below.
64 bit Ubuntu Linux: gush-bin-64bit.tgz
32 bit Ubuntu Linux: gush-bin-32bit.tgz
After downloading the code from SVN, follow the instructions below to compile Gush. NOTE: This dynamically compiles Gush and the Gush client. You can compile statically using the --with-static-gush and --with-static-client compile flags. Statically compiling is usually a bit more error prone because of some issues with curl. If static compilation is required, we recommend installing curl directly from source and using --disable-ldap while compiling curl. For most installations, dynamically compiling should be sufficient.
$ cd gush/trunk $ autoconf (Make sure you use autoconf version 2.50 or higher.) $ ./configure $ make dep $ make
Configuring Python for Omni
According to the Omni QuickStart page, the following installation methods may be helpful:
- Debian / Ubuntu
$ sudo apt-get install python-m2crypto python-dateutil python-pyopenssl libxmlsec1 xmlsec1 libxmlsec1-openssl libxmlsec1-dev
- Mac OS X (using MacPorts)
$ sudo port install py26-m2crypto py26-dateutil py26-openssl xmlsec $ cd /opt/local/lib $ sudo ln -s libxmlsec1-openssl.dylib libxmlsec1-openssl.so
You must configure the omni_config file before using Omni to connect to any control frameworks. Details about omni_config can be found on the OmniOverview page and OmniConfiguration page, but a sample omni_config is included for reference in the Gush SVN repository. It looks like this:
[omni] #default_cf = plc #default_cf = pg users = jeannie [plc] type=sfa authority=plc.williams user=plc.williams.jeannie cert=~/.gcf/plc.williams.jeannie.gid key=~/.gcf/jeannie.pem registry=http://www.planet-lab.org:12345 slicemgr=http://www.planet-lab.org:12347 #[pg] #type=pg #ch = https://www.emulab.net:443/protogeni/xmlrpc/ch #sa = https://www.emulab.net:443/protogeni/xmlrpc/sa #cert = ~/.ssl/encrypted.pem #key = ~/.ssl/encrypted.pem [jeannie] urn=urn:publicid:IDN+plc:williams+user+jeannie #urn = urn:publicid:IDN+emulab.net+user+jeannie keys=~/.ssh/id_rsa.pub
In this sample omni_config file, your PlanetLab private key is stored in the ~/.gcf directory. Note that this key cannot require a passphrase to work with Gush. You can remove the passphrase of a private key called keyin.pem, for example, by running:
$ openssl rsa -in keyin.pem -out keyout.pem
In this case, your omni_config file should include a reference to keyout.pem. You can use the handle-geni.py script included with Gush (in the helper-scripts directory) to automatically download your certificate. This is also a good way to make sure you have Omni correctly configured to talk to the aggregate managers. To do this, insert your PlanetLab slice name into the following command:
$ helper-scripts/handle-geni.py -f plc -a http://www.planet-lab.org:12346 sliverstatus <slice_name>
or your ProtoGENI slice name into the following command:
$ helper-scripts/handle-geni.py -f pg -a https://www.emulab.net/protogeni/xmlrpc/am sliverstatus <slice_name>
If it prompts you to download your certificate file, say "Y". If everything is working correctly, you should see something like this (for PlanetLab):
$ helper-scripts/handle-geni.py -f plc -a http://www.planet-lab.org:12346 sliverstatus williams_gush INFO:omni:Loading config file omni_config INFO:omni:Using control framework plc Your certificate file (/home/jeannie/.gcf/plc.williams.jeannie.gid) was not found, would you like me to download it for you to /home/jeannie/.gcf/plc.williams.jeannie.gid? (Y/n)Y INFO:omni.sfa:SFA Registry: http://www.planet-lab.org:12345 INFO:omni.sfa:SFA Slice Manager: http://www.planet-lab.org:12347 INFO:omni:Sliver at http://www.planet-lab.org:12346: <?xml version="1.0"?> <gush> <slice name="williams_gush"> <expires>2011-04-28 13:41:35</expires> <user name=""/> <node name="planetlab1.williams.edu"/> <node name="planetlab2.williams.edu"/> <node name="planetlab4.williams.edu"/> <node name="planetlab5.williams.edu"/> <node name="planetlab1.ucsd.edu"/> <node name="planetlab2.ucsd.edu"/> <node name="planetlab3.ucsd.edu"/> <node name="planetlab3.williams.edu"/> </slice> </gush>
Please double check and make sure you have a valid key and certificate, and that Omni is correctly configured before moving on!
You need three configuration files to use Gush: directory.xml, gush.prefs, and omni_config. Details about omni_config are described above. For most installations, gush.prefs should not need to be modified. In directory.xml (shown below), you should modify the SVN version of the file to include your slice name instead of williams_gush2. For the port number, choose a port greater than 61000. Please change the port! You probably should reserve a port here.
directory.xml (in SVN):
<?xml version="1.0" encoding="UTF-8"?> <gush> <resource_manager type="geni-plc"> <port_map slice="williams_gush2" port="61500"/> </resource_manager> <!-- <resource_manager type="geni-pg"> <port_map slice="jeannie" port="61600"/> </resource_manager> --> </gush>
To summarize, you must modify directory.xml and omni_config before Gush will work!
That should be it! Let us know if you have problems. This is still a work in progress!
You can now start Gush with "./gush -P <port number>". Using the default command-line user interface, Gush tries to act like a shell (well, almost -- unfortunately many of the commands are asynchronous, so you don't have the nice background/foreground distinction present in normal shells). A very-partial list of those commands is listed below; the official source is the "terminal_parser.y" YACC file.
For instant gratification, try the following commands:
$ ./gush -P 15555 Start Gush on port 15555. Wait until Gush prints out two messages with your slice in it. gush> connect email@example.com Connect to planetlab1.williams.edu. This basically installs/starts the Gush client. Wait until you see a message about the host joining the mesh. gush> info mesh Print out the current connection status of various hosts. gush> shell "ps aux" Run "ps aux" on every connected host, and print the results locally. gush> disconnect Disconnect hosts from the mesh--this kills remote clients. gush> quit
There are several example application description XML files in the "tests" directory that can be used to experiment with different aspects of Gush.
gush> load path: Read a project file. gush> run: Start executing the first experiment in the active project. gush> connect hostname: Start a Gush client on a remote host. gush> connect pat <regex> <num>: Connect to <num> hosts that match the regular expression pattern <regex>. gush> debug fail node <hostname>: Mark node as failed and find replacement (only during experiment). gush> disconnect: Close all open connections. gush> disconnect hostname: Disconnect from a host. gush> info control: Print controller state information. gush> info nodes: Print summary information on all known nodes. gush> info node hostname: Print information about a node. gush> info mesh: Print the mesh status (membership). gush> prefer <regex>: Tell Gush to increase preference value for nodes matching regex. Nodes with higher preference values are used first. gush> shell <quoted string>: Run the string as a shell command on all connected nodes. gush> quit: Quit Gush.
Additional PlanetLab Commands
gush> connect slice slice_name: Connect to all nodes assigned a slice. gush> info slices: Print summary information on all known slices. gush> info slice slice_name: Print information about a slice. gush> slice update slice_name: Get updated information for a slice from the PLC Database. gush> slice list slice_name: Show information on a slice. gush> slice renew slice_name: Renew the given slice. gush> slice add slice_name regex: Assign hosts that match regex (regular expression) to the given slice. gush> slice remove slice_name regex: Unassign hosts that match regex from the slice.