Home | About Gush | Using Gush | Gush Examples | Gush Tutorial at GEC 11 | GENI Summer Camp

Gush Tutorial at GEC 11

This page summarizes the Gush tutorial that was presented at GEC 11. Slides can be found here.

Tutorial examples quick links:

Tutorial Pre-work and VM Startup

  1. Install VirtualBox: Download the latest  VirtualBox software. If you already have VirtualBox installed on your machine, make sure it is version 4.08 or above.

Note to Ubuntu users: The version of VirtualBox distributed with Ubuntu 10.04 is old. Download the latest version from  here.

  1. Download VirtualBox VM image: Download the VirtualBox vm image (gec11-tutorials-final.ova) needed for the Gush tutorial from  here. A README with a copy of these instructions is also available at the same location.
  1. Install the gec11-tutorials-final.ova image: Start up VirtualBox, select File->Import Appliance..., and follow the instructions. Accept the default VM settings during the import. To run the virtual machine, go to the Oracle VM VirtualBox Manager window, select the virtual machine gec11-tutorial-final and click the green arrow labeled Start at the top of this window.

If the install was successful, you should see the logon screen for the Ubuntu OS. Passwords for logging into the VM will be handed out at the tutorial.

  1. Turn off the VM: Turn off the virtual machine by clicking on the power button icon on the bottom right of the screen.

Note that this release of tutorial software is intended for use at the GEC tutorials only. We will have a post-GEC release with updated software at a later date.

Gush Configuration

Open a terminal on the VM. Type the following commands to add Gush to your home directory (copy/paste introduces hidden characters and can mess things up):

$ bash 
$ keychain ~/.ssh/id_rsa
$ echo 'StrictHostKeyChecking no' >> ~/.ssh/config
$ mkdir gush 
$ cd gush 
$ cp /usr/local/geni/gush/gush-bin-32bit.tgz .
$ tar xzvf gush-bin-32bit.tgz

Make sure hidden binary characters do not appear in ~/.ssh/config. This was a common problem at the tutorial session. The following files must be modified for Gush to work correctly.

  • omni_config - Gush uses a modified omni client to connect to aggregate managers. Configure your omni_config to look like this, replacing guest31 with the appropriate username. Make sure ALL paths are correctly configured!
users = guest31 

  • directory.xml - This file describes the resources available to Gush. For our example, we will use PlanetLab and ProtoGENI resources. Configure your directory.xml like this, again replacing guest31 and slice31 as appropriate. Take note of the "cf=plc" attribute. This tells Gush that we are using PlanetLab credentials to create ProtoGENI slivers. Without it, things won't work. Also pick a different port to avoid conflicts.
<?xml version="1.0" encoding="UTF-8"?>
    <resource_manager type="geni-plc">
      <port_map slice="gec11_slice31" port="61531"/>

    <resource_manager type="geni-pg" cf="plc">
      <port_map slice="slice31" port="61631"/>

ProtoGENI Sliver Creation

Using the following RSpec (called myslice.rspec), we reserve two Fedora 10 VMs and run a command to reset their hostname to be that of the physical machine.

<?xml version="1.0" encoding="UTF-8"?>
<rspec type="request" xsi:schemaLocation="http://www.protogeni.net/resources/rspec/2
  <node client_id="geni1" component_manager_id="urn:publicid:IDN+emulab.net+authority+cm" exclusive="true">
    <sliver_type name="raw-pc">
      <disk_image name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD"/>
      <execute shell="sh" command="sudo hostname `cat /var/emulab/boot/realname`.`cat /var/emulab/boot/mydomain`"/>
  <node client_id="geni2" component_manager_id="urn:publicid:IDN+emulab.net+authority+cm" exclusive="true">
    <sliver_type name="raw-pc">
      <disk_image name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD"/>
      <execute shell="sh" command="sudo hostname `cat /var/emulab/boot/realname`.`cat /var/emulab/boot/mydomain`"/>

We request a ProtoGENI sliver using our PlanetLab slice credentials. We use a Gush helper-script to do this. (Note that handle-geni.py is nothing more than a modified omni client.)

$ helper-scripts/handle-geni.py -n -f plc -a https://www.emulab.net/protogeni/xmlrpc/am createsliver slice31 myslice.rspec

INFO:omni:Loading config file omni_config
INFO:omni:Using control framework plc
INFO:omni.sfa:SFA Registry: http://www.planet-lab.org:12345
INFO:omni.sfa:SFA Slice Manager: http://www.planet-lab.org:12347
Asked https://www.emulab.net/protogeni/xmlrpc/am to reserve resources. Result:
<?xml version="1.0" ?><rspec type="manifest" xmlns="http://www.protogeni.net/resources/rspec/2" xmlns:flack="http://www.protogeni.net/resources/rspec/ext/flack/1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.protogeni.net/resources/rspec/2                                           http://www.protogeni.net/resources/rspec/2/manifest.xsd">  
    <node client_id="geni1" component_id="urn:publicid:IDN+emulab.net+node+pc82" component_manager_id="urn:publicid:IDN+emulab.net+authority+cm" exclusive="true" sliver_id="urn:publicid:IDN+emulab.net+sliver+45415">    
        <sliver_type name="raw">      
            <disk_image name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD"/>      
          <login authentication="ssh-keys" hostname="pc82.emulab.net" port="22" username="guest31"/>    </services>    
      <rs:vnode name="pc82" xmlns:rs="http://www.protogeni.net/resources/rspec/ext/emulab/1"/>  </node>  
    <node client_id="geni2" component_id="urn:publicid:IDN+emulab.net+node+pc68" component_manager_id="urn:publicid:IDN+emulab.net+authority+cm" exclusive="true" sliver_id="urn:publicid:IDN+emulab.net+sliver+45416">    
        <sliver_type name="raw">      
            <disk_image name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD"/>      
          <login authentication="ssh-keys" hostname="pc68.emulab.net" port="22" username="guest31"/>    </services>    
      <rs:vnode name="pc68" xmlns:rs="http://www.protogeni.net/resources/rspec/ext/emulab/1"/>  </node>  

If you get a "server WAY too busy" error, please wait a few moments and try again.

Start Gush!

We are ready to start Gush and make sure our resource pools are being populated correctly. Try this:

guest31@geni-vm:~/gush$ ./gush -P 15000
gush> Gush has learned about the slice gec11_slice31.
Gush has learned about the slice slice31.

gush> Updated information on the slice gec11_slice31 is available.
Updated information on the slice slice31 is available.

If you do not see these messages, something is not configured correctly. Check omni_config and directory.xml for typos.

To verify that our resource pools are being populated, try this:

gush> info nodes
There are 17 known nodes:
[ P         ] gec11_slice31@planet2.scs.cs.nyu.edu:61500(pref=0) (Disconnected.)
[ P         ] gec11_slice31@ricepl-1.cs.rice.edu:61500(pref=0) (Disconnected.)
[ P         ] gec11_slice31@planet1.scs.stanford.edu:61500(pref=0) (Disconnected.) 
[ P         ] gec11_slice31@planetlab1.cs.uchicago.edu:61500(pref=0) (Disconnected.)
[ P         ] gec11_slice31@planetlab1.cs.umass.edu:61500(pref=0) (Disconnected.)
[ P         ] gec11_slice31@planetlab1.cs.uoregon.edu:61500(pref=0) (Disconnected.)
[ P         ] gec11_slice31@planetlab2.cs.uoregon.edu:61500(pref=0) (Disconnected.)
[ P         ] gec11_slice31@planetlab3.cs.uoregon.edu:61500(pref=0) (Disconnected.)
[ P         ] gec11_slice31@planetlab4.cs.uoregon.edu:61500(pref=0) (Disconnected.)
[ P         ] gec11_slice31@pl2.eecs.utk.edu:61500(pref=0) (Disconnected.)
[ P         ] gec11_slice31@planetlab02.cs.washington.edu:61500(pref=0) (Disconnected.)
[ P         ] gec11_slice31@planetlab1.koganei.wide.ad.jp:61500(pref=0) (Disconnected.)
[ P         ] gec11_slice31@planetlab1.arizona-gigapop.net:61500(pref=0) (Disconnected.)
[ P         ] gec11_slice31@planetlab2.arizona-gigapop.net:61500(pref=0) (Disconnected.)
[ P         ] gec11_slice31@planetlab3.arizona-gigapop.net:61500(pref=0) (Disconnected.)
[ P         ] guest31@pc419.emulab.net:61600(pref=0) (Disconnected.)
[ P         ] guest31@pc430.emulab.net:61600(pref=0) (Disconnected.)

If you do not see resources from both PlanetLab and ProtoGENI, something is wrong.

Useful Commands

The following commands are useful for manipulating remote resources using Gush.

 gush> connect <hostname>: Start a Gush client on a remote host.
 gush> connect pat <regex> <num>: Connect to <num> hosts that match the regular expression pattern <regex>.
 gush> disconnect: Close all open connections.
 gush> disconnect <hostname>: Disconnect from a host.
 gush> info nodes: Print summary information on all known nodes.
 gush> info node <hostname>: Print information about a node.
 gush> info mesh: Print the mesh status (membership).
 gush> shell "<quoted string>": Run the string as a shell command on all connected nodes.
 gush> quit: Quit Gush.

The following commands are useful when running experiments with Gush.

 gush> prefer <regex>: Tell Gush to increase preference value for nodes matching regex.  Nodes with higher preference values are used first.
 gush> load <file.xml>: Read a project file.
 gush> run: Start executing the first experiment in the active project.
 gush> info control: Print controller state information.
 gush> debug fail node <hostname>: Mark node as failed and find replacement (only during experiment).
 gush> save prefer <prefer.xml>: Save current host preferences (can load them back in later).

Always remember to disconnect from resources when finished running an experiment. If you forget, you can reconnect and run "killall -9 client" to clean things up. Also, periodically clean up your local Gush directory by removing logfiles (rm *logfile*).


Now we are ready to try some simple examples. Check out the "tutorial examples quick links" on the top of this page for more info.