Home | About Gush | Using Gush | Gush Examples | Gush Tutorial at GEC 11 | GENI Summer Camp


Gush Tutorial at GENI Summer Camp

This page summarizes the Gush tutorial that was presented at the GENI Summer Camp at RIT. Slides can be found here.

Tutorial examples quick links:

Pre-work and VM Startup

Before doing any of the steps described on this page, please make sure you have followed all of the instructions found  here. IMPORTANT: Make sure you have configured the SSH workaround found  here.

Gush Configuration

Open a terminal on the VM. Type (or copy/paste) the following commands to add Gush to your VM. (Note that copy/paste introduces hidden characters in the echo command, which can mess things up. You should type that line rather than using copy/paste.):

$ sudo apt-get install keychain
$ cp /home/geni/.ssh/geni_key /home/geni/.ssh/id_rsa
$ cp /home/geni/.ssh/geni_key.pub /home/geni/.ssh/id_rsa.pub 
$ keychain ~/.ssh/id_rsa
$ echo "Host *" >> ~/.ssh/config
$ echo "  StrictHostKeyChecking no" >> ~/.ssh/config
$ mkdir gush 
$ cd gush 
$ wget http://gush.cs.williams.edu/gush-bin-32bit.tgz
$ tar xzvf gush-bin-32bit.tgz
$ cp /home/geni/.gcf/omni_config .

Make sure hidden binary characters do not appear in ~/.ssh/config. (This was a common problem at a tutorial session last year.)

Next, directory.xml must be modified for Gush to work correctly. This file describes the resources available to Gush. For our example, we will use PlanetLab and ProtoGENI resources. Configure your directory.xml like this, replacing SLICENAME and PG_SLICE with your slices. (If you do not have a slice yet on PlanetLab or ProtoGENI, you will create them in the next step. Just choose a name for now, and remember it for the next step.) For the attributes that say "port=61531" and port="61631", pick different ports in the range 61000 - 62000 to avoid potential conflicts. Initially, we will comment out the PlanetLab part and use only ProtoGENI resources.

<?xml version="1.0" encoding="UTF-8"?>
<gush>
<!--    
    <resource_manager type="geni-pg" am="http://localhost:8348">
      <port_map slice="pgenigpolabbbncom_SLICENAME" port="61531"/>
    </resource_manager>
-->

   <resource_manager type="geni-pg">
      <port_map slice="PG_SLICE" port="61631"/>
    </resource_manager>

</gush>

PlanetLab Slice and Sliver Creation

First we need an RSpec request file containing the resources you want to reserve. Here is a sample (referred to as plab.rspec in remaining instructions). Copy and paste the following XML into a file on your machine.

<?xml version="1.0" ?>
<!-- Resources at AM:
        URN: unspecified_AM_URN
        URL: http://www.planet-lab.org:12346
 -->
<rspec type="request"
       xmlns="http://www.protogeni.net/resources/rspec/2"
       xmlns:emulab="http://www.protogeni.net/resources/rspec/ext/emulab/1"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.protogeni.net/resources/rspec/2
       http://www.protogeni.net/resources/rspec/2/request.xsd
       http://www.protogeni.net/resources/rspec/ext/emulab/1
       http://www.protogeni.net/resources/rspec/ext/emulab/1/ptop_extension.xsd">

  <node 
   component_id="urn:publicid:IDN+plc:ucsd+node+planetlab3.ucsd.edu" 
   component_manager_id="urn:publicid:IDN+plc+authority+cm" 
   component_name="planetlab3.ucsd.edu" exclusive="false"
   client_id="ucsdpl3">
      <hardware_type name="plab-pc"/>    
      <hardware_type name="pc"/>    
      <location country="unknown" latitude="32.877" longitude="-117.237"/>      
      <sliver_type name="plab-vnode"/>    
  </node>    
  <node 
    component_id="urn:publicid:IDN+plc:williams+node+planetlab1.williams.edu" 
    component_manager_id="urn:publicid:IDN+plc+authority+cm" 
    component_name="planetlab1.williams.edu" exclusive="false"
    client_id="williamspl1">
      <hardware_type name="plab-pc"/>    
      <hardware_type name="pc"/>    
      <location country="unknown" latitude="42.71" longitude="-73.2"/>      
      <sliver_type name="plab-vnode"/>    
  </node>   
  <node
    component_id="urn:publicid:IDN+plc:williams+node+planetlab3.williams.edu"
    component_manager_id="urn:publicid:IDN+plc+authority+am"
    component_name="planetlab3.williams.edu" exclusive="false" 
    client_id="williamspl3">    
      <hardware_type name="plab-pc"/>    
      <hardware_type name="pc"/>    
      <location country="us" latitude="42.71" longitude="-73.2"/>    
      <sliver_type name="plab-vnode"/>    
  </node>  
  <node
    component_id="urn:publicid:IDN+plc:bbn+node+plnode-03.gpolab.bbn.com"
    component_manager_id="urn:publicid:IDN+plc+authority+am"
    component_name="plnode-03.gpolab.bbn.com" exclusive="false" 
    client_id="gpopl3">    
      <location country="us" latitude="42.3907" longitude="-71.1478"/>    
      <hardware_type name="plab-pc"/>    
      <hardware_type name="pc"/>    
      <sliver_type name="plab-vnode"/>    
  </node>  
  <node 
    component_id="urn:publicid:IDN+plc:cornell+node+planetlab2.cs.cornell.edu" 
    component_manager_id="urn:publicid:IDN+plc+authority+cm" 
    component_name="planetlab2.cs.cornell.edu" exclusive="false"
    client_id="cornellpl2">
       <location country="unknown" latitude="42.4436" longitude="-76.4816"/>      
      <hardware_type name="plab-pc"/>    
      <hardware_type name="pc"/>    
      <sliver_type name="plab-vnode"/>    
  </node>    
  <node 
    component_id="urn:publicid:IDN+plc:umassd+node+planetlab2.umassd.edu" 
    component_manager_id="urn:publicid:IDN+plc+authority+cm" 
    component_name="planetlab2.umassd.edu" exclusive="false"
    client_id="umassdpl2">
      <location country="unknown" latitude="41.6286" longitude="-71.0061"/>      
      <hardware_type name="plab-pc"/>    
      <hardware_type name="pc"/>    
      <sliver_type name="plab-vnode"/>    
  </node>    
</rspec>

Following the instructions below to create a PlanetLab slice and sliver, inserting your own slice name in for SLICENAME.

$ omni.py createslice SLICENAME
$ omni.py -a plc createsliver --api-version 2 SLICENAME plab.rspec

Now we have to wait for our keys to be copied out to the PlanetLab machines (this usually takes ~15 minutes). In the meantime, we can setup ProtoGENI.

ProtoGENI Sliver Creation

Using the following RSpec (called pg.rspec), we reserve two Fedora 10 VMs and run a command to reset their hostname to be that of the physical machine.

<?xml version="1.0" encoding="UTF-8"?>
<rspec type="request" xsi:schemaLocation="http://www.protogeni.net/resources/rspec/2
                                          http://www.protogeni.net/resources/rspec/2/request.xsd"
       xmlns:flack="http://www.protogeni.net/resources/rspec/ext/flack/1"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns="http://www.protogeni.net/resources/rspec/2">
  <node client_id="geni1" component_manager_id="urn:publicid:IDN+emulab.net+authority+cm" exclusive="true">
    <sliver_type name="raw-pc">
      <disk_image name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD"/>
    </sliver_type>
    <services>
      <execute shell="sh" command="sudo hostname `cat /var/emulab/boot/realname`.`cat /var/emulab/boot/mydomain`"/>
    </services>
  </node>
  <node client_id="geni2" component_manager_id="urn:publicid:IDN+emulab.net+authority+cm" exclusive="true">
    <sliver_type name="raw-pc">
      <disk_image name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD"/>
    </sliver_type>
    <services>
      <execute shell="sh" command="sudo hostname `cat /var/emulab/boot/realname`.`cat /var/emulab/boot/mydomain`"/>
    </services>
  </node>
</rspec>

We request a ProtoGENI slice and sliver. We use omni to do this.

$ omni.py -a pg createslice PG_SLICE
$ onmi.py -n -a https://www.emulab.net/protogeni/xmlrpc/am createsliver PG_SLICE pg.rspec

If you get a "server WAY too busy" error, please wait a few moments and try again.

Start Gush!

We are ready to start Gush and make sure our resource pools are being populated correctly. Try this:

geni@geni-vm:~/gush$ ./gush -P 15000
gush> Gush has learned about the slice jeannieslicePG.
Updated information on the slice jeannieslicePG is available.

gush> info nodes
There are 2 known nodes:
[ P         ] jeannie@pc279.emulab.net:61600(pref=0) (Disconnected.)
[ P         ] jeannie@pc321.emulab.net:61600(pref=0) (Disconnected.)

gush> connect jeannie@pc279.emulab.net
Connecting to host jeannie@pc279.emulab.net:61600.

gush> jeannie@pc279.emulab.net:61600 has joined the mesh.

gush> info mesh
Mesh:
jeannie@pc279.emulab.net:61600: connected; (fds=7,7 last_heard=1338317610)
Summary: 1 mesh members; 0 nodes connecting.

gush> disconnect
gush> jeannie@pc279.emulab.net:61600 has decided to leave the mesh.

gush> quit

Now, edit directory.xml so that the bottom part is commented out instead of the top. This enables PlanetLab and disables ProtoGENI. Then rerun Gush:

geni@geni-vm:~/gush$ ./gush -P 15000
gush> Gush has learned about the slice pgenigpolabbbncom_jeannieslice.
Updated information on the slice pgenigpolabbbncom_jeannieslice is available.

gush> info nodes
There are 5 known nodes:
[ P         ] pgenigpolabbbncom_jeannieslice@plnode-03.gpolab.bbn.com:61500(pref=0) (Disconnected.)
[ P         ] pgenigpolabbbncom_jeannieslice@planetlab2.cs.cornell.edu:61500(pref=0) (Disconnected.)
[ P         ] pgenigpolabbbncom_jeannieslice@planetlab2.umassd.edu:61500(pref=0) (Disconnected.)
[ P         ] pgenigpolabbbncom_jeannieslice@planetlab1.williams.edu:61500(pref=0) (Disconnected.)
[ P         ] pgenigpolabbbncom_jeannieslice@planetlab3.williams.edu:61500(pref=0) (Disconnected.)

gush> connect pgenigpolabbbncom_jeannieslice@planetlab1.williams.edu
Connecting to host pgenigpolabbbncom_jeannieslice@planetlab1.williams.edu:61500.

gush> pgenigpolabbbncom_jeannieslice@planetlab1.williams.edu:61500 has joined the mesh.

gush> info mesh
Mesh:
pgenigpolabbbncom_jeannieslice@planetlab1.williams.edu:61500: connected; (fds=7,7 last_heard=1338316464)
Summary: 1 mesh members; 0 nodes connecting.

gush> disconnect
gush> pgenigpolabbbncom_jeannieslice@planetlab1.williams.edu:61500 has decided to leave the mesh.

gush> quit

Useful Commands

The following commands are useful for manipulating remote resources using Gush.

 gush> connect <hostname>: Start a Gush client on a remote host.
 gush> connect pat <regex> <num>: Connect to <num> hosts that match the regular expression pattern <regex>.
 gush> disconnect: Close all open connections.
 gush> disconnect <hostname>: Disconnect from a host.
 gush> info nodes: Print summary information on all known nodes.
 gush> info node <hostname>: Print information about a node.
 gush> info mesh: Print the mesh status (membership).
 gush> shell "<quoted string>": Run the string as a shell command on all connected nodes.
 gush> quit: Quit Gush.

The following commands are useful when running experiments with Gush.

 gush> prefer <regex>: Tell Gush to increase preference value for nodes matching regex.  Nodes with higher preference values are used first.
 gush> load <file.xml>: Read a project file.
 gush> run: Start executing the first experiment in the active project.
 gush> info control: Print controller state information.
 gush> debug fail node <hostname>: Mark node as failed and find replacement (only during experiment).
 gush> save prefer <prefer.xml>: Save current host preferences (can load them back in later).

Always remember to disconnect from resources when finished running an experiment. If you forget, you can reconnect and run "killall -9 client" to clean things up. Also, periodically clean up your local Gush directory by removing logfiles (rm *logfile*).

Experimentation

Now we are ready to try some simple examples. Check out the "tutorial examples quick links" on the top of this page for more info.