Context Navigation

Home | About Gush | Using Gush | Gush Examples | Gush Tutorial at GEC 11 | GENI Summer Camp

Using ProtoGENI and PlanetLab with Gush (only PlanetLab credentials)

In this example, we walk through our SimpleExample yet again. As in ProtoGeniExample, we will copy our tarball to 1 PlanetLab host and 1 ProtoGENI host, untar it, and then cat the file included. This simple application goes over the basic building blocks needed to manage applications using Gush on both the PlanetLab and ProtoGENI frameworks. Unlike ProtoGeniExample, this example assumes that you are using PlanetLab credentials to create and manage slices and slivers on ProtoGENI and PlanetLab.

The following files are needed to run this application and are described below. They must be modified slightly (look for machine names and user names) to run on your machines.

omni_config
directory.xml
simple.xml
gush.prefs (see SimpleExample)
software.tar (see SimpleExample)

Configuring Gush for ProtoGENI

First our omni_config file must be modified to include information about our PlanetLab credentials. My omni_config file is shown below as an example. For more information on what each of these parameters mean in Omni, please see the OmniConfiguration page.

[omni]
users = jeannie

[plc]
type=sfa
authority=plc.williams
user=plc.williams.jeannie
cert=~/.gcf/plc.williams.jeannie.gid
key=~/.gcf/jeannie.pem
registry=http://www.planet-lab.org:12345
slicemgr=http://www.planet-lab.org:12347

[jeannie]
urn=urn:publicid:IDN+plc:williams+user+jeannie
keys=~/.ssh/id_rsa.pub

Before using Gush for ProtoGENI, you must create a sliver. There are many ways to do this. If you want to use the scripts included with Gush, the following technique can be used:

Create an RSPec that will be used to create your sliver. Details and samples can be found on the RSpecExamples page. Here is one that I used (called myslice.rspec) to create two nodes with Fedora 10:

<?xml version="1.0" encoding="UTF-8"?>
<rspec xmlns="http://www.protogeni.net/resources/rspec/0.1"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.protogeni.net/resources/rspec/0.1 http://www.protogeni.net/resources/rspec/0.1/request.xsd"
       type="request" >
 <node virtual_id="geni1" 
       exclusive="1"
       virtualization_type="emulab-vnode"
       virtualization_subtype="raw">
   <disk_image
      name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD" />
 </node>
 <node virtual_id="geni2" 
       exclusive="1"
       virtualization_type="emulab-vnode"
       virtualization_subtype="raw">
   <disk_image
      name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD" />
 </node>
</rspec>

Here is the same request using RSpec V2.

<?xml version="1.0" encoding="UTF-8"?>
<rspec type="request" xsi:schemaLocation="http://www.protogeni.net/resources/rspec/2
                                          http://www.protogeni.net/resources/rspec/2/request.xsd"
       xmlns:flack="http://www.protogeni.net/resources/rspec/ext/flack/1"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns="http://www.protogeni.net/resources/rspec/2">
  <node client_id="geni1" component_manager_id="urn:publicid:IDN+emulab.net+authority+cm" exclusive="true">
    <sliver_type name="raw-pc">
      <disk_image name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD"/>
    </sliver_type>
  </node>
  <node client_id="geni2" component_manager_id="urn:publicid:IDN+emulab.net+authority+cm" exclusive="true">
    <sliver_type name="raw-pc">
      <disk_image name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD"/>
    </sliver_type>
  </node>
</rspec>

Create your sliver using your existing PlanetLab slice. Assuming your slice is williams_gush, one way to do this is to run the following command. Notice how the "williams_" prefix of the slice name is dropped:

$ helper-scripts/handle-geni.py -n -f plc -a https://www.emulab.net/protogeni/xmlrpc/am createsliver gush myslice.rspec
INFO:omni:Loading config file omni_config
INFO:omni:Using control framework plc
INFO:omni.sfa:SFA Registry: http://www.planet-lab.org:12345
INFO:omni.sfa:SFA Slice Manager: http://www.planet-lab.org:12347
Asked https://www.emulab.net/protogeni/xmlrpc/am to reserve resources. Result:
<?xml version="1.0" ?><rspec type="manifest" xmlns="http://www.protogeni.net/resources/rspec/2" xmlns:flack="http://www.protogeni.net/resources/rspec/ext/flack/1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.protogeni.net/resources/rspec/2                                           http://www.protogeni.net/resources/rspec/2/manifest.xsd">  
    <node client_id="geni1" component_id="urn:publicid:IDN+emulab.net+node+pc82" component_manager_id="urn:publicid:IDN+emulab.net+authority+cm" exclusive="true" sliver_id="urn:publicid:IDN+emulab.net+sliver+45415">    
        <sliver_type name="raw">      
            <disk_image name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD"/>      
        </sliver_type>    
        <services>      
          <login authentication="ssh-keys" hostname="pc82.emulab.net" port="22" username="jeannie"/>    </services>    
      <rs:vnode name="pc82" xmlns:rs="http://www.protogeni.net/resources/rspec/ext/emulab/1"/>  </node>  
    <node client_id="geni2" component_id="urn:publicid:IDN+emulab.net+node+pc68" component_manager_id="urn:publicid:IDN+emulab.net+authority+cm" exclusive="true" sliver_id="urn:publicid:IDN+emulab.net+sliver+45416">    
        <sliver_type name="raw">      
            <disk_image name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD"/>      
        </sliver_type>    
        <services>      
          <login authentication="ssh-keys" hostname="pc68.emulab.net" port="22" username="jeannie"/>    </services>    
      <rs:vnode name="pc68" xmlns:rs="http://www.protogeni.net/resources/rspec/ext/emulab/1"/>  </node>  
</rspec>

Environment configuration: directory.xml

directory.xml defines the Gush resource directory. This file tells the Gush default matcher where to locate resources during execution. In the following example, we specify PlanetLab (type=geni-plc) and ProtoGENI (type=geni-pg) slices available to us, and define a mapping between slice names and port numbers. Notice how we specify the "cf=plc" for our ProtoGENI resources. This is how we are able to use our PlanetLab credentials to create ProtoGENI resources.

<?xml version="1.0" encoding="UTF-8"?>
<gush>
    <resource_manager type="geni-plc">
      <port_map slice="williams_gush" port="61500"/>
    </resource_manager>
    <resource_manager type="geni-pg" cf="plc">
      <port_map slice="gush" port="61600"/>
    </resource_manager>
</gush>

Gush preferences: gush.prefs

The gush.prefs file defines basic preferences that customize the way applications are run. In this example we specify the ClientPrefix and ClientPath which control how the client is started ({ClientPrefix}/client) and what working directory the client uses when executing commands (chdir {ClientPath}).

<preferences>
  <pref key="ClientPrefix">./</pref>
  <pref key="ClientPath">./</pref>
  <pref key="UsePlanetLab">true</pref>
</preferences>

Application Description: simple.xml

simple.xml is a Gush application description file. At the highest level, we define the project, which in this case is called "simple." Next we define any software packages that are needed. Gush application descriptions are comprised of different types of "blocks."

<gush>
  <project name="simple">
    <software name="SimpleSoftwareName" type="none">
      <package name="Package" type="web">
        <path>http://sysnet.cs.williams.edu/~jeannie/software.tar</path>
        <dest_path>software.tar</dest_path>
      </package>
    </software>
    <component name="Cluster1">
      <rspec>
        <num_hosts>2</num_hosts>
      </rspec>
      <software name="SimpleSoftwareName" />
      <resources>
        <resource type="geni-plc" group="williams_gush" /> 
        <resource type="geni-pg" group="gush" />
      </resources>
    </component>
    <experiment name="simple">
      <execution>
        <component_block name="cb1">
          <component name="Cluster1" />
          <process_block name="p2">
            <process name="cat">
              <path>cat</path>
              <cmdline>
                <arg>software.txt</arg>
              </cmdline>
              <cwd />
            </process>
          </process_block>
        </component_block>
      </execution>
    </experiment>
  </project>
</gush>

Start Gush

Now we are ready to run our example. The following shows the output from a run using two clients. You only need to type the commands that appear after "gush>".

Note that there are a few extra steps here to ensure that 1) we get both a PlanetLab and ProtoGENI host, and 2) we connect to the ProtoGENI host in advance. The reason we must connect in advance is due to a strange property of ProtoGENI hostnames. By connecting to resources before running the experiment, we avoid confusing the Gush controller. (Alternatively, you can follow the steps described in ProtoGeniFixHostname.)

albrecht:trunk jeannie$ ./gush -P 15000
gush> 
Gush has learned about the slice williams_gush.
Gush has learned about the slice gush.

gush> 
Updated information on the slice williams_gush is available.
Updated information on the slice gush is available.

gush> prefer planetlab1.williams

gush> info nodes
There are 10 known nodes:
[ P         ] williams_gush@planetlab1.ucsd.edu:61500(pref=0) (Disconnected.)
[ P         ] williams_gush@planetlab2.ucsd.edu:61500(pref=0) (Disconnected.)
[ P         ] williams_gush@planetlab3.ucsd.edu:61500(pref=0) (Disconnected.)
[ P         ] williams_gush@planetlab1.williams.edu:61500(pref=10) (Disconnected.)
[ P         ] williams_gush@planetlab2.williams.edu:61500(pref=0) (Disconnected.)
[ P         ] williams_gush@planetlab3.williams.edu:61500(pref=0) (Disconnected.)
[ P         ] williams_gush@planetlab4.williams.edu:61500(pref=0) (Disconnected.)
[ P         ] williams_gush@planetlab5.williams.edu:61500(pref=0) (Disconnected.)
[ P         ] jeannie@pc481.emulab.net:61600(pref=0) (Disconnected.)
[ P         ] jeannie@pc491.emulab.net:61600(pref=0) (Disconnected.)

gush> connect pat emulab 1
Initiated connections to 1 of 1 hosts.
jeannie@geni2.slice5675.genislices.emulab.net:61600 has joined the mesh.

gush> load tests/simple-pg.xml
Project "simple" is selected.
Experiment "simple" is selected.

gush> run
Starting experiment run.
Running experiment simple...

The configuration matcher has finished matching.
The resource allocator has finished successfully.

The file transfer of Package to geni2.slice5675.genislices.emulab.net has been completed.
The software installation of Package on geni2.slice5675.genislices.emulab.net was successful.

williams_gush@planetlab1.williams.edu:61500 has joined the mesh.
The file transfer of Package to planetlab1.williams.edu has been completed.
The software installation of Package on planetlab1.williams.edu was successful.

williams_gush@planetlab1.williams.edu:61500,20930: Hello World

jeannie@geni2.slice5675.genislices.emulab.net:61600,3724: Hello World

The experiment has ended.

gush> disconnect
williams_gush@planetlab1.williams.edu:61500 has decided to leave the mesh.
jeannie@geni2.slice5675.genislices.emulab.net:61600 has decided to leave the mesh.

gush> quit

Deleting Our Sliver

When we are finished with our experiment, we can delete our sliver using the Gush helper script:

$ helper-scripts/handle-geni.py -n -f plc -a https://www.emulab.net/protogeni/xmlrpc/am deletesliver gush myslice.rspec 
INFO:omni:Loading config file omni_config
INFO:omni:Using control framework plc
INFO:omni.sfa:SFA Registry: http://www.planet-lab.org:12345
INFO:omni.sfa:SFA Slice Manager: http://www.planet-lab.org:12347
Deleted sliver urn:publicid:IDN+plc:williams+slice+gush on unspecified_AM_URN at https://www.emulab.net/protogeni/xmlrpc/am

Download in other formats:

Plain Text