Context Navigation

Home | About Gush | Using Gush | Gush Examples | Gush Tutorial at GEC 11 | GENI Summer Camp

Using ProtoGENI and PlanetLab with Gush (separate credentials)

In this example, we walk through our SimpleExample again. However this time we will copy our tarball to 1 PlanetLab host and 1 ProtoGENI host, untar it, and then cat the file included. This simple application goes over the basic building blocks needed to manage applications using Gush on both the PlanetLab and ProtoGENI frameworks. Note that this example assumes that you have accounts/credentials for both PlanetLab and ProtoGENI. If you want to use ProtoGENI resources uses only PlanetLab credentials, please see the ProtoGeniExampleTwo.

The following files are needed to run this application and are described below. They must be modified slightly (look for machine names and user names) to run on your machines.

omni_config
directory.xml
simple.xml
gush.prefs (see SimpleExample)
software.tar (see SimpleExample)

Configuring Gush for ProtoGENI

First our omni_config file must be modified to include information about both the PlanetLab and ProtoGENI aggregate managers. My omni_config file is shown below as an example. For more information on what each of these parameters mean in Omni, please see the OmniConfiguration page.

[omni]
users = jeannie

[plc]
type=sfa
authority=plc.williams
user=plc.williams.jeannie
cert=~/.gcf/plc.williams.jeannie.gid
key=~/.gcf/jeannie.pem
registry=http://www.planet-lab.org:12345
slicemgr=http://www.planet-lab.org:12347

[pg]
type=pg
ch = https://www.emulab.net:443/protogeni/xmlrpc/ch
sa = https://www.emulab.net:443/protogeni/xmlrpc/sa
cert = ~/.ssl/encrypted.pem
key = ~/.ssl/encrypted.pem

[jeannie]
urn=urn:publicid:IDN+plc:williams+user+jeannie
keys=~/.ssh/id_rsa.pub

Before using Gush for ProtoGENI, you must create a slice and sliver. There are many ways to do this. If you want to use the scripts included with Gush, the following technique can be used:

Get an account on Emulab and download your key/certificate (~/.ssl/encrypted.pem). Detailed instructions can be found on the ProtoGENI page. You should also remove the passphrase on the key. The best way to do this is to download your key from the Emulab webpage, and then run:
```
openssl rsa -in encrypted.pem -out keyout.pem
```

to generate a passphrase-less key. Note that you will also have to copy your certificate back into the keyout.pem file. When you are finished, rename keyout.pem to be encrypted.pem. If you cat the file, it should have two parts. The top half should be your RSA key, and the bottom should be your certificate.

Create a ProtoGENI slice. One way to do this (assuming you have set up omni_config as described above) is to run:

$ helper-scripts/handle-geni.py -f pg -a https://www.emulab.net/protogeni/xmlrpc/am createslice jeannie
INFO:omni:Loading config file omni_config
INFO:omni:Using control framework pg
Created slice with Name jeannie, URN urn:publicid:IDN+emulab.net+slice+jeannie

Create an RSPec that will be used to create your sliver. Details and samples can be found on the RSpecExamples page. Here is one that I used (called myslice.rspec) to create two nodes with Fedora 10:

<?xml version="1.0" encoding="UTF-8"?>
<rspec xmlns="http://www.protogeni.net/resources/rspec/0.1"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.protogeni.net/resources/rspec/0.1 http://www.protogeni.net/resources/rspec/0.1/request.xsd"
       type="request" >
 <node virtual_id="geni1" 
       exclusive="1"
       virtualization_type="emulab-vnode"
       virtualization_subtype="raw">
   <disk_image
      name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD" />
 </node>
 <node virtual_id="geni2" 
       exclusive="1"
       virtualization_type="emulab-vnode"
       virtualization_subtype="raw">
   <disk_image
      name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD" />
 </node>
</rspec>

Here is the same request using RSpec V2.

<?xml version="1.0" encoding="UTF-8"?>
<rspec type="request" xsi:schemaLocation="http://www.protogeni.net/resources/rspec/2
                                          http://www.protogeni.net/resources/rspec/2/request.xsd"
       xmlns:flack="http://www.protogeni.net/resources/rspec/ext/flack/1"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns="http://www.protogeni.net/resources/rspec/2">
  <node client_id="geni1" component_manager_id="urn:publicid:IDN+emulab.net+authority+cm" exclusive="true">
    <sliver_type name="raw-pc">
      <disk_image name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD"/>
    </sliver_type>
  </node>
  <node client_id="geni2" component_manager_id="urn:publicid:IDN+emulab.net+authority+cm" exclusive="true">
    <sliver_type name="raw-pc">
      <disk_image name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD"/>
    </sliver_type>
  </node>
</rspec>

Create your sliver. One way to do this is to run:

$ helper-scripts/handle-geni.py -n -f pg -a https://www.emulab.net/protogeni/xmlrpc/am createsliver jeannie myslice.rspec
INFO:omni:Loading config file omni_config
INFO:omni:Using control framework pg
Asked https://www.emulab.net/protogeni/xmlrpc/am to reserve resources. Result:
<?xml version="1.0" ?><rspec type="request" xmlns="http://www.protogeni.net/resources/rspec/0.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.protogeni.net/resources/rspec/0.1 http://www.protogeni.net/resources/rspec/0.1/request.xsd">  
   <node component_manager_urn="urn:publicid:IDN+emulab.net+authority+cm" component_manager_uuid="28a10955-aa00-11dd-ad1f-001143e453fe" component_urn="urn:publicid:IDN+emulab.net+node+pc293" component_uuid="de9f23c5-773e-102b-8eb4-001143e453fe" exclusive="1" hostname="pc293.emulab.net" sliver_urn="urn:publicid:IDN+emulab.net+sliver+37780" sliver_uuid="d8f35bb8-5d2e-11e0-b35d-001143e453fe" sshdport="22" virtual_id="geni1" virtualization_subtype="raw" virtualization_type="raw">    
       <disk_image name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD"/>    
       <services>      
         <login authentication="ssh-keys" hostname="pc293.emulab.net" port="22" username="jeannie"/>    
       </services>  
   </node>  
   <node component_manager_urn="urn:publicid:IDN+emulab.net+authority+cm" component_manager_uuid="28a10955-aa00-11dd-ad1f-001143e453fe" component_urn="urn:publicid:IDN+emulab.net+node+pc314" component_uuid="de9ec775-773e-102b-8eb4-001143e453fe" exclusive="1" hostname="pc314.emulab.net" sliver_urn="urn:publicid:IDN+emulab.net+sliver+37781" sliver_uuid="d98b035d-5d2e-11e0-b35d-001143e453fe" sshdport="22" virtual_id="geni2" virtualization_subtype="raw" virtualization_type="raw">    
       <disk_image name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD"/>    
       <services>      
         <login authentication="ssh-keys" hostname="pc314.emulab.net" port="22" username="jeannie"/>    
       </services>  
   </node>  
</rspec>

Environment configuration: directory.xml

directory.xml defines the Gush resource directory. This file tells the Gush default matcher where to locate resources during execution. In the following example, we specify PlanetLab (type=geni-plc) and ProtoGENI (type=geni-pg) slices available to us, and define a mapping between slice names and port numbers.

<?xml version="1.0" encoding="UTF-8"?>
<gush>
    <resource_manager type="geni-plc">
      <port_map slice="williams_gush" port="61500"/>
    </resource_manager>
    <resource_manager type="geni-pg">
      <port_map slice="jeannie" port="61600"/>
    </resource_manager>
</gush>

Gush preferences: gush.prefs

The gush.prefs file defines basic preferences that customize the way applications are run. In this example we specify the ClientPrefix and ClientPath which control how the client is started ({ClientPrefix}/client) and what working directory the client uses when executing commands (chdir {ClientPath}).

<preferences>
  <pref key="ClientPrefix">./</pref>
  <pref key="ClientPath">./</pref>
  <pref key="UsePlanetLab">true</pref>
</preferences>

Application Description: simple.xml

simple.xml is a Gush application description file. At the highest level, we define the project, which in this case is called "simple." Next we define any software packages that are needed. Gush application descriptions are comprised of different types of "blocks."

<gush>
  <project name="simple">
    <software name="SimpleSoftwareName" type="none">
      <package name="Package" type="web">
        <path>http://sysnet.cs.williams.edu/~jeannie/software.tar</path>
        <dest_path>software.tar</dest_path>
      </package>
    </software>
    <component name="Cluster1">
      <rspec>
        <num_hosts>2</num_hosts>
      </rspec>
      <software name="SimpleSoftwareName" />
      <resources>
        <resource type="geni-plc" group="williams_gush" /> 
        <resource type="geni-pg" group="jeannie" />
      </resources>
    </component>
    <experiment name="simple">
      <execution>
        <component_block name="cb1">
          <component name="Cluster1" />
          <process_block name="p2">
            <process name="cat">
              <path>cat</path>
              <cmdline>
                <arg>software.txt</arg>
              </cmdline>
              <cwd />
            </process>
          </process_block>
        </component_block>
      </execution>
    </experiment>
  </project>
</gush>

Start Gush

Now we are ready to run our example. The following shows the output from a run using two clients. You only need to type the commands that appear after "gush>".

Note that there are a few extra steps here to ensure that 1) we get both a PlanetLab and ProtoGENI host, and 2) we connect to the ProtoGENI host in advance. The reason we must connect in advance is due to a strange property of ProtoGENI hostnames. By connecting to resources before running the experiment, we avoid confusing the Gush controller. (Alternatively, you can follow the steps described in ProtoGeniFixHostname. This is recommended.)

albrecht:trunk jeannie$ ./gush -P 15000
gush> 
Gush has learned about the slice williams_gush.
Gush has learned about the slice jeannie.

gush> 
Updated information on the slice williams_gush is available.
Updated information on the slice jeannie is available.

gush> prefer planetlab1.williams

gush> info nodes
There are 10 known nodes:
[ P         ] williams_gush@planetlab1.ucsd.edu:61500(pref=0) (Disconnected.)
[ P         ] williams_gush@planetlab2.ucsd.edu:61500(pref=0) (Disconnected.)
[ P         ] williams_gush@planetlab3.ucsd.edu:61500(pref=0) (Disconnected.)
[ P         ] williams_gush@planetlab1.williams.edu:61500(pref=10) (Disconnected.)
[ P         ] williams_gush@planetlab2.williams.edu:61500(pref=0) (Disconnected.)
[ P         ] williams_gush@planetlab3.williams.edu:61500(pref=0) (Disconnected.)
[ P         ] williams_gush@planetlab4.williams.edu:61500(pref=0) (Disconnected.)
[ P         ] williams_gush@planetlab5.williams.edu:61500(pref=0) (Disconnected.)
[ P         ] jeannie@pc49.emulab.net:61600(pref=0) (Disconnected.)
[ P         ] jeannie@pc63.emulab.net:61600(pref=0) (Disconnected.)

gush> connect pat emulab 1
Initiated connections to 1 of 1 hosts.
jeannie@geni2.slice5675.genislices.emulab.net:61600 has joined the mesh.

gush> load tests/simple.xml
Project "simple" is selected.
Experiment "simple" is selected.

gush> run
Starting experiment run.
Running experiment simple...

The configuration matcher has finished matching.
The resource allocator has finished successfully.

The file transfer of Package to geni2.slice5675.genislices.emulab.net has been completed.
The software installation of Package on geni2.slice5675.genislices.emulab.net was successful.

williams_gush@planetlab1.williams.edu:61500 has joined the mesh.
The file transfer of Package to planetlab1.williams.edu has been completed.
The software installation of Package on planetlab1.williams.edu was successful.

williams_gush@planetlab1.williams.edu:61500,20930: Hello World

jeannie@geni2.slice5675.genislices.emulab.net:61600,3724: Hello World

The experiment has ended.

gush> disconnect
williams_gush@planetlab1.williams.edu:61500 has decided to leave the mesh.
jeannie@geni2.slice5675.genislices.emulab.net:61600 has decided to leave the mesh.

gush> quit

Download in other formats:

Plain Text