ExoGENI: tools and capabilities for developers

Ilya Baldin, Paul Ruth, Anirban Mandal, Jonathan Mills, Claris Castillo, Chris HeermannRENCI, UNC-CH

Jeff Chase, Victor OrlikowskiDuke University

Image snapshot script◦ Easy way to capture the ExoGENI image◦ Tested and ‘GPO approved’◦ Replaces ‘playpen’

Speaks-for in controllers New libraries for building tools

What’s new for GEC20?

Release ndlLib◦ Port Flukes to use ndlLib◦ Develop other applications with it

Image capture◦ Integrate with NEuca◦Make it work for bare-metal nodes◦Obsolete playpen – it was a temporary awkward solution.

Plans for GEC21

Part of upcoming ORCA 5.0◦ More storage slivering for

Cisco racks NetApp New IBM racks and Dell Racks with server-based storage With I/O isolation Add support to attach to bare-metal nodes

◦ Virtual wire support for OVS To improve network performance isolation for VMs

◦ Hybrid network support for switches Ethernet VLANs for those who care about performance OpenFlow for those who actually care about it

◦ Fully dynamic slices Add/remove links/nodes at runtime

◦ Inter-slice stitching

More on what’s next

Details on image snapshot utility Image portability with InstaGENI Turning tools into libraries ExoGENI Storage slivering ExoGENI Stitching

Overview

ExoGENI Image Background Images defined by XML metadata file◦ Image (AMI) - mandatory◦ Kernel (AKI) - optional◦ Ramdisk (ARI) - optional

Hosted on HTTP server◦ Permanent server at your institution◦ Soon will work with DropBox and Google Drive

Slice requests require◦ URL to metadata file – goes into the ‘name’ in RSpec◦ Hash of metadata file – goes into the ‘vesrion’ in RSpec

ExoGENI Image Background: Metadata<images> <image> <type>ZFILESYSTEM</type> <signature>9efee8fa9c6705455c87a42477bd4ebd04d39c81</signature> <url>http://geni-images.renci.org/images/standard/centos/centos6.3-v1.0.10.tgz</url> </image> <image> <type>KERNEL</type> <signature>5e76d034e27cb861275fc3d1bd9d97c3905ef5a2</signature> <url>http://geni-images.renci.org/images/standard/centos/kernels/vmlinuz-2.6.32-431.5.1.el6.x86_64</url> </image> <image>`

<type>RAMDISK</type><signature>c999527cc0d8739a3d3f38d48f03fa6c73151cca</signature><url>http://geni-images.renci.org/images/standard/centos/kernels/initramfs-

2.6.32-431.5.1.el6.x86_64.img</url> </image></images>

ExoGENI Image Snapshot Snapshot of running ExoGENI VM◦ Creates: image, kernel, ramdisk, metadata

High level steps◦ Create/modify a VM – update OS, install needed packages◦ Run the script, make sure you have enough disk space

Use ExoGENI storage slivering to help here (next topic)◦ Copy the new image files to an http server◦ Insert metadata URL and hash into a request

More details on the ExoBlog:http://www.exogeni.net/2014/06/creating-a-custom-image-from-an-

existing-virtual-machine/

Porting from EG to IG is a complicated and error-prone process

Porting from IG to EG has not yet been achieved Ultimately image snapshot utility should work for IG

nodes to capture image that works on EG

Image portability with IG

◦ Log into your VM

◦ GET EVEN MORE COFFEE

◦ Submit RSpec

◦ Your RSpec “disk_image” is the “Metadata URL”

◦ Create a new Rspec

Boot a VM

◦ Click: <Your Image Name>

◦ Click: Experimentation -> List ImageIDs

◦ Toggle “Global” to “Yes”

◦ GET COFFEE / WAIT FOR EMAIL FROM EMULAB DECLARING VICTORY

◦ Click “Submit”

Set types to only “pcvm”

Set “EC2 User@Node”: i.e. root@152.54.14.44

Set “Filename”: should be preset with project dir and “Description Name”. Example: “/proj/GENISciShakedown/images/Centos6.3.ndz”

Set “OS Version”: Version of the image??? I’m not sure what this is for.

Set “Operating System”: Linux

Set “Description”: choose a description

Set "Descriptor Name”: choose a name

◦ Click: “Import an Amazon EC2 Instance Image”

◦ Click: Experimentation -> List ImageIDs

◦ Log into www.emulab.net

Perform ec2 import

cat key to >> .ssh/authorized_keys”

Copy emulab keys into ExoGENI instance

Make user the “EC2 user@node" is in the shudders file (i.e. pruth@node )

yum install ruby zip unzip sudo e2fsprogs rsync

◦ setup for ec2 import

edit /usr/local/etc/emulab/rc/rc.mounts and change this line:

Create a file called /etc/emulab/genvmtype and stick in one line that says “xen”

Allow sudo to run without a tty. Add the following to the sudoers file: "Defaults !requiretty”

Add /users

Fix resov.conf and hostname

◦ Install client side tools

◦ Install pubsub

◦ yum install perl dhclient openssh mysql bison byacc flex boost git mysql-server mysql-devel mysql-embedded-devel mysql-proxy-devel libtool perl-Time-HiRes yum install python-devel httpd man httpd-devel dos2unix swig swig-devel boost-devel pcap-devel libpcap-devel mysql-devel db4-devel db4 libdb-devel compat-db tcsh

install Emulab tools

Modify the VM as you want.

◦ cleanup anything the post boot script did

◦ cleanup hostname

◦ cleanup sudoers

◦ remove users if desired

◦ remove public keys (from at least root)

◦ chkconfig neuca off

◦ /etc/init.d/neuca stop

Clean up VM

Boot a Centos VM on ExoGENI.

ExoGENI has a number of tools, that are now being converted into libraries for developing more tools

Existing and newly-developed libraries ◦ Written in Java◦ orca.ndl – NDL parsing and generation

Used by ORCA, Flukes, NDL converter◦ orca.ndl-conversion.core – Conversions between NDL and other formats

(GENI RSpec, PG Rspec, graph representations)◦ ndlLib – easy application-level slice management abstractions– in progress

Integration with pubsub for status notifications Slice visualization, graph layouts

Available from snapshot Maven repository◦ http://ci-dev.renci.org/nexus/content/repositories/geni-orca-snapshot/

Turning tools into libraries

Built on CouchDB REST APIs Registries for ◦ Images◦ Stitchports◦ Actors◦…

Registries

Overview of architecture and implementation

Storage Slivering

Storage Architecture

Slices with storage

Lun Creation Time EstimatesLun Size Lun Creation Time Disk Formatted (Accessible)

1 GB 25 sec Before login

2 GB 50 sec Before login

4 GB 1 min 50 sec Before login

8 GB 3 min 6 sec ~Time of login

16 GB 6 min 30 sec 3 min 48 sec

32 GB 12 min 51 sec 4 min 30 sec

64 GB 25 min 7 min 12 sec

128 GB 50 min 13 min 49 sec

256 GB 1 hour 49 min 30 mins

Unlike GENI stitching, ExoGENI stitching logic is◦ Integrated into ORCA `controller’ that speaks GENI AM API

Makes it easy to place requests for complex inter-domain topologies

◦ Integrated with ORCA resource brokering/ticketing system Provides reliable single-shot provisioning that (except when

OSCARS on AL2S or ESnet decides to fall over) We considered pulling it out of the controller and

making a toolkit Still in consideration, no immediate plans though

ExoGENI stitching

Reasons to use ExoGENI native stitching ExoGENI stitching◦ Any XO rack to any other XO rack transparently to the user

AL2S, ION, ESnet, BEN◦Works with various corner cases

Internal label translations by transit providers E.g. NICTA

Inability of AL2S to connect racks that share the same AL2S port E.g. FIU to UFL

◦Multipoint VLAN support – create large Layer 2 domains◦ Storage stitching support – create large slivers of iSCSI storage◦ Stitch ports – connect slices to the outside world

Ways to use it GENI Tools (omni, Flack, GENI Portal):+Works with other rack types◦ Limited topology embedding support

ExoGENI native tools:+Expose full range of capabilities+Multi-point+Storage slivering and stitching+Slice modifications (elastic slices)

+Use regular GENI credentials◦ Work only with ExoGENI racks

More information: +http://tinyurl.com/l5qsbwy+http://www.exogeni.net

http://www.exogeni.net http://wiki.exogeni.net

Thank you!