Sponsored by the National Science Foundation

GMOC Support for GENI Racks

Kevin Bohan, GMOC

Eldar Urumbaev, GMOC

Sponsored by the National Science Foundation 2October 24, 2012

Topics

• Meta-Operations Overview• What’s Changed• Code Examples• What’s Connected• Future Directions

Sponsored by the National Science Foundation 3October 24, 2012

Introduction

• Core set of operational data needed to jointly run the network

• Monitoring of aggregates, resources, and circuits that make up GENI

• Display operational data in a web UI– For Operators– For Experimenters

• Orchestrate Emergency Stop

Sponsored by the National Science Foundation 4October 24, 2012

Meta-Operations Model

Campus ACampus AAggregates/Resources

Meta-operationsDatabase

Slice AuthoritySlice Sliver

Resource

SliverResource

Slice

User Interface

Sponsored by the National Science Foundation 5October 24, 2012

What’s Changed

• API called GMOC Objects– Makes it easier to report data to GMOC– Python module

• Use of URNs• Aggregate Naming• Support for Circuits

Sponsored by the National Science Foundation 6October 24, 2012

GMOC Objects

• Models the state of things in GENI• Loosely couples submission of metadata and

time-series data• Allows partial data submission• Can be instantiated anywhere• Much easier to use than previous solutions

Sponsored by the National Science Foundation 7October 24, 2012

Modeled Network Elements

• POP• Organization• Aggregate• Sliver• Resource• Interface

– IPv4, IPv6, MAC address

– VLANs

• Slice Authority• Slice• Network• Circuit

– Endpoint interfaces– Circuit type

• 1/10/40/100 Gigabit Ethernet

• 802.11• WiMAX

– Provisioned bandwidth

Sponsored by the National Science Foundation 8October 24, 2012

Operational Measurement Objects

• CPUUtilization• VMCount• TargetPingable• AMAPIListResources• AMAPIGetVersion

• DiskUtilization• OpenFlowSliverStats• SliverState• DatapathStats• NetworkStats

Sponsored by the National Science Foundation 9October 24, 2012

Changing Aggregate State

import gmoc

pop = gmoc.POP(“urn:publicid:IDN+gmoc.geni.net+pop+MOXI-Indiana”)

agg = gmoc.Aggregate(“moxifoam.ictc.indiana.gigapop.net:3626”, pop)

agg.state = gmoc.AM_STATE_DOWNclient.store(agg)

Sponsored by the National Science Foundation 10October 24, 2012

Adding a Resource

import gmoc

pop = gmoc.POP(“urn:publicid:IDN+gmoc.geni.net+pop+MOXI-Indiana”)

agg = gmoc.Aggregate(“moxifoam.ictc.indiana.gigapop.net:3626”, pop)

res = gmoc.Resource(“urn:publicid:IDN+openflow:foam:foam7.gpolab.bbn.com+datapath+04:4d:00:12:e2:b8:a5:d0”, “datapath”, pop, agg)

client.store(agg)

Sponsored by the National Science Foundation 11October 24, 2012

Manual Measurements

import gmoc

pop = gmoc.POP(“urn:publicid:IDN+gmoc.geni.net+pop+MOXI-Indiana”)

agg = gmoc.Aggregate(“moxifoam.ictc.indiana.gigapop.net:3626”, pop)

cpu = CPUUtilization()cpu.addData(1349725335, { "cpu_idle": 3.1 })

agg.addMeasurement(cpu)

client.storeMeasurement(pop)

Sponsored by the National Science Foundation 12October 24, 2012

Measurement from RRDs

import gmoc

pop = gmoc.POP(“urn:publicid:IDN+gmoc.geni.net+pop+MOXI-Indiana”)

agg = gmoc.Aggregate(“moxifoam.ictc.indiana.gigapop.net:3626”, pop)

cpu = CPUUtilization()cpu.loadRRD(”/array/moxifoam_node_cpu_idle.rrd”, 1318102932, 1349729656)

agg.addMeasurement(cpu)

client.storeMeasurement(pop)

Sponsored by the National Science Foundation 13October 24, 2012

What’s Connected?

• 20 FOAM aggregates• pgeni SA• GENI Clearinghouse & Portal• ExoGENI

– Metadata– Operational monitoring

• InstaGENI– Metadata– Some operational monitoring

Sponsored by the National Science Foundation 14October 24, 2012

GMOC UI Screenshot

Sponsored by the National Science Foundation 15October 24, 2012

Future Directions?

• Populate the API with data from GMOC– Query current operational data about GENI things

• More measurements• RSpec parser/generator

– Monitoring extension

• Additional languages?• Integration with Active Measurement projects• Use Circuit data operationally

Sponsored by the National Science Foundation 16October 24, 2012

GMOC Support for GENI Racks

• Campus/Racks Team contacts GMOC to announce Outage or upcoming Maintenance- Initially important to include very specific information

(Campus, type of outage/maintenance, what’s affected, outage/maintenance times, who’s working to perform maintenance or resolving an outage, ETR, etc…)

- Eventually through pro-active monitoring. - Contact GMOC as Racks come up.

Sponsored by the National Science Foundation 17October 24, 2012

GMOC Support for GENI Racks

• GMOC tracks racks/campus events and notifies the GENI community:– Initially through the GENI Ops list and Ops Calendars

(Web, RSS, iCal)– Long-term goal is to notify individual GENI

Experimenters via targeted notifications based on which Aggregate is affected and slice ownership

Sponsored by the National Science Foundation 18October 24, 2012

GMOC Operations Calendars

Sponsored by the National Science Foundation 19October 24, 2012

Contact GMOC

• Email: GMOC@GRNOC.IU.EDU• Phone: 317-274-7783• Website: http://gmoc.grnoc.iu.edu• GENI Ops Calendars (Web, RSS, iCal):

http://gmoc.grnoc.iu.edu/gmoc/index/support/gmoc-operations-calendars.html

• GENI Bi-Weekly Operations Report: http://gmoc.grnoc.iu.edu/gmoc/index/support/gmoc-reports/gmoc-reports.html

Sponsored by the National Science Foundation 20October 24, 2012

Some discussion questions…

• How meaningful are current notifications?• How much does the GENI Community care about

a single rack/campus outage?• How important are notifications targeted to

individual experimenters/operators?• What else is important to the Community?

Sponsored by the National Science Foundation 21October 24, 2012

Contact & Links

• Kevin Bohan: kevin@iu.edu• Eldar Urumbaev: eurumbae@indiana.edu

• GMOC Database– https://gmoc-db.grnoc.iu.edu/protected

• GMOC Objects API Reference– https://gmoc-db.grnoc.iu.edu/public_docs/gmoc-objects-v1.p

df

• GMOC Objects Python Implementation– https://gmoc-db.grnoc.iu.edu/public_docs/gmoc.py