sponsored by the national science foundation train-the-admin sarah edwards, gpo june 22, 2014
TRANSCRIPT
Sponsored by the National Science Foundation
Train-the-Admin
Sarah Edwards, GPO
June 22, 2014
Sponsored by the National Science Foundation 2Train the Admin – GEC20 – June 21, 2014
Outline
• GENI in the classroom: Technical Issues• Who can use a GENI rack?• Rack Layout & Strengths• Inter-aggregate links• OpenFlow and FOAM• GENI Meta-operations (GMOC)• Tools for rack administration• Getting Help• Q&A
Sponsored by the National Science Foundation 3Train the Admin – GEC20 – June 21, 2014
GENI IN THE CLASSROOM: TECHNICAL ISSUES
Sponsored by the National Science Foundation 4Train the Admin – GEC20 – June 21, 2014
How can you help?
• Four areas where campus IT can help:#1 InCommon
#2 ssh (especially from Windows machines)
#3 GENI Client Tool access
#4 Known GENI Ports
Sponsored by the National Science Foundation 5Train the Admin – GEC20 – June 21, 2014
Access via the GENI Portal
For many experimenters:• no new passwords• familiar login screens
The GENI Portal leverages InCommon for single sign-on authentication
Experimenters from 304 educational and research institutions have InCommon accounts
GENI Project Office runs a federated IdP to provide accounts for non-federated organizations.
Portal needs certain attributes (more in minute)
#1
Sponsored by the National Science Foundation 6Train the Admin – GEC20 – June 21, 2014
What can you do?
• The GENI Portal gives access to real resources• Therefore, we need to be able to contact experimenters if
something goes wrong• GENI Portal requires:
– eppn (eduPersonPrincipalName)– e-mail address
• GENI Portal prefers to receive:– affiliation– given name– surname
• InCommon members can easily share these attributes by enabling the Research & Scholarship (R&S) category
R&S: https://spaces.internet2.edu/display/InCCollaborate/Research+and+Scholarship+Category
Sponsored by the National Science Foundation 7Train the Admin – GEC20 – June 21, 2014
Try logging in
Want to know if your institution is an InCommon member which shares the needed attributes?• The GENI Portal is at:
https://portal.geni.net • Click “Use GENI”• Pick your institution from the list• Login using your usual username and password• Does this work? You’re done• If not, e-mail [email protected]
– We will contact the appropriate person at your institution
Sponsored by the National Science Foundation 8Train the Admin – GEC20 – June 21, 2014
SSH from Windows
SSH with keypair from Windows is non-trivial– No built-in ssh client
Possible Solutions
http://groups.geni.net/geni/wiki/HowTo/LoginToNodes
Does your campus have a standard solution?
– BitVise – FireSSH – javascript plugin for Firefox– SecureCRT (not free)– cygwin– Linux VM – make use of a slim OS– PuTTy (private key format different)
#2
Sponsored by the National Science Foundation 9Train the Admin – GEC20 – June 21, 2014
Student access to client tools
• Three options:– Use their personal laptop
• LabZero is a good way to get setup• There are Mac/Windows Binaries for Omni
– Use Lab computers• Go through the exercises in lab computers• stress-test the resources or split students
– Use a VM with all the software loaded• http://groups.geni.net/geni/wiki/HowTo/CreateTutorialVM
Work with professors to determine how students will access client tools
#3
Sponsored by the National Science Foundation 10Train the Admin – GEC20 – June 21, 2014
Known GENI Ports
• GENI is: – highly distributed– runs many services on many unusual ports– clients sometimes demonstrate unusual behavior
• eg many SSL connections because contacting many aggregates, repeated ssh connections to the same host, etc
• As a result, we’ve found that some networks will block some legitimate GENI traffic– In particular, campus Guest WiFi frequently has this
problem
• Checkout the Known GENI Ports
Known GENI Ports: http://groups.geni.net/geni/wiki/KnownGENIPorts
#4
Sponsored by the National Science Foundation
First Exercise in GENI
□ Bulk-add students to project
Email Prework:□ GENI account□ Computer setup*□ Other?
1 week before
Class Prep:□ GENI Account Access□ Project for the Class□ Test Exercises□ Notify GMOC□ Figure Student Setup□ Email [email protected]
2 weeks before
(or sooner)
* Include steps for testing the setup
Instructor Timeline/Checklist
Instructor Checklist: http://groups.geni.net/geni/wiki/GENIEducation/Resources
Sponsored by the National Science Foundation 13Train the Admin – GEC20 – June 21, 2014
WHO CAN USE A GENI RACK?
Sponsored by the National Science Foundation 14Train the Admin – GEC20 – June 21, 2014
Who can use a GENI rack?
• Rack users can have accounts from one of three places:– GENI Portal/Clearinghouse (most common)– Emulab – PlanetLab
• In all three cases:– experimenters must be vouched for in order to access
GENI resources the first time– The operations groups coordinate to contact
experimenters if needed
Sponsored by the National Science Foundation 15Train the Admin – GEC20 – June 21, 2014
Slice credentials
GENI: Terms and Definitions• Clearinghouse
– Slice authority: Creates and registers slices– GENI slice authorities: GENI Portal, PlanetLab, ProtoGENI
• Aggregate: Provides resources to GENI experimenters– Typically owned and managed by an organization– Examples: GENI Racks, Internet2, Emulab, PlanetLab– Aggregates implement the GENI AM API
Create & Register Slice
Researcher
Portal/Clearinghouse
Aggregate Manager API - listResources - createSliver … Aggregate
ManagerAggregate Resources
Slice credentials
Sponsored by the National Science Foundation 16Train the Admin – GEC20 – June 21, 2014
GENI Portal/Clearinghouse
• The clearinghouse is a set of services to track:– experimenters, – projects, – slices, and – authorization
• The portal is a web-based user interface for experimenters to access the clearinghouse services and GENI aggregates– Accounts used in tutorials
• Anyone can get an account, but you must be a member of a project to do anything interesting
Sponsored by the National Science Foundation 17Train the Admin – GEC20 – June 21, 2014
Project Lead Policies
Projects organize research in GENI– Projects contain both
people and their
experiments– A project is led by a single
responsible individual: the project lead
– Who can be a project lead?• Academic Faculty
• Senior technical staff in non-academic environments
Project
Lead
Members
Slice
We can contact both the experimenter and the project lead
Sponsored by the National Science Foundation 18Train the Admin – GEC20 – June 21, 2014
Identifiers and URNs
• Slice URNs identify the issuing authority and the slice name– Use (URN, UUID) to uniquely identify slices over time– Exampleurn:publicid:IDN+ch.geni.net:tutorial+slice+monitor
• User URNs– Example
urn:publicid:IDN+ch.geni.net+user+sedwards
authority (GENI CH : project) slice name
authority username
Sponsored by the National Science Foundation 19Train the Admin – GEC20 – June 21, 2014
RACK LAYOUT & STRENGTHS
Sponsored by the National Science Foundation 21Train the Admin – GEC20 – June 21, 2014
InstaGENI Rack ExoGENI Rack
GENI Racks
Sponsored by the National Science Foundation 22Train the Admin – GEC20 – June 21, 2014
What is a GENI Rack?
A GENI Rack contains compute resources, an OpenFlow dataplane switch and speaks the GENI AM API.
Racks are connected to each other across campuses, regionals, and backbone providers
Production GENI racks are developed by two teams: ExoGENI (IBM/RENCI), InstaGENI (HP/University of Utah)
In addition, GENI racks are being prototyped by:• Dell (prototype at Clemson and GPO) and • Cisco (prototype at WVNet and NCSU)
Sponsored by the National Science Foundation 23Train the Admin – GEC20 – June 21, 2014
Getting a GENI Rack
Interested in getting a GENI Rack?
Email [email protected]
What’s involved?– High Level checklist for new racks from delivery to production release http
://groups.geni.net/geni/wiki/GENIRacksHome/RacksChecklist– InstaGENI requirements for configuring a rack http
://www.protogeni.net/wiki/instageni/checklist
Who has a production GENI aggregate? – http://groups.geni.net/geni/wiki/GENIProduction
– All aggregates (including racks) that have been tested and released for production use by experimenters.
– Includes "dev" racks (GPO, RENCI, Utah, Utah DDC, and Kansas)• "dev" designation indicates that rack might be taken down with less warning than others,
and might be running newer versions of software under test than others
Sponsored by the National Science Foundation 24Train the Admin – GEC20 – June 21, 2014©2010 HP Created on xx/xx/xxxxof 222
InstaGENI Rack Diagram
Diagram from InstaGENI slide deckJuly 10, 2012
Control plane switch
Data plane switch (OF)
Control node
Experiment nodes (5)
Expansion space(empty/PDUs)
bossops
foamflowvisor
Sponsored by the National Science Foundation 25Train the Admin – GEC20 – June 21, 2014
ExoGENI Rack Diagram
Diagram from ExoGENI cabling diagram
Front Back
Control plane switchData plane switch (OF)
Head node
Worker nodes (10)
VPN (Juniper SSG)iSCSI storage
Console
PDUs not shown
Sponsored by the National Science Foundation 26Train the Admin – GEC20 – June 21, 2014
Details: Hardware specs
InstaGENI ExoGENI *
Nodes per rack 5 + 1 control node 10 workers + 1 head
Cores per rack 60 120
Network interfaces 4x 1Gbit (upgradeable to 10 Gbps)
2x 10Gbit with SR-IOV (upgradeable to 40 Gbps)
Storage 1 TB local 150GB+500 GB local + 6 TB SAN (upgradeable)
Dataplane Switch
HP ProCurve 5406 (VLAN-based OpenFlow)
IBM G8264R (Port-based OpenFlow)
* Listed are the specs for ExoGENI IBM-based racks.
There are 110V and 220V versions of each rack
Sponsored by the National Science Foundation 27Train the Admin – GEC20 – June 21, 2014
Compute resources
• Bare-metal host:– “expensive” resource, limits the lifetime of your
experiment
– ExoGENI : 2 bare-metal hosts per rack
– InstaGENI : 2 bare-metal hosts per rack
Experimenters should use VMs unless they have a very good reason
• ExoGENI : KVM VMs
• InstaGENI, ProtoGENI: linux containers, XEN VMs
Sponsored by the National Science Foundation 28Train the Admin – GEC20 – June 21, 2014
Networking
• Speed of the NICs:– ExoGENI: 10Gbps (2x per host) (upgradeable to 40
Gbps)– InstaGENI: 1Gbps (4x per host) (upgradeable to 10
Gbps)
• Sharing:– ExoGENI: Virtual NICs share physical 10G NICs and
are bandwidth-provisioned via OVS among VMs unless using bare-metal hosts
– InstaGENI: Can get dedicated NICs even on shared nodes, but only 1Gb
Sponsored by the National Science Foundation 29Train the Admin – GEC20 – June 21, 2014
Tool support
• ExoGENI & InstaGENI both support:– Reservation tools: Omni, GENI Portal, Flack– I&M: GIMI, GEMINI
• InstaGENI, ExoGENI also have testbed specific tools
Sponsored by the National Science Foundation 33Train the Admin – GEC20 – June 21, 2014
INTER-AGGREGATE CONNECTIVITY
Sponsored by the National Science Foundation 34Train the Admin – GEC20 – June 21, 2014
Inter-aggregate Links
• GENI does not own any networks. Instead GENI connects racks via regional and backbone providers.
• Connect racks in one of four ways:– Stitching: dynamically configured inter-domain VLANs
• Stitcher: Distributed with gcf/omni• Flukes: ExoGENI-only (not via GENI AM API)
– OpenFlow Mesoscale & AL2S & some regionals• e.g. CENIC, SOX, MOXI are OpenFlow regionals
– Preconfigured VLANs (Shared and Unshared)• Some pre-configured inter-domain VLANs are available• Some are OpenFlow-enabled
– GRE tunnels over control interface• Use Flack to connect IG nodes via a GRE tunnel
Sponsored by the National Science Foundation 35Train the Admin – GEC20 – June 21, 2014
GENI Stitching: Under the Hood
How does GENI Stitching Work?
1.Rack Configuration: Long. Done once in advance.
2.Tool & Aggregates make Reservations: Quick, Live, Easy
Sponsored by the National Science Foundation 36Train the Admin – GEC20 – June 21, 2014
Rack Stitching Configuration
• Identify a path or paths from the rack to other GENI aggregates. – Typically a connection to a national backbone
• Identify the network providers – Typically a campus, a regional, and the backbone
• Identify the endpoints and VLAN tags that can be used to connect to the rack
Backbone Regional Campus
Rack
GENI Aggregates
Static VLANs
Sponsored by the National Science Foundation 37Train the Admin – GEC20 – June 21, 2014
SCS
Tool
Experimenter View: Creating a Circuit
<link client_id=”mylink"> <component_manager… <component_manager…
Aggregate 2
Aggregate 1
1. Simple Request
2. Send Path Request to Stitching Computation Service (SCS)
3. Get Expanded Request
4. Send Request to Aggregate 1
5. Get Manifest
6. Repeat for Other Aggregates
<link client_id=”mylink"> <component_manager… <component_manager… …<stitching> <path id=“mylink”> <hop id=“1”> <link id=“switch1:port1”… <vlan…>3747</vlan…>
7. Manifest Back
Sponsored by the National Science Foundation 39Train the Admin – GEC20 – June 21, 2014
OPENFLOW AND FOAM
Sponsored by the National Science Foundation 40Train the Admin – GEC20 – June 21, 2014
GENI META-OPERATIONS (GMOC)
Sponsored by the National Science Foundation 41Train the Admin – GEC20 – June 21, 2014
GENI Meta-operations
GMOC: GENI Meta-operation Center• Keeps track of outages• Notification system for resource reservations• Emergency Stop
How to notify GMOC: http://groups.geni.net/geni/wiki/HowTo/PreReserveGENIResourcesGMOC Calendar: http://globalnoc.iu.edu/gmoc/index/support/gmoc-operations-calendars.html
GMOC Google Calendar keeps track of reservations/outages
Sponsored by the National Science Foundation 42Train the Admin – GEC20 – June 21, 2014
Monitoring
• Racks are being monitored• You may get email from someone if something
looks amiss.• Example:
– Can’t reach anything on the rack via ping or an AM API GetVersion call
Sponsored by the National Science Foundation 43Train the Admin – GEC20 – June 21, 2014
GMOC Notifications
• GMOC sends many notifications• Three types of interest:
– Outages– Resources pre-reservation for a class, tutorial, or demo– Potentially disruptive experiment
• All notifications are tracked on the calendar:– http://globalnoc.iu.edu/gmoc/index/support/gmoc-
operations-calendars.html
Sponsored by the National Science Foundation 44Train the Admin – GEC20 – June 21, 2014
GMOC Resource Pre-Reservation
What?– A class, tutorial, or experiment needs a well defined set of
resources at a specified time
Why does GMOC send notifications?– Help us deconflict use of GENI and maintenances,
planned outages, etc.
What should you do?– Is there something happening on your campus that would
conflict with this event? Contact GMOC and let them know.
– Conflicts are a rare event. Usually the rack teams will take care of this for you.
– Please report events on your campus that only you can know about. e.g a known power outage
Sponsored by the National Science Foundation 45Train the Admin – GEC20 – June 21, 2014
GMOC Resource Pre-ReservationSUBJECT: Resource Reservation - GENI Spring Train the TA TutorialAFFECTED: GPO Portal and Clearinghouse InstaGENI racks: GPO, Clemson, Missouri, Illinois, Nysernet GENI Tools: Flack, OmniSCHEDULED START TIME: Friday, January 24, 2014, 7:00 PM (1900) UTCSCHEDULED END TIME: Friday, January 24, 2014, 10:00 PM (2200) UTCDESCRIPTION: At the 2014 Sprint Train the TA there are going to be multiple GENI tutorials that will use the GENI resources listed above. We would kindly ask operators to refrain for any outages during this period without coordinating with the tutorial organizers first at [email protected]. Only light traffic volumes are expected.TICKET NO.: 753:126TIMESTAMP: Thu Jan 23 20:04:53 2014 UTC
Message ID: gmoc.753.2.1Thank You,GENI Meta Operations CenterIndiana [email protected], 317-274-7783Visit the GMOC Home Page athttp://gmoc.grnoc.iu.edu/
_______________________________________________GENI-Ops mailing [email protected]://mail1.grnoc.iu.edu/mailman/listinfo/geni-ops
Name of Event
Who’s affected?
Duration
Description
GMOC Ticket #
GMOC Website for more information
Is your rack listed?If so, keep reading
Mention in e-mail to GMOC
STOP
Sponsored by the National Science Foundation 46Train the Admin – GEC20 – June 21, 2014
Potentially Disruptive Experiment
What?– An experiment that might cause problems for other
experimenters or resource owners.– Examples:
• Reserving a large amount of bandwidth between two sites
Why does GMOC send notifications?– Alerts other experimenters and resource owners that
something unusual might happen
What should you do?– If you see unexpected behavior related to your rack,
notify GMOC. We can ask the experimenter to stop.
Sponsored by the National Science Foundation 47Train the Admin – GEC20 – June 21, 2014
Potentially Disruptive ExperimentSUBJECT: Disruptive Experiment Reservation for GENI Participants U. Missouri and U. Utah DDCAFFECTED: U. of Missouri InstaGENI Rack Utah DDC InstaGENI Rack Internet2SCHEDULED START TIME: Thursday, December 12, 2013, 3:00 PM (1500) UTCSCHEDULED END TIME: Thursday, December 12, 2013, 6:00 PM (1800) UTCDESCRIPTION: An experiment will be reserving a 1GB link between the U. of Missouri and Utah DDC InstaGENI racks.Traffic spikes and other potential disruptions may occur across the GENI OpenFlow Backbone during the experiment. The entire window has been
reserved.TICKET NO.: 714:126TIMESTAMP: Wed Dec 11 20:23:12 2013 UTC
Message ID: gmoc.714.1.1Thank You,GENI Meta Operations CenterIndiana [email protected], 317-274-7783Visit the GMOC Home Page athttp://gmoc.grnoc.iu.edu/
Name of Event
Who’s affected?
Duration
Description
GMOC Ticket #
GMOC Website for more information
Is your rack listed?If so, keep reading
Mention in e-mail to GMOC
STOP
Sponsored by the National Science Foundation 48Train the Admin – GEC20 – June 21, 2014
GENI Emergency Stop process
48
http://gmoc.grnoc.iu.edu/gmoc/documents.html
• Emergency Stop is a well defined process for what to do if something goes wrong.
• There is 24/7 staffing to respond in case of emergency
Slide by Eldar Urumbaev
Sponsored by the National Science Foundation 49Train the Admin – GEC20 – June 21, 2014
What is Emergency Stop?• One of the essential early operational requirements for the GENI
facility is the need to manage and coordinate the stop and/or containment of GENI resources among all GENI projects in the case of an urgent request or misbehavior of the GENI experiments.
• Emergency stop is the system used to respond to incidents of interference or resource exhaustion caused either unintentionally (misconfiguration), or intentionally (malware).
The emergency stop system has 3 main goals: • To give experimenters and other GENI stakeholders a single place
to go for notification of emergency stop issues• To facilitate emergency stop with GENI aggregates • To provide an easily understood, flexible service that minimizes
disruptions for GENI experiments or GENI aggregates
49Slide by Eldar Urumbaev
Sponsored by the National Science Foundation 50Train the Admin – GEC20 – June 21, 2014
These goals are accomplished through two mechanisms:• A coordination process to identify related GENI
aggregates and/or slices for a given stop request (via GMOC-DB), notify, facilitate communication, verify resolution and report.
• A last resort isolation mechanism (via Internet2 and NLR Core) to effectively quarantine aggregates with issues from the rest of the GENI infrastructure.
What Is Emergency Stop?
Sponsored by the National Science Foundation 51Train the Admin – GEC20 – June 21, 2014
• GMOC – GMOC • Aggregate Operators • GENI-Interconnected Networks & Experimenters • GENI LLR Representative
Emergency Stop Participants & Stakeholders
Slide by Eldar Urumbaev
Sponsored by the National Science Foundation 52Train the Admin – GEC20 – June 21, 2014
GMOC Emergency Stop Contact Information:
• Phone: (317) 274-7783• Email: [email protected] • Web:
http://gmoc.grnoc.iu.edu/gmoc/support/report-a-problem.html
Slide by Eldar Urumbaev
Sponsored by the National Science Foundation 53Train the Admin – GEC20 – June 21, 2014
TOOLS FOR RACK ADMINISTRATION
Sponsored by the National Science Foundation 54Train the Admin – GEC20 – June 21, 2014
• Currently the rack teams do most actual rack administration
• However tools exist which you can get access to for your rack
• Tools are specific to your rack type• Request admin accounts from the
InstaGENI/ExoGENI teams
Sponsored by the National Science Foundation 55Train the Admin – GEC20 – June 21, 2014
InstaGENI Administrative E-mail
• InstaGENI sends mail when:– Experimenters act on slivers (create/renew/delete/etc)– Routine non-alarming events occur– Something more serious is wrong– http://www.protogeni.net/wiki/rackmaillists has more
info
IG rack wiki: http://www.protogeni.net/wiki/instageni
Sponsored by the National Science Foundation 56Train the Admin – GEC20 – June 21, 2014
InstaGENI Administrative E-mailFrom: [email protected]: [email protected]: [email protected]: [gpo-ops] BBNINSTAGENI: protogeni-wrapper.plDate: Wed, 15 Jan 2014 10:12:11 -0500 (EST)
URN : urn:publicid:IDN+ch.geni.net+user+jbsModule : amMethod : CreateSliverVersion : 2.0StartTime : 10:11:49:815689slice_urn : urn:publicid:IDN+ch.geni.net:gpo-infra+slice+ps103slice_idx : 13430slice_uuid : ec5dbd9d-cc1d-4c06-acb5-0ebd6b11f366EndTime : 10:12:11:184891Elapsed : 21.37LogURN : urn:publicid:IDN+instageni.gpolab.bbn.com+log+de38e4abc56126289876f71d6e130013LogURL : https://boss.instageni.gpolab.bbn.com/spewlogfile.php3?logfile=de38e4abc56126289876f71d6e130013Code : 0
Who acted?
AM API Call
Time
Slice
Log URLCheck here for more details /
Mention in e-mail
Sponsored by the National Science Foundation 57Train the Admin – GEC20 – June 21, 2014
InstaGENI: Red dot mode
The web interface for each rack allows administrators to see extra information about what is happening on their rack.
Press to enter red dot mode
… but only if you are an administrator
Sponsored by the National Science Foundation 58Train the Admin – GEC20 – June 21, 2014
Red Dot Example
For example, see what slices are running on your nodes and who has reserved those resources
Sponsored by the National Science Foundation 59Train the Admin – GEC20 – June 21, 2014
ExoGENI Administrative Tools
• ExoGENI does not send administrative email
• ExoGENI provides – pequod, a command line tool to access your rack– A monitoring webpage– Wiki documentation
Pequod documentation: https://geni-orca.renci.org/trac/wiki/orca-pequodExoGENI monitoring: https://control.exogeni.net/monitor/check_mk/ExoGENI wiki: https://wiki.exogeni.net/doku.php
Sponsored by the National Science Foundation 60Train the Admin – GEC20 – June 21, 2014
$ ssh bbn-hn.exogeni.gpolab.bbn.comLast login: Wed Jan 29 15:50:04 2014 from dhcp89-081-159.bbn.com|-----------------------------------------------------------------|| ____ ____ ____ ____ ____ ____ ____ || ||E |||x |||o |||G |||E |||N |||I || || ||__|||__|||__|||__|||__|||__|||__|| || |/__\|/__\|/__\|/__\|/__\|/__\|/__\| || ||-----------------------------------------------------------------|
$ pequodPequod ORCA Shell v.4.0-SNAPSHOT.build-5300 built on 07/16/2013 18:38 (c) 2012-2013 RENCI/UNC Chapel Hill
help: Returns help for individual commands file: File-related commands set: Modify internal set variables show: Show the state of things aux: Auxiliary commands manage: Manage actor state history: show command history (!<command index> invokes the command) exit: Exit from the shell (Ctrl-D or Ctrl-C also works)Type the entire command, or enter the first word of the command to enter subcommand with intelligent auto-completion (Using TAB).<snip>pequod>show reservations for all actor bbn-vm-am state active 68fc1bb0-3085-4e91-baa7-330a9ac3a37b bbn-vm-am Slice: 6146e4bf-0ebf-4072-8238-18cd688aeb53 1 bbnvmsite.vm [ active, nascent] Notices: Reservation 68fc1bb0-3085-4e91-baa7-330a9ac3a37b (Slice urn:publicid:IDN+ch.geni.net:gpoamcanary+slice+sitemon) is in state [Active,None] Start: Tue Jan 14 16:40:22 EST 2014 End:Thu Jan 30 19:00:00 EST 20144be1e2f8-7be1-456e-862c-cffdbebcdc9c bbn-vm-am Slice: 733fc88a-14e7-44ed-a2c5-1ea7ec4da223 1 bbnvmsite.vm [ active, nascent] Notices: Reservation 4be1e2f8-7be1-456e-862c-cffdbebcdc9c (Slice urn:publicid:IDN+ch.geni.net:gpo-infra+slice+ps103) is in state [Active,None] Start: Wed Jan 15 10:11:44 EST 2014 End:Sun Feb 02 18:00:00 EST 2014<snip>Total: 19 reservationspequod>exit
Logging out of containersShutting down commands help file set show aux manage Resetting terminal and exiting. Goodbye.
Login to head node on your rack
Run pequod
Query state of resources
Sponsored by the National Science Foundation 61Train the Admin – GEC20 – June 21, 2014
ExoGENI Admin Webpages (nagios)
ExoGENI wiki: https://wiki.exogeni.net/doku.phpExoGENI monitoring: https://control.exogeni.net/monitor/check_mk/For a single rack: https://<site>-hn.exogeni.net/rack_<site>/For example: https://bbn-hn.exogeni.net/rack_bbn/
Status of racks
interfaces
Sponsored by the National Science Foundation
Getting Help
Sponsored by the National Science Foundation 63Train the Admin – GEC20 – June 21, 2014
General bug reporting advice
Gather as much information as you can– What did you do? What did you expect to happen? What
actually happened?– Include what you see (screenshots, omni output errors)– Where relevant, include:
• type of account you are using (eg portal)• the tool you are using (eg Flack, omni, portal)• your slice name or URN • aggregates you are using• a detailed description of what's wrong including any error messages
Sponsored by the National Science Foundation 64Train the Admin – GEC20 – June 21, 2014
Getting help: General mailing lists
[email protected]– General place to get help using GENI– More aimed at experimenters than admins– Any questions about using GENI are
certainly [email protected]
– At least one admin/ops person from each GENI site– You should probably subscribe (directly or a local list)– Mostly outage announcements these days
[email protected]– General catch-all list for getting help– geni-users is usually better, but this is an ok last resort
[email protected] (not a mailing list)– Send outage notifications to this list– Get help with Internet2 and AL2S
http://groups.geni.net/geni/wiki/NikySandbox/GENIExperimenter/GENICommunity
Sponsored by the National Science Foundation 65Train the Admin – GEC20 – June 21, 2014
Getting help: Topical mailing lists
[email protected]– General list for people running Emulab– Site admins with InstaGENI racks should
join– Any questions about running an IG rack
– RENCI list for their ExoGENI ops team– Any questions about running an EG rack are welcome
[email protected]– GPO list for their infrastructure team– The best place for FOAM questions for now– Also fine to ask about anything else
http://groups.geni.net/geni/wiki/NikySandbox/GENIExperimenter/GENICommunity
Sponsored by the National Science Foundation 66Train the Admin – GEC20 – June 21, 2014
Getting help: Other resources
The GENI wiki:– http://groups.geni.net/– http://groups.geni.net/geni/wiki/GENIOperationsWelcome– http://groups.geni.net/geni/wiki/GENIRacksHome/InstageniRacks/Operators
– For operators– http://groups.geni.net/geni/wiki/ConnectivityHome
– IP Addresses, VLANs assigned to GENI sites– http://groups.geni.net/geni/wiki/GeniAggregate– http://groups.geni.net/geni/wiki/GENIRacksHome
IRC:– http://groups.geni.net/geni/wiki/HowTo/ConnectToGENIChatRoom– Various GPO and other GENI people are often there
Other mailing lists:– http://groups.geni.net/geni/wiki/NikySandbox/GENIExperimenter/GENICommunity– Many other topic-specific lists– Using one of the general lists is often better
http://groups.geni.net/geni/wiki/GENIExperimenter/GetHelp
Sponsored by the National Science Foundation 68Train the Admin – GEC20 – June 21, 2014
Questions?