1

GENI: Global Environment for Network Innovations

Jennifer Rexford

Princeton Universityhttp://www.cs.princeton.edu/~jrex

2

Outline• Networking research challenges

– Security, economic incentives, management, layer-2 technologies

• Importance of building and deploying– Bridging the gap between simulation/testbeds and real deployment

• Global Environment for Network Innovations (GENI)– Major NSF initiative to support experimental networking research– Key ideas: virtualization, programmability, and user opt-in

• GENI backbone design– Programmable routers, flexible optics, and connection to Internet

• Virtual Network Infrastructure (VINI)– Initial experimental network facility in NLR and Abilene

• Conclusions

3

Is the Internet broken?• It is great at what it does.

– Everyone should be proud of this. – All sorts of things can be built on top of it.

• But…– Security is weak and not getting better.– Availability continues to be a challenge.– It is hard to manage and getting harder. – It does not handle mobility well.– A long list, once you start…

4

Challenges Facing the Internet• Security and robustness

– Naming and identity– Availability

• Economic incentives– Difficulty of providing end-to-end services– Commoditization of the Internet infrastructure

• Network management– No framework in the original Internet design– Tuning, troubleshooting, accountability, …

• Interacting with underlying network technologies – Advanced optics: dynamic capacity allocation– Wireless: mobility, dynamic impairments– Sensors: small embedded devices at large scale

5

FIND: Future Internet Design• NSF research initiative

– Requirements for global network of 10-15 years out?– Re-conceive the network, if we could design from scratch?

• Conceive the future, by letting go of the present:– This is not change for the sake of change– Rather, it is a chance to free our minds– Figuring out where to go, and then how to get there

• Perhaps a header format is not the defining piece of a new architecture– Definition and placement of functionality– Not just data plane, but also control and management– And division between end hosts and the network

6

The Importance of Building• Systems-oriented computer science research

needs to build and try out its ideas to be effective– Paper designs are just idle speculation– Simulation is only occasionally a substitute

• We need:– Real implementation– Real experience– Real network conditions– Real users– To live in the future

7

Need for Experimental Facility

Analysis Simulation / Emulation Experiment At Scale

Deployment

(models) (code)

(results)

(measurements)

Goal: Seamless conception-to-deployment process

8

Today’s Tools Have Limitations• Simulation based on simple models

– Topologies, administrative policies, workloads, failures…

• Emulation (and “in lab” tests) are similarly limited– Only as good as the models

• Traditional testbeds are targeted– Not cost-effective to test every good idea– Often of limited reach– Often with limited programmability

• Testbed dilemma– Production network: real users, but hard to make changes– Research testbed: easy to make changes, but no users

9

Bridging the Chasm

This chasm is a majorbarrier to realizing the

future designs

Maturity

Time

Foundational Research

Simulation and Research Prototypes

Small Scale Testbeds

DeployedFuture

InternetGlobal Experimental

Facility

10

GENI• Experimental facility

– MREFC proposal to build a large-scale facility– Jointly from NSF’s CS directorate, & research community– We are currently at the “Conceptual Design” stage– Will eventually require Congressional approval

• Global Environment for Network Innovations– Prototyping new architectures– Realistic evaluation– Controlled evaluation– Shared facility– Connecting to real users– Enabling new services

See http://www.geni.net

11

Three Key Ideas in GENI• Virtualization

– Multiple architectures on a shared facility– Amortizes the cost of building the facility– Enables long-running experiments and services

• Programmable– Enable prototyping and evaluation of new architectures– Enable a revisiting of today’s “layers”

• Opt-in on a per-user / per-application basis– Attract real users

• Demand drives deployment / adoption– Connect to the Internet

• To reach users, and to connect to existing services

12

Slices

13

Slices

14

User Opt-in

Client

Server

Proxy

15

Realizing the Ideas• Slices embedded in a substrate of resources

– Physical network substrate• Expandable collection of building block components• Nodes / links / subnets

– Software management framework• Knits building blocks together into a coherent facility• Embeds slices in the physical substrate

• Builds on ideas in past systems– PlanetLab, Emulab, ORBIT, X-Bone, …

16

National Fiber Facility

17

+ Programmable Routers

18

+ Clusters at Edge Sites

19

+ Wireless Subnets

20

+ ISP Peers

ISP 2

ISP 1

21

Closer Look

Internet

backbone wavelength

backbone switch

Sensor Network

Edge SiteWireless Subnet

Customizable Router

DynamicConfigurable

Swith

22

GENI Substrate: Summary• Node components

– Edge devices– Customizable routers– Optical switches

• Bandwidth– National fiber facility– Tail circuits

• Wireless subnets– Urban 802.11– Wide-area 3G/WiMax– Cognitive radio– Sensor net– Emulation

23

GENI Management Core

GMC

Management Services

Substrate Components

- name space for users, slices, & components

- set of interfaces (“plug in” new components)- support for federation (“plug in” new partners)

24

Hardware Components

Substrate HW Substrate HW Substrate HW

25

Virtualization Software

Virtualization SWSubstrate HW

Virtualization SWSubstrate HW

Virtualization SWSubstrate HW

26

Component Manager

Substrate HW Substrate HWSubstrate HW

CM

Virtualization SW

CM

Virtualization SW

CM

Virtualization SW

27

GENI Management Core (GMC)

Resource Controller Auditing Archive

Slice ManagerGMC

nodecontrol

sensordata

CM

Virtualization SWSubstrate HW

CM

Virtualization SWSubstrate HW

CM

Virtualization SWSubstrate HW

slice_spec(object hierarchy)

28

Federation

. . .

GMC GMC

29

User Front-End(s)

. . .

Front-End(set of management services)

GUI

GMC GMC

provisioning servicefile & naming serviceinformation plane

30

Virtualization in GENI• Multiple levels possible

– Different level required by different experiments– Different level depending on the technology

• Example “base cases”– Virtual server (socket interface / overlay tunnels)– Virtual router (virtual line card / static circuits)– Virtual switch (virtual control interface / dynamic circuits)– Virtual AP (virtual MAC / fixed spectrum allocation)

• Specialization– The ability to install software in your own virtual-*

31

Distributed Services in GENI• Goals

– Complete the GENI management story– Lower the barrier-to-entry for researchers (students)

• Example focus areas– Provisioning (slice embedder)– Security– Information plane– Resource allocation– Files and naming– Topology discovery– Development tools– Interfacing with the Internet, and IP

32

GENI Security• Limits placed on a slice’s “reach”

– Restricted to slice and GENI components– Restricted to GENI sites– Allowed to compose with other slices– Allowed to interoperate with legacy Internet

• Limits on resources consumed by slices– Cycles, bandwidth, disk, memory– Rate of particular packet types, unique addrs per second

• Mistakes (and abuse) will still happen– Auditing will be essential– Network activity slice responsible user(s)

33

Success Scenarios• Change the research process

– Sound foundation for future network architectures– Experimental evaluation, rather than paper designs

• Create new services– Demonstrate new services at scale– Attract real users

• Aid the evolution of the Internet– Demonstrate ideas that ultimately see real deployment– Provide architectural clarity for evolutionary path

• Lead to a future global network– Purist: converge on a single new architecture– Pluralist: virtualization supporting many architectures

34

Working Groups to Flesh Out Design• Research (Dave Clark and Scott Shenker)

– Usage policy / requirements / instrumentation

• Architecture (Larry Peterson and John Wroclawski)– Define core modules and interfaces

• Backbone (Jen Rexford and Dan Blumenthal)– Fiber facility / routers & switches / tail circuits / peering

• Wireless (Dipankar Raychaudhuri and Deborah Estrin)– RF technologies / deployment

• Services (Tom Anderson, Reiter)– Edge sites / infrastructure and underlay services

• Education– Training / outreach / course development

35

GENI Backbone Requirements• Programmability

– Flexible routing, forwarding, addressing, circuit set-up, …

• Isolation– Dedicated bandwidth, circuits, CPU, memory, disk

• Realism– User traffic, upstream connections, propagation delays, equipment

failure modes, …

• Control– Inject failures, create circuits, exchange routing messages

• Performance– High-speed packet forwarding and low delays

• Security– Preventing attacks on the Internet, and on GENI itself

36

A Researcher’s View of GENI Backbone• Virtual network topology

– Nodes and links in a particular topology– Resources and capabilities per node/link– Embedded in the GENI backbone

• Virtual router and virtual switch– Abstraction of a router and switch per node– To evaluate new architectures (routing, switching,

addressing, framing, grooming, layering, …)

• GENI backbone capabilities evolve over time – To realize the abstractions at finer detail– To scale to a larger number of experiments

37

Creating a Virtual Topology

Some links and nodes unused

Some links created by cutting through other nodes

Allocating a fraction of a link and node

38

GENI Backbone

ISP 1

ISP 2

PC Clusters

ProgrammableRouters

WirelessSubnets

Dynamic ROADMs

39

GENI Backbone Node Components• Phase 0 – General purpose blade server

– Single node with collection of assignable resources– Virtual Router may be assigned VM, blade or >1 blades

• Phase 1 – Adding higher performance components– Assignable Network Processor blades and FPGA blades– NPs also used for I/O for better control of I/O bandwidth

• Phase 2 – Adding reconfigurable cross-connect– Enable experiments with configurable transport layer– Provide “true circuits” between backbone virtual routers

• Phase 3 – Adding dynamic optical switch– Dynamic optical switch with programmable groomer and

framer, and reconfigurable add/drop multiplexers

40

GENI Backbone Node Components• Phase 0 – General

purpose blade server– Node with collection of

assignable resources– Virtual Router may be

assigned a virtual machine, blade, or multiple blades

Mgm

t. Pr

oc.

SwitchSwitch

P 1 . . .P 2 P 3 P n

Mgm

t. Pr

oc.

41

GENI Backbone Node Components• Phase 1 – Adding higher

performance components– Assignable Network

Processor blades and FPGA blades

– NPs also used for I/O for better control of bandwidth

– ATCA chassis and blades

. . .

10 GigE Links

Mgm

t. Pr

oc.

SwitchSwitch

PE1 . . .PE2

PEn

LCk

Mgm

t. Pr

oc.

GP BladeServer

LC1

42

GENI Backbone Node Components• Phase 2 – Reconfigurable

cross-connect– Enable experiments with

configurable transport layer– Provide “true circuits”

between backbone virtual routers

– Cut-through traffic circumvents the router

VX VX

VR VR

VRVR

VR

CustomizableRouter

10GE+VLAN

1 GE

Wavelength tunable transponders/combinerWDM Fiber

ProgrammableCross-Connect/

Groomer

Cont

rol P

lane

43

GENI Backbone Node Components• Phase 3 – Adding dynamic

optical switch– Dynamic optical switch with

programmable groomer and framer, and reconfigurable add/drop multiplexers

– Maleable bandwidth– Arbitrary framing

VC1 VC1Customizable

Router10GE+VLAN

1 GE

Wavelength tunable transponders

ProgrammableCross-Connect/

Groomer

Cont

rol P

lane

ROADM

44

GENI Backbone Software• Component manager and virtualization layer

– Abstraction of virtual router and virtual switch– Setting scheduling parameters for subdividing resources

• Multiplexers for resources hard to share– Single BGP session with the outside world– Single interface to an element-management system

• Exchanging traffic with the outside world– Routing and forwarding software to evaluate & extend– VPN servers and NATs at the GENI/Internet boundary

• Libraries to support experimentation– Specifying, controlling, and measuring experiments– Auditing and accounting to detect misbehavior

45

Feasibility• Industrial trends and standards

– Advanced Telecom Computing Architecture (ATCA)– Network processors and FPGAs– SONET cross connects and ROADMs

• Open-source networking software– Routing protocols, packet forwarding, network address

translation, diverting traffic to an overlay

• Existing infrastructure– PlanetLab nodes, software, and experiences– National Lambda Rail and Abilene backbones

46

VINI: Step Toward GENI Backbone• Virtual Network Infrastructure (VINI)

– Multiple network experiments in parallel– Connections to end users and upstream providers– Supporting Internet protocols and new designs

• VINI as an initial experimental platform – Support researchers doing network experiments– Explore software challenges of building GENI backbone

• GENI will have a much wider scope– Programmable hardware routers– Flexible control of the optical components– Wireless and sensor networks at the edge

47

Network Infrastructure• Network topology

– Points of Presence– Link bandwidth– Upstream connectivity

• Two backbones– Abilene Internet2– National Lambda Rail

48

Building Virtual Networks• Physical nodes

– Initially, high-end computers– Later, network processors and FPGAs

• Virtual routers (a la PlanetLab)– Multiple virtual servers on a single node– Separate shares of resources (e.g., CPU, bandwidth)– Extensions for resource guarantees and priority

49

Building Virtual Links• Creating the illusion of interfaces

– Create a tunnel for each link in the topology– Assign IP addresses to the end-points of tunnels– Match tunnels with one-hop links in the real topology

50

Building Multiple Virtual Topologies• Separate topology per experiment

– Routers are virtual servers– Links are a subset of possible tunnels

• Creating a customized environment – Running User Mode Linux (UML) in a virtual server– Configuring UML to see multiple interfaces– Enables running the routing software “as is”

Operating System

R R R R R

51

Overcoming Efficiency Challenges• Packet forwarding must be fast

– But, we are doing packet forwarding in software– And don’t want the extra overhead of UML

• Solution: separate packet forwarding– Routing protocols running within UML– Packet forwarding running outside of UML

UML

Click

XORP

XORP: routing softwareClick: forwarding software

52

Carrying Real User Traffic• Users opt in to VINI

– User runs VPN client– Connects to VINI node

• External Internet hosts– VINI connects to Internet– Apply NAT at boundary

UML

UML

UML

Click

Click

Click

Client Server

UDPtunnels

XORP XORP

XORP

OpenVPN

NetworkAddressTranslation

routing-protocolmessages

VINI

53

Example VINI Experiment• Configure VINI just like Abilene

– VINI node per PoP– VINI link per inter-PoP link – Routing configuration as real routers

• Network event– Inject link failure between two PoPs– … in midst of an ongoing file transfer

• Measuring routing convergence– Packet monitoring of the data transfer– Active probes of round-trip time & loss– Detailed view of effects on data traffic

54

VINI Current Status• Initial Abilene deployment

– Eleven sites– Nodes running XORP and Click on UML

• Upcoming deployment– Six sites in National Lambda Rail– … with direct BGP sessions with CRS-1 routers– Dedicated 1 Gbps bandwidth between Abilene sites

• In the works– Upstream connectivity via a commercial ISP in NYC– Speaking interdomain routing with the Internet

Initial write-up: http://www.cs.princeton.edu/~jrex/papers/vini.pdf

55

Conclusions• Future Internet poses many research challenges

– Security, network management, economics, layer-2, …

• Research community should rise to the challenge– Conceive of future network architectures– Prototype and evaluate architectures in realistic settings

• Global Environment for Network Innovations (GENI)– Facility for evaluating new network architectures– Virtualization, programmability, and user opt-in

• GENI backbone design– Fiber facility, tail circuits, and upstream connectivity– Programmable router and dynamical optical switch

• VINI prototype– Concrete step along the way to the GENI backbone