emulab.net: an emulation testbed for networks and distributed systems jay lepreau and many others...
Post on 19-Dec-2015
217 views
TRANSCRIPT
![Page 1: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/1.jpg)
emulab.net:An Emulation Testbed for Networks and Distributed
Systems
Jay Lepreau
and
many others
University of Utah
Intel IXA University Workshop
June 21, 2001
![Page 2: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/2.jpg)
The Main Players Undergrads
– Chris Alfeld, Chad Barb, Rob Ricci
Grads– Dave Andersen, Shashi Guruprasad, Abhijeet
Joglekar, Indrajeet Kumar, Mac Newbold Staff
– Mike Hibler, Leigh Stoller
Alumni– Various
(Red: here at Intel today)
![Page 3: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/3.jpg)
What?
A configurable Internet emulator in a room– Today: 200 nodes, 500 wires, 2x BFS (switch)– virtualizable topology, links, software
Bare hardware with lots of tools An instrument for experimental CS research Universally available to any remote
experimenter Simple to use
![Page 4: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/4.jpg)
What’s a Node?
Physical hardware: PCs, StrongARMs
Virtual node:– Router (network emulation)– Host, middlebox (distributed system)
Future physical hardware: IXP1200 +
![Page 5: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/5.jpg)
Why?
“We evaluated our system on five nodes.” -job talk from university with 300-node cluster
“We evaluated our Web proxy design with 10 clients on 100Mbit ethernet.”
“Simulation results indicate ...” “Memory and CPU demands on the individual
nodes were not measured, but we believe will be modest.”
“The authors ignore interrupt handling overhead in their evaluation, which likely dominates all other costs.”
“Resource control remains an open problem.”
![Page 6: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/6.jpg)
Why 2
“You have to know the right people to get access to the cluster.”
“The cluster is hard to use.” “<Experimental network X> runs FreeBSD
2.2.x.” “October’s schedule for <experimental
network Y> is…” “<Experimental network Z> is tunneled
through the Internet”
![Page 7: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/7.jpg)
Complementary to Other Experimental Environments
Simulation Small static testbeds Live networks Maybe someday, a large scale set of
distributed small testbeds (“Access”)
![Page 8: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/8.jpg)
Switched “Backplane”
PCPC
Web/DB/SNMPSwitch MgmtUsers
Internet
Control Switch/Router
Serial
Sharks Sharks
16040
PowerCntl
![Page 9: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/9.jpg)
Zoom In: One Node
![Page 10: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/10.jpg)
Fundamental Leverage:
Extremely ConfigurableEasy to Use
![Page 11: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/11.jpg)
Key Design Aspects
Allow experimenter complete control … but provide fast tools for common
cases– OS’s, disk loading, state mgmt tools, IP, traffic
generation, batch, ...
Virtualization– of all experimenter-visible resources– node names, network interface names,
network addresses– Allows swapin/swapout
![Page 12: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/12.jpg)
Design Aspects (cont’d)
Flexible, extensible, powerful allocation algorithm
Persistent state maintenance:– none on nodes– all in database– leverage node boot time: only known state!
Separate control network Familiar, powerful, extensible
configuration language: ns
![Page 13: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/13.jpg)
Some Unique Characteristics
User-configurable control of “physical” characteristics: shaping of link latency/bandwidth/drops/errors(via invisibly interposed “shaping nodes”), router processing power, buffer space, …
Node breakdown today:– 40 core, 160 edge
![Page 14: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/14.jpg)
More Unique Characteristics
Capture of low-level node behavior such as interrupt load and memory bandwidth
User-replaceable node OS softwareUser-configurable physical link
topologyCompletely configurable and usable
by external researchers, including node power cycling
![Page 15: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/15.jpg)
Obligatory Pictures
![Page 16: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/16.jpg)
Then
![Page 17: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/17.jpg)
Now
![Page 18: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/18.jpg)
A Few Research Issues and Challenges
Network management of unknown entities
Security Scheduling of experiments Calibration, validation, and scaling Artifact detection and control NP-hard virtual --> physical mapping
problem Providing a reasonable user interface ….
![Page 19: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/19.jpg)
An “Experiment”
emulab’s central operational entity Directly generated by an ns script, … then represented entirely by
database state
Steps: Web, compile ns script, map, allocate, provide access, assign IP addrs, host names, configure VLANs, load disks, reboot, configure OS’s, run, report
![Page 20: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/20.jpg)
Mapping Example
![Page 21: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/21.jpg)
Automatic mapping of desired topologies and
characteristics to physical resources
Algorithm goals:– minimize likelihood of experimental artifacts (bottlenecks)– “optimal” packing of multiple simultaneous experiments– Extensible for heterogenous hardware, software, new
features
Randomized heuristic algorithm: simulated annealing
May move to genetic algorithm
![Page 22: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/22.jpg)
Virtual Topology
![Page 23: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/23.jpg)
Mapping into Physical Topology
![Page 24: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/24.jpg)
Mapping Results
< 1 second for first solution, 40 nodes
“Good” solution within 5 secondsApparently insensitive to number
of node “features”
![Page 25: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/25.jpg)
Disk Loading
13 GB generic IDE 7200 rpm drivesWas 20 minutes for 6 GB imageNow 88 seconds
![Page 26: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/26.jpg)
How?
Do obvious compression… and a little less obvious: zero
the fsDisk writes become the bottleneckHack the disk driverCarefully overlap I/O and
decompression==> 6 minutes
![Page 27: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/27.jpg)
Last Step
Domain-specific compressionType the filesystem blocks:
– allocated– free
Never write the free ones
![Page 28: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/28.jpg)
Experiment Creation Time
![Page 29: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/29.jpg)
Experiment Termination Time
![Page 30: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/30.jpg)
Ongoing and Future Work
Multicast disk images IXP1200 nodes, tools, code fragments
– Routers, high-capacity shapers
Event system Scheduling system Topology generation tools and GUI Simulation/enulation transparency Linked testbeds Wireless nodes, Mobile nodes Logging. Visualization tools Microsoft OSs, high speed links, more nodes!
![Page 31: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/31.jpg)
Final Remarks
18 projects have used it (14 external)– Plus several class projects
Two OSDI’00 and three SOSP’01 papers– 20% SOSP general acceptance rate– 60% SOSP acceptance rate for emulab users!
More emulab’s under construction:– Yes: Univ. of Kentucky, Stuttgart– Maybe: WUSTL, Duke
Sponsors (red: major ones, current or expected)– NSF, DARPA, University of Utah– Cisco, Intel, Compaq, Microsoft, Novell, Nortel
![Page 32: Emulab.net: An Emulation Testbed for Networks and Distributed Systems Jay Lepreau and many others University of Utah Intel IXA University Workshop June](https://reader030.vdocuments.site/reader030/viewer/2022032800/56649d375503460f94a0f3d2/html5/thumbnails/32.jpg)
Available for universities, labs, and
companies at:
www.emulab.net