Why Optical NetworksWill Become the 21st Century Driver

Scientific American, January 2001

Number of Years0 1 2 3 4 5

Pe

rfo

rma

nc

e p

er

Do

llar

Sp

en

t

Data Storage(bits per square inch)

(Doubling time 12 Months)

Optical Fiber(bits per second)

(Doubling time 9 Months)

Silicon Computer Chips(Number of Transistors)

(Doubling time 18 Months)

Imagining a Fiber Optic Infrastructure Supporting Interactive Visualization--SIGGRAPH 1989

“Using satellite technology…demo ofWhat It might be like to have high-speed fiber-optic links between advanced computers in two different geographic locations.”

― Al Gore, SenatorChair, US Senate Subcommittee on Science, Technology and Space

“What we really have to do is eliminate distance between individuals who want to interact with other people and with other computers.”

― Larry Smarr, DirectorNational Center for Supercomputing Applications, UIUC

ATT & Sun

Source: Maxine Brown

http://sunsite.lanet.lv/ftp/sun-info/sunflash/1989/Aug/08.21.89.tele.video

The OptIPuter Project – Removing Bandwidth as an Obstacle In Data Intensive Sciences

• NSF Large Information Technology Research Proposal– UCSD and UIC Lead Campuses—Larry Smarr PI– USC, UCI, SDSU, NW Partnering Campuses

• Industrial Partners: IBM, Sun, Telcordia/SAIC, Chiaro, Calient• $13.5 Million Over Five Years• Optical IP Streams From Lab Clusters to Large Data Objects NIH Biomedical Informatics Research Network

NSF EarthScope

http://ncmir.ucsd.edu/gallery.html

siovizcenter.ucsd.edu/library/gallery/shoot1/index.shtml

Interactive Visual Analysis of Large Datasets: Lake Tahoe Tidal Wave Threat Analysis

http://siovizcenter.ucsd.edu/library/gallery/shoot1/index.shtml

Cal-(IT)2 Visualization Center at Scripps Institution of Oceanography

Graham Kent, Scripps Institution of

Oceanography

3 Megapixel Panoram Display

OptIPuter End User Building Blocks:Scalable Compute + Storage + Viz Linux Clusters

• Cluster: 16 – 128 Nodes (Typically Two Intel Processors)• Storage: 0.1-1 TB per Node• Graphics: Nvidia Card Per Node• Visualization Displays: Desktop, Wall, Theatre, Tiled, VR• Specialized Data Source/Sink Instruments• All Nodes Have 1 or 10 GigE I/O• Clusters Connected by Lambdas or Fibers

Commodity GigE Switch

Fibers or Lambdas

OptIPuter is Studying the Best Application Usagefor Both Routed vs. Switched Lambdas

• OptIPuter Evaluating Both:– Routers

– Chiaro, Juniper, Cisco, Force10

– Optical Switches– Calient, Glimmerglass, BigBangWidth

• UCSD Focusing on Routing Initially• UIC Focusing on Switching initially• Next Year Merge into Mixed Optical Fabric

MEMS Switching (Layer 2) Routing (Layer 3) 1x 10x 100x OOO OEO OEOEO Calient, Glimmerglass Cisco Chiaro Data Plane Control Plane Routing

Optical Switch Workshop October 2002

OptIPuter Software Architecture for Distributed Virtual Computers v1.1

Layer 4: XCPNode Operating Systems

-configuration, Net Management

Grid and Web Middleware – (Globus/OGSA/WebServices/J2EE)

Physical Resources

DVC #1

OptIPuter Applications

DVC #2 DVC #3

Layer 5: SABUL, RBUDP, Fast, GTP

Real-Time Objects

Security Models

Data Services:DWTP

Higher Level Grid Services

VisualizationDVC/

Middleware

High-Speed Transport

Optical Signaling/Mgmt

Andrew ChienOptIPuter Systems Software Architect

½ Mile

SIO

SDSC

CRCA

Phys. Sci -Keck

SOM

JSOE Preuss

6th College

SDSCAnnex

Node M

Earth Sciences

SDSC

Medicine

Engineering High School

To CENIC

Collocation

Source: Phil Papadopoulos, SDSC; Greg Hidley, Cal-(IT)2

The UCSD OptIPuter DeploymentThe Dedicated Optical Grid: OptIPuter Campus-Scale Experimental Network

Forged a New LevelOf Campus Collaboration

In Networking Infrastructure

SDSC Annex

JuniperT320

0.320 TbpsBackplaneBandwidth

20X

ChiaroEstara

6.4 TbpsBackplaneBandwidth

2 Miles0.01 ms

OptIPuter 2004 @ UCSDCoupling Linux Clusters with High Resolution Visualization

10

1

4

102

OptIPuter Project Goal:Scaling to 100 Million Pixels

• JuxtaView (UIC EVL) on PerspecTile LCD Wall– Digital Montage

Viewer – 8000x3600 Pixel

Resolution~30M Pixels

• Display Is Powered By – 16 PCs with

Graphics Cards– 2 Gigabit Networking

per PC

Source: Jason Leigh, EVL, UIC; USGS EROS

NCMIR – Brain Microscopy

(2800x4000 24 layers)

Multi-Latency OptIPuter LaboratoryNational-Scale Experimental Network

Source: John Silvester, Dave Reese, Tom West-CENIC

Chicago OptIPuter

StarLightNU, UIC

SoCalOptIPuter

USC, UCI UCSD, SDSU

2000 Miles 10 ms

=1000x Campus Latency

“National Lambda Rail” PartnershipServes Very High-End Experimental and Research Applications

4 x 10GB Wavelengths Initially Capable of 40 x 10Gb wavelengths at Buildout

UIC

Chicago

Chicago Metro-Scale OptIPuter Uses I-WIRE and OMNInet Fiber

• Optically linking EVL and NU using I-WIRE and OMNInet Fiber

– OMNInet is a 10GigE Metro-Scale Testbed

– I-WIRE is a $7,500,000 State of Illinois Initiative

Source: Tom DeFanti, EVL, UIC

An International-Scale OptIPuter is Operational over the First Set of 76 International GE TransLight Lambdas

European lambdas to US–8 GEs Amsterdam— Chicago–8 GEs London—ChicagoCanadian lambdas to US–8 GEs Chicago—Canada —NYC–8 GEs Chicago—Canada —SeattleUS lambdas to Europe–4 GEs Chicago—Amsterdam–3 GEs Chicago—CERNEuropean lambdas–8 GEs Amsterdam—CERN –2 GEs Prague—Amsterdam–2 GEs Stockholm—Amsterdam–8 GEs London—AmsterdamTransPAC lambda–1 GE Chicago—TokyoIEEAF lambdas (blue)–8 GEs NYC—Amsterdam–8 GEs Seattle—Tokyo

UKLight

CERN

NorthernLight

Source: Tom DeFanti, EVL, UIC

The OptIPuter GeoWall2 at Supercomputing ‘03

See the National Center for Data MiningBooth 2935 SC ‘03