module 11: fiber optic networks and the internet

Slide #1CENTER FORINTEGRATED ACCESS NETWORKS

Module 11: Fiber Optic Networks and the Internet

Dr. Joe TouchPostel Center Director, USC/ISIResearch Assoc. Prof., USC CS and EE/Systems Depts.


What is a network?

Nodes Sources and/or sinks of bits

Links A way to exchange bits between nodes

What’s hard, then? Many types of nodes Many types of links Many ways to interconnect them


Simple Nodes

1-link nodes Sources emits signals

Sinks receives signals

What about the rest? All transform input signal into output signal

Difference is how “deep” into the signal you look…


OSI stack

Layers look increasingly deep ata signal (from the bottom up): App = user program Pres = formatting Sess = multi-transport coord. Transp = order, reliability, flow Net = logical addr, path Link = bits to codes, real addr. Phys = codes over a medium

7 - Application

6 - Presentation

5 - Session

4 - Transport

3 - Network

2 - Link

1 - Physical


Multi-Link Nodes 2-link nodes Amplifier

analog; decreases SNRphysical

Repeater digital; increases SNRphysical

Multi-link nodes Add-drop-mux (ADM)

adds &/or removes part of signalphysical, link layer

Switch permutes inputsphysical, link, network layer


Link types

Direction Simplex – unidirectional Duplex – bidirectional Half-duplex – time-share a single simplex link Radio, e.g., “here I am, over. OK, over.”

Full-duplex – two separate simplex links Multiaccess – more than 2 possible transmitters Broadcast multiaccess Any transmission is received by all nodes

Non-broadcast multiaccess (NBMA)


What’s inside a node?

One or more transmitters

One or more receivers

Both = transciever

Driver

LED or laser

Electrical output

Amp

Photodiode

Electrical input


Transparent vs. opaque

Transparent Not specific to a physical encoding E.g., analog amplifiers, analog switches only

Opaque Specific to a physical encoding E.g., repeaters, frame/packet switches

It’s all relative… WDM is opaque to frequency WDM is transparent to encoding within a frequency


Why are links difficult?

Delay Transmission = bits x bits/symbol x symbols/sec

(symbols/sec = baud rate) Propagation delay

Fiber = 1/R, e.g., 0.6c Coax = 0.6c, ladder = 0.95c, air = 0.9997c

Attenuation Decrease in signal magnitude

Noise Decrease in signal relative to noise

Dispersion Components of signal separate (wavelength, time,…)


Delay

Propagation is SLOWER than other media Symbol rate is FASTER

distance

time

ElectricalHigher propagation (shallow slope)Lower symbol rate (long dashes)

OpticalLower propagation (steep slope)Higher symbol rate (short dashes)


Fixing other link issues

Amplify Mitigates attenuation Increases noise

Filter Decreases noise Can also attenuate

Compensate E.g., fix chromatic dispersion (e.g., as a doublet does)

Regenerate E.g., as a repeater does (OEO)


Typical optical links

Span = fiber without any ‘fixes’ Terrestrial = 80km (60-100km) If one-hop, can be up to 200km Submarine = 50km

Type of optical links Dark = ‘transparent’ AND not connected at its ends Most are asymmetric due to amplifiers, repeaters, etc.


Multihop links

Multihop is easier to ‘wire’ N2 links vs. N

Ways to go multihop Passive switches Switches with amplification OEO


Multihop Topologies

Star Simple core – passive coupler; complex to wire Doesn’t scale well

Ring Easy to wire; hard to maintain during faults Connectivity scales; bandwidth does not

Mesh Most robust Requires more intelligent switching

Bus Like a star, but inconsistent attenuation, timing


Multihop considerations

Devices don’t all work in all topologies Complexity is only one issue

Fault tolerance “self-healing

Symmetry Timing Signal strength Fairness

Active vs. passive OEO “wiring”


Ways to share a link

Dimension of sharing Space division – one party per wire Time division – one party per timeslot Wavelength division – one party per frequency Code division – one party per code

Granularity of sharing Static vs. dynamic Synchronous vs. asynchronous


SONET – sync. TDM

N frames every 125µs90 octets per row

9 ro

ws

3 bytes S,L overhead per row

LS

P

P

PP


SONET speeds

Voice channel = 56 Kbps data + 8 Kbps signalling 7-bit samples, 8,000/sec (resolves 0-4Khz, i.e., voice)

OC = Optical Connect OC-1 – one frame every 125µs

Raw = 9 rows x 90 bytes/row = 51.84 Mbps total Payload = 9 rows x 87 bytes/row = 50.112 Mbps data Exactly 783 voice channels

OC-N = N frames per 125µs Frames are byte-interleaved (why?)

Common rates are OC-3 (155Mbps), OC-12 (622Mbps), OC-196 (10Mbps), OC-768 (40G)


SONET Hops Path

Between endpoints of SONET circuit; controls path Line

Terminated at ADMs, muxes; controls multiplexing, timing offsets Section

Framing over a single hop; alarms, parity check, control channel Regenerated from scratch at a repeater

T

T

T

Mux

MuxM

ux Mux

TADM


SONET Extensions

VCAT – virtual concatenation Inverse muxing (a.k.a. striping) Aggregate K*STS-1 to emulate STS-K (a.k.a. STS-Kv) vs. Contiguous concat (CCAT)

LCAS – link capacity adjustment scheme Dynamic, “hitless” VCAT Adjust VCAT without tearing down/restarting a path

NB: SDH (ITU) != SONET (US) Like ethernet framing != 802.3 VCAT, LCAS are ITU

11/15/2010 20


What SONET Really Means

Precise timing Fixed-frequency bitstreams Fixed-latency (bit stream arrival doesn’t shift)

Point to point links with add/drop muxing Looks like a train to you Traincars interleave like cars

Fixed bandwidth boundaries (K*51Mb) Still just a bit stream Needs GFP, HDLC, etc. to frame IP packets


ATM – async. TDM

Originally the “killer” replacement for IP Small, fixed-sized cells, but NOT time-aligned like SONET Efficient switching (multistage switches) Efficient interleaving without per-packet delays Complete, kitchen-sink system

Self-inhibiting design 48 byte payload – bad compromise between US and EU Cell size based on 64 Kbps telephone circuit Prime cell size (53B) defeats alignment efficiencies Complexity shifted to endpoints (segmentation and reassembly)

Result Temporarily used as L2 layer; now basically gone

11/15/2010 22


Problems with TDM

Electronics can’t keep up with optics Can drive 10Gbps, not much faster But optics can support Thz bandwidths

Not transparent Electrical signals don’t propagate far without

repeaters Use WDM Smaller per-channel BW; easier to drive electronics Avoids dispersion problems


Dispersion and its compensation

Dispersion

Dispersion compensation

Time (or distance along a fiber)

Use two kinds of fiber (works like an achromatic doublet)


So what is WDM?

Like TV or radio channels Coarse = few channels, wide gaps Fine = many channels, narrow gaps

Basic components: Separate frequencies – gratings, filters Combine frequencies – couplers Change frequencies – wavelength converters


Typical WDM configuration

Demux = grating Mux = coupler (can use grating in reverse)

ROADM

MUX DEMUX

Dispersion Comp.


Combining WDM and TDM

Goal – avoid the need to reserve wavelengths Planning wavelengths is like booking trains Hard to plan; requires global coordination

Shared access without coordination? Collisions – detect and retry Replace collision with increased noise – coding

OCDMA Optical CDMA with non-orthogonal codes Shared access with graceful degradation under overload


OCDMA

Code = K chips Chip = multidimensional

value, e.g., wavelength and polarization

Codes are not orthogonal Electronic ones are; signals

can cancel Too hard to phase-align optics

to enable destructive interference

λ1

λ2

λ3

λ4

λ5

λ

2 - Dimensional User

Tc1 Tc2 Tc3 Tc4 Tc5

t

λ1

λ2

λ3λ4

λ5

λ1

λ2

λ3λ4

λ5

λ

Codeword (λ, t)Tc1 Tc2 Tc3 Tc4 Tc5


Media Access Control

Shared media challenge Without control, throughput is low and collapses easily

Determines: Transmission order – avoid starvation Transmission priority – for control frames, QoS Line acquisition method – sequence (token), idle channel

detection, timing (TDMA), explicit allocation (FDMA) LAN dimension – size of the ring/bus Transmission duration – coupled to line acquisition, order, and

efficiency (e.g., burst extension in 1Gbps)

Effi

cien

cy

Load

Effi

cien

cy

Load


Pre-Internet

Different network stacks Gateway translators between each pair

Net ANetB Net C


Heterogeneity leads to layering

M different interacting parties need M2 translators

or

M translators + common format… i.e., a layer


What is the Internet?

One protocol to bind them all… IP datagrams as the common interoperation layer

Internet

Net ANetB Net C


The Hourglass Principle

Common interchange format between layers

HTTP/DNS/FTP/NFS/IM

TCP/UDP/SCTP/RTP

Ethernet/FDDI/Sonet

λ PPM, λ CDMA, e- NRZ, e- PCM

HTTP DNS FTP NFS IM

λPPM λCDMA eNRZ ePCM


Timeline (RFC2235)

1945 – WWII ends; V. Bush “As We May Think” (WWW) 1958 – Sputnik launched; ARPA created 1969 – Woodstock; ARPA project sends first packets 1973 – Ethernet 1977 – email standardized 1983 – NCP to TCP/IPv4 switchover 1985 – DNS (vs. hosts.txt file) 1988 – TCP congestion control 1989 – BGP 1991 – WWW 1992 – IP multicast; IP overlays 1997 – 802.11/WiFi; QoS/RSVP 1998 – IPsec 1999 – P2P (Napster) 2000 – IPv6


OSI stack

Travel top-down to the physical App = user program Pres = formatting Sess = multi-transport coord. Transp = order, reliability, flow Net = logical addr, path Link = bits to codes, real addr. Phys = codes over a medium

7 - Application

6 - Presentation

5 - Session

4 - Transport

3 - Network

2 - Link

1 - Physical


Layer details – RFC1208

Physical: given bits, send them on a link Optical: 4B/5B encoding (e.g., Fiberchannel, FDDI), PPM Electrical: Manchester, NRZ, QPSK, QAM

Link: transfer IP packets between adjacent IP nodes “Frame” IP to link address, IP packet to link frame translations

Network: transfer IP packets between non-adjacent IP nodes “Packet” Endian conversion (IEN 137) for addresses (common byte order)

Transport: convert services to IP packets “Segment” TCP: convert user ordered bytestream to segments that fit in IP packets, provide

reliable copy of bytestream at receiver, with congestion control UDP: convert user messages to segments in IP packets DCCP: convert user messages to segments in IP packets, with congestion control


Stack Traversal

7 - HTTP

6 - XML

5 – (none)

4 - TCP

3 - IP

2 – Eth./802.3

1 – 802.11

HTTP

XML

(none)

TCP

IP

PPP

WDM

E/802.3

802.11 GbE

IP

E/802.3

GbE

SONET

WDM WDM


What IS the Internet?

Packets Variable-sized, self-addressed data units

Common message format One universal interchange for data unit: IP packet

Best effort delivery NO guarantees

Globally unique IDs Name = location = forwarding indicator

Local forwarding decisions Based on longest-prefix matching

internet->Internet A deliberately contagious disease (transitive closure)

11/15/2010 38


Circuits – a sure thing

Trains on a train track Scheduled in advance Allocated whether in use or not Resources locked along entire path Path is fixed

Guarantees no competing traffic Fixed delay, fixed jitter, fixed capacity, lossless Can’t share resources concurrently


Internet Best Effort – NO guarantees

Cars on a highway No need to schedule Resource used only during transit Path can vary, even given identical header

Focuses on sharing Aggregate, concurrent resource use Results in variable delay, variable jitter, variable capacity, and

loss


Bellheads vs. Packetheads

Bellhead Smart core, simple edge

Cheaper for a monopoly Scarce resources motivate

Provider controls services

Packethead Simple core, smart edge Users control services

Slide #42CENTER FORINTEGRATED ACCESS NETWORKSInternet-2 July

20 200442

Path to Optical Routers

Evolution of electronics

Evolution of optics

VCswitches

Tag-switchedpaths

Line-raterouters

WDM Burstswitching

Opticalrouters


20 200443

Current Optical Focus

WDM as a bonus Needed to overcome dispersion Can be used to partition? or route?

Connection-based/-like traffic ATM/MPLS flow-based setups (MPλS, SWAP) BUT: Setup doubles connection latency

Packet-train setup on-the-fly (OBS, TBS) BUT: Setup requires large gap after first packet

BUT: Both expect long flows or aggregation


20 200444

Goal – Optical Internet

IP over light No setup Single terabit channels

(no WDM ) Works for short flows,

or for single packets

Implications One channel for routers Use wavelengths for link coding


20 200445

Challenges

Router Design Forwarding via partial filters TTL decrement IP checksum Queue-free switch design

LAN Issues (second part of this talk) OCDMA MAC design NIC design LAN architecture issues


IPv4 Header

Ver IHL TOS Total Length

ID Flags Fragment Offset

TTL Protocol Header Checksum

Source Address

Destination Address

Red = changes per hopOrange = indexed per hopYellow = changes on generation

Key IPv6 changes:longer addressesno ID or checksum


20 200447

xElectronicswitch fabric

Inside Current Routers

Forwarder + switch fabric Queues everywhere

Forwarding tableForwarderO/Econverter


Forwarder Functions

Filter Mark, drop as needed

Lookup next hop Longest prefix match

Decrement the TTL To prevent loops

Recompute the IP checksum IPv4 only, but seems persistent

Queue Random access storage for reordering, delay

Checksum

Decr. TTL

Addr match

Filter

Queue


Optical correlator

Sequence of Bragg filters Tuned to match 0,1,X 0,1 requires pairs, X is pass-through

0

1

1 1 1 0

0 0 0 1


Forward via Filters

Bit-subset groups share next-hops Remainder to helper router

R = 0%

1 1 0 1‘MATCH’SignalAND

Input

Threshold = 3

“1” “1” “0” “1”

R = 0%R = 0%R = 0%

Threshold = 0

“1” bits correlator

Match = ‘high’

Match = ‘low’NOT

“0” bits correlator

“1” “1” “0” “1”


TTL Decrement

Unsigned, 8-bit field Decrement by 1 each IP hop Drop if zero before decrement

Current design: 8-bit parallel Arithmetic subtract-by-1

+

0 1 1 0 1 0 0 0

1 1 1 1 1 1 1 1

0 1 1 0 0 1 1 1


Optical Decrementer

Serial LSB-first: Invert until 1 Stop @ 1st “1 (delete if no “1”)

D


All-Optical Decrementer

Implemented using optical latch

Replace latch with fast latching laser

Electronic controlElectronic control

λ MOD

SOA λ1(CW)

Signal inversion10 Gbit/s NRZ

“ databar”

PD

“data”

D-flip flop

MODλpPPLN

D Q

Q

MODλpPPLN

λ packet out w/updated TTL

1 MOD

SOA λ1(CW)

Signal inversion10 Gbit/s NRZ

“ ”

PD

“data”

TTL start

D-flip flop

MODλpPPLN

D Q

Q

MODλpPPLN

2


Internet Checksum

16-bit, 1’s complement sum In 2’s complement sum Add carry back in Can be done in words, doubles, etc. with a folded

result…

Current electronic hardware: 2’s complement accumulators Groups of full-adds; carries wired in a loop


Fast Parallelized Checksum

Recognizes symmetry in 1s complement adds Carries loop around

Xi

CoYi

Ci

SiXj

CoYj

Cj

Sj

Xk

Cok

Yk

Cik

Sk

Xl

Col

Yl

Cl

S

Ii Ii IiIi


Optical Checksum

Serial 1-bit full-adder

Xi

Co

Yi

Ci

S

16 bit delay

16 bit delay


Avoiding Queuing

Electronic routers queue via RAM VOQ requires random-access storage Store for delay Store for reordering

Optical routers can queue But cannot store for arbitrary periods Consider other kinds of queues, e.g., “conveyor

queues”


Packet Aggregator

Supports access networks

With a multiplexer (like horizontal Tetris)


PC Switch Architecture


Various PC-OQ Tests


Number of Packets in Contention

CDF of the number of packets in a contention graph for Poisson arrival with multiple distributions of packet length (mean = 165). The distribution with lower variance is listed first.


Results – no impact

Throughput of a 32x32 switch for quasipoisson-expo(165) for unbuffered switch and precognition optical switch running exhaustive search algorithm.


Lookahead-Shift Mux


Key Properties

Packet-oriented No seg/reassy Native support for variable length

Shifts only No reordering No recirculation

Simple processing One-pass Encourages batch processing Small holding area


Results are Promising…


Optical Interfaces

Back to wavelengths Here they help

MAC protocols Manage shared-access media Avoid congestion collapse, starvation

NIC design Push functions into optics (performance) Coordinate coding, power, timing issues

Overall arch. Issues “Impedence matching” LAN/Access Net/WAN

11/15/2010 9:34 AM

66


Optical MAC Issues

Electronic collision detection (CD) 2 outcomes: 1 or none on channel

“If I can’t hear, nobody can”

Detected by matching sent to received (trivial)

Optical interference detection (ID) 4 outcomes:

I’m OK, they’re OK (no interference) I’m OK, they’re not (destructive) I’m not, they’re OK (self-destructive) I’m not, they’re not (mutually destructive)

Infeasible to detect state of others Amplified by additive nature of noncoherent light


Does a MAC matter?

Throughput improves with good scheduling

• Simulation results• Results are

averaged over 100 trials

• Random phases are chosen uniformly over N

• Results are similar for smaller codesets

• The “good” phasing algorithm is not the optimal algorithm


Interference Avoidance

Electronic: State detection (feedback + matching) Output scheduling (backoff algorithms primarily)

Optical: State Estimation Line state cannot be completely known

State is the set of codes in active use Can measure sum of codes Cannot decompose sum into component codes

No way to factor a sum! Transmission Scheduling

Chip shifting to avoid interference


Interference Avoidance MAC

More complex than CDMA State Estimation Transmission Scheduling

Three variants All avoid collapse in simulation

Vs. Aloha (“no MAC”) One has lowest loss in simulation

Threshold Scheduling (TS) All compatible with CCM

Loss proven by real testbed 4 user testbed 6dBM benefit for TS

Implemented Hardware demonstrated Aloha

(no MAC)

IA-MAC(TS)


Overload analysis N=100

• Simulation results– N=100– Same parameters

otherwise

100 users transmitting

at load of 0.01 each

10000 users transmitting

at load of 0.01 each


Hardware design

Bus

Optical CDMAReceiver

Transmitbuffer

State estimation

module

Transmission scheduling

module

Sampling module

Codewordbuffer

Optical CDMATransmitter

Synchronization module

Receivebuffer

Ranging module

Bus

Transmitfiber

Receivefiber

Controller Feasibillity based on:

10 Gc/s N=100, w =3, = 1, K=3 Diameter of network =

2000m 100 nodes Transmission scheduling

Threshold, th = 0.3 State estimation

Continuous, window ne= 16

Expected utility @load=1.0 Hardware MAC

= 0.25 through No MAC = 0.05 through

Network interface card


MAC Implementation


OCDMA NIC design

Asynchronous OCDMA Avoids need for endpoint sync Code-cycle modulation (CCM) is pair-wise equivalent

of sync; increases throughput of n-chip code by a factor of log2(n)

CCM matches on all n-chip shifts Splitting the signal n-ways defeats the benefit We developed a novel receiver that matches on any

n-chip shift with out splitters


Circular Correllator

Matches any chip-shift without splitter


Receiver Module

Circular correlator on all wavelengths to generate match signal w/shift indicated

11/15/2010 9:34 AM

76


Continuous Reception

Single buffer needs 1-codeword gap Use “double-buffering” tactic:

11/15/2010 9:34 AM

77


Network Arch. Issues

Location of tuners Fixed receiver, variable transmitter fits IP Converse requires a coordination channel

Avoid sequences and hierarchies of contention channels Tokens avoid contention sequences Full routers needed to avoid hierarchies

“Impedence matching” Changes in broadcast, MAC creates boundary

interactions that decrease efficiency, need buffering

11/15/2010 9:34 AM

78


Grand challenges

Ps switching Random-access buffering LSI integration Low signal loss switching Designing to “think optical”


More info…

Many collaborators: Alan Willner, USC Joe Bannister, Aerospace Corp. PhD students: P. Kamanth, S. Suryaputra

Many papers And a few patents…

Many project URLs: www.isi.edu/odcma - NIC/MAC issues www.isi.edu/pow - TTL, header match www.cian-erc.org – precog, Tetris switch, checksum

module 11: fiber optic networks and the internet

Documents