specifying optical fiber for high-speed data centers - john kamino - ofs

Specifying Optical Fiber for HighSpecifying Optical Fiber for High--Speed Data CentersSpeed Data Centers

John KaminoJohn [email protected]@ofsoptics.com

Agendag• Multimode Fiber Market Drivers• Multimode Standards and Applications• Value Propositionp• Future of Multimode Fiber• Conclusions• Conclusions

IP Traffic Growth• Global IP traffic will quadruple from 2010 to 2015• Annual global IP traffic will reach nearly 1

zettabyte (1x1021 bytes) by the end of 2015• 15% of traffic will be from non-PC devices by 2015• In 2015 wired devices will account for 46% of IP

ff h l d l d lltraffic, while Wi-Fi and wireless devices will account for 54%.

Cisco Visual Networking Index:F t d M th d l 2010 2105Forecast and Methodology, 2010-2105June 2, 2011

IP Traffic Growth" Cisco Visual Networking Index (VNI): Forecast and Methodology, 2010-2015"

June 1, 2011PB/Month June 1, 2011

70000

80000

90000PB/Month

Mobile data

32% CAGR!

50000

60000

70000 Mobile data Managed IPFixed Internet

30000

40000

0

10000

20000

Mobile: Includes mobile data and Internet traffic generated by handsets, notebook cards, and mobile broadband gatewaysInternet: Denotes all IP traffic that crosses an Internet backbone

2010 2011 2012 2013 2014 2015

Internet: Denotes all IP traffic that crosses an Internet backboneManaged IP: Includes corporate IP WAN traffic, IP transport of TV/VoD

Worldwide Data Center Server Growth

• Data rates continue to increase

• Server port growth

Ethernet Fiber TransceiversLightCounting "Worldwide Sales of Optical Transceivers (Historical Data and Forecast)"

March 28, 2011

14,000,000

16,000,000100 GigE

40 GigE

10 GigE LRM total• Server port growth resumed in 2010

• Fiber volumes will continue to increase 6 000 000

8,000,000

10,000,000

12,000,000g

10 GigE LX4

10 GigE SR Total

GigE Fiber Total

Fast Ethernet

continue to increase• Significant 10GigE

growth beginning in 2010 -

2,000,000

4,000,000

6,000,000

2010 2008 2009 2010 2011 2012 2013 2014 2015

What is happening today?pp g y

PERCS Supercomputer2011

Roadrunner Supercomputer2008

“Need for Higher Density and Lower Power Interconnects for Future HPC and Servers”, January, 2011Petar Pepeljugoski, Marc Taubenblatt, and Jeff KashIBM

New York Times, February 16, 2011

IBM

Next Generation Speeds Coming!p g

http://connectedplanetonline.com/business_services/news/ieee-eyes-next-ethernet-speed-standard/

North America Multimode Fiber Demand

Multimode Fiber Demand North AmericaMultimode Fiber Demand - North AmericaCRU February 2011

1 000

1,200OM4OM3

IP traffic and server growth drive fiber demand

Virtualization increasing server d b d idth d d

600

800

1,000OM2OM1

usage and bandwidth demands

Servers requiring multiple Ethernet connections

Bandwidth requirements

200

400

600Redundancy

10Gbps server links drive 40Gbps uplinks

0

200

2008 2009 2010 2011 2012 2013 2014 2015 2016

Multimode Fiber TypesypMultimode (described in the industry using primarily the ISO/IEC 11801 designations)

ISO/IEC 11801 IEC 60793-2-10 TIA/EIA ITU-T62.5/125 OM1(1) A1b 492AAAA ---

Fiber TypeIndustry Standards

50/125 OM2(2) A1a.1 492AAAB G.651.150/125 OM3 A1a.2 492AAAC ---50/125 OM4 A1a.3 492AAAD ---

(1) OM1 is typically a 62 5um fiber but can also be a 50um fiber

ISO/IEC 11801

IEC 60793-2-10

"Generic Cabling for Customer Premises""Product Specifications - Sectional Specification for Category A1 Multimode

( ) OM1 is typically a 62.5um fiber, but can also be a 50um fiber.(2) OM2 is typically a 50um fiber, but can also be a 62.5um fiber.

IEC 60793-2-10

TIA/EIA-492AAAx

ITU-T G.651.1 "Characteristics of a 50/125 um Multimode Graded Index Optical Fibre Cable for the Optical Access Network"

Fibres""Detail Specification for… Class 1a Graded-Index Multimode Optical Fibers"

Multimode Fiber Types, Performance Grades

Min BandwidthMin Bandwidth(MHzkm) Fiber

Type

Wavelength

(nm) Max Loss (dB/km)

OFL BW

EMB

OM1 850 3 5 200 n s

OFL BW = Overfilled Launch BandwidthOM1

62.5 µm 850

1300 3.51.5

200500

n.s.n.s.

OM2 50 µm

850 1300

3.5 1.5

500 500

n.s. n.s.

EMB = Effective Modal Bandwidth

Overfilled Launch Bandwidth

50 µm OM3 50 µm

850 1300

3.5 1.5

1500 500

2000 n.s.

OM4 850 3.5 3500 4700

(also known as “Laser” BW)

OM4 50 µm

850 1300

3.51.5

3500500

4700n.s.

OMx designations are from ISO/IEC 11801International Cabling Standard

OM4 Multimode Fiber Standardization

• Complete in both TIA and IECTIA 492AAAD– TIA-492AAAD

– IEC 60793-2-10, Fiber Type A1a.3

• Specifications:Specifications:– Effective Modal Bandwidth (EMB) >/= 4700 MHz-km

• Allows 2 methods for verification– DMD Masks or EMBc• DMD Mask method shown to be more stringent

– OFL Bandwidth @850nm >/= 3500 MHz-km• Ensures performance with sources that launch more power in outer modes.

– OFL Bandwidth @1300nm >/= 500 MHz-km • Ensures backward compatibility for FDDI, 100BASE-FX, 1000BASE-LX, etc.

Why is OM4 important?Fiber dominates in Access to Distribution and Distribution to Core links.

Alan Flatman – Principal Consultant, LAN Technologies, UK“Long Data Center Links vs. Length”IEEE802.3ba, Jan. 2008, Flatman_01_0108

Evolution Of Short Reach Applications

40 000

Data Rate(Mbps)

T d 40/100 GbE

10,000

40,000 Trends:LEDs Lasers (faster)OM1 OM4 (farther)

1 GbE

10 GbE

850 or

40/100 GbE

850 or 1300 Laser

50 OM31000

( )

Fast

1 GbE

850 or 1300

850 or 1300Laser

50 OM3

50 OM350 OM4

SM

100 FDDI

1300 LEDEthernet

Fast Ethernet

1300 LED62.5 μm

1300Laser

50 OM250 OM3

50 OM4SM

1

10LED

62.5μmEthernet

850 LED62.5 µm

μ50 OM4

62.5 OM1SM

20101985 1990 1995 2000

Applications Mappingpp pp g

Application Data Center Lg. Data Center Very Lg. Data Center Building Backbone Campus Campus

Link Speed

1 Gb/

Building Backbone Building Backbone Building Backbone Campus Backbone Backbone Backbone

1 Gb/s

10 Gb/s OM4 MultimodeLaserWave 550 Fiber

OM3/OM4 Multimode FiberLaserWave® 300 or LaserWave 550 Fiber

40 Gb/s

OM4 MultimodeLaserWave 550 Fiber

OS1/OS2 Single-mode FiberAllWave® Fiber

100 Gb/s

Link Distance 100m 150m 300m 550m 1000m >1000m

40G & 100G Ethernet (IEEE 802.3ba)

Reach & Media: 40 Gb/s for servers, HPC, SAN, NAS

− 10km on SMF (1310nm) 40GBASE-LR4

− 100m on OM3 MMF (850nm) 40GBASE-SR4

150 OM4 MMF (850 ) 40GBASE SR4− 150m on OM4 MMF (850 nm) 40GBASE-SR4

− 7m over copper 40GBASE-CR4

− 1m over backplane 40GBASE-KR4

100 Gb/s for switching, routing, aggregation− 40km on SMF (1310nm) 100GBASE-ER4

− 10km on SMF (1310nm) 100GBASE-LR4



− 7m over copper 100GBASE-CR10

High Speed Short Reach Technologies:Multiple Fiber Parallel SystemsMultiple Fiber Parallel Systems

for 40G:

• One 12-fiber cable– duplex link– 8 active fibers

• 12 Fiber MPO connector• One wavelength per fiber• 4 x 10 Gb/s

High Speed Short Reach Technologies:Multiple Fiber Parallel Systems

for 100G:

Multiple Fiber Parallel Systems

for 100G:

• Two 12 Fiber Cables, or 24 fiber Cable

– 20 Active– Duplex link

• MPO connector– 2 x 12 fiber– 1 x 24 fiber

• One wavelength per fiber• 10 x 10 Gb/s

40G & 100G Ethernet – MDI RecommendationsReferences MPO interface req’s/specs of IEC 61754-7.f f q / p f

40GBASE SR440GBASE-SR4

Left 4 fibers are TxRight 4 fibers are Rx

(inner 4 fibers unused)

100GBASE-SR10

( )

I 10 fib T R R I 10 fib L ft Sid T

Option BOption A Option C

Inner 10 fibers, Top Row are RxInner 10 fibers, Bot Row are Tx(outermost fibers both rows unused)

Inner 10 fibers, Left Side are TxInner 10 fibers, Right Side are Rx

(outermost fibers each side unused)

Inner 10 fibers, Top are RxInner 10 fibers, Bot are Tx

(outermost fiber Top & Bot unused)

p(recommended)

p

40G & 100G Ethernet (continued)

• Reduced reach of 100m on OM3, 150m on OM4 d 10G (300 OM3 550 OM4) icompared to 10G (300m on OM3, 550m on OM4) is

due to relaxation of transmitter spectral width:

– from 0.45 to 0.65 nm

• There have been no changes to the fiberThere have been no changes to the fiber itself!

150 OM4 t d t t 95% f D t• 150m on OM4 expected to support 95%+ of Data Center links.

Data Center Environment

Migration to 10Gb/sCat6/6a copperActive Cable

OM3 or OM4 fiberOM3 or OM4 fiber(OM4 fiber recommended in latest draft of TIA-942A)

Migration to40 & 100Gb/s

OM3 OM4 fib 942A)OM3 or OM4 fiber

(OM4 fiber recommended in latest draft of TIA-942A)

Data from Alan Flatmann presented to IEE 802.3 High Speed Study Group January 2008

Applicationspp• Data Center – Today’s Focus

– OM4 multimode fiber is the recommended media in the– OM4 multimode fiber is the recommended media in the latest draft of TIA-942A, Telecommunications Infrastructure Standard for Data CentersO l OM3 d OM4 fib i d i th l t t– Only OM3 and OM4 fibers are recognized in the latest draft of TIA-942A

– Moving rapidly to optical based networking• Driven by bandwidth requirements

– Servers are capable of utilizing 10G speeds– Availability of 40/100G uplinks will drive adoption of 10G at the

server

– Driver for multimode fiber growth• Enterprise Local Area Networkp

Cost implications (100 G)p ( )CopperTwinax

OM3Multimode

OM4Multimode

OS1 Single-modeTwinax

CableMultimode

FiberMultimode

FiberSingle mode

Fiber

Distance 7m 100m 150m 10 km

TransceiverPrice

? CAPEXPrice

Cable price CAPEX

Power use per port 50 w 5 w 5 w 20+ w OPEX

Example Pricingp g• 100GBase-LR4 single-mode

http://www.pcmicrostore.com/SearchResult.aspx?q=kw:Juniper+100Gbase-LR4

• 100GBase-SR10 multimode (est) $15,000 Scott Kipp, BrocadeOFCNFOEC 2011 March 2, 2011March 2, 2011

Power Consumptionp• Lower power consumption critical as link density and speed

increaseincrease– 100G CFP single-mode transceivers consume 20+ watts– 100G CXP multimode transceivers consume ~5 watts

• Savings ~ 15 watts/transceiverC li th 15 tt /t i• Cooling – another 15 watts/transceiver

i / ( h d) l i li d b• 4 transceivers/server (2 on each end), multiplied by hundreds or thousands of servers per data center can consume significant amounts of power!!consume significant amounts of power!!

Single-mode vs. Multimode Module Size

• Significantly larger f t i t f i l dfootprint for single-mode CFP module

• Much lower faceplate densitydensity

• 4 single-mode modules in 1U footprint vs. 16-32 multimode modules!

Comparison between Single-mode and Multimode Fibermode and Multimode Fiber

SystemsTraditionally, optoelectronics have driven the cost difference between single mode and multimodesingle-mode and multimode

Single-mode CWDM system– Pro: Lower cabling cost– Con: Significantly higher transceiver cost– Con: Higher power consumption– Con: Larger size

• OM3 and OM4 multimode parallel systems– Pro: Much lower transceiver cost using existing 10Gb/s VCSELs– Pro: Lower power consumption– Pro: Smaller footprint– Con: Higher cabling cost

What does this tell us?• Data centers will move towards more optical

interconnects as bandwidth requirements increaseinterconnects as bandwidth requirements increase

• Multimode solutions have key advantages over both i l d d l isingle-mode and copper solutions

• Fiber bandwidth will be a key parameter

• Lower power consumption will be a key driver

So what about new bend insensitive multimode fibers?

What is bend insensitive multimode fiber?(The simple description)(The simple description)

Bend loss100 turns

Bend loss 2 turns

Bend loss 2 turns

37.5 mm radius

15 mm radius

7.5 mm radius

7 5 mmStandard 50/125 fiber

850 nm0.5 dB1300 nm

850 nm1 dB1300 nm

850 nm? dB1300 nm

7.5 mm attenuation

performance not specified for

standard 50/1251300 nm0.5 dB

1300 nm1 dB

1300 nm? dB

Bend 850 nm 850 nm 850 nm

standard 50/125 fiber

Bend insensitive multimode fiber

850 nm0.5 dB1300 nm0 5 dB

850 nm0.1 dB1300 nm0 3 dB

850 nm0.2 dB1300 nm0 5 dB0.5 dB 0.3 dB 0.5 dB

What parameters impact system performance?performance?

• Interoperabilit / Connection properties• Interoperability / Connection properties

• BandwidthBandwidth

• Reliability

It is desirable for optical fibers to be optimized for system performance

Comparison of wave guides

Bend Insensitive Multimode FiberStandard Multimode Fiber

• Trench is added to the BI-MMF to improve macrobend performance

Comparison of wave guidesp gBend Insensitive Multimode FiberStandard Multimode Fiber

Strongly guided modes

Weakly guided modesy g

Leaky modes

• In both STD and BI-MMF mode groups 1 – 17 are strongly guidedIn both STD and BI MMF mode groups 1 17 are strongly guided• In STD MMF mode groups 18 – 19 are loosely guided• In BI-MMF mode groups 18 and 19 are strongly guided

– leaky mode groups > 19 can also propagatey g p p p g

Mode groups 18-19 are ignored in bandwidth calculations and IEEE 10Gb/s link model!!

Impact of BI-MMF beam expansionReduced PerformanceReduced Performance

• Higher loss when BI-MMF is connected to Std-MMF Fiber A Fiber B

Median (dB)

Std.Dev. (dB)

μ + 3σ (dB)

Stand. Stand. 0 22 0 06 0 40• Similar trends likely when

different BI-MMFs are connected

OM3 OM3 0.22 0.06 0.40

BI-MMF Design Y

Stand. OM3 0.29 0.09 0.56

• Interoperability may be an issue with some BI-MMFs

BI-MMF Design X

Stand. OM3 0.31 0.11 0.64

issue with some BI MMFs

What about other parameters that impact system performance?system performance?



• Reliability


DMD Measurements of Comparable EMBc FibersEMBc Fibers

MW18 MW23 SLW0.088 0.223 0.094EMBc OFL8505512 3522

BI-MMF 2

MW18 MW23 SLW0.094 0.094 0.081EMBc OFL8505555 6876

BI-MMF 1

Light traveling in high order modes was ignored when EMBc weight functions were developed, is this assumption still valid for BI-MMF?

Systems Performance of Comparable EMBc fibers

1 E 061.E-051.E-04

EMBc fibers

1.E-091.E-081.E-071.E-06

ER

Back to BackBI-MMF 1BI-MMF 2 Note: EMBc

measurements are

1 E 131.E-121.E-111.E-10B

E measurements are similar, but DMD Outer Mask Width shows significant higher order mode problems

1.2dB3dB

1.E-151.E-141.E-13

-16 -15 -14 -13 -12 -11 -10 -9 -8( )received average power (dBm)

• Some BI-MMF fibers have large system penalties and others do not!

• Current OM3 and OM4 bandwidth

MW18 MW23 SLW EMBc OFL850BI-MMF 1 0.094 0.094 0.081 5555 6876BI-MMF 2 0.088 0.223 0.094 5512 3522• Current OM3 and OM4 bandwidth

measurement criteria developed for standard MMF may need to be modified

Standard Multimode FibersHigher order modes quickly attenuatedMaximum transmission distance!

1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

10 Gbps Detector10 Gbps850 nm

Laser

Detector

Core

Cladding

Bend Insensitive Multimode FiberHigher Order Modes Propagate in BIMMF!!Lower Bandwidth and Transmission Distance

1 0 1 0 1 0 1 0 1 0 1 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 10 Gbps850 nm

Detector850 nm

Laser

Core

CladdingHigher order modes not accounted for with current bandwidth measurements and standards!

What about other parameters that impact system performance?system performance?



• Reliability


Fiber Reliability

• Fiber reliability is a function of:O ti l R li bilit– Optical Reliability

• Maintaining optical signal through the link• Sufficient bandwidth to support future system upgrades

– Mechanical Reliability• Inherent glass quality – intrinsic strength• Inherent glass quality – intrinsic strength• Proof test level – extrinsic strength• Packaging – cable design• Deployment

Reliability yBend loss at 850-nm (2.5 Turns) for OM3 fiber types

10Bend loss at 850-nm (2.5 Turns) for OM3 fiber types

10

789

10)

Stand.BOMMF 1BOMMF 2BOMMF 3

BIMMF 1BIMMF 2BIMMF 37

89

10)

Stand.BOMMF 1BOMMF 2BOMMF 3

BIMMF 1BIMMF 2BIMMF 3

Mechanicalreliability concerns

4567

ndlo

ss (

dB) BOMMF 3BIMMF 3

4567

ndlo

ss (

dB) BOMMF 3BIMMF 3concerns

Optical reliability concerns

1234

Ben

1234

Ben

Risky Bend Warning!

01

2 3 4 5 6 7 8 9 10 11 12 13 14 15Mandrel radius (mm)

01

2 3 4 5 6 7 8 9 10 11 12 13 14 15Mandrel radius (mm)

Bend insensitive fiber is not a substitute for poor installation practices!

Relative diameter

r 3 5 mmr ?? mm

• Sharp 1/4 turns around fiber shelves can have very small radius

r=7.5 mm r=8.95 mmr=3.5 mmStd. Pencil

r=?? mmShelf Edge

p / y• Optical reliability can be maintained• Long term mechanical reliability is at risk

• 7.5mm is a reasonable radius to maintain, without additional risk of mechanical failure

• Bend optimized fiber is not a substitute for good cable management!!!p g g

Reliability of bare optical fiber with bends and tensionsingle ¼ turn calculation for 250 μm fibersingle ¼ turn calculation for 250 μm fiber

Fiber bend radius

Pull force on fiber

Fiber strain kpsi

20 year failure probability

(pounds force)

6.5 mm 0 ~100 < 1 ppm

6 5 4 ~300 50%6.5 mm 4 ~300 > 50%

4 lbs pulling force (~200 kpsi)Maximum stress atstress at corner

(~100 kpsi b di

Combined bends and tension is now part of new revision to:

bending strain) IEC TR 62048 Optical fibers -

Reliability - Power law theory

Mechanical Reliability CriteriaWhat is the criteria for a data center?What is the criteria for a data center?

MDU FTTx Trunks and Metro Applications

Long haul Submarinepp

Users per fiber 1 10-100 100-1000 100-10,000

Accepted Less than 10 Less than 1 ppm in Less than 1 ppm in 0 ppm in 40 yearsAccepted Reliability Criteria

Less than 10 ppm in 20 years



0 ppm in 40 years target

Cost of failure 1 customer down, truck roll

Several customers down, need for immediate repair

FCC reportable incident, need for immediate repair

FCC reportable incident. Failure extremely costly (Ship must be used)

Minimum design bend

5 mm Radius 10 mm Radius 15 mm Radius 30 mm Radius

Summary of bend insensitive multimode fiber

• It is prudent to balance macrobend performance i h h fib i l di b d id h dwith other fiber parameters including bandwidth and

interoperability to optimize systems performance

• Bend insensitive multimode fibers are a new and un-standardized product. Efforts are currently underway in the standards bodies to study this new fiberin the standards bodies to study this new fiber.

• Wide spread adoption of this fiber should be limitedWide spread adoption of this fiber should be limited until technical issues are resolved in standards bodies

Why the higher speeds?y g p

M i t tMore interconnects and switches required

High speed connections simplify the network

Pictures presented by Adam Bechtel Yahoo! Chief ArchitectPictures presented by Adam Bechtel – Yahoo! Chief Architect IEEE 802.3 Plenary March 2007

Where is Multimode Going?Towards 4x25 Gb/s transmission!

• Data centers are overflowing with connections

Towards 4x25 Gb/s transmission!

g• The current Ethernet evolution path

– 10 Gb/s is a single fiber (Duplex cable) solution40 Gb/s is a 4X10 sol tion ith a 12 fiber cable– 40 Gb/s is a 4X10 solution with a 12 fiber cable

– 100 Gb/s is a 10X10 solution using a 24 fiber cable

• On this path, transition from 40 G to 100 G will require additional fiber

• 100 Gb/s 4X25 solution is an easier upgrade path!

OPPORTUNITY: A new fiber (OM5) may be required for the longest links

Conclusions• Data centers are shifting from copper to fiber

Copper is more costly and has limited transmission distance– Copper is more costly and has limited transmission distance– Fiber has much lower power consumption and can support longer links– Multimode fiber has key advantages over both copper and single-mode fiber

OM3 d OM4 M lti d fib i th di f h i f 40 100 Gb/• OM3 and OM4 Multimode fiber is the media of choice for 40-100 Gb/s transmission rates in data centers– OM1 and OM2 not recognized media in latest TIA-942A Data Center Cabling

draft

• OM4 is well accepted and has been incorporated in system standards– Recommended fiber in latest TIA-942A Data Center Cabling draft!

Th ill b hif ll l i i l i d fib i h• There will be a shift to parallel transmission over multimode fiber with MPO connections as network speeds exceed 10 Gb/s

• 4x25Gb/s transmission is on the horizon

Th k Y !Thank You!Visit OFS at the Exhibit Hall

specifying optical fiber for high-speed data centers - john kamino - ofs

Documents