ip ngn backbone routers for the next decade€¦ · ip/mpls backbone ip/mpls internet peering pdh...

43
Cisco Public 1 © 2010 Cisco and/or its affiliates. All rights reserved. IP NGN Backbone Routers for the Next Decade Josef Ungerman Consulting SE, CCIE #6167

Upload: others

Post on 14-Aug-2020

10 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

Cisco Public 1© 2010 Cisco and/or its affiliates. All rights reserved.

IP NGN Backbone Routersfor the Next DecadeJosef Ungerman

Consulting SE, CCIE #6167

Page 2: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 2

• Motivations for IP NGN

• Trends is IP/MPLS Core Design

• Router Anatomy Trends

• Latest Product Updates

• Switching Fabric Technologies

• Network Processor Technologies

Page 3: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

Cisco Public 3© 2010 Cisco and/or its affiliates. All rights reserved.

Exponential Growth and Evolving Traffic Mix

Global IP Traffic2009 2014

Video & Multimedia

Mobile Internet

IT Services via Cloud

300+%Market Growth

39X Increasein Traffic

90% ConsumerTraffic

IP traffic will increase 4X

(767 exabytes by 2014)

Very Different

Traffic Profile

• IPv6 and IPv4 Address Exhaustion

• LTE moving from circuits to packets

• new access technologies – WiFi, FTTX

Source: Cisco Visual Networking Index—Forecast, 2009-2014

More Issues:

Page 4: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 4

Traffic

Revenue

Animated slide

MonetizationNew revenue streams

OptimizationEfficient delivery

Profitability

Page 5: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

Cisco Public 5© 2010 Cisco and/or its affiliates. All rights reserved.

Network Architecture Trends

Page 6: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 6

ACCESS AGGREGATION BACKBONE

SDH/ATM

Mobile

SDH

FR/ATM

Metro Ethernet

PDH MW SDH

FMC/

Packet

Core

ATM

IP/MPLS Internet

Peering

PDH

CPE

DSLAM

MSAN

CPE

IGW

BRAS

CPE

Regional

DWDM

Backbone

DWDM

SGSN

GGSN

IGW

INTERNET

100’s 10’s 1’s1000’s

2G

3G

Data

Centers

Page 7: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 7

ANYACCESS

AGGREGATION

CarrierE AGGREGATIONIP/MPLS

BACKBONEIP/MPLS

Internet

Peering

PDH MW

2G

3G

CPE

DSLAM

IGW

CGv6

BRAS

DWDM

INTERNET

CORE

Ethernet

MPLS-TP

MPLS

PON

LTE, WiFI…

EDGE

IP NGN

• single multiservice IP/MPLS network

• operational hierarchy

• Circuits (Pseudowires) and Clouds (IP)

100’s 10’s1000’s 1’s

Data

Centers

Page 8: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 8

ANYACCESS

AGGREGATION EDGE

IP/MPLS Core Internet

Peering

PDH MW

DPI

2G

3G

CPE

DSLAM

IGWBRAS

DN

VPN PE

IPTV PE

FMC PE

BB PE

P P BB PE

DWDM

INTERNET

CORE

Ethernet

MPLS-TP

MPLS

PON

LTE, WiFI…

AN

100’s 10’s1000’s 1’s

CarrierE AGGREGATIONIP/MPLS

Page 9: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 9

ANYACCESS

AGGREGATION EDGE

IP/MPLS Core Internet

Peering

PDH MW

2G

3G

CPE

DSLAM

IGWAN

P BB PE

DWDM

INTERNET

CORE

Ethernet

MPLS-TP

MPLS

PON

LTE, WiFI…

Backbone Optimization

• IP Offload links, 100GE

• Flexible Edge placement

• Consolidation techniques

Existing PE’s

Existing PE’s

DN

100’s 10’s1000’s 1’s

CarrierE AGGIP/MPLS

Page 10: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

Cisco Public 10© 2010 Cisco and/or its affiliates. All rights reserved.

Statistical Multiplexing(VPN optimized)

- O-E-O regeneration avoided as much as possibleno need for OTN Switching cross-connects in CEE countries

- Static long lambdas are usedno need for dynamic G.MPLS in the static Internet backbone

- Importance of OTN interfaces in routers (IPoDWDM)STM-256 (OTU3) and 100GE (OTU4)

PE P P PEPE PE

P P

PE P P PEPE PE

P P

Direct Link(Broadband optimized)

Saves Money, and Keeps the Core Hierarchy!

l1 l1

Traffic

Generator

Mngmt 1GE

Optical Monitoring Points

OSA

CRS-1

Poznan

CRS-1

Warszawa

40GE40GE613 km

Real-Life Example:

Warszawa-Poznan, 613km

40G over Siemens 10G WDM

Page 11: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

Cisco Public 11© 2010 Cisco and/or its affiliates. All rights reserved.

100GE•IEEE 802.3ba, ITU OTU4

• router interface

• router router or transponder

40GE

•IEEE 802.3ba, ITU OTU3e

•data center interface

• router to transponder

Link Bundling (LACP)•up to 64x TGE today (32x deployed)

• dynamic adaptable hash (also 3,5,7,11 links)

• 7-tuple hash for equal load-sharing

100GE•throughput (100GE is like a bundle 12-14 TGEs)

• no hashing inefficiencies, easy troubleshooting

• contribution HDTV is 1.24Gbps single stream!

Page 12: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco Systems, Inc. All rights reserved. Cisco Public 12

RSP

LC

LC

LC

LC

RSP

LC

LC

LC

LC

Cluster (ASR9000)

Key motivation is in the Access edge:

Simpler Access Dual-homing• scaling the L2/L3 control plane (not data plane)

Key motivation is in the Core:

Simpler Core PoP• scaling the non-blocking data plane

• back-to-back, 2+1, 8+2, etc.

Cluster(one L2 & IP/MPLS control plane)

Cluster + Satellites(remote linecards)

Multi-Chassis(one router)

RP

LC

LC

RP

LC

Multi-Chassis (CRS)

LC

LC

LC

LC

dRP

Page 13: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 13

1) Reduce the number of networks

IP NGN = single multiservice network

2) Reduce the number of layers

IP NGN = IP/MPLS + DWDM

3) Reduce the number of nodes

Direct Links = huge broadband traffic takes shortest path

4) Reduce the number of links

MPLS Technology = statistical multiplex and hierarchy

5) Innovate – make use of modern technologies

Moore’s Law = Lower TCO, Price/Gigabit, Watt/Gigabit

Optimization: How to move bits cheaper......reduce OPEX, CAPEX, and keep reasonable quality?

Page 14: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 14

7.757.49 7.65

2.77 2.69 2.74

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

8.00

9.00

0

2000

4000

6000

8000

10000

12000

14000

4/S 8/S 16/S

Watt

s

CRS Form Factor

Power @ 40G/slot

Power @ 140G/slot

Total Power

W/Gbps @ 40G/slot

W/Gbps @ 140G/slot

Page 15: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 15

• Silicon has fundamentally followed Moore’s law

• Optics is fundamentally an analog problem

$-

$5,000.00

$10,000.00

$15,000.00

$20,000.00

$25,000.00

$30,000.00

$35,000.00

$40,000.00

$45,000.00

$50,000.00

1997 1999 2001 2003 2005 2007 2010 2012

$/Gbps

Routers: 23% Cumulative Average $/Gbps Drop per year / fewer ASICsOptics: $/G stays flat (best case) or increases from one technology to the next

Cisco Core Router Example

0

5000

10000

15000

20000

25000

30000

35000

40000

45000

0.00

500.00

1000.00

1500.00

2000.00

2500.00

2007 2008 2009 2010 2011 2012 2013 2014

Cap

acit

y i

n G

b/s

Co

st/

Gb

($U

SD

)

DWDM Optic Cost and DWDM Capacity

10Gb 40Gb 100Gb Capacity

10G/40G/100G Networking Ports Biannual Worldwide

and Regional Market Size and Forecasts

May 2010

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

10G % ASP

40G % ASP

100G % ASP

Future Trend

Cisco Core Routing Example

Switch Engine

Client Optics

DWDM Optics

Infrastructure

Page 16: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

Cisco Public 16© 2010 Cisco and/or its affiliates. All rights reserved.

Router Anatomy Trends

Page 17: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 17

2004: Cisco CRS-1 – 40G (STM-256) per slotFocus on Quality (scale, modularity, resiliency)

IOS IOS IOS IOS

RP (active) RP (standby)

SPA

SPA

SIP-800

mid

plan

e 4

0G

Switch Fabric Cards

(all 8 active)

MSC-40 – 40G

NP

NP

Qbuff.

IOS

Q

Q

buff.

FP-40 – 40G

NP

NP

Qbuff.

IOS

Q

Q

buff.

56G rx

112G tx

49G rx

98G tx

SPP (Silicon Packet Processor)

- 40 Gbps, 80 Mpps [u-programmable]

- one for Rx, one for Tx processing

PLIM

mid

plan

e 4

0G

4, 8 or 16 Linecard slots + 2 RP slots

56G rx

112G tx

49G rx

98G tx

56G rx

112G tx

Page 18: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 18

IOS IOS IOS IOS

RP (active) RP (standby)

FP-140 – 140G MSC-140– 140G

SPA

SPA

SIP-800

mid

plan

e 4

0G

MSC-40 – 40G

NP

NP

Qbuff.

IOS

Q

Q

buff.

FP-40 – 40G

NP

NP

Qbuff.

IOS

Q

Q

buff.

NP’

NP’

Qbuff.

IOS

Q

Q

buff.

NP’

NP’

Qbuff.

IOS

Q

Q

buff.

PLIM

mid

plan

e 4

0G

14x 10GE

Switch Fabric Cards

(all 8 active)

4, 8 or 16 Linecard slots + 2 RP slots

100GE

QFA (Quantum Flow Array)

- 140 Gbps, 125 Mpps [u-programmable]

- one for Rx, one for Tx processing

141G rx

226G tx

mid

plan

e 1

40G

mid

plan

e 1

40G

2010: Cisco CRS-3 – 140G per slotFocus on Quality (scale, modularity, resiliency)

Page 19: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 19

2009: Cisco ASR9000 – 8x 10GE per slotCompact Router/Switch

IOS IOS

RSP (active)

IOS IOS

RSP (fab. active)

NP

NP

NP

NP

buff.

buff.

buff.

buff.

IOS

Transport LC – 40G

NP

NP

NP

NP

buff.

buff.

buff.

buff.

IOS

NP

NP

NP

NP

buff.

buff.

buff.

buff.

Transport LC – 80G

4 or 8 Linecard slots

Trident Network Processor

- 30 Gbps, 28 Mpps [u-programmable]

- shared for Rx and Tx processing

- one per 10GE (up to 8 per card)

NP

NP

buff.

buff.

Transport LC – 16x TGE OS

NP

NP

buff.

buff.

IOSNP

NP

buff.

buff.

NP

NP

buff.

buff.

NP

NP

NP

NP

buff.

buff.

buff.

buff.

IOS

Transport LC – 8x TGE OS

RSP (Route/Switch Processor)• CPU + Switch Fabric

• active/active fabric elements

92G

92G

46G

46G

92G

92G

X

Page 20: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 20

NP

NP

NP

NP

buff.

buff.

buff.

buff.

IOS

Transport LC – 40G

4 or 8 Linecard slots

IOS IOS

RSP’ (active)

IOS IOS

RSP’ (fab. active)

NP

NP

buff.

buff.

Transport LC – 24x TGE

NP’

NP’

buff.

buff.

IOSNP

NP

buff.

buff.

NP’

NP’

buff.

buff.

NP’

NP’

NP’

NP’

buff.

buff.

buff.

buff.

Transport LC – 2x 100GE

IOS

NP

NP

NP

NP

buff.

buff.

buff.

buff.

IOS

NP

NP

NP

NP

buff.

buff.

buff.

buff.

Transport LC – 80G

92G

92G

2011: Cisco ASR9000 – 2x 100GE per slotCompact Router/Switch

>400G

Page 21: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 21

IOS IOS

RSP (active)

F

buff.SP RP

IOS IOS

RSP (standby)

F

buff.

SP RP

2003: Cisco 7600 – 4x 10GE per slotThe Switch/Router

NP

NP

NP

NP

buff.

buff.

buff.

buff.

F

IOS

ES+

3, 4, 6, 9 or 13 Linecard/RSP slots

NP

NP

buff.

buff.F

IOS

ES+

NP

NP

NP

NP

buff.

buff.

buff.

buff.

F

IOS

ES+

NP

NP

NP

NP

buff.

buff.

buff.

buff.

F

IOS

ES+

20G

20G

Edge or Core LC – 40G Edge or Core LC – 20G

32G local

switching

Trident Network Processor

- 10Gbps full-duplex

- H-QoS, VPLS, u-programmable

EARL Switching ASIC

- 48 Mpps

- Catalyst 6500 compatibility

RSP (Route/Switch Processor)• CPU + Switch Fabric + Switch ASIC

• active/standby fabric elements

Page 22: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 22

IOS IOS

RSP (active)

F

buff.SP RP

IOS IOS

RSP (standby)

F

buff.

SP RP

2003: Cisco 7600 – 4x 10GE per slotThe Switch/Router

NP

NP

NP

NP

buff.

buff.

buff.

buff.

F

IOS

ES+

3, 4, 6, 9 or 13 Linecard/RSP slots

NP

NP

buff.

buff.F

IOS

ES+

NP

NP

NP

NP

buff.

buff.

buff.

buff.

F

IOS

ES+

NP

NP

NP

NP

buff.

buff.

buff.

buff.

F

IOS

ES+

Edge or Core LC – 40G Edge or Core LC – 20G

SIP-400

SPA

SPANP

buff.

NP

buff.

Bus (headers only) IOSbuff.

IOS

buff.

SPA

SPA

SIP-200

buff.

F

SPA

SPAIOS

NP

SIP-600

buff.

IPSEC

FWSM

ACE

buff.

buff.

buff.

buff.

buff.

buff.

buff.

Page 23: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 23

...and keep a reasonable quality?

1) Compact Anatomy

RSP, Route/Switch Processor (instead of RP and FC)

Ethernet-oriented Linecard (non-modular, less memory)

2) Linecard Architecture

Multiple smaller NP’s (eg. 4x 10G instead of 1x 40G)

One NP is shared for Rx and Tx (not dedicated NP’s per Rx and Tx)

Multiple smaller Fabric Ports (eg. 2x 20G instead of 1x 40G)

3) Special Core-facing Linecards

8/16 queues per port (instead of thousands)

lower-scale NP (no need for thousands of interfaces)

licenses for features that not everybody uses (eg. VPN, OTN, scale)

4) Oversubscribed Cards

2:1 ingress overbooking (eg. PON OLT Aggregation)

Page 24: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

Cisco Public 24© 2010 Cisco and/or its affiliates. All rights reserved.

IP NGN Routers Update

Page 25: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 25

# of Slots

Height

Linecard [Gbps]

System [Tbps]

Linecard [Gbps]

System [Tbps]

Per Slot [Gbps]

CRS-8/S

8 (+2 RP)

½ rack

40

.640

140/105

2.24

400

CRS-16/S

16 (+2 RP)

1 rack

40

1.28

140/122

4.48

400

Core

Maximum BW / Protected BW with a failed fabric element

CRS-MC (8+2)

128 (+2 RP)

10 racks

40

10.24

140/122

35.84

400

CRS-4/S

4 (+2 RP)

1/3 rack

40

.320

140/105

1.12

-

Chassis

2004

(CRS-1)

2010

(CRS-3)

Future

~7500 systems

~450 customers

(~80 are CRS-3)

Page 26: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 26

14x 10GBE-WL-XFP

Line-rate performance (140Gbps)

Configurable LAN/WAN PHY

Interface Modules (PLIMs)

Each PLIM requires FP140 or other forwarding card

20x 10GBE-WL-XFP

Oversubscribed (140Gbps)

Configurable LAN/WAN PHY

1x 100GBE

Line-rate performance (100Gbps)

CFP optics (LR4, 10km)

Page 27: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 27

MSC-40 – High-speed edge @ 40Gbps

H-QoS (8,000 queues), 800 interfaces, WAN

FP-40 – IP/MPLS Core &Peering @ 40Gbps

Per-port QoS, IP/MPLS, ACL, Netflow…

Forwarding Cards

MSC-140 – High-speed edge

H-QoS (64,000 queues), scale (12,000 vlans)

FP-140 – IP/MPLS Core & Peering

Per-port QoS, IP/MPLS, ACL, Netflow…

LSP-140 – MPLS Core P

Per-port QoS, MPLS, IP Multicast, limited IP

Page 28: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 28

IPv6 Transition: CGSE Carrier-Grade Services Engine PLIM

Introducing the new engine for massive Cisco CGv6 deployments (XR 3.9.1)

now: Cisco CRS

2011: XR12K, ASR9K

Cisco CGSE

20+ million active translations

100s of thousands of subscribers

1+ million connections per second

20Gb/s of throughput

XML API (eg. port-forwarding)

Netflow V9 translation logging

Security

IPv6 Transition solution feature set

Carrier-Grade NAT44 (3.9.1)

NAT64 stateless (3.9.3)

6rd BR (3.9.3)

NAT64 stateful (4.1.2)

DS-Lite, 4rd, dIVI – planned

Page 29: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 29

# of Slots

Height

Linecard [Gbps]

System [Tbps]

Linecard [Gbps]

System [Tbps]

Per Slot [Gbps]

ASR-9006

4 (+2 RSP)

¼ rack

120/80

.960

240

1.92

ASR-9010

8 (+2 RSP)

½ rack

120/80

1.92

240

3.84

Edge and Aggregation

~3500 systems

~500 customers

Maximum BW / Protected BW with a failed fabric element

Full Rack Box

20 (+2 RSP)

1 rack

240

9.6

360

Pizza Box

2 sub-slots

2 RU

-

.120

Chassis

2009

2012

Future

Page 30: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 30

Fixed Ethernet LCs:

Line Rate: 40xGE, 2x10GE+20xGE, 4x10GE, 8x10GE

Oversubscribed: 8x10GE (60G), 16x10GE (90G/120G)

Ingress/Egress H-QoS, Netflow, IPoDWDM (G.709, FEC, XFP),

Video monitoring, SyncE, E-OAM

L2 Scalability: 1MMACs, 8kBDs, 32kPWs

L3 Scalability:1M routes, 4kVRFs, 4kL3intfs

3 LC versions (16x10GE OS “Medium Queue” only):

+licenses: L3VPN (/LC), G.709 (/LC), vidmon (/chassis)

Modular LCs: SIP-700 + max 4x SPA

QFP based

20Gbps Full Duplex

Ph1: ChOC12

Line Card EFPsEgress

QueuesPolicers Buffering

Low Queue -L 4k 8/port 8k 50ms

Medium Queue -B 16k 64k 128k 50ms

High Queue -E 32k 256k 256k 150ms

ASR 9000 #10GE LR #10GE OS

6-slot 32 64

10-slot 64 128

Page 31: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 31

...and keep a reasonable quality?

1) Compact Anatomy

RSP, Route/Switch Processor (instead of RP and FC)

Ethernet-oriented Linecard (non-modular, less memory)

2) Linecard Architecture

Multiple smaller NP’s (eg. 4x 10G instead of 1x 40G)

One NP is shared for Rx and Tx (not dedicated NP’s per Rx and Tx)

Multiple smaller Fabric Ports (eg. 2x 20G instead of 1x 40G)

3) Special Core-facing Linecards

8/16 queues per port (instead of thousands)

lower-scale NP (no need for thousands of interfaces)

licenses for features that not everybody uses (eg. VPN, OTN, scale)

4) Oversubscribed Cards

2:1 ingress overbooking (eg. PON OLT Aggregation)

Page 32: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

Cisco Public 32© 2010 Cisco and/or its affiliates. All rights reserved.

Architecture Matters!

Page 33: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

IP/MPLS

NPU

IP/MPLS

NPU

IP/MPLS

NPUIP/MPLS

NPU

Forwarding Architecture 101

Ingress

Linecards

Egress

Linecards

TX

RX

TX

TX

TX

RX RX RX

UNICAST

to slot 3

MULTICAST

to slots 2,3,4

1

2

3

4

1 2 3 4

Fabric Port

• addressable entity

• singleduplex pipe

What’s the capacity?

4 fab.ports @10Gbps

non-blocking

ENGINEERING:

4 * 10 = 40Gbps fdx

MARKETING:

4 * 10 *2 = 80Gbps

IP/MPLS

NPU

IP/MPLS

NPU

IP/MPLS

NPU

IP/MPLS

NPU

SWICTHING FABRIC

IP/MPLS unaware part

speaks cells/frames

NETWORK PROCESSOR

IP/MPLS aware part

speaks packets

Page 34: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

CISCO INTERNAL

© 2010 Cisco and/or its affiliates. All rights reserved.

© 2011 Cisco and/or its affiliates. All rights reserved.

SMP Network Processor Example2010: CRS QFA (Quantum Flow Array)

BQS ASIC

- 64K queues

- 3L shaping

DRAM

Processing Pool

256 Engines

Resources & Memory Interconnect

headers only

Distribute & Gather Logic

SRAMRLDRAMDRAM TCAM4

PLU8/TLU5 Stats/Police ACL 125 Mpps, 140 Gbps

65nm, u-programmable

2 per LC (Rx, Tx)

Page 35: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 35

Bad Network Processor Example (non-Cisco)ACL performance impact

IP Performance Range

L3

L4

Vendor is Saving on Memory – ACL memory is shared with Route memory•Effect #1: ACL drastically impacts forwarding performance

•Effect #2: FIB cannot be hierarchical slow BGP convergence

Page 36: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 36

Switching Fabric and Multicast Good vs. Bad IPTV Experience

TX

TX

RX

RX

RX

RX

Ingress

LinecardsGood:

Egress Replication

• Cisco CRS, 12000

• Cisco ASR9K, 7600

10Gbps of multicast

eats 10Gbps fabric bw!

Bad: Binary Ingress Replication

• dumb switch fabric

• non-Cisco

10Gbps of multicast

eats 80Gbps fabric bw!

(10G multicast impossible)

TX RX

Ingress

Linecards

TX RXTX RXTX RX

TX RXTX RXTX RXTX RX

RXRXRXRX

RXRXRXRX

161

7181

9202

1222

3242

5262

7293

0313

2

333

46

364

08

09

10

11

12

13

14

15

04

05

06

07

02

01

03

363

8404

2444

648

Page 37: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 37

TX

ASR9000

• 80G eth. non-blocking with failed RSP

• cell dip is not an issue (super-frame based fabric, not cell based)

Active RSP80G 80G

TX RX

RX

CRS-1

• 40G eth. non-blocking with 1 or 2 failed fabric cards

TX RX

Good Fabric Redundancy

Active RSP

40G100G

40G

100G

X

X

CRS-1: 56G 49G 42G

CRS-3: 141G 123G 106G

X

failed RSP: 184G 92G

CRS-3

• 100G eth. non-blocking with 1 or 2 failed fabric cards

CRS-1: 112G 98G 84G

CRS-3: 226G 197G 169G

Page 38: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 38

Non-Cisco

• 40G system sold as 100G

•if one CMP fails, system turns blocking

• unreliable 100GE

Bad Fabric Redundancy

TX

Active SFM

Active SFM

100G 100GX

failed SFM/CMP: 105G 52G

- huge cell dip at >40G

- blocking with 100GE

RX

Non-Cisco

• 70G system sold as 80G

•minimum speedup

• huge cell dip

• multicast impacts unicast

TX

Active SCB

80G 80G

TX RX

RXActive SCB

RX

RX

TX

TX

Active SCB

X

failed SCB: 126G 84G

- huge cell dip at >75G

- blocking with m’cast

x8

x8

Page 39: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 39

Cell dip and SpeedupP

erc

en

tage

of

Lin

era

te

80%

60%

40%

20%

0%

100%

1N0%

40

10

0

20

0

30

0

40

0

50

0

60

0

70

0

80

0

90

0

10

00

11

00

12

00

13

00

14

00

15

00

Cell dip – per vendor @ 41B, 46B, 53B, 103B,...

The lower speedup, the more multicast, the more cell dips!

cell payload

[48B]

cell

hdr

[5B]

cell format

example:

Fixed overhead [cell header, ~10%]

Relative overhead [fabric header]

Variable overhead [padding]

40B IP

Packet:

IP Packet

[40B]

cell

hdr

[5B]

buffer

hdr

[8B]

Good efficiency1Mpps = 1Mcps

1Gb/s 1.33Gb/s

41B IP

Packet:

IP Packet

[first 40B]

cell

hdr

[5B]

buffer

hdr

[8B]

empty

[47B padding]

cell

hdr

[5B]

IP Packet [last 1B]

Poor efficiency1Mpps = 2Mcps

1Gb/s 2.6Gb/s

L3 Packet Size [B]

Cell Dip mitigation:1. Enough Speedup

2. Packet Packing (CRS)

3. Super-framing (ASR9K)

Another IP PacketPoor efficiency1Mpps = 2Mcps

1Gb/s 2.6Gb/s

Page 40: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 40

Quality Differences in 40G Solutions40 ≠ 40

Good:

40 Gbps per slot

non-blocking

Ingress Linecard Egress Linecard

56G 112G

Cisco CRS-1 (MSC-40)

16 slots

[16x16 matrix]

40G

NPU

40G

NPU

40G 40GTX RX

Good-enough?:

40 Gbps per slot

non-blocking!

10G 18.8G 18.8G

18.8G 18.8G

18.8G 18.8G

18.8G 18.8G

Non-Cisco

12 slots

[48x48 matrix]

10G

10G

10G

10G

10G

10G

10G

10G

10G

10G

10G

10G

10G

10G

10G

TX

TX

TX

TX

RX

RX

RX

RX

26G 26G

Good-enough:

40 Gbps per slot

non-blocking

Ingress Linecard Egress Linecard

26G 26G

Non-Cisco

8 slots

[16x16 matrix]20G

ASIC

20G

ASIC

20G

ASIC

20G

ASIC

20G 20G

20G 20G

TX

TX

RX

RX

Page 41: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco Systems, Inc. All rights reserved.BRKIPM-2012 41

Good 100GE100GE non-blocking even with failed fabric card

1.4x 1.2x speedup

Cisco CRS-3

Ingress Linecard Egress Linecard

141G 225G140G NPU

Switching Fabric

4.48 Tbps

7+1 planes

140G NPU

TX RX

Bad 100GE

Non-Cisco

54G 54G

54G 54G

sub-interface 1

sub-interface 2

Ingress Linecard Egress Linecard

50G NPU

50GNPU

Switching Fabric

1.6 Tbps

4+1 SIB’s

50GNPUECMP/Link-Bundling

No 100G processor

available, per-destination

IP load-sharing across two

old 50G processors with

two old 54G fabric ports

50GNPU

sub-interfce 1

sub-interface 2

TX

TX

RX

RX

Impact of too many fabric connections“How to do 100GE?”

More than 54G Fabric Port gets overloaded!

Head of Line Blocking?

Port-Channel across 2 vlans on the same physical port

(to keep 1 IP and 1 MAC address per port)

no multi-vendor interoperability

Page 42: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public 42

• Motivation for IP NGN = traffic growth

• How to make the Network cheaper

• How to make the Router cheaper

• Quality differences

There are Good, Good-enough or Bad Solutions.

Page 43: IP NGN Backbone Routers for the Next Decade€¦ · IP/MPLS BACKBONE IP/MPLS Internet Peering PDH MW 2G 3G CPE DSLAM IGW CGv6 BRAS DWDM INTERNET ... - Importance of OTN interfaces

Thank you.