router architecture and ios internals · presentation_id © 2001, cisco systems, inc. all rights...
TRANSCRIPT
![Page 1: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/1.jpg)
1© 2001, Cisco Systems, Inc. All rights reserved.
Session NumberPresentation_ID
Router Architecture And IOS Internals
![Page 2: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/2.jpg)
222© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Agenda
• Routing and Switching
• Cisco IOS Switching Paths
• Cisco Express Forwarding
• Router Architectures & Parallel Express Forwarding
![Page 3: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/3.jpg)
333© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Routing and Switching
![Page 4: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/4.jpg)
444© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Switching
• The destination in the layer two header remains the same when a packet passes through a switch
A
B
L3: to B
data
L2: to B
L3: to B
data
L2: to B
![Page 5: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/5.jpg)
555© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Routing
• Host A transmits the packet to the router
• The router determines the correct outbound port, then rewrites the layer 2 header so the packet is now destined to B
A
B
data
L3: to B
data
L3: to B
L2: to router
L2: to B
![Page 6: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/6.jpg)
666© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Routing
• Switching, in the context of routers, involves this process of looking up the next hop, finding the layer 2 rewrite “string,” rewriting the layer 2 header, and transmitting the packet
switchingtable
![Page 7: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/7.jpg)
777© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Layer 3 Switching
• When the term “layer 3 switch” was first coined, it meant switching packets in hardware based on the layer 3 information
• However, the lines are rarely so neatly drawn in the real world Switching
Table
![Page 8: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/8.jpg)
888© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Layer 3 Switching
RoutingTable
InformationSources
InformationSources
Co
ntr
ol P
lan
eD
ata
Pla
neRouting Protocols and
Other Sources Are Used to Build the Routing Table
![Page 9: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/9.jpg)
999© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Layer 3 Switching
RoutingTable
InformationSources
InformationSources
Co
ntr
ol P
lan
eD
ata
Pla
ne
Layer 2MappingLayer 2
Mapping
ARP and Other Methods Are Used
to Build the Layer 2 Mapping Tables
![Page 10: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/10.jpg)
101010© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Layer 3 Switching
RoutingTable
InformationSources
InformationSources
Co
ntr
ol P
lan
eD
ata
Pla
ne
Layer 2MappingLayer 2
Mapping
These Tables Are Used to Build a Switching Table
SwitchingTable
![Page 11: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/11.jpg)
111111© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Layer 3 Switching
RoutingTable
InformationSources
InformationSources
SwitchingTable
Co
ntr
ol P
lan
eD
ata
Pla
ne
Layer 2MappingLayer 2
Mapping
The Switching Table Is then Used to Switch Packets
![Page 12: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/12.jpg)
121212© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Layer 3 Switching
• Where is switching done?
On the main processor, in a “normal” process
On the main processor in a special mode (interrupt context)
On a separate general purpose processor
On an application specific chip (ASIC)
SwitchingTable
![Page 13: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/13.jpg)
131313© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Cisco IOS Switching Paths
![Page 14: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/14.jpg)
141414© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
IOS Process Scheduling
Critical Priority
High Priority Medium Priority
Low Priority
•Each disk represents a Process in the Process Ready Queue.
•Each Process is assigned a Priority (Critical, High, Medium or Low)
Scheduler
Ready Queue
![Page 15: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/15.jpg)
151515© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
CPUSoftware ‘Processes’….
Interface Processor DMA’s packet into RX Ring Buffer
Shared memory
Router Switching Operation “Process Switching”
Buffer
Buffer
BufferRx Ring
1
![Page 16: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/16.jpg)
161616© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
CPUSoftware ‘Processes’….
Interface Driver CodeDecodes packet header and builds Buffer Header with L3 Info
Shared memory
Router Switching Operation“Process Switching”
Header
Buffer
Buffer
BufferRx Ring
2
![Page 17: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/17.jpg)
171717© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
CPU
Interface processor generates RX Interrupt to CPU.
X
RX Interrupt
Shared memory
Software ‘Processes’….
Router Switching Operation“Process Switching”
Header
Buffer
Buffer
BufferRx Ring
3
![Page 18: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/18.jpg)
181818© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
CPU
Software ‘Processes’ are resumed at the point the were suspended when the RX Interrupt arrived
When Packet passed to Processor, Buffer ownership transferred to Processor.
As Ownership has passed Interrupt released.
Shared memory
Router Switching Operation“Process Switching”
Header
Buffer
Buffer
BufferRx Ring
4
![Page 19: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/19.jpg)
191919© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
CPUSoftware ‘Processes’….
Processor returns to scheduled tasks. Packet is placed on Input Hold Q (protocol dependant).Packet is idle waiting for Input Process to deal with Packet.
Shared memory
Router Switching Operation“Process Switching”
Header
Buffer
Buffer
BufferRx Ring
ip_input
5
![Page 20: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/20.jpg)
202020© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Input Process Looks up Destination in Forwarding Table. Determines O/P interface. Writes new MAC header. Places Packet in Output Q
Forwarding Table
1.1.0.0/16 via 172.16.2.1
10.1.1.0/24 via 172.16.1.1CPUCPU
Shared memory
Router Switching Operation“Process Switching”
Buffer | Header
ip_outputip_input
6
![Page 21: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/21.jpg)
212121© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
System buffer
Output Process places packet on output interface TX Ring Buffer.
7
CPUCPU
Router Switching Operation“Process Switching”
Buffer | Header
Tx Ring Shared memory
X
Software ‘Processes’….
ip_output
![Page 22: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/22.jpg)
222222© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
CPUCPU
Interface polls TX ring and DMA’s packets for transmission
8
Router Switching Operation“Process Switching”
Buffer | Header
Shared memory
X
Software ‘Processes’….
TxRing
![Page 23: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/23.jpg)
232323© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
CPUCPU
9Interface Instigates a TX interrupt. Increment counters, SNMP etc..
TX Interrupt
Router Switching Operation"Process Switching”
Buffer
Buffer
Buffer
Rx Ring
X
Software ‘Processes’….
![Page 24: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/24.jpg)
242424© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
CPU
CPU Memory
Fast CacheFast CachePrefix/Length Prefix/Length AgeAge Interface Next Hop Interface Next Hop 1.1.0.0/16 1.1.0.0/16 00:00:15 00:00:15 Ethernet0 172.16.2.1 14 00000C7EF7CF00E0B06423F60800 Ethernet0 172.16.2.1 14 00000C7EF7CF00E0B06423F60800 10.1.1.0/2410.1.1.0/24 00:00:15 00:00:15 Serial1 Serial1 172.16.1.1 4 0F000800 172.16.1.1 4 0F000800
ARP Table
172.16.1.1: 0F000800
172.16.2.1: 10134567A...ECE030178654
Forwarding Table
1.1.0.0/16 via 172.16.2.1
10.1.1.0/24 via 172.16.1.1
Demand Generated Cache Based Switching (“Fast” Switching)
![Page 25: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/25.jpg)
252525© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
1Interface Processor DMA’s packet into RX Ring Buffer
Router Switching Operation”Fast” Switching
Buffer
Buffer
BufferRX
Ring
CPUX
Software ‘Processes’….
![Page 26: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/26.jpg)
262626© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
2Interface Driver CodeDecodes packet header and builds Buffer Header with L3 Info
Router Switching Operation”Fast” Switching
Header
Buffer
Buffer
Buffer
CPU
Rx Ring
X
Software ‘Processes’….
![Page 27: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/27.jpg)
272727© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Prefix Age I/F Next Hop10.1.2.3/32 00:00:15 E0 10.1.2.1 14 aae0cd..11.1.2.0/24 00:00:15 S1 10.2.3.1 4 0f000800
Simplified Optimum Cache
3Interface processor generates RX Interrupt to CPU.
CPU Halts current process and attempts to fast switch packet
Router Switching Operation ”Fast” Switching
Header
Buffer
Buffer
Buffer
Rx Ring
CPU
RX
Inte
rru
pt
X
Software ‘Processes’
![Page 28: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/28.jpg)
282828© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
4Optimum Cache entry used to Write MAC header
Router Switching Operation ”Fast” Switching
CPUX
Software ‘Processes’….
RX
Inte
rru
pt
Header
Buffer
Buffer
Buffer
Rx Ring
Prefix Age I/F Next Hop10.1.2.3/32 00:00:15 E0 10.1.2.1 14 aae0cd..11.1.2.0/24 00:00:15 S1 10.2.3.1 4 0f000800
Simplified Optimum Cache
![Page 29: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/29.jpg)
292929© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
TX
Ring
If Output Q is Empty packet is placed directly on the TX Ring.
A packet in the Output Hold Q, will force other packets destined for that interface to be placed in the Q.
Shared memory
Router Switching Operation”Fast” Switching
Buffer | Header
CPUX
Software ‘Processes’….
RX
Inte
rru
pt
ip_output
5
![Page 30: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/30.jpg)
303030© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Interface Instigates a TX Interface Instigates a TX interrupt. Increment interrupt. Increment counters, SNMP etc..counters, SNMP etc..
Router Switching OperationRouter Switching Operation”Fast” Switching”Fast” Switching
Buffer
Buffer
Buffer
CPU
Tx
Inte
rru
pt
Rx Ring
6
![Page 31: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/31.jpg)
313131© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Demand Generated Cache Based Switching Issues
•• First packet towards a given destination is always First packet towards a given destination is always process switched process switched
•• Fast cache entries must be timed out periodically to Fast cache entries must be timed out periodically to prevent stale information from being used in switching prevent stale information from being used in switching
•• When an arp entry or the routing table changes, we When an arp entry or the routing table changes, we must clear some portion of the fast cache and wait for must clear some portion of the fast cache and wait for process switched traffic to rebuild it process switched traffic to rebuild it
•• We store a prebuilt mac header for each possible We store a prebuilt mac header for each possible destination. This waste space and causes duplicated destination. This waste space and causes duplicated efforteffort
![Page 32: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/32.jpg)
323232© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Show Processes
7206#show processes
CPU utilization for five seconds: 0%/0%; one minute: 0%; five minutes: 0%
PID QTy PC Runtime (ms) Invoked uSecs Stacks TTY Process
2 M* 0 8 86 93 9888/12000 0 Exec
3 Lst 60655C58 345736 129733 2664 5740/6000 0 Check heaps
4 Cwe 6064C268 4 1 4000 5568/6000 0 Chunk Manager
5 Cwe 6065BC70 12 17 705 5596/6000 0 Pool Manager
14 Lwe 60719100 5604 103710 54 5236/6000 0 ARP Input
20 Cwe 60661090 0 1 0 5608/6000 0 Critical Bkgnd
21 Mwe 6061BC70 232 209650 110164/12000 0 Net Background
22 Lwe 605ACD38 0 26 011504/12000 0 Logger
24 Msp 6061B1C0 32336 1277140 25 6920/9000 0 Per-Second Jobs
35 Mwe 60747998 4276 64668 6610648/12000 0 IP Input
82 Msp 6061B200 85188 21328 3994 5660/6000 0 Per-minute Jobs
For the 5 Sec window we have both the total CPU time and the Interrupt time
![Page 33: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/33.jpg)
333333© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Show Processes CPU
7206#show processes cpu
CPU utilization for five seconds: 0%/0%; one minute: 0%; five minutes: 0%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
2 68 227 299 0.00% 0.00% 0.00% 0 Exec
3 368920 138425 2665 0.08% 0.02% 0.00% 0 Check heaps
4 4 1 4000 0.00% 0.00% 0.00% 0 Chunk Manager
5 20 21 952 0.00% 0.00% 0.00% 0 Pool Manager
14 6608 119562 55 0.00% 0.00% 0.00% 0 ARP Input
20 0 1 0 0.00% 0.00% 0.00% 0 Critical Bkgnd
21 248 218242 1 0.00% 0.00% 0.00% 0 Net Background
22 0 28 0 0.00% 0.00% 0.00% 0 Logger
24 35704 1362619 26 0.00% 0.00% 0.00% 0 Per-Second Jobs
35 4520 68993 65 0.00% 0.00% 0.00% 0 IP Input
82 90896 22759 3993 0.00% 0.00% 0.00% 0 Per-minute Jobs
More specific information on the CPU time occupied by the Processes
![Page 34: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/34.jpg)
343434© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Cisco Express Forwarding
![Page 35: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/35.jpg)
353535© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Cisco Express Forwarding
• Background
• CEF Theory
• The CEF Mtrie
• The Adjacency Table
• Adjacency Table Entries
• Load Sharing with CEF
• CEF Accounting
![Page 36: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/36.jpg)
363636© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Background: Process Level Switching
• Process Level Switching has speed limitations on high speed networks
![Page 37: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/37.jpg)
373737© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Background: Fast Switching
• Caching the results of the lookup routines was the first solution and is known as Fast Switching
• This solution encounters scalability problems on Internet backbone routers where the routing table is changing rapidly and there are many different flows of traffic
• CEF (Cisco Express Forwarding) was developed to address the scalability issues of Process and Fast Switching
• CEF doesn’t cache switching information, it builds switching tables
![Page 38: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/38.jpg)
383838© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
CEF Theory
What Do We Need to Switch a Packet?
192.168.100.1192.168.100.1Destination Address
0002B9AD16E0000002B9AD16E000B06454F9410800B06454F9410800MAC Header Rewrite String
Outbound Interface Information
![Page 39: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/39.jpg)
393939© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
CEF Theory
CEF Builds Two Tables to Contain this Information:
The CEF Mtrie
The Adjacency Table
![Page 40: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/40.jpg)
404040© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
CEF Packet Switching
• Read in packet from the interface and store packet into memory
• Raise an interrupt to the processor; the rest of the packet switching takes place within the interrupt
• Use CEF mtrie to lookup packet destination; determine correct next-hop info by following pointer in the last CEF mtrie node
• Use Adjacency table info to rewrite physical layer header
• Place packet on the outbound interface queue
![Page 41: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/41.jpg)
414141© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
CEF Theory
What’s the Difference between a Tree and a Trie?
The MAC Header Rewrite Information Is Stored in the Tree Itself
pointer to parent
pointer to child
MAC header rewrite
….
….
![Page 42: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/42.jpg)
424242© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
CEF Theory
A Pointer to the MAC Header Information Is Stored in the Trie, and the MAC Header Information Itself Is Stored in a
Separate Table
pointer to parent
pointer to child
MAC header rewrite
….
….
What’s the Difference between a Tree and a Trie?
![Page 43: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/43.jpg)
434343© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The CEF Mtrie
172.16.1.0
256 Children 2550
256 Children2550 1
256 Children 2550 16
256 Children 2550 172
root
![Page 44: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/44.jpg)
444444© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The CEF Mtrie
• Nodes point to other nodes or leaves
2550
2550
2550
2550 172
16
1
root
![Page 45: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/45.jpg)
454545© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
172
16
1
The CEF Mtrie
• Leaves point to the adjacency table 2550
2550
2550
2550
root
![Page 46: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/46.jpg)
464646© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The CEF MTrie
Router#sh ip cef summary
IP CEF with switching (Table Version 4)
4 routes, 0 reresolve, 0 unresolved (0 old, 0 new), peak 0
4 leaves, 8 nodes, 8832 bytes, 4 inserts, 0 invalidations
0 load sharing elements, 0 bytes, 0 references
universal per-destination load sharing algorithm, id 20340B24
1 CEF resets, 0 revisions of existing leaves
0 in-place/0 aborted modifications
Resolution Timer: Exponential (currently 1s, peak 1s)
refcounts: 533 leaf, 536 node
![Page 47: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/47.jpg)
474747© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The CEF Mtrie
The Pipe The Pipe
MainProcessor
RX TX
InputQueue
OutputQueue
RIBCache
![Page 48: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/48.jpg)
484848© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The CEF Mtrie Notes
• Where in the switching path do we build the CEF table?
• Nowhere! The CEF table is built from the routing table before (and while) packets are being switched
• Because the CEF table is directly related to the routing table, we can build it for every destination in the routing table without waiting on any packets to be switched
![Page 49: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/49.jpg)
494949© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Two Separate Tables
The Routing Table and the CEF Mtrie Are Directly Related
The CEF Table Contains Reachability and Next Hop Information
router#show ip route
172.30.0.0/24 is subnetted, 1 subnets
S 172.30.100.8/32 via Serial 0/0
C 192.168.1.0/24 is directly connected, Ethernet 1/2
S 10.0.0.0/8 via POS 4/1
0
172
30
192
168
100
10
1
8
![Page 50: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/50.jpg)
505050© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The CEF Mtrie
NULL
ROOT0-255
Empty Table
![Page 51: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/51.jpg)
515151© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The CEF Mtrie
NULL
ROOT0-255
Add 10.0.0.0/8
![Page 52: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/52.jpg)
525252© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The CEF Mtrie
10.0.0.0/8 NULLNULL
ROOT0-9 10 11-255
Add 20.1.0.0/16
![Page 53: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/53.jpg)
535353© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The CEF Mtrie
NULLNULL
ROOT0-9 10 11-19 20 21-255
200 1 2-255
NULLNULL
10.0.0.0/8
20.1.0.0/16
NULL
Add 20.1.1.0/24
![Page 54: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/54.jpg)
545454© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The CEF Mtrie
NULLNULL
ROOT0-9 10 11-19 20 21-255
200 1 2-255
NULL
NULLNULL 10 1 2-255
10.0.0.0/8
20.1.1.0/2420.1.0.0/16 20.1.0.0/16
Add 30.1.1.0/29
![Page 55: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/55.jpg)
555555© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The CEF Mtrie
NULLNULL
ROOT0-9 10 11-19 20 21-29
200 1 2-255
NULL
NULLNULL 10 1 2-255
30 31-255
300 1 2-255
NULL
NULL 10 1 2-255
NULL
10-7 8-255
10.0.0.0/8
20.1.1.0/2420.1.0.0/16 20.1.0.0/16
30.1.1.0/29
NULLNULL
NULLAdd 30.1.1.0/30
![Page 56: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/56.jpg)
565656© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The CEF Mtrie
ROOT0-9 10 11-19 20 21-29
200 1 2-255
10 1 2-255
30 31-255
300 1 2-255
10 1 2-255
10-3 4-7 8-255
10.0.0.0/8
20.1.1.0/2420.1.0.0/16 20.1.0.0/16
30.1.1.0/30 30.1.1.0/29
NULLNULL NULL
NULLNULL
NULL
NULL NULL
NULL
NULL NULL
Add 30.1.0.0/16
![Page 57: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/57.jpg)
575757© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The CEF Mtrie
NULLNULL
ROOT0-9 10 11-19 20 21-29
200 1 2-255
NULL
NULLNULL 10 1 2-255
30 31-255
300 1 2-255
NULL
NULL 10 1 2-255
NULL
10-3 4-7 8-255
10.0.0.0/8
20.1.1.0/2420.1.0.0/16 20.1.0.0/16
30.1.1.0/30 30.1.1.0/29 30.1.0.0/16
30.1.0.0/16 30.1.0.0/16
Add 0.0.0.0/0
![Page 58: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/58.jpg)
585858© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The CEF Mtrie
ROOT0-9 10 11-19 20 21-29
200 1 2-255
10 1 2-255
30 31-255
300 1 2-255
10 1 2-255
10-3 4-7 8-255
10.0.0.0/8 0.0.0.0/00.0.0.0/0 0.0.0.0/0
0.0.0.0/00.0.0.0/0
20.1.1.0/24
0.0.0.0/0
0.0.0.0/0 0.0.0.0/0
20.1.0.0/16 20.1.0.0/16
30.1.1.0/30 30.1.1.0/29 30.1.0.0/16
30.1.0.0/16 30.1.0.0/16
![Page 59: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/59.jpg)
595959© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The CEF Mtrie
• Normally there are 4 levels of nodes with each node having 255 children
• Prefix and traffic distribution sometimes makes the mtrie perform better if there are different numbers of children for nodes at each level
![Page 60: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/60.jpg)
606060© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The CEF MTrie
256 Children
65,536 Children
256 Children
2550
2550
64K0 172
16
root
1
16-8-8 512 Children
1,024 Children
32 Children
256 Children
5120
320
1K172
16
root
1
2550 1
10-9-5-8
0
256 Children
2,048 Children
32 Children
256 Children
2560
320
2K0 172
16
root
1
2550 1
11-8-5-8
![Page 61: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/61.jpg)
616161© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Interface Processor Interface Processor DMA’s packet into RX DMA’s packet into RX Ring BufferRing Buffer
Buffer
Buffer
BufferRX
Ring
CPUX
Software ‘Processes’…. Software ‘Processes’….
Path Through the CEF Switch Code
![Page 62: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/62.jpg)
626262© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
1. A packet arrives at an input interface, RX Interrupt generate1. A packet arrives at an input interface, RX Interrupt generatedd
2. Read IP Destination Prefix2. Read IP Destination Prefix
3. Search 3. Search CEF’sCEF’s FIB DB, using the Destination Prefix as Search KeyFIB DB, using the Destination Prefix as Search Key
3
Destination PrefixDestination Prefix
Path Through the CEF Switch Code
Search for FIB Entry
FIB DBFIB DB
256
way
256
way
MT
RIE
MT
RIE
![Page 63: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/63.jpg)
636363© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
4. A Successful 4. A Successful MTRIEMTRIE Lookup will result in a Lookup will result in a FIB EntryFIB Entry being Foundbeing Found4a. If the MTRIE Lookup is 4a. If the MTRIE Lookup is unsuccessfulunsuccessful, the packet will , the packet will be droppedbe dropped
Not Switched – Packet DROPPED 4a
mtrie_infomtrie_info
path[0]path[0]
fastadjfastadj44
Destination PrefixDestination Prefix
Path Through the CEF Switch Code
1. A packet arrives at an input interface, RX Interrupt generate1. A packet arrives at an input interface, RX Interrupt generatedd
2. Read IP Destination Prefix2. Read IP Destination Prefix
3. Search 3. Search CEF’sCEF’s FIB DB, using the Destination Prefix as Search KeyFIB DB, using the Destination Prefix as Search Key
Search for FIB Entry
FIB DBFIB DB
256
way
256
way
MT
RIE
MT
RIE
![Page 64: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/64.jpg)
646464© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
5. FIB Path is selected
55
Search for FIB Entry
mtrie_infomtrie_info
path[0]path[0]
fastadjfastadjDestination PrefixDestination Prefix
4. A Successful MTRIE Lookup will result in a FIB Entry being Found4a. If the MTRIE Lookup is unsuccessful, the packet will be dropped
1. A packet arrives at an input interface, RX Interrupt generated, RX Interrupt generated
2. Read IP Destination Prefix
3. Search CEF’s FIB DB, using the Destination Prefix as Search Key
Path Through the CEF Switch Code
FIB DBFIB DB
256
way
256
way
MT
RIE
MT
RIE
![Page 65: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/65.jpg)
656565© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
6. Selected FIB Path will point to necessary entry in Adjacency Table
AdjacencyAdjacencyTableTable
L2 HeaderL2 Header
Output I/FOutput I/F6
55
Search for FIB
mtrie_infomtrie_info
path[0]path[0]
fastadjfastadj44
Destination PrefixDestination PrefixFIB DBFIB DB
256
way
256
way
MT
RIE
MT
RIE
5. FIB Path is selected
4. A Successful MTRIE Lookup will result in a FIB Entry being Found4a. If the MTRIE Lookup is unsuccessful, the packet will be dropped
1. A packet arrives at an input interface, RX Interrupt generated, RX Interrupt generated
2. Read IP Destination Prefix
3. Search CEF’s FIB DB, using the Destination Prefix as Search Key
Path Through the CEF Switch Code
![Page 66: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/66.jpg)
666666© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Switch During the Receive Interrupt
• Features are processed along each switching path.
• Each feature represents a function call which may fail, succeed, or just not exist.
Fast
CEF
Input Access List
Network AddressTranslation
Switch
![Page 67: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/67.jpg)
676767© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Switch During the Receive Interrupt
• At any point while the packet is being processed, it can be punted to the next slower process by allowing the processor to jump to the next pointer in the chain.
Fast
CEF
Punt from CEF to Fast
![Page 68: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/68.jpg)
686868© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Switch During the Receive Interrupt
• At any point in the chain, the packet may be also be enqueued for process switching. Fast
CEF
Enqueue packet and terminate interrupt
![Page 69: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/69.jpg)
696969© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The CEF Mtrie
• Attached
• Connected
• Receive
• Recursive
Depending on the Type of Route, a CEF Table Entry Can Be Several Different Types
![Page 70: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/70.jpg)
707070© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The CEF Mtrie
• Attached—An “attached” mtrie entry means the destination is attached to the router
• Connected—A “connected” entry is the result of an ip address being configured on an interface
• An entry may be both Attached and Connected
![Page 71: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/71.jpg)
717171© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The CEF Mtrie
• Receive—Indicates packets that are destined to the router and do not need to be switched to another interface
• Recursive—References another node to find the next-hop information
![Page 72: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/72.jpg)
727272© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The Adjacency Table
• The Mtrie is used to look up the next-hop for a prefix
• The final node encountered in the Mtrie during a prefix lookup includes a pointer to the correct next-hop in the adjacency table
Outbound InterfaceMAC Rewrite StringOutbound InterfaceMAC Rewrite String
Adjacency Table
CEF MTrie
![Page 73: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/73.jpg)
737373© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The Adjacency Table
• The ARP Cache and the Adjacency Table are directly related
• The adjacency table doesn’t contain any information about networks; it only contains information about next hops
router#show arp
Address Hardware Addr Interface
192.168.1.4 2B2B.2B2B.2B2B Ethernet 1/2
10.1.1.1 3C3C.3C3C.3C3C POS 4/1
Serial 0/0Point2PointSerial 0/0
Point2Point
Ethernet 1/22B2B.2B2B.2B2B
Ethernet 1/22B2B.2B2B.2B2B
POS 4/13C3C.3C3C.3C3C
POS 4/13C3C.3C3C.3C3C
![Page 74: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/74.jpg)
747474© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The Adjacency Table
• Allows next-hops to change without changing the mtrie
• A change in next-hop just requires the final mtrie node’s pointer to the adjacency table to be updated
• Routing table changes also don’t impact the adjacency table
![Page 75: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/75.jpg)
757575© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The Adjacency Table
• Update the FIB when changes in the routing table occur
• Update the adjacency table when changes in connected adjacencies occur
![Page 76: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/76.jpg)
767676© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Adjacency Table Entries
• Auto adjacencies
• Punt Adjacencies
• Glean Adjacency
• Drop Adjacencies
• Discard Adjacencies
• Null Adjacencies
• Cached Adjacencies
![Page 77: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/77.jpg)
777777© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Adjacency Table Entries (Auto)
• Auto Adjacencies—The most common type of adjacency; include all the information needed to rewrite the packet header and place the packet in the proper interfaces output queue
![Page 78: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/78.jpg)
787878© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Adjacency Table Entries (Auto)
10.1.1.2
10.1.1.1
ADJADJ
MAC Header Rewrite String
MAC Header Rewrite String
Router(config)#ip route 70.0.0.0 255.0.0.0 10.1.1.2
70.0.0.0/8
![Page 79: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/79.jpg)
797979© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Adjacency Table Entries
• Punt Adjacencies—A punt adjacency indicates that the packet should be switched by the next slower switching scheme
![Page 80: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/80.jpg)
808080© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Adjacency Table Entries (Glean)
• Glean Adjacency—Only one per router; indicates that the destination is attached to the router but the layer two information has not been acquired; results in an ARP request when a packet is switched to this destination
![Page 81: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/81.jpg)
818181© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Adjacency Table Entries (Glean)
Router#sh ip interface brief
Interface IP-Address OK? Method Status Protocol
Ethernet0/0 20.0.0.1 YES manual up up
Router#sh ip cef adjacency glean
Prefix Next Hop Interface
20.0.0.0/8 attached Ethernet0/0
![Page 82: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/82.jpg)
828282© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Adjacency Table Entries (Glean)
10.1.1.2
10.1.1.0/24
10.1.1.1
10.1.1.0/32 Receive10.1.1.255/32 Receive10.1.1.1/32 Receive10.1.1.0/24 Attached
Glean
ADJADJ
![Page 83: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/83.jpg)
838383© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Adjacency Table Entries (Glean)
10.1.1.2
10.1.1.0/24
10.1.1.1
10.1.1.0/32 Receive10.1.1.255/32 Receive10.1.1.1/32 Receive10.1.1.0/24 Attached10.1.1.2/32 Attached
Glean
MAC
ADJADJ
![Page 84: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/84.jpg)
848484© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Adjacency Table Entries
• Drop Adjacency—Indicates the packet should be dropped
![Page 85: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/85.jpg)
858585© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Adjacency Table Entries (Drop)
Router#sh ip cef adjacency drop
Prefix Next Hop Interface
224.0.0.0/4 drop
![Page 86: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/86.jpg)
868686© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Adjacency Table Entries
• Discard Adjacency—Indicates destinations which are part of a loopback’s subnet, but are not the actual ip address configured on the interface
![Page 87: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/87.jpg)
878787© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Adjacency Table Entries (Discard)
40.0.0.0/24
Router(config)#int loop0
Router(config-if)#ip addr 40.0.0.1 255.255.255.0
40.0.0.1/32Router#sh ip cef 40.0.0.2
40.0.0.0/24, version 3, attached, connected
0 packets, 0 bytes
via Loopback0, 0 dependencies
valid discard adjacency
ADJADJ
DiscardDiscard
![Page 88: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/88.jpg)
888888© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Adjacency Table Entries
• Null Adjacency—Indicates the packet should be switched to a Null interface on the router
![Page 89: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/89.jpg)
898989© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Adjacency Table Entries (Null)
Router(config)#ip route 60.0.0.0 255.0.0.0 null0
Router#sh ip cef adjacency null
Prefix Next Hop Interface
60.0.0.0/8 attached Null0
![Page 90: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/90.jpg)
909090© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
CEF Show Commands
router#show ip cef
Prefix Next Hop Interface
0.0.0.0/32 receive
10.97.1.0/24 attached Serial4/3
10.97.1.0/32 receive
10.97.1.255/32 receive
42.40.183.0/24 12.51.142.5 POS1/0
Prefix: The Prefix of the Destination Network
Next Hop: The Type of Adjacency or the Next Hop Towards This Destination
Interface: The Interface Out Which to Send Traffic for This Destination
![Page 91: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/91.jpg)
919191© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
CEF Show Commands
router#show ip cef 33.97.1.0 255.255.255.0 detail
33.97.1.0/24, version 13, attached, connected, cached adjacency to Serial4/3
0 packets, 0 bytes
via Serial4/3, 0 dependencies
valid cached adjacency
The Type of Adjacency This CEF Table Entry Points to
Number of Table Entries Which Point to (Depend On) This Entry
Number of Packets and Bytes Which Have Been Switched Through This Entry; Configure IP CEF Accounting Per-prefix for This to Work
![Page 92: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/92.jpg)
929292© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
CEF Show Commands
router#show ip cef summary
IP CEF with switching (Table Version 46), flags=0x0
22 routes, 0 reresolve, 0 unresolved (0 old, 0 new), peak 0
25 leaves, 19 nodes, 22960 bytes, 49 inserts, 24 invalidations
0 load sharing elements, 0 bytes, 0 references
universal per-destination load sharing algorithm, id F2F8D257
Total Number of Entries in the CEF Table
Number of Entries Which Need to Be Re-resolved
Number of Entries Which Do Not Have Resolved Recursions
![Page 93: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/93.jpg)
939393© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
CEF Show Commands
router#show adjacency detail
Protocol Interface Address
IP Serial4/0 point2point(5)
0 packets, 0 bytes
0F000800
CEF expires: 00:02:32
refresh: 00:00:32
A: Packets and Bytes Switched Through This Adjacency
AABBCC
B: MAC Header Rewrite String
C: When This Entry Will Be Refreshed; In This Case, All Point2Points Are Refreshed Every Minute
![Page 94: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/94.jpg)
949494© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
CEF Show Commands
router#show int ethernet1/0 stat
Ethernet1/0
Switching path Pkts In Chars In Pkts Out Chars Out
Processor 977121 70149655 578014 56457133
Route cache 0 0 0 0
Total 977121 70149655 578014 56457133
Route cache Includes CEF Switched Packets
![Page 95: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/95.jpg)
959595© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Router Architectures & Parallel Express Forwarding
![Page 96: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/96.jpg)
969696© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
IntroductionIntroduction
• Routers have to deal in three “Planes” of operation:-
The “Control” Plane
Building and maintaining data structures such as “forwarding tables”
The “Management” Plane
Dealing with configuration files, gathering and providing statistics, providing and responding to control protocol messages
The “Data” Plane
Switching of packets, manipulation of packet (header and content), packet delivery scheduling (queuing)
![Page 97: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/97.jpg)
979797© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Introduction Introduction –– Consumable ResourcesConsumable Resources
CPU Memory
Bandwidth
When any all or all of the resources are exhausted, inconsistentbehavior will be observed
![Page 98: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/98.jpg)
989898© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Routers OperationallyRouters Operationally
• Maintain/manipulate routing informationListen for updates/update neighbors
• Classify packets for manipulation/queuing/permit-deny, etc.Compare packets to classification lists and perform control
• Perform Layer 3 switchingCreate outbound Layer 2 encapsulation
Layer 3 checksum
TTL/hop count update
• Management/billing (statistics)Interface statistics—NetFlow export
Telnet, SNMP, ping, trace route, HTTP
![Page 99: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/99.jpg)
999999© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Routers FunctionallyRouters Functionally
• (Attempt to) switch packetsLayer 3 switching based on routing information
• (Attempt to) transmit packetsAccess outbound media
• Manipulate packetsChange contents of packet (CAR/NAT/compression/encryption)
• Consume packetsRouting protocol updates etc…/services advertisements(SAP)/ICMP/SNMP
• Generate packetsRouting protocol packets/SAPs/ICMP/SNMP
Tunnels—GRE, IPSec, DLSw etc…
![Page 100: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/100.jpg)
100100100© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Router Hardware
• Interface Processors
• The Central Processing Unit
• Memory
• The Backplane
![Page 101: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/101.jpg)
101101101© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
The Central Processing Unit
• Provides horsepower for all control plane functions, such as system maintenance, building routing tables, etc.
• On some platforms, it also provides the horsepower for actually switching packets
![Page 102: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/102.jpg)
102102102© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Shared Memory ArchitectureShared Memory ArchitectureShared Memory Architecture
•• Applicable PlatformsApplicable Platforms
Cisco 1xxxCisco 1xxxCisco 2xxxCisco 2xxxCisco 3xxxCisco 3xxxCisco 4xxxCisco 4xxx
![Page 103: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/103.jpg)
103103103© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
CPU Allocated Memory
Data/Address/Control Bus’s
Shared Memory ArchitectureShared Memory Architecture
Packet Switching Allocated
Memory
BuffersQueuesPointersHeaders
S/W Image/Files
CPU Buffers
Switching Tables
CPU Queues
Inte
rfac
eIn
terf
ace
Inte
rfac
eIn
terf
ace
Inte
rfac
eIn
terf
ace
Inte
rfac
eIn
terf
ace
Inte
rfac
eIn
terf
ace
Inte
rfac
eIn
terf
ace
Inte
rfac
eIn
terf
ace
Physical Media Interfaces (Fixed or Modular)
General Purpose
CPU
Memory
![Page 104: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/104.jpg)
104104104© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Shared Memory Architecture (Hardware “Assist”)Shared Memory Architecture (Hardware “Assist”)Shared Memory Architecture (Hardware “Assist”)
•• Applicable PlatformsApplicable Platforms
Cisco 7200Cisco 7200Cisco 7300Cisco 7300Cisco 7400Cisco 7400Cisco 10000Cisco 10000
![Page 105: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/105.jpg)
105105105© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Data/Address/Control Bus’s
Shared Memory Architecture (Hardware “Assist”)Shared Memory Architecture (Hardware “Assist”)
Inte
rfac
eIn
terf
ace
Inte
rfac
eIn
terf
ace
Inte
rfac
eIn
terf
ace
Inte
rfac
eIn
terf
ace
Inte
rfac
eIn
terf
ace
Inte
rfac
eIn
terf
ace
Inte
rfac
eIn
terf
ace
General Purpose
CPU
Physical Media Interfaces (Fixed or Modular)
Function Specific
Hardware
Memory
CPU Allocated Memory
Packet Switching Allocated
Memory
BuffersQueuesPointersHeaders
S/W Image/Files
CPU Buffers
Switching Tables
CPU Queues
![Page 106: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/106.jpg)
106106106© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Distributed Shared MemoryDistributed Shared MemoryDistributed Shared Memory
•• Applicable PlatformsApplicable Platforms
Cisco 7500Cisco 7500Catalyst 5xxx RSMCatalyst 5xxx RSM
![Page 107: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/107.jpg)
107107107© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Some Line cards have Packet Memory, Forwarding Table Memory and a discrete
Switching Hardware.
Sha
red
I/O M
emor
y(S
(D)R
AM
) <6
4MB
Sha
red
I/O M
emor
y(S
(D)R
AM
) <6
4MB
Distributed Shared MemoryDistributed Shared MemoryDistributed Shared Memory
I/O Buffer
I/O Buffer
I/O Buffer
I/O Buffer
I/O Buffer
I/O Buffer
I/O Buffer
I/O Buffer
CPU
General Purpose CPU
InterfaceInterface
CPU Memory
I/O Memory
CPU Memory >64MBCPU Memory >64MBs/w Image
RIBConfigurationCEF Tables
CPU MemoryCPU Memory
s/w Image
CEF Tables
CPU Memory
I/O Memory
CPU Memory
I/O Memory
I/O MemoryI/O MemoryI/O Buffers
![Page 108: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/108.jpg)
108108108© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Distributed Cross BarDistributed Cross BarDistributed Cross Bar
•• Applicable PlatformsApplicable Platforms
Cisco 6500/7600 OSRCisco 6500/7600 OSRCisco 12000 (GSR)Cisco 12000 (GSR)
![Page 109: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/109.jpg)
109109109© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Cross Bar Data PathCross Bar Data PathCross Bar Data Path
TxTx
RxRx
Tx
Rx
Tx
Rx
CPU Memory(DRAM)
CPU Memory(DRAM)
(C) ForwardingTable
CPU
CPU
InterfaceCard(D) FT
Packet Memory
CPU
InterfaceCard(D) FT
Packet Memory
CPU
InterfaceCard(D) FT
Packet Memory
CPU
InterfaceCard(D) FT
Packet Memory
Serial Input/Output
Lines, “To” and “From” Fabric
ASIC X-Bar Fabric
![Page 110: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/110.jpg)
110110110© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Cross Bar Data Path (Multiple Fabrics)Cross Bar Data Path (Multiple Fabrics)Cross Bar Data Path (Multiple Fabrics)
Packet Slicing Allows Multiple Switching
Fabrics
256 bytes
64
64
64
64
256 bytes
64
64
64
64
Packets split into cells
Packets reassembled
Tx
Rx
Tx
Rx
Tx
Rx
Tx
Rx
Tx
Rx
Tx
Rx
Tx
Rx
Tx
Rx
Tx
Rx
Tx
Rx
XOR
XOR
CPU
InterfaceCard
(D) FT
Packet Memory
CPU
InterfaceCard(D) FT
Packet Memory
Serialization time = t
Serialization time = t/4
![Page 111: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/111.jpg)
111111111© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Parallel Express Forwarding
• PXF is one kind of “Function Specific Hardware”
• PXF Architecture
• PXF packet switching
![Page 112: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/112.jpg)
112112112© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Cisco-Developed Unique Value-Add PXF IP Services Processor
• Internally-Developed Cisco Processing Technology• Internally-Developed Cisco Processing Technology
US Patent 6,101,599US Patent 6,101,599 IM IMIMIM
IM IMIMIM
IM IMIMIM
IM IMIMIM
Input DemuxInput Demux
Output MuxOutput Mux
Feedback
.
.
.
ColumnMem.
ColumnMem.
ColumnMem.
ColumnMem.
55 13139911
44 12128800
77 1515111133
66 1414101022
![Page 113: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/113.jpg)
113113113© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Benefit of PXF Acceleration
Number of Services Added to the Network
Per
form
ance
PXF Hardware Acceleration
Non-PXF Processing
Delta in CPUUsage
![Page 114: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/114.jpg)
114114114© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Parallel eXpress Forwarding (PXF) Engine
• New technology switching engine for high-touch L3 services with optimized throughput
• Programmable architecture to allow for future feature upgrades
• Based on custom pipelined array processor (ASIC)
![Page 115: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/115.jpg)
115115115© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Power of CiscoParallel Processing
• Matrix of separate processors• Implements “assembly line” for exceptional performance• “Assembly line” enables consistent throughout• Little division when services are enabled/disabled
PXF Network Processor
ProcessorProcessor
ProcessorProcessor
ProcessorProcessor
ProcessorProcessor
ProcessorProcessor
ProcessorProcessor
ProcessorProcessor
ProcessorProcessor
ProcessorProcessor
ProcessorProcessor
ProcessorProcessor
ProcessorProcessor
ProcessorProcessor
ProcessorProcessor
ProcessorProcessor
ProcessorProcessor
![Page 116: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/116.jpg)
116116116© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
PXF Processor ServicesExample
IM
12
IM
13
IM
14
IM
15
Packet Q Packet M Packet I Packet E
Packet A Packet U
CEF
CBWFQ, WRED, LLQ
NAT
Classification
![Page 117: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/117.jpg)
117117117© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Parallel eXpress Forwarding (PXF)
• Each PXF ASIC has 16 processors arranged in 4 rows x 4 columns
• Two PXF ASICs connected serially: 4x8, 32 CPUs total for an ESR• Parallelism and pipelining => Improved feature throughput
IM
InstructionMemory
ProcessorCore
NextContext
CurrentContext
14
22
30
Col. Mem Col. Mem Col. Mem Col. Mem
Col. MemCol. MemCol. MemCol. Mem
Inp
ut D
emu
x
Out
put M
ux
FeedbackFeedback
IM0
IM8
IM16
IM24
IM
IM
IM
IM
IM IM71 5
9
17
25
IM
IM10
IM18
IM26
2
11
19
27
3IM
4
IM12
IM20
IM28
IM13
IM21
IM29
IM
IM
IM
IM
IM15
IM23
IM31
IM
IM
IM
IM
6
![Page 118: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/118.jpg)
118118118© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
PXF
BufferSDRAM
OUT
JOINPXFIN
SDRAM SDRAM
SDRAM
To Line CardsFrom Line Cards
SDRAM
From-Toaster Complex
Output Queue Controller
Input Packet Memory
Interface Interface
To-Toaster Complex
PXF Packet Forwarding
Headers Passthrough Toaster Modified Headers
and Bodies AreMoved to PacketBuffer Memory
Complete Packets Are Moved from
SDRAM to Output Line Cards
![Page 119: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/119.jpg)
119119119© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Summary
![Page 120: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/120.jpg)
120120120© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
SummarySummary
• 90 minutes is way too much to summarize in one slide, and not enough time to cover these topics!
• Routers scale based on CPU, processing hardware, memory and bandwidth
• Resource exhaustion results in dropped packets
• No one architecture has all the answers different platforms are appropriate for different roles in your network
![Page 121: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/121.jpg)
121121121© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Recommended Reading
Inside Cisco IOS Software Architecture(CCIE Professional Development)ISBN: 1-57870-181-3
IP Routing FundamentalsISBN: 1-57870-071-X
Cisco Router Configuration, Second Edition ISBN: 1-57870-241-0
CEF Whitepaper: http://www.cisco.com/warp/public/732/Tech/switching/docs/cef_ov_final.pdf
![Page 122: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/122.jpg)
122122122© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID
Life Is Short, Eat Dessert First!
122122122© 2002, Cisco Systems, Inc. All rights reserved.
PS-5404970_05_2002_c1
![Page 123: Router Architecture And IOS Internals · Presentation_ID © 2001, Cisco Systems, Inc. All rights reserved. 3 Routing and Switching](https://reader034.vdocuments.site/reader034/viewer/2022042200/5e9ffeb1fe661635fe769313/html5/thumbnails/123.jpg)
123123123© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID 123123123© 2001, Cisco Systems, Inc. All rights reserved.Presentation_ID