cisco arch easy

Copyright © 1998, Cisco Systems, Inc. All rights reserved. Printed in USA.Presentation_ID.scr 1

16011094_06F9_c4 © 1999, Cisco Systems, Inc. 16011094_06F9_c1 © 1999, Cisco Systems, Inc. 1

6011094_06F9_c4 © 1999, Cisco Systems, Inc.


Cisco Router ArchitectureCisco Router Architecture

Session 601Session 601



AgendaAgenda

• Router Fundamentals

• Layered Switching

• Router Architectures/Switching Paths

• Optimized Network Design

• Troubleshooting


Router FundamentalsRouter Fundamentals




List of ReachableList of ReachableNetworksNetworks

Layer 3SwitchPacketPacketFrame FrameFrame

OSPF, EIGRP,BGPStatic Routes, Etc…OSPF, EIGRP,BGP

Static Routes, Etc…

Layer 2 EncapsulationTTL-1Layer 3 Checksum

What Is a Router?What Is a Router?

• Routers performtwo main functions

• Control pathroutines

• Data path control(switching)

PacketPacket PacketPacket

PacketPacket


Routers (Operationally)Routers (Operationally)

• Maintain/manipulate routing informationListen for updates/update neighbors

• Classify packets for manipulation/queuing/permit-deny, etc.Compare packets to classification lists and perform control

• Perform Layer 3 switchingCreate outbound Layer 2 encapsulation

Layer 3 checksum

TTL/hop count update

• Management/billing (statistics)Interface statistics—Netflow export

Telnet, SNMP, ping, trace route, HTTP



Router FunctionalityRouter Functionality

System Level TasksSystem Level Tasks

CPU

Packet SwitchingPacket Switching

• Run routing protocols• Maintain routing tables• Check for CLI commands• Increment accounting counters• ICMP• Queue packets, etc…

Layer 1: Retime, Regenerate SignalLayer 2: Rewrite Header and CRCLayer 3: Decrement TTL and CRC Rewrite

Control Plane

Data Plane


RoutersRouters(Layer 3 Packet Functionally)(Layer 3 Packet Functionally)

• (Attempt to) switch packetsLayer 3 switching based on routing information

• (Attempt to) transmit packetsAccess outbound media

• Manipulate packetsChange contents of packet (CAR/NAT/compression/encryption)

• Consume packetsRouting protocol updates etc…/servicesadvertisements(SAP)/ICMP/SNMP

• Generate packetsRouting protocol packets/SAPs/ICMP/SNMP

Tunnels—GRE, IPSec, DLSw etc…



Generic RouterGeneric RouterArchitecturesArchitectures



CPU MemoryCPU Memory

Routing TableRouting Table

CPU

Shared MemoryShared Memory

Shared Bus Electrical Limitations Limit ofShared Bus Switches <= 20 GbpsThus 10 Gbps/No. Line Cards =

Max Line Rate

Inte

rfac

eIn

terf

ace

Inte

rfac

eIn

terf

ace

Inte

rfac

eIn

terf

ace

Inte

rfac

eIn

terf

ace




One Packet/Switching Cycle



CPU


Inte

rfac

eIn

terf

ace

Inte

rfac

eIn

terf

ace

Inte

rfac

eIn

terf

ace

Inte

rfac

eIn

terf

ace

Shared Bus

SM Data Path (Conventional)SM Data Path (Conventional)



SM Data PathSM Data Path(Distributed Processors)(Distributed Processors)



CPU

InterfaceInterface

CPU

InterfaceInterface

CPU

InterfaceInterface

CPU

InterfaceInterface

CPU



Cross Bar Data PathCross Bar Data Path

i/p

o/p



CPU

InterfaceInterface

CPU

InterfaceInterface

CPU

InterfaceInterface

CPU

InterfaceInterface

CPU


Cross Bar Data PathCross Bar Data Path

Bit Slicing AllowsMultiple Switching

Fabrics

CPU Routing TableRouting Table


CPU

InterfaceInterface

CPU

InterfaceInterface

CPU

InterfaceInterface

CPU

InterfaceInterface



Cisco RouterCisco RouterComponentsComponents



CPUCPU

ROMROM RAMRAM

FlashFlash

BusInterface

BusInterface

InterfaceInterface InterfaceInterface InterfaceInterface

System Bus

NVRAMNVRAM

NetworkController

NetworkController

NetworkController

General Router HardwareGeneral Router Hardware



Software ComponentsSoftware Components

• ROM monitorStartup diagnostic codeContains exception handling

• RxBootHost mode Operating SystemUsed for downloading fullCisco IOS®

• Cisco IOSInternetwork operatingsystemContains process scheduler,memory manager, parserAlso contains protocol-specific code for packethandling

• Device microcodePart of Cisco IOS that dealswith network controllersMicrocode deals with modularinterface processors

• Configuration register16 Bits, specifies routerstartup parameters

• Configuration fileStartup-config: Containsconfiguration infoRunning-config: Currentlyactive configuration


CPUCPU

RAM

Boot Flash

RxBootRxBoot

NVRAM

Startup-ConfigConfig Reg.

Startup-ConfigConfig Reg.

ROM

ROM MonitorROM Monitor

PCMCIA Flash

Cisco IOS FileCisco IOS File

Cisco IOSExec

Cisco IOSExec

Runningconfig

Runningconfig

DataStructures

DataStructures

RoutingTable

RoutingTable

SystemBuffersSystemBuffers

InterfaceBuffers

InterfaceBuffers

Main I/O

Layer 2/3Cache

Layer 2/3Cache

Memory UsageMemory Usage



Cisco Router BuffersCisco Router Buffersand Queuesand Queues



Router Interface BuffersRouter Interface Buffers

• Interface FIFOA very small amount of buffer memory (for large MTUs noteven one packet in size) used to store bits as they arrives fromthe wire and are dealt with by the interface driver

These buffers are NOT configurable

• Interface Rx and Tx RINGOn some platforms these buffers are used by the interfacedriver for reception and transmission of packets

Each interface has a FIFO Rx/Tx RingInbound (Rx) they are used to store a packet until anInterface buffer is available. Outbound (Tx) until theinterface can transmit to the wireThey can exist on the interface card or in shared memoryThese buffers are NOT normally configurable



Router Internal BuffersRouter Internal Buffers

• Buffer HeadersThese data structures containing information about related buffer(e.g. location pointers, size, etc.). They are mostly located in mainprocessor memory for all buffers. In some cases headers or particleheaders may be stored in shared I/O memory for speed

The purpose of buffer headers is to keep track of buffers andenqueue them for various processes

• Shared or main memory system buffersThese buffers are used when packets are bound for the processorfor either consumption or process switching

System buffers are sized as small (104), middle (600), big (1,524),very big (4,520), large (5,024), and huge (18,024)

The total number of buffers depends on available DRAM

System buffers can grow or Trim on demand and canbe configured

These buffers are public. All interfaces can use them


Router Internal BuffersRouter Internal Buffers

• Shared memory interface buffersThese buffers are used to store packets between the interface driverand the switching path (not process switching) software(e.g optimum switching)

These buffers are allocated at startup or after OIR. The number ofbuffers depends on the speed and MTU of the interfaces available

These buffers are Interface specific


• Shared memory interface particle buffersUsed as Packet buffers on some platforms and VIP cards

Particle Buffers are located in Shared I/O Memory

Their size is 512 bytes (or 128 byte multicast/broadcast)

The incoming packets are “scattered” into 512 byte particles andthen “gathered” into a contiguous packets for transmission”




Contiguous BuffersContiguous Buffers• Contiguous buffers

Contiguous buffers (older platforms as well as RSP/RSM)store packets in buffers sized with respect to the interfacemedia MTU and speed

The amount of buffers is based on grouping the MTUs of theinterfaces e.g. Ethernet = 1500b MTU, Token Ring/FDDI =4500b MTU

Let’s assume we have 6 Ethernet and 2 FDDI = 2 buffer pools.(all the Ethernet I/f share the 1500b pool and all the FDDIshare the 4500b pool)

Based on the aggregate bandwidth we get: Ethernet =(60/60+200) = 23% and FDDI = (200/60+200) = 77%We then apportion the available buffers (let’s say it’s a 7K SP)

(.23*504K)/1500 = 79 buffers and (.77*504K)/4500 = 88 Buffers

Therefore, 79/6 = 12 buffers/Ethernet interface and88/2 = 44 buffers/FDDI interface


Ethernet InterfaceBuffer Pool

FDDI InterfaceBuffer Pool

1500 4500

Contiguous BuffersContiguous Buffers

• Contiguous buffersPacket buffers (contiguous) can be wasteful in terms ofmemory (Frames are rarely all full size), but more importantlyare not as efficient as particles when it comes to packetreplicationIn contiguous buffers the packets is treated as a whole,therefore to create a replication of the packet with a differentoutput header requires a completely new packet to be created



I/P BufferHeader

I/P BufferHeader

Particle Headers

Packet MemoryDivided IntoParticles

O/P BufferHeaderParticle Headers

Particles vs. Contiguous BuffersParticles vs. Contiguous Buffers

• On some newer platforms (36xx, 72xx, 71xx, VIP2) packet memory isallocated in “particles”. Particles are either 1024b (36xx), 512b or 128bchunks of memory

• Packets are divided into particle size blocks and stored in free particles• Particles have an associated Particle Buffer header which stores

information as to which particles constitute an entire frame

and or an?


I/P BufferHeader

I/P BufferHeader

Particle Headers

Packet MemoryDivided intoParticles

O/P BufferHeader

ClonedParticle

HeadersO/P Buffer

HeaderO/P Buffer

Header

ParticleHeaders

New Header

New Header

Particles vs. Contiguous BuffersParticles vs. Contiguous Buffers

• We have the ability to “clone” particle buffer headers

• This allows us to “replicate” a packets particles a number of timeswithout actually replicating the packet itself for every outboundinterface. This significantly improved multicast performance



Types of QueuesTypes of Queues

• System interface queuesInput hold queues

Used to queue packets in system buffers for process switching

These queues are based in Main Processor Memory

The size of Input Hold queue is configurable per interface basis

Use “Show Interface” to look at Input Queue statistics

Output hold queues

These queues are used for packets in System buffers after theyhave been process switched and are waiting to be transmitted bythe interface driver

These queues exist in main processor memory

The size of input hold queue is configurable per interface basis

Use “Show Interface” to look at output queue statistics


Types of QueuesTypes of Queues

• Interface QueuesReceive Queues

The queues are used for incoming packets that wait ininterface buffers for Fast Switching codeReceive queue usually has a size limit (called RQL) whichis calculated based upon various factorsBy default the RQL = Total buffers in a pool/no. interfacesin that pool(If for any reason this results in less than 16KB of buffersRQL= 16384/MTU)

Transmit QueuesThese queues are used for outgoing packets that wait ininterface buffers for transmission by outgoing interfacedriver codeTransmit queue usually has a size limit (called TQL) whichis calculated based upon various factors



Router OperatingRouter OperatingSystem DetailsSystem Details



Cisco IOS Image and File StorageCisco IOS Image and File Storage

• BOOT ROMEPROM used for startup diagnostic code and ROMmonitor (read only) and to load Cisco IOS

• NVRAMUsed to store startup configuration (rewritable).Configuration register settings

• PCMCIA FLASHPortable storage of Cisco IOS image, configurationfiles etc… (rewritable)

• FLASHOnboard storage of full Cisco IOS (rewritable). Insome cases Cisco IOS execution



Operational StorageOperational Storage

• DRAMUsed to store loaded Cisco IOS, runningconfiguration, route tables, switching caches,processor and switched packets

• SRAMUsed on platforms to deal with high-speedswitching and high-speed interfaces

• FIFOFirst-In, First-Out memory used for interfacebuffering (not queued—circular buffer or RING)


Elements of ExecutionElements of Execution

• ROM MONITOR (System Bootstrap, or Bootstrap code)Hardware configuration and system diagnostics performed atsystem startup

• RxBOOT (Boot Helper image, Helper Cisco IOS, orBootstrap Image)

A Subset of the Cisco IOS, used when a valid Cisco IOS image is notpresent allowing a user to download a full Cisco IOS image fromthe network

• Cisco IOSCisco IOS normally resides in Flash or PCMCIA Flash card and is loadedinto processor memory (DRAM) for executionIn some platforms it may also run from Flash memory to save DRAM

• ROUTER CONFIGThe Startup configuration is stored in NVRAM and a copy is loaded inDRAM at startup



Configuration Register FunctionConfiguration Register Function

• Force the system into the bootstrap program

• Select a boot source and default boot filename

• Recognition of break signal from console

• Control broadcast addresses

• Set the console terminal baud rate

• Load operating software from ROM

• Enable booting from a Trivial File TransferProtocol (TFTP) server


ROM MonitorROM MonitorDiagnostic Check, Console Setup, Memory Sizing, Config

Register Check

Loads Rxboot, or Stays in ROMMON [rommon>]

Diagnostic Check, Console Setup, Memory Sizing, ConfigRegister Check

Loads Rxboot, or Stays in ROMMON [rommon>]

RxBootRxBootBuilds Basic Data Structures, Interface Setup, Host Mode

Functionality, Startup-Config Check

loads Cisco IOS, or Stays in RxBoot [router(boot)>]

Builds Basic Data Structures, Interface Setup, Host ModeFunctionality, Startup-Config Check

loads Cisco IOS, or Stays in RxBoot [router(boot)>]

Cisco IOSCisco IOSInterface Setup, Router Functionality, Allocate Buffers,

Loads Startup-Config

Boot Process Complete [router>]

Interface Setup, Router Functionality, Allocate Buffers,Loads Startup-Config

Boot Process Complete [router>]

Router StartupRouter Startup



How Routers ReceiveHow Routers Receiveand Transmit Packetsand Transmit Packets



CPU

Rx

In

te

rr

up

t

Increment “Ignore”Increment “NoBuffer”*Drop Packet*Return to ScheduledProcesses

Free BufferHeader Available

Free BufferHeader Available

Interface DriverCode

Interface DriverCode

Packet onInterface

Packet onInterface

Yes

No

Move Packet toShared MemoryMove Packet toShared Memory

11

22

55

22

33

Assign a FreeBuffer HeaderAssign a FreeBuffer Header

Discard FrameIncrement“Overrun”If CPU Too Slow

No

YesInterfaceFIFOBuffer

Frame CheckFrame Check

OK

33

44

66

Discard FrameUpdate Frameerrors, e.g.“Runt, Giant, CRC”

Fail

ProcessScheduler

Shared Memory Buffer Headers

11

66 4455

Receiving PacketReceiving Packet



If “Input Hold Queue”Full, *Drop Packet*Increment “InputQueue Drop”

No

Go toFastSwitching

Yes

ClassifyPacket

ClassifyPacket

Can the PacketBe Fast Switched

Can the PacketBe Fast Switched

Assign System BufferReturn Packet Buffer

to Free List

Assign System BufferReturn Packet Buffer

to Free List

IP ProcessInput Hold

Queue

Rx InterruptComplete

Return CPU toScheduled Tasks

Rx InterruptComplete

Return CPU toScheduled Tasks

Fast Cache

CPU

Rx InterruptInterfaceFIFOBuffer

System Memory

System BuffersSystem Buffers

99

How Is ThisBox Related?How Is This

Box Related?

88

77

77

88

99

88

Receiving a PacketReceiving a Packet(Process Switching)(Process Switching)

Enqueue Packetfor Process Switching

Enqueue Packetfor Process Switching


If Output Hold Queue IsFull, Packet Is DroppedWith an “Output QueueDrop”

ForwardingDecision

ForwardingDecision

Packet Is Enqueuedin for Output I/f in

Output Hold Queue

Packet Is Enqueuedin for Output I/f in

Output Hold Queue

Output Hold Queue

CPU

InterfaceFIFOBuffer

System Memory

Write NewHeader OverOld Header

Write NewHeader OverOld Header


Packet IsTransmitted

Packet IsTransmitted

1212

1111

1010

1212

1111

1010

How Is thisBox Related?How Is this

Box Related?

Transmitting a PacketTransmitting a Packet(Process Switching)(Process Switching)



Go toFastSwitchingYes

ClassifyPacket

ClassifyPacket

Can the Packet be Fast Switched

Can the Packet be Fast Switched

Fast Cache

CPU

InterfaceFIFOBuffer

Packet Memory

Packet Buffers

i/p o/p Obtain OutputHeader FromFast Cache

Rewrite Headerand Move

Buffer Headeror Move Packet

to OutputInterface Buffer

If No Output BufferAvailable Drop Packet

With“Output Buffer Failure”

Note: If“Transmit BufferBacking-Store”

Is Enabled the PacketWill Be Moved to

System Buffers and De-queued as with Process

SwitchingAn Output Buffer Swap

Is Incremented

Enqueue Packetfor Interface

Transmit Queue

TQL

Transmit Packet

99

1111

1010

88

777788

1111

1010

99

Transmitting a PacketTransmitting a Packet(Fast Switching)(Fast Switching)


Packet SwitchingPacket Switching“Processes” and“Processes” and

“Functions”“Functions”




IP InputIP Input CompletedCompletedCompressionCompression EncryptionEncryption I/P ACLsI/P ACLs CARCAR

Process Functions

Processes and FunctionsProcesses and Functions

• Process switching is a collection of functions basedon a particular protocol that is being switched

• The Scheduler gives processor time to each “Process”which may call a number of “Functions” before the“Process” is complete

• Each “Process” is run to completion unless a higher-priority “Process” is invoked. The original “Process” issuspended until such time as the higher-priority“Process” is completed at which time the original“Process” will continue to be run


MLPPPMLPPP NATNAT IPSecIPSec

Rx Interrupt

Function Calls


• For Fast and ½ Fast Switching there is no concept of a“Input Process”

• The Fast Switching code, invoked after an Rx Interrupt,calls various functions, independently as defined by theconfiguration file for an interface or protocol

• Some features are run in Rx Interrupt mode, but thepackets are moved to system buffers depending on therequirements of the feature (e.g. NAT)



Shared Packet Memory(SRAM)

Shared System Memory(DRAM)

Output I/fOutput I/f


Packet BuffersPacket Buffers

Output Hold QOutput Hold Q

“Fancy” QueuedPackets“Fancy” QueuedPackets


• Dequeuing is the function used when a packethas been process switched and needs to bepassed to the output interface, or when aninterface has “fancy” queuing enabled and againpackets are stored in system buffers until theyare dequeued (based on queuing mechanism)


FIFO

Process Switching

HMNL

Priority Queuing

3210

Custom Queuing

Output Hold Q


• In the case of “fancy” queuing being applied to aninterface, the Output Hold Queue represents either the (4)Priority, (>16) Custom or (>256) Weighted Fair queues

• In Process switching this is a FIFO queue by default



CompressedCompressed

EncryptedEncrypted

Input ACLInput ACL Deny *Drop Packet*

Decompress Packet

Decrypt PacketYes

No

No

Yes

CARCAR

Permit

CAR Action

Conform/Not Conform

NAT TransformNAT TransformYes Out->In NAT

Is TTL>0Is TTL>0 *Drop Packet*No

TTL=TTL-1TTL=TTL-1

Yes

= Packet Compared To AccessList in Configuration

IP Process InputHold Queue

CPUCPU

Sys

tem

Mem

ory

Input Processing

Dequeue Packet fromInput Hold Queue

Dequeue Packet fromInput Hold Queue

Process SwitchingProcess Switching

Attempt toForward Packet

Attempt toForward Packet


BroadcastBroadcast

Policy RoutePolicy Route

NAT TransformNAT TransformYes

In->Out

Broadcast Packet

Policy RouteYes

No

No

Yes

Look Up RouteLook Up RouteNo

CompressionCompressionYes

Compress Packet

Output ACLOutput ACL *Drop Packet*Deny

Permit

IP Process OutputHold Queue

CPU

Sys

tem

Mem

ory

Output Processing

EncryptionEncryptionYes

Encrypt Packet

Route Table

= Packet Compared To AccessList in Configuration

Process SwitchingProcess Switching



How Routers MakeHow Routers MakeSwitching DecisionsSwitching Decisions



c3620#sh ip routeCodes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, * - candidate default U - per-user static route, o - ODR Gateway of last resort is 0.0.0.0 to network 0.0.0.0 1.0.0.0/24 is subnetted, 2 subnetsC 1.1.1.0 is directly connected, Ethernet0/0C 1.1.2.0 is directly connected, Loopback1 10.0.0.0/24 is subnetted, 1 subnetsC 10.64.217.0 is directly connected, Ethernet0/1S* 0.0.0.0/0 is directly connected, Ethernet0/1

Routing TablesRouting Tables

• Forwarding information populated by IGPsand EGPs

RIP, OSPF, Statics, EIGRP etc…



c3620#sh arpProtocol Address Age (min) Hardware Addr Type InterfaceInternet 1.1.1.1 - 0010.7b1f.4a61 ARPA Ethernet0/0Internet 1.1.1.5 118 00a0.c903.6077 ARPA Ethernet0/0Internet 32.97.105.46 153 0000.0caa.2350 ARPA Ethernet0/1Internet 1.1.1.6 4 00a0.c903.6064 ARPA Ethernet0/0Internet 171.68.225.9 145 0000.0caa.2350 ARPA Ethernet0/1Internet 209.17.176.120 142 0000.0caa.2350 ARPA Ethernet0/1Internet 204.71.200.74 91 0000.0caa.2350 ARPA Ethernet0/1Internet 128.32.18.166 156 0000.0caa.2350 ARPA Ethernet0/1Internet 152.163.241.223 161 0000.0caa.2350 ARPA Ethernet0/1Internet 38.15.254.206 144 0000.0caa.2350 ARPA Ethernet0/1Internet 206.79.171.51 153 0000.0caa.2350 ARPA Ethernet0/1

Adjacency InformationAdjacency Information

• Discovered in various ways includingARP protocol


c3620#sh ip ca verbIP routing cache 3 entries, 516 bytes 232 adds, 229 invalidates, 0 refcountsMinimum invalidation interval 2 seconds, maximum interval 5 seconds, quiet interval 3 seconds, threshold 0 requestsInvalidation rate 0 in last second, 0 in last 3 seconds Prefix/Length Age Interface Next Hop1.1.1.5/32-24 21:07:47 Ethernet0/0 1.1.1.5 14 00A0C903607700107B1F4A6108001.1.1.151/32-24 00:06:22 Ethernet0/0 1.1.1.151 14 000039542C0800107B1F4A610800171.69.0.0/16-0 01:16:08 Ethernet0/1 171.69.10.34 14 00000CAA235000107B1F4A620800

A Cached Table of ForwardingA Cached Table of Forwardingand Adjacency Informationand Adjacency Information

• Initialized after a packet has beensuccessfully Process Switched and*Can* be cached…



Prefix/Length Age Interface Next Hop1.1.1.5/32 20:50:18 Ethernet0/0 1.1.1.51.1.1.151/32 00:08:20 Ethernet0/0 1.1.1.151171.68.0.0/16 00:43:59 Ethernet0/1 171.70.10.70 This would come first171.68.1.0/24 00:08:39 Ethernet0/2 171.68.11.34 This is more specific199.2.54.0/24 00:03:54 Ethernet0/1 199.2.54.193

Finding an Entry in a TableFinding an Entry in a Table

• We could look sequentially through each entry untilwe find the one we are looking for

• This could end up being very time consuming and(unless the tables were sorted inversely with time)probably end up with the most useful entries (i.e. themost recent ones) at the end

• Also it is possible that a more specific and moreoptimized entry maybe missed if we take the first onewe find sequentially


X=n

X=n

X=n

X=n

This Is OK for Small Tables ofInformation Where the MaximumLook up Time Is the Number onEntries in the List

üü

Linear List Look-UpLinear List Look-Up

Find X=4Find X=4

123456789

123456789

How Binary Trees Help UsHow Binary Trees Help UsSpeed up PerformanceSpeed up Performance



Find Midpointof the List

Find Midpointof the List

üü

Or…

13468911

13468911

If X= -> L, Found,If X< -> L, look in first half of the list,else look in the second half




Binary List Look-UpBinary List Look-Up

How Binary Trees Help UsHow Binary Trees Help UsSpeed up PerformanceSpeed up Performance

An Entry in a Table W/2 BillionEntries Will Take 32 IterationsAn Entry in a Table W/2 BillionEntries Will Take 32 Iterations

Find X=4Find X=4

134

134 44


66

1111884411

33 99

This Is a Node(In This Case the Root Node)

LHS RHS

Compare X to Root NodeIf X = RN DoneIf X < RN Search in LHS If X > RN Search in RHS

How Binary Trees Help UsHow Binary Trees Help UsSpeed-Up PerformanceSpeed-Up Performance

Binary List Look-UpBinary List Look-Up

Find X=4Find X=4

Look-Up Time = (Max = Logn, Where N=no. Nodes)Look-Up Time = (Max = Logn, Where N=no. Nodes)



Cisco Router InternalsCisco Router Internals(Low-Mid Range)(Low-Mid Range)



Contiguous System BuffersContiguous System Buffers (Small, Large, Huge, Etc.) (Small, Large, Huge, Etc.)

Public, Dynamic, ConfigurablePublic, Dynamic, Configurable

Contiguous Interface BuffersContiguous Interface BuffersPrivate, Static, ConfigurablePrivate, Static, Configurable

Particle Based in 7200 (512 B) 3600Particle Based in 7200 (512 B) 3600(1524 B), Nonconfigurable(1524 B), Nonconfigurable

Shared DRAMShared DRAM

Tx and Rx Descriptor RingsTx and Rx Descriptor Rings Private, Circular Linked Lists Private, Circular Linked Lists

SRAM 7200 (Npe150/200)SRAM 7200 (Npe150/200)

Particle-basedParticle-basedInterface Buffer.Interface Buffer.

(3 Hi BW PA Only),(3 Hi BW PA Only),NonconfigurableNonconfigurable

Tx and Rx RingsTx and Rx Rings

Main DRAMMain DRAM

Cisco IOSCisco IOS(Not 1600/2500)(Not 1600/2500)

RunningRunningConfigConfigDataData

StructuresStructures

RoutingRoutingTablesTables

SwitchSwitchCacheCache

BufferBufferHeadersHeaders

Low-End SystemsLow-End Systems



Cisco 1000/2500 FamilyCisco 1000/2500 Family



CPUCPU Boot ROM

DRAM

SerialCSU/DSUBRI S/T, U

NVRAM

Ethernet

ConsoleI/O BusesI/O Buses

PCMCIA

CP

U B

usC

PU

Bus

Cisco 100x SeriesCisco 100x Series

M 68360SCC

M 68360SCC



WICWIC

CPUCPU

SerialCSU/DSUBRI S/T, U Ethernet

ConsoleI/O BusesI/O Buses

Boot ROM

PCMCIA

CP

U B

usC

PU

Bus

NVRAM

DRAMSIMM

On BoardDRAM

WIC Slot


M 68360SCC

M 68360SCC


WICWIC

On BoardDRAM

Hub Ports2505, 2507

2516

DRAMSIMM

WIC Slots2524, 2525

Sys

tem

Bus

WAN Intf

Async2509-2512

Ether/TR

Mgmt Card2517-2519

Daughter andHub Cards

Boot ROM

PCMCIA

CP

U B

usC

PU

Bus

Flash

DualUART

NVRAM

M 68030M 68030

Sys CtrlASIC

Sys CtrlASIC

Cisco 25xx SeriesCisco 25xx Series



Cisco 3600 FamilyCisco 3600 Family



NetworkModulesNetworkModules

R 4700R 4700

NetworkModulesNetworkModules

Sys CtrlGT 64010Sys Ctrl

GT 64010

PC

I Bus

0

PCMCIA

CPU BusCPU Bus

PC

I Bus

2

PCIBridge

PC

I Bus

1

PCIBridge

Boot ROM

I/O B

us

DRAM

Flash

NVRAM

DualUART

Cisco 36x0 SeriesCisco 36x0 Series



Cisco 4000 FamilyCisco 4000 Family



NIMsNIMsM 68030M 68030

DB

us Sys CtrlLogic

Sys CtrlLogic

Boot ROM

SharedDRAM

NVRAM

CP

U B

usC

PU

Bus

DualUART

Flash /EPROM

MainDRAM

Con

trol

Bus

Cisco 4000 and 4000MCisco 4000 and 4000M



NIMsNIMsD

Bus

Con

trol

Bus

PowerSupply

R 4600R 4700R 4600R 4700

Sys CtrlASICs

Sys CtrlASICs

CPU BusCPU Bus

SharedDRAM

MainDRAM

Layer 2Cache

4700 Only

OtherLogic I/O

Bus

Boot ROM

Flash

DualUART

NVRAM

Boot Flash

Cisco 4500 and 4700Cisco 4500 and 4700


40004000

CPUCPU

M68030M68030 CISCCISC 32 bit32 bit NIMNIM --40 MHz40 MHz

4000M4000M M68030M68030 CISCCISC 32 bit32 bit NIMNIM --40 MHz40 MHz

45004500 R4600R4600 RISCRISC 64 bit64 bit NIMNIM --100 MHz100 MHz

PROCESSORPROCESSOR TYPETYPE CLOCKCLOCK CPU BusCPU Bus INTERFACESINTERFACES Layer 2 CACHELayer 2 CACHE

4500M4500M R4600R4600 RISCRISC 64 bit64 bit NIMNIM --100 MHz100 MHz

47004700 R4700R4700 RISCRISC 64 bit64 bit NIMNIM 512 KB512 KB133 MHz133 MHz

4700M4700M R4700R4700 RISCRISC 64 bit64 bit NIMNIM 512 KB512 KB133 MHz133 MHz

36203620 R4700R4700 RISCRISC 64 bit64 bit NM,WIC,VICNM,WIC,VIC --80 MHz80 MHz

36403640 R4700R4700 RISCRISC 64 bit64 bit NM,WIC,VICNM,WIC,VIC --100 MHz100 MHz

25xx25xx M68360M68360 CISCCISC 32 bit32 bit Built inBuilt in --20 MHz20 MHz

17xx17xx ?????????? ?????????? ???????????? BI,WIC,VICBI,WIC,VIC --??????????

160x160x M68360M68360 CISCCISC 32 bit32 bit BI,WIC,VICBI,WIC,VIC --33 MHz33 MHz

100x100x M68360M68360 CISCCISC 32 bit32 bit Built inBuilt in --25 MHz25 MHz

4xxx

4xxx

36XX

36XX

Low-Mid Router ComparisonLow-Mid Router Comparison



High-End SystemsHigh-End Systems

CPUCPUCPU

Contiguous System BuffersPublic, Dynamic, Configurable

Contiguous System BuffersContiguous System BuffersPublic, Dynamic, ConfigurablePublic, Dynamic, Configurable

Contiguous Interface BuffersContiguous Interface BuffersShared, Static, NonconfigurableShared, Static, Nonconfigurable

SRAMSRAM

DRAMDRAM

Local BuffLocal Buff

RSPRSPRSPRPSP/SSPRPRPSP/SSPSP/SSP

MicroMicroProcessorProcessorNetworkNetwork

ControllersControllers

MicrocodeMicrocode

Tx andTx andRx RingsRx Rings

IProcIProc SRAMSRAM

Particle BasedParticle Based256 B256 B

Local BuffsLocal Buffs

MicroMicroProcessorProcessor

VIPVIP

DRAMDRAM

Tx and RxTx and RxRingsRings

VIPVIPCiscoCiscoIOSIOS

PAPA

DistributedDistributedCacheCache

Main DRAMMain DRAMCisco IOS

(Not 1600/2500)Cisco IOSCisco IOS

(Not 1600/2500)(Not 1600/2500)Running

ConfigurationRunningRunning

ConfigurationConfigurationData

StructuresDataData

StructuresStructures

RoutingTables

RoutingRoutingTablesTables

SwitchCacheSwitchSwitchCacheCache

BufferHeadersBufferBuffer

HeadersHeaders

NetworkNetworkControllersControllers

PAPANetworkNetwork

ControllersControllers


Cisco 7200/7500 FamilyCisco 7200/7500 Family




CPUNPECPUNPEMidplaneMidplane

I/O ControllerI/O Controller

I/O Bus

PCI Bus 0

EEPROM

PCIBridge

R 4700R 5000R 4700R 5000

SRAM! NPE-100

Layer 2Cache

NPE-200

CP

U B

usC

PU

Bus

DRAM

PCIBridge

PCI Bridge

PCI Bridge

PCI Bridge

PA 6PA 6

PA 4PA 4

PA 2PA 2

PCI Bridge

PCI Bridge

PCI Bridge


PA 5PA 5

PA 3PA 3

PA 1PA 1

Sys CtrlGT 64010Sys Ctrl

GT 64010P

CI B

us 2

PC

I Bus

1

PCMCIAFast Ether

Boot ROMDual

UART

NVRAM Boot Flash


FanTray

CPU BusCPU Bus

M 68040M 68040Bit SliceProc.

Bit SliceProc.EEPROM

Multi BusIntf Logic

SP/SSPSP/SSP

Diag BusIntf LogicSRAM

DRAM

RPRP

Mul

ti B

us

CxBus IntfDMA Logic

Multi BusIntf Logic

Cx Bus

Cisco 70x0—RP and SP/SSPCisco 70x0—RP and SP/SSP

Local BusLocal Bus

I/O CtrlI/O

Devices

Diag BusIntf Logic

Intf Proc.Intf Proc.Cx BusArbiter

Diag BusDiag Bus

Intf Proc.Intf Proc.



Cisco 70x0—RSP7000Cisco 70x0—RSP7000

FanTray IP / VIPIP / VIP

DRAMRSP7KRSP7KCI BoardCI Board

QA ASIC

SRAM

RegisterFPGA

Diag BusIntf Logic

EnvmLogic

EEPROM

Cx BusArbiter

I/O B

us

Boot ROM

PCMCIA

DualUART

NVRAM

Boot Flash

CPU BusCPU Bus

R 4600R 4600Sys CtrlASICs

Sys CtrlASICs

IP/VIPIP/VIP

Diag BusDiag Bus

Cx Bus

Diag BusFPGA

MemD CtrlASICs

MemD CtrlASICs


Boot ROM

PCMCIA

DualUART

NVRAM

Boot Flash

RSPRSP

QA ASIC

SRAM

RegisterFPGA

Cy Bus 0 Cy Bus 1

Cisco 75xx SeriesCisco 75xx Series

DRAMSys CtrlASICs

Sys CtrlASICs Layer 2

Cache

CPU BusCPU Bus

R 4600R 4700R 5000

R 4600R 4700R 5000

I/O B

us

Diag BusDiag Bus

IP/VIPIP/VIP IP/VIPIP/VIP Cy BusArbiter IP/VIPIP/VIP IP/VIPIP/VIP

Diag BusFPGA

MemD CtrlASICs

MemD CtrlASICs



R 4600R 4700R 5000

R 4600R 4700R 5000

Boot ROM

PCIBridge 2

DRAMVIPVIP

Cisco 75xx Series—VIPCisco 75xx Series—VIP

PCIBridge 1

PC

I Bus

1P

CI B

us 2

PAPA

PAPAPMA

ASICsPMA

ASICs

SRAM

Diag BusDiag BusCBus

Layer 2Cache

CPU BusCPU Bus

PC

I Bus

0

Pac

ket B

us

CYAASICsCYA

ASICs

I/O CtrlASIC

I/O CtrlASIC

EEPROM

DRAM CtrlASICs

DRAM CtrlASICs


RSP1RSP1

CPUCPU

R4600R4600 RISCRISC 64 bit64 bit IP,VIP1,VIP2IP,VIP1,VIP2 --100 MHz100 MHz

RSP2RSP2 R4600/R4700R4600/R4700 RISCRISC 64 bit64 bit IP,VIP1,VIP2IP,VIP1,VIP2 --100 MHz100 MHz

RSP4RSP4 R5000R5000 RISCRISC 64 bit64 bit IP,VIP1,VIP2IP,VIP1,VIP2 512 KB512 KB200 MHz200 MHz

PROCESSORPROCESSOR TYPETYPE CLOCKCLOCK CPU BusCPU Bus INTERFACESINTERFACES Layer 2Layer 2CACHECACHE

VIP2-15VIP2-15 R4700R4700 RISCRISC 64 bit64 bit PAPA 512 KB512 KB100 MHz100 MHz



NPE100NPE100 R4700R4700 RISCRISC 64 bit64 bit PA,IO -FEPA,IO -FE 512 KB512 KB150 MHz150 MHz



RPRP M68040M68040 RISCRISC 32 bit32 bit IP, VIP1IP, VIP1 --40 MHz40 MHz

RPRP M68040M68040 RISCRISC 32 bit32 bit IP, VIP1IP, VIP1 --40 MHz40 MHz

RSP7KRSP7K R4600R4600 RISCRISC 64 bit64 bit IP, VIP1,VIP2IP, VIP1,VIP2 --100 MHz100 MHz

7500

7500

7200

7200

7000

7000

High End Router ComparisonHigh End Router Comparison


756011094_06F9_c4 © 1999, Cisco Systems, Inc. 756011094_06F9_c1 © 1999, Cisco Systems, Inc.

GSR FamilyGSR Family


CPU I/f

Clock and Scheduler Cards

Switch Fabric Cards

RPCPU

RPCPU

CPU I/f

CPU I/f

CPU I/f

CPU I/fCPU I/f

CPU I/f

CPU I/f

CPU I/f

CPU I/f

GSR Switch FabricGSR Switch Fabric



Transceivers

Eac

h C

ard

can

Sw

itch

15G

bps

1.25Gbps per Trace

RPCPU

CPU I/f

GSR ThroughputGSR Throughput


R5000R5000CPUCPU

TigerTigerAsicAsic

NVRAMNVRAM

Boot Boot ROMROM

Layer 2Layer 2CacheCache

DRAMDRAM SRAMSRAM

TrancieverTranciever

TrancieverTranciever

SLISLI

SLISLI

EthernetEthernet

PCMCIAPCMCIA

ConsoleConsole

Flash SIMMFlash SIMM

I/O BUSI/O BUS

CSARCSARFIAFIA

FIAFIA

R5000 CPU Executes Cisco IOS Software—200mhz RISCProcessor

Layer 2 CACH—512KB Write through Data and Instruction Cache

TIGER ASIC Connects RP to DRAM, CSAR I/O Bus

DRAM Memory for Main Storage Supporing up to 256MB

CSAR Segments and Reassembles Packets

SAR SRAM Provides a Cache for the CSAR

FABRIC INTERFACE ASICS Manage Cisco Cells to and fromthe Fabric

SLI ASICS Encode Packets for Transmission on the Fabric

8B/10B Encoding Similar to Gigabit Ethernet or FDDI

GSR RPGSR RP



Mode Mode CSC CSC SFC SFC Bandwidth Clock Redundancy Bandwidth Clock Redundancy Fabric RedundancyFabric Redundancy

Entry Level Entry Level 1 1 0 0 622Mbps 622Mbps none none none none

Redundant Entry Redundant Entry 2 2 0 0 622Mbps 622Mbps 1:1 1:1 1:1 1:1

High BW High BW 1 1 3 3 2.4Gbps 2.4Gbps none none none none

High BWHigh BW 2 2 2 2 2.4Gbps 2.4Gbps 1:1 1:1 none none

Redundant High BW Redundant High BW 2 2 3 3 2.4Gbps 2.4Gbps 1:1 1:1 1:N1:N

GSR SwitchGSR SwitchProcessor ConfigurationsProcessor Configurations


GSR Interface CardGSR Interface Card

Tx Tx Rx Rx

512KbBurst Buffer

512KbBurst Buffer

512KbBurst Buffer

512KbBurst Buffer

Processor

SQE16-64MBBuffer

Memory

16-64MBBuffer

Memory

Fabric InterfaceFabric Interface

16-64MBBuffer

Memory

16-64MBBuffer

Memory

CPU I/f



Blocked (grrrrr!!)

Head of Line Blocking (HOL)Head of Line Blocking (HOL)


Shared Memory

Output QueueOutput Queue

CPU

Interface

Interface

Interface

A

B

C

CC CC BBCC

CongestedCongested

Delayed/Dropped



“Lane Control”“Lane Control”

A

B

C

D

B B

B

D D

D

C

C

C

A

A

A


A CB B

B

D D

D

C

C

C

A

A

A

No interface(Outbound) Canaffect anotherInterface

B

D

All Destinations Have a LaneAll Destinations Have a Lane



Shared Memory

Virtual Input QueuingVirtual Input Queuing

CPU

Interface

Interface

Interface

A

B

C

CC CC

BBCC

CongestedCongested

Delayed/Dropped

FIFO

FIFO


Shared Memory

CPU

Virtual Input QueueVirtual Input Queue

Interface

Interface

Interface

A

B

C

CC CC CC

CongestedCongested

FIFO

FIFOQueueScheduler

BB



Please Complete YourPlease Complete YourEvaluation FormEvaluation Form

Session 601Session 601



cisco arch easy

Documents

shared memory601

sram qa asic

interface

shared memory

system buffers

output hold

drop packet

pci bus 2