introduction to ethernet switches

Introduction to Ethernet Switches

Contents

Ethernet/802.3CSMA/CDEthernet/Fast Ethernet/Gigabit Ethernet

Layer 2 Ethernet SwitchesBridge vs. Layer 2 Switch

Layer 3 Ethernet Switches (Routing Switches)Router vs. Layer 3 Switch

Router Accelerator or Router BoosterProduct Positioning

802.3 CSMA/CD[Carrier Sense Multiple Access with Collision Detection]

802.3 Standard Evolution

802.3 Reference Model

Typical Node Hardware

Ethernet CSMA/CD

Ethernet CSMA/CD with Collision

Packet Formats

• 802.3 Packet Format

1010….1010 10101011

• Ethernet Packet Format

1010….1010 11

Pre-Pre-ambleamble62 Bits62 Bits

SynSyn2 Bits2 Bits

TypeTypeFieldField2 Bytes2 Bytes

Destination Address Fields

I/G U/LOrganizationally Unique Identifier

Assigned by IEEEVendor Assigned

24 bits 24 bits

0 = Individual Address1 = Group Address

0 = Globally Administered Address1 = Locally Administered

Fast Ethernet (802.3u)

Acute Introduction 12

802.3u Functional Overview

Gigabit Ethernet (802.3z)

802.3z MAC Control

Carrier Extension

Extend Carrier to 512ByteAchieve acceptable performanceExtension is non-data symbols

DA SA Type/LEN DATA FCSPreamble ExtensionSFD

64 Bytes Min Frame

512 Bytes Min

Duration of Carrier Event

Frame Bursting

Extend first frame if necessary

Transmit another frame if burst timer not expired

Inter packet gap is same symbols as extension

Frame 1 Frame 2 Frame 3Extension IPG

Frame 1 IPG Frame 2

512 Bytes

8192 (8K) Bytes

802.3z Full Duplex

CSMA/CD Parameters

Contents





An Example Network

...

...

校園網路 (FDDI)

...

1st

Qtr

2nd

Qtr

0

20

40

60

80

1st

Qtr

2nd

Qtr

1st

Qtr

2nd

Qtr

0

20

40

60

80

1st

Qtr

2nd

Qtr

1st

Qtr

2nd

Qtr

0

20

40

60

80

1st

Qtr

2nd

Qtr

1st

Qtr

2nd

Qtr

0

20

40

60

80

1st

Qtr

2nd

Qtr

1st

Qtr

2nd

Qtr

0

20

40

60

80

1st

Qtr

2nd

Qtr

1st

Qtr

2nd

Qtr

0

20

40

60

80

1st

Qtr

2nd

Qtr

Router

Router

Router

廣域網路

Bridge Bridge

區域網路

區域網路

Bridge

Bridge區域網路

區域網路

區域網路

Bridges - Layer 2

Extends a LAN by relaying frames

Forward frames based on Layer 2 address

High throughput and low latency

Low per port cost

Transparent

Operate according to IEEE 802. 1D standard


Bridge Functions (1)

Learning Records addresses appearing in SA fields in the address table (Filtering

Database) with the associated port

Filtering If DA exists and the same as incoming port, then discarding the local frame

Forwarding If DA exists and not the same as incoming port, then forwarding the frame

Flooding If DA does not exist, forward to all ports except the incoming port

Aging Time out address entries periodically

Spanning Tree (Protocol) Loop resolution


Bridge Functions (2)


Multi-Port Bridge

CPU

Multi-Port Bridge

CollisionDomain Collision

Domain

Packet Memory

• One segment per port• Packet forwarding via CPU• Only one packet to be processed in one time• The bridge performance relies on the computing power of CPU


Ethernet Switch[Wirespeed Bridge]

Ethernet Switch

Switch Fabric

• One station per port• Packet forwarding via H/W• Handle multiple packets in one time


Contents






IP Routing


Traditional Router

MAC MAC

Bus

DMATransfer

Memory- Packet Buffer- Routing Tables

Route Processor- Address Lookup- Routing Protocols

MAC MAC

Routed Packets


LAN Switching[Old Paradigm]

LAN Workgroup computing (80% local, 20%

inter-subnet traffic)Local fileservers, printers, etc

Requirements:High speed layer- 2 switches

Routers to provide broadcast containment, inter- work

group communication, security


LAN Switching[New Paradigm]

Global computing (80% inter-subnet, 20% local

traffic)

Content rich applications increase congestion

Requirements:High performance throughout network

Routing controls where services are accessed

Performance of Layer 2 switches


Hardware-Based Router

Switching Fabric

L2Table

L3Table

Fast Path

Route Processor- Routing Protocols

Memory- Routing Tables

Slow Path


Routing Switch[Wirespeed Hardware Based Router]

The performance of Layer 2 switching

The intelligence of Layer 3 routing


Multi-Layer Switching

1 2 3 4 5 6

IP Router

a b c

Layer 2 switching

Layer 3 (IP) switching


Applications

16-24 ports 10/100Mb16-24 ports 10/100MbRouting SwitchRouting Switch

Server FarmServer Farm

L2 Switch

AccountingAccountingDepartmentDepartment

ManufacturingManufacturingDepartmentDepartment

AdministrationAdministrationDepartmentDepartment

ManagementManagementConsoleConsole

Small to Mid-sized BusinessSmall to Mid-sized Business

L2 Switch

L2 Switch

L2 Switch

L2 Switch

L2 Switch SoftwareEngineering

Hardware Engineering

QualityControl

Sales /Marketing

Subnet 1

Subnet 2

Subnet 3


Routing Switch Evolution


IPv4 Header Format


Sending IP Packets

RouterRouter

Intra-Subnet Communication Test under MaskTest under Mask is “true”.Next hop’s address is exactly the

destination MAC address.

Inter-Subnet Communication Test under MaskTest under Mask is “false”.Next hop’s address is the router’s

MAC address.

Host1

Host2

Inter-Subnet

Intra-Subnet


Switching Decisions (1)


Switching Decisions (2)

Does the Destination MACAddress == Switch's

MAC Address?

Layer-2 switching usingDestination MAC Address

No

YesDoes the Destination IP

Address == Switch'sIP Address?

IncomingPacket

Forward IP datagram to CPUas Management Frame

Yes

No

Layer-3 forwarding orrouting of packet

Search forDestinationAddress in

switching database

Was Destination IPAddress found in

database?

Write MACaddress of Next

Hop to DestinationMAC Address

Update SourceMAC address

Decrement "Timeto Live", and

recalculate IPHeader Checksum

Use uP IP Protocolsoftware for route

resolution

No

Was Destination AddressResolved?

Yes

Yes

Layer-3 forwarding toNext Hop Address

Layer-3 forwarding toDefault Routing Address


Contents






Switching Terminology

LAN Switch

Routing Switch

Layer 2 device for segmentation and fast forwarding

Layer 3 Switch that does containRoute Server and Topology Protocols

Router Accelerator -Layer 3 (IP) forwarding devices-Does not necessarily contain Route Server and Topology Database


Router Front-End Processor

Router

RouterAccelerator

Router


Advantages

No Infrastructure Impact

Reduced Price

Increased Performance

Maximum Scalability

Implementation Cost

No new protocols

1/10th of router price($500 vs. 5,000/100M port)

10~20x Boost

Routing protocols - not SpanningTree

A little higher than LAN switch


Learning/Forwarding inRouter Accelerator

RouterAccelerator

Router

Network PortsNetwork Ports

Router PortsRouter Ports

Learning: packets from router ports

Forwarding: packets from network portsand router ports


An Example of Inter-Subnet Communication

RouterAccelerator

NetworkNetworkPortsPorts

RouterRouterPortsPorts

FFaa

aaAA??RR

FFaa

aaAA??RR

1

23

4

DA2SA2

source Ethernet address (SA3)source IP address (SIP)destination Ethernet address (DA3)destination IP address (DIP)

(ARP_Req)

aarr

aaAArr

RR

(ARP_Res)

IP MAC

BB rr

HOSTARP cache

BB cc 3

IP MACsub portRouter Accelerator

IP cache

port 4

port 3

rraa

AA

BB

(IP Pkt)

ccrr

AA

BB

port 4

port 4

Router

Router

HOSTsend a packet

to destination IP: BBTest under Mask: false


Route Once, Switch Many

RouterAccelerator Router

Inter-Subnet traffic: Switched rather than RoutedInter-Subnet traffic: Switched rather than Routed

routingswitching


Issues of Dynamic Routing


4 3 2 1

1 2 4

3


4 3 2 1 null

1 2 3 4

OUT-BAND route refresh

IN-BAND route refresh


Routing Switch vs. Router Accelerator

Routing Switch– Route construction by RIP or

OSPF

– Longest-match prefixes lookup

– Layer 2 & 3 switching

Router Accelerator– Route construction by IP learning

– Exact-match IP address lookup

– Layer 3 switching

– Additional router’s ports (double increase of the number of ports)

– Host Movement between subnets

– Dynamic route

– IP multicast

– VLAN management

Network Technologies

Acute Communications Corp.

Victor Yao-Tzung Wang

[email protected]


Contents

Fast IP Lookup MechanismsLongest Prefix Match (LPM)Direct Lookup and Indirect Lookup

Switch ArchitecturesDesign ConsiderationsSwitch Fabric

Traffic Scheduling AlgorithmsWeighted Round-RobinWeighted Fair Queuing


General Model of a Routing Switch

IPC

IPC

OPC

OPC

Input PortController

Output PortController

Packets

Packets

Packets

Packets

Switching Fabric

(IP-in, Port-out, Next-hop-MAC)

Standard interfaces Proprietary internal architecture


Routing in an Ethernet Switch

1. Next-hop MAC substitution

2. Transport of packets from the input to the appropriate output

Switch FabricInputport

Incoming Outgoing

IP PortNext-hopMAC

OutputportBuffers

Routing Table


Three Major Functions

IP Lookups

Switching

Output Scheduling


Packet/Flow ClassifierQoS-Ready

Select packets based on the headerMF (Multi-Field) Classifier

classify packets based on a combination of one or more header

fields (source/destination address, protocol, source/destination

port)

Flow - A sequence of packets sent from a particular source to a particular destination forwarded through particular ports with a particular QoS


IP Routing Longest Prefix Match (LPM)

Composing of IP Route PrefixPrefix Address

General IP address

Prefix Mask (Prefix Length n)

Continuous n-bit 1 from high to low (n = [0, 32], Prefix Length n)

IP Route Prefix = Prefix Address & Prefix Mask

Longest Prefix MatchPrefix Match means the continuous n-bit of destination IP (DIP)

address are the same with the IP Route Prefix of length n.

Longest Prefix Match is to choose the item which has the longest prefix length of all matched route prefixes in the routing table.


An Example

D IP

139 .118 .58 .9

140 .114 .178 .66

140 .114 .128 .3

140 .114 .78 .8

168 .98 .122 .3

R ou te Lookup

B it M ap10001011011101100011101000001001

10001100011100101011001001000010

10001100011100101000000000000011

10001100011100100100111000001000

10101000011000100111101000000011

140 .114 .35 .1 10001100011100100010001100000001

M atch ing E n tries ' N o .

0

2

9

2 , 7

N u ll

2

Longes t M a tch

139

140 .114

140 .114 .128

140 .114 .78

N U LL

140 .114

E ntry N o .

0

1

2

3

4

5

6

7

8

9

10

R ou ting T ab le

P re fix A ddress P re fix M ask Leng th

139 11111111000000000000000000000000 8

140 .116 11111111111111100000000000000000 15

140 .114 11111111111111110000000000000000 16

140 .115 11111111111111110000000000000000 16

140 .118 .168 11111111111111111100000000000000 18

168 .98 .177 11111111111111111111000000000000 20

140 .114 .36 11111111111111111111110000000000 22

140 .114 .78 11111111111111111111111100000000 24

140 .117 .168 11111111111111111111111100000000 24

140 .114 .128 .0 11111111111111111111111111000000 26

140 .117 .188 .98 11111111111111111111111111111111 32

IP R ou te P re fix10001011000000000000000000000000

10001100011101000000000000000000

10001100011100100000000000000000

10001100011100110000000000000000

10001100011101101010100000000000

10101000011000101011000100000000

10001100011100100010010000000000

10001100011100100100111000000000

10001100011101011010100000000000

10001100011100101000000000000000

10001100011101011011110000000000

P ort

6

9

1

8

8

10

3

2

16

4

18


192.168.1.x/25192.168.1.x/24

A Real-World Configuration

3 2

1

InternetDIP=192.168.1.210

LPM

Router 0.0.0.0192.168.1.127192.168.1.x

192.168.1.254192.168.1.x

….

132 Me24 332 Me25 2…. ….


Routing Table vs. Layer 3 Table

CPU

Link Interfaces

..............

Switching Fabric ForwardingEngine

ip_addr ip_mask gateway if_no . . .

140.96.x.x 2. . .

Routing TableRouting Table

MAC Address(MSB)

Port_out

Static

Age

High-PriorityCPU

03163 15

IP Address

MAC Address (LSB)

reserved Block

Layer3 TableLayer3 Table


IP Routing Engine

H eaderV erifica tion

T T LD ecrem ent

andC hecksum

U pda te

M A CA ddress

S ubs titu tion

R ou teLookup

32

Bits

Ext

ern

al D

ata

Bu

s

32

Bits

In

tern

al D

ata

Bu

s

IP S

W B E

B ottlenecko f

F orw ard ing E ng ine

S B S

Ne

xt H

op

Speedup by ASIC hardware

ASIC IP Forwarding Needs Header Verification

Route Lookup (Bottleneck)

MAC Address Substitution

TTL Decrement and Checksum

Update


Route Lookup Engine

Primary GoalSpeedup and Scale the Operation

Memory Accesses Times

Size of Forwarding Table

Lookup MechanismsFast IP Lookup Mechanisms

Direct Lookup

Indirect Lookup

Indirect Lookup with Reducing Next Hop Array


Difficulty of IP LookupsAn Illustrative Example

Address: 1011 0001 10000 1

0 1

1

10

10

1


Direct Lookup

IPv4 Address

..............

Directly Spreadfor

Exactly Matching

Next Hop Array (4 GB)

32 Bits


Indirect Lookup

S egm en t O ffse tIP v4 A ddress

16 B its 16 B its

....

S egm en ta tion T ab le (64K E n tries )

........64 K B

N ex t H op A rray

N ex t H op

P o in t to N ex t H op A rray

S egm en t

O ffse t O ffse t

........64 K B

N ex t H op A rray

O ffse t

........64 K B

N ex t H op A rray

O ffse t

........64 K B

N ex t H op A rray

O ffse t

........64 K B

N ex t H op A rray

......

P o in te r/N ex t H op

F orm at

32 B its

V a lue < 256 ==> N ex t H op (W ithou t N H A ) V a lue > 255 ==> P o in te r


Indirect Lookup with Reducing Next Hop Array

S egm en tIP v4 A ddress

16 B its

....

S egm en ta tion T ab le (64K E n tries )

........2 k 0 B y tes

N ex t H op A rray

N ex t H op

P o in t to N ex t H op A rray

S egm en t

O ffse t(k 0) O ffse t(k 1)

........2 k 1 B y tes

N ex t H op A rray

O ffse t(k i)

........2 k i B y tes

N ex t H op A rray

O ffse t(k i+2)

........2 k i+2 B y tes

N ex t H op A rray

O ffse t(k 3)

........2 k 3 B y tes

N ex t H op A rray

......

F orm at

P o in te r/N ex t H op

4 B its to ind ica teO ffse t leng th - 1

28 B its

V a lue < 256 ==> N ex t H op (W ithou t N H A ) V a lue > 255 ==> P o in te r

k B itsO ffse t(k )

16 -k B itsR em a inde r B its


Contents

Fast IP Lookup MechanismsLongest Prefix MatchDirect Lookup and Indirect Lookup




Design Considerations of Ethernet Switch

8

Ethernet SwitchEthernet Switch

2. . .packet

.

.

.1 2

1

N. . .

1

N

.

.

.

Two major functions: packet routing and output port contention resolution

Buffer placements: input buffer, internal buffer, output buffer, or combinations

Output-queued (nonblocking) switch provides the best delay/throughput performance


Input vs. Output Queuing

Input Queue Output Queue2 1

2 1

2

2 1

2

1

1 1

2

1 1

2 2

2 1

2 1

2

2

11

1 1

22

Throughput of an input-queuedswitch is limited to 58.6% due toHead-of-the-Line (HOL) contention

The complexity of an output-queuedswitch is usually higher for nonblockingtransfer


Performance of Input and Output Queuing


Ethernet Switching

De-multiplexingSeparates incoming packets based on destination IP address

RoutingMoves packets to output port

Multiplexing and SchedulingCombines traffic streams at output port, taking into account QoS

parameters Buffering

Needed to absorb short bursts without packet losses Discarding

Chooses packets to discard when buffers are exhausted


Switch Fabric Classification

Switch Fabric

time division space division

sharedmemory

sharedmedia

single path multi-path

crossbarfully

interconnectedbanyan


Shared-Media Switch


Shared-Memory Switch

MUXandS/P

DMUXandP/S

RAM

. . . . . ....

.

.

.

Sequentially serve each input and output port Queues are managed as a set of linked lists


Requirement of Memory Bandwidth

N: number of portsV: port speed

Memory Bandwidth=2NV

Example: 32-line switch with line speeds of 150 Mbps

Memory Bandwidth > 9.6 Gbps


Fully Interconnected Switch Fabric


The Knockout Switch:Y. Yeh et al. (1987)

01

N-1

Input

R

...

0

R

...

N-1

...

Filters

Knockoutconcentrator

Output

When R equals 8 and the input load is close to 100 percent (under uniform traffic), the probability of loss due to output contention is below 10E-6.


Banyan Networks


Contention Resolution

Many techniques for contention resolution

Input buffering

Output buffering

Fabric buffering

Switch speedup/replication

No single technique works well in all cases

Switches employing 2 or more techniques do well


Contents

Fast IP Lookup MechanismsLongest Prefix matchDirect Lookup and Indirect Lookup




Packet Scheduling in an Ethernet Switch (1)

1

2

n

...

Problem: when and in what order to service buffered packets (queues) to meetQoS guarantees.

Difficulties:

Multiple constraints (delay, bandwidth, jitter, loss rate)Complexity of implementationNetwork load fluctuations


Packet Scheduling in an Ethernet Switch (2)

Two conflicting goals:

Sharing: to increase utilization of network resources

Isolation: to minimize effect of one flow on the QoS experienced by another

Scheduling algorithms (service disciplines) must make tradeoffs betweensharing and isolation.


Scheduling Policies

Scheduling policies may be either:Work Conserving - Link is not allowed to go idle until all queues

empty, even if queued packets are not yet scheduled to be transmitted FCFS (FIFO) Strict Priority Queuing Fair Queuing Weighted Round-Robin Queuing Weighted Fair Queuing

Non-Work Conserving - Link is allowed to go idle if queued packets are not yet scheduled to be transmitted


FCFS Queuing

All packets placed in the same queue in Fist Come First Served order

The single queue is scheduled each time the physical layer can accept a new packet


FCFS Queuing Example

Mux


FCFS Queuing: Pros and Cons

ProsExtremely simpleLow overheadMaximizes sharing (no isolation)

ConsAll packets placed in the same queue

Guaranteed, Best-Effort, Bursty, etc.

Without separation of Best-Effort flows, delays cannot be controlled

Cannot be used to guarantee QoS

Bursty flows cause delay variation for all other flows


Strict Priority Queuing

All packets classified as guaranteed-QoS or notEach class given a different queue

The classes/queues are scheduled in priority orderIf the highest priority queue has a packet, it goes

If not, if the second highest priority has a packet, it goes, etc.

Within a class, all packets still scheduled in FCFS order


Strict Priority Queuing Example

Cla

ssif

ier P

rior

itiz

er

1

2

3


Strict Priority Queuing: Pros and Cons

ProsStill very simple with low overheadMultiple traffic classes/qualities of service isolated from

each other

ConsGuaranteed flows with large burst sizes cause large delay

variations for other flows in the same class


Fair Queuing

Fair Queuing was developed to ensure fairness and prevent bursty flows from interfering with other flows.Fair Queuing sets up firewall between each flow

Each flow is allocated its own dedicated queue.Each of the queues are serviced one packet at a time.Also called per-flow queuing.


Fair Queuing Example

Cla

ssif

ier

Sch

edu

ler


Fair Queuing: Pros and Cons

ProsBounded delaysBursty flows don’t cause delay variation in other flowsExcellent for giving fair treatment to many best-effort

flowsCons

Requires per-flow queues High overhead and complexity

Each flow is given the same amount of bandwidth Cannot support many flows with a large difference in bandwidth


Weighted Round-Robin Queuing

Weighted Round-Robin was developed to allow Fair Queuing to be used with a large number of flows with a wide variation in bandwidth.

Very similar to Fair QueuingEach flow is allocated its own dedicated queue.

Each of the queues are serviced one-at-a-time, in order, but when a queue is serviced, it may send more than one packet.Amount is limited to a predetermined number.


Weighted Round-Robin Queuing Example

Cla

ssif

ier

Sch

edu

ler

1

5

20

100

2

65


Weighted Round-Robin Queuing: Pros and Cons

ProsFlows can be given controlled amounts of bandwidthBounded delays

ConsSlightly higher overhead and complexity than Fair QueuingDelay variation may be high

Sum of all weights of other flow in worst case

Sends very bursty traffic to downstream switch May degenerate maximum burstiness to downstream


An Improved WRR Scheduling Algorithm

AA

FF

E C

DD

BB

Round Cycle 1

Round Cycle 2

The adjacent visit to A are separated by either three or five other visits.What is solution for a large number of round cycles to have a smallerinter-service delay jitter?


Weighted Fair Queuing

Weighted Fair Queuing was developed to have all of the advantages of Weighted Round-Robin, but to eliminate the burstiness.

The queues are scheduled out-of-order on precisely their required frequency.

When a queue is scheduled, it may send exactly one packet.


Sorted Priority Mechanism

Switch

PriorityAssignment

Ordering andTransmission


Weighted Fair Queuing Example

Cla

ssif

ier

Sch

edu

ler

Queues almost empty

1

5

20

100

2

65

sorted on priority


Weighted Fair Queuing: Pros and Cons

ProsExtremely fine control over bandwidthCan compensate for delay and jitter across switch

ConsVery complexRequires queue sorted on timestamp containing one packet

from each queueSimple timestamp computation algorithm can introduce

jitter

introduction to ethernet switches

Documents