onl np router

1 - John DeHart, Jing Lu - 04/20/23

SRAM

ONL NP Router

Rx(2 ME)

HdrFmt(1 ME)

Parse, Lookup,

Copy(3 MEs)

TCAM SRAM

Mux(1 ME)

Tx(1 ME)

QM(1 ME)

xScale xScale

xScale

Assoc. DataZBT-SRAM

Plu

gin

0

Plu

gin

1

Plu

gin

2

Plu

gin

3

Plu

gin

4NN NN NN NN

FreeList Mgr(1 ME)

Tx, QMParsePluginXScale

Stats(1 ME)

QMCopyPlugins SRAM

NN

SRAMRing

ScratchRing

NNRingNN

SRAM

64KW

64KW64KWEach


ONL SRAM Buffer DescriptorBuffer_Next (32b)LW0

LW1

LW2

LW3

LW4

Reserved (16b)LW5

LW6

Packet_Next (32b)LW7

Reserved (4b)

Free_list0000 (4b)

Ref_Cnt (8b)

MAC DAddr_47_32 (16b)Stats Index (16b)

MAC DAddr_31_00 (32b)

Reserved (32b)

Written by Freelist Mgr

Written by Rx

Written by Copy

Written by QM

Ref_Cnt (8b)

1 Written by Rx,Added to by CopyDecremented byFreelist Mgr

Buffer_Size (16b)

Packet_Size (16b)

Offset (16b)

Written by Rx and Plugins

EtherType (16b)


MUX -> PLC

Flags: Src: Source (2b): 00: Rx 01: XScale 10: Plugin 11: Undefined PT(1b): PassThrough(1)/Classify(0) Reserved (5b)

Reserved (5b)Src(2b)

PT(1b)

01237

PLCMux

L3 (IP, ARP, …) Pkt Length (16b)

Stats Index (16b)

QID(16b)In

Port(3b)

PluginTag(5b)

Flags(8b)

Rsv(4b)

OutPort(4b)

Buffer Handle(24b)

• Assume QID is user visible QID – low 13 bits are valid


Flags(8b): Why pkt is being sent to XScale TTL(1b): TTL expired Options(1b): IP Options present NoRoute(1b): No matching route or filter NonIP(1b): Non IP Packet received ARP_Needed(1b): NH_IP valid, but no MAC NH_Invalid(1b): NH_IP AND NH_MAC both invalid Reserved(2b): currently unused

PLC -> XScale

PLC

XScale

xScale

SRAML3 (IP, ARP, …)Pkt Length (16b)

Buffer Handle(24b)

Stats Index (16b)

QID(16b)In

Port(3b)

Flags(8b)

PluginTag(5b)

NH MAC DA[47:16] (32b)

NH MAC DA[15:0](16b)

Unicast/MCast bits (16b)

Rsv(8b)

Reserved (16b)

EtherType (16b)

Same as ring_in

Based on Parse &Lookup results

01237

Reserved(2b)

NR(1b)

TTL(1b)

Opt(1b)

NI(1b)

ARP(1b)

NHINV(1b)

45

• Assume QID is user visible QID – low 13 bits are valid• Port number(s) is available in Unicast/Mcast bits


PLC -> Plugins

PLC

Plugins 0-4

01237

Reserved(2b)

NR(1b)

TTL(1b)

Opt(1b)

NI(1b)

ARP(1b)

NHINV(1b)

45

Flags(8b): Why pkt is being sent to XScale TTL(1b): TTL expired Options(1b): IP Options present NoRoute(1b): No matching route or filter NonIP(1b): Non IP Packet received ARP_Needed(1b): NH_IP valid, but no MAC NH_Invalid(1b): NH_IP AND NH_MAC both invalid Reserved(2b): currently unused

xScale

SRAML3 (IP, ARP, …)Pkt Length (16b)

Buffer Handle(24b)

Stats Index (16b)

QID(16b)In

Port(3b)

Flags(8b)

PluginTag(5b)

NH MAC DA[47:16] (32b)

NH MAC DA[15:0](16b)

Unicast/MCast bits (16b)

Rsv(8b)

Reserved (16b)

EtherType (16b)

Same as ring_in

Based on Parse &Lookup results

• Assume QID is user visible QID – low 13 bits are valid• Port number(s) is available in Unicast/Mcast bits


PLC -> QM

PLC QM

Buffer Handle(24b)

QID(16b)Rsv(4b)

OutPort(4b)

L3 (IP, ARP, …)Pkt Length (16b)Reserved(16b)

Rsv(8b)

Rsv(8b)

Same as ring_in

Based on Parse & Lookup results

Maybe the added hdr buffer handle


Types of Pkts Arriving at PLC From Rx:

»Only have a payload buf, ref_cnt == 1.»To be classified except IP pkts with mal-formed hdr (these pkts

will be counted and dropped by PLC).

From XScale/Plugins:»Passthrough (PT) pkt

May/May not have a hdr buf, ref_cnt >= 1. Not processed by PLC. Copy sends it to QM.

»Non-PT pkt Only have a payload buf at arrival, ref_cnt >= 1. Is IP pkt. To be classified if it passes IP hdr validation; otherwise, pkt gets

counted and dropped by PLC.


PLC()PLC(){ dlNextBlock = BID_QM; //default downstream block if (!ring_in.PT) {

Parse(); if (dlNextBlock != BID_FREELISTMGR) Lookup();}Copy();

}

Inside Copy(), dl_sink() is called to enqueue pkts to downstream blocks.


dl_sink Special design of dl_sink(first, last, try_action)

»“first” == TRUE, means current pkt is the first one of a sequence of pkts to be sunk.

»“last” == TRUE, means current pkt is the last one of a sequence of pkts to be sunk.

»“try_action” == 1, drop the pkt if ring is full; otherwise try till succeeds.»dl_sink() has a return value to indicate if enqueue operation is successful

or not.» If DL_ORDERED is defined and “first” == TRUE, thread waits for signal

from previous context before enqueuing the first pkt.» If DL_ORDERED is defined and “last” == TRUE, thread passes signal to

next context after euqueuing the last pkt.


Where are the files located? PLC:

»ONL_Router\src\plc\ONL\plc[.c.h] Parse and Copy:

»ONL_Router\src\plc\ONL\parse[.c.h]»ONL_Router\src\plc\ONL\copy[.c.h]

Dispatch_loop:»ONL_Router\src\dispatch_loop\ONL\plc_dl.c»ONL_Router\src\dispatch_loop\ONL\dl_source[.c.h]

Ring related C functions:»ONL_Router\src\dispatch_loop\ONL\ring_formats.h»ONL_Router\src\dispatch_loop\ONL\sram_rings_WU[.c.h]»ONL_Router\src\dispatch_loop\ONL\scratch_rings_WU[.c.h]»ONL_Router\src\dispatch_loop\ONL\nn_rings[.c.h]


Parse Functions:

»Do IP Router checks (wrong ver, Hlen, Plen, cksum), count error pkts.»Decrement TTL on pkts from Rx and recompute IP cksum»Detect exceptions (expired TTL, IP option, Non-IP)»Extract lookup key

Input: »ring_in data

Output: » lookup key, dlNextBlock, eth_type, DG QID.

IP DAddr (32b) IP SAddr (32b) DPort (16b)SPort (16b)Proto(8b)

TCPFlags(12b)

Exceptions(16b)

P(3b)

PTag(5b)

140 Bit Key:

RL

PF and AF

Non-IP(1b)

ARP(1b)

IP Opt(1b)

TTL(1b)Reserved (12b)


Parse Operations:

» If pkt is from Rx: Read eth_type in Ethernet hdr

If eth_type != 0x0800, set NIP in exception bits. – If eth_type == 0x0806, set ARP in exception bits.

Return.»eth_type <= 0x0800.»For pkt from Rx, offset <= 0x18E; for pkt from XScale/plugins, get offset

from payload buf desc. »Read 20B IP hdr.

Check for ver, Hlen, Plen. If fails, count and dlNextBlock <= BID_FREELISTMGR. Return.

If Hlen > 5, set OPT in exception bits, read IP option. Verify cksum. If fails, count and dlNextBlock <= BID_FREELISTMGR. Return. If pkt is from Rx and TTL <= 1, or pkt is from plugins/XScale and TTL < 1, set

TTL in exception bits. If pkt is from Rx and TTL > 1, decrement TTL, recompute cksum and write them

back to dram. Form Datagram QID using DG QID = SA[9:8] SA[6:5] DA[6:5] (Used by Copy in

case of a zero QID). Extract key from IP hdr.

If IP protocol is TCP/UDP, read 14B, and extract key from TCP/UDP hdr. Copy Plugin Tag and In Port to lookup key.


Pseudo code - parse()parse() { //process typical case first if (pkt is from RX) {//L3 pkt hdr starts from offset 0x18E in dram pkt buffer read 16W from dram buffer offset 0x188; //this will read eth_type if (pkt is IPv4) { dg_qid = SA[9:8]SA[6:5]DA[6:5]; if (pkt has correct IP version, hdr length and pkt length) { if (pkt has no option) { compute cksum; if (cksum is correct) { if (TTL > 1) { decrement TTL and re-compute cksum; write new TTL and cksum back to dram pkt buffer; construct lookup key; return; } else {//TTL <= 1 construct lookup key and set TTL bit in exception bits; return; } } else {//cksum is wrong count the pkt and set dlNextBlock to BID_FREELISTMGR; return; } }


Pseudo code - parse() else {//pkt has option read 6 more words from dram pkt buffer and align IP/TCP hdr; based on hdr length, compute cksum; if (cksum is correct) { if (TTL > 1) { decrement TTL and re-compute cksum; write new TTL and cksum back to dram pkt buffer; construct lookup key; return; } else {//TTL <= 1 construct lookup key and set TTL bit in exception bits; return; } } else {//cksum is wrong count the pkt and set dlNextBlock to BID_FREELISTMGR; return; } } } else {//pkt has wrong IP version, or hdr length or pkt length count the pkt and set dlNextBlock to BID_FREELISTMGR; return; } }


Pseudo code - parse() else {//pkt is not IPv4 construct lookup key and set NI bit in exception bits; if (it is an ARP pkt) { set ARP bit in exception bits; } return; } } else {//pkt is from plugins or XScale read 22W from dram pkt buffer starting from IP pkt hdr, and align IP/TCP hdr; dg_qid = SA[9:8]SA[6:5]DA[6:5]; if (pkt has correct IP version, hdr length and pkt length) { if (pkt has no option) { compute cksum; if (cksum is correct) { construct lookup key; if (TTL == 0) { set TTL bit in exception bits; } return; } else {//cksum is wrong count the pkt and set dlNextBlock to BID_FREELISTMGR; return; } }


Pseudo code - parse() else {//pkt has option based on hdr length, compute cksum; if (cksum is correct) { construct lookup key; if (TTL == 0) { set TTL bit in exception bits; } return; } else {//cksum is wrong count the pkt and set dlNextBlock to BID_FREELISTMGR; return; } } } else {//pkt has wrong IP version, or hdr length or pkt length count the pkt and set dlNextBlock to BID_FREELISTMGR; return; } }}


Lookup Key and Results FormatsIP DAddr (32b) IP SAddr (32b) DPort (16b)SPort (16b)

Proto(8b)

TCPFlags(12b)

Exceptions(16b)

P(3b)

PTag(5b)

QID (16b)Stats Index

(16b)

UCastMCast(12b)

V(4b)

Prio(8b)

D(1b)

H(1b)

MH

(1b)

Address (21b)

32 Bit Result in TCAM Assoc. Data SRAM:

96 Bit Result in QDR SRAM Bank0:

PF


(16b)

UniCast(8b)

V(4b)

Res(8b)

D(1b)

H(1b)

MH

(1b)

Address (21b)AF


(16b)

UCastMCast(12b)

V(4b)

Res(8b)

D(1b)

H(1b)

MH

(1b)

Address (21b)RL

SB

(2b)

EntryValid(1b)

NHIP

Valid(1b)

NHMACValid(1b)

IPMC

Valid(1b)

NH_MAC (48b)

NH_IP (32b)Res (16b)

NH_MAC (48b)


NH_MAC (48b)


Multicast Copy Vector (11b)PPS(1b)

•If IP MC Valid = 1

D(1b)

PPS(1b)

UCast Out Port

(3b)

UCast Out Plugin

(3b)

Reserved(4b)


140 Bit Key:

RL

PF and AF

TCAM CtrlBits:

D:DoneH:HIT

MH:Multi-Hit

Res

(2b)


Copy Functions:

»Drop error pkts detected by Parse.»Count and send PT pkts to QM. »Process lookup results:

When “control error” (NH_IP and NH_MAC both invalid or both valid), or “ARP request needed” (Uncast pkt with valid NH_IP but invalid NH_MAC), or “no route” (invalid PR and AR) is detected, pkt should be sent to XScale, or one of five plugins, or dropped based on user preference (dynamically configurable).

Compute MAC DAddr for IP multicast pkts. Update total ref_cnt in paylod buf desc for each classified pkt. If total ref_cnt == 1 and pkt goes to QM, fill in payload buf desc with NH_MAC,

Stats index and EtherType. If total ref_cnt > 1, add hdr buf to each pkt going to QM, and fill in hdr buf desc

with buffer_next, packet_size, ref_cnt (=1), NH_MAC, Stats index, EtherType. For each copy, form ring_out data and enqueue it to the outgoing ring.

Input: »ring_in data, exception bits in lookup key, dlNextBlock, eth_type, DG

QID, lookup results Output:

»Ring_out data


Copy Control Block Located in SRAM (COPY_BLOCK_CONTROL_MEM_BASE) Dynamically configurable through XScale control software A sampling rate must be an integer between 0 and 0x7FFF (largest random number)

typedef __declspec(packed) union _sampling_rate{ struct { unsigned int sp_rate_00; unsigned int sp_rate_01; unsigned int sp_rate_10; unsigned int sp_rate_11; }; unsigned int i[4];} sampling_rate;

typedef __declspec(packed) struct _user_preference{ unsigned int no_route : 3;// "000" ~ "100" - plugin_0 ~ plugin_4, "101" - XScale, "111" - drop unsigned int arp_needed : 3; unsigned int nh_inv : 3;} user_pref;

typedef struct _copy_control_block{ sampling_rate sp_rate; user_pref pref;} copy_control_block;


Copy Operations:

»Error pkt detected by Parse (to freelist_mgr): Construct ring_out data to Freelist_mgr Call dl_sink(TRUE, TRUE, 0). Return.

»PT pkt (to QM): dlNextBlock <= BID_QM Construct ring_out data to QM from ring_in data Call dl_sink(TRUE, TRUE, 0). Return.

»Classified pkt: Pre-check:

Copy exception bits to flags. If PR and AR are both invalid, this pkt has no route.

– Set NR bit in flags. – Set dlNextBlock to user preference. – Construct ring_out data. – Call dl_sink(TRUE, TRUE, 1). Return.

Pre-process lookup results, because: Both PR and AR can generate pkt copies to QM and XScale/plugins. Need to know total num of copies to

– Decide whether hdr buffer decs should be added to copy going to QM.– Set “last” bit in dl_sink().– Update total ref_cnt in payload buf desc using one atomic addition.

Post-process lookup results, copy and send pkts to proper rings.


Copy Pre-processing data structure:

struct copy_plugins_prep{ unsigned int pkt_cnt; bool pr_arp; bool pr_nh_inv; bool pr_ucast; bool pr_mcast; bool ar_arp; bool ar_nh_inv; bool ar_ucast;} plugins;


(16b)

UCastMCast(12b)

V(4b)

NH_MAC (48b)



(16b)

UniCast(8b)

V(4b)

SB

(2b)

NH_MAC (48b)


Res

(2b)

PR:

AR:

Pre-process fields

Post-process fields

struct copy_qm_prep{ unsigned int pkt_cnt; bool pr_ucast; bool pr_mcast; bool ar_ucast;} qm;

Only one can be TRUE


Copy (pre-process PR)

If PR is valid, » If NH_IP is valid and NH_MAC is invalid and IP_MC is invalid, this pkt

needs ARP. Set APR bit in flags. If pkt should be sent to XSclae/plugins,

plugins.pkt_cnt ++; plugins.pr_arp <= TRUE.

»Else if NH_IP and NH_MAC are both invalid or both valid, lookup entry is mis-configured.

Set NH_INV bit in flags. If pkt should be sent to XSclae/plugins,

plugins.pkt_cnt ++; plugins.pr_nh_inv <= TRUE.


(16b)

UCastMCast(12b)

V(4b)

EntryValid(1b)

NHIP

Valid(1b)

NHMACValid(1b)

IPMC

Valid(1b)

NH_MAC (48b)




D(1b)

PPS(1b)

UCast Out Port

(3b)

UCast Out Plugin

(3b)

Reserved(4b)



Copy (pre-process PR)»Else,

If MC_valid == 0 (unicast), If Drop == 0,

– If PPS == 0, qm.pkt_cnt ++, qm.pr_ucast <= TRUE.– Else (PPS == 1), plugins.pkt_cnt ++, plugins.pr_ucast <= TRUE.

Else (MC_valid == 1 (Multicast)), If PPS == 0,

– qm.pkt_cnt <= total 1’s in high 5 bits in MCast_copyVector; – plugins.pkt_cnt <= total 1’s in low 6 bits in MCAST_copyVector. – If qm.pkt_cnt > 0, qm.pr_mcast <= TRUE.– If plugins.pkt_cnt > 0, plugins.pr_mcast <= TRUE.

Else (PPS == 1), – plugins.pkt_cnt <= total 1’s in low 6 bits in Mcast_copyVector;– If plugins.pkt_cnt > 0, plugins.pr_mcast <= TURE.


D(1b)

PPS(1b)

UCast Out Port

(3b)

UCast Out Plugin

(3b)

Reserved(4b)


Copy (pre-process AR)

If AR is valid,»If NH_IP is valid and NH_MAC is invalid, this pkt needs ARP.

Set APR bit in flags. If pkt should be sent to XSclae/plugins,

plugins.pkt_cnt ++; plugins.pr_arp <= TRUE.

»Else if NH_IP and NH_MAC are both invalid or both valid, lookup entry is mis-configured.

Set NH_INV bit in flags. If pkt should be sent to XSclae/plugins,

plugins.pkt_cnt ++; plugins.pr_nh_inv <= TRUE.


(16b)

UniCast(8b)

V(4b)

SB

(2b)

NH_MAC (48b)


Res

(2b)

EntryValid(1b)

NHIP

Valid(1b)

NHMACValid(1b)

IPMC

Valid(1b)

D(1b)

PPS(1b)

UCast Out Port

(3b)

UCast Out Plugin

(3b)


Copy (pre-process AR)

»Else, Each SB value is associated with a sampling rate rt(SB) that is

dynamically configurable by users. Generate a random number rd. If rd <= rt(SB),

If PPS == 0, – qm.ar_ucast <= TRUE, qm.pkt_cnt ++.

Else (PPS == 1), – plugins.ar_ucast <= TRUE, plugins.pkt_cnt ++.

D(1b)

PPS(1b)

UCast Out Port

(3b)

UCast Out Plugin

(3b)

Rsv(2b)

SB(2b)


Copy (post-process)

If qm.pkt_cnt + plugins.pkt_cnt == 0, dlNextBlock <= BID_FREELISTMGR, dl_sink(TRUE, TRUE, 0). Return.

Read ref_cnt from payload buf desc. If qm.pkt_cnt + plugins.pkt_cnt + (ref_cnt – 1) == 1,

» If qm.pkt_cnt == 1, DO NOT add hdr buf desc. Based on qm.pr_ucast, qm.pr_mcast, qm.ar_ucast,

Fill in NH_MAC, stats index, and eth_type in payload buf desc. Construct QM ring_out data. dlNextBlock <= BID_QM. dl_sink(TRUE, TRUE, 0). Return.

» Else (plugins.pkt_cnt == 1), Based on plugins.*,

construct plugins/XScale ring_out data. Set dlNextBlock to XScale or one of the five plugins. dl_sink(TRUE, TRUE, 1). Return.

struct copy_plugins_prep{ unsigned int pkt_cnt; bool pr_arp; bool pr_nh_inv; bool pr_ucast; bool pr_mcast; bool ar_arp; bool ar_nh_inv; bool ar_ucast;} plugins;

struct copy_qm_prep{ unsigned int pkt_cnt; bool pr_ucast; bool pr_mcast; bool ar_ucast;} qm;


(16b)

UCastMCast(12b)

V(4b)

NH_MAC (48b)



(16b)

UniCast(8b)

V(4b)

SB

(2b)

NH_MAC (48b)


Res

(2b)

PR:

AR:


Copy (post-process) Else (qm.pkt_cnt + plugins.pkt_cnt + (ref_cnt – 1) > 1),

»Add (qm.pkt_cnt + plugins.pkt_cnt – 1) to ref_cnt in payload buf desc.» If qm.pkt_cnt >0,

for each copy to QM, add hdr buf desc. Based on qm.pr_ucast, qm.pr_mcast, qm.ar_ucast,

Fill in buffer_next, packet_size, ref_cnt (=1), NH, stats index, and eth_type in hdr buf desc.

Construct QM ring_out data. dlNextBlock <= BID_QM. Call dl_sink() qm.pkt_cnt times.

– If plugins.pkt_cnt == 0, Set “last” bit in dl_sink() for the last copy and return. » If plugins.pkt_cnt > 0,

Based on plugins.*, Construct plugins/XScale ring_out data. Set dlNextBlock to XScale or one of the five plugins. Call dl_sink() plugins.pkt_cnt times.

– Set “last” bit in dl_sink() for the last copy and return.


Pseudo code - copy()copy() { //process typical case first if (it is a normal unicast pkt from RX, and PR is valid with invalid NH_IP and valid NH_MAC, and AR is invalid ) { write to payload buffer desc with stats_index, NH_MAC and eth_type; prepare QM ring data; //in qid, 3-bit port is numbered from 1 to 5, while port in lookup result is from 0 to 4 update pre-Q counter by writing to stats ring; enqueue pkt to QM ring; return; } //process all other cases drop error pkt by writing buf handle to Freelist_mgr ring; return; send PT pkt to QM, update pre-Q counter, and enqueue pkt to QM ring; return;

//pre-check for invalid PR and AR if (both PR and AR are invalid) { set dlNextBlock to user preference, and construct ring data; enqueue pkt to the specified ring; if (ring is full and pkt gets dropped) { update drop counter by writing to stats ring; } return; }


Pseudo code - copy() //pre-process lookup results initialize qm_copy and plugins_copy; //qm_copy and plugins_copy store pre-processing results //pre-process PR if (PR is valid ) { if (it’s unicast with valid NH_IP and invalid NH_MAC) {//ARP is needed if (pkt shouldn’t be dropped based on user preference) { plugins_copy.pkt_cnt++; plugins_copy.pr_arp = TRUE; } } else if (it’s unicast with NH_IP and NH_MAC both valid or both invalid) {//lookup entry is mis-configured if (pkt shouldn’t be dropped based on user preference) { plugins_copy.pkt_cnt++; plugins_copy.pr_nh_inv = TRUE; } } else {//process uc_mc_bits if (it’s unicast pkt) { if (no drop) { if (PPS == 0) {//port unicast qm_copy.pkt_cnt++; qm_copy.pr_ucast = TRUE; } else {//plugin unicast plugins_copy.pkt_cnt++; plugins_copy.pr_ucast = TRUE; } } }


Pseudo code - copy() else {//multicast if (PPS == 0) {//port and plugin multicast qm_copy.pkt_cnt = number of 1’s in high 5 bits of uc_mc_bits; if (qm_copy.pkt_cnt != 0) { qm_copy.pr_mcast = TRUE; } plugins_copy.pkt_cnt = number of 1’s in low 6 bits of uc_mc_bits; if (plugins_copy.pkt_cnt != 0) { plugins_copy.pr_mcast = TRUE; } } else {//plugin multicast plugins_copy.pkt_cnt = number of 1’s in low 6 bits of uc_mc_bits; if (plugins_copy.pkt_cnt != 0) { plugins_copy.pr_mcast = TRUE; } } } } }


Pseudo code - copy() //pre-process AR if (AR is valid ) { if (pkt has valid NH_IP and invalid NH_MAC) {//ARP is needed if (pkt shouldn’t be dropped based on user preference) { plugins_copy.pkt_cnt++; plugins_copy.ar_arp = TRUE; } } else if (NH_IP and NH_MAC both valid or both invalid) {//lookup entry is mis-configured if (pkt shouldn’t be dropped based on user preference) { plugins_copy.pkt_cnt++; plugins_copy.ar_nh_inv = TRUE; } } else {//process uc_bits generate a rand number rd; if (PPS == 0) {//port unicast if (rd <= sp_rate) {//compare rd with sampling rate sp_rate associated with sp_bits; qm_copy.pkt_cnt++; qm_copy.ar_ucast = TRUE; } } else {//plugin unicast if (rd <= sp_rate) {//compare rd with sampling rate sp_rate associated with sp_bits; plugins_copy.pkt_cnt++; plugins_copy.ar_ucast = TRUE; } } } }


Pseudo code - copy() //post-process if (qm_copy.pkt_cnt == 0 and plugins_copy.pkt_cnt == 0) {//no copy is made, drop the pkt set dlNextBlock = BID_FREELISTMGR; drop pkt by writing buf handle to Freelist_mgr ring; update drop counter by writing to stats ring; return; } if (ref_cnt == 1) {//only one copy is made if (qm_copy.pkt_cnt == 1) {//pkt goes to QM if (qm_copy.pr_ucast == TRUE) {//unicast pkt resulted from PR write to payload buffer desc with stats_index, NH_MAC and eth_type; prepare QM ring data; update pre-Q counter by writing to stats ring; } else (qm_copy.pr_mcast == TRUE) {//multicast pkt resulted from PR compute multicast NH_MAC; write to payload buffer desc with stats_index, NH_MAC and eth_type; based on port number, prepare QM ring data; update pre-Q counter by writing to stats ring; } else {//unicast pkt resulted from AR write to payload buffer desc with stats_index, NH_MAC and eth_type; prepare QM ring data; update pre-Q counter by writing to stats ring; } enqueue pkt to QM ring; return; }


Pseudo code - copy() else {//pkt goes to plugins/XScale if (plugins_copy.pr_arp == TRUE) {//APR needed resulted from PR set dlNextBlock to user preference, and construct ring data; } else if (plugins_copy.pr_nh_inv == TRUE) {//NH_INV resulted from PR set dlNextBlock to user preference, and construct ring data; } else if (qm_copy.pr_ucast == TRUE) {//unicast pkt resulted from PR set dlNextBlock to one of the plugins/XScale, and construct ring data; } else (qm_copy.pr_mcast == TRUE) {//multicast pkt resulted from PR set dlNextBlock to one of the plugins/XScale, and construct ring data; //use multicast NH_MAC } else if (plugins_copy.ar_arp == TRUE) {//APR needed resulted from AR set dlNextBlock to user preference, and construct ring data; } else if (plugins_copy.ar_nh_inv == TRUE) {//NH_INV resulted from AR set dlNextBlock to user preference, and construct ring data; } else {//unicast pkt resulted from AR set dlNextBlock to the plugins/XScale specified in uc_bits, and construct ring data; } enqueue pkt to the ring specified in dlNextBlock; if (ring is full and pkt gets dropped) { update drop counter by writing to stats ring; } return; } }


Pseudo code - copy() else {//multiple copies are made update ref_cnt in payload buffer desc; if (qm_copy.pkt_cnt > 0) {//at least one pkt goes to QM set dlNextBlock to BID_QM; if (qm_copy.pr_ucast == TRUE) {//unicast pkt resulted from PR allocate a hdr buffer desc; write to hdr buffer desc with buffer next, L3 pkt length, ref_cnt, stats_index, NH_MAC, eth_type;

prepare QM ring data, and enqueue pkt to QM; update pre-Q counter by writing to stats ring; } else if (qm_copy.pr_mcast == TRUE) {//multicast pkt(s) resulted from PR compute multicast NH_MAC; for (each output port set in uc_mc_bits) { allocate a hdr buffer desc; write to payload buffer desc with stats_index, NH_MAC and eth_type; based on port number, prepare QM ring data; enqueue pkt to QM ring; update pre-Q counter by writing to stats ring; } } if (qm_copy.ar_ucast == TRUE) {//unicast pkt resulted from AR allocate a hdr buffer desc; write to hdr buffer desc with buffer next, L3 pkt length, ref_cnt, stats_index, NH_MAC, eth_type;

prepare QM ring data, and enqueue pkt to QM; update pre-Q counter by writing to stats ring; } }


Pseudo code - copy() if (plugins_copy.pkt_cnt > 0) {//at least one pkt goes to plugins/XScale if (plugins_copy.pr_arp == TRUE) {//APR needed resulted from PR set dlNextBlock to user preference, construct ring data, and enqueue pkt; if (ring is full and pkt gets dropped) { update drop counter by writing to stats ring; } } else if (plugins_copy.pr_nh_inv == TRUE) {//NH_INV resulted from PR set dlNextBlock to user preference, construct ring data, and enqueue pkt; if (ring is full and pkt gets dropped) { update drop counter by writing to stats ring; } } else if (qm_copy.pr_ucast == TRUE) {//unicast pkt resulted from PR set dlNextBlock to the plugins/XScale specified in uc_mc_bits; construct ring data, and enqueue pkt; if (ring is full and pkt gets dropped) { update drop counter by writing to stats ring; } } else if (qm_copy.pr_mcast == TRUE) {//multicast pkt(s) resulted from PR compute multicast NH_MAC, and construct ring data; for (each plugin/XScale set in uc_mc_bits) { set dlNextBlock, and enqueue pkt; if (ring is full and pkt gets dropped) { update drop counter by writing to stats ring; } } }


Pseudo code - copy() if (plugins_copy.ar_arp == TRUE) {//APR needed resulted from AR set dlNextBlock to user preference, construct ring data, and enqueue pkt; if (ring is full and pkt gets dropped) { update drop counter by writing to stats ring; } } else if (plugins_copy.ar_nh_inv == TRUE) {//NH_INV resulted from AR set dlNextBlock to user preference, construct ring data, and enqueue pkt; if (ring is full and pkt gets dropped) { update drop counter by writing to stats ring; } } else if {//unicast pkt resulted from AR set dlNextBlock to the plugins/XScale specified in uc_bits; construct ring data, and enqueue pkt; if (ring is full and pkt gets dropped) { update drop counter by writing to stats ring; } } return; } }}


Estimated latency for PLC Typical Case Typical case: Valid IP Pkt from Rx, no option, goes to one port as a result of

PR. AR entry is invalid.

Read eth_type, IP, TCP/UDP hdr

DRAM read: 16W 300

Operation

Memory Access

Cycle Count

Write TTL, IP cksum DRAM write: 4W 300 Parse

Write NH MAC, stats index, eth_type SRAM write: 3W 150

Enqueue pkt to QM Scratch Ring write: 3W 60

Copy

Lookup 915 Lookup

Total: 1785

Dequeue pkt from MUX Scratch Ring read: 3W 60

dl_sink

dl_source


PLC performance (typical case) System is unloaded, 8 threads per ME

»Processing time: 330 cycles»Idle time: 1354 cycles


Tests Performed Parse:

»Typical case: Pkt from Rx, no error, no option, no exception.


Tests Performed Copy:

» Typical case: Valid PR with port unicast, invalid AR

» Unicast case: Valid PR with plugin unicast, invalid AR

» Multi-copy cases: Valid PR with port/plugins multicast, invalid AR

5 copies go to 5 output ports 7 copies go to one output port, 5 plugins and XScale

Valid PR with plugins multicast, invalid AR 6 copies go to 5 plugins and XScale

» Exception cases: Invalid PR and invalid AR (no route): drop pkt Valid PR, unicast, ARP needed: send pkt to XScale Valid PR, unicast, NH invalid:send pkt to XScale Valid PR, unicast with drop bit set; invalid AR: drop pkt

onl np router

Documents