netfpga hands-on training day 1
DESCRIPTION
Presented by: Adam Covington (Stanford University) Indiana University, Bloomington June 18 - 19, 2012 http://NetFPGA.org. NetFPGA Hands-on Training Day 1. Tutorial Outline. Background Introduction The NetFPGA Platform The Stanford Base Reference Router Motivation: Basic IP review - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/1.jpg)
Hands-on Training – June 18-19, 2012 1
NetFPGA Hands-on TrainingDay 1
Presented by: Adam Covington (Stanford University)
Indiana University, BloomingtonJune 18 - 19, 2012
http://NetFPGA.org
![Page 2: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/2.jpg)
Hands-on Training – June 18-19, 2012 2
Tutorial Outline• Background
– Introduction– The NetFPGA Platform
• The Stanford Base Reference Router– Motivation: Basic IP review– Example: Reference Router running on the NetFPGA
• Infrastructure– Tree– Build System– Scripts
• The Life of a Packet Through the NetFPGA– Hardware Datapath – Interface to software: Exceptions and Host I/O
• Implementation– Module Template– User Data Path– Write Crypto NIC using a static key
• Simulation and Debug– Write and Run Simulations for Crypto NIC
• Concluding Remarks
![Page 3: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/3.jpg)
Hands-on Training – June 18-19, 2012 3
Section I: Motivation
![Page 4: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/4.jpg)
Hands-on Training – June 18-19, 2012 4
NetFPGA = Networked FPGA
A line-rate, flexible, open networking platform for teaching and research
![Page 5: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/5.jpg)
Hands-on Training – June 18-19, 2012 5
NetFPGA 1G Board
NetFPGA consists of…
Four elements:
• NetFPGA board
• Tools + reference designs
• Contributed projects
• CommunityNetFPGA 10G Board
![Page 6: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/6.jpg)
Hands-on Training – June 18-19, 2012 6
NetFPGA 1G NetFPGA 10G
4 x 1Gbps Ethernet Ports 4 x 10Gbps Ethernet Ports
4.5 MB ZBT SRAM64 MB DDR2 SDRAM
27 MB QDRII-SRAM288 MB RLDRAM-II
PCI PCI Express x8
Virtex II-Pro 50 Virtex 5 TX240T
NetFPGA Board Comparison
![Page 7: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/7.jpg)
Hands-on Training – June 18-19, 2012 7
FPGAFPGA
MemoryMemory
1GE1GE
1GE1GE
1GE1GE
1GE1GE
NetFPGA board
PCI
CPUCPU MemoryMemory
NetFPGA Board
PC with NetFPGA
NetworkingSoftwarerunning on a standard PC
A hardware acceleratorbuilt with a Field Programmable Gate Arraydriving Gigabit network links
![Page 8: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/8.jpg)
Hands-on Training – June 18-19, 2012 8
Tools + Reference Designs
Tools:• Compile designs• Verify designs• Interact with hardware
Reference designs:• Router (HW)• Switch (HW)• Network Interface Card (HW)• Router Kit (SW)• SCONE (SW)
![Page 9: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/9.jpg)
Hands-on Training – June 18-19, 2012 9
Contributed Projects
More projects:http://netfpga.org/foswiki/NetFPGA/OneGig/ProjectTable
Project Contributor
OpenFlow switch Stanford University
Packet generator Stanford University
NetFlow Probe Brno University
NetThreads University of Toronto
zFilter (Sp)router Ericsson
Traffic Monitor University of Catania
DFA UMass Lowell
![Page 10: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/10.jpg)
Hands-on Training – June 18-19, 2012 10
Community
Wiki• Documentation
– User’s Guide– Developer’s Guide
• Encourage users to contribute
Forums• Support by users for users• Active community - 10s-100s of posts/week
![Page 11: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/11.jpg)
Hands-on Training – June 18-19, 2012 11
International Community
Over 1,000 users, using 2,000 cards at150 universities in 40 countries
![Page 12: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/12.jpg)
Hands-on Training – June 18-19, 2012 12
NetFPGA’s Defining Characteristics
• Line-Rate– Processes back-to-back packets
• Without dropping packets • At full rate of Gigabit Ethernet Links
– Operating on packet headers • For switching, routing, and firewall rules
– And packet payloads• For content processing and intrusion prevention
• Open-source Hardware – Similar to open-source software
• Full source code available • BSD-Style License
– But harder, because • Hardware modules must meeting timing• Verilog & VHDL Components have more complex interfaces • Hardware designers need high confidence in specification of modules
![Page 13: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/13.jpg)
Hands-on Training – June 18-19, 2012 13
Test-Driven Design
• Regression tests– Have repeatable results – Define the supported features– Provide clear expectation on functionality
• Example: Internet Router– Drops packets with bad IP checksum– Performs Longest Prefix Matching on destination address– Forwards IPv4 packets of length 64-1500 bytes– Generates ICMP message for packets with TTL <= 1– Defines how packets with IP options or non IPv4
… and dozens more … Every feature is defined by a regression test
![Page 14: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/14.jpg)
Hands-on Training – June 18-19, 2012 14
Who, How, Why
Who uses the NetFPGA?– Teachers– Students– Researchers
How do they use the NetFPGA?– To run the Router Kit– To build modular reference designs
• IPv4 router• 4-port NIC• Ethernet switch, …
Why do they use the NetFPGA?– To measure performance of Internet systems– To prototype new networking systems
![Page 15: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/15.jpg)
Hands-on Training – June 18-19, 2012 15
Summer Camp Objectives
• Overall picture of NetFPGA• How reference designs work• How you can work on a project
– NetFPGA Design Flow– Directory Structure, library modules and projects– How to utilize contributed projects
• Interface/Registers– How to verify a design (Simulation and Regression
Tests)– Things to do when you get stuck
AND… You can build your own projects!
![Page 16: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/16.jpg)
16 S T A N F O R D U N I V E R S I T Y Hands-on Training – June 18-19, 2012 16
Section II: Network review
![Page 17: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/17.jpg)
17 S T A N F O R D U N I V E R S I T Y Hands-on Training – June 18-19, 2012 17
Internet Protocol (IP)
Data
DataIP
Hdr
Eth Hdr
DataIP
Hdr
Data to betransmitted:
IP packets:
EthernetFrames:
DataIP
HdrData
IP Hdr
Eth Hdr
DataIP
HdrEth Hdr
DataIP
Hdr
…
…
![Page 18: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/18.jpg)
18 S T A N F O R D U N I V E R S I T Y Hands-on Training – June 18-19, 2012 18
Internet Protocol (IP)
Data
DataIP
Hdr…
16 3241
Options (if any)
Destination Address
Source Address
Header ChecksumProtocolTTL
Fragment OffsetFlagsFragment ID
Total Packet LengthT.ServiceHLenVer
20 b
ytes
![Page 19: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/19.jpg)
19 S T A N F O R D U N I V E R S I T Y Hands-on Training – June 18-19, 2012 19
Basic operation of an IP router
R3
A
B
C
R1
R2
R4 D
E
FR5
R5F
R3E
R3D
Next HopDestination
D
![Page 20: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/20.jpg)
20 S T A N F O R D U N I V E R S I T Y Hands-on Training – June 18-19, 2012 20
Basic operation of an IP router
A
B
C
R1
R2
R3
R4 D
E
FR5
![Page 21: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/21.jpg)
21 S T A N F O R D U N I V E R S I T Y Hands-on Training – June 18-19, 2012 21
Forwarding tables
Entry Destination Port
12⋮ 232
0.0.0.00.0.0.1⋮
255.255.255.255
12⋮12
~ 4 billion entries
Naïve approach:One entry per address
Improved approach:Group entries to reduce table sizeEntry Destination Port
12⋮50
0.0.0.0 – 127.255.255.255128.0.0.1 – 128.255.255.255
⋮248.0.0.0 – 255.255.255.255
12⋮12
IP address 32 bits wide → ~ 4 billion unique address
![Page 22: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/22.jpg)
22 S T A N F O R D U N I V E R S I T Y Hands-on Training – June 18-19, 2012 22
IP addresses as a line
0 232-1
Entry Destination Port
12345
StanfordBerkeley
North AmericaAsia
Everywhere (default)
12345
All IP addresses
North AmericaAsia
BerkeleyStanford
Your computer My computer
![Page 23: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/23.jpg)
23 S T A N F O R D U N I V E R S I T Y Hands-on Training – June 18-19, 2012 23
Longest Prefix Match (LPM)
Entry Destination Port
12345
StanfordBerkeley
North AmericaAsia
Everywhere (default)
12345
Universities
Continents
Planet
DataTo:
Stanford
Matching entries:•Stanford•North America•Everywhere
Most specific
![Page 24: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/24.jpg)
24 S T A N F O R D U N I V E R S I T Y Hands-on Training – June 18-19, 2012 24
Longest Prefix Match (LPM)
Entry Destination Port
12345
StanfordBerkeley
North AmericaAsia
Everywhere (default)
12345
Universities
Continents
Planet
DataTo:
Canada
Matching entries:•North America•Everywhere
Most specific
![Page 25: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/25.jpg)
25 S T A N F O R D U N I V E R S I T Y Hands-on Training – June 18-19, 2012 25
Implementing Longest Prefix Match
Entry Destination Port
12345
StanfordBerkeley
North AmericaAsia
Everywhere (default)
12345
Most specific
Least specific
Searching
FOUND
![Page 26: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/26.jpg)
26 S T A N F O R D U N I V E R S I T Y Hands-on Training – June 18-19, 2012 26
Basic components of an IP router
Control Plane
Data Planeper-packet processing
SwitchingForwarding
Table
Routing Table
Routing Protocols
Management& CLI
Softw
areH
ardware
Queuing
![Page 27: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/27.jpg)
27 S T A N F O R D U N I V E R S I T Y Hands-on Training – June 18-19, 2012 27
IP router components in NetFPGA
SCONE
Routing Table
Routing Protocols
Management& CLI
Output PortLookup
ForwardingTable
InputArbiter
OutputQueues
Switching Queuing
Linux
Routing Table
Routing Protocols
Management& CLI
Router Kit
OR
Softw
areH
ardware
![Page 28: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/28.jpg)
28 Hands-on Training – June 18-19, 2012 28
Section III: Example
![Page 29: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/29.jpg)
29 Hands-on Training – June 18-19, 2012 29
Operational IPv4 router
Control Plane
Data Planeper-packet processing
Softw
areH
ardware
Routing Table
Routing Protocols
Management& CLI
SCONE
SwitchingForwarding
TableQueuing
Reference router
Java GUI
![Page 30: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/30.jpg)
30 Hands-on Training – June 18-19, 2012 30
Streaming video
![Page 31: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/31.jpg)
31 Hands-on Training – June 18-19, 2012 31
Streaming video
PC & NetFPGA(NetFPGA in PC)
NetFPGA runningreference router
![Page 32: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/32.jpg)
32 Hands-on Training – June 18-19, 2012 32
Streaming video
Video streaming over shortest path
VideoclientVideo
server
![Page 33: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/33.jpg)
33 Hands-on Training – June 18-19, 2012 33
Streaming video
VideoclientVideo
server
Link breaks
![Page 34: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/34.jpg)
34 Hands-on Training – June 18-19, 2012 34
Streaming video
.1.1
.1.2.3.1
.30.2
.4.1
.4.2
.6.1.3.2
.7.1
.7.2
.9.1
.6.2
.10.1
.10.2
.12.1
.9.2
.13.1
.13.2
.15.1
.12.2
.16.1
.16.2.15.2
.28.1
.28.2.27.1
.30.1
.25.1
.25.2.24.1
.27.2
.22.1
.22.2.21.1
.24.2
.19.1
.19.2
.21.2.18.2
.18.1
.23.1
![Page 35: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/35.jpg)
35 Hands-on Training – June 18-19, 2012 35
Observing the routing tables
Columns:•Subnet address•Subnet mask•Next hop IP•Output ports
![Page 36: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/36.jpg)
36 Hands-on Training – June 18-19, 2012 36
Demo
![Page 37: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/37.jpg)
37 Hands-on Training – June 18-19, 2012 37
Review
NetFPGA as IPv4 router:•Reference hardware + SCONE software•Routing protocol discovers topology
Demo:•Ring topology•Traffic flows over shortest path•Broken link: automatically route around failure
![Page 38: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/38.jpg)
38 Hands-on Training – June 18-19, 2012 38
Section IV: Infrastructure
![Page 39: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/39.jpg)
39 Hands-on Training – June 18-19, 2012 39
Infrastructure
• Tree structure
• NetFPGA package contents– Reusable Verilog modules– Verification infrastructure– Build infrastructure– Utilities– Software libraries
![Page 40: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/40.jpg)
40 Hands-on Training – June 18-19, 2012 40
Tree Structure (1)
netfpga
binbin
liblib
projectsprojects
bitfilesbitfiles
(scripts for running simulations and setting up the environment)(scripts for running simulations and setting up the environment)
(contains the bitfiles for all projects that have been synthesized)(contains the bitfiles for all projects that have been synthesized)
(shared Verilog modules, libraries needed for simulation/synthesis/design)(shared Verilog modules, libraries needed for simulation/synthesis/design)
(user projects, including reference designs)(user projects, including reference designs)
![Page 41: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/41.jpg)
41 Hands-on Training – June 18-19, 2012 41
Tree Structure (2)
lib
CC
javajava
MakefilesMakefiles
Perl5Perl5
pythonpython
scriptsscriptsverilogverilog
(common software and code for reference designs)(common software and code for reference designs)
(contains software for the graphical user interface)(contains software for the graphical user interface)
(makefiles for simulation and synthesis)(makefiles for simulation and synthesis)
(libraries to interact with reference designs, create test data, and manage simulations/regression tests)(libraries to interact with reference designs, create test data, and manage simulations/regression tests)
(common libraries to aid in regression tests)(common libraries to aid in regression tests)
(utility scripts – less commonly used than those in the bin directory)(utility scripts – less commonly used than those in the bin directory)
(modules that can be reused in designs)(modules that can be reused in designs)
![Page 42: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/42.jpg)
42 Hands-on Training – June 18-19, 2012 42
Tree Structure (3)
projects/crypto_nic
docdoc
includeinclude
srcsrc
swsw
synthsynth
testtest
(project specific documentation)(project specific documentation)
(XML files defining project and any local modules, auto-generated Verilog register defines)(XML files defining project and any local modules, auto-generated Verilog register defines)
(non-library Verilog code used for synthesis and simulation)(non-library Verilog code used for synthesis and simulation)
(software elements of the project)(software elements of the project)
(project-specific .xco files to generate cores, Makefile to implement the design)(project-specific .xco files to generate cores, Makefile to implement the design)
(simulation and hardware tests)(simulation and hardware tests)
liblib (C/Perl defines for registers)(C/Perl defines for registers)
![Page 43: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/43.jpg)
43 Hands-on Training – June 18-19, 2012 43
NetFPGA package contents
• Projects:– HW: router, switch, NIC, buffer sizing router– SW: router kit, SCONE
• Reusable Verilog modules• Verification infrastructure:
– simulate full board with PCI + physical interfaces– run tests against hardware– test data generation libraries (eg. packets)
• Build infrastructure• Utilities:
– register I/O, packaging, …• Software libraries
![Page 44: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/44.jpg)
44 Hands-on Training – June 18-19, 2012 44
Reusable Verilog modules
Category Modules
I/O interfaces Ethernet MACCPU DMA queuesCPU register queues
MDIOPCI
Output queues SRAM-basedDRAM-based
BRAM-based
Output port lookup Router (CAM-based)Learning switch (CAM-based)
NICHardwired
Memory interfaces SRAM DRAM
Miscellaneous FIFOsGeneric register module
Rate limiter
![Page 45: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/45.jpg)
45 Hands-on Training – June 18-19, 2012 45
Verification Infrastructure
• Simulation: nf_test.py sim– allows testing before synthesis– catches many bugs
• Hardware tests: nf_test.py hw– test synthesized hardware
• Test data generation libraries:– easily create test data:– many standard packet formats supported out of
the box– easily add support for custom formats
![Page 46: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/46.jpg)
46 Hands-on Training – June 18-19, 2012 46
Build infrastructure
• Register system:– allocates memory to modules– generates “include” files for various languages
• Build/synthesis:– required shared modules documented XML
(shared with register system)– shared modules pulled in during synthesis– resultant bitfile checked for timing errors
![Page 47: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/47.jpg)
47 Hands-on Training – June 18-19, 2012 47
Utilities
• Bitfile download: nf_download• Register I/O: regread, regwrite• Device querying: nf_info• SRAM dumping: lib/scripts/sram_dump
![Page 48: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/48.jpg)
48 Hands-on Training – June 18-19, 2012 48
Software libraries
• Libraries for interfacing with NetFPGA:– C, Perl, Java, Python support
![Page 49: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/49.jpg)
Hands-on Training – June 18-19, 2012 49
Section V: Life of a Packet
![Page 50: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/50.jpg)
Hands-on Training – June 18-19, 2012 50
Reference Router Pipeline
• Five stages– Input– Input arbitration– Routing decision and
packet modification– Output queuing– Output
• Packet-based module interface
• Pluggable design
MACRxQMACRxQ
CPURxQCPURxQ
MACRxQMACRxQ
CPURxQCPURxQ
MACRxQMACRxQ
CPURxQCPURxQ
MACRxQMACRxQ
CPURxQCPURxQ
Input ArbiterInput Arbiter
Output Port LookupOutput Port Lookup
MACTxQMACTxQ
CPUTxQCPUTxQ
MACTxQMACTxQ
CPUTxQCPUTxQ
MACTxQMACTxQ
CPUTxQCPUTxQ
MACTxQMACTxQ
CPUTxQCPUTxQ
Output QueuesOutput Queues
![Page 51: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/51.jpg)
Hands-on Training – June 18-19, 2012 51
Full System Components
Software
PCI Bus
NetFPGA
CPURxQCPURxQ
CPUTxQ
nf2_reg_grp
user data path
nf2c0 nf2c1 nf2c2 nf2c3 ioctl
MACTxQMACTxQ
MACRxQMACRxQ
Ethernet
CPURxQCPURxQ
CPUTxQ
CPURxQCPURxQ
CPUTxQ
CPURxQCPURxQ
CPUTxQ
MACTxQMACTxQ
MACRxQMACRxQ
MACTxQMACTxQ
MACRxQMACRxQ
MACTxQMACTxQ
MACRxQMACRxQ
![Page 52: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/52.jpg)
Hands-on Training – June 18-19, 2012 52
port0 port2192.168.2.y192.168.1.x
Life of a Packet through the Hardware
IP packet
![Page 53: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/53.jpg)
Hands-on Training – June 18-19, 2012 53
Inter-Module Communication
Using “Module Headers”:
IP Hdr
Eth Hdr
…
0
0
0
Last word of packet0x10
Last Module Hdry
……
Module Hdrx Contain information such as packet length, input port, output port, …
Data Word(64 bits)
Ctrl Word(8 bits)
![Page 54: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/54.jpg)
Hands-on Training – June 18-19, 2012 54
Module i
Module i+1
data
Inter-Module Communication
ctrlwr
rdy
![Page 55: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/55.jpg)
Hands-on Training – June 18-19, 2012 55
MAC Rx Queue
MAC Rx Queue
![Page 56: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/56.jpg)
Hands-on Training – June 18-19, 2012 56
Rx Queue
Rx Queue
IP Hdr:IP Dst: 192.168.2.3,
TTL: 64, Csum:0x3ab4
Eth Hdr:Dst MAC = port 0,
Ethertype = IP
Data
0
0
0
Pkt length,input port = 0
0xff
![Page 57: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/57.jpg)
Hands-on Training – June 18-19, 2012 57
Input Arbiter
Input Arbiter
Rx Q 0
Rx Q 1
…
Rx Q 7
Pkt
Pkt
Pkt
![Page 58: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/58.jpg)
Hands-on Training – June 18-19, 2012 58
Output Port Lookup
Output Port
Lookup
![Page 59: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/59.jpg)
Hands-on Training – June 18-19, 2012 59
Output Port
LookupIP Hdr:IP Dst: 192.168.2.3,
TTL: 64, Csum:0x3ab4
IP Hdr:IP Dst: 192.168.2.3,
TTL: 63, Csum:0x3ac2
Output Port Lookup
EthHdr: Dst MAC = 0Src MAC = x,
Ethertype = IP
Data
0
0
0
Pkt length,input port = 00xff
1- Check input port matches
Dst MAC
2- Check TTL, checksum
3- Lookup next hop IP & output port
(LPM)
4- Lookup next hop MAC address (ARP)
5- Add output port header
6- Modify MAC Dst and Src addresses
7-Decrement TTL and update
checksum
EthHdr: Dst MAC = nextHop Src MAC = port 4,
Ethertype = IP
Pkt length,input port = 0
output port = 4
![Page 60: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/60.jpg)
Hands-on Training – June 18-19, 2012 60
Output Queues
Output Queues
OQ0
OQ4
OQ7
![Page 61: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/61.jpg)
Hands-on Training – June 18-19, 2012 61
MAC Tx Queue
MAC Tx Queue
![Page 62: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/62.jpg)
Hands-on Training – June 18-19, 2012 62
MAC Tx Queue
MAC Tx Queue
IP Hdr:IP Dst: 192.168.2.3,
TTL: 64, Csum:0x3ab4
IP Hdr:IP Dst: 192.168.2.3,
TTL: 63, Csum:0x3ac2
EthHdr: Dst MAC = nextHop Src MAC = port 4,
Ethertype = IP
Data
0
0
0
Pkt length,input port = 0
output port = 40xff
![Page 63: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/63.jpg)
Hands-on Training – June 18-19, 2012 63
Exception Packets
• Example: TTL = 0 or TTL = 1
• Packet has to be sent to the CPU• Host generates an ICMP packet response• Difference starts at the Output Port Lookup
![Page 64: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/64.jpg)
Hands-on Training – June 18-19, 2012 64
nf2c0 nf2c1 nf2c2 nf2c3 ioctl
Ethernet
Exception Packet Path
CPURxQCPURxQ
CPUTxQCPUTxQ
CPURxQCPURxQ
CPUTxQCPUTxQ
CPURxQCPURxQ
CPUTxQCPUTxQ
CPURxQCPURxQ
CPUTxQCPUTxQ
nf2_reg_grpnf2_reg_grp
user data pathuser data path
MACTxQMACTxQ
MACRxQMACRxQ
MACTxQMACTxQ
MACRxQMACRxQ
MACTxQMACTxQ
MACRxQMACRxQ
MACTxQMACTxQ
MACRxQMACRxQ
Ethernet
Software
PCI Bus
NetFPGA
![Page 65: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/65.jpg)
Hands-on Training – June 18-19, 2012 65
Output Port
LookupIP Hdr:IP Dst: 192.168.2.3,
TTL: 1, Csum:0x3ab4
Output Port Lookup
EthHdr: Dst MAC = 0,Src MAC = x,
Ethertype = IP
Data
0
0
0
Pkt length,input port = 00xff
1- Check input port matches
Dst MAC
2- Check TTL, checksum – EXCEPTION!
3- Add output port module
Pkt length,input port = 0
output port = 1
![Page 66: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/66.jpg)
Hands-on Training – June 18-19, 2012 66
Output Queues
Output Queues
OQ0
OQ1
OQ2
OQ7
![Page 67: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/67.jpg)
Hands-on Training – June 18-19, 2012 67
CPU Tx Queue
CPU Tx Queue
![Page 68: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/68.jpg)
Hands-on Training – June 18-19, 2012 68
CPU Tx Queue
CPU Tx Queue
IP Hdr:IP Dst: 192.168.2.3,
TTL: 1, Csum:0x3ab4
EthHdr: Dst MAC = 0, Src MAC = x,
Ethertype = IP
Data
0
0
0
Pkt length,input port = 0
output port = 10xff
![Page 69: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/69.jpg)
Hands-on Training – June 18-19, 2012 69
ICMP Packet
• Packet arrives at the CPU Rx Queue from the PCI Bus
• Same path as a packet from the MAC until it reaches the Output Port Lookup (OPL)
• The OPL module sees the packet is from the CPU Rx Queue 1 and sets the output port directly to 0
• The packet continues on the same path as the non-exception packet to the Output Queues and then MAC Tx queue 0
![Page 70: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/70.jpg)
Hands-on Training – June 18-19, 2012 70
nf2c0 nf2c1 nf2c2 nf2c3 ioctl
Ethernet
ICMP Packet Path
CPURxQCPURxQ
CPUTxQCPUTxQ
CPURxQCPURxQ
CPUTxQCPUTxQ
CPURxQCPURxQ
CPUTxQCPUTxQ
CPURxQCPURxQ
CPUTxQCPUTxQ
nf2_reg_grpnf2_reg_grp
user data pathuser data path
MACTxQMACTxQ
MACRxQMACRxQ
MACTxQMACTxQ
MACRxQMACRxQ
MACTxQMACTxQ
MACRxQMACRxQ
MACTxQMACTxQ
MACRxQMACRxQ
Ethernet
Software
PCI Bus
NetFPGA
![Page 71: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/71.jpg)
Hands-on Training – June 18-19, 2012 71
NetFPGA-Host Interaction
• Linux driver interfaces with hardware– Packet interface via standard Linux network
stack
– Register reads/writes via ioctl system call with wrapper functions:
• readReg(nf2device *dev, int address, unsigned *rd_data);• writeReg(nf2device *dev, int address, unsigned *wr_data);
eg:readReg(&nf2, OQ_NUM_PKTS_STORED_0, &val);
![Page 72: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/72.jpg)
Hands-on Training – June 18-19, 2012 72
NetFPGA-Host Interaction
NetFPGA to host packet transferNetFPGA to host packet transfer
PC
I Bu
sP
CI B
us
2. Interrupt notifies driver of packet arrival
2. Interrupt notifies driver of packet arrival
3. Driver sets up and initiates DMA transfer
3. Driver sets up and initiates DMA transfer
1. Packet arrives – forwarding table sends to CPU queue
1. Packet arrives – forwarding table sends to CPU queue
![Page 73: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/73.jpg)
Hands-on Training – June 18-19, 2012 73
NetFPGA-Host Interaction
NetFPGA to host packet transfer (cont.)NetFPGA to host packet transfer (cont.)
PC
I Bu
sP
CI B
us
4. NetFPGA transfers packet via DMA
4. NetFPGA transfers packet via DMA
5. Interrupt signals completion of DMA
5. Interrupt signals completion of DMA
6. Driver passes packet to network stack6. Driver passes packet to network stack
![Page 74: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/74.jpg)
Hands-on Training – June 18-19, 2012 74
NetFPGA-Host Interaction
Host to NetFPGA packet transfersHost to NetFPGA packet transfers
PC
I Bu
sP
CI B
us
3. Interrupt signals completion of DMA
3. Interrupt signals completion of DMA
1. Software sends packet via network sockets
Packet delivered to driver
1. Software sends packet via network sockets
Packet delivered to driver
2. Driver sets up and initiates DMA transfer
2. Driver sets up and initiates DMA transfer
![Page 75: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/75.jpg)
Hands-on Training – June 18-19, 2012 75
NetFPGA-Host Interaction
Register accessRegister access
PC
I Bu
sP
CI B
us
1. Software makes ioctl call on network socket
ioctl passed to driver
1. Software makes ioctl call on network socket
ioctl passed to driver
2. Driver performs PCI memory read/write
2. Driver performs PCI memory read/write
![Page 76: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/76.jpg)
Hands-on Training – June 18-19, 2012 76
NetFPGA-Host Interaction
• Packet transfers shown using DMA interface
• Alternative: use programmed IO to transfer packets via register reads/writes– slower but eliminates the need to deal with
network sockets
![Page 77: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/77.jpg)
Hands-on Training – June 18-19, 2012 77
Section VI: Example Project
![Page 78: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/78.jpg)
Hands-on Training – June 18-19, 2012 78
Project: Cryptographic NIC
Implement a network interface card (NIC) that encrypts upon transmission and
decrypts upon reception
![Page 79: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/79.jpg)
Hands-on Training – June 18-19, 2012 79
Cryptography
XOR function
XOR written as: ^ ⊻ ⨁XOR is commutative: (A ^ B) ^ C = A ^ (B ^ C)
A B A ^ B
0 0 0
0 1 1
1 0 1
1 1 0
XORing a value with itself always yields 0
![Page 80: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/80.jpg)
Hands-on Training – June 18-19, 2012 80
Cryptography (cont.)
Simple cryptography:– Generate a secret key– Encrypt the message by XORing the message and key– Decrypt the ciphertext by XORing with the key
Explanation:
(M ^ K) ^ K = M ^ (K ^ K)
= M
= M ^ 0
Commutativity
A ^ A = 0
![Page 81: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/81.jpg)
Hands-on Training – June 18-19, 2012 81
Cryptography (cont.)
Example:
Message: 00111011
Key: 10110001
Message ^ Key: 10001010
Key: 10110001
Message ^ Key ^ Key: 00111011
![Page 82: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/82.jpg)
Hands-on Training – June 18-19, 2012 82
Cryptography (cont.)
Idea: Implement simple cryptography using XOR– 32-bit key– Encrypt every word in payload with key
Note: XORing with a one-time pad of the same length of the message is secure/uncrackable. See: http://en.wikipedia.org/wiki/One-time_pad
PayloadHeader
Key Key Key Key Key
⨁
![Page 83: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/83.jpg)
Hands-on Training – June 18-19, 2012 83
Section VII: Implementation
![Page 84: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/84.jpg)
Hands-on Training – June 18-19, 2012 84
Getting started with a new project (1)
• Projects:– Each design represented by a project
– Location: netfpga/projects/<proj_name>• netfpga/projects/crypto_nic
– Consists of:• Verilog source• Simulation tests• Hardware tests• Libraries• Optional software
![Page 85: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/85.jpg)
Hands-on Training – June 18-19, 2012 85
Getting started with a new project (2)
– Normally:• copy an existing project as the starting point
– Today:• pre-created project
– Missing from pre-created project:• Verilog files (with crypto implementation)• Simulation tests• Hardware tests• Custom software
![Page 86: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/86.jpg)
Hands-on Training – June 18-19, 2012 86
Getting started with a new project (3)
Typically implement functionality in one or more modules inside the user data path
MACRxQMACRxQ
CPURxQCPURxQ
MACRxQMACRxQ
CPURxQCPURxQ
MACRxQMACRxQ
CPURxQCPURxQ
MACRxQMACRxQ
CPURxQCPURxQ
Input ArbiterInput Arbiter
Output Port LookupOutput Port Lookup
MACTxQMACTxQ
CPUTxQCPUTxQ
MACTxQMACTxQ
CPUTxQCPUTxQ
MACTxQMACTxQ
CPUTxQCPUTxQ
MACTxQMACTxQ
CPUTxQCPUTxQ
Output QueuesOutput Queues
CryptoCrypto
Crypto module to encrypt and decrypt packets
User data path
![Page 87: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/87.jpg)
Hands-on Training – June 18-19, 2012 87
Getting started with a new project (4)
– Shared modules included from netfpga/lib/verilog• Generic modules that are re-used in multiple projects• Specify shared modules in project’s include/project.xml
– Local src modules override shared modules
– crypto_nic:
Local Shared
user_data_path.vcrypto.v
Everything else
![Page 88: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/88.jpg)
Hands-on Training – June 18-19, 2012 88
Exploring project.xml (1)
• Location: project/<proj_name>/include<?xml version="1.0" encoding="UTF-8"?><nf:project …>
<nf:name>Crypto NIC</nf:name>
<nf:description>NIC with basic crypto support</nf:description>
<nf:version_major>0</nf:version_major> <nf:version_minor>1</nf:version_minor> <nf:version_revision>0</nf:version_revision>
<nf:dev_id>0</nf:dev_id>
Short nameDescription
Version information•indicate bitfile version
Unique ID to identify projectSee: http://netfpga.org/foswiki/bin/view/NetFPGA/OneGig/DeviceIDList
![Page 89: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/89.jpg)
Hands-on Training – June 18-19, 2012 89
Exploring project.xml (2) <nf:use_modules> core/io_queues/cpu_dma_queue core/io_queues/ethernet_mac core/input_arbiter/rr_input_arbiter core/nf2/generic_top core/nf2/reference_core core/output_port_lookup/nic core/output_queues/sram_rr_output_queues core/sram_arbiter/sram_weighted_rr core/user_data_path/reference_user_data_path core/io/mdio core/cpci_bus core/dma core/user_data_path/udp_reg_master core/io_queues/add_rm_hdr core/strip_headers/keep_length core/utils/generic_regs core/utils </nf:use_modules>
Shared modules toload from lib/verilog
![Page 90: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/90.jpg)
Hands-on Training – June 18-19, 2012 90
Exploring project.xml (3) <nf:memalloc layout="reference"> <nf:group name="core1"> <nf:instance name="device_id" /> <nf:instance name="dma" base="0x0500000"/> <nf:instance name="mdio" /> <nf:instance name="nf2_mac_grp" count="4" /> <nf:instance name="cpu_dma_queue" count="4" /> </nf:group> <nf:group name="udp"> <nf:instance name="in_arb" /> <nf:instance name="crypto" /> <nf:instance name="strip_headers" /> <nf:instance name="output_queues" /> </nf:group> </nf:memalloc></nf:project>
Specify where to instantiate modules, the number of instances, and the memory addresses to use
![Page 91: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/91.jpg)
Hands-on Training – June 18-19, 2012 91
Getting started with a new project (5)
Tasks:Set the project that we’ll be working with:
1. Add the following lines to the end of ~/.bashrc:export NF_DESIGN_DIR=$NF_ROOT/projects/crypto_nicexport
PERL5LIB=$NF_ROOT/lib/Perl5:$NF_DESIGN_DIR/lib/Perl5
2. Type: source ~/.bashrc
Copy reference files as starting points:
3. Copy the following files from netfpga/lib/verilog/core into netfpga/projects/crpyto_nic/src
user_data_path/reference_user_data_path/src/user_data_path.vmodule_template/src/module_template.v
![Page 92: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/92.jpg)
Hands-on Training – June 18-19, 2012 92
Getting started with a new project (6)
Create crypto.v from module_template.v:
1. Rename the local module_template.v to crypto.v2. Change the module name inside crypto.v (first non-
comment line of the file)
3. Add the crypto module to the user data path
![Page 93: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/93.jpg)
Hands-on Training – June 18-19, 2012 93
user_data_path.v (1)module user_data_path #( parameter DATA_WIDTH = 64, ... ) ( ... )
//------------------ Internal parameters ----------------------- ...
//----------------- Input arbiter wires/regs ------------------- ...
Module port declaration
![Page 94: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/94.jpg)
Hands-on Training – June 18-19, 2012 94
user_data_path.v (2) //-------------- output port lut wires/regs -------------------- wire [CTRL_WIDTH-1:0] op_lut_in_ctrl; wire [DATA_WIDTH-1:0] op_lut_in_data; wire op_lut_in_wr; wire op_lut_in_rdy;
...
//------- output queues wires/regs ------ ...
Wire declarations for the output port lookup module.Duplicate this section, and replace op_lut with crypto
![Page 95: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/95.jpg)
Hands-on Training – June 18-19, 2012 95
user_data_path.v (3) //--------- Connect the data path ----------- input_arbiter #( ... ) input_arbiter ( ... )
output_port_lookup #( ... ) output_port_lookup ( ... )
...
Module instantiations.
1.Duplicate the output_port_lookup instantiation 2.Rename to crypto3.Remove all parameters (inside the first set or parentheses)4.In the output_port_lookup instantiation, replace oq_ with crypto_5.In the crypto instantiation, replace op_lut_ with crypto_
We’ve inserted the new module into the pipeline
![Page 96: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/96.jpg)
Hands-on Training – June 18-19, 2012 96
Getting started with a new project (7)
Run a simulation to verify changes:
1. nf_test.py sim --major nic --minor short
Now we can implement the crypto functionality
![Page 97: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/97.jpg)
Hands-on Training – June 18-19, 2012 97
Implementing the Crypto Module (1)
• What do we want to encrypt?– IP payload only
• Plaintext IP header allows routing• Content is hidden
– Encrypt bytes 35 onward• Bytes 1-14 – Ethernet header• Bytes 15-34 – IPv4 header (assume no options)
– Assume all packets are IPv4 for simplicity
![Page 98: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/98.jpg)
Hands-on Training – June 18-19, 2012 98
Implementing the Crypto Module (2)
• State machine (draw on next page):– Module headers on each packet– Datapath 64-bits wide
• 34 / 8 is not an integer! • Inside the crypto module
![Page 99: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/99.jpg)
Hands-on Training – June 18-19, 2012 99
Crypto Module State Diagram
Hint: We suggest 4 states (or 3 if you’re feeling adventurous)
SkipModuleHeaders
![Page 100: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/100.jpg)
Hands-on Training – June 18-19, 2012 100
Implementing the Crypto Module (3)
Implement your state machine inside crypto.v– Use a static key initially
Suggested sequence of steps:1. Create a static key value
• Constants can be declared in the module with localparam:localparam MY_EXAMPLE = 32’h01234567;
2. Implement your state machine without modifying the packet
3. Update your state machine to modify the packet by XORing the key and the payload
• Use two copies of the key to create a 64-bit value to XOR with data words
![Page 101: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/101.jpg)
Hands-on Training – June 18-19, 2012 101
module_template.v (1)module module_template #( parameter DATA_WIDTH = 64, parameter CTRL_WIDTH = DATA_WIDTH/8, parameter UDP_REG_SRC_WIDTH = 2 ) ( ... )
//----------------------- Signals---------------------------- ...
//------------------ Local assignments ----------------------- ...
Module port declaration
![Page 102: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/102.jpg)
Hands-on Training – June 18-19, 2012 102
module_template.v (2) //------------------------- Modules-------------------------------
fallthrough_small_fifo #( .WIDTH(CTRL_WIDTH+DATA_WIDTH), .MAX_DEPTH_BITS(2) ) input_fifo ( .din ({in_ctrl, in_data}), // Data in .wr_en (in_wr), // Write enable .rd_en (in_fifo_rd_en), // Read the next word .dout ({in_fifo_ctrl, in_fifo_data}), .full (), .nearly_full (in_fifo_nearly_full), .prog_full (), .empty (in_fifo_empty), .reset (reset), .clk (clk) );
Packet data dumped in a FIFO. Allows some “decoupling” between input and output.
![Page 103: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/103.jpg)
Hands-on Training – June 18-19, 2012 103
module_template.v (3) generic_regs #( .UDP_REG_SRC_WIDTH (UDP_REG_SRC_WIDTH), .TAG (0), .REG_ADDR_WIDTH (1), .NUM_COUNTERS (0), .NUM_SOFTWARE_REGS (0), .NUM_HARDWARE_REGS (0) ) module_regs ( ... );
Generic registers.
Ignore for now – we’ll explore this later
![Page 104: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/104.jpg)
Hands-on Training – June 18-19, 2012 104
module_template.v (4) //------------------------- Logic-------------------------------
always @(*) begin // Default values out_wr_int = 0; in_fifo_rd_en = 0;
if (!in_fifo_empty && out_rdy) begin out_wr_int = 1; in_fifo_rd_en = 1; end end
Combinational logic to read data from the FIFO. (Data is output to output ports.)
You’ll want to add your state in this section.
![Page 105: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/105.jpg)
Hands-on Training – June 18-19, 2012 105
Inter-module Communication
Module i+1
`
Module i
datadata
ctrlctrlwrwrrdyrdy
![Page 106: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/106.jpg)
Hands-on Training – June 18-19, 2012 106
Implementing the Crypto Module (3)
Implement your state machine inside crypto.v– Use a static key initially
Suggested sequence of steps:1. Create a static key value
• Constants can be declared in the module with localparam:localparam MY_EXAMPLE = 32’h01234567;
2. Implement your state machine without modifying the packet
3. Update your state machine to modify the packet by XORing the key and the payload
• Use two copies of the key to create a 64-bit value to XOR with data words
![Page 107: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/107.jpg)
Hands-on Training – June 18-19, 2012 107
Section VIII: Simulation and Debug
![Page 108: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/108.jpg)
Hands-on Training – June 18-19, 2012 108
Testing: Simulation (1)
• Simulation allows testing without requiring lengthy synthesis process
• NetFPGA simulation environment allows:– Send/receive packets
• Physical ports and CPU– Read/write registers– Verify results
• Simulations run in ModelSim/VCS/ISim
![Page 109: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/109.jpg)
Hands-on Training – June 18-19, 2012 109
Testing: Simulation (2)
• Simulations located in project/test• Multiple simulations per project
– Test different features• Example:
– crypto_nic/test/both_nic_short• Send one packet from CPU, expect packet out
physical port• Send one packet in physical port, expect packet to
CPU
Note: This test will not work once your crypto module is implemented!
![Page 110: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/110.jpg)
Hands-on Training – June 18-19, 2012 110
Testing: Simulation (3)
Useful functions:Register access:
nftest_regwrite(addr, value)nftest_regread_expect(addr, expect)
Packet generation:make_IP_pkt(…) – see documentationencrypt_pkt(key, pkt)decrypt_pkt(key, pkt)
Packet transmission/reception:nftest_send_phy(interface, pkt)nftest_expect_phy(interface, pkt)nftest_send_dma(interface, pkt)nftest_expect_dma(interface, pkt)
![Page 111: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/111.jpg)
Hands-on Training – June 18-19, 2012 111
Testing: Simulation (4)
Task:Implement tests for encryption and decryption
Modify the following tests:netfpga/projects/crypto_nic/test/both_crypto_encrypt/run.pynetfpga/projects/crypto_nic/test/both_crypto_decrypt/run.py
Look at both_nic_short as an example of creating IP packets and sending/receiving them
![Page 112: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/112.jpg)
Hands-on Training – June 18-19, 2012 112
Running Simulations
• Use command nf_test.py– Required Parameter
• sim or hw (right now only use sim)– Optional parameters
• --major <major_name>• --minor <minor_name>• --gui (starts the default viewing environment)
both_crypto_encrypt
majormajor minorminor
![Page 113: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/113.jpg)
Hands-on Training – June 18-19, 2012 113
Running Simulations
Non-GUI execution example:# 10756.00ns testbench.host32.service_interrupt: Info: Interrupt signaled# 10935 Host read 0x00000044 with cmd 0x6: Disconnect with Data, # 10995 CPCI Interrupt: DMA ingress xfer complete# 11175 Host read 0x00000148 with cmd 0x6: Disconnect with Data, # 11415 Host read 0x00000150 with cmd 0x6: Disconnect with Data, # 11475.00ns testbench.host32.service_interrupt: Info: DMA ingress transfer complete. # 11655 Host read 0x00000040 with cmd 0x6: Disconnect with Data, # Timecheck: 13645.00ns# 20100 Simulation has reached finish time - ending.# ** Note: $finish : /home/summercamp/netfpga/lib/verilog/core/testbench/target32.v# Time: 20100 ns Iteration: 0 Instance: /testbench/target32--- Simulation is complete. Validating the output.
Comparing simulation output for port 1 ...Port 1 matches [1 packets]Comparing simulation output for port 2 ...Port 2 matches [0 packets]
--- Test PASSED (test_nic_short) Test test_nic_short passed!------------SUMMARY---------------PASSING TESTS:
test_nic_shortFAILING TESTS: TOTAL: 1 PASS: 1 FAIL: 0
![Page 114: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/114.jpg)
Hands-on Training – June 18-19, 2012 114
Running Simulations
GUI execution example:
![Page 115: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/115.jpg)
Hands-on Training – June 18-19, 2012 115
Running Simulations
GUI execution example (cont)Try the following:
nf_test.py sim --major crypto --minor encrypt –gui
In the transcript window of the GUI:do wave.dorun 10us
You should see waveforms of packets going in and coming out of the crypto module
![Page 116: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/116.jpg)
Hands-on Training – June 18-19, 2012 116
Running Simulations
• When running ModelSim interactively:– Click "no" when simulator prompts to finish
– Changes to code can be recompiled without quitting ModelSim:
• bash# cd /tmp/$(whoami)/test/<projname>; make model_sim• VSIM 5> restart -f; run -a
– Ensure $NF_DESIGN_DIR is correct
![Page 117: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/117.jpg)
Hands-on Training – June 18-19, 2012 117
Section IX: Conclusion
![Page 118: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/118.jpg)
Hands-on Training – June 18-19, 2012 118
Nick McKeown, Glen Gibb, Jad Naous, David Erickson, G. Adam Covington, John W. Lockwood, Jianying Luo, Brandon Heller,
Paul Hartke, Neda Beheshti, Sara Bolouki, James Zeng, Jonathan Ellithorpe, Sachidanandan Sambandan, Eric Lo,
Sam D’Amico
Acknowledgments
NetFPGA Team at Stanford University (Past and Present):
NetFPGA Team at University of Cambridge (Past and Present):
Andrew Moore, David Miller, Martin Zadnik, Muhammad Shahbaz
All Community members (including but not limited to):
Paul Rodman, Kumar Sanghvi, Wojciech A. Koszek, Yahsar Ganjali, Martin Labrecque, Jeff Shafer,
Eric Keller , Tatsuya Yabe, Bilal Anwer,Yashar Ganjali, Martin Labrecque
Ram Subramanian, Kees Vissers, Michaela Blott, Shep Siegel
![Page 119: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/119.jpg)
Hands-on Training – June 18-19, 2012 119
Special thanks to our Partners:
Other NetFPGA Tutorials Presented At:
SIGMETRICS
Ram Subramanian, Patrick Lysaght, Veena Kumar, Paul Hartke, Anna Acevedo
Xilinx University Program (XUP)
See: http://NetFPGA.org/tutorials/
![Page 120: NetFPGA Hands-on Training Day 1](https://reader035.vdocuments.site/reader035/viewer/2022062315/568159c3550346895dc7148e/html5/thumbnails/120.jpg)
Hands-on Training – June 18-19, 2012 120
Thanks to our Sponsors:
• Support for the NetFPGA project has been provided by the following companies and institutions
Disclaimer: Any opinions, findings, conclusions, or recommendations expressed in these materials do not necessarily reflect the views of the National Science Foundation or of any other sponsors supporting this project.