liberouter / cesnet tmc group | programmable …...hardware accelerated nic (hanic) •accelerated...
TRANSCRIPT
![Page 1: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/1.jpg)
Nadpis 1
Nadpis 2
Nadpis 3
Jméno Příjmení
Vysoké učení technické v Brně, Fakulta informačních technologií v Brně
Božetěchova 2, 612 66 Brno
99.99.2008
Flexible network monitoring at 100Gbps
and beyond
Lukáš Kekely, Viktor Puš
{kekely,pus}@cesnet.cz
2nd SIG-PMV meeting
17th May 2017
![Page 2: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/2.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 2
CESNET
• Czech NREN with over 400,000 connected users
![Page 3: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/3.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 3
CESNET monitoring (Liberouter group)
• 7 metering points guarding the perimeter @ 40/100 Gbps
![Page 4: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/4.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 4
Monitoring point
• TAPed network link
• commodity Linux server(s)
• production and testing
• FPGA accelerated NICs
![Page 5: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/5.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 5
Monitoring overview
![Page 6: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/6.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 6
Monitoring overview
![Page 7: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/7.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 7
Family of accelerated NICs
![Page 8: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/8.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 8
NFB-100G2Q
• Virtex7 H580T FPGA
• 2x QSFP28 transciever cage
• 100GE or 4x 10GE
• PCIe x16 gen3 (100Gbps to RAM)
• 3x QDRIIIe (3x72Mb)
• precise timestamp input
• Intel DPDK support
![Page 9: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/9.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 9
NetCOPE platform
• rapid development of network applications on our NICs
• multi-card support (porting) made easy
• commonly usable IP cores (network modules, parsers …)
• generic data transfer protocol towards used accelerators
• fast DMA transfers of packets into host memory
![Page 10: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/10.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 10
DMA bus-master: proprietary SZE2
• the fastest DMAs available – full-duplex 100GE line-rate
![Page 11: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/11.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 11
DMA bus-master: Intel DPDK
• DPDK performance record set in April
![Page 12: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/12.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 12
P4 language
• high-level language for description of packet processing
• protocol stack independent header parsing of incoming packets
• decision making and related actions (match-action tables)
• modification and assembly of outgoing packets
• development of unique P4-to-VHDL translator (generator)
• parsing & de-parsing done; match-action underway
• live demonstration today at P4 Workshop @ Stanford
• P4 generated 100GE In-Band Network Telemetry (INT) sink
• delay heatmap of the whole network visualized as a result
![Page 13: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/13.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 13
Hardware accelerated NIC (HaNIC)
• accelerated packet capture solution with extra features
• flow-aware (hash-based) traffic distribution
• packet filtering/classification – IP prefixes, ports, protocol …
• bi-directional flows, sampling, trimming, headers
![Page 14: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/14.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 14
Software Defined Monitoring (SDM)
• new concept of hardware accelerated flow monitoring
• extensible application-specific processor for stateful flow processing
• SW applications can offload processing of bulk traffic to HW
• aimed to enable high-speed application layer monitoring
![Page 15: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/15.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 15
Software Defined Monitoring (SDM)
![Page 16: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/16.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 16
Flow exporter
• we use FlowMonExp from our partner Flowmon Technologies
• highly optimized implementation (hugepages, NUMA aware …)
• comfigurable management of flow cache records
• flexible architecture supporting user defined plugins
• input – PCAP, DPDK, our SZE2 format, preprocessed packets
• processing – DNS & HTTP analyzers, Heartbleed detector
• export – CSV, NetFlow, IPFIX
![Page 17: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/17.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 17
DDoS scrubber
• separate DDoS packets from legitimate traffic
• HaNIC firmware with extra features (rate limit, VLAN tag)
• measurement of statistics and mitigation of detected attacks
• 100 Gbps (10x10GE) prototype already deployed in network
![Page 18: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/18.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 18
Monitoring overview
![Page 19: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/19.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 19
IPFIXcol
• collector fully supporting IPFIX including enterprise elements
• include tools for subsequent data processing and mediation
• high-performance sufficient for 100GE environment
• extensible by various plugins (input, intermediate, storage)
• open-source in C++ - https://github.com/CESNET/ipfixcol/
![Page 20: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/20.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 20
SecurityCloud
• distributed flow-based collector in development
• master-slaves and proxy architecture
• based on IPFIXcol to store and distribute data
• fdistdump to execute queries on slaves
![Page 21: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/21.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 21
NEtwork MEasurements Analysis (NEMEA)
• framework for automated real-time analysis of flow data
• build as a user-defined collection of various modules
• TRAP + UniRec = high-performance and easy distribution
• detected threads reported to CERTS/CSIRT systems
• open-source - https://github.com/CESNET/NEMEA
![Page 22: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/22.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 22
NETCONF and YANG
• development of tools for full remote control of our devices
• in cooperation with IETF’s NETCONF & NETMOD groups
• libyang - YANG parser and validator with API in C
• libnetconf - NETCONF protocol implementation for Linux
• generic client-server communication API written in C
• device data modeling - v1 uses XML, v2 uses YANG
• Netopeer - set of applications with NETCONF protocol
• implementations of server, clients (webGUI or CLI) and more
https://github.com/CESNET/{libyang,libnetconf,libnetconf2,netopeer}
![Page 23: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/23.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 23
Cooperation (National)
technology transfer
(since 2003)
spin-off company (since 2007)
![Page 24: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/24.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 24
Cooperation (National)
• Best Cooperation of the Year
• project TA03010561: Distributed System for Complex Monitoring of High-Speed Networks
• highest national research award Czech Head, Industry award
• world’s first 100 Gbps Ethernet interface card
![Page 25: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/25.jpg)
• University of Twente, DACS group
• network monitoring and intrusion detection
• University of Cambridge, NetOS group
• packet classification/filtering and dynamic reconfiguration
• part of GÉANT network and projects
• PROTECTIVE, Firewall on Demand
L. Kekely: Flexible network monitoring at 100Gbps and beyond 25
Cooperation (International)
![Page 26: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/26.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 26
Cooperation (International)
• BEBA (BEhavioural BAsed forwarding) H2020 EU project
• finished last week with “Excellent“ rating
![Page 27: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/27.jpg)
L. Kekely: Flexible network monitoring at 100Gbps and beyond 27
Summary
• direct access to a lot of high-speed network data
• high-performance production and test monitoring probes
• reconfigurable FPGA acceleration cards and extensible SW
• collection, analysis and storage of flow data
• flexible and modifiable open-source tools
• large database of collected IPFIX flow records
• close connections with university and industry environment
• years of experience with national and EU research projects
We are open to new cooperation possibilities!
![Page 28: Liberouter / Cesnet TMC group | Programmable …...Hardware accelerated NIC (HaNIC) •accelerated packet capture solution with extra features •flow-aware (hash-based) traffic distribution](https://reader033.vdocuments.site/reader033/viewer/2022050104/5f43181a610b9d07720be3d6/html5/thumbnails/28.jpg)
L. Kekely: HANIC 28
Thank you for your attention!
More info:• https://www.liberouter.org/• @liberouter• [email protected]