trex realistic traffic generator · what problem is being solved? ... different ip pool for each...

TRexRealistic Traffic Generator

Hanoch Haim, Principal Engineer

DEVNET-1120

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

Cisco Spark

Questions? Use Cisco Spark to communicate with the speaker after the session

1. Find this session in the Cisco Live Mobile App

2. Click “Join the Discussion”

3. Install Spark or go directly to the space

4. Enter messages/questions in the space

How

cs.co/ciscolivebot#DEVNET-1120


Agenda

• Overview

• Stateless

• Stateful

• Advance Stateful


TRex – Results

• Open Source

• Cisco Customers

DEVNET-1120 5


TRex Usage Analytics monthly report (*)

(**) Users are distinct

(*) ~1200 distinct returningusers,

DEVNET-1120 6


TRex models of operation

• L7, DUT terminate TCP/SSL, flow based

• DUT inspect L7. does not change TCP. Flow based

• DUT L2/L3 Switch , packet based

DEVNET-1120 7


What Problem is Being Solved?

• Networks include complex L4-7 features, such as• Load Balancer, DPI/AVC, Firewall, NAT

• Requires testing with stateful and realistic traffic mix

Firewall, NATDPI/AVCLB

DEVNET-1120 8


What Problem is Being Solved?

• Traffic generators for realistic traffic are• Expensive ~$100-500K

• Not scalable for high rates

• Not flexible

• Implication • Limited and late testing

• Different benchmarks and test methodologies

• Real life bottlenecks and design issues

DEVNET-1120 9


What is TRex?

• Linux user-space application uses DPDK library

• Stateless: Stream based uses Scapy

• Stateful: flow basedGenerates, manipulates and amplifies based on templates of real, captured flows (W/O TCP stack)

• High performance: up to 200 Gb/sec

• Low cost: Standard server hardware

• Flexible and Open Software

• Virtualization

• Easy installation and deployment

DEVNET-1120 10

Stateless


Stateless High level functionality

• High scale – ~10M-35MPPS/core

• Profile can support multiple streams, scalable to 20K parallel streams

• Interactive support – GUI/TUI

• Statistic per port/ stream (e.g. latency/ Jitter)

• Python automation support

• Multi-user support

• Capture to Wireshark

• Scalable services using plugins (e.g. DHCP,IPv6)

DEVNET-1120 13


Traffic Profile Example

DEVNET-1120 14


Control plane High level

DEVNET-1120 15


One stream with two directions

DEVNET-1120 16


Python Automation example

DEVNET-1120 17


link

Performance XL710 MPPS/Core

DEVNET-1120 18

trex-tgn.cisco.com/trex/doc/trex_stateless_bench.html

Stateful


L7 inspection features

• Networks include complex L4-7 features, such as

• Load Balancer, DPI/AVC, Firewall, NAT

• Requires testing with stateful and realistic traffic mix

Firewall, NATDPI/AVCLB

DEVNET-1120 21


Stateful Traffic Generation Model

DEVNET-1120 22


High level software architecture

CP

U/S

ha

re m

em

ory

Sh

are

m

em

ory

Inte

l N

IC

DP -thread 1

CP – thread 0

DP -thread 2

Rx – thread 5

IF0Client

IF1Server

IF2Client

IF3Server

DP -thread 3

DP -thread 4

• DPDK/Multi-Threaded

• Scales linearly

• ~5MPPS/~20 Gb/sec per core

• Supports 1/10/40 Gb Intel NICs

• Flow-based

• Fast event scheduler

• Generates flow templates

• Can support 1K templates

• Scales up to 100K clients,1M servers

• Flexible

• Client/server generation models• Measures jitter/latency/flow order

• NAT translation/IPv6/Tunnels

• User Interface

• Python API

• Benchmark automation

• GUI

DEVNET-1120 23


Flow Generation

• Example of one flow with four packets

24DEVNET-1120

Total PPS =

𝑘=0

𝑛

𝐶𝑃𝑆𝑘 × 𝑓𝑙𝑜𝑤_𝑝𝑘𝑡𝑠𝑘

Total CPS =

𝑘=0

𝑛

𝐶𝑃𝑆𝑘

Concurrent flows

=

𝑘=0

𝑛

𝐶𝑃𝑆𝑘 × 𝑓𝑙𝑜𝑤_𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛𝑘

FIF

FIF

FIF


DNS simple profile example

DEVNET-1120 25


DNS output

DEVNET-1120 26


HTTP & DNS

DEVNET-1120 27


HTTP & DNS

DEVNET-1120 28


Enterprise traffic profile

• Includes protocols with Control/Data

dependency • SIP

• RTSP

DEVNET-1120 29


EMIX - YAML

DEVNET-1120 30


EMIX

DEVNET-1120 31


NAT/FW learning translation/randomization

1-Tx-SYN

1-Rx-SYN (learn translation per flow)

2-Tx-SYN-ACK

2-Rx-SYN-ACK (learn)

3-Tx-ACK

3-Rx-ACK (learn seq number randomization)

DEVNET-1120 32


Client Clustering• To simulate a big network

• Scale of the number of clients

• For Controller testing

DEVNET-1120 33


Performance MPPS/Core @ 8M flows

DEVNET-1120 34


Gbps extrapolation, average packet size 600B

DEVNET-1120 35

Advanced Stateful


User space TCP stack – Why ?

DEVNET-1120 37


User space TCP stack – Why ?

DEVNET-1120 38


TRex ASTF features

• High scale

• TCP is the core component

• Can be tuned MSS/initwnd/delay-ack

• TCP is based on BSD with acceleration

• Interactive

• Accurate latency measurement – usec

• Simulation of latency/jitter/drop in high rate

• OpenSSL integration

• L7 emulation layer

• Emulate application using “micro-instructions”

• Field engine

DEVNET-1120 39


TRex ASTF features status

• High scale

• TCP is the core component

• Can be tuned MSS/initwnd/delay-ack

• TCP is based on BSD with acceleration

• Interactive

• Accurate latency measurement – usec

• Simulation of latency/jitter/drop in high rate

• OpenSSL integration

• L7 emulation layer

• Emulate application using “micro-instructions”

• Field engine

DEVNET-1120 40


L7 Emulation layer Client side

Server side

DEVNET-1120 41


HTTP simple profile

DEVNET-1120 42


Client side pseudo code

DEVNET-1120 43


Server side pseudo code

DEVNET-1120 44


Profile with two template

DEVNET-1120 45


Different IP pool for each template

DEVNET-1120 46


Statistic

DEVNET-1120 47


Client/Server only

DEVNET-1120 48


Under the hood

DEVNET-1120 49


TCP stack – Flow Scale -TX

10M flows

320GByte

320M mbuf

=10GB

0.01GB

DEVNET-1120 50


TCP stack – Flow Scale issue - RX

10M flows

320GByte

3-50GB for 1%

drop rate

0.01GB

DEVNET-1120 51


TCP stack – Delay/Jitter/Drop simulation

100MPPS *100msec

=10MPPS in Queue

100MPPS *100msec

=10*16MPPS=0.16GB

DEVNET-1120 52


TRex vs NGINX

DEVNET-1120 53


Performance setup #2

DEVNET-1120 54


Performance numbers

https://trex-tgn.cisco.com/trex/doc/trex_astf_vs_nginx.html

x80 faster

x2000 less memory

DEVNET-1120 55

wrapup


TRex models of operation

• L7, DUT terminate TCP/SSL, flow based

• DUT inspect L7. does not change TCP. Flow based

• DUT L2/L3 Switch , packet based

DEVNET-1120 57


Cisco Spark

Questions? Use Cisco Spark to communicate with the speaker after the session

1. Find this session in the Cisco Live Mobile App

2. Click “Join the Discussion”

3. Install Spark or go directly to the space

4. Enter messages/questions in the space

How

cs.co/ciscolivebot#DEVNET-1120


• Please complete your Online Session Evaluations after each session

• Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt

• All surveys can be completed via the Cisco Live Mobile App or the Communication Stations

Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at www.ciscolive.com/global/on-demand-library/.

Complete Your Online Session Evaluation

http://ciscolive.com/Online

http://www.ciscolive.com/global/on-demand-library/


Continue Your Education

• Stateless manual

• TRex documents Index

• GitHub

• DevNet zone

DEVNET-1120 61

http://trex-tgn.cisco.com/trex/doc/trex_stateless.html

http://trex-tgn.cisco.com/trex/doc/

https://github.com/cisco-system-traffic-generator/trex-core

https://developer.cisco.com/site/trex/


Continue Your Education

• Demos in the Cisco campus

• Walk-in Self-Paced Labs

• Tech Circle

• Meet the Engineer 1:1 meetings

• Related sessions

62DEVNET-1120

Thank you

Backup

Stateless –Traffic profile


Simple Interleaving streams -profile


Multi burst

69DEVNET-1120


Multi burst profile


Field Engine

71DEVNET-1120

• Flexible engine for changing packet fields

• Examples

– Change TOS 1-20

– Range of client IPv4/IPv6 e.g. 10.0.0.1-10.0.0.254

– Random packet size 64-9k

– Random destination IPv4/IPv6

– Support any tunnel even not valid packet like

QinQ/GRE/MPLS/Ipv6/UDP/Ipv4/HTTP


Field Engine, Syn attack


Pcap File Conversion to Streams

DEVNET-1120 73


Per Stream Statistics

DEVNET-1120 74


Per Stream Statistics - TUI

DEVNET-1120 75


• Base on per stream stats hardware assist

• Software measures latency and jitter resolution is

~usec (not nsec)

Per Stream Latency/Jitter

76DEVNET-1120


Per Stream Statistics - TUI

Stateless –Service mode


Service Mode

DEVNET-1120 79


Service Mode

• Ping / ARP/DHCP Client

• IPv6 ND /Multicast setup

• Scan6 support

• Traffic Capturing

• Functional test

• New protocols multiplex framework

80DEVNET-1120


Service Mode Capture Monitoring – Wireshark Pipe

81DEVNET-1120

trex realistic traffic generator · what problem is being solved? ... different ip pool for each...

Documents