bof - what can bpf do for you? · bof - what can bpf do for you? brenden blanco ... developer...
TRANSCRIPT
![Page 1: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/1.jpg)
BoF - What Can BPF Do For You?
Brenden BlancoAug. 22, 2016
![Page 2: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/2.jpg)
Agenda
2
A bit of history and project motivation
An introduction to eBPF in the Linux kernel
An introduction to the BCC toolkit
Show how Clang/LLVM is integrated into BCC
Demo how to use IO Visor+XDP for DDoS mitigation
Demo how to use IO Visor to debug a live system
Q+A
www.iovisor.org
![Page 3: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/3.jpg)
Thank You to Sponsoring Members
3www.iovisor.org
![Page 4: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/4.jpg)
What we want
Started with building networking applications for SDN
An SDK to extend low-level infrastructure
But…
Don’t want to become a kernel developer
4www.iovisor.org
![Page 5: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/5.jpg)
Compare to a server app framework (e.g. Node.js)
Recognize that writing multithreaded apps is hard
Syntax that mirrors thought process, not the CPU arch (events vs threads)
Don’t sacrifice performance (v8 jit)
Make it easy to get code from the devs to deployment (npm)
Foster a community via sharing of code
5www.iovisor.org
![Page 6: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/6.jpg)
What do you need to write infrastructure apps
High performance access to data
Reliability...it must never crash
In-place upgrades
Debug tools
A programming language abstraction
6www.iovisor.org
![Page 7: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/7.jpg)
But there are restrictions
No custom kernels
No custom kernel modules
No kernels with debug symbols
No reboots
(some of these are nice-to-haves)
7www.iovisor.org
![Page 8: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/8.jpg)
8
IO Visor Project, What is in it?
• A set of development tools, IO Visor Dev Tools
• A set of IO Visor Tools for management and operations of the IO Visor Engine
• A set of Applications, Tools and open IO Modules build on top of the IO Visor framework
• A set of possible use cases & applications like Networking, Security, Tracing & others
www.iovisor.org
![Page 9: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/9.jpg)
Hello, World! Demo
#!/usr/bin/pythonimport bccb = bcc.BPF(text=”””int kprobe__sys_clone(void *ctx) { bpf_trace_printk("Hello, World!\\n"); return 0;}”””)b.trace_print()
9www.iovisor.org
![Page 10: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/10.jpg)
BPF
10www.iovisor.org
![Page 11: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/11.jpg)
In a very simplified way:A safe, runtime way to extend Linux kernel capabilitiesFunctions, Maps, Attachment Points, Syscall
11
f1()
fnext
()f2()
fnext
()f3()
fnext
()
1) Set of Tables
2) Set of Functions
??
3) A way to hook it to Kernel events
Kernel internals
events
4) A way to interact with the kernel components from user space
Kernel
User Space
www.iovisor.org
What are BPF Programs?
![Page 12: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/12.jpg)
More on BPF Programs
Berkeley Packet Filters around since 1990, extensions started Linux 3.18
Well, not really a program (no pid)...an event handler
A small piece of code, executed when an event occurs
In-kernel virtual machine executes the code
Assembly instruction set
See ‘man 2 bpf’ for details
12www.iovisor.org
![Page 13: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/13.jpg)
The eBPF Instruction Set
Instructions▪ 10x 64bit registers▪ 512B stack▪ 1-8B load/store▪ conditional jump▪ arithmetic▪ function call
13
Helper functions▪ forward/clone/drop packet▪ load/store packet data▪ load/store packet metadata▪ checksum (incremental)▪ push/pop vlan▪ access kernel mem (kprobes)
Data structures▪ lookup/update/delete
▪ in-kernel or from userspace▪ hash, array, ...
www.iovisor.org
![Page 14: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/14.jpg)
BPF Kernel Hook Points
A program can be attached to:
kprobes or uprobes
socket filters (original tcpdump use case)
seccomp
tc filters or actions, either ingress or egress
XDP (NEW)
14www.iovisor.org
![Page 15: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/15.jpg)
BPF Verifier
A program is declared with a type (kprobe, filter, etc.)
Only allows permitted helper functions
Kernel parses BPF instructions into a DAG
Disallows: back edges, unreachable blocks, illegal insns, finite execution
No memory accesses from off-stack, or from unverified source
Program ok? => JIT compile to native instructions (x86_64, arm64, s390)
15www.iovisor.org
![Page 16: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/16.jpg)
Developer Workflow
16
eBPF program written in C
Translated into eBPF instructions (LLVM)
Loaded in kernel
Hooked at different levels of Linux Networking Stack
(as an example)
HW/veth/tap
TAP/Raw
driver
netif_receive_skb()
TC / traffic control
Bridge hook
IP / routing
Socket (TCP/UDP)
BPF
BPF
BPF
www.iovisor.org
![Page 17: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/17.jpg)
Using Clang and LLVM in BCC
17www.iovisor.org
![Page 18: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/18.jpg)
How BCC uses Clang
18
import bccb = bcc.BPF(“hello.c”)
www.iovisor.org
clang::Rewriter => hello.c’
clang -c hello.c’ -o <memory>
clang -c hello.c -o <memory> llvm MCJIT => hello.o
b.load_func(...)
![Page 19: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/19.jpg)
How BCC uses Clang
19
import bccbcc.BPF(“hello.c”)
BPFModule
clang pass 2llvm::Module => IR
llvm PassManagerIR => -O3 => optimized IR
clang pass 1- extract key/leaf types- fixup tracing fn args- fixup packet load/store- bpf_map_create() => fd- fixup map accesses w/ fd- share externed maps b/w programs
clang::Rewriter
llvm MCJITIR => BPF bytecode
bpf_prog_load()
www.iovisor.org
![Page 20: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/20.jpg)
Rewrite Sample #1
#include <uapi/linux/ptrace.h>int do_request(struct pt_regs *ctx, int req) { bpf_trace_printk("req ptr: 0x%x\n", req); return 0;}
20www.iovisor.org
#include <uapi/linux/ptrace.h>int do_request(struct pt_regs *ctx, int req) { ({ char _fmt[] = "req ptr: 0x%x\n"; bpf_trace_printk_(_fmt, sizeof_(fmt), ((u64)ctx->di)); }); return 0;}
![Page 21: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/21.jpg)
Rewrite Sample #2
#include <linux/sched.h>
#include <uapi/linux/ptrace.h>
int count_sched(struct pt_regs *ctx,
struct task_struct *prev) {
pid_t p = prev->pid; return p != -1;
}
21www.iovisor.org
![Page 22: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/22.jpg)
Rewrite Sample #2
#include <linux/sched.h>
#include <uapi/linux/ptrace.h>
int count_sched(struct pt_regs *ctx,
struct task_struct *prev) {
pid_t p = ({ pid_t _val; memset(&_val, 0, sizeof(_val)); bpf_probe_read(&_val, sizeof(_val), ((u64)ctx->di) + offsetof(struct task_struct, pid)); _val; }); return p != -1;
}
22www.iovisor.org
![Page 23: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/23.jpg)
Rewrite Sample #3
#include <bcc/proto.h>
struct IPKey { u32 dip; u32 sip; };
BPF_TABLE("hash", struct IPKey, int, mytable, 1024);
int recv_packet(struct __sk_buff *skb) {
struct IPKey key;
u8 *cursor = 0;
struct ethernet_t *ethernet = cursor_advance(cursor, sizeof(*ethernet));
struct ip_t *ip = cursor_advance(cursor, sizeof(*ip));
key.dip = ip->dst;
key.sip = ip->src;
int *leaf = mytable.lookup(&key);
if (leaf)
*(leaf)++;
return 0;
}
23www.iovisor.org
![Page 24: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/24.jpg)
Rewrite Sample #3
#include <bcc/proto.h>
struct IPKey { u32 dip; u32 sip; };
BPF_TABLE("hash", struct IPKey, int, mytable, 1024);
int recv_packet(struct __sk_buff *skb) {
struct IPKey key;
u8 *cursor = 0;
struct ethernet_t *ethernet = cursor_advance(cursor, sizeof(*ethernet));
struct ip_t *ip = cursor_advance(cursor, sizeof(*ip));
key.dip = bpf_dext_pkt(skb, (u64)ip+16, 0, 32);
key.sip = bpf_dext_pkt(skb, (u64)ip+12, 0, 32);
int *leaf = bpf_map_lookup_elem((void *)bpf_pseudo_fd(1, 3), &key);
if (leaf)
*(leaf)++;
return 0;
}
24www.iovisor.org
![Page 25: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/25.jpg)
Using BCC for Tracing
25www.iovisor.org
![Page 26: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/26.jpg)
Tracing Demo
https://github.com/iovisor/bcc
http://www.brendangregg.com/blog
26www.iovisor.org
![Page 27: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/27.jpg)
XDP for Networking
27www.iovisor.org
![Page 28: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/28.jpg)
28www.iovisor.org
What is XDP?
▪ A programmable, high performance, specialized application, packet processor in the networking data path
▪ Bare metal packet processing at lowest point in the SW stack▪ Use cases include
▪ Pre-stack processing like filtering to do DOS mitigation▪ Forwarding and load balancing▪ Batching techniques▪ Flow sampling, monitoring
![Page 29: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/29.jpg)
29www.iovisor.org
XDP Properties
▪ XDP is designed for high performance. It uses known techniques and applies selective constraints to achieve performance goals
▪ XDP is also designed for programmability. New functionality can be implemented on the fly without needing kernel modification
▪ XDP is not kernel bypass. It is an integrated fast path in the kernel stack▪ XDP does not replace the TCP/IP stack. It augments the stack and works in
concert▪ XDP does not require any specialized hardware. Less-is-more principle for
networking hardware
![Page 30: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/30.jpg)
eXpress Data Path (XDP)
30www.iovisor.org
![Page 31: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/31.jpg)
XDP Benchmark Setup
31www.iovisor.org
Receiver
Xeon E5-1630 @3.70GHz
Mellanox MT27520 ConnectX-3 Pro
Sender
Xeon E5645 @2.40GHz
Mellanox MT27520 ConnectX-3 Pro
40G
![Page 32: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/32.jpg)
Thank You!
32www.iovisor.org
![Page 33: BoF - What Can BPF Do For You? · BoF - What Can BPF Do For You? Brenden Blanco ... Developer Workflow 16 ... BPF_TABLE("hash", struct IPKey, int, mytable, 1024);](https://reader034.vdocuments.site/reader034/viewer/2022051800/5ac6d7417f8b9acb7c8b73a5/html5/thumbnails/33.jpg)
Learn More and Contribute
https://iovisor.org
https://github.com/iovisor
#iovisor irc.oftc.net
@IOVisor
33www.iovisor.org