packet mangling for fun & profit a brief intro to netfilter, user mode linux, and the linux...
Post on 20-Dec-2015
230 views
TRANSCRIPT
Packet Mangling for Fun & Profit
A Brief Intro to Netfilter, User Mode Linux, and the Linux
TCP/IP Stack
Topics SplitTCP Summary (Context) Netfilter Introduction One buffer to rule them all:
sk_buffs User Mode Linux Current Status
SplitTCP In a Nutshell Work started by Michalis, Srikanth, and
Swastik. The idea is to add transport layer proxies to
long (in terms of hop count) TCP connections.
If a link fails (due to mobility, etc.) the packet can be re-transmitted by the proxy closest to the failure.
A “demo-quality” implementation was done which made many simplifying assumptions.
So What? My current task is to take that
demo implementation and turn it into a general implementation.
This presentation discusses the lessons learned up to this point in that process.
Design Goals Any and all modifications made to
the behavior of TCP should be backwards compatible. I.e. nodes should interoperate
regardless of weather or not they’re “split tcp enabled”.)
If possible, no changes should be made to the kernel proper. It’s just better for everyone that way…
Enter Netfilter Fortunately, Linux has a plugin API,
called Netfilter, which allows kernel modules to hook into strategic spots in the networking stack.
Unfortunately, in the grand tradition of open source, documentation takes a back seat to implementation – so there’s a non-trivial learning curve.
Sideline: Plugin APIs An example of the Delegation
design pattern (GoF) – a work unit is passed through a series of cooperating steps to produce the final result.
A well-designed plugin architecture can facilitate otherwise impossible tasks and simply possible ones.
Sideline: Kernel Modules Most of the linux kernel can be
built as a module – a bit of code that’s loaded into (and unloaded from) the kernel dynamically.
Device drivers, file systems, even networking protocols (IPX, Appletalk) are candidates for modularization.
Writing a Module Just define a few preprocessor
macros (__KERNEL__, MODULE) Include a few header files
(linux/module.h, linux/version.h, linux/config.h)
And honor a small interface (module_init(), module_exit())
You’ll have a no-op module.
Netfilter Overview Initial design by Paul “Rusty” Russell Netfilter is a series of callback
functions within the network stack. The API is non-portable and appeared in linux 2.3.x
Each protocol has it’s own set of callback points. We care about IPv4.
Netfilter Concepts A module expresses interest in being
invoked at an arbitrary subset of the available callback points – specifying the function and the (global) priority in which it should be called.
That function is passed (among other things) a pointer to a pointer to a packet buffer ( sk_buff ** ).
Return Values The netfilter function has five
possible return values: NF_ACCEPT: continue callback chain NF_DROP: drop the packet and stop the
chain NF_STOLEN: stop the chain NF_QUEUE: send the packet to
userspace NF_REPEAT: call the hook again
Netfilter Hooks in IPv
Routing Engine
Local Sockets
1In Out
2
34
5
Say that Again? 1: NF_IP_PREROUTING
any received packet which checksums OK. 2: NF_IP_LOCAL_IN
packets destined for local sockets 3: NF_IP_FORWARD
foreign packets being forwarded 4: NF_IP_POST_ROUTING
any outbound packet 5: NF_IP_LOCAL_OUT
packets originating from local sockets
Routing Engine
Local Sockets
1In Out
2
4
5
An sk_what?? Linux uses a structure called an
sk_buff to store packet data internally.
It contains a handful of pointers to other structures as well as a packet data region.
Data
Poin
ters
Head
room
Tailr
oom
Sk_buff’s and you The data area is like a stack, only
you can insert at the head and the tail (deque?). The kernel provides a handful of
helper functions to manage sk_buff’s and their data areas.
The various header pointers point into the data area – which can be thought of as a serialized packet.
Why do We Care? An sk_buff is built as a packet travels
down the stack – each layer (TCP, IP, Ethernet) adds their own special sauce.
This means that each header is “squashed” in against the next – so while modifying existing data is relatively easy, adding new header data is a bit trickier.
Don’t leave me in suspense… Basically, you make a copy of the
sk_buff, and ask it to “grow” a bit during the copy.
Once you have the copy – you “slide” the IP and TCP headers backwards a bit, insert the new option bytes, and re-checksum the packet.
A tale of “n” checksum’s Sounds easy, right? Remember
that these sk_buffs are built one layer at a time?
There is no nice friendly function which will take a TCP sk_buff and compute all the needed checksums.
Funny thing about checksums – almost isn’t good enough.
Ok, insmod and <BOOM> Remember developing on a system
without memory protection and having to reboot ?
Kernel modules execute in kernel space – so no one’s watching your back. If you goof, it’s time to reboot.
User Mode Linux to the rescue
User Mode Linux (UML) A kernel patch that allows running
the linux kernel as a user-mode process on a linux machine.
If you crash the user-mode kernel, you just restart the process, no reboot required.
Where do I sign up? Setup is (in principal) fairly easy – only it
turns out that the standard distribution doesn’t have netfilter enabled.
So I re-built with the appropriate options and placed the binaries in ~swift/user-mode-linux/bin
There’s a README there – some support executables must be installed as root on your workstation.
That’s It? Not quite – you also need a file system
to boot this kernel off. Good News: you can download a file
system image – you don’t have to make one. Bad News: it’s really big. (up to 700MB)
Good News: you can share one among many people Bad News: you have to read the HOWTO.
Building Modules Under UML Building a kernel module is the
same under UML as under “KML” – it is important that you build it against the source used to build the target kernel.
For “our” UML build – that source is in ~swift/user-mode-linux/src
Current Status A skeleton of a splittcp module
(tcpproxy.c) exists. It can inspect locally generated packets
and add our newly defined PROXY option, rechecksum the packet, and send it on it’s way.
It can inspect arriving packets, check for the option, and decide if that packet should be proxied.
So What’s Left? It doesn’t (yet) actually proxy the
packet. Nor does it send an
acknowledgement of receipt to the upstream proxy.
There are also issues around ICMP error messages, and what should be done about them.
Conclusion Once the code is “alpha” quality,
I’ll commit it to the swift CVS repository for your collective viewing pleasure.
Until then, if you have questions or suggestions, see me.