listifyed (udp) grovger.kernel.org/netconf2019_files/udp_gro.pdf · listifyed (udp) gro generic...
TRANSCRIPT
Listifyed (UDP) GRO
Listifyed (UDP) GRO
Steffen Klassert
secunet Security Networks AG
Dresden
Linux Netconf, Boston, June, 2019
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Generic Receive Offload (GRO)
I Software packet aggregation on Layer 2
I Merges packets of a flow into one big packet (up to 64 KB)
I Reduces the number of network stack traversals
I Good performance if a local socket can handle GRO
I Good performance on forwarding if the TX NIC can do LSO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Generic Receive Offload (GRO)
I Software packet aggregation on Layer 2
I Merges packets of a flow into one big packet (up to 64 KB)
I Reduces the number of network stack traversals
I Good performance if a local socket can handle GRO
I Good performance on forwarding if the TX NIC can do LSO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Generic Receive Offload (GRO)
I Software packet aggregation on Layer 2
I Merges packets of a flow into one big packet (up to 64 KB)
I Reduces the number of network stack traversals
I Good performance if a local socket can handle GRO
I Good performance on forwarding if the TX NIC can do LSO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Generic Receive Offload (GRO)
I Software packet aggregation on Layer 2
I Merges packets of a flow into one big packet (up to 64 KB)
I Reduces the number of network stack traversals
I Good performance if a local socket can handle GRO
I Good performance on forwarding if the TX NIC can do LSO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Generic Receive Offload (GRO)
I Software packet aggregation on Layer 2
I Merges packets of a flow into one big packet (up to 64 KB)
I Reduces the number of network stack traversals
I Good performance if a local socket can handle GRO
I Good performance on forwarding if the TX NIC can do LSO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Generic Receive Offload (GRO)
I Software packet aggregation on Layer 2
I Merges packets of a flow into one big packet (up to 64 KB)
I Reduces the number of network stack traversals
I Good performance if a local socket can handle GRO
I Good performance on forwarding if the TX NIC can do LSO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Generic Receive Offload (GRO)
I Software packet aggregation on Layer 2
I Merges packets of a flow into one big packet (up to 64 KB)
I Reduces the number of network stack traversals
I Good performance if a local socket can handle GRO
I Good performance on forwarding if the TX NIC can do LSO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Problematic GRO usecases
I Does not perform well if GSO is neededI GRO and GSO add overhead
I calculating checksumsI modifies headersI touches packet data
I UDP needs needs software GSO if NIC can’t do LSO
I IPsec needs software GSO if NIC can’t offload ESP
I Is there a better way to do GRO if software GSO is needed?
I Yes: Listifyed GRO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Problematic GRO usecases
I Does not perform well if GSO is neededI GRO and GSO add overhead
I calculating checksumsI modifies headersI touches packet data
I UDP needs needs software GSO if NIC can’t do LSO
I IPsec needs software GSO if NIC can’t offload ESP
I Is there a better way to do GRO if software GSO is needed?
I Yes: Listifyed GRO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Problematic GRO usecases
I Does not perform well if GSO is neededI GRO and GSO add overhead
I calculating checksumsI modifies headersI touches packet data
I UDP needs needs software GSO if NIC can’t do LSO
I IPsec needs software GSO if NIC can’t offload ESP
I Is there a better way to do GRO if software GSO is needed?
I Yes: Listifyed GRO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Problematic GRO usecases
I Does not perform well if GSO is neededI GRO and GSO add overhead
I calculating checksumsI modifies headersI touches packet data
I UDP needs needs software GSO if NIC can’t do LSO
I IPsec needs software GSO if NIC can’t offload ESP
I Is there a better way to do GRO if software GSO is needed?
I Yes: Listifyed GRO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Problematic GRO usecases
I Does not perform well if GSO is neededI GRO and GSO add overhead
I calculating checksumsI modifies headersI touches packet data
I UDP needs needs software GSO if NIC can’t do LSO
I IPsec needs software GSO if NIC can’t offload ESP
I Is there a better way to do GRO if software GSO is needed?
I Yes: Listifyed GRO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Problematic GRO usecases
I Does not perform well if GSO is neededI GRO and GSO add overhead
I calculating checksumsI modifies headersI touches packet data
I UDP needs needs software GSO if NIC can’t do LSO
I IPsec needs software GSO if NIC can’t offload ESP
I Is there a better way to do GRO if software GSO is needed?
I Yes: Listifyed GRO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Problematic GRO usecases
I Does not perform well if GSO is neededI GRO and GSO add overhead
I calculating checksumsI modifies headersI touches packet data
I UDP needs needs software GSO if NIC can’t do LSO
I IPsec needs software GSO if NIC can’t offload ESP
I Is there a better way to do GRO if software GSO is needed?
I Yes: Listifyed GRO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Problematic GRO usecases
I Does not perform well if GSO is neededI GRO and GSO add overhead
I calculating checksumsI modifies headersI touches packet data
I UDP needs needs software GSO if NIC can’t do LSO
I IPsec needs software GSO if NIC can’t offload ESP
I Is there a better way to do GRO if software GSO is needed?
I Yes: Listifyed GRO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Problematic GRO usecases
I Does not perform well if GSO is neededI GRO and GSO add overhead
I calculating checksumsI modifies headersI touches packet data
I UDP needs needs software GSO if NIC can’t do LSO
I IPsec needs software GSO if NIC can’t offload ESP
I Is there a better way to do GRO if software GSO is needed?
I Yes: Listifyed GRO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Problematic GRO usecases
I Does not perform well if GSO is neededI GRO and GSO add overhead
I calculating checksumsI modifies headersI touches packet data
I UDP needs needs software GSO if NIC can’t do LSO
I IPsec needs software GSO if NIC can’t offload ESP
I Is there a better way to do GRO if software GSO is needed?
I Yes: Listifyed GRO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Listifyed GRO
I Does not merge packets into a big packet
I Builds a linked list of packets
I Linked packets travel together though the stack
I GSO just needs to unlink the packets
I Listifyed GRO/GSO does not need to touch packet data
I Listifyed GRO/GSO performs better than standard GRO/GSO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Listifyed GRO
I Does not merge packets into a big packet
I Builds a linked list of packets
I Linked packets travel together though the stack
I GSO just needs to unlink the packets
I Listifyed GRO/GSO does not need to touch packet data
I Listifyed GRO/GSO performs better than standard GRO/GSO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Listifyed GRO
I Does not merge packets into a big packet
I Builds a linked list of packets
I Linked packets travel together though the stack
I GSO just needs to unlink the packets
I Listifyed GRO/GSO does not need to touch packet data
I Listifyed GRO/GSO performs better than standard GRO/GSO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Listifyed GRO
I Does not merge packets into a big packet
I Builds a linked list of packets
I Linked packets travel together though the stack
I GSO just needs to unlink the packets
I Listifyed GRO/GSO does not need to touch packet data
I Listifyed GRO/GSO performs better than standard GRO/GSO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Listifyed GRO
I Does not merge packets into a big packet
I Builds a linked list of packets
I Linked packets travel together though the stack
I GSO just needs to unlink the packets
I Listifyed GRO/GSO does not need to touch packet data
I Listifyed GRO/GSO performs better than standard GRO/GSO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Listifyed GRO
I Does not merge packets into a big packet
I Builds a linked list of packets
I Linked packets travel together though the stack
I GSO just needs to unlink the packets
I Listifyed GRO/GSO does not need to touch packet data
I Listifyed GRO/GSO performs better than standard GRO/GSO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Listifyed GRO
I Does not merge packets into a big packet
I Builds a linked list of packets
I Linked packets travel together though the stack
I GSO just needs to unlink the packets
I Listifyed GRO/GSO does not need to touch packet data
I Listifyed GRO/GSO performs better than standard GRO/GSO
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Max UDP Throughput in Mbps (single flow / 30 sec)
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Avg Latency at Max Throughput
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Min Latency at Max Throughput
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Max Latency at Max Throughput
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Status of listifyed GRO
I Implemented for UDP (RFC state)I TCP forwarding could benefit too if software GSO is needed
I Should listifyed GRO support TCP too?
I Problem: How to decide between standard and listifyed GRO?I Global knob to enable/disableI Do a route lookup at GRO layer
I Gives a good guess on output NIC and xfrmI But: Could be rerouted by netfilter etc.
I Add some TC/NF hook before GRO that can add route infosI NF flowtable?I eBPF programm?
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Status of listifyed GRO
I Implemented for UDP (RFC state)I TCP forwarding could benefit too if software GSO is needed
I Should listifyed GRO support TCP too?
I Problem: How to decide between standard and listifyed GRO?I Global knob to enable/disableI Do a route lookup at GRO layer
I Gives a good guess on output NIC and xfrmI But: Could be rerouted by netfilter etc.
I Add some TC/NF hook before GRO that can add route infosI NF flowtable?I eBPF programm?
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Status of listifyed GRO
I Implemented for UDP (RFC state)I TCP forwarding could benefit too if software GSO is needed
I Should listifyed GRO support TCP too?
I Problem: How to decide between standard and listifyed GRO?I Global knob to enable/disableI Do a route lookup at GRO layer
I Gives a good guess on output NIC and xfrmI But: Could be rerouted by netfilter etc.
I Add some TC/NF hook before GRO that can add route infosI NF flowtable?I eBPF programm?
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Status of listifyed GRO
I Implemented for UDP (RFC state)I TCP forwarding could benefit too if software GSO is needed
I Should listifyed GRO support TCP too?
I Problem: How to decide between standard and listifyed GRO?I Global knob to enable/disableI Do a route lookup at GRO layer
I Gives a good guess on output NIC and xfrmI But: Could be rerouted by netfilter etc.
I Add some TC/NF hook before GRO that can add route infosI NF flowtable?I eBPF programm?
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Status of listifyed GRO
I Implemented for UDP (RFC state)I TCP forwarding could benefit too if software GSO is needed
I Should listifyed GRO support TCP too?
I Problem: How to decide between standard and listifyed GRO?I Global knob to enable/disableI Do a route lookup at GRO layer
I Gives a good guess on output NIC and xfrmI But: Could be rerouted by netfilter etc.
I Add some TC/NF hook before GRO that can add route infosI NF flowtable?I eBPF programm?
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Status of listifyed GRO
I Implemented for UDP (RFC state)I TCP forwarding could benefit too if software GSO is needed
I Should listifyed GRO support TCP too?
I Problem: How to decide between standard and listifyed GRO?I Global knob to enable/disableI Do a route lookup at GRO layer
I Gives a good guess on output NIC and xfrmI But: Could be rerouted by netfilter etc.
I Add some TC/NF hook before GRO that can add route infosI NF flowtable?I eBPF programm?
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Status of listifyed GRO
I Implemented for UDP (RFC state)I TCP forwarding could benefit too if software GSO is needed
I Should listifyed GRO support TCP too?
I Problem: How to decide between standard and listifyed GRO?I Global knob to enable/disableI Do a route lookup at GRO layer
I Gives a good guess on output NIC and xfrmI But: Could be rerouted by netfilter etc.
I Add some TC/NF hook before GRO that can add route infosI NF flowtable?I eBPF programm?
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Status of listifyed GRO
I Implemented for UDP (RFC state)I TCP forwarding could benefit too if software GSO is needed
I Should listifyed GRO support TCP too?
I Problem: How to decide between standard and listifyed GRO?I Global knob to enable/disableI Do a route lookup at GRO layer
I Gives a good guess on output NIC and xfrmI But: Could be rerouted by netfilter etc.
I Add some TC/NF hook before GRO that can add route infosI NF flowtable?I eBPF programm?
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Status of listifyed GRO
I Implemented for UDP (RFC state)I TCP forwarding could benefit too if software GSO is needed
I Should listifyed GRO support TCP too?
I Problem: How to decide between standard and listifyed GRO?I Global knob to enable/disableI Do a route lookup at GRO layer
I Gives a good guess on output NIC and xfrmI But: Could be rerouted by netfilter etc.
I Add some TC/NF hook before GRO that can add route infosI NF flowtable?I eBPF programm?
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Status of listifyed GRO
I Implemented for UDP (RFC state)I TCP forwarding could benefit too if software GSO is needed
I Should listifyed GRO support TCP too?
I Problem: How to decide between standard and listifyed GRO?I Global knob to enable/disableI Do a route lookup at GRO layer
I Gives a good guess on output NIC and xfrmI But: Could be rerouted by netfilter etc.
I Add some TC/NF hook before GRO that can add route infosI NF flowtable?I eBPF programm?
Listifyed (UDP) GRO
Generic Receive Offload (GRO)
Status of listifyed GRO
I Implemented for UDP (RFC state)I TCP forwarding could benefit too if software GSO is needed
I Should listifyed GRO support TCP too?
I Problem: How to decide between standard and listifyed GRO?I Global knob to enable/disableI Do a route lookup at GRO layer
I Gives a good guess on output NIC and xfrmI But: Could be rerouted by netfilter etc.
I Add some TC/NF hook before GRO that can add route infosI NF flowtable?I eBPF programm?