1

SDN: ExtensionsMiddleboxes

Ack: Vyas Sekar, Aaron Gember, Felipe Huici, Zafar Qazi

Need for Network Evolution

2

New devices

New applications

Evolving threats Policy

constraintsPerformance, Security, Compliance

3

Type of appliance NumberFirewalls 166NIDS 127Media gateways 110Load balancers 67Proxies 66VPN gateways 45WAN Optimizers 44Voice gateways 11Total Middleboxes 636Total routers ~900

Network Evolution today: Middleboxes!

Data from a large enterprise: >80K users across tens of sites

Just network security$10 billion

How many middleboxes do you deploy?

Typically on par with # routers and switches.

How do administrators spend their time?

Misconfig. Overload Physical/Electrical

Firewalls 67.3% 16.3% 16.3%Proxies 63.2% 15.7% 21.1%

IDS 54.45% 11.4% 34%

Most administrators spent 1-5 hrs/week dealing with failures; 9% spent 6-10 hrs/week.

6

Specialized boxes

Narrowinterfaces

“Point”solutions!

Increases capital expenses & sprawl Increases operating expensesLimits extensibility and flexibility

Management ManagementManagement

Key “pain points”

7

Controller PlatformSwitch API (OpenFlow)

Controller

Switches

App

Runtime

SDN Stack

Control Flow, Data Structures, etc.

Applications

8

Outline

• Why middleboxes?

• SIMPLE

• OpenMB

• Slick

9

Can SDN simplify middlebox management?Centralized Controller

“Flow” FwdAction… …

“Flow” FwdAction… …

OpenFlow

Proxy IDS

Necessity + Opportunity: Incorporate functions markets views as important

Scope: Enforce middlebox-specific steering policies

Firewall IDS ProxyWeb

10

What makes this problem challenging?Centralized Controller

“Flow” FwdAction… …

“Flow” FwdAction… …

OpenFlow

Proxy IDS

Middleboxes introduce new dimensions beyond L2/L3 tasks.

Achieve this with unmodified middleboxes and existing SDN APIs

Firewall IDS ProxyWeb

Firewall IDS ProxyWeb

SIMPLE overview

LegacyMiddleboxes

OpenFlow capable

Flow Action… …

Flow Action… …

11

Policy enforcement layer for middlebox-specific “traffic steering”

12

Challenge: Policy Composition

S1 S2

Firewall Proxy IDS

Firewall IDS Proxy*Policy Chain:

Oops! Forward Pkt to IDS or Dst?

Dst

“Loops” Traditional flow rules may not suffice!

13

Challenge: Resource Constraints

S1

S2S4

S3

ProxyFirewall

IDS1 = 50%

IDS2 = 50%

Space for traffic split?

Can we set up “feasible” forwarding rules?

14

S1Proxy

S2User 1

User 2

Proxy may modify flows

Are forwarding rules at S2 correct?

Challenge: Dynamic Modifications

Firewall

User1: Proxy FirewallUser2: Proxy

15

New dimensions beyond Layer 2-3 tasks

1) Policy Composition Potential loops

3) Dynamic Modifications Correctness?

2) Resource Constraints Switch + Middlebox

Can we address these with unmodified middleboxes and existing SDN APIs?

16

Rule Generator

Resource Manager Modifications Handler

SIMPLE System Overview

LegacyMiddleboxes

OpenFlow capable

Flow Action… …

Flow Action… …

Firewall IDS ProxyWeb

17

Composition Tag Processing StateFirewall IDS Proxy

*Policy Chain:

S1 S2

Firewall Proxy IDS

DstORIGINAL Post-Firewall

Post-IDSPost-Proxy

Fwd to Dst

Insight: Distinguish different instances of the same packet

18

Rule Generator

Resource Manager Modifications Handler

SIMPLE System Overview

LegacyMiddleboxes

OpenFlow capable

Flow Action… …

Flow Action… …

Firewall IDS ProxyWeb

19

Resource Constraints Joint Optimization

Resource Manager

Topology & Traffic

Switch TCAM

MiddleboxCapacity + Footprints

Policy Spec

Optimal & Feasible load balancing

Theoretically hard! Not obvious if some configuration is feasible!

20

Offline + Online Decomposition

Offline Stage Online Step

Deals with Switch constraints Deals with only load balancing

Resource Manager

Network Topology

Switch TCAM

Policy Spec

TrafficMatrix

Mbox Capacity + Footprints

21

Offline Stage: ILP based pruning

Set of all possible middlebox load distributionsPruned Set

Balance the middlebox load

• Feasible • Sufficient freedom

22

FW IDS ProxyWeb

Rule Generator

Resource Manager Modifications Handler

SIMPLE System Overview

LegacyMiddleboxes

OpenFlow capable

Flow Action… …

Flow Action… …

23

Modifications Infer flow correlations

Correlate flows

Install rules

S1Proxy

S2User 1

User 2 Firewall

User1: Proxy FirewallUser2: Proxy

Payload Similarity

24

FW IDS ProxyWeb

Rule Generator (Policy Composition)

Resource Manager(Resource Constraint)

Modifications Handler(Dynamic modifications)

SIMPLE Implementation

OpenFlow 1.0Flow Tag/

TunnelAction

… …

Flow Tag/Tunnel

Action

… …

POX extensions

CPLEX

25

Evaluation and Methodology• What benefits SIMPLE offers? load balancing? • How scalable is the SIMPLE optimizer?• How close is the SIMPLE optimizer to the optimal?• How accurate is the dynamic inference?• Methodology

– Small-scale real test bed experiments (Emulab) – Evaluation over Mininet (with up to 60 nodes)– Large-scale trace driven simulations (for convergence times)

26

Summary of SIMPLE• Middleboxes: Necessity and opportunity for SDN

• Goal: Simplify middlebox-specific policy enforcement

• Challenges: Composition, resource constraints, modifications

• SIMPLE: policy enforcement layer – Does not modify middleboxes– No changes to SDN APIs– No visibility required into the internal of middleboxes

• Scalable and offers 4-7X improvement in load balancing

27

Outline

• Why middleboxes?

• SIMPLE

• OpenMB

• Slick

28

Middlebox Deployment Models

• Arbitrary middlebox placement• New forms of middlebox deployment

(VMs, ETTM [NSDI 2011], CoMB [NSDI 2012])

29

• Move between software-defined data centers

• Existing VM and network migration methods– Unsuitable for changing underlying substrate

Live Data Center Migration

Data Center A Data Center B

Programmatic control over middlebox state

30

• Add or remove middlebox VMs based on load

• Clone VM (logic, policy, and internal state)– Unsuitable for scaling down or some scaling up

Middlebox Scaling

Fine-grained control

31

Contributions

• Classify middlebox state, and discuss what should be controlled

• Abstractions and interfaces– Representing state– Manipulating where state resides– Announcing state-related events

• Control logic design sketches

32

Controller

Middlebox

App App

Middlebox

SDN-like Middleboxes

IPS

Software-Defined Middlebox Networking

Today

33

Controller

Key Issues

Middlebox

1. How is the logic divided?

2. Where is state manipulated?

3. What interfaces

are exposed?

App App

Middlebox

34

• Configuration input

Middlebox State

State: ESTABSeq #: 3423

Server: BCPU: 50%

Hash: 34225Content: ABCDE

Significant state diversity

+ detailed internal records

Balance Method:Round Robin

Cache size: 100

Src: HostAServer: B

Proto: TCPPort: 22

35

Balance Method:Round Robin

Cache size: 100

Src: HostAServer: B

Proto: TCPPort: 22

Classification of State

State: ESTABSeq #: 3423

Server: BCPU: 50%

Hash: 34225Content: ABCDE

Action Supporting Tuning

Internal & dynamic Many forms

Only affects performance,

not correctness

36

PolicyLanguage

Src: HostAServer: B

Proto: TCPPort: 22

State: ESTABSeq #: 3423

Server: BCPU: 50%

Hash: 34225Content: ABCDE

How to Represent State?

Unknown structure

Significant diversity

May be shared

Per flow

SharedCommonality among middlebox operations

1000101

1101010

0101001

1111000

1010110

37

State Representation

• Key: protocol header field/value pairs identify traffic subsets to which state applies

• Action: transformation function to change parts of packet to new constants

• Supporting: binary blob

Key Action Supporting

Binary Blob

Field1 = Value1…

FieldN = ValueN

Offset1 → Const1…

OffsetN → ConstN

• Only suitable for per-flow state• Not fully vendor independent

38

Controller

Middlebox

How to Manipulate State?

• Today: only control some state– Constrains flexibility and sophistication

• Manipulate all state at controller– Removes too much functionality from middleboxes

39

State Manipulation

• Control over state placement1. Broad operations interface2. Expose state-related events

Controller

IPS 1 IPS 2 Create and update state

Determine wherestate resides

40

Action

*

KeySrcIP = 10.10.0.0/16DPort = 22

KeySrcIP = 10.10.54.41DstIP = 10.20.1.23SPort = 12983DPort = 22

State = ESTAB

Supporting

Operations Interface

get ( , )FilterSrcIP = 10.10.54.41

add ( , )ActionDROP

KeyDstIP = 10.20.1.0/24

Source Destination Proto Other Action

* 10.20.1.0/24 TCP * DROP

remove( , )Filter…

• Need atomic blocks of operations• Potential for invalid manipulations of state

41

Firewall

Events Interface

• Triggers– Created/updated state– Require state to

complete operation• Contents

– Key– Copy of packet?– Copy of new state?

Controller

Balance visibility and overhead

42

Conclusion

• Need fine-grained, centralized control over middlebox state to support rich scenarios

• Challenges: state diversity, unknown semantics

get/add/remove ( , )…

ActionOffset1 → Const1

…

KeyField1 = Value1

…

Supporting

Binary Blob

43

Open Questions

• Encoding supporting state/other action state?• Preventing invalid state manipulations?• Exposing events with sufficient detail?• Maintaining operation during state changes? • Designing a variety of control logics?• Providing middlebox fault tolerance?

44

Outline

• Why middleboxes?

• SIMPLE

• OpenMB

• Slick

Network Policies

• Reachability– Alice can not send packets to Bob

• Application classification– Place Skype traffic in the gold queue

Limitations of SDN Data Plane

10.2.3.4:10.2.3.3 Fwd Port 1

A2:e3:f1:ba:ea:23:* Drop

Match Action

• Limited actions and matching– Match: Ethernet, IP, TCP/UDP port numbers– Action: forward, drop, rewrite header, etc.

Extending SDN’s Data Plane

• Expand the OpenFlow standards– Requires hardware support

• Implement richer data plane in controller– Introduces additional latency to packets

• Add new devices (Middleboxes)

Example: Detecting Network Attacks

• Inspect all DNS traffic with a DPI device• If suspicious lookup takes place, send to traffic scrubber

Example: Detecting Network Attacks

• Inspect all DNS traffic with a DPI device• If suspicious lookup takes place, send to traffic scrubber

Example: Detecting Network Attacks

• Inspect all DNS traffic with a DPI device• If suspicious lookup takes place, send to traffic scrubber

Example: Detecting Network Attacks

• Inspect all DNS traffic with a DPI device• If suspicious lookup takes place, send to traffic scrubber

Challenges

• Specify network policies across middleboxes– Difficult to automatically react to middlebox events

• Dynamically place sophisticated middleboxes– Difficult to determine efficient placement– Difficult to adjust placement to traffic patterns

• Support for arbitrary middlebox functionality– Difficult to capture hardware requirements

Slick Contributions

• Abstraction for programming middleboxes– Simplifies the development of network policies– Separates specification of intent from implementation

• Dynamic placement of middlebox functionality– Online resource allocation algorithm

• Support for heterogeneous devices– Maintains performance profiles of middlebox

Slick Architecture

Slick Controller

MiddleboxElement

MiddleboxElement

Application• Encodes network policy• Provides handlers for

triggers

• Piece of code encapsulating middlebox functions

Your network operator

3rd party elementdevelopers

Programmable device: NetFPGA, x86 server

Virtual Switch

Triggers from elements

Slick Architecture

Slick Controller

Application• Runs applications• Runs resource allocation algo.

• Places middlebox elements• Steers traffic through middleboxes

• Configures switches

• Installs/uninstalls middlebox functions

DeployMiddlebox code

MiddleboxElement

MiddleboxElement

Programmable device: NetFPGA, x86 server

Virtual Switch

Resource Allocation Heuristic

Resource allocation heuristic

Traffic Steering

OpenFlow Controller

Placement Decisions

Traffic matrixAnd topology

Network policies inapplications

Middlebox perfprofile

Hardwareconstraints

Programmable device

Virtual Switch

Programmable device

Virtual Switch

Objective: minimize latency (path lengths)

Summary

• Slick: control plane for middleboxes– Presented an initial architecture– Discussed algorithmic challenge

• Slick is implemented in python– Slick controller as a module on NoX 0.5.0– Developed 2 applications and 3 middlebox elements

• Open questions– How can developers help guide placement?– What is the optimal solution for resource allocation?

58

Discussion: Likes/dislikes?