Adaptive CPU Allocation forSoftware basedRouter Systems

Puneet Zaroo

Software based routers

Implement packet forwarding/processing in software. E.g a PC with multiple NICs.

Provide value added services like encryption, network address translation esp. at the network edge.

Issues Software architecture.

Per flow threads / per-packet threads Division of input, forwarding and output functions

CPU scheduling. How to determine CPU shares How to enforce CPU shares.

Objective

Leverage the advantages of a component based software router system. Flexibility in designing routers Reusability of software components Dynamic addition of element modules

Overlay a QoS provisioning mechanism on top of the component based system.

Develop an adaptive QoS system Adaptive to varying input rate and per-packet

processing costs.

Some Software Router Systems

Router Plugins : ETH Zurich, Uwash St. Louis Per flow code modules or plugins. Implemented in the NetBSD kernel.

Click Modular router : MIT Routers made of elements composed into a flow graph.

ANTS Programmable and customizable networks. Customizable applications acting on packets / packets carrying code as well

as data. X-kernel : University of Arizona

Object oriented interface to protocols. Can be used on end systems as well as routers.

Scout : University if Arizona, Princeton University Communication oriented OS based on x-kernel. Path based abstraction. Advanced CPU scheduling.

OS support for CPU scheduling

Scout Proportional scheduling. CPU balance (extension of work on livelock)

Resource Containers : Rice University Decoupling of protection domain/resource

domain. Proper accounting of resources to processes.

Resources include threads as well as kernel data structures and memory,bound to containers.

E.g a web server serving multiple connections. Processor Capacity reserves : CMU

Provides support for both time-sharing and real-time systems. The OS enforces the reservations (cpu share, time period). Applications free to change their reservations subject to admission control.

Nemesis : Cambridge OS does low level resource multiplexing. Avoiding QoS cross-talk

Support for I/O in user level libraries.

Click

Composable flow-graphs from router elements Packets travel along graph edges Element based processing (push/pull). Element based scheduling. Multithreaded SMP Click

Issues in flow level QoS on top of an element based architecture Flow level accounting and scheduling. CPU balance b/w input, output and processing. CPU conservation of idle elements.

CROSS/Linux – Resource reservation with containers

Containers Group of related elements

Elements doing per flow processing. Container – CPU resource reservation unit.

Why use containers and not flows ? Types of Containers

Input Output Forwarding

Best Effort QoS - Packet rate reservations

Example Router Configuration

CROSS/Linux - CPU scheduling

Three level scheduler Linux schedules CROSS

Linux process schedulerCROSS schedules Containers

Proportional (Dynamic stride scheduling)Containers schedule Elements

Simple Round Robin scheduling

CROSS/Linux – Architectural Enhancements to Click

CPU conservation through sleep/wakeup Elements tested for scheduling eligibility Containers tested for scheduling eligibilty Notifier Queues - wake up elements (make eligible

for scheduling) Delayed wakeup Network interface Input Element

Switching between polling and interrupt Based on a threshold packet input rate to reduce

programmed I/O overhead Topology discovery

Discovering input/output queues for a container

CROSS/Linux – Enhancements to Click

virtual Interface queues – especially for interface statistics gathering

Linux /proc interface – One directory for each container Directory provides information about

Container tickets CPU cycles consumed Packet rate/drop rate Elements Input/Output queues Set container tickets

CROSS/Linux – Share adaptation

Why ? Inability to do a-priori CPU share calculation Variations in packet input rate Variations in per-packet processing cost

How ? Scheduler for each container keeps track of

Packet input rate. Packet drop rate. CPU cycles used.

Recomputes container shares to remove packet drops.

CROSS/Linux – Share adaptation

Statistics maintained by QueuesPacket ratesPacket drop rates

Queues used to connect containersPacket pass/drop rates at Queues

indicate the difference between the required and the actual CPU shares for the container

Share adaptation Algorithm

Invoked every 1 second Notation used

T – Ticket share C – Current CPU share p – Input packet rate d – packet drop rate m – maximum input rate

General idea Increase ticket share of a container so that the drop

rate is removed at all the containers

Input Container share adaptation (Issues)

Pass as many packets as possible upto a maximum.How to arrive at this maximum?Forwarding more than the maximum

adversely affects the effective router throughput.

Reduce share on observing over allocation.

Input Container – Share adaptation(Algorithm)

if p > m /* Input rate too high */ /* reduce share */ T = C * (m/p)else if d > 0 /* Increase share to */ /*remove packet drops */ drate = min(d + p,m) T = C * (drate/p) else if (T – C) >= delta

/* Over allocation *//* reduce share */T = T – eps

QoS container – Share adaptation(Issues)

Always forward till reserved rate. Target a forwarding rate range.

Reduce share in case of over allocation

QoS container – Share adaptation(Algorithm)

If p ε [ R – Dt, R + Dt] /* No change */ return if p > R + Dt /* Reduce share */ T = C * (R/p) else if d > 0 /* Increase share */ drate = min(p + d,R) T = C * (drate/p) else if (T-C) >= delta /* Reduce share */ T = T – eps

Output Container – Share adaptation (Issues)

Try to forward all that is receivedThrottling if any has happened upstream

Reduce share in case of over allocation

Output Container – Share adaptation (Algorithm)

if d > 0 /*Increase share */ T = C * (1 + d/p)else if (T – C) >= delta / * Reduce Share */ T = T - eps

Best Effort Container – Share adaptation

No action takenSystem makes no guarantees

Discussion

Packet rate based reservation Reservations based on packet rates more intuitive CPU shares may vary for the same packet rates

C (Actual share) - How is it calculated? Input container

Only include CPU cycles used in packet processing as opposed to idle polling.

Other containers Easy to calculate since no idle polling.

m (Maximum forwarding rate) Constant determined at router initialization Evaluated at each iteration

Evaluation

Using a simulatorCalculates the forwarding rate , drop rate

based on the CPU shares.Mimics the actions of the adaptive algorithmEases loading the “router” and testing of

diverse workloadsUsing a real implementation

CROSS/Linux on 866 Mhz Pentium III CPU.

Adaptive vs. Non Adaptive(Experimental setup)

Input (2 µs), Output (2 µs) , Best Effort Container (6 µs).

Router – 1 MHz CPU => max forwarding = 100,000 packets/s

Static ticket assignment = 1:1:1Input varied for 0 to 110,000 packets/s in

increments of 10,000 packet/s every 10s.

Adaptive vs. Non Adaptive(Variation with time)

Adaptive vs. Non Adaptive(Maximum loss free forwarding rate)

Variable packet processing time(Experimental Setup)

Input (2µs), Best Effort/QoS (6µs), Output Container (2µs) Observe different convergence behavior for QoS /

Best Effort Router – 1 MHz CPU => max forwarding rate

initially = 100,000 packets/s Constant input = 50,000 packets/s Per packet processing cost increased by 2 µs

every 10 secs. Max. forwarding rate = 50,000 packets/s at

t=50s.

Variable packet processing time(Adaptive vs. Non Adaptive)

Variable packet processing time-(Best Effort vs. QoS)

Adaptation in m

Hard to determine m at router initializationMay vary with variations in per packet

processing costs.

m = maxi (TOTAL_CPU_CPS/cpu_cpp(ci))

where ci ε C TOTAL_CPU_CPS - Total CPU cycles per second available to the router cpu_cpp(ci) - cycles/packet being used by the flow serviced by container ci

cpu_cpp(ci) = cpu_cpi() + cpu_cycles(ci)/num_packets(ci) + cpu_cpo()

C - The set of containers servicing active flows

Fixed vs adaptive m - (Experimental setup)

Input (8µs), Best Effort/QoS (1µs), Output Container (1µs)

Router – 1 MHz CPU => max forwarding rate, initially = 100,000 packets/s

Constant input = 50,000 packets/s Per packet processing cost increased by 2 µs

every 5 secs Max forwarding rate = 50,000 packets/s at

t=30 s.

Fixed vs adaptive m - (Effective Best Effort Forwarding)

Fixed vs. adaptive m(Effective QoS forwarding)

Fixed vs. Adaptive m(Best Effort, QoS , Theoretical maximum)

Advanced Adaptation in m

Previous algorithm gives too much stress to the least expensive flow. Fine if all packets destined for that flow. The packet rate to different flows can be variable.

m =(TOTAL_CPU_CPS/weighted_cpu_cpp) weighted_cpu_cpp

= Σ (cpu_cpp(ci) * rate(ci))/ (Σ rate(ci))

where ci ε C

Adaptive m vs. advanced adaptive m(Experimental Setup)

Input container (5 µs), Output Container(5 µs) Router (1 MHz CPU) 2 flows

QoS container (50,000 p/s,30 µs) => max forwarding rate achievable = 25,000 packets/s

Best Effort container (3 µs) => max forwarding rate achievable = 77,000 packets/s

Input rate to best effort container = 500 packets/s Input rate to QoS container varied from 15,000

packets/s to 50,000 packets/s in increments of 5,000 packets/s every 5 s.

Adaptive m vs. advanced adaptive m(Forwarding rate vs. time)

Evaluation on a Router

CROSS/Linux software router platformP III 866 MHZ pc.3 network interface cards.

QoS Forwarding (Experimental setup)

866 MHz , PIII router Input Container(4.5 µs) , Best Effort

Container(3 µs),QoS container (32,000 packets/s), Output Container (4.9 µs)

3 different per – packet processing costsfor the QoS container 3, 9.7 and 15.2 µs

Input to QoS => 32,000 packets/ Input to Best Effort => 27,000 packets/s

QoS Forwarding (Forwarding rate)

QoS Forwarding (Ticket Share)

QoS forwarding (Ticket Shares)

Case Input Output Best

Effort

QoS

3 µs 0.29 0.236 0.236 0.236

9.7 µs 0.27 0.282 0.153 0.293

15.2 µs 0.213 0.245 0.068 0.47

QoS forwarding (CPU Shares)

Case Input Output Best

Effort

QoS

3 µs 0.51 0.29 0.08 0.10

9.7 µs 0.31 0.299 0.087 0.30

15.2 µs 0.21 0.24 0.066 0.48

Effective Forwarding rate(Experimental setup)

Input (4.5 µs), best effort (8.3 µs) and output (4.9 µs)

Maximum forwarding rate = 57,000 p/s 3 different scenarios

No AdaptationCPU share Adaptation and m = 65000

packets/sCPU share Adaptation and m = 110000

packets/s

Effective Forwarding rate

Future Work

Conjoint CPU – Buffer Allocation Insufficient CPU share => always packet drops Once sufficient CPU shares, more buffering =>

more efficiency More buffering => higher packet delays and

packets getting dropped at line cards.

Share adaptation between Linux/CROSS Can use the SFQ scheduler already implemented

Conclusion

Provide a QoS provisioning layer on top of a component based system.

Adaptive in response to variable packet input and processing costs.

THANK YOU