demand-aware network design with minimal congestion and … · 2019-05-05 · challenges!6 low...

Demand-Aware Network Design with Minimal Congestion

and Route Lengths

INFOCOM 2019

Chen Avin , Kaushik Mondal, Stefan Schmid

Motivation

!2

Motivation• Traditionally, networks topologies

designs are demand oblivious:

!2


designs are demand oblivious:• optimized for the “worst-case”:

all-to-all communications

!2



all-to-all communications• e.g., Fat-tree topologies

provide (almost) a fullbisection bandwidth

!2



all-to-all communications• e.g., Fat-tree topologies

provide (almost) a fullbisection bandwidth

• But…

!2

Motivation

!3

Motivation• Traffic is growing fast

!3Aggregate Server Traffic in Google datacenter

Jupiter rising @ SIGCOMM 2015

Motivation• Traffic is growing fast• Real communication patterns are sparse

and feature structure (?!)


Jupiter rising @ SIGCOMM 2015Heatmap of rack-to-rack traffic ProjecToR @ SIGCOMM 2016

Motivation• Traffic is growing fast• Real communication patterns are sparse

and feature structure (?!)• Can be exploited if demand is known:

demand-aware design


Jupiter rising @ SIGCOMM 2015Heatmap of rack-to-rack traffic ProjecToR @ SIGCOMM 2016

Demand-Aware Design?

!4

Application Layer

Transport Layer

Network Layer

Link / Physical Layer

Networks Capable of Change.Jennifer Rexford.

Infocom 2019 Keynote.


!4

Application Layer

Transport Layer

Network Layer

Link / Physical Layer ?!




!4

Application Layer

Transport Layer

Network Layer

Link / Physical Layer ?!Even in real time!



Goal

!5

Goal• Design a demand aware, bounded

degree (scalable) networks with:

!5


degree (scalable) networks with:1. Short average route length (l),

and

!5



and 2. Low congestion (c):

!5



and 2. Low congestion (c):

• Both are importantmeasures of efficiency

!5

Challenges

!6

Challenges

!6

Short route length: bottleneck

Chen Avin, Kaushik Mondal and Stefan Schmid Demand-Aware Network Designs of Bounded Degree. DISC 2017

Challenges

!6

Low congestion: high degree/long routes



Challenges

!6

Low congestion: high degree/long routes

Demand-Aware Network Design withMinimal Congestion and Route Lengths

Chen AvinCommunication Systems Engineering Dept.Ben Gurion University of the Negev, Israel

Kaushik MondalCommunication Systems Engineering Dept.Ben Gurion University of the Negev, Israel

Stefan SchmidFaculty of Computer ScienceUniversity of Vienna, Austria

Abstract—Emerging communication technologies allow to re-

configure the physical network topology at runtime, enabling

demand-aware networks (DANs): networks whose topology is opti-

mized toward the workload they serve. However, today, only little

is known about the fundamental algorithmic problems underlying

the design of such demand-aware networks. This paper presents

the first bounded-degree, demand-aware network, cl-DAN, which

minimizes both congestion and route lengths. The designed

network is provably (asymptotically) optimal in each dimension

individually: we show that there do not exist any bounded-degree

networks providing shorter routes (independently of the load),

nor do there exist networks providing lower loads (independently

of the route lengths). The main building block of the designed

cl-DAN networks are ego-trees: communication sources arrange

their communication partners in an optimal tree, individually.

While the union of these ego-trees forms the basic structure of

cl-DANs, further techniques are presented to ensure bounded

degrees (for scalability).

I. INTRODUCTION

A. MotivationData center networks have become a critical infrastructure of

our digital society. With the trend toward more data-intensiveapplications, data center network traffic is growing quickly [7],[31]. As much of this traffic is internal to the data center (e.g.,traffic due to scatter-gather and batch computing applications),the design of efficient data center networks has received muchattention over the last years [23].

Traditionally, data center designs are demand-oblivious andstatic: they are optimized for the “worst-case”, e.g., they(almost) provide a full bisection bandwidth, allowing to servedense, all-to-all communication patterns. Empirical studieshowever show that real communication patterns are usually farfrom all-to-all. Rather, traffic patterns feature spatial localityand are sparse [5], [9], [13], [19], [21], [26]: only a smallfraction of all possible source-destination pairs are involved inintensive communications at any time.

The advent of novel optical technologies which allow toreconfigure the physical network topology [10], [15], [20], [21],heralds a paradigm shift: using these technologies, data centerdesigns can be reconfigured and optimized toward their demand,i.e., they become demand-aware. In particular, a demand-awarenetwork design may connect frequently communicating nodes“better”: the network provides shorter routes between suchnodes (lower latency, energy consumption etc.) and aims toreduce congestion by keeping traffic local (lower load, lessqueuing delays, etc.).

(a) (b) (c)Fig. 1. Challenge of designing demand-aware networks: (a) Optimizing forroute lengths only may result in bottlenecks and high loads. (b) Optimizing forcongestion only, by distributing load across multiple paths, can result in longroutes. (c) Ideally, we aim to design networks that minimize both congestionand route lengths, using a small number of links (constant degree).

However, only little is known today about the algorithmicchallenge of designing demand-aware networks which providelow congestion and short routes (in the number of hops), fora given communication pattern. This is the topic of our paper(see also Figure 1).

At first sight, it may seem that designing networks providingboth short routes and minimal load is hard and faces a tradeoff:to better balance loads, it may be necessary to route flows alonglonger paths. Yet, as we show in this paper, a solution can beefficiently computed which is almost optimal both in termsof route length and congestion, independently (i.e., withouttradeoff).

B. The Demand-Aware Network Design Problem

Intuitively, the demand-aware network design problem canbe stated as follows (a formal model will follow later). Weare given a set of n nodes (e.g., top-of-rack switches [21])interacting according to a certain communication pattern: afrequency distribution represented as matrix or (weighted)demand graph.

Our goal is to design a demand-aware network, cl-DAN,together with a routing scheme, which serves this communica-tion pattern providing low congestion and shorth route lengths.The designed network should be scalable, i.e., of boundeddegree (e.g., reconfigurable links may consume space and/or



Challenges: An Example

!7

Demand Distribution

Goal: design an optimal network with bounded degree 3


!7

Demand Distribution Route length is minimum, not congestion



!7


Congestion is minimum, not route length



!7


Congestion is minimum, not route length

Can we optimize both simultaneously?


Model and Definitions

!8

Model and Definitions• Demand joint distribution

!8

D<latexit sha1_base64="MhRJrXArGYUXlM+4bhaEvXNooeM=">AAAB8nicbVBNS8NAFHypX7V+VT16WSyCp5KooMeiHjxWsLaQhrLZbtulm03YfRFK6M/w4kERr/4ab/4bN20O2jqwMMy8x86bMJHCoOt+O6WV1bX1jfJmZWt7Z3evun/waOJUM95isYx1J6SGS6F4CwVK3kk0p1EoeTsc3+R++4lrI2L1gJOEBxEdKjEQjKKV/G5EccSozG6nvWrNrbszkGXiFaQGBZq96le3H7M04gqZpMb4nptgkFGNgkk+rXRTwxPKxnTIfUsVjbgJslnkKTmxSp8MYm2fQjJTf29kNDJmEoV2Mo9oFr1c/M/zUxxcBZlQSYpcsflHg1QSjEl+P+kLzRnKiSWUaWGzEjaimjK0LVVsCd7iycvk8azunde9+4ta47qoowxHcAyn4MElNOAOmtACBjE8wyu8Oei8OO/Ox3y05BQ7h/AHzucPddmRXQ==</latexit>

Model and Definitions• Demand joint distribution• A network

!8


N<latexit sha1_base64="0oJp+RPrFPkdBwfDyH67uygMDug=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KokKeix4EQRpwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut7Oyura+sVnYKm7v7O7tlw4OmzpOFcMGi0Ws2gHVKLjEhuFGYDtRSKNAYCsY3Uz91hMqzWP5YMYJ+hEdSB5yRo2V6ve9UtmtuDOQZeLlpAw5ar3SV7cfszRCaZigWnc8NzF+RpXhTOCk2E01JpSN6AA7lkoaofaz2aETcmqVPgljZUsaMlN/T2Q00nocBbYzomaoF72p+J/XSU147WdcJqlByeaLwlQQE5Pp16TPFTIjxpZQpri9lbAhVZQZm03RhuAtvrxMmucV76Li1S/L1bs8jgIcwwmcgQdXUIVbqEEDGCA8wyu8OY/Oi/PufMxbV5x85gj+wPn8AapCjN4=</latexit>

Model and Definitions• Demand joint distribution• A network• A routing scheme

!8



�(N)<latexit sha1_base64="d5eQ+89/rBqwjvTNfYoSgOF9IwI=">AAAB8HicbVDLSgNBEOz1GeMr6tHLYBDiJeyqoMeABwVBIpiHJEvonUySITO7y8ysEJZ8hRcPinj1c7z5N06SPWhiQUNR1U13VxALro3rfjtLyyura+u5jfzm1vbObmFvv66jRFFWo5GIVDNAzQQPWc1wI1gzVgxlIFgjGF5N/MYTU5pH4YMZxcyX2A95j1M0VnpsX6OUWLo76RSKbtmdgiwSLyNFyFDtFL7a3YgmkoWGCtS65bmx8VNUhlPBxvl2olmMdIh91rI0RMm0n04PHpNjq3RJL1K2QkOm6u+JFKXWIxnYTolmoOe9ifif10pM79JPeRgnhoV0tqiXCGIiMvmedLli1IiRJUgVt7cSOkCF1NiM8jYEb/7lRVI/LXtnZe/+vFi5zeLIwSEcQQk8uIAK3EAVakBBwjO8wpujnBfn3fmYtS452cwB/IHz+QPA0o++</latexit>

Model and Definitions• Demand joint distribution• A network• A routing scheme • The congestion:

!8




Model and Definitions• Demand joint distribution• A network• A routing scheme • The congestion:

• The weighted path length

!8




Goal: cl-DAN Design

!9

(↵,�)<latexit sha1_base64="jLKg44kMhMs2M1Xfd8/ZYyLpPoI=">AAAB+HicbVDLSgNBEOz1GeMjqx69DAYhgoRdFfQY8CJ4iWAekF1C72Q2GTL7YGZWiCFf4sWDIl79FG/+jZNkD5pY0FBUddPdFaSCK+0439bK6tr6xmZhq7i9s7tXsvcPmirJJGUNmohEtgNUTPCYNTTXgrVTyTAKBGsFw5up33pkUvEkftCjlPkR9mMecoraSF27VPFQpAM8I17ANJ527bJTdWYgy8TNSRly1Lv2l9dLaBaxWFOBSnVcJ9X+GKXmVLBJ0csUS5EOsc86hsYYMeWPZ4dPyIlReiRMpKlYk5n6e2KMkVKjKDCdEeqBWvSm4n9eJ9PhtT/mcZppFtP5ojATRCdkmgLpccmoFiNDkEpubiV0gBKpNlkVTQju4svLpHledS+q7v1luXaXx1GAIziGCrhwBTW4hTo0gEIGz/AKb9aT9WK9Wx/z1hUrnzmEP7A+fwBau5JE</latexit>

Goal: cl-DAN Design

!9


Input: matrix, degree

D,�<latexit sha1_base64="dhHBLegVOrV+0dzk94dPReYAq5I=">AAAB/HicbVDLSsNAFJ34rPUV7dLNYBFcSElU0GXBLgQ3FewDmlAm09t26GQSZiZCCPVX3LhQxK0f4s6/cdJmoa0HBg7n3Ms9c4KYM6Ud59taWV1b39gsbZW3d3b39u2Dw7aKEkmhRSMeyW5AFHAmoKWZ5tCNJZAw4NAJJje533kEqVgkHnQagx+SkWBDRok2Ut+ueCHRY0p41pieYa8BXJO+XXVqzgx4mbgFqaICzb795Q0imoQgNOVEqZ7rxNrPiNSMcpiWvURBTOiEjKBnqCAhKD+bhZ/iE6MM8DCS5gmNZ+rvjYyESqVhYCbzqGrRy8X/vF6ih9d+xkScaBB0fmiYcKwjnDeBB0wC1Tw1hFDJTFZMx0QSqk1fZVOCu/jlZdI+r7kXNff+slq/K+oooSN0jE6Ri65QHd2iJmohilL0jF7Rm/VkvVjv1sd8dMUqdiroD6zPHyk0lHk=</latexit>

Goal: cl-DAN Design

!9




Output: cl-DAN

N 2 N�,�(N)<latexit sha1_base64="wLnTFDnBaqnp4QhRinGOdX/I5lY=">AAACDnicbVDLSgNBEJyNrxhfUY9eBkMggoRdFfQYUFAQQgTzgGwIvZNJMmRmdpmZFcKSL/Dir3jxoIhXz978GyePgyYWNBRV3XR3BRFn2rjut5NaWl5ZXUuvZzY2t7Z3srt7NR3GitAqCXmoGgFoypmkVcMMp41IURABp/VgcDn26w9UaRbKezOMaEtAT7IuI2Cs1M7my9hnEvsCTJ8AT8qjduJfUW5gdIz9axACCuWjdjbnFt0J8CLxZiSHZqi0s19+JySxoNIQDlo3PTcyrQSUYYTTUcaPNY2ADKBHm5ZKEFS3ksk7I5y3Sgd3Q2VLGjxRf08kILQeisB2js/W895Y/M9rxqZ70UqYjGJDJZku6sYcmxCPs8EdpigxfGgJEMXsrZj0QQExNsGMDcGbf3mR1E6K3mnRuzvLlW5ncaTRATpEBeShc1RCN6iCqoigR/SMXtGb8+S8OO/Ox7Q15cxm9tEfOJ8/jqabLQ==</latexit>

Goal: cl-DAN Design

!9




Output: cl-DAN


Where

Optimal congestion

Goal: cl-DAN Design

!9




Output: cl-DAN


Where

Optimal congestion

&Optimal path length

!10

A Building Block: EgoTree

!10

• A single source multiple destination problem


!10



S

p̄ = {p1, p2, …, pk}

!10

…Δ


•


EgoTree(s, p̄, Δ) S

p̄ = {p1, p2, …, pk}

Binary Trees

!10

…Δ


•

• Optimizes both C and L


EgoTree(s, p̄, Δ) S

p̄ = {p1, p2, …, pk}

Binary Trees

EgoTree: A Greedy Construction

• Sort = {p1, p2, …, pk} from large to small• Greedily Place destination nodes (one at a

time)

!11

…

S

T1 T2 … TΔ

EgoTree: An Example• Ego-tree is not balanced w.r.t. sizes of

the subtrees• Almost balanced w.r.t. the probability

mass that defines congestion

!12

Analysis on Congestion

!13

Analysis on Congestion• Minimizing congestion is a NP-Hard problem

• provides 4/3 approximation to the minimum congestion (i.e., Longest Processing Time [Graham69])

!13

EgoTree(s, p̄, Δ)

Analysis on Route-Length

!14

Analysis on Route-Length• Considering all the binary subtrees and

using entropy grouping properties, we have an Entropy bound:

!14

Analysis on Route-Length• On any optimal Δ-ary tree:

• Combining all, we now have optimality on L

!15

H(p̄)log2(Δ + 1)

≤ L(p̄, T*Δ) ≤ L(p̄, T*s ) ≤ H(p̄)

!16

From Trees to Networks

!17


• Real distributions are sparse: datacentre's traffic shows demand distributions are sparse

!17



• We assume a distribution D with average degree ⍴

!17




• ⍴ is a constant

!17




• ⍴ is a constant

• Half of the nodes of lowest degree are defined as low degree nodes; others are high degree nodes

!17


Sparse Distributions• Proof idea i

!18

Sparse Distributions• Proof idea i i

Optimal bounded degree tree

!18


i j

i

Optimal bounded degree treeProblem

!18


i j

i


Solution - “helper” nodes!18


i j i j

i


Solution - “helper” nodes!18

Ego-tree Modification • Modify ego-tree Tu of a high degree node u

to T’u

• let v be a high degree neighbor of u: b be the helper node• we remove v from ego-tree of u if p(u,b)>p(u,v) • else we put b in place of v

!19

cl-DANs : Sparse Distributions

!20


• For Δ=12 ⍴

!20


• For Δ=12 ⍴• Degree of nodes are at most Δ

!20


• For Δ=12 ⍴• Degree of nodes are at most Δ• Congestion is at most:

!20

C(D,�(N)) 1 + 8/9�C⇤(D,�)<latexit sha1_base64="mbbXKXL7pcKOIymk13piKOMv1tA=">AAACLXicbVDLSgMxFM34tr6qLt0Ei9Cq1BkV1F2hBQVBFKwKnVrupLc1mMwMSUYoQ3/Ijb8igouKuPU3TB8LXwcCh3POJfeeIBZcG9ftOWPjE5NT0zOzmbn5hcWl7PLKlY4SxbDKIhGpmwA0Ch5i1XAj8CZWCDIQeB3cl/v+9QMqzaPw0nRirEtoh7zFGRgrNbKVct6XYO4YiLTS3ab+MUgJ+bNCgfoCqUe36OHOEfUrKAzQ8u3mj/hQLjSyObfoDkD/Em9EcmSE80b2xW9GLJEYGiZA65rnxqaegjKcCexm/ERjDOwe2lizNASJup4Oru3SDas0aStS9oWGDtTvEylIrTsysMn+qvq31xf/82qJaR3WUx7GicGQDT9qJYKaiParo02ukBnRsQSY4nZXyu5AATO24Iwtwft98l9ytVv09orexX6udDqqY4askXWSJx45ICVyQs5JlTDySJ5Jj7w5T86r8+58DKNjzmhmlfyA8/kFlQqkvg==</latexit>


• For Δ=12 ⍴• Degree of nodes are at most Δ• Congestion is at most:

• Average Path length is at most

!20

C(D,�(N)) 1 + 8/9�C⇤(D,�)<latexit sha1_base64="mbbXKXL7pcKOIymk13piKOMv1tA=">AAACLXicbVDLSgMxFM34tr6qLt0Ei9Cq1BkV1F2hBQVBFKwKnVrupLc1mMwMSUYoQ3/Ijb8igouKuPU3TB8LXwcCh3POJfeeIBZcG9ftOWPjE5NT0zOzmbn5hcWl7PLKlY4SxbDKIhGpmwA0Ch5i1XAj8CZWCDIQeB3cl/v+9QMqzaPw0nRirEtoh7zFGRgrNbKVct6XYO4YiLTS3ab+MUgJ+bNCgfoCqUe36OHOEfUrKAzQ8u3mj/hQLjSyObfoDkD/Em9EcmSE80b2xW9GLJEYGiZA65rnxqaegjKcCexm/ERjDOwe2lizNASJup4Oru3SDas0aStS9oWGDtTvEylIrTsysMn+qvq31xf/82qJaR3WUx7GicGQDT9qJYKaiParo02ukBnRsQSY4nZXyu5AATO24Iwtwft98l9ytVv09orexX6udDqqY4askXWSJx45ICVyQs5JlTDySJ5Jj7w5T86r8+58DKNjzmhmlfyA8/kFlQqkvg==</latexit>


• What have we shown?• For sparse distributions we can have

!21


• What have we shown?• For sparse distributions we can have

!21

Near optimal congestion

Near optimal path length

Take home

!22

Take home•Demand Aware Networks are possible

!22


•The “challenge” is Self-Adjusting Networks

•without knowing the future

!22




•What about:

• Δ is forced.

• routing protocols?

•cost metrics?

•Sync with upper layers

!22




•What about:

• Δ is forced.

• routing protocols?

•cost metrics?

•Sync with upper layers

•Much work and Interesting…

!22

Thank you!Toward Demand-Aware Networking: A Theory for Self-Adjusting NetworksChen Avin and Stefan Schmid.SIGCOMM CCR, October 2018.Demand-Aware Network Designs of Bounded DegreeChen Avin, Kaushik Mondal, and Stefan Schmid.31st International Symposium on Distributed Computing (DISC), Vienna, Austria, October 2017.

Online Balanced RepartitioningChen Avin, Andreas Loukas, Maciej Pacut, and Stefan Schmid.30th International Symposium on Distributed Computing (DISC), Paris, France, September 2016.

rDAN: Toward Robust Demand-Aware Network DesignsChen Avin, Alexandr Hercules, Andreas Loukas, and Stefan Schmid.Information Processing Letters (IPL), Elsevier, 2018.SplayNet: Towards Locally Self-Adjusting NetworksStefan Schmid, Chen Avin, Christian Scheideler, Michael Borokhovich, Bernhard Haeupler, and Zvi Lotker.IEEE/ACM Transactions on Networking (TON), Volume 24, Issue 3, 2016. Early version: IEEE IPDPS 2013.

Demand-aware network design with minimal congestion and route lengthsC. Avin, K. Mondal, and S. Schmid, IEEE INFOCOM, 2019.Distributed self-adjusting tree networksB. Peres, O. Souza, O. Goussevskaia, S. Schmid, and C. Avin,Proc. IEEE INFOCOM, 2019.

Furth

er R

eadi

ng

!23

https://net.t-labs.tu-berlin.de/~stefan/dan-san.pdf

https://net.t-labs.tu-berlin.de/~stefan/disc17.pdf

https://www.net.t-labs.tu-berlin.de/~stefan/disc16repartition.pdf

https://net.t-labs.tu-berlin.de/~stefan/ipl18.pdf

http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=7066977&url=http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber%3D7066977

demand-aware network design with minimal congestion and … · 2019-05-05 · challenges!6 low...

Documents