IEEE COMMUNICATIONS LETTERS, VOL. 18, NO. 10, OCTOBER 2014 1739

Buffer Stopping Time Analysis in Data Center NetworksGan Luan

Abstract—This letter investigates buffer behavior in data centernetworks (DCNs). An analytical framework to model a switchbuffer in a DCN is proposed. Based on a martingale perspective,we derive the expectation of the stopping time, and we provideexplicit expression for the relationship between overflow proba-bility of the stopping time and buffer size. Simulations are givento validate the analysis. In addition, an example is provided toexplain how the method can be applied to switch design.

Index Terms—Buffer size, data center network, stopping time,overflow probability.

I. INTRODUCTION

BUFFER SIZE and buffer behavior analysis have long beenhot topics. Average buffer occupancy and the relationship

between buffer size and overflow probability are discussed in[1]–[4]. Different topologies and routing schemes employed indata center networks (DCNs) are proposed in [5]–[8]. They re-quire different switch capabilities at different positions. There-fore, switch design in DCNs is of great importance.

On the basis of an empirical study of the traffic characteris-tics in DCNs, the per-packet arrival process in a DCN followsan ON/OFF traffic pattern. The distributions of ON perioddurations, OFF period durations, and packet inter-arrival timeswithin ON periods present positive skews and heavy tails. Thebest fits are produced by either lognormal curves or Weibullcurves. Furthermore, the packet sizes in the data centers underdiscussion exhibit a bimodal pattern, with most packet sizesclustering around either 200 Bytes or 1400 Bytes [9], [10].

In this letter, we focus on issues about which end (emptyend or full end) of the buffer is reached first and when thishappens during the transmission. The remainder of this letteris organized as follows. We propose an analytical frameworkaccording to traffic characteristics in DCNs to model the bufferbehavior (Section II). Then, we construct martingales to obtainthe expectation of the stopping time and the overflow probabil-ity along with it (Section III). During the derivation, we giveasymptotics for overshoots of a random walk with left heavy-tailed increments. Simulations demonstrated the accuracy ofthe study, followed by an example showing how this methodapplies to buffer design (Section IV). Finally, we conclude thisletter (Section V).

Manuscript received March 3, 2014; revised August 16, 2014; acceptedSeptember 3, 2014. Date of publication September 10, 2014; date of currentversion October 8, 2014. The associate editor coordinating the review of thispaper and approving it for publication was W. Fawaz.

The author is with the State Key Laboratory of Networking and SwitchingTechnology, Beijing University of Posts and Telecommunications, Beijing100876, China (e-mail: luangan@bupt.edu.cn).

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/LCOMM.2014.2356478

Fig. 1. Embedded Markov chain of the ON/OFF semi-Markov process.

II. MODEL DESCRIPTION

In a DCN switch, each output port is allocated with a buffer.It is assumed that the output ports are independent of eachother. The analysis is concerned with the buffer performanceof a single output port.

A. Buffer Notations

The buffer size of the output port under study is denoted asL in kBytes. The buffer is preset to be occupied with l0 kBytesof data, and the initial occupancy is l0/L.

B. ON/OFF Traffic Pattern

The ON/OFF traffic pattern found in DCNs can be describedwith semi-Markov process. The time intervals between transi-tions have subexponential distributions as illustrated in Fig. 1.

The embedded Markov chain is irreducible and positiverecurrent. The steady state probabilities of the embedded chainare {πon = 0.5, πoff = 0.5}.

The random variables (rv’s) Ton and Toff represent thedurations of an ON period and an OFF period. The rv Tint

denotes an inter-arrival time within an ON period. They areall in milliseconds (ms), and their cumulative distribution func-tions (CDFs) are denoted as Fon(x), Foff (x), and Fint(x),respectively.

It is noticed that every Ton consists of successive Tint, andToff is actually an inter-arrival time much longer than Tint

in the whole process. The inter-arrival time for the wholeprocess is denoted as T . We can use the following scheme toapproximate the packet arrival process. The rv T takes on avalue of Tint with probability p, and takes on a value Toff withprobability q, where p+ q = 1. The parameters of p and q aredescribed in the limiting time-average fraction of time spent inOFF state, which is expressed as

πoffμoff

πonμon + πoffμoff=

qμoff

qμoff + pμint. (1)

Therefore, we have

p =μon

μint + μon, q =

μint

μint + μon. (2)

These two expressions are then substituted into the CDF of T ,which is FT (x) = pFint(x) + qFoff (x).

It is easy to prove that the distribution of durations ofON periods still preserves heavy-tail property. The probabil-ity that an ON period contains n inter-arrival times is pnq.

1089-7798 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

1740 IEEE COMMUNICATIONS LETTERS, VOL. 18, NO. 10, OCTOBER 2014

Summing this over n, we have Fon(x) =∑+∞

n=1 pnqF ∗n

int(x),where F ∗n

int(x) denotes the n-th convolution of Fint(x).Thereby, the distribution of the right tail of an ON period obeys

limx→+∞

Fon(x) = limx→+∞

+∞∑n=1

pnqF ∗nint(x)

= limx→+∞

qFint(x)+∞∑n=1

npn

= limx→+∞

p

qFint(x)= lim

x→+∞

μon

μintFint(x) (3)

where F (x) = 1− F (x).Since the class of subexponential distributions is closed

under tail-equivalence, the ON period under this approximationscheme is also right heavy-tailed. Accordingly, this approxima-tion scheme is proper.

C. Transmission Model

Packets in the buffer are transmitted during the whole arrivalprocess with a fixed rate of R in kByte/ms. The flow transmittedduring an inter-arrival time is B = −T ·R, in kBytes. Theminus sign indicates that the data is leaving the buffer. Hence,the CDF of B would be FB(x) = FT (−x/R).

In this letter, it is assumed that the rv’s Toff and Tint

follow truncated lognormal distributions, whose power seriesexpansion for moment generating function (MGF) is abso-lutely convergent (i.e., Toff ∼ LNT (μoff , σ

2off ) and Tint ∼

LNT (μint, σ2int), and they are defined on the respective in-

tervals of (0, Coff ] and (0, Cint]). The probability densityfunction of a truncated lognormal distribution defined on (0, C]is expressed as

gT (x) =

{e− (log x−log μ)2

2σ2

xσ√2πΦ( log C−log μ

σ )0 < x ≤ C

0 otherwise(4)

where Φ(x) is the CDF of standard normal distribution (i.e.,

Φ(x) =∫ x

−∞ e−( t2

2 ) dt). The term Φ( logC−logμσ ) works as a

scale factor so that∫ +∞0 gT (x) dx =

∫ C

0 gT (x) dx = 1.Let A ∼ Dpkt(μpkt, σ

2pkt) denote the size of a packet follows

some bimodal distribution defined on the interval of (0, Amax],in kBytes, and FA(x) denotes the CDF of A.

Define U(n = 2k − 1) = Bk, which is the quantity ofkBytes transmitted during the k-th inter-arrival time. Mean-while, define U(n = 2k) = Ak, which is the size of k-th arriv-ing packet. The subscripts k in both Ak and Bk satisfy k ∈ N+.According to previous definitions, we can tell U(n) is negativewhen n is odd and positive when n is even.

In this way, the whole process is represented as {S(n) =l0 +

∑ni=1 U(i), n ≥ 0}. Also, it is assumed that there is no

mass arriving at time zero. As a result, The value of S(n) showsthe amount of usage of the buffer at the n-th step.

D. Buffer Stopping Time

In the process {S(n), n ≥ 0}, let N be the smallest n forwhich S(n) /∈ (0, L). For each n ≥ 0, the indicator rv I{N=n}is a function of {S(0), S(1), . . . , S(n)}. Therefore, N is astopping time for {S(n), n ≥ 0}. And it is the buffer stoppingtime we discuss throughout this letter.

It is noticed that N is a non-defective rv (i.e., limh→+∞Pr{N ≥ h} = 0). Due to the fact that U(n) is not iden-tically equal to 0, there is some n for which Pr{S(n) /∈(−L,L)} > 0. For any such n, define ε as ε = max[Pr{S(n) ≤−L},Pr{S(n) ≥ L}]. For any integer d ≥ 1, it holds thatPr{N > nd|N > n(d− 1)} ≤ 1− ε. Iterating on d, we havePr{N > nd} ≤ (1− ε)d. So, N is finite with probability 1.

The rv N corresponds to another rv K, where K = N+12 �.

Here, the operation of ·� will round down the argument to thenearest integer. The value of K represents the stopping timehappens either during the K-th inter-arrival time or at the epochof the K-th packet arrival. So, finding E[K] is equivalent tofinding E[N ].

When the stopping time occurs, denote the overflow proba-bility as Po, and the empty probability as Pe. With Pe + Po =1, it is easy to find E[N ] = 2E[K]− (1− Po).

Overshoots of the stopping process at these two ends arerepresented as Oo and Oe.

III. MATHEMATICAL DERIVATIONS

Define Z0 = S(0) = l0 and Zn = Zn−1 + U(n)− E[U(n)]for all n ≥ 1. Therefore, {Zn, n ≥ 0} is a martingale.

Theorem 1:

E[K]=(1−Po) (E[Oe]+E[A])+Po (L+E[Oo])−l0

E[B]+E[A]. (5)

Proof: In the martingale of {Zn, n ≥ 0}, it is known thatE[N ] < +∞. Since U(n) is defined on finite interval, we have

E [|Zn − Zn−1||Zn−1, Zn−2, . . . , Z0]= E [|U(n)− E [U(n)]|] < +∞.

By applying the stopping time theory of martingale, we canget E[Z0] = E[ZN ]. It is noticed that E[Z0] = S(0) = l0 and

E[ZN ] = PeE [S(N)−KE[B]− (K − 1)E[A]]

+ PoE [S(N)−KE[B]−KE[A]] .

According to the definition of U(n), buffer empty can onlyoccur at odd steps and buffer overflow can only occur ateven steps. At the same time we have Pe + Po = 1. When thebuffer is emptied, E[S(N)] = E[0 +Oe] = E[Oe] ≤ 0. Sym-metrically, when buffer overflow happens, E[S(N)] = L+E[Oo] ≥ L.

Therefore,

l0 =Pe (E[Oe]− E[K]E[B]− E[K]E[A] + E[A])

+ Po (L+ E[Oo]− E[K]E[B]− E[K]E[A])

E[K] =(1− Po) (E[Oe] + E[A]) + Po (L+ E[Oo])− l0

E[B] + E[A].

�In (5), the unknown variables are Po, E[Oe], and E[Oo]. To

solve the probability of overflow, we need to construct anothermartingale.

Since both distributions of B and A are defined on finiteintervals and their MGFs exist, the MGFs of B and A areexpressed as MB(r) = E[erB ] and MA(t) = E[etA], wherer �= 0 and t �= 0.

LUAN: BUFFER STOPPING TIME ANALYSIS IN DATA CENTER NETWORKS 1741

Denote

Qodd(k) =M−kB (r)M

−(k−1)A (t)e

rk∑

i=1

U(2i−1)+tk−1∑i=1

U(2i)

(6)

Qeven(k) =M−kB (r)M−k

A (t)er

k∑i=1

U(2i−1)+t

k∑i=1

U(2i)

. (7)

Define Y0 = 1 and Yn =

{Qodd(k) n = 2k − 1Qeven(k) n = 2k

for all

n ≥ 1. Therefore, {Yn, n ≥ 0} is a martingale.Let r = t and MB(r)MA(t) = 1, and we can get the non-

zero solution of r= t=γ. Thus, in the definition of {Yn, n≥0},it holds that

∑i U(2i− 1) +

∑i U(2i) = S(n)− l0.

Theorem 2:

Po =eγl0 −MA(γ)E[e

γOe ]

eγLE[eγOo ]−MA(γ)E[eγOe ]. (8)

Proof: By the same stopping rule, we have E[N ] < +∞.And E[sup |Yh|] < +∞, where h = min{N,n}.

Applying the stopping time theory of martingale, we haveE[Y0] = E[YN ].

With r = t = γ, and notice that E[Y0] = 1 and E[YN ] =PeE[Qodd(K)] + PoE[Qeven(K)], we have

E[YN ] =PeE[MA(γ)e

γ(S(N)−l0)]+ PoE

[eγ(S(N)−l0)

]= e−γl0

((1− Po)MA(γ)E[eγOe ] + Poe

γLE[eγOo ])

=1

Po =eγl0 −MA(γ)E[e

γOe ]

eγLE[eγOo ]−MA(γ)E[eγOe ].

�Next, we will investigate the distributions of Oe and Oo,

to obtain E[Oe], E[Oo], E[eγOo ], and E[eγOe ]. We can repre-sent the process {S(n), n ≥ 0} in another way, i.e., from theperspective of random walk. Combining an odd step with thefollowing even step, we can get independent and identicallydistributed rv’s {V (k) = U(2k − 1) + U(2k), k ≥ 1}. The rvV (k) may take on both positive and negative values, andFV (x) denotes its CDF. These rv’s indicate the left heavy-tailedincrements in the non-arithmetic random walk of {W (k) =

l0 +∑k

i=1 V (i) = S(2k), k ≥ 0}. In such a random walk, thedistributions of overshoots are solvable.

In this letter, H+(x) and H−(x) denote the CDFs of ascend-ing and descending ladder height distributions of the randomwalk. Asymptotics of the ladder height distributions are adoptedto solve the distributions of overshoots, as can be found in [11]and [12].

The defective CDF of the ascending ladder height distribu-tion is

H+(x) ≈ 1

|E[B]|

x∫0

FA(y) dy, x > 0 (9)

and the scaled version can be expressed as

H+s (x) ≈ 1

|E[B]|H+(Amax)

x∫0

FA(y) dy, x > 0. (10)

As for the descending ladder height distribution,

H−(x) ≈ 1∫ 0

−∞ FV (y)dy

x∫−∞

FV (y) dy, x < 0 (11)

where FV (x) equals the convolution of CDFs of A and B (i.e.,FV (x) = (FB ∗ FA)(x)).

It is known that the distributions of overshoots in a non-arithmetic random walk equal to the residual lifetime distribu-tions of ladder height renewal processes [13]. Therefore,

Pr{Oo ≤ x} =1

μH+s

x∫0

H+s (y) dy, x > 0 (12)

Pr{Oe > x} =1

μH−

x∫0

H−(y) dy, x < 0. (13)

In a simulation check, results under such asymptotics matchperfectly with the experimental data. Hence, adoption of thoseasymptotics is valid.

Plugging values of E[Oe], E[Oo], E[eγOo ], and E[eγOe ] into(5) and (8), we will obtain E[K]. Consequently, the expectedtime taken for the stopping time to occur can be counted asE[Tstop] = E[K]E[T ] + (1− Po)E[Oe]/(−R).

IV. SIMULATION

In this simulation the parameters are set as μoff = 483.3582,μon = 483.3700, μint = 9.6674, σoff = σint = 1, Coff =5000, Cint = 100, all in milliseconds. So, we can get p =50/51 and q = 1/51. Besides, the transmission rate R is setto be 0.044 kByte/ms.

We simplify the distribution of A by assuming A to take onthe two discrete values 0.2 and 1.4 (in kBytes) with probabilitiesof p0.2 = p1.4 = 0.5.

The buffer sizes under test are 500, 1000, 10 000, and100 000, in kBytes. And the occupancies in the test are setfrom 1% to 100%, with the step of 1%. Ten thousand timessimulations are performed for each percentage point.

Through the comparison between sample averages and the-oretical values, we can demonstrate the consistency of themathematical results and the experimental situations in a buffer,shown in Figs. 2 and 3.

In Fig. 2, the stopping time climbs all the way up beforereaching its maximum, and then it decreases to zero. Themaximum point is obtained by taking the derivative of E[K].

In Fig. 3, the overflow probabilities grow up as initial occu-pancy increases, but the growth rates are different for differentbuffer sizes. With a longer buffer, the overflow probabilitywould remain low for a wider range of initial occupancy.For instance, as shown in the figure, the overflow probabilitiesremain less than 0.1, when the occupancies are lower than19.95%, 42.36%, 93.71%, and 99.37% for the four respectivebuffer sizes of 500, 1000, 10 000, and 100 000, in kBytes.Meanwhile, with a fixed initial occupancy value, the overflowprobability drops exponentially as L increases, which can bederived from (8).

The method in this letter can be applied as guidelines whendesigning a buffer in DCNs. An extensive literature introducedalgorithms of computing stationary distributions in a buffer

1742 IEEE COMMUNICATIONS LETTERS, VOL. 18, NO. 10, OCTOBER 2014

Fig. 2. Sample averages of stopping time compared with theoretical results.

Fig. 3. Sample averages of overflow probability of the stopping time com-pared with theoretical results.

[1]–[3]. Here, we can regard queue length in stationary con-dition as l0, which is the preset amount of data in the buffer. Itwould lead to a reasonable buffer size to take when comparingbuffer stopping time and overflow probability with networkrequirements.

For example, suppose the stationary queue length of a buffer is50 kBytes, and the buffer size L varies from 50 to 2000 kBytes.The initial occupancy would then be 50/L. In Fig. 4, theoreticalresults of stopping time and its overflow probability are shownunder the same transmission conditions as in simulation. Itwould be a good choice to set the buffer size in range of [800,1200] kBytes, since it can save costs and achieve relatively goodnetwork performance at the same time.

V. CONCLUSION

To sum up, in a DCN with tiered-design, switches with differ-ent abilities are required on different hierarchical levels.

Fig. 4. Stopping times and their overflow probabilities of a fixed queue lengthwith vary buffer sizes.

The method presented in this letter can be put into use fordesigning switch and understanding buffer performance. Gen-eralizations of the analysis can be realized by adjusting thetransmission model.

REFERENCES

[1] M. Cello, G. Gnecco, M. Marchese, and M. Sanguineti, “A model of bufferoccupancy for ICNs,” IEEE Commun. Lett., vol. 16, no. 6, pp. 862–865,Jun. 2012.

[2] Karmeshu and S. Sharma, “Queue length distribution of network packettraffic: Tsallis entropy maximization with fractional moments,” IEEECommun. Lett., vol. 10, no. 1, pp. 34–36, Jan. 2006.

[3] K. Jagannathan, M. Markakis, E. Modiano, and J. N. Tsitsiklis, “Queue-length asymptotics for generalized max-weight scheduling in the presenceof heavy-tailed traffic,” IEEE/ACM Trans. Netw., vol. 20, no. 4, pp. 1096–1111, Aug. 2012.

[4] G. R. de los Santos, M. Uruena, A. Munoz, and J. A. Hernandez, “Bufferdesign under bursty traffic with applications in FCoE storage area net-works,” IEEE Commun. Lett., vol. 17, no. 2, pp. 413–416, Feb. 2013.

[5] M. F. Bari et al., “Data center network virtualization: A survey,” IEEECommun. Surveys Tuts., vol. 15, no. 2, pp. 909–928, 2013, 2nd Quarter.

[6] M. Al-Fares, A. Loukissas, and A. Vahdat, “A scalable, commodity datacenter network architecture,” in Proc. ACM SIGCOMM, 2008, pp. 63–74.

[7] X. Yuan, W. Nienaber, Z. Duan, and R. Melhem, “Oblivious routing infat-tree based system area networks with uncertain traffic demands,” IEEETrans. Netw., vol. 17, no. 5, pp. 1439–1452, Oct. 2009.

[8] Y. Chen, T. Chen, and C. Huang, “SOM: Spiral-fat-tree-based on-demandmulticast protocol in a wireless ad-hoc network,” Comput. Commun.,vol. 25, no. 17, pp. 1684–1695, Nov. 2002.

[9] T. Benson, A. Akella, and D. A. Maltz, “Network traffic characteristicsof data centers in the wild,” in Proc. ACM SIGCOMM IMC, 2010,pp. 267–280.

[10] T. Benson, A. Anand, A. Akella, and M. Zhang, “Understanding datacenter traffic characteristics,” ACM SIGCOMM Comput. Commun. Rev.,vol. 40, no. 1, pp. 92–99, Jan. 2010.

[11] S. Asmussen and H. Albrecher, “Further General Tools and Results,” inRuin Probabilities, 2nd ed, vol. 14. Singapore: World Scientific, 2010,pp. 39–70.

[12] S. Asmussen and H. Albrecher, “Ruin probabilities in the presence ofheavy tails,” in Ruin Probabilities, 2nd ed, vol. 14. Singapore: WorldScientific, 2010, pp. 293–328.

[13] G. Allan, “Renewal Theory for Random Walks with Positive Drift,”in Stopped Random Walks Limit Theorems and Applications, 2nd ed.New York, NY, USA: Springer-Verlag, 1988, pp. 79–113.