RandomEarlyDetectionGatewaysfor CongestionAvoidance

SallyFloyd andVanJacobson�

LawrenceBerkeley LaboratoryUniversityof California

floyd@ee.lbl.govvan@ee.lbl.gov

To appearin theAugust1993IEEE/ACM TransactionsonNetworking

Abstract

ThispaperpresentsRandomEarlyDetection(RED)gate-ways for congestionavoidancein packet-switchednet-works.Thegatewaydetectsincipientcongestionby com-putingtheaveragequeuesize. Thegateway couldnotifyconnectionsof congestioneitherby droppingpacketsar-riving at thegatewayor by settingabit in packetheaders.Whentheaveragequeuesizeexceedsa presetthreshold,the gateway dropsor markseacharriving packet with acertainprobability, wheretheexactprobability is a func-tion of theaveragequeuesize.

REDgatewayskeeptheaveragequeuesizelow whileallowingoccasionalburstsof packetsin thequeue.Duringcongestion,theprobabilitythatthegatewaynotifiesapar-ticularconnectionto reduceits window is roughlypropor-tional to thatconnection’sshareof thebandwidththroughthegateway. REDgatewaysaredesignedto accompany atransport-layercongestioncontrol protocolsuchasTCP.The RED gateway hasno biasagainstbursty traffic andavoids the global synchronizationof many connectionsdecreasingtheirwindow at thesametime. Simulationsofa TCP/IPnetwork areusedto illustratethe performanceof REDgateways.

1 Intr oduction

In high-speednetworkswith connectionswith largedelay-bandwidthproducts,gatewaysare likely to be designedwith correspondinglylarge maximumqueuesto accom-modatetransientcongestion.In the currentInternet,theTCPtransportprotocoldetectscongestiononlyafterapackethasbeendroppedatthegateway. However, it wouldclearlybeundesirableto havelargequeues(possiblyontheorder�

This work was supportedby the Director, Office of Energy Re-search,ScientificComputingStaff, of the U.S. Departmentof EnergyunderContractNo. DE-AC03-76SF00098.

of a delay-bandwidthproduct)thatwerefull muchof thetime; this would significantlyincreasethe averagedelayin the network. Therefore,with increasinglyhigh-speednetworks,it is increasinglyimportantto havemechanismsthatkeepthroughputhighbut averagequeuesizeslow.

In theabsenceof explicit feedbackfrom thegateway,thereare a numberof mechanismsthat have beenpro-posedfor transport-layerprotocolstomaintainhighthrough-putandlow delayin thenetwork. Someof theseproposedmechanismsaredesignedto work with currentgateways[15, 23, 31, 33, 34], while other mechanismsare cou-pledwith gatewayschedulingalgorithmsthatrequireper-connectionstatein thegateway[20, 22]. In theabsenceofexplicit feedbackfrom thegateway, transport-layerproto-colscouldinfer congestionfrom theestimatedbottleneckservicetime, from changesin throughput,from changesin end-to-enddelay, aswell asfrom packetdropsor othermethods.Nevertheless,theview of anindividualconnec-tion is limited by the timescalesof the connection,thetraffic patternof the connection,the lack of knowledgeof thenumberof congestedgateways,thepossibilitiesofroutingchanges,aswell asby otherdifficultiesin distin-guishingpropagationdelayfrom persistentqueueingde-lay.

Themosteffective detectionof congestioncanoccurin the gateway itself. The gateway can reliably distin-guishbetweenpropagationdelayandpersistentqueueingdelay. Only thegateway hasa unifiedview of thequeue-ing behavior over time; theperspectiveof individualcon-nectionsis limited by thepacketarrival patternsfor thoseconnections.In addition,agatewayis sharedby many ac-tiveconnectionswith awiderangeof roundtriptimes,tol-erancesof delay, throughputrequirements,etc.;decisionsaboutthedurationandmagnitudeof transientcongestionto beallowedatthegatewayarebestmadeby thegatewayitself.

The methodof monitoringthe averagequeuesizeat

1

thegateway, andof notifyingconnectionsof incipientcon-gestion,is basedof the assumptionthat it will continueto be useful to have queuesat the gateway where traf-fic from a numberof connectionsis multiplexedtogether,with FIFO scheduling.Not only is FIFO schedulinguse-ful for sharingdelayamongconnections,reducingdelayfor aparticularconnectionduringits periodsof burstiness[4], but it scaleswell andis easyto implementefficiently.In analternateapproach,somecongestioncontrolmech-anismsthatusevariantsof Fair Queueing[20] or hop-by-hop flow control schemes[22] proposethat the gatewayschedulingalgorithmmakeuseof per-connectionstateforevery active connection.We would suggestinsteadthatper-connectiongatewaymechanismsshouldbeusedonlyin thosecircumstanceswheregatewayschedulingmecha-nismswithout per-connectionmechanismsareclearlyin-adequate.

The DECbit congestionavoidancescheme[18], de-scribedlater in this paper, is an early exampleof con-gestiondetectionat the gateway; DECbit gatewaysgiveexplicit feedbackwhentheaveragequeuesizeexceedsacertainthreshold. This paperproposesa different con-gestionavoidancemechanismat thegateway, RED(Ran-domEarly Detection)gateways,with somewhatdifferentmethodsfor detectingcongestionandfor choosingwhichconnectionsto notify of thiscongestion.

While theprinciplesbehindRED gatewaysarefairlygeneral,andRED gatewayscanbe useful in controllingtheaveragequeuesizeevenin anetwork wherethetrans-port protocolcannot be trustedto be cooperative, REDgatewaysareintendedfor a network wherethe transportprotocolrespondsto congestionindicationsfrom thenet-work. Thegatewaycongestioncontrolmechanismin REDgatewayssimplifiesthecongestioncontroljob requiredofthetransportprotocol,andshouldbeapplicableto transport-layer congestioncontrol mechanismsotherthanthe cur-rent versionof TCP, including protocolswith rate-basedratherthanwindow-basedflow control.

However, someaspectsof RED gatewaysarespecifi-cally targetedto TCP/IPnetworks. TheRED gateway isdesignedfor anetwork whereasinglemarkedor droppedpacket is sufficient to signal the presenceof congestionto the transport-layerprotocol. This is differentfrom theDECbit congestioncontrol scheme,wherethe transport-layer protocolcomputesthe fraction of arriving packetsthathave thecongestionindicationbit set.

In addition,theemphasison avoiding theglobalsyn-chronizationthatresultsfrom many connectionsreducingtheirwindowsatthesametimeis particularlyrelevantin anetwork with 4.3-TahoeBSD TCP[14], whereeachcon-nectiongoesthroughSlow-Start,reducingthewindow toone,in responseto a droppedpacket. In theDECbit con-gestioncontrolscheme,for example,whereeachconnec-tion’s responseto congestionis lesssevere,it is alsoless

critical to avoid thisglobalsynchronization.REDgatewayscanbeusefulin gatewayswith a range

of packet-schedulingandpacket-droppingalgorithms.Forexample,RED congestioncontrol mechanismscould beimplementedin gatewayswith droppreference,wherepack-etsaremarkedaseither“essential”or “optional”, and“op-tional” packetsaredroppedfirst whenthequeueexceedsacertainsize.Similarly, for agatewaywith separatequeuesfor realtimeandnon-realtimetraffic, for example,REDcongestioncontrol mechanismscould be appliedto thequeuefor oneof thesetraffic classes.

TheREDcongestioncontrolmechanismsmonitortheaveragequeuesizefor eachoutputqueue,and,usingran-domization,chooseconnectionsto notify of thatconges-tion. Transientcongestionis accommodatedby a tem-poraryincreasein thequeue.Longer-livedcongestionisreflectedby an increasein the computedaveragequeuesize,andresultsin randomizedfeedbackto someof theconnectionsto decreasetheir windows. The probabilitythata connectionis notifiedof congestionis proportionalto that connection’s shareof the throughputthroughthegateway.

Gatewaysthatdetectcongestionbeforethequeueover-flows arenot limited to packet dropsas the methodfornotifying connectionsof congestion.RED gatewayscanmarka packetby droppingit at thegateway or by settingabit in thepacketheader, dependingon thetransportpro-tocol. Whentheaveragequeuesizeexceedsa maximumthreshold,the RED gateway marksevery packet that ar-rivesat the gateway. If RED gatewaysmark packetsbydroppingthem,ratherthanby settinga bit in the packetheader, when the averagequeuesize exceedsthe maxi-mumthreshold,thentheRED gateway controlstheaver-agequeuesizeevenin theabsenceof acooperatingtrans-portprotocol.

Oneadvantageof agatewaycongestioncontrolmech-anismthatworkswith currenttransportprotocols,andthatdoesnot requirethat all gatewaysin the internetusethesamegateway congestioncontrol mechanism,is that itcouldbedeployedgraduallyin thecurrentInternet.REDgatewaysarea simplemechanismfor congestionavoid-ancethatcouldbeimplementedgraduallyin currentTCP/IPnetworkswith nochangesto transportprotocols.

Section2 discussesprevious researchon Early Ran-domDropgatewaysandothercongestionavoidancegate-ways.Section3 outlinesdesignguidelinesfor REDgate-ways.Section4 presentstheREDgatewayalgorithm,andSection5 describessimple simulations. Section6 dis-cussesin detailtheparametersusedin calculatingtheav-eragequeuesize,andSection7 discussesthe algorithmusedin calculatingthepacket-markingprobability.

Section8examinestheperformanceof REDgateways,including the robustnessof RED gateways for a rangeof traffic and for a rangeof parametervalues. Simula-

2

tions in Section9 demonstrate,amongother things, theRED gateway’s lack of biasagainstbursty traffic. Sec-tion 10describeshow REDgatewayscanbeusedto iden-tify thoseusersthatareusinga largefractionof theband-width througha congestedgateway. Section11 discussesmethodsfor efficiently implementingREDgateways.Sec-tion 12 givesconclusionsanddescribesareasfor futurework.

2 Previouswork oncongestionavoid-ancegateways

2.1 Early RandomDrop gateways

SeveralresearchershavestudiedEarlyRandomDropgate-waysasa methodfor providing congestionavoidanceatthegateway.1

Hashem[11] discussessomeof the shortcomingsofRandomDrop2 and Drop Tail gateways,andbriefly in-vestigatesEarly RandomDrop gateways. In the imple-mentationof Early RandomDropgatewaysin [11], if thequeuelengthexceedsa certaindrop level, thenthe gate-waydropseachpacketarriving atthegatewaywith afixeddrop probability. This is discussedasa roughinitial im-plementation.Hashem[11] stressesthat in future imple-mentationsthedroplevel andthedropprobabilityshouldbeadjusteddynamically, dependingonnetwork traffic.

Hashem[11] pointsout thatwith Drop Tail gatewayseachcongestionperiodintroducesglobalsynchronizationin the network. Whenthe queueoverflows, packetsareoften droppedfrom several connections,and thesecon-nectionsdecreasetheir windows at the sametime. Thisresultsin a lossof throughputat thegateway. Thepapershows thatEarly RandomDrop gatewayshave a broaderview of traffic distribution thando Drop Tail or RandomDrop gatewaysandreduceglobal synchronization.Thepapersuggeststhat becauseof this broaderview of traf-fic distribution,EarlyRandomDropgatewayshaveabet-ter chancethanDrop Tail gatewaysof targetingaggres-siveusers.Theconclusionsin [11] arethatEarlyRandomDropgatewaysdeservefurtherinvestigation.

For theversionof EarlyRandomDropgatewaysusedin thesimulationsin [36], if thequeueis morethanhalf

1Jacobson[14] proposedgatewaysto monitortheaveragequeuesizeto detectincipientcongestion,andto randomlydroppacketswhencon-gestionisdetected.Theseproposedgatewaysareaprecursorto theEarlyRandomDrop gatewaysthathave beenstudiedby several authors[11][36]. Wereferto thegatewaysin thispaperasRandomEarlyDetectionor RED gateways. RED gatewaysdiffer from theearlierEarlyRandomDrop gatewaysin severalrespects:theaverage queuesizeis measured;thegateway is not limited to droppingpackets;andthepacket-markingprobabilityis a functionof theaveragequeuesize.

2With RandomDropgateways,whenapacket arrivesat thegatewayandthequeueis full, the gateway randomlychoosesa packet from thegateway queueto drop.

full thenthegatewaydropseacharrivingpacketwith prob-ability 0.02. Zhang[36] shows that this versionof EarlyRandomDropgatewayswasnotsuccessfulin controllingmisbehaving users.In thesesimulations,with both Ran-dom Drop and Early RandomDrop gateways, the mis-behaving usersreceived roughly 75% higher throughputthantheusersimplementingstandard4.3BSD TCP.

TheGatewayCongestionControlSurvey [21] consid-erstheversionsof Early RandomDrop describedabove.The survey cites the resultsin which the Early RandomDropgateway is unsuccessfulin controllingmisbehavingusers[36]. As mentionedin [32], Early RandomDropgateways are not expectedto solve all of the problemsof unequalthroughputgiven connectionswith differentroundtriptimesandmultiplecongestedgateways.In [21],thegoalsof EarlyRandomDropgatewaysfor congestionavoidancearedescribedas“uniform, dynamictreatmentof users(streams/flows), of low overhead,and of goodscalingcharacteristicsin largeandloadednetworks”. It isleft asanopenquestionwhetheror not thesegoalscanbeachieved.

2.2 Other approachesto gatewaymechanismsfor congestionavoidance

Early descriptionsof IP SourceQuenchmessagessug-gestthat gatewayscould sendSourceQuenchmessagesto sourcehostsbefore the buffer spaceat the gatewayreachescapacity[26], andbeforepacketshavetobedroppedat thegateway. Oneproposal[27] suggeststhat thegate-way sendSourceQuenchmessageswhenthe queuesizeexceedsacertainthreshold,andoutlinesapossiblemethodfor flow control at the sourcehostsin responseto thesemessages.Theproposalalsosuggeststhatwhenthegate-way queuesizeapproachesthe maximumlevel thegate-waycoulddiscardarrivingpacketsotherthanICMP pack-ets.

The DECbit congestionavoidancescheme,a binaryfeedbackschemefor congestionavoidance,is describedin [29]. In theDECbitschemethegatewayusesacongestion-indicationbit in packetheadersto providefeedbackaboutcongestionin the network. Whena packet arrivesat thegateway, thegateway calculatestheaveragequeuelengthfor the last (busy+ idle) periodplusthecurrentbusype-riod. (Thegatewayis busywhenit is transmittingpackets,and idle otherwise.)Whenthe averagequeuelengthex-ceedsone,thenthegatewaysetsthecongestion-indicationbit in thepacketheaderof arriving packets.

Thesourceuseswindow flow control,andupdatesitswindow onceeverytwo roundtriptimes.If at leasthalf ofthepacketsin thelastwindow hadthecongestionindica-tion bit set, thenthe window is decreasedexponentially.Otherwise,thewindow is increasedlinearly.

ThereareseveralsignificantdifferencesbetweenDECbit

3

gatewaysandtheRED gatewaysdescribedin this paper.Thefirst differenceconcernsthemethodof computingtheaveragequeuesize.BecausetheDECbit schemechoosesthe last (busy + idle) cycle plus the currentbusy periodfor averagingthe queuesize, the queuesize can some-timesbe averagedover a fairly shortperiodof time. Inhigh-speednetworkswith largebuffersat thegateway, itwould bedesirableto explicitly control thetime constantfor thecomputedaveragequeuesize;this is donein REDgatewaysusingtime-basedexponentialdecay. In [29] theauthorsreport that they rejectedthe idea of a weightedexponentialrunningaverageof thequeuelengthbecausewhen the time interval was far from the roundtrip time,therewasbiasin thenetwork. This problemof biasdoesnot arisewith RED gatewaysbecauseRED gatewaysusea randomizedalgorithmfor markingpackets,andassumethat the sourcesusea differentalgorithmfor respondingto markedpackets.In aDECbitnetwork, thesourcelooksat thefractionof packetsthathavebeenmarkedin thelastroundtrip time. For a network with RED gateways, thesourceshouldreduceits window evenif thereis only onemarkedpacket.

A seconddifferencebetweenDECbit gateways andREDgatewaysconcernsthemethodfor choosingconnec-tionsto notify of congestion.In theDECbitschemethereis no conceptualseparationbetweenthealgorithmto de-tectcongestionandthealgorithmto setthecongestionin-dicationbit. Whenapacketarrivesat thegatewayandthecomputedaveragequeuesizeis too high, thecongestionindicationbit is setin theheaderof thatpacket. Becauseof thismethodfor markingpackets,DECbitnetworkscanexhibit a biasagainstburstytraffic [seeSection9]; this isavoidedin RED gatewaysby usingrandomizationin themethodfor marking packets. For congestionavoidancegatewaysdesignedto work with TCP, anadditionalmoti-vationfor usingrandomizationin themethodfor markingpacketsis to avoid theglobalsynchronizationthatresultsfrom many TCPconnectionsreducingtheirwindow atthesametime. This is lessof a concernin networkswith theDECbitcongestionavoidancescheme,whereeachsourcedecreasesitswindow fairly moderatelyin responsetocon-gestion.

Anotherproposalfor adaptivewindow schemeswherethe sourcenodesincreaseor decreasetheir windows ac-cordingto feedbackconcerningthe queuelengthsat thegatewaysis presentedin [25]. EachgatewayhasanupperthresholdUT indicatingcongestion,andalowerthresholdLT indicatinglight loadconditions.Informationaboutthequeuesizesat the gateways is addedto eachpacket. Asourcenodeincreasesits window only if all thegatewayqueuelengthsin thepatharebelow thelower thresholds.If the queuelengthis above the upperthresholdfor anyqueuealongthe path,thenthe sourcenodedecreasesitswindow. Onedisadvantageof thisproposalis thatthenet-

work respondsto the instantaneousqueuelengths,not tothe averagequeuelengths. We believe that this schemewould bevulnerableto traffic phaseeffectsandto biasesagainstbursty traffic, andwould not accommodatetran-sientincreasesin thequeuesize.

3 Designguidelines

This sectionsummarizessomeof the designgoalsandguidelinesfor REDgateways.Themaingoalis to providecongestionavoidanceby controlling the averagequeuesize. Additional goals include the avoidanceof globalsynchronizationandof abiasagainstburstytraffic andtheability to maintainanupperboundon theaveragequeuesize even in the absenceof cooperationfrom transport-layerprotocols.

Thefirst job of a congestionavoidancemechanismatthegateway is to detectincipientcongestion.As definedin [18], acongestionavoidanceschememaintainsthenet-work in a region of low delayandhigh throughput.Theaveragequeuesizeshouldbekeptlow, while fluctuationsin the actualqueuesizeshouldbe allowed to accommo-datebursty traffic andtransientcongestion.Becausethegateway canmonitor thesizeof thequeueover time, thegateway is the appropriateagentto detectincipient con-gestion. Becausethe gateway hasa unified view of thevarioussourcescontributing to this congestion,thegate-way is alsotheappropriateagentto decidewhichsourcesto notify of thiscongestion.

In anetworkwith connectionswith arangeof roundtriptimes, throughputrequirements,and delay sensitivities,thegatewayis themostappropriateagentto determinethesizeanddurationof short-livedburstsin queuesizeto beaccommodatedby thegateway. Thegateway cando thisby controllingthetimeconstantsusedby thelow-passfil-ter for computingtheaveragequeuesize.Thegoalof thegatewayis to detectincipientcongestionthathaspersistedfor a “long time” (severalroundtriptimes).

Thesecondjob of a congestionavoidancegateway isto decidewhich connectionsto notify of congestionatthe gateway. If congestionis detectedbeforethe gate-way buffer is full, it is not necessaryfor the gateway todroppacketsto notify sourcesof congestion.In this pa-per, we saythat thegateway marksa packet, andnotifiesthesourceto reducethewindow for thatconnection.Thismarkingandnotificationcanconsistof droppingapacket,settinga bit in a packetheader, or someothermethodun-derstoodby the transportprotocol. Thecurrentfeedbackmechanismin TCP/IPnetworksis for thegatewayto droppackets,andthesimulationsof RED gatewaysin this pa-perusethisapproach.

Onegoalis to avoid a biasagainstburstytraffic. Net-workscontainconnectionswith a rangeof burstiness,and

4

gatewayssuchasDrop Tail andRandomDrop gatewayshave a biasagainstbursty traffic. With Drop Tail gate-ways, the morebursty the traffic from a particularcon-nection,themorelikely it is that thegateway queuewilloverflow whenpacketsfrom thatconnectionarrive at thegateway[7].

Anothergoalin decidingwhich connectionsto notifyof congestionis to avoid the global synchronizationthatresultsfrom notifying all connectionsto reducetheirwin-dows at thesametime. Globalsynchronizationhasbeenstudiedin networkswith DropTail gateways[37], andre-sultsin lossof throughputin the network. Synchroniza-tion asa generalnetwork phenomenahasbeenexploredin [8].

In ordertoavoidproblemssuchasbiasesagainstburstytraffic andglobal synchronization,congestionavoidancegatewayscan usedistinct algorithmsfor congestionde-tectionand for decidingwhich connectionsto notify ofthiscongestion.TheREDgatewayusesrandomizationinchoosingwhicharrivingpacketstomark;with thismethod,theprobabilityof markingapacket from aparticularcon-nectionis roughlyproportionalto thatconnection’sshareof thebandwidththroughthe gateway. This methodcanbeefficiently implementedwithoutmaintainingper-connectionstateat thegateway.

Onegoal for a congestionavoidancegateway is theability to control the averagequeuesizeeven in the ab-senceof cooperatingsources. This can be doneif thegateway dropsarriving packetswhen the averagequeuesizeexceedssomemaximumthreshold(ratherthan set-ting a bit in the packet header). This methodcould beusedto control theaveragequeuesizeeven if mostcon-nectionslast lessthan a roundtrip time (as could occurwith modified transportprotocolsin increasinglyhigh-speednetworks), and even if connectionsfail to reducetheir throughputin responseto markedor droppedpack-ets.

4 The RED algorithm

This sectiondescribesthe algorithmfor RED gateways.TheRED gateway calculatesthe averagequeuesize,us-ing a low-passfilter with an exponentialweightedmov-ing average.Theaveragequeuesizeis comparedto twothresholds,a minimumthresholdanda maximumthresh-old. Whenthe averagequeuesize is lessthan the min-imum threshold,no packets aremarked. When the av-eragequeuesizeis greaterthanthemaximumthreshold,everyarriving packet is marked. If markedpacketsareinfact dropped,or if all sourcenodesarecooperative, thisensuresthattheaveragequeuesizedoesnot significantlyexceedthemaximumthreshold.

When the averagequeuesize is betweenthe mini-

mum and the maximumthreshold,eacharriving packetis markedwith probability ��� , where��� is a functionofthe averagequeuesize ���� . Eachtime that a packet ismarked, the probability that a packet is marked from aparticularconnectionis roughlyproportionalto thatcon-nection’sshareof thebandwidthat thegateway. Thegen-eralREDgatewayalgorithmis givenin Figure1.

for each packet arrivalcalculate the average queue size ����if � ��������������������������

calculate probability ���with probability ��� :

mark the arriving packetelse if � ��������������

mark the arriving packet

Figure1: Generalalgorithmfor REDgateways.

Thusthe RED gateway hastwo separatealgorithms.Thealgorithmfor computingtheaveragequeuesizedeter-minesthedegreeof burstinessthatwill beallowedin thegatewayqueue.Thealgorithmfor calculatingthepacket-markingprobabilitydetermineshow frequentlythegate-way markspackets,giventhecurrentlevel of congestion.Thegoalis for thegatewaytomarkpacketsatfairly evenly-spacedintervals, in order to avoid biasesand to avoidglobal synchronization,andto mark packetssufficientlyfrequentlyto controltheaveragequeuesize.

Thedetailedalgorithmfor theRED gateway is givenin Figure2. Section11 discussesefficient implementa-tionsof thesealgorithms.

Thegateway’s calculationsof theaveragequeuesizetake into accounttheperiodwhenthequeueis empty(theidle period)by estimatingthenumber� of smallpacketsthat could have beentransmittedby the gateway duringthe idle period. After the idle period the gateway com-putesthe averagequeuesizeasif � packetshadarrivedto anemptyqueueduringthatperiod.

As ���� variesfrom ��� � ��� to � ��� ��� , thepacket-markingprobability ��! varieslinearly from 0 to � ���#" :� !%$ � ��� "#& ����(')��� �����*,+ & � �������-')��� ������*/.Thefinal packet-markingprobability ��� increasesslowlyasthecountincreasessincethelastmarkedpacket:� � $ ��! + &10 ')24365���798 ��! *As discussedin Section7, this ensuresthat the gatewaydoesnotwait too longbeforemarkingapacket.

Thegatewaymarkseachpacketthatarrivesatthegate-waywhentheaveragequeuesize ���� exceeds�������� .

One option for the RED gateway is to measurethequeuein bytesratherthan in packets. With this option,

5

Initialization:���� $;:2<365���7 $ ' 0for each packet arrival

calculate new avg. queue size ���� :if the queue is nonempty��� $ &10 '>=@?A*B����DCE=@?GFelse� $;H & 7I� ��J@'KF 7I� � JL*��� $ &10 '>=@?A*BMN����

if ��� �����O�P����Q��� �������increment 24365���7calculate probability � � :��! $ � ���R" & ����(')��� � ��� *,+ & � ��� ��� ')��� � ��� *� � $ ��! + &10 ')2<365���798 ��! *with probability � � :

mark the arriving packet2<365���7 $S:else if � ��������������

mark the arriving packet2<365���7 $;:else 2<365���7 $ ' 0

when queue becomes emptyF 7I� � J $ 7I� � JSavedVariables:���� : average queue sizeF 7I� � J : start of the queue idle time2<365���7 : packets since last marked pkt.

Fixed parameters:= ? : queue weight��� � ��� : minimum threshold for queue� ��� ��� : maximum threshold for queue� ���#" : maximum value for ��!Other:��� : current pkt-marking probabilityF : current queue size7I� � J : current timeH & 71* : a linear function of the time 7

Figure2: Detailedalgorithmfor REDgateways.

theaveragequeuesizeaccuratelyreflectstheaveragede-lay at the gateway. Whenthis option is used,the algo-rithm wouldbemodifiedtoensurethattheprobabilitythata packet is marked is proportionalto the packet size inbytes:��! $ �����R" & ����T'>� � � ��� *U+ & � ��� ��� '>� � � ��� *� !V$ � !XWGYZ\[]<^/_a`cbd] +Le YgfR`ch(i#h�WGYZ\[]4^\_a`jb4]

��� $ � ! + &B0 'K2<365���7k8 � ! *In thiscasealargeFTPpacketis morelikely to bemarkedthanis a smallTELNET packet.

Sections6 and7 discussin detailthesettingof thevar-ious parametersfor RED gateways. Section6 discussesthecalculationof theaveragequeuesize.Thequeueweight= ? is determinedby the size and durationof bursts inqueuesize that are allowed at the gateway. The mini-mumandmaximumthresholds� � ����� and �������� arede-terminedby thedesiredaveragequeuesize. Theaveragequeuesizewhichmakesthedesiredtradeoffs (suchasthetradeoff betweenmaximizingthroughputandminimizingdelay)dependson network characteristics,andis left asa questionfor further research.Section7 discussesthecalculationof thepacket-markingprobability.

In this paperour primary interestis in the functionaloperationof theRED gateways.Specificquestionsaboutthe mostefficient implementationof the RED algorithmarediscussedin Section11.

5 A simplesimulation

Thissectiondescribesoursimulatorandpresentsasimplesimulationwith REDgateways.Oursimulatorisaversionof theREAL simulator[19] built onColumbia’sNestsim-ulationpackage[1], with extensivemodificationsandbugfixes madeby Steven McCanneat LBL. In the simula-tor, FTPsourcesalwayshaveapacket to sendandalwayssenda maximal-sized(1000-byte)packet assoonasthecongestioncontrolwindow allows themto do so. A sinkimmediatelysendsanACK packetwhenit receivesadatapacket. ThegatewaysuseFIFOqueueing.

Sourceandsink nodesimplementa congestioncon-trol algorithmequivalentto that in 4.3-TahoeBSD TCP.3

Briefly, therearetwo phasesto thewindow-adjustmental-gorithm. A thresholdis setinitially to half the receiver’sadvertisedwindow. In the slow-startphase,the currentwindow is doubledeachroundtriptime until thewindowreachesthethreshold.Thenthecongestion-avoidancephaseis entered,andthecurrentwindow is increasedby roughlyonepacket eachroundtriptime. Thewindow is never al-lowed to increaseto morethanthe receiver’s advertisedwindow, whichthispaperrefersto asthe“maximumwin-dow size”. In 4.3-TahoeBSDTCP, packetloss(adroppedpacket) is treatedas a “congestionexperienced”signal.Thesourcereactsto apacket lossby settingthethresholdto half thecurrentwindow, decreasingthecurrentwindowto onepacket,andenteringtheslow-startphase.

3Oursimulatordoesnotusethe4.3-TahoeTCPcodedirectlybut webelieve it is functionallyidentical.

6

Queue size (solid line) and average queue size (dashed line).lTimem

Qu

eu

en

0.0 0.2 0.4 0.6 0.8 1.0

0o 10

30

max-thpmin-thq

.................................................................................................................................................................

..................

..............................................................................................................................

................

....................................................................................................................

.

.

.........................................................................................................................

..........

..........

.................

.......................................................................

.

.

...............

..........

..........

.................

..........

..........

...........

...............

..........

.

..........

..........

..........

..........

..........

..........

..........

..........

..........

..........

..........

..........

..........

..........

..........

..........

..........

..........

..........

..........

..........

..........

..........

..........

..........

..........

..........

.........

.

.

.

..................

..................

.............

............

..........

..........

..........

.

.

..

..

.

..................

....

.

.

..................

..................

.............

..........

..........

......

.

.

..

..

.

.

..

..

.

...

.

..............

..............

..............

..............

......

.

.

..........

..........

.........

.

.

..

..

.

.

..

..

.

.

..

..

.

.

..

..

.

...

.

...

.

...

.

......

......

......

......

......

......

.......

............

.............

..........

....

.

.

..

..

.

.

..

..

.

.

..

..

.

.

..

..

.

.

..

..

.

.

..

..

.

.

..

..

.

.

..

..

.

...

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

..

.

..

.

..

.

..

.

..

...

......

............

..........

.

.

..

..

.

.

..

..

.

.

..

..

.

.

..

..

.

.

..

..

.

.

..

..

.

.

..

..

.

.

..

..

.

.

..

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

..

.

..

.

.

..

.

.

.

..

.

.

.

..

.

.

.

.

.

.

.

.

.

..

.

...

.

...

.

.

.

.

.

..

.

..

..

..

.

..

..

..

..

..

..

..

..

..

..

.

..

.

.

.

..

.

.

.

..

.

.

.

..

.

.

.

..

.

.

.

..

.

.

.

..

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

...

.

.

.

.

.

.

.

.

..

.

..

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

...

.

.

.

.

.

..

.

..

.

.

.

.

.

.

...

.

..

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

..

.

...

.

.

.

.

.

.

..

.

.

..

.

.

..

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

..

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

..

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

..

.

.

.

.

.

...

.

......

.

...

..

.................

.

..

.

.....

..........

.

.

.

..

...........

................

.............

..

.

.

...

...

...................

..............

.........

............

.....................

..........

...............

.................

.

.

.

.......

...........

................

..........

..................

.

.

.

..

.

..................

.................

....

.................

......

...

................

..

.

...

..

..

.

.

...

.

........

.

.

.

.

.

.

.............

..........

...............

.....................

.

...

.

..

.

....

..

.

.

.

.

.

...

.

.........

.

.

.

.

.

.

.

.......

.

.

.

....

.

.

.

....

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.....

.....

.......

...........

.

.

.

......

.

...

.

..

.

.....

.

.

..

..

.

.

.

.

.

.

.

.

..

.

.

.

.

..

.

..

.

..

.

.

.

...

.

...

.

......

.....

....

.

..

.

.

.

.

.

.

.

.

............

................

..

..............

.

.

..

..

.

.

.

......

.

...

..

....

.

.

.

...

.

...

..

....

.

..

..

...

..

.

.

.

.

.

...

.

...

.

...

.

....

...

..

....

.

.

.

..

.

.

.

.

.

.

.

.

...........

...............

........

....

.

.

.

.

....

.

...

.

....

.

.....

..

......

.

.

.

...

.

....

.

.

.

..

.

..

.

.

..

.

.

.

..

.

.

..

..

..

.

.

..

.

.

..

..

..

.

.

..

..

.

..

..

.

.

.

..

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

..

...

.

.

.

..

.

.

.

.

..

.

...

.

...

.

....

..

...

..

...

..

..

.........

..

.

.

.

..

.

..

.

.

.

..

.

.

.

....

.

.

.

....

.

...

.

.

.

.....

.

.

.

....

..

.....

.

..

.

.

.

.

..

.

.

.

....

.

..

.

.

.

..

..............

.

.

.

.

..

..

.

....

..

.

.

.

...

.

.

.

..

.

..

..

.

.

.

.

..

.

.

.

.

.

..

.

.

.

.

.

.

...

.

.

.

.

.

.

..

.

.

.

.

..

..

.

.

.

.

.

.

..

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

..

..

.

.

..

.

.

.

.

.

.

.

.

.

.

...

.

.

.

..

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

..

.

.

.

.

....

.....................

..

.

....

...

.

.

...

..

..

..

..

....

...

.

.

..

..

.

....

..

...

.

..

.

.

...

..

..

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

..

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

..

.

.

.

.

.

..

.

.

.

.

.

.

..

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

..

.

...

.

..

.

..

...

.......

.........................

.

.

..................

.

.

...

.

..

...

..

..........

..

.

...............

..............

.....

..

.

..

.

..

.

..

.

...

............

.

.....

.................

..........

..................

.......

..............

.....................................

.

..................

...

.

..

..

..

....

.

.

.

.

.

.

.

.

.

.

.

.

.........

........

.

.

.

.

.

....

.......................

.

..

.

.

..

..

....

.

...

.

........

..

........

..

...

..........

...............

..

.

.

.

...

..

..

...

...

.

...

..

..

.

.

.

.

......

.

.

.

.

..

.

.

.

.

.

.

.

.

..

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

..

...

.

.

.

.

.

..

.

.

.

.

...

.

.

.....

...

.

.

............

.................

..........

.

.

.

.

....

.

...

.

.

.

....

.

...

..

....

.

.

..

.

..

.

......

.

.

.

.

..

..

.

...

.

.....

.

.

.

..

.

...

.

..

.

..

.

..

.

.

.

.

.

.

..

..

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

...

.

...

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.......

.........

...........

..

........................................

...

.

..

.

....

.

.

.

..

..

.

.

.

.

.

.

..

.

.

.

.

.

.

..

.

.

.

.

.

.

..

.

.

.

.

.

..

.

..

.

.

.

.

..

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

...

.

.

..

.

..

.

..

.

.

.

..

.

.

..

.

.

..

.

...

.

.

..

.

.

.

..

...

.....

.

.

.

.

.

.

.

..

.

.

.

...

.

.

..

.

.

.

.

.

.

.

.

.

..

.

.

.

..

.

.

.

.

.

..

.

.

.

...

.

.

.

..

.

.

.

.

.

..

..

.

.

.

.

.

...

.

.....

........

.

..

.

.

.

.

..

.

..

.

.

.

.....

.

..

.

.

...

...

........

...

.......

.....

.

....

.

...

.

......

..

.

.

...

.

.....

.

...

..

.

..

.

.

.

..

..

..

..

......

.

...

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.....

.

.

..

.

.

.

.

..

.

.

.

.

.

...............

...

.

.

.

.

.....

.

.

..

..

....

..

.

.

.

.

.

.

.

...

.

.........

..

.

..

..

..

.

.

....

.

...

.

.

..

..

.

.

.

.

.

..

...

.

.

.

.

.

.

.

.

.

.

.

...

.

.

.

.

..

.

...

.

..

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

...............

.........................................

....

.

.

..

.

.

.

.

....

.

..

.

...

.

...

.

..................

..

.

.

.

..

..

..

.

..

....

...

...

.

.

......

..........

...

.

.

..

..

.

.

.

...............

.

.

.

.

.

.

.

.

...

..

..

.

.

.

.

.

..

.

.

..

..

.

.

....

.

........................

..

.

....

.

...

.

.

..

.

.

.

......

.

.

.

.

...........

..

.

..

................................

..

.

...

.

.

.

....

.

..

.

.

.

.

.

.

..........

.

.

..

.

.

..

.

.

..

..

..

.

...

..

..

.

..

...

.

.

.

.

.

.

..

..

..

..

.

.

..

.

.

.....

.

.

.

.

.

.

.

................

...............................

....

.

..

.

...

.

....

.

.

.

....

.

...

..

.

.

..

.

..

.

.

.

.

..

.

..

.

.

.

.

..

.

.

.

.

..

.

.

.

..

.

.

.

..

..

..

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

..

.

..

.

.

.

.

.

..

.

.

..

.

.

.

.

.

..

.

..

.

......

.............

..

.

.

.

..

.

.

.

..

.

.

.

.

..

.

..

..

..

.

.

..

.

.

.

.

.

.

.

.

..

.

.

..

.

.

..

.

.

...

..

.

..

.

.

.

.

..

.

.

...

.

.

.

.

.

.

.

.

.

..

..

..

..

.

.

..

.

.

..

.

.

.

.

.

.

......

.

.

..

.

...

.

........

.

...

.

...

.

.

.

.

..

......

..

.

.

.

.

..

.

.

.

...

...

..

....

.

.

..

.

..

.

.

.

.

.

.

.

.

..

.

..

..

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

...

.

....

.

.

.

.

..

.

.

.

.

...

.

...

.

.

..

....

.

.

.

..

.

.......

...........

...............

.

...

..

...

.

.....

.

.

...

.

.

..

..

.

.

.

.

..

.

..

.

.

.

.

.

.

..

.

...

.

..

.

.

.

.

..

.

..

.

..

.

.

...

..

...

.

.

.

.

.

.

.

..

.

..

.

...

.

.

.

.

.

.

.

.

.

.

.

..

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

..

.

.

...

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

..

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

..

.

.

.

.

.

.

..

.

.

.

.

.

.

..

.

.

.

.

.

.

...

...

..

..

..

....

...

.

.

.

...

.

..

..

.

.

.....

.

.

.

.

.

.

.

..

.

....

.

...

.

.

.

.....

.

....

.

.......

.

.

.

.

.

.

....

.

.......

.

.

..

..

..

.

......

.

...

.

..

.

..

..

....

..

..

.

..

.

..

.

.....

.

.....

.

....

.

.....

.

..

.

..

.

..

.

....

...

...

.

...

.

.....

.

.

.

.

.

....

.

.

..

...

.

.

.

..

...

.

..

...

.....

.

.

..

.

.

.

..

.

.

.

.

.

..

.

.

...

.........

.

.

..

.

.

.

..

.

..

...............

.....

..............

........

.........

.....

.

.

.

.

.

..

.

.

..

..

.

.

.

..

..

..

.

..

.

...

.

.

.

...

.

.

..

.

.

.

.

.

.

.

.

.

.....

.

.

.

.

.

.

.

.

.

..

.

.

.

...

..

..

..

..

.

..

.

.

..

.

.

...

.

..

.

.......................

..

.

.

.

...

...

..

..

.

...

.

.

..

.

.

..

.

.

.

.

.

..

..

.

.

.

.

..

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

..

..

.

.

.

....

.

.....

..

.......

.

.

.

.

.

.

....

.

..

.

.

.

.

.

.

....

.

.

..

.

..

.

..

.

..

.

.....

.

.....

.

.

.

....

.

.

..

.

.

...

.

..

....

..

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

..........

.............

.......

...

.

.

...

.

.

..

..

.

.

.

......

.

...

.

.........

.

.......

.

..

.

.

..

.

...

.

.

..

.

..

.

.

.

..

.

.

.

.

.

..

..

.

.

.

.

..

.

..

.

..

.

.

.

.

.

.

...

.

.

..

.

.

.

.

.

..

.

..

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.....

.

..

.

......

.

.

..........

.

.

.

.........

.

....

.

.

..

....

.

.

.

.

...

.

.

.

.....

.

.

.

...

.

..

.

..

..

..

.

..

.

.

.

.

.

.

.

.

.

.

..

.

.

.

..

..

..

.

..

..

...

.

....

.

..

...

....

.

.....

..

.

....

.

........

........

.

.

.

.

.

.

.

..

....

.

.

.

...

.

....

.

...

.

.

.

.

...

.

..

.

..

.

..

.

.

.

..

..

.

.

.

..

.

.

..

.

.

.

..

.

...

.

.

...

..

...

.

.

.

...

.

.

.

.

.

.

.

..

.

...

.

.

.

.

..

.

.

.

.

..

.

..

.

.

...

.

..

..

..

..

......

.

..

.

..

.

....

..

.......

..

.....

.

..

.

....

.

.

.

...

.

.......

.

..

..

..

.

.

.

.

.

...

.

.

.

.

...

.

.

.

..

.

..

...

.

.

....

.

....

..

.

.

..

.......

.

.

.

.

..

.

.

.

.

..

.

....

..

.....

..

..

.

.

.

....

.

.......

.

.

.

.

.

........

.

...

.

.

..

.

.....

.

.

.

...

..

......

.

..

..

...

.

....

..

...

.

.

.

...

.

..

.

..

...

.....

.

.

.

..

.

.

..

.

..

.

.

.

.

.

.

..........

..............

..............

.

..

.

.....

.

..

.

....

..

..

.

....

.

.

.

..

.

.

..

..

.

.

..

..

..

.

..

.

.

.

..

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

...

..

.

..

.

...

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

...

.

...

..

....

....

.

......

.

...

.

....

..

........

.

.....

.

....

..

..

.

.

.

...

.

.

.

.

.

.

.

.

.

.

.

...

.

.

.

.....

.

......

.

.......

..

.....

..

........

..

..

.

...

..

..

.

....

..

..

.

..

.

.....

.

.

.

...

..

.

.

..

.

...

.

..

.

......

.

.

.

.

.

..

..

..

.

.

..

.

..

..

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

...

.

.

.

.

.

.

..

.

.

.

.

.

.

..

..

.

..

.

..

.

..

..

...

............

..

.....

.....

.

.

.

.

.

....

.

.

.

.

.....

.

..

.

.

...

..

..

.

...

.

.

.

.

.

.

...

.

.

..

...

............

..

..

......

............

.

..

..

...

.

..........

.

..

.

..

...

.

.........

.....

....

...

..

...

.

.

.

.

.

.

.

.

.

..

..

.........

....

.

.

..

..

...

..

.

.

.

.

.

.

..

.

..

..

........

.

.........

.......

.....

.

.

.

.

..

.

...

.

...

.

...

.

.

..

.

..

..

............

.

.

..

.

.

.

..

.

.

..

.

.

..

...

.

.

.

..

..

.

.

..

....

.......

......

.

..

..

.

.................

.

.

..

.

.

..

.

.

....

...

.

......

..

...

.

.

.

.

..

.

.....

......................

.

.

.

.

.....

...

...

.

.

...

.

......

......

.

..

..

..

.

.

.

.

.

..

.

.

..

.

..

.

.....

.

....

.

...

.

.

.

.

.

.

.

..

....

..

.

.....................

.

.

..

..

....

.

..........

..

.

.

.

.

....

.

..

..

..

.

.

..

..

.

.

..

..

.

..

.

.

.

..

.

.

.

..

.

.

.

.

.

.

..

...

..

..

.

..

..

.

.

.

.

..

.

.

.

..........

.

.

.

..

.

.

..

.

...

.

.

.

..

.

....

.

..

.

.

.

....

........

..

.

.

.

.

..

.

.

.

.

.

.

.

...

.

..

.

.

...

.

...

.

..

...

.

...

.

...

..

.

.

..

.

...

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

....

.

..

.

.

..

..

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

..

...

...

.

..

.

...

.

.....

.

...

.

.

.

.

.

..

.

...

..

..

.

..

..

.

.

.

.

.

.

.

.

.

......

.

..

..

.

...

.

..

.

.

.

..

..

.

...

.

.

.

.

.

....

..

.

.

.

.

.

.

.

.

.

..

..

..

.

.

..

.

..

..

..

..

..

..

.

..

.

..

.

..

.

.............

....

.

.

.

.

..

..

.

.

.

..

...

..

...

.

.

.

..

.

.

..

.

.

.

.

.

..

..

.

.

.

.

..

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

..

.

.

.

.

.

.

..

.

..

.

.

.

..

.

.

.

.

.

.

..

.

.

.

.

..

.

.

..

.

.

.

..

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

........

........

.

...

..

..

..

.

.

.

.

.

..

.

..

..

.

.

.

.

..

.

.

...

.

..

.

....

.

.

.

.

..

.

..

.

...

.

.

..

..

.

.

.

.....

.

..

.

.

.

..

.

.

.

..

..

.

.

..

..

.

.

..

.

.

..

.

.

.

.

..

..

...

.

..

..

.

.

.

.

..

.

.

..

.

.

...

.

.

.

..

.

..

..

.

.

.

.

.

.

.

.

.

.

...

.

..

.

.

.

.

..

..

.

.

.

...

.

.

.

.

.

.

..

.

..

.

.

.

.

.

.

.

.

.

..

.

..

.

.

.

.

.

.

.

.

.

.

..

...

.

.

.

.....

..

..

.

..

..

.

.

.

.

.

..

.

....

.

.

...

.

..

..

.

.

.

....

..

..

.

.

.

.

..

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

...

.

.

..

.

...

....

.

.

.

.

..

.

........

....

..

..

.

.

.....

.

.

.

.

.

.

.

.

.

..

..

.

..

...

.

.

.

.

.

.

.

..

.

..

.

..

..

.

.

.

.

.

..

.

..

.

..

.

.

.

.

.

.

...

.

.

..

.

.

.

.

.

..

.

..

.

..

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

..

.

.

.

.

.

..

.

.

.

.

..

.

.

..

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

..

.

.

.

.

.

..

.

.

.

.

.

.

..

.

.

.

.

.

.

..

.

..

.

.

..

.

.

.

.

.

.

.

..

.

...

.

.

.

.

...

.

..

..

.

.

.

.

..

.

..

.

..

..

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

..

.

..

.

..

.

.

.

.

....

.

..

.

.

.

.

.

...

..

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

..

.

..

.

...

.

.

.

..

.

.

.

..

.

.

.

...

..

..

.

..

.

.

.

.

.

.......

.

.

...

.

......

.

.

.

.

.

.

.

.

.

...

.

..

.

.

.

..

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

...

.

..

.

.

.

..

.

.

..

.

.

.

.

..

..

.

.

.

.

.

.

.

..

..

.

.

..

.

.

..

....

..

.

.

.

.

.

.

.

.

.

.

....

...

.

.

......

.

.

....

.

.

.

.

.

.

.......

.

.

.

.

.

...

.

.

.

.

..

.

.

.

.

.....

.

....

..

.

.

.

.

.

.

....

.

.

.

.

.

.

..

.

.

...

.

.

.

.

.

.

..

.

.

.

.

.

..

.

.

.

.

.

..

.

.

.

..

.

.

.

.

..

.

.

.

.

.

.

..

..

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

..

..

.

.

.

.

.

.

.

.

....

.

.

.

.

.

.

.

.

..

.

.

.

.

..

...

.

.

..

.

...

.

..

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

...

.

.

..

.

.

...

.

...

.

..

.

..

.

.......

.

.

.

...

..

....

..

....

.

.

.

.

..

..

.

.

.

.

.

.

.

.

..

.

.

.

.

.

..

.

.

.

.

.

.

.

..

.

..

.

.

.

.

.

.

.

.

.

.

.

....

.

...

.

.......

.

.

.

.

.

..

..

..

.

.

..

.....

.

..

.....

..

.

...

.

..

...........

.

.

.

.........

....

.

.

.

..

..

...

.

.

..

.

...

..

.

.

.

.

...

.

.

..

.

....

.

.....

.

.

.

..

.

.

.

.

.

.

..

.

.

.

.

.

.

....

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

........

..

.

.

.

.

.

.

..

..

.

.

.

.

...

.

.

.

..

..

.

.

..

.

.

..

.

........

.

...

.

......

.

..

.

..

.

.

.

.

.

.

..

..

.

...

..

.

.

..

.

.

.

.

.

..

.

.

.

.

.

.

...

.

..

.

.

.

.

.

.

...

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

..

.

..

.

.

...

..

.

.

.

.

.

.

.

.

.

..

.

..

.

.

.

.

.

..

..

..

.

..

.

.

...

.

.

.

.

.

..

.

.

...

.

.

.

.

.

.

.

.

.

.

.

.

..

.....

.

.....

.

...

.

.

.

..

..

..

..

.

.

.

........

..

.

..

.

.

.

....

.

...

.

..

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

..

.

..

.

.

.

.

.

.

.

.

...

.

.

.

..

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

..

.

..

.

.

.

.

.

.

.

..

.

.

..

.

.

.

..

.

.

.

.

...

..

..

.

.

.

..

.

....

..

....

.

...

.

.

..

..

.

.

.

..

.

.

.

.

.

.

..

.

...

.

.

.

.

..

.

.

..

.

.

..

.

..

.

.

.

..

.

..

.

.

.

..

.

..

.

..

.

.

.

.

...

.

.

.

.

.

.

..

...

.

..

.

.

...

..

.

..

..

.

...

..

.

.

...

....

.

.

....

.

.

..

.

.

.

.

.

..

..

..

.

.

..

..

.

.

.

..

.

.

....

.

.

..

.

.

.

...

..

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

..

.

.

.

.

...

.

.

.

..

...

.

.

.

.

..

..

.

.

..

..

.

.

.

..

.

..

.

.

..

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

..

.

...

.

.

.

.

.

.

..

.

.

.

.

.

..

..

.

.

.

.

.

..

.

...

.

.

..

.

.

.

.

..

..

.

.

..

..

..

..

.

.

.

.

.

.

.

.

..

.

.....

.

.

..

.

.

.

.

..

.

.

..

.

.

.

.

.

.

.

.

.

.

.

....

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

..

.

.

.

.

..

.

.

.

.

.

..

..

.

..

.

.

.

.

.

.

.

.

...

.

..

.

.

..

.

..

.

.

.

.

.

.

.

...

.

.

.

.

.

.

.

.

..

.

.

.

.

.

..

.

..

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

...

..

.

..

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

..

..

.

..

.

..

.

.

.

.

.

.

.

.

.

.

..

.

..

...

.

.

..

.

.

..

.

.

.

...

.

..

.

.

.

.

.

.

..

..

.

..

.

.

...

..

.

.

..

..

.

.

.

.

.

...

.

.

.

.

..

.

.

..

.

..

.

.

.

.

.

.

.

..

.

..

..

..

.

.

.

.

.

..

.

.

.

..

.

.

.

.

..

.

.

.

.

..

.

.

.

.

..

.

.

.

.

..

.

.

.

.

..

.

.

.

.

...

..

.

..

..

...

..

.

.

..

.

.

.

.

.

.

..

..

.

.

.

.

.

..

.

.

.

.

.

..

.

.

.

.

.

..

.

.

..

.

.

.

.

.

.

.

.

.

.

...

...

..

.

.

.

....

.

.

.

.

..

.

....

.

.

.

.

.

.

.

.

.

..

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

..

...

.

..

.

.

.

...

.

..

.

..

.

.

.

..

.

.

..

.

..

.

.

.

.

.

......

.

..

..

.

.

......

.

.

.

.

..

..

.

..

.

.

.

.

.........

..

.

.............

...

...........

.

.

..

..

.

...

..

.

...

..

...

.

.

.

.

.

........

.

..

.

......

..........

.

.

.

.

.

.

...............

..

.

.

..

.

.

.

.

.

.

.

.....

...................

...

..

.

..

.

.

.

......

.

..

.

..

.

...

.

........

.

..

.

....

.

.

.

...

.

...

.

..

.

.

.

.

..

.

.

.

.

.

..

.

.

..

.

.

..

.

.

.

..

.

.

.

.

..

.

.

.

...

..

.....

.

...

..

.

.

..........

.

.

.

.

...

......

.

.

..

..

...

.

...

...

..

.

..

....

.

.

.

...

.

.

.

.

.....

.

.

..

.

.

..

.

.

..

.

..

.

....

..

...

.

..

.

..

.

.

..

.

..

.

.

.

....

.

..

.

..

.

.

.

..

.

.

..

.

.

..

..

..

.....

.

.

.

......

..

..

.

..

.

.

.

..

.

.

...

.

.

.

.

...

.

.

..

..

.

.

.

.

..

..

.

..

.

.

.

.

.

.

.

..

..

.

.

..

..

...

.

.

...

.

.

.

.....

.

....

.

.

.

......

.

...

.

.

.

..

.

.

..

.

.

.

.

.

.......

.

..

.

.....

.

...

..

....

..

.

.

..

.

..

...

.

...

.

...

.

...

.

..

.

...

.

.

.

.

.

....

.

.

...

.

..

...

.

..

..

.

..

.

..

.

.

..

.

..

.

.

.

.

.

..

..

..

.

.

..

.

.

.

.

..

.

...

.

..

.

......

..

.....

.

.

..

...

.

..

..

.

.

.

.

....

..

.

..

.

..

.

.

..

.

.

..

..

.

.

.

.

.

..

...

.

.

.

.

.

.

..

.

.

.

.

..

.

.

.

..

.

.

.

.

.

.

.

.

.

..

.

..

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

...

.

..

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

....

.

.

.

..

.

.

.

..

.

.

.....

.

..

.

..

.

..

.

......

..

..

.

.

.

.

.

.

.....

.

...

.

..

.

.

.

...

.

...

..

.

.

.

.

.

.

.

..

..

..

..

.

.

.

.

.

.

.

.

.

.

.

.

..

..

.

.

.

..

.

.

...

.

.

.

..

.

..

.

.

.

.

.

...

.

.

.

..

.

.

..

..

.

...

..

..

.

........

.

.

.

....

.

.

.

.

.

.

.

..

.

.

.

..

.

..

.

.

.

.

.

.

..

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

..

.

..

.

.

..

..

.

.

.

.

..

.........

.

.

..

.

....

.

....

..

....

.

...

.

..

..

.

..

.

.

..

.

.

..

..

.

.

.

.

.

......

.

...

.

......

.

.

.

.

..

.

..

...

..

.

.

.

.

..

..

...

.

.

.

..

.

...

..

.

.

..

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

....

.

.

.

.

.

...

.

..

......

.

..

.

.

.

...

.

....

.

.

.

.

..

.

.

.

....

.

..

.

.

.

..

.

.

..

.

.

.

..

.

....

.

.

..

.

..

.

.

.

..

.

.

.

.

...

.

.

..

.

..

.

.

.

.

.

..

.

.

..

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

...

.

.

.

.

.

.

.

..

.

.

...

..

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

....

.

..

..

.

..

.

.

..

.

..

.

.

.

.

..

.

.

..

....

.

....

.

.

.

.

..

.

.

.

...

.

....

.

.

.

.

.

.

.

...

.

.

.

.

..

..

..

.

...

.

...

.

...

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

..

.

..

.

.

.

.

.

..

.

.

.

.

.

.

..

.

.

.

.

.

.

..

.

.

.

.

.

..

..

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

..

.

.

.

.

.

.

.

.

..

.

.

..

..

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

..

.

..

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

..

.

..

.

.

...

.

.

.

...

..

.

.

.

..

.

.

.

.

.

...

.

.

.

.

.

.

....

.

.

.

..

..

.

.

..

.

..

..

.

.

.

.

.

.

....

..

.

.

.

.

.

.

.

.

.

..

..

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

...

.

.

..

..

..

.

.

...

..

.

...

.

.

.

..

.

.

.

.

..

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

..

.

.

.

.

.

.........

.

.

.

.

.

..

.

..

.

.

..

.

.

..

.

.

.

.

.

.

.

.

.

.

..

.

.

...

.

.

..

.

....

.

.

..

.

.

.

.

.

.

.

.

.

.

....

.

.

.

..

..

...

.

.

..

..

....

..

.

.

....

...

...

...

.

.

..

...

..

.

.

....

.

.

.....

.

.....

..

.......

.

.

.

.

...

.

.

...

.

....

.

..

..

.

.

.

....

.

..

.

..

.

..

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

..

...

..

.

.

.

.

.

.

..

.

.

..

.

.............

.

........................................

...

.

..

..

.

..

...

.

.

.

.

.

.

.

.

.

.......

..

.......

.....

.

.

.

..............

....

.

.

..............

..

.

......

.......

...............

..

.

.

...........

..........

.

.

.

.

..

.

..

.

....

.

....

..........

...............

...............

.

.

...

...

.

.

..

.

..

..

.

.

.

..

..

.

..

..

..

.

..

.

..

.

..

.

.

.

.

....

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

...

...

.

.

.

.

.

.

...

..........

.....

.

.

..

.

..

.

..

.

.

.

....

.

.....

.

.......

.

....

...

.

.

.

..

.

...

..

..

.

...

..

...

..

..

.

.

.

.

..

.

.

.

..

.

.

....

.

..

.

.

.

.

.

.

..

...

.

.

.

.

.

..

.

.

.

..

.

.

.

.

..

.

..

..

.

..

.

.

.

.

..

.

.

.

..

.

.

..

.

.

.

..

.

...

..

.

..

.

..

.

.....

.

........

..

..

.

..

...

.....

.

..

.

.

.

.

.

.

.

.

.

..

..

..

..

..

.

.

..

.

.

.

.

.

..

.

...

.

........

.

.

.

...

.

..

.

.

..............

.

..

.

.

.

.

.

.

.

.

.....

.

....

.

..

.

..

.

.

.

..

.

..

..

..

.

.

...

.

..

.

.

.

......

.

...

.

..

.

.

.

.

..

.

.

.

.

..

.

.

.

.

.

...

.

.

.

.

..

..

..

..

...

...

.

.

..

..

.

.

...

..

.

.

.

..

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

..

..

.

..

.

.

............

............

..

.

.

.

..

.

..

.

..

.

.

.

.

.

.

..

..

.

..

.

.

..

.

.

.

.

..

.

..

.

.

.

.

.

.

.

...

.

.

.

.

.

.

.

..

.

..

.

.

...

.

.

.

.

..

..

.

.

.

.

..

.

..

..

.

.

..

....

.

.

.

....

.

..

..

.

.

.

.

..

...

.

..

..

..

...

.

.

.

.

..

.

.

.......

...

.

.

.

.

.

..

.

..

.

....

.

.

.

.

..

..

.

..

..

.

...........

..

.

.

..

.

.

....

.

.

.

.

.

..

....

.

.

.

.

.

..

..

..

.

.

.

.

..

.

..

.

..

.

.

.

.

.

.

...

.....

.

.

.

..

.

..

..

.

..

.

.

..

..

.

.

..

.

.

.

....

.

.

.

..

.

....

..

..

.

.........

....

.

.

.

....

.

.

.

.

.

.

.

....

.

.

..

.

...

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

..

.

.

.

.

.

.

.

..

.

.

.

.

.

..

.

..

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

....

.

.

.

.

.

.

.

.

.

..

.

..

.

.

.

.

.

..

..

.

..

.

.

.

.

.

.

...

.

.

.

..

...

.

.

.

.

..

..

.

.

..

.

.

.

.

..

...

..

...

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

..

.

.

.

.

..

...

..

.

.

..

.

.

.

.

.

.

..

.

.

.

.

..

..

.

.

.

..

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

..

.

..

..

.

.

.

.

.

..

.

.

..

.

.

.

.

.

..

.

.

..

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

..

.

...

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

...

..

..

..

..

..

.

.

.

.

.

.

.

.

.

.

.

.

..

....

.

.

.

...

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

...

..

.

..

..

..

.

.

.

..

...

..

.

.

..

.

..

.

.

..

..

.

.

.

...

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

...

...

.

.

.

.

...

..

.

.

.

.

..

.

.

.

...

..

.

.

.

..

.

.

.

..

.

.

..

..

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

..

..

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

..

.

.

..

..

.

.

.

.

...

.

.

..

..

.

.

.

..

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

..

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

..

.

.

.

.

.

.

..

.

.

.

.

..

.

..

.

.

.

....

.

.

..

..

.

..

.

.

.

.

.

.

..

.

...

.

..

..

.

.

..

.

.

..

.

.

.

..

.

.

.

..

..

.

.

.

..

..

.

.

.

..

.

....

.

.

.

.

.

.

.

.

.

.

...

.

..

..

.

..

.

..

.

..

.

.

.

...

...

.

.

.

..

.

.

.

.

.

.

.

.

...

.

.

..

.

...

...

...

.

.

.

.

..

.

.

.

...

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.....

.

.

.

.

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

..

..

.

.

.

..

.

.

..

.

.

..

.

.

..

..

.

.

..

.

..

..

...

.

......

.

..

.

.

.

.

.

.

.

.

.

.....

.

..

.

......

.

.

..

.

..

..

.

.

..

.

.

.

.

.

.

..

.

..

..

.

.

.

.

.

.

.

..

..

.

..

.

.

.

.

.

.

.

.

..

.

...

.

...

.

.

..

.

.

.

.

.

.

..

.

.

.

.

.

.

...

.

.

..

..

.

.

.

.

.

.

.

.

..

.

.

.

..

.

..

.

..

.

..

.

..

...

.

.

..

.

.

.

.

.

.

...

..

.

.

..

..

.

.

.

..

..

...

.

.

.

.

.

..

.

.

.

....

.

..

.

.

.

.

..

.

.

.

.

.

.

.

..

.

....

..

.

.

.

.

..

.

...

.

.

..

..

.

..

.

.

..

.

..

.

.

.

...

.

.

..

..

..

.

..

.

..

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.

.

.

..

.

..

..

.

.

...

.

.

....

.

.

.

.

.

...

..

...

...

....

.

.

.

....

..

..

..

.

.

...

.

..

.

..

.

.

.

.

.

.....

..........

.

..

.

.

.

.

.

..

.

.

..

.

.

..

.

.

..

...

.......

...

.

.

.

.

..

.

...

.

...

.

...

.

.

.

.

..

..

..

.

.

.

.

.

.

..

.

....

.

.

.

.

.

.

.....

.

.

.....

.....

...

...

.

...

.

...

.

.....

Timer

Pack

et N

um

ber

(Mod 9

0)

for

Four

Connect

ions

s

0.0 0.2 0.4 0.6 0.8 1.0

0s

100s

200s

300s

400s

Figure3: A simulationwith four FTPconnectionswith staggeredstarttimes.

7

Figure3 shows a simplesimulationwith RED gate-ways. The network is shown in Figure 4. The simula-tioncontainsfourFTPconnections,eachwith amaximumwindow roughly equal to the delay-bandwidthproduct,which rangesfrom 33 to 112packets.TheRED gatewayparametersare set as follows: =@?�t : . ::u , ��� �����vtw

packets, � �������xt 0 w packets, and ����� " t 0 + w : .Thebuffer sizeis sufficiently largethatpacketsareneverdroppedatthegatewaydueto bufferoverflow; in thissim-ulationtheREDgatewaycontrolstheaveragequeuesize,andtheactualqueuesizeneverexceedsforty packets.

3y

1

2z 4

{

SINK|

GATEWAY}1

5~6�

4{

45Mbps�100Mbps�

2ms�

FTP SOURCES

1ms� 4ms

� 8ms�

5ms�

Figure4: Simulationnetwork.

For thechartsin Figure3, thex-axisshowsthetimeinseconds.Thebottomchartshows thepacketsfrom nodes1-4. Eachof the four main rows shows thepacketsfromoneof thefour connections;thebottomrow showsnode1packets,andthetoprow showsnode4 packets.Thereis amarkfor eachdatapacketasit arrivesat thegateway andasit departsfrom thegateway; at this time scale,thetwomarksareoftenindistinguishable.They-axisis afunctionof thepacketsequencenumber;for packetnumber� fromnode � , they-axisshows � h��R�(�: C & ��' 0 * 0 :: . Thus,eachvertical‘line’ represents90consecutively-numberedpacketsfromoneconnectionarrivingatthegateway. Each‘X’ showsapacketdroppedby thegateway, andeach‘X’is followedby a mark showing the retransmittedpacket.Node1 startssendingpacketsat time 0, node2 startsaf-ter0.2seconds,node3 startsafter0.4seconds,andnode4startsafter0.6seconds.

Thetopchartof Figure3 showstheinstantaneousqueuesize F andthecalculatedaveragequeuesize ��� . Thedot-tedlinesshow ��� ����� and �������� , theminimumandmax-imumthresholdsfor theaveragequeuesize.Notethatthecalculatedaveragequeuesize ���� changesfairly slowlycomparedto F . The bottom row of X’s on the bottomchartshowsagainthetimeof eachdroppedpacket.

This simulationshows the successof the RED gate-way in controllingtheaveragequeuesizeat thegateway

in responseto a dynamicallychangingload. As thenum-berof connectionsincreases,thefrequency with whichthegateway dropspacketsalsoincreases.Thereis no globalsynchronization.The higherthroughputfor the connec-tions with shorterroundtrip times is due to the bias ofTCP’swindow increasealgorithmin favor of connectionswith shorterroundtriptimes(asdiscussedin [6, 7]). Forthe simulationin Figure3 the averagelink utilization is76%. For the following secondof the simulation,whenall four sourcesareactive, the averagelink utilization is82%.(This is notshown in Figure3.)

(‘triangle’ for RED, ‘square’ for Drop Tail)�Throughput (%)�A

vera

ge Q

ueue�

0.4 0.6 0.8 1.0

0� 2040

6080

100�

Figure5: ComparingDropTail andREDgateways.

3�

4�

SINK�

GATEWAY�

5�6�

4�

45Mbps�

100Mbps

FTP SOURCES

20ms�

1ms

Figure6: Simulationnetwork.

BecauseREDgatewayscancontroltheaveragequeuesizewhileaccommodatingtransientcongestion,REDgate-waysarewell-suitedto provide high throughputandlowaverage delayin high-speednetworkswith TCPconnec-tionsthathave largewindows. TheRED gatewaycanac-commodatetheshortburstin thequeuerequiredby TCP’sslow-start phase;thus RED gateways control the aver-age queuesize while still allowing TCP connectionstosmoothlyopentheir windows. Figure5 shows theresultsof simulationsof the network in Figure6 with two TCPconnections,eachwith a maximumwindow of 240pack-ets, roughly equalto the delay-bandwidthproduct. Thetwo connectionsarestartedatslightlydifferenttimes.Thesimulationscomparetheperformanceof DropTail andofREDgateways.

In Figure5 thex-axisshows thetotal throughputasa

8

fractionof themaximumpossiblethroughputon thecon-gestedlink. The y-axisshows the averagequeuesizeinpackets(asseenby arriving packets).Five 5-secondsim-ulationswere run for eachof 11 setsof parametersforDropTail gateways,andfor 11setsof parametersfor REDgateways;eachmarkin Figure5 shows theresultsof oneof thesefive-secondsimulations. The simulationswithDrop Tail gatewayswererun with thebuffer sizerangingfrom 15 to 140 packets; as the buffer size is increased,thethroughputandtheaveragequeuesizeincreasecorre-spondingly. In orderto avoid phaseeffectsin the simu-lationswith Drop Tail gateways,thesourcenodetakesarandomtimedrawn from theuniformdistributionon[0, t]secondsto prepareanFTPpacketfor transmission,where7 is thebottleneckservicetimeof 0.17ms.[7].

Thesimulationswith REDgatewayswereall runwitha buffer size of 100 packets, with ��� ����� rangingfrom3 to 50 packets. For the RED gateways, � ������� is setto �G��� ����� , with =@?�t : . ::u and ���� " t 0 + w : . Thedashedlinesshow theaveragedelay(asafunctionof through-put) approximatedby 0 .��L�a+ &10 '���* for the simulationswith RED gateways,andapproximatedby : . 0 + &B0 '���*B�for the simulationswith Drop Tail gateways. For thissimple network with TCP connectionswith large win-dows, the network power (the ratio of throughputto de-lay) is higher with RED gateways than with Drop Tailgateways. Thereareseveral reasonsfor this difference.With Drop Tail gatewayswith a small maximumqueue,the queuedropspacketswhile the TCP connectionis intheslow-startphaseof rapidly increasingits window, re-ducing throughput. On the other hand,with Drop Tailgatewayswith a largemaximumqueuetheaveragedelayis unacceptablylarge.In addition,DropTail gatewaysaremorelikely to droppacketsfrom bothconnectionsat thesametime, resultingin globalsynchronizationanda fur-therlossof throughput.

Laterin thepaper, we discusssimulationresultsfromnetworkswith a morediverserangeof connections.TheRED gateway is not specificallydesignedfor a networkdominatedby bulk datatransfer;this is simply an easyway to simulateincreasingly-heavy congestionat a gate-way.

6 Calculating theaveragequeuelength

The RED gateway usesa low-passfilter to calculatetheaveragequeuesize. Thus,theshort-termincreasesin thequeuesizethatresultfrom burstytraffic or from transientcongestiondo not result in a significantincreasein theaveragequeuesize.

Thelow-passfilter is anexponentialweightedmovingaverage(EWMA):���� $ &10 ')= ? *B����DCE= ? Fa. (1)

The weight =@? determinesthe time constantof thelow-passfilter. Thefollowing sectionsdiscussupperandlower boundsfor setting =@? . Thecalculationof theaver-agequeuesizecanbeimplementedparticularlyefficientlywhen=@? is a(negative)powerof two,asshown in Section11.

6.1 An upper bound for ���If = ? is too large, thenthe averagingprocedurewill notfilter out transientcongestionat thegateway.

Assumethatthequeueis initially empty, with anaver-agequeuesizeof zero,andthenthequeueincreasesfrom0 to � packetsover � packetarrivals.After the � th packetarrivesat thegateway, theaveragequeuesize ����� is

���� � t �  ¡ ¢k£ ��= ? &B0 ')= ? * ��¤ ¡t = ? &10 '>= ? * � �  ¡ ¢k£ � & 00 '>=@? * ¡t �¥C 0 C &10 '>= ? * �#¦ £ ' 0=@? . (2)

Thisderivationusesthefollowing identity [9, p. 65]:�  ¡ ¢k£ � � ¡ t �OC & §X¨ ')�)' 0 *a� �#¦ £&10 '>��*1© .

0.001

0.002

0.003

0.004

0.005

w_q

0

25

50

75

100

L

0

5

10

15

20

ave_L

0.001

0.002

0.003

0.004

0.005

w_q

0

25

50

75

100

L

0

5

10

15

20

ave_L

Figure7: ���� asa functionof =@? and � .

Figure 7 shows the averagequeuesize ���� � for arangeof valuesfor = ? and � . The � -axisshows = ? from0.001to 0.005,andthe ª -axis shows � from 10 to 100.For example,for = ? t : . :: 0 , afteraqueueincreasefrom0 to 100 packets, the averagequeuesize ���� £I«U« is 4.88packets.

Givenaminimumthreshold� � ����� , andgiventhatwewish to allow burstsof � packetsarriving at thegateway,

9

then=@? shouldbechosento satisfythefollowingequationfor �������� ������� :�¥C 0 C &B0 ')=¬?d* �#¦ £ ' 0= ? ����� �����#. (3)

Given ��� �����Qt w, and �­t w : , for example,it is neces-

saryto choose=@?N� : . ::g®au .

6.2 A lower bound for ���REDgatewaysaredesignedtokeepthecalculatedaveragequeuesize ���� below a certainthreshold.However, thisserveslittle purposeif thecalculatedaverage���� is not areasonablereflectionof thecurrentaveragequeuesize.If=@? is settoolow, then ��� respondstooslowly to changesin theactualqueuesize.In thiscase,thegatewayis unableto detecttheinitial stagesof congestion.

Assumethat the queuechangesfrom empty to onepacket, andthat,aspacketsarrive anddepartat thesamerate,thequeueremainsatonepacket. Furtherassumethatinitially the averagequeuesizewaszero. In this caseittakes ' 0 +6¯j° &B0 ')= ? * packetarrivals(with thequeuesizeremainingatone)until theaveragequeuesize ���� reachs: . ±�)t 0 ' 0 +gJ [35]. For = ? t : . :: 0 , this takes1000packet arrivals;for = ? t : . ::�u , this takes500packet ar-rivals; for =@?�t : . :: � , this takes333packet arrivals. Inmostof our simulationsweuse=@?@t : . ::u .

6.3 Setting ²´³,µ·¶¹¸ and ²�º�»G¶¹¸Theoptimalvaluesfor ��� � ��� and � ��� ��� dependon thedesiredaveragequeuesize. If the typical traffic is fairlybursty, then ��� ����� mustbe correspondinglylarge to al-low thelink utilization to bemaintainedat anacceptablyhigh level. For the typical traffic in our simulations,forconnectionswith reasonablylargedelay-bandwidthprod-ucts,a minimumthresholdof onepacket would resultinunacceptablylow link utilization. The discussionof theoptimalaveragequeuesizefor a particulartraffic mix isleft asa questionfor futureresearch.

The optimal valuefor ���� ��� dependsin part on themaximumaveragedelaythatcanbeallowedby thegate-way.

The RED gateway functionsmost effectively when���� ��� '¼��� � ��� is larger thanthe typical increasein thecalculatedaveragequeuesize in one roundtrip time. Ausefulrule-of-thumbis toset�������� toatleasttwice � � ����� .

7 Calculating thepacket-marking prob-ability

Theinitial packet-markingprobability � ! is calculatedasa linearfunctionof theaveragequeuesize.In thissection

wecomparetwo methodsfor calculatingthefinal packet-markingprobability, and demonstratethe advantagesofthe secondmethod. In the first method,whenthe aver-agequeuesizeis constantthenumberof arriving packetsbetweenmarkedpacketsis a geometricrandomvariable;in thesecondmethodthenumberof arriving packetsbe-tweenmarkedpacketsis a uniformrandomvariable.

Theinitial packet-markingprobabilityis computedasfollows:� !%$ � ��� "#& ����(')��� �����*,+ & � �������-')��� ������*/.The parameter����� " gives the maximumvalue for thepacket-markingprobability��! , achievedwhentheaveragequeuesizereachesthemaximumthreshold.

Method 1: Geometricrandom variables. In Method1, let eachpacket bemarkedwith probability ��! . Let theintermarkingtime ½ bethenumberof packetsthatarrive,after a marked packet, until the next packet is marked.Becauseeachpacket is markedwith probability � ! ,¾D¿ 3gÀLÁ ½Ât´��Ã�t &10 ' � ! *1Ä ¤ £ � ! .Thuswith Method1, ½ is a geometricrandomvariablewith parameter� ! , and Å�Á ½ÆÃ�t 0 + � ! .

With aconstantaveragequeuesize,thegoalis tomarkpacketsat fairly regularintervals.It is undesirableto havetoo many marked packets close together, and it is alsoundesirableto have too long an interval betweenmarkedpackets.Bothof theseeventscanresultin globalsynchro-nization,with severalconnectionsreducingtheirwindowsat thesametime,andbothof theseeventscanoccurwhen½ is ageometricrandomvariable. Ç

Method 2: Uniform random variables. A morede-sirablealternative is for ½ to bea uniform randomvari-able from È 1, 2, ..., 0 + ��!/É (assumingfor simplicity that0 + � ! is aninteger). This is achievedif themarkingprob-ability for eacharriving packet is � ! + &B0 'Ê2<365���7@8 � ! * ,where2<365���7 is thenumberof unmarkedpacketsthathavearrivedsincethelastmarkedpacket. Call this Method2.In thiscase,¾D¿ 3gÀgÁ ½ÂtP��ÃËt � !0 ' & � ' 0 * � ! Ä ¤ ©Ì¡ ¢X« Í 0 ' � !0 '>� � !Ît � !(Ϲ�Ð 0 ���>� 0 + � !/Ñand ¾D¿ 3gÀLÁ ½Òt���Ã�t :¥Ï¹�Ð �¥Ó 0 + � ! .For Method2, Å�Á ½�Ã�t 0 + & u � ! *XC 0 + u . Ç

Figure8showsanexperimentcomparingthetwo meth-ods for marking packets. The top line shows Method1, whereeachpacket is marked with probability � , for� t : . :�u . Thebottomline shows Method2, whereeachpacketis markedwith probability� + &10 CD� � * , for � t : . : 0 ,

10

(top row for Method 1, bottom row for Method 2)Packet NumberÔ0Õ 1000 2000 3000 4000 5000

..... .. . ... ..... .. . . . . ...... . . .. . .. . . .. . ... .. . . .... . .. .. . .... ... ... .. .. . .. . . .. . ... . . . ... . .. . . . . . . . .. .. . . ... . .. . .. . .. ... ... .. .... .. . . . .. . ... .. .. ... . .... . . .. . .. . ..... . ... . ...... . . ..... .. .. . .. .. . . .

... .... ... . . ...... .... . . ... .. . . .. . ........ . . .... ..... ......... ..... . .. ........ ..... . . ........ ..... .... . .. .... ..... .. . . .... ... ... . . ... ...... . .. ... .. .. . ..... .. . .... .. .. ...... . .... . .... . .. . .. .... .... .. .. . .....1:

2:Ö

Figure8: Randomly-markedpackets,comparingtwo packet-markingmethods.

andfor � the numberof unmarkedpacketssincethe lastmarkedpacket. Bothmethodsmarkedroughly100outofthe 5000arriving packets. The � -axis shows the packetnumber. For eachmethod,thereis a dot for eachmarkedpacket. As expected,the markedpacketsaremoreclus-teredwith Method1 thanwith Method2.

For thesimulationsin thispaper, weset� ��� " to 1/50.Whenthe averagequeuesizeis halfway between��� �����and � ������� , the gateway drops,on the average,roughlyone out of 50 (or one out of 0 +6���� " ) of the arrivingpackets. RED gatewaysperformbestwhen the packet-markingprobabilitychangesfairly slowly astheaveragequeuesizechanges;thishelpstodiscourageoscillationsintheaveragequeuesizeandin thepacket-markingproba-bility. Thereshouldneverbeareasonto set �����R" greaterthan0.1, for example.When � ���#"�t : . 0 , thentheREDgatewaymarkscloseto 1/5thof thearriving packetswhentheaveragequeuesizeis closeto themaximumthreshold(usingMethod2 to calculatethe packet-markingproba-bility). If congestionis sufficiently heavy thattheaveragequeuesizecannotbecontrolledby markingcloseto 1/5thof thearriving packets,thenafter theaveragequeuesizeexceedsthe maximumthreshold,the gateway will markeveryarriving packet.

8 Evaluation of RED gateways

In addition to the designgoalsdiscussedin Section3,several generalgoalshave beenoutlined for congestionavoidanceschemes[14, 16]. In this sectionwe describehow ourgoalshavebeenmetby REDgateways.× Congestionavoidance. If theRED gateway in factdrops packetsarriving at the gateway when the averagequeuesizereachesthemaximumthreshold,thentheREDgatewayguaranteesthatthecalculatedaveragequeuesizedoesnotexceedthemaximumthreshold.If theweight =@?for theEWMA procedurehasbeensetappropriately[seeSection6.2], thenthe RED gateway in fact controlstheactualaveragequeuesize. If theRED gateway setsa bitin packetheaderswhentheaveragequeuesizeexceedsthemaximumthreshold,ratherthan droppingpackets, then

theRED gatewayrelieson thecooperationof thesourcesto controltheaveragequeuesize.× Appropriate time scales.After notifying aconnec-tion of congestionby markinga packet, it takesat leasta roundtriptime for thegateway to seea reductionin thearrival rate.In REDgatewaysthetimescalefor thedetec-tionof congestionroughlymatchesthetimescalerequiredfor connectionsto respondto congestion.RED gatewaysdon’t notify connectionsto reducetheir windows asa re-sultof transientcongestionat thegateway.× No global synchronization. TherateatwhichREDgatewaysmark packetsdependson the level of conges-tion. During low congestion,thegatewayhasa low prob-ability of marking eacharriving packet, and as conges-tion increases,theprobabilityof markingeachpacket in-creases.RED gatewaysavoid global synchronizationbymarkingpacketsataslow arateaspossible.× Simplicity. The RED gateway algorithmcould beimplementedwith moderateoverheadin currentnetworks,asdiscussedfurtherin Section11.× Maximizing global power4. TheREDgatewayex-plicitly controlsthe averagequeuesize. Figure5 showsthatfor simulationswith highlink utilization,globalpoweris higherwith RED gatewaysthanwith Drop Tail gate-ways. Futureresearchis neededto determinethe opti-mumaveragequeuesizefor differentnetwork andtrafficconditions.× Fairness.Onegoalfor acongestionavoidancemech-anismis fairness.Thisgoalof fairnessis notwell-defined,sowe simply describetheperformanceof theRED gate-way in this regard. The RED gateway doesnot discrim-inateagainstparticularconnectionsor classesof connec-tions. (This is in contrastto Drop Tail or RandomDropgateways,asdescribedin [7]). For theRED gateway, thefractionof markedpacketsfor eachconnectionis roughlyproportionalto thatconnection’s shareof thebandwidth.However, RED gateways do not attemptto ensurethateachconnectionreceives the samefraction of the totalthroughput,anddonotexplicitly controlmisbehavingusers.RED gatewaysprovide a mechanismto identify the level

4Power is definedastheratioof throughputto delay.

11

of congestion,andRED gatewayscould alsobe usedtoidentify connectionsusinga largeshareof thetotalband-width. If desired,additionalmechanismscouldbeaddedto RED gatewaysto control the throughputof suchcon-nectionsduringperiodsof congestion.× Appropriate for a wide range of envir onments.The randomizedmechanismfor marking packets is ap-propriatefor networks with connectionswith a rangeofroundtriptimesandthroughput,andfor a large rangeinthe numberof active connectionsat onetime. Changesin the load aredetectedthroughchangesin the averagequeuesize,andtherateatwhichpacketsaremarkedis ad-justedcorrespondingly. TheREDgateway’sperformanceis discussedfurtherin thefollowing section.

Evenin a network whereRED gatewayssignalscon-gestionby droppingmarkedpackets,therearemany oc-casionsin a TCP/IPnetwork whenadroppedpacketdoesnot result in any decreasein load at the gateway. If thegateway dropsa datapacket for a TCP connection,thispacket drop will be detectedby the source,possiblyaf-teraretransmissiontimerexpires.If thegatewaydropsanACK packetfor aTCPconnection,orapacketfromanon-TCPconnection,this packet dropcouldgo unnoticedbythesource.However, even for a congestednetwork witha traffic mix dominatedby shortTCP connectionsor bynon-TCPconnections,theREDgatewaystill controlstheaveragequeuesizeby droppingall arriving packetswhentheaveragequeuesizeexceedsa maximumthreshold.

8.1 Parameter sensitivity

This sectiondiscussesthe parametersensitivity of REDgateways. Unlike Drop Tail gateways, where the onlyfreeparameteris thebuffer size,RED gatewayshave ad-ditionalparametersthatdeterminetheupperboundontheaveragequeuesize,the time interval over which the av-eragequeuesizeis computed,andthemaximumrateformarkingpackets. The congestionavoidancemechanismshouldhave low parametersensitivity, and the parame-tersshouldbeapplicableto networkswith widely varyingbandwidths.

TheREDgatewayparameters=¬? , � ������� , and ���������arenecessarysothatthenetwork designercanmakecon-sciousdecisionsaboutthedesiredaveragequeuesize,andaboutthesizeanddurationin queueburststo beallowedat thegateway. Theparameter����� " canbechosenfroma fairly wide range,becauseit is only anupperboundonthe actualmarkingprobability ��! . If congestionis suffi-ciently heavy thatthegatewaycannotcontroltheaveragequeuesize by marking at most a fraction � ���#" of thepackets,thentheaveragequeuesizewill exceedthemax-imum threshold,andthegateway will markevery packetuntil congestionis controlled.

Wegiveafew rulesthatgiveadequateperformanceof

theREDgatewayundera widerangeof traffic conditionsandgatewayparameters.

1: Ensureadequatecalculationof theaveragequeuesize:set =@?DØ : . :: 0 . Theaveragequeuesizeat thegate-wayis limitedby � ������� , aslongasthecalculatedaveragequeuesize ��� is a fairly accuratereflectionof theactualaveragequeuesize. Theweight =@? shouldnot besettoolow, so that thecalculatedaveragequeuelengthdoesnotdelaytoo long in reflectingincreasesin theactualqueuelength [SeeSection6]. Equation3 describesthe upperboundon = ? requiredto allow thequeueto accommodateburstsof � packetswithoutmarkingpackets.

2: Set ��� � ��� sufficiently high to maximizenetworkpower. The thresholds� � ����� and ��������� shouldbe setsufficientlyhightomaximizenetworkpower. Aswestatedearlier, moreresearchis neededon determiningtheopti-mal averagequeuesize for variousnetwork conditions.Becausenetwork traffic is often bursty, the actualqueuesizecanalsobequitebursty; if theaveragequeuesizeiskepttoo low, thentheoutputlink will beunderutilized.

3: Make ��������-')��� ����� sufficiently largeto avoidglobal synchronization. Make ����� ��� '­� � � ��� largerthanthetypical increasein theaveragequeuesizeduringa roundtriptime, to avoid theglobalsynchronizationthatresultswhenthegatewaymarksmany packetsatonetime.Oneruleof thumbwouldbeto set ���� ��� to at leasttwice� � ����� . If � �������Ù'T� � ����� is toosmall,thenthecomputedaveragequeuesizecanregularly oscillateup to �������� ;thisbehavior is similar to theoscillationsof thequeueupto themaximumqueuesizewith DropTail gateways.

12

Queue size (solid line) and average queue size (dashed line).lTimem

Qu

eu

e f

or

ga

te A

Ú0.0 0.2 0.4 0.6 0.8 1.0

0o 20

40

max-thpmin-thq

Queue size (solid line) and average queue size (dashed line).lTimem

Qu

eu

e f

or

ga

te B

Ú0.0 0.2 0.4 0.6 0.8 1.0

0o 20

40

max-thpmin-thq

..........................................................................................................................................................................

.

...

..

.................................................

..

.

..

.

..

...................

.........

.

.......................

..................

...........

.

..

........................

.....................

...

.......................

.

.

..

..................

.......

.................

...................

.....

.

...

.

..

..

.

.

..

..

.

.

..

...

..

.......................

.

..

..

..

.

..

......

....

.

.

...

...

.

.

...

..

....

..

............

........

.

..

........

.

.

.......

.

....

...

....

..................

.

..

.

.

....

.........

....

.........

......

.

.

....

..

.

.

......

.

...

....

.....

...

.

.

.

.

...

..

..

.

.

.

..

..

..

..

..

..

..

..

..

.......

..

........

.

......

...

.

...

...

..

..

..............

....

.

.....

...

...

.

..

..

..

.

...

.

.

...

....

.

.

.

..

..

..

..

.

...

..

...

..

.

...

..

.

.

..

..

.

.

..

.

..

...

...

.........

..

........

..

.

..

..................

.

...

...

..

.....

.

..

.

..

.

.

..

.

..

.

.

...

.

..

.

.

..

.

.

..

.

..

.

..

.

...

.

...

..

..

...

..

..

....

..

.

.

.

..

.

.

...

..

..

.

..

...

..

.

..

....

..

...................

.

...

.

.

.

..

..

....

..........................

.

.

.

.

..

.

.

....

.......

.

....

.

.

.

..

..

.

.

..

.

.

.

....

...

....

..

..

.

..

..

.

...

..

.

.

....

...

....

....

.

.

..

.

..

..

.

.

..

....

..

......

..

...................

..........

......

.......

.

.

..

..

...

.

..

.

.

..

....

.....

.....

.

..

.

..

..............

..

.

.

....

..

.

...

.

...

.

.....

.

..

...

.

.

...

...

..

.

.

..

.

.

.

..

.

..

.

.

..

.

.

.

..

.

.

.

.

..

.

..

..

..

..

..

....

..

...

..

.

..

...

..

.

....

.

..

..

..

..

..

..

........................

..

.....

.

........

.

..

.

..

.

.

...

.

..

.

..

..

..

.

.

.

...

..

.

..

.

.

.

..

..

.

..

.

...

.

.....

..

..

.

.

.

..

.

..

..

..

.

..

..

..

..

...

..

....

..

.

..

...

.

..

.

..

.

.

......

.

..

.

.

..

.

..

.

.

..

.

.....

..

.

.

..

..

.

.

..

.

..

...

..

..

..

....

..

..

..

..

.

..

.........

...

.......

.

....

.

.

.

.

.

....

...

..

.

..

.

...

.

.

...

.

..

.

....

..

..

....

..

..

.

..

.

..

.

.

..

.

..

.

..

....

.

..

....

..

.

...

..

.

..

.

.

..

.

..

........

...

....

.

.

...

..

.

.

..

..

..

.........

.

..

.

.......

.

.....

..

....

.

....

..

.

.

.

..

.

.

..

.

..

.

.

..

.

.

..

.

.

.

..

..

.

....

.

..

.

.

..

..

.

..

.

.....

..

.

.

.

....

..

..

..

..

.

.

....

..

.

......

...

..

.

.

..

..

.

..

.

.

........

..

.

.

..

.....

..

..

..

...

.

..

.

..

.

.

.

..

..

..

....

..

.

..

..

..

......

...

.

...

.

...

...

.

.

..

..

..

.

.

.

...

..

...

.

..

..

.....

..

.

.

.....

..

.

..

.......

.

.

.

.

..

..

.

..

......

...

.

.

.

.

...

..

...

...

....

.....

.

.

..

.

.

...

..

..

.

..

.

.

..

..

..

...

.

.

.....

.

...

.

..

.

...

..

.

.

..

.

.

.

.

...

.

...

.

..

..

.

..

..

.

.

..

..

.

...

..

.

..

..

..

..

..

.

...

....

.....

.

.

..

.

.

..

.

..

...

..

..

..

...

.

...

..

..

.....

.

..

..

..

....

..

..

..

........

..

.

..

.

..

.

.

..

.....

..

..

.

..

..

......

..

..

...

.

...

.

..

..

..

..

.

.

..

.

...

.

..

...

..

..

.

..

.

..

..

...

....

...

..

...

..

..

..

.

..

.

..

.

.

.

..

.

..

..

...

.

.

.....

..

.

.

..

..

...

.

..

..

.

.

..

.

....

..

.

.

.

..

.

..

..

.

.

..

.

..

..

.

..

..

..

..

.

.

..

..

..

.

..

.

....

..

.

....

.

..

..

..

....

..

..

...

..

.

.

..

.

..

.

..

.

..

.

.

..

.

....

.

.

..

..

.

.

..

.

..

..

...

..

.

.

..

..

.

.

..

.

.

.

..

..

..

..

..

.

.

.

..

.

..

.

.

.

..

..

.

.

...

.

.

..

..

..

..

.

..

..

.

.

..

.

..

.

..

.

..

.

.

.

..

..

.

..

.

...

.

.

..

.

..

.

.

.

..

..

.

..

.

.

.

..

.

..

.

.

..

..

.

......

.

..

.

..

..

..

.

....

..

...

.

.

..

.

..

.

..

.

.

...

.

.

..

.

..

..

.

..

.

..

..

.

..

.

..

..

.

..

.

..

.

..

.

.....

..

..

.

.

..

.

..

.

..

...

..

.

.

..

..

..

.

..

.

.

.

..

.

..

.

.

..

.

.

..

.

..

.

..

.

..

....

..

.

..

.

.

.

.

......

..

...

.

.

...

..

.

.

..

.

.

..

..

....

..

.

.

..

..

..

..

..

.

.

..

.

.

...

..

..

..

..

.....

.

...

.

....

..

.

...

...

..

..

..

..

....

..

..

..

..

..

..

..

.

...

..

.

......

.

....

..

.

....

..

..

..

..

....

.

..

..

.

..

..

.

..

.

.

..

..

.

..

.

..

.

..

.

..

..

.

....

..

.

....

.

....

..

..

.

.

..

..

..

...

......

..

....

....

..

......

..

..

.

..

.

..

..

.

..

.

.

..

.

.

..

.

..

..

.

....

....

...

..

.

.

..

....

.

..

.

.

...

..

.

.

....

..

.

.

.

.

.

..

..

.

.

.

.

.

..

...

.

...

...

.

.

..

.

.

..

.

.

..

.

..

.

.

..

...

..

.

.

...

..

..

...

.

...

.

..

....

..

.

...

.

..

.

...

...

..

...

..

..

.

..

..

.

..

..

.

..

..

..

..

..

..

....

..

..

..

.....

..

.

.

..

....

.

.

.

..

.

.

.

..

...

..

...

.

.

.

.

........

..

..

.

.

..

.

.

..

..

..

....

.

...

..

.

.

.

..

.

..

..

..

.

..

.

....

..

.

..

.

..

..

.

.

..

..

.

.

..

.

.

..

.

..

.

..

..

.

.

.

.

..

.

.

..

.

....

.

.

..

..

..

..

...

..

..

..

.

...

.

.

..

.

..

..

.

.

.

.

..

..

..

.

.

..

..

..

..

.

..

..

.

..

..

..

..

..

.

..

..

..

....

..

.

..

....

.

..

..

..

......

..

..

....

.

...

..

..

..

.

..

.

..

.

..

..

...

.......

.

.

..

.

.

....

..

.

.

..

..

.

.

...

..

.

.

.

....

..

..

..

..

.

..

.

.

..

..

.

..

..

..

..

..

.

..

..

.

...

.

.

..

..

.

.

.

.

..

.

.

.

.

..

.

.

.

..

..

.

.

..

.

.

.

.

..

.

.

.

..

.

.

.

.

..

..

.

..

.

.

..

.

.

.

..

.

.

.

..

..

.

..

..

.

.

..

..

.

.

..

.

.

..

..

....

..

..

..

.

..

...

..

..

.

..

..

.

..

..

.

.

.

..

...

..

.

.

..

..

..

.

.

.

..

.

.

..

.

...

.

..

.

..

..

..

.

.

....

..

.

..

..

.

.

..

.

.

.

..

.

...

..

.

...

.

..

..

.

.

..

.

..

..

.

..

..

.

.

.

..

..

..

..

.

.

....

..

..

..

..

..

..

..

.

..

..

..

..

....

..

.

..

..

..

..

..

..

.....

..

.

..

..

...

..

.

..

.

..

.

.

.

..

.

..

.

..

.

..

.

.

.

....

..

..

.

.

.

.

..

........

....

.

..

.

.

..

.

..

..

..

...

.

..

..

..

.

..

.

.

..

..

.

....

..

...

.

..

.

.

..

.

.

.

...

.

....

.

......

.

.

..

..

..

.

.

..

.

..

..

.....

....

.

..

...

.

....

.

.

..

..

.

....

.

..

...

..

..

..

.

....

..

.

.

....

..

.

.

.

.

..

...

.

.

.

..

..

..

.

.

..

..

...

.

....

.

...

...

.

.

.

...

.

..

.

.

..

.

.

..

.

..

..

..

.

.

..

.

.

.

.

..

..

..

..

..

...

.

.

..

.

..

.

.

.

..

........

..

.

.

..

.

.

.

.

..

..

.

....

..

...

.

...

.

..

.

...

.

..

.

...

.

.

..

..

...

.

..

..

.

..

...

.

....

..

.

.

..

..

.

.

.

..

.

..

.

..

..

.

.

..

.

..

..

..

.

....

.

.

..

.

..

.

..

.

..

..

.

..

..

.

..

.

.

..

..

..

.

..

.

..

.

..

.

..

.

..

.....

.

.

..

..

.

..

.

.

.

..

.

..

..

.

.

.

..

.

..

..

..

...

....

...

..

.

..

.

....

..

.

.

..

.

..

..

..

......

..

...

..

...

..

..

..

.

.

.

.

.

..

..

..

..

....

.

...

.

.....

.

.

..

..

..

.

..

..

..

.

.....

..

..

.

..

.

.

.

..

..

.

.

.

..

..

.....

..

..

..

..

..

.

...

.

..

.

.

...

.

..

..

..

..

..

..

..

....

...

.

.

..

..

.

.

..

.

..

.

.

..

.

...

..

.

.

..

..

..

.

....

..

..

..

..

..

..

..

..

..

..

..

....

.

...

....

....

.

..

...

.

.

....

.

.

..

....

.

.

..

..

.

.

..

.

.

..

.

.

..

.....

..

..

...

..

..

..

...

..

...

.

..

.

.

.

....

.

..........

....

..

..

.

...

......

.

.

....

..

.

.

.

.......

.

..

..

.

.

....

.

..

..

..

.

..

....

..

.

.

.

..

..

..

.

.

...

..

.

.

..

..

.

..

..

...

.

...

.

.

.

.

...

..

...

...

..

.

..

..

.....

....

.

..

..

.

...

.

.

.

.

..

.

..

..

...

...

.

.

.

....

.

...

.

...

.

..

.

.

.

..

.

.

.

.

.

.

..

.

.

.

..

..

.

..

.

..

.

.

..

.

.

..

..

.

.

.

.

..

..

.

.

....

.

.

..

..

..

..

..

..

..

....

..

..

..

..

..

..

.

..

...

....

..

.

.

..

.

.

.

....

.

.

.

.

.

.

..

...

..

..

..

.

.

......

.

..

....

..

....

...

...

.

..

.

.

.

..

.

.

.

...

....

.

..

.

.

..

.

..

..

.

.

.

..

.

..

.

.

...

.

...

..

.

.

..

..

.

..

.

..

..

.

..

..

..

..

....

..

...

....

..

..

..

.

..

..

.

.

..

..

..

.

.

...

.

.....

.

..

.

..

.

.

....

..

.

..

..

.

..

..

..

..

.

..

..

..

..

.

..

.

....

.

.

..

.

..

..

.

.

..

..

....

..

..

...

..

..

.

...

..

.

.

.

.

..

..

.

.

.

..

..

.

.

.

.

.

.

.

.

..

.

.

.

.

..

..

.

..

.

.

..

...

.

..

..

..

.

....

.....

..

.......

...

......

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

..

.

..

..

..

.

.

.

.

..

..

..

..

.

.

.......

..

........

.

..

.

..

..

.......

.

.

..

.

.....

.

.

.

..

..

.

..

..

..

.

..

.

..

..

.

.

..

..

....

.

.

.

.

.

.

..

.

.

..

..

..

..

.

..

..

.

..

...

.

.

.

..

.

.

..

.

..

.

.

...

.

..

..

.

..

....

..

.

..

..

..

..

...

.

.

.

.

.

...

.

.

...

...

.

.

.

.

..

.

.

..

..

.

...

..

..

..

.

..

.

..

..

..

.

..

.

...

..

..

..

..

..

..

...

.

.

.

..

.

..

.

.

.

.

.

..

....

..

..

.

.

..

.

..

.

..

..

..

..

..

....

.........

.

....

.

...

..

.

...

..

..

..

.

.......

.

.

..

......

..

..

.

.

...

...

..

..

.

.

.

..

....

.

.

..

..

..

.

..

....

.

....

.....

.

..

..

..

..

..

........

..

.

..

.

..

.

..

..

.

..

.

..

...

.

....

..

..

...

.

..

....

.

.

..

.

..

.

..

..

..

.

..

..

.

.

..

...

.

....

.

.

.

..

..

..

.

..

.

..

..

.

.

..

.

..

.

....

..

.....

..

....

....

..

.

...

.

........

.

...

..

..........

.

.

....

.

.

..

..

.......

..

.

..

.

...

.

..

..

...

.

..

..

.....

.

.

..

..

......

.

.

.

..

.

.

..

.

.

..

.

..

.

.

.

..

....

...

...

..

...

..

..

..

..

..

..

.

.

..

.

..

..

....

..

..

.

..

.

.

..

.

.

....

..

.....

.

.

.

...

.

.

..

..

......

..

..

..

..

..

..

.......

.

.

.

..

..

...

..

.......

..

.

......

......

.

......

..

....

.

.....

.

...

...

....

..........

.

..

.........

.

.

..

...

...

..

..

.......

.

.

.

..

.

.

..

.

.

..

.

.

.

.

.

..

.

.

.

.

..

.

.

..

.

.

.

..

.

....

.

.

..

.

.

..

..

..

..

..

..

.

..

..

.

..

..

..

..

.

.

..

.

.

..

..

.

..

..

.

..

..

..

.

.

..

..

.

.

.

..

..

.

.

.

.

..

...

..

.

.

..

.

.

.

....

..

....

..

..

.

.

..

..

..

..

.

....

..

...

..

...

..

..

..

...

..

..

.

..

..

..

....

..

..

.....

......

.

....

....

....

........

.

..

.

..

.

...

.

..

..

.

.

..

.

.

..

..

.

..

.

..

..

..

..

....

.

..

.

...

............

.....

.

.....

.....

.

..

.....

.

.....

.

......

.

.

.

.

...

.

..

.

.

..

.

..

.

..

.

.

....

...

.

.

.

..

.

.

.

..

.

..

.

..

..

..

.

..

..

..

.

..

.

..

..

.

..

..

.

...

..

.

.

..

.

.......

..

.

.

...

.

.

..

..

..

..

..

.

..

.

....

.

..

..

..

..

.

.

..

..

..

..

..

....

.

..

.

.

..

.

..

....

.

.

..

..

.

.

..

...

....

.

.

.

....

.

.

.

..

.

.

..

.

.

....

.

.

..

.

.

..

.

..

..

..

.

..

..

..

..

....

..

..

....

..

..

..

..

..

..

..

..

..

..

....

..

.

...

...

....

.

....

.

......

.

.

.

.

.

..

..

..

..

.

..

..

..

.

..

..

....

..

.

..

..

..

.

....

.

.

..

..

..

..

.

...

...

.

.

..

..

..

.

...

...

..

.....

.

...

..

..

..

..

.

.

..

.

.

..

.

.

..

.

.

..

..

..

.

.

...

...

.

.

..

.

..

..

..

..

.

.

..

..

.

..

..

...

.

..

...

..

.

...

.

..

.

..

..

.

..

.

.

.....

..

..

....

..

.

....

..

.

.

..

.

...

.

..

..

.

..

.

...

.

..

....

..

..

.

.

....

..

.

..

..

..

...

..

...

..

.

..

..

.

..

.

..

..

...

.

.

.

.

.

..

..

.

...

...

....

.

...

.

..

.

.

...

..

..

.

..

..

.

.

.

..

..

..

...

..

.

......

....

...

.

..

.

...

.

..

..

..

.

..

.....

..

.

.

..

.

...

..

..

..

....

.

.

.

.

.

...

.

.

..

.

..

..

.

..

.

..

.

..

.....

.

.

.

..

..

....

.

.

.

...

.

.

..

..

..

..

.

.

..

...

..

..

....

...

....

....

.

.

..

..

....

.

...

.

...

.

.

........

...

.

...

.

.

.

.

.

..

..

.

.

.

.

..

...

...

..

.

.

...

.

.

..

..

..

..

.

..

.

...

..

..

..

.

..

..

..

.

.

.

.

.

.

.

.

.

..

.

.

..

.

.

..

....

..

......

.

.

.

.

.

...

...

..

..

.

.....

.

..

..

......

.

.......

...

.......

.

..

.

.

.

..

..

....

.

.

.

..

.

.

..

.

..

.

.

..

.

..

.

.

..

..

..

.

.

..

..

.

..

.

..

..

..

..

.

.

....

.

.

..

..

.

.

.

..

..

..

.

.

..

.

.

..

..

....

.

...

..

..

..

.

..

.

..

..

.

..

.

.

..

.

...

.

.

..

..

.

.

..

.

....

..

..

..

.

...

..

..

....

...

.

...

...

...

..

..

.

..

..

....

..

...

...

.

..

.

.

.

.

..

..

..

.

.

..

......

.

..

...

.

.

...

.

...

.

.

..

.

.

.

..

.

..

.

.

.

..

..

..

.

.

.

..

.

.

..

..

...

.

.

.

..

..

..

..

.

..

...

..

...

.

..

..

.

...

.

.

.

.

..

..

.

.

..

.

.

.

.

...

.

.

...

.......

...

.

.

..

....

....

.

..

..

..

.

..

..

..

.

.

..

.

..

..

..

.

.

..

..

..

..

..

...

..

..

..

..

....

..

.

..

.

..

..

.

.

.

...

..

.

.

.

..

..

..

..

.

..

..

....

..

..

..

..

..

..

..

..

..

.

..

..

.

.....

.

....

..

.

....

.

.

...

..

.....

.

..

..

.

..

......

.

....

.

.

.

.

.

....

..

.

.

..

..

....

.

......

........

..

..

..

.

...

..

...

...

..

..

...

.

.

.

.

.

.

..

..

.

.

..

..

.

.

...

...

...

...

..

.

..

...

...

.

..

..

..

.

...

..

.

.

...

.

...

.

.

..

....

..

.

..

.

.

..

.

.

..

.

..

.

.

..

.

..

..

.

.

..

..

..

..

..

..

....

..

.

...

.

.

.

.

.

..

...

..

.

...

...

..

.

...

.

.

...

..

.

.

..

..

..

.

.

..

...

..

..

....

..

...

.

.

.

..

..

...

.

.

...

.

......

.

..

.

..

....

.

.

..

.

...

...

.

.

..

..

.

...

.

.

..

..

.

.

.

.

.

.

.

.

..

..

.

...

....

.

..

.

.

..

..

...

......

....

.....

.

....

.

.

...

..

.

...

.

....

.

..

..

...

...

.

...

...

.

..

..

..

.

...

..

.....

.

.

.

.

...

....

.

...

...

....

.

...

.

.

..

.....

.

...

...

.

.

.

..

.

..

..

.

.

.

.

.

.

.

.

.

.

.

.

.

..

..

..

.

.

.

.

.

.

.

.

.

.

....

.

.

.

..

..

.

.

.

.

....

....

..

.

..

...

..

.

.

.

..

.

.

..

.

..

..

.

.

..

.

..

.

........

.

..

..

..

....

..

...

..

........

.

.

.

...

.

.

.

.

....

.

.

..

...

...

...

...

...

.....

...

...

.

.

.

..

.

..

..

.

.

.

.

..

..

...

.

.

...

...

.

.

.

..

...

..

..

.

.

..

...

...

.

.

....

..

.

...

..

.

.

..

...

...

.

.

....

..

...

..

.....

.

.

.

...

.

.

...

......

...

.

.

..

.

...

.

.

.

...

.

.

.

..

..

..

.

.

.

.

..

.

.

..

..

.

..

.

.

..

..

.

..

....

.

.

..

.

..

..

..

..

.

...

....

.

.

...

..

.....

.

.

.

...

.

....

.

.

.

.

.

..

..

.

..

.

..

.

...

..

.

..

.

.

...

..

..

.

..

..

..

.

....

..

..

.

....

...

..

.

..

.

.

..

..

..

.....

.

.....

.

....

.

..

..

.

..

..

...

....

.

.

..

......

.

.

.

.

..

.

.

.

.

.

.

...

..

..

..

..

..

..

.

..

.

.

.

..

.

..

..

.

..

.

..

.

....

..

..

..

........

....

.

..

.

...

..

.

..

...

..

..

.....

.

.

.

.

.

.

....

.

.

.

..

..

..

.

..

..

.

...

..

..

.

.

..

.

.

.

.

.

.

..

.

.

.

.

.

..

.

...

.

..

..

.

..

..

.

..

..

....

...

.

.....

.

..

..

.

..

.

.

.

...

..

...

....

.

....

.

..

.

.

..

..

.....

...

....

...

....

.

.

..

..

..

.

.

..

..

.

...

..

..

.

.

..

..

..

..

..

..

...

.

.

..

.

...

....

.

.

...

.

.

.

..

..

.

...

..

..

.

.

.

.

.

.

.

....

..

......

.......

.

..

.

.

.

...

.

.

.

..

...

.

.

.

..

..

.

.

.

.

.

.

.

.

..

...

.

.

.

.

.

..

.

.

.

.

.

..

.

..

..

.

.

.

.....

.....

..

.

.

...

.

.

.

..

..

..

.

..

..

.

..

..

..

..

....

.

..

..

.

.

..

..

.

.

..

.

.

..

..

.

.

..

..

.

......

...

..

..

..

..

.

..

..

........

..

.

..

.....

..

...

..

.

....

..................

.............

.

....

.

..

...

..

...

.

..

.

.

..

.

...

.

.

.

.

.....

...

.

......

...

.

.

.

.

..

..

.

..

...

......

.

........

...

.

.

..

..

.

.

....

.

.

..

..

..

.

..

..

...

.

.

..

..

.

..

..

.

.

..

.

.

.

..

..

.

.

.

.

..

...

.

.

.

..

.

.

..

..

.

.

.

..

..

..

.

.

..

.

..

.

.

..

.

.

..

....

..

......

..

......

..

..

..

..

..

.

..

.

..

...

..

..

.

.

..

..

.

.

...

.

.....

.....

.

.

..

.

.

.

.

..

..

.

.

.

..

..

.

.

.

.

.

..

..

.

.

.

..

..

..

.

..

.

.

.

.

...

...

.

.

...

..

..

.

.

.

..

.

.

..

..

...

.

.

.

....

..

..

..

..

..

.

.

..

..

..

..

..

.

..

..

....

..

..

..

.

..

.

....

...

.

..

...

.

.

..

.

......

.

......

..

.

..

......

..

..

..

..

..

....

..

.

....

.

.

..

..

.

.

.

.

.

.

.

..

...

.

...

...

..

..

..

..

...

.

....

...

.

..

.

..

...

.

...

.

.

.

.

...

..

...

.

.

.

....

.

.

.

..

..

.

..

.

.

..

..

.

...

.

.

..

...

.

.

..

..

.

..

..

.

..

.....

...

....

.

..

.

..

...

.

..

..

....

..

...

.

..

..

.

...

.

.

.

.

...

...

.

.

.

....

......

.

.

.

.

.

.

...

.

.

..

...

..

..

.

.

...

..

.

.

.

..

..

..

..

.

..

...

...

....

.

.

.

.

.

..

.

..

.

..

..

..

.

.

..

.

..

.

.

...

..

......

..

.

..

.

.

..

......

.....

.

..

....

..

.

..

.....

..

...

.

..

.

...

.

.

...

.

...

...

.

...

.

..

...

..

.

.

.

..

..

..

..

..

..

..

..

..

..

..

.

.

.

..

..

..

..

..

..

..

..

......

.

.

..

.

...

..

.

.

.

.

..

..

..

.

.

...

..

.

.

.

..

.

.

..

.

.

.....

..

..

..

..

...

..

.

.

..

..

..

..

.

.

..

.

.

..

.

...

.

..

..

..

.

.

.

..

.

.

..

..

.

....

..

..

..

..

.

.

.

.

..

.

..

..

..

..

.

.

..

.

..

..

.

..

..

..

..

..

...

.

.

.

.

..

..

.

..

...

.

.

..

.

..

.

.

..

.

.

..

..

.

.

...

...

.

..

.

.

..

..

.

..

.

..

.

..

.

..

.

.

.

.

..

.

.

.

.

..

.

.

.

..

..

..

..

..

.

.

..

.

.

.

..

..

.

.

...

...

.

...

..

..

.

...

..

..

..

..

.

..

.

..

.

.

..

..

..

..

..

..

......

..

......

.

....

....

.....

.

..

..

.

.

..

..

..

.

....

..

...............

.

.

.

.

.

.

..

.

.

..

.

.

..

.

..

..

..

..

.

.

.

.

..

.

..

...

....

..

...

.

..

.

.....

.

..

.

.

.

..

..

..

..

..

..

..

..

...

.

.

.

....

......

.

.

..

..

..

..

.

..

.

.....

.

.

..

.

.

.

......

.

.

..

..

.

..

..

..

..

..

..

.

..

..

.

.

..

.

..

.

.

.

..

..

..

.

.

.

.

...

..

..

..

..

..

.

.

...

..

....

....

.

..

..

..

..

.

.

......

..

....

.

.

.

.

..

.

.

..

.

.

.

.....

..

.

.

...

.

...

.

..

.

.

..

.

...

.

..

..

.

.

..

.

.

.

..

..

.

..

..

..

.

..

..

..

..

.

.

.

..

.

..

.

.

..

.

..

.

.

..

.....

..

..

...

.

..

.

..

.

.

.

..

..

..

..

..

.

..

.

..

.

.

...

....

..

..

..

..

..

..

..

.

.

.

.

...

.

....

.

.

...

...

...

.

.

.

.

..

..

.

.

.

..

..

.

.

..

.

.

..

..

..

..

..

..

.

.

..

..

.

.

..

.

.

.

.

.

.

.

.

.

.

...

.

....

.

......

.

.

.

.

.

..

..

.

.

.

.

..

...

.

.

.

..

...

.

..

.

.

..

.

.

..

.

.

..

..

.

..

..

.

.

.

....

..

..

..

..

.

.

.

..

....

..

..

.

.

..

..

..

.

..

.

.

..

.

..

.

..

..

..

.

.

...

.

.

..

..

.

..

.

.

...

..

....

..

..

.

.

..

..

.

.

.

...

.

.

...

.

.

.

..

.

.

.

.....

.

...

...

..

.

.

.

.

..

..

..

.

..

..

.....

..

.

....

..

.

..

..

.

..

..

.

..

.

.

..

..

.

..

.

..

.

..

.........

..

..

..

.

.

.

...

...

..

.

..

.

...

..

...

......

..

.

..

..

..

...

.

..

.

.

..

..

..

...

..

.

.

..

....

.

.

..

..

.

..

.....

.

....

.

.

.

.

.

.

.

.

..

.....

.

.

.

.

.

.

.

..

.

.

...

.

.

..

..

..

..

.

..

.

...

.

.

..

...

.

.

..

..

..

.

.

...

.

.

.

.

..

...

.

...

..

.

.

.

..

.

.

.

...

.

.

.

...

.

.

.

..

...

.

.

.

.

.

..

...

..

..

..

..

..

..

..

..

.

.

..

..

.

..

..

..

.

..

....

..

.

..

..

..

..

..

.

.

..

..

..

..

....

..

.

...

.

..

...

.

.

.

..

...

.

.

...

.

.

..

..

...

.

.

..

.

...

.

.

..

.

..

..

.

.

..

..

.

..

..

......

..

..

.......

..

...

..

...

.

.

....

.

.

.

..

.

..

...

.

...

.

..

..

....

.

..

.

..

.

....

...

...........

.

.

......

...

..

.

..

....

...

..

..

.

....

.

.

..

.

.

.

..

...

.

.

..

.

.

.

.

..

.

..

.

.

.

.

...

.

..

..

..

..

.

.

...

..

..

..

..

.

..

.

...

..

..

..

..

.

...

.

.

..

..

..

.

.

.

.

.

..

..

.

..

.

..

.

.

.

.

.

.

.

..

...

.

.

.

.

..

...

.

..

..

..

..

.

..

.

..

..

..

..

..

...

..

..

..

.

..

..

..

..

..

..

..

....

..

..

.

.

..

.

..

.

...

.

..

.

..

.

.

....

.

..

..

...

..

.

..

.....

...

.

.

..

..

.

...

..

..

..

.

..

....

.

..

.

.

..

..

.

.

.

..

.....

..

.

..

.

.

.

...

.

.

..

..

..

...

.

..

.

.

.

..

..

.

..

.

.

..

..

...

...

.

..

..

..

..

..

.

.

..

.

.

.

.

..

..

.

.

.

.

..

.

...

.

.

...

.

.

.....

.

.

..

.

.

..

.

.

.

...

....

.

.

....

.

.

.

.

..

.

....

..

.....

.

..

..

.

.

..

..

..

..

..

..

..

..

..

..

..

..

..

..

..

..

.

.

..

..

.

..

..

..

..

.

.

..

.

..

...

..

.

..

..

.

....

..

.

.

..

..

..

...

......

..

..

.

..

..

....

..

..

.............

..

...............

.

.

.

.

.

.

....

.

.

..

..

.

.

..

..

.

.

.

.

.

..

..

.

.

.

.

...

.

.

.

.

..

.

.

..

.

.

...

.

.

.

.

.

.

..

.

.

.

..

.

.

.

..

..

.

..

..

..

..

.

.

.

.

.

....

..

..

.

.

..

.

.

.

..

.

.

..

.

..

..

..

.

...

..

..

..

..

..

..

.

.

....

.

.

.

..

..

...

.

.

...

..

..

.

..

.

..

.

.

.

..

.

.

..

.

.

.

..

.

.

..

.

.

..

....

..

.

.

.

..

...

..

..

....

..

..

....

..

..

.

.

..

..

.

.

..

.

.

..

..

.

.

..

..

.

.

..

..

..

..

..

..

..

....

....

..

.

.

.

...

.

..

..

..

..

..

..

..

..

..

.

..

..

.

.

.

....

.

..

...

.

.

.

..

.

.

.

..

.

.

.

...

.

.

..

.

.

..

.

..

.

..

.

.

..

.

..

.

..

.

..

..

.

..

.

..

.

..

.

..

.

..

.

.

..

.

..

..

...

.

.

..

.

..

..

...

..

..

.

..

...

................

..

.

..

..

..

..

...

.

.

..

..

...

....

.

..

..

..

...

....

...

...

.

.

.

..

..

.

.

.

.

...

.

..

..

..

.

.

.

.

...

.

..

.

.

.

.

.

.

...

...

..

.

..

.

..

..

.

.....

.

.

..

.

..

..

.

..

..

.

..

.

.

.

..

.....

..

.

.

..

.

.

....

..

..

.

..

.

.

..

...

..

.

.

....

......

.

.

...

..

..

...

.

.

..

.

..

....

..

.

....

..

...

.

.

.

..

.

..

..

.

...

.

..

..

..

..

..

.

..

.

..

.

..

.

..

.

.

....

..

.

.

.

..

..

....

.

.

.

...

.

.

.

.

..

...

....

.

.

..

..

.

...

.

.

.

..

..

...

...

...

.

..

.

.

.

.

.

.

.

.

.

.

.

..

.

..

.

.

.

..

..

.

..

..

..

..

..

.

.

..

..

.

.

.

.

.

.

.

..

..

.

.

.

.

..

.

.

...

.

...

...

.

.

.

.

.

.

..

...

.

.

.

.

...

.

.

..

.

.

...

..

..

.

.....

..

....

..

.

.

..

..

.

...

.

..

.

..

..

..

..

.

..

..

.

.

..

..

.

..

..

.

..

..

.

..........

..

....

..

..

....

....

.

..

..

...

......

....

...

..

......

.

.

....

...

.

.

..

....

.

..

.

..

.

.

.

......

...

.

.

...

..

.

...

.

.

.

..

..

..

..

.

.

.

..

..

..

..

..

..

.

...

.

.

.

..

..

....

...

.

...

.

.

....

.

..

..

....

...

.......

.

........

.

.

.

.

.

.

.

.

.

..

..

.

.

.

.

.

.

...

...

.

.

.....

...

.

...

...

...

.

.

.

..

...

.

..

..

.

.

..

.

.

.

..

.

.

..

..

....

..

.

.

..

.

..

..

.

.

.

..

.

.

.

..

.

.

.

..

.

..

..

.

.

..

..

.

..

..

.

.

...

..

....

...

...

..

..

.

....

..

.

...

..

..

.

.

..

.

.

..

.

..

.

..

.

..

.

..

..

.

.

..

.

..

.

.

..

.

.

..

..

..

..

..

..

..

.

..

..

..

.

..

..

..

..

..

..

..

......

...

..

.

...

.

.

.

....

..

.

.

..

..

.

.

...

.

...

.

..

..

..

..

..

..

..

..

..

....

.

....

........

.

.

.

.

.

..

..

.

..

.

...

.

.

.

..

.

..

...

.

..

..

....

.

.

.

.

..

...

.

.

.

.

..

..

.

.

.

.

..

..

..

..

..

.

.

..

..

...

..

.

.

.

.

....

.

..

..

.

....

..

.

..

..

..

..

..

.

.

.

.

.

....

.

..

.

.

..

..

..

.

.

.

...

..

.

.

.

.

...

..

..

.

.

.

.

..

..

..

......

.

..

.

.

.

...

.

.

...

.

...

.

.

.

..

.

..

..

..

.

..

.

.

...

.

...

..

.

.

..

..

..

.

.

..

..

.

..

.

.

..

.

..

..

.

.

..

..

.

.

..

.

..

..

.

.

..

..

.

..

.

...

..

.

.

.

..

.

..

.

.

..

..

..

.

.

.

..

..

.

.

.

....

..

.

.

....

..

.

.

..

.

.

.

..

.

.

..

.

..

.

..

..

.

.

...

.

.

..

....

.

...

..

..

..

..

....

..

....

....

....

..

..

..

..

..

..

.

.

.

....

......

.

.

..

....

.

.

..

..

..

.

.

....

..

....

..

....

.

.

.

....

.

.

.

.

..

.

....

.

.

.

.

.

.

...

.

.

.

.

.

.

.

.

.

..

..

.

..

.

.

..

.

..

..

.

..

..

.

.

..

..

.

.

.

.

.

.

.

.

....

...

.

.

.

.

.

.

.

.

..

.

..

...

.

.

.

.

..

..

..

.

.

..

..

.

.

.

.

.

.

.

.

..

....

..

.

.

.

.

...

.

....

..

.

.

..

...

.

..

.

.

.

..

..

.

.

.

.

..

..

.

.

.

..

.....

.

.

..

..

.

.

..

..

..

.

.

..

..

.

.

..

..

..

..

...

.

....

...

...

.

.

..

..

.

..

..

..

....

.

.

..

..

.

..

..

..

...

..

.

..

.

..

....

.

..

.

.

..

..

..

.

..

..

..

..

.

..

....

..

.

..

.

....

..

..

...

...

..

.

.

.

.

........

.

..

.

....

..

..

..

.

....

.......

.....

..

..

.

.

.

.

....

.

.

.

.

.

.

..

.

..

...

.

..

..

.

.

.

.

.

.

..

.

..

..

..

.

....

..

.

...

.

.

...

.

...

...

...

.

.

....

.

.

.

.

..

...

.

..

...

..

.

.

.

.

.

..

.

..

..

.

....

.

..

.

...

.

...

.

...

.

.

.

....

.

.

.

.

...

..

..

.

..

..

.

..

...

.

.

.

..

..

.

...

...

.

..

.

..

..

..

..

.

.

.

.

..

......

.

.

.

.

..

.

..

..

.

..

.

..

.

...

.

.

..

.

.

.

..

.

..

.

..

.

..

..

..

.

..

.

......

..

..

..

..

..

.

.

...

..

..

..

....

..

..

..

..

..

.

.

..

...

..

...

.

....

.

.

..

...

..

....

...

.

..

.

.

.

..

.

......

.

..........

.

...

...

..

..

..

.

.

...

.

.

.

.

.

..

.

..

..

...

...

.

...

...

.

.

.

.

.

.

.

..

...

.....

........

.

........

.

..

..

.

..

.

..

.

..

........

.

...

.

.

...

........

...

...

...

..

.

...

..

...

.

....

...

...

.

...

.

..

...

...

..

.

.

....

..

.

.

.

.

.

..

..

..

.

.

....

.

.

..

..

..

..

.

.

..

.

..

..

..

..

..

..

..

..

.

....

..

..

.

....

.

..

......

...

......

..

........

.

.

.

.

...

.....

.

.

.

..

.

.

.

.

..

..

.

.....

.

.

....

..

.

...

.

.

..

..

.

.

.

..

..

..

...

..

..

.

.

.

.

.

........

.

...

.

.

..

.

.

.

..

..

..

.

..

...

.

.

..

..

.

.

..

.

.

..

...

..

..

..

....

..

.

.

...

.

..

.

.

.

..

.

..

.

..

.

..

..

..

.

.

..

..

.

.

..

.

..

..

.

.

.

.

.

..

..

.

.

.

.

.

..

...

.

..

..

..

.

.

.

.

..

..

.

.

.

....

.

.

..

..

..

..

.

..

..

.

..

..

.

..

.

..

..

.

..

....

....

...

....

..

.

.

.

..

.

.

.

....

.

.

.

...

.

.

..

...

..

..

..

....

..

.

..

.

......

...

.

......

...

...

....

.

..

..

.

.

..

..

..

.

.

.

...

.

.

..

.

.

....

...

.

...

.

...

...

.

.

..

..

.

.

..

..

.

.

.

..

..

..

....

.

.

.

...

...

...

.

.

.

...

.

.

.

.

..

.

......

..

...

.

.

..

..

.

....

..

...

..

....

..

........

...

.

.

.

.

.......

.

...

.

...

.

.

..

.

....

..

.

.

...

...

.

.

.

.

.

..

..

.

.

...

.

..

.

...

.

....

..

.

.

.

.

..

.

.

...

.

.

.

..

..

...

..

.

...

.

..

.

.

..

.

..

.

.

.

...

.

.

..

...

...

......

.

...

..

.....

.

.

..

.

.....

.

....

.

.

..

.

..

..

..

.

..

.

.

....

..

..

.

.

..

..

.

.

.

.

.

..

..

.

.

.

.

..

..

..

..

.

.

.

.

....

.

.

..

..

..

.

.

.

.

.

..

.

..

..

..

.

.

..

..

..

..

..

.

..

..

....

.

....

..

.

.

.

..

..

.

.

....

.

.

.

......

.

.

.

.

...

.

.

...

.

.

.

....

.

.

.

.

.

...

.

.

.

.

.

...

.

.

..

.

..

..

.

..

....

.

.

....

.

..

..

........

.

..

..

.

...

.

.

.

....

.

.

....

.

.

.

...

.

.

...

.

.

...

....

.

.

........

.

.

.

...

.

...

.

.

.

.

..

...

..

.

..

..

.

.

..

..

.

.

.

...

.

.

.

.

.

.

.

.

..

.

.

...

.

.

.

..

.

.

.

.

..

.

.

.

..

..

.....................

...

.

...

.

.....

.

...

...

..

..

.

..

...

..

....

.

..

..

.

.

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.

..

.

.....

.

...

...

..

.....

..

..

...

.

.

...

..

.

...

..

...

.

...

..

..

.

.

..

.................

...

..

..

.

..

...

.

...

.

.

.

.

.

.

...

.

.

.

..

.

.

.

.

.

.

.

.

.

.

.

.....

..

..

....

....

.

..............

.

..

.

..

.........

.

...................

.......

.......

...

.

..

..

..

.

.

.

.

.

.

.

.....

......

.

.

......

..

.

...

....

.

...

..

..

.

.

.

..

..

..

.

.

..

.

.

.

.

....

..

..

.

.

.......

..

..

.

...

.

..

...

.

....

.

..

.

..

..

.

....

...

....

....

.....

..........

.

.....

.

..

.

.

.

.

...

.

.

.

.

..

...

..

.

...

..

...

....

..........

...

..

..

.

..

.

.

...

.

.

..

........

.

.

..

...

.

.

.

.

..

.

.

.

.

.....

.

.

.

..

.

..........

..

.

......

...

.

..

.

.

.

.

.

.

..

.

.

.

.....

...

.....

...

........

........

.

.

.

...

.

....

.

.

....

...........

........

.........

.........

.

.

..

..

.....

..

.................

..

.

......

..

....

....

.......

.

. ........

.....

......

.

..

.

........

...

........

..

........

....

........

.

..

.. .. ... ...

....

.......

.

.

.

.. ...

..... . .

.

.

. ........... .

.

.

..

.. .....

.............................................................

.

....................

.

..

....

....

..

TimeÛ

Pack

et N

umbe

r for

Eac

h C

onne

ctio

n

Ü

0.0 0.2 0.4 0.6 0.8 1.0

0Ý

100Ý

200Ý

300Ý

Figure9: A RED gateway simulationwith heavy congestion,two-way traffic, andmany shortFTP andTELNETconnections.

13

3Þ1ß2à

4á

GATEWAYâ1

ß

4á

GATEWAYâ

1ß

4á

Aã

5ä6å7æ8ç

B

45Mbpsè

100Mbpsé100Mbps

é2msê1ms

é5msë 2ms

ê1msé5msë

0.5msì

3msí

0.5msì

Figure10: A network with many shortconnections.

To investigatetheperformanceof RED gatewaysin arangeof traffic conditions,thissectiondiscussesasimula-tion with two-waytraffic, wherethereis heavy congestionresultingfrom many FTPandTELNETconnections,eachwith a smallwindow andlimited datato send.TheREDgatewayparametersarethesameasin thesimplesimula-tion in Figure3, but thenetwork traffic is quitedifferent.

Figure9 showsthesimulation,whichusesthenetworkin Figure10. Roughlyhalf of the41connectionsgo fromone of the left-handnodes1-4 to one of the right-handnodes5-8; theotherconnectionsgo in theoppositedirec-tion. The roundtrip timesfor the connectionsvary by afactorof 4 to 1. Most of the connectionsareFTP con-nections,but therearea few TELNET connections.(Oneof the reasonsto keepthe averagequeuesizesmall is toensurelow averagedelayfor theTELNET connections.)Unlike theprevioussimulations,in this simulationall ofthe connectionshave a maximumwindow of either8 or16 packets.Thetotal numberof packetsfor a connectionrangesfrom 20 to 400packets.Thestartingtimesandthetotal numberof packetsfor eachconnectionwerechosenratherarbitrarily; we arenot claimingto representrealis-tic traffic models. The intentionis simply to show REDgatewaysin a rangeof environments.

Becauseof the effectsof ack-compressionwith two-way traffic, thepacketsarriving at thegateway from eachconnectionaresomewhat bursty. Whenack-packetsare‘compressed’in a queue,the ack packets arrive at thesourcenodein a burst. In response,the sourcesendsaburstof datapackets[38].

The top chart in Figure9 shows the queuefor gate-wayA, andthenext chartshowsthequeuefor gatewayB.For eachchart,each‘X’ indicatesapacketdroppedatthatgateway. The bottom chart shows the packets for eachconnectionarriving anddepartingfrom gateway A (andheadingtowardsgatewayB). For eachconnection,thereisa markfor eachpacket arriving anddepartingfrom gate-way A, thoughat this time scalethetwo marksareindis-tinguishable.Unlikethechartin Figures3, in Figure9 the

packetsfor the differentconnectionsaredisplayedover-lapped,ratherthan displayedon separaterows. The x-axisshows time,andthey-axisshows thepacketnumberfor thatconnection,whereeachconnectionstartsatpacketnumber0. Forexample,theleftmost‘strand’showsacon-nectionthat startsat time 0, andthat sends220 packetsin all. Each‘X’ shows a packet droppedby oneof thetwo gateways. The queueis measuredin packetsratherin bytes;shortpacketsarejust aslikely to bedroppedasarelongerpackets. The bottomline of the bottomchartshowsagainan‘X’ for eachpacketdroppedby oneof thetwo gateways.

BecauseFigure 9 shows many overlappingconnec-tions, it is not possibleto tracethe behavior of eachoftheconnections.As Figure9 shows, theRED gateway iseffectivein controllingtheaveragequeuesize.Whencon-gestionis low at oneof thegateways,theaveragequeuesize and the rateof marking packets is also low at thatgateway. As congestionincreasesat thegateway, theav-eragequeuesizeandtherateof markingpacketsbothin-crease.Becausethissimulationconsistsof heavy conges-tion causedby many connections,eachwith asmallmax-imum window, the RED gatewayshave to drop a fairlylarge numberof packets in order to control congestion.Theaveragelink utilizationover theone-secondperiodis61%for thecongestedlink in onedirection,and59%fortheotherdirection.As thefigureshows,thereareperiodsat the beginning andthe endof the simulationwhenthearrival rateat thegatewaysis low.

Note that the traffic in Figures3 and9 in quite var-ied, andin eachcasetheRED gateway adjustsits rateofmarkingpacketsto maintainanacceptableaveragequeuesize. For the simulationsin Figure 9 with many shortconnections,thereareoccasionalperiodsof heavy con-gestion,and a higher rate of packet drops is neededtocontrol congestion. In contrast,with the simulationsinFigure3 with a small numberof connectionswith largemaximumwindows,thecongestioncanbecontrolledwitha small numberof droppedpackets. For the simulationsin Figure9, the burstinessof the queueis dominatedbyshort-termburstinessaspacket burstsarrive at the gate-way from individual connections.For thesimulationsinFigure3, theburstinessof thequeueis dominatedby thewindow increase/decreasecycles of the individual con-nections.Note that theRED gateway parametersareun-changedin thesetwo simulations.

Theperformanceof aslightlydifferentversionof REDgatewayswith connectionswith differentroundtriptimesandwith connectionswith multiple congestedgatewayshasbeenanalyzedandexploredelsewhere[5].

14

9 Bursty traffic

ThissectionshowsthatunlikeDropTail or RandomDropgateways,REDgatewaysdonothaveabiasagainstburstytraffic.5 Bursty traffic at the gateway canresult from anFTPconnectionwith a long delay-bandwidthproductbutasmallwindow; awindow of traffic will besent,andthentherewill bea delayuntil theackpacketsreturnandan-otherwindow of datacanbesent.Variable-bit-ratevideotraffic andsomeforms of interactive traffic areotherex-amplesof burstytraffic seenby thegateway.

In this sectionwe useFTP connectionswith infinitedata,smallwindows,andsmall roundtriptimesto modelthe less-bursty traffic, andwe useFTP connectionswithsmallerwindowsandlongerroundtriptimesto modelthemore-burstytraffic.

We considersimulationsof thenetwork in Figure11.Node5 packetshavearoundtriptimethatis six timesthatof the otherpackets. Connections1-4 have a maximumwindow of 12 packets, while connection5 hasa maxi-mum window of 8 packets. Becausenode5 hasa largeroundtriptime anda smallwindow, node5 packetsoftenarrive at thegateway in a loosecluster. By this,we meanthat consideringonly node5 packets, thereis one longinterarrival time,andmany smallerinterarrival times.

1î

SINKï

GATEWAYð

FTP SOURCES

2 3 4

5

6ñ7ò

d5,6

d6,7

= 2ms

45Mbps

100Mbps

d(1,2,3,4),6 = 1ms

= 16ms

45Mbps

Figure11: A simulationnetwork with five FTP connec-tions.

Figures12 through14show theresultsof simulationsof thenetwork in Figure11with DropTail, RandomDrop,andRED gatewaysrespectively. Thesimulationsin Fig-ures12and13wererunwith thebuffer sizerangingfrom8 packets to 22 packets. The simulationsin Figure 14wererun many timeswith a minimumthresholdrangingfrom 3 to 14packets,andabuffer sizerangingfrom 12to56packets.

5By bursty traffic we mean traffic from a connectionwhere theamountof datatransmittedin oneroundtriptime is small comparedtothedelay-bandwidthproduct,but wheremultiplepacketsfrom thatcon-nectionarrive at thegateway in ashortperiodof time.

++

+

+

+

+++

+

+

+

+

+

+++

++

+

+

+

+

+

++++

+

+

+

+

+

+

+++++

++

+

+

+++

+++

+

+

+

++++

+

++

+

+

+

+++

++

+

++

+

++++++

++

++

+

++

+

++

+

+

++

+

++

+

+

++

+++

+

+++++++++

+

+++++++++

+

+++++++++

+

+++++++++

+

+++++++++

Buffer Sizeó

No

de

5 T

hro

ug

hp

ut

(%)

ô

8õ

10ö 12÷ 14ø 16ù 18õ

20ö 22÷0

ú

1û

2

ü

3

ý

4

þ

..

. ..

. . . ..

. . . . .

Buffer SizeÿAver

age

Que

ue (i

n pa

cket

s)

�8�

10 12� 14� 16 18 20 22�0� 5� 1015

..

. ..

. . . . . . . . . .

Buffer SizeÿAver

age

Link

Utili

zatio

n

�8�

10 12� 14� 16 18 20 22�0.80�

0.90�

1.00�

Figure12: Simulationswith DropTail gateways.

Eachsimulationwas run for ten seconds,and eachmarkrepresentsoneone-secondperiodof thatsimulation.For Figures12 and13, the x-axis shows the buffer size,andthey-axisshowsnode5’s throughputasa percentageof the total throughputthroughthe gateway. In ordertoavoid traffic phaseeffects(effectscausedby the precisetiming of packet arrivals at the gateway), in the simula-tionswith Drop Tail gatewaysthesourcetakesa randomtimedrawn from theuniformdistributionon[0, t] secondsto prepareanFTPpacket for transmission,where 7 is thebottleneckservicetime of 0.17ms. [7]. In thesesimula-tionsourconcernis to examinethegateway’sbiasagainstburstytraffic.

For eachset of simulationsthere is a secondfigureshowing the averagequeuesize(in packets)seenby ar-riving packetsatthebottleneckgateway, andathird figureshowing theaveragelink utilizationonthecongestedlink.

15

+

+

+

++

+++++

++

+++

+

++

++

+

+

+

+

+

++

+++

+

++

++

+++

+

+

++

++++++

+

+

+

+

+

++

+

+

++

+

+

+++

+

+

+++

++

+++

+

++

+++

+

+++

+

+

++

+

+

++

+

+++++++

++

+

+++++++

+

+++++++++

+

+++++++++

+

+++++++++

+

+++++++++

Buffer Sizeó

No

de

5 T

hro

ug

hp

ut

(%)

ô

8õ

10ö 12÷ 14ø 16ù 18õ

20ö 22÷0

ú

1û

2

ü3

ý4

þ

. . .. .

. .. .

. . . . . .

Buffer SizeÿAver

age

Que

ue (i

n pa

cket

s)

�8�

10 12� 14� 16 18 20 22�0� 5� 1015

. . .. .

. . . . . . . . . .

Buffer SizeÿAver

age

Link

Utili

zatio

n

�8�

10 12� 14� 16 18 20 22�0.80�

0.90�

1.00�

Figure13: Simulationswith RandomDropgateways.

BecauseREDgatewaysarequitedifferentfrom DropTailor RandomDrop gateways,thegatewayscannotbecom-paredsimplyby comparingthemaximumqueuesize;themostappropriatecomparisonis betweenaDropTail gate-way anda RED gateway thatmaintainthesameaveragequeuesize.

With Drop Tail or RandomDropgateways,thequeueis morelikely to overflow whenthequeuecontainssomepackets from node5. In this case,with eitherRandomDropor DropTail gateways,node5 packetshaveadispro-portionateprobability of beingdropped;the queuecon-tentswhenthe queueoverflows arenot representative oftheaveragequeuecontents.

Figure14 shows the resultof simulationswith REDgateways.Thex-axisshows � � ����� andthey-axisshowsnode5’sthroughput.Thethroughputfor node5 is closetothemaximumpossiblethroughput,givennode5’sroundtrip

Minimum Threshold

Node 5

Thro

ughput (%

)

4�

6�

8

10� 12� 14�0�

1�

2�

3�

4�

..

..

..

..

. .. .

Minimum Threshold�Aver

age

Que

ue (i

n pa

cket

s)

�4� 6� 8

�10 12� 14�0� 5� 10

15

..

. .. . . . . . . .

Minimum Threshold�

Aver

age

Link

Utili

zatio

n

�4� 6� 8

�10 12� 14�0.

80�

0.90�

1.00�

Figure14: Simulationswith REDgateways

timeandmaximumwindow. Theparametersfor theREDgateway areasfollows: =@?(t : . ::�u and ����� " t 0 + w : .Themaximumthresholdis twice theminimumthresholdandthebuffer size,which rangesfrom 12 to 56 packets,is four timestheminimumthreshold.

Figure15 shows thatwith thesimulationswith DropTail or with RandomDrop gateways,node5 receivesadisproportionateshareof thepacket drops.EachmarkinFigure15 shows theresultsfrom a one-secondperiodofsimulation. The boxesshow the simulationswith DropTail gatewaysfrom Figure12, thetrianglesshow thesim-ulationswith RandomDropgatewaysfromFigure13,andthe dotsshow the simulationswith RED gatewaysfromFigure 14. For eachone-secondperiod of simulation,the x-axis shows node 5’s throughput(as a percentageof the total throughput)and the y-axis shows node5’spacket drops(asa percentageof the total packet drops).

16

(square for Drop Tail, triangle for Random Drop, dot for RED)�Node 5 Throughput (%)�

Nod

e 5

Dro

ps (%

)�

0� 2� 4� 6� 8� 10

0�

1020

3040

50

....

..

..

.

.

...

.

.

...

..

.

.

.

.

. .

.

.

. .

.

..

..

.

..

.

.

.

.

...

.

..

.

.

.

.....

.

.

. .

.

.

...

.

.

..

..

.

.

.

.

..

.

.

.

.

.

....

.

.

.

......

.

.

. ....

.

.....

Figure15: Scatterplot, packetdropsvs. throughput

The numberof packetsdroppedin oneone-secondsim-ulationperiodrangesfrom zeroto 61; thechartexcludesthoseone-secondsimulationperiodswith lessthanthreedroppedpackets.

Thedashedline in Figure15showsthepositionwherenode5’s shareof packet dropsexactly equalsnode5’sshareof the throughput. The clusterof dots is roughlycenteredon the dashedline, indicatingthat for the REDgateways,node5’sshareof droppedpacketsreflectsnode5’sshareof thethroughput.In contrast,for simulationswithRandomDrop (or with Drop Tail) gateways node5 re-ceivesa small fractionof thethroughputbut a largefrac-tion of thepacketdrops.Thisshowsthebiasof DropTailandRandomDropgatewaysagainsttheburstytraffic fromnode5.

Our simulationswith an ISO TP4 network usingtheDECbit congestionavoidanceschemealso show a biasagainstburstytraffic. With theDECbit congestionavoid-anceschemenode5packetshaveadisproportionatechanceof having theircongestionindicationbitsset.TheDECbitcongestionavoidancescheme’s biasagainstburstytrafficwouldbecorrectedby DECbitcongestionavoidancewithselective feedback[28], which hasbeenproposedwith afairnessgoalof dividing eachresourceequallyamongallof theuserssharingit. This modificationusesa selectivefeedbackalgorithmat the gateway. The gateway deter-mineswhich usersareusingmorethantheir “f air share”of thebandwidth,andonly setsthecongestion-indicationbit in packetsbelongingto thoseusers.We have not runsimulationswith thisalgorithm.

10 Identifying misbehaving users

In this sectionwe show that RED gateways provide anefficient mechanismfor identifying connectionsthat use

a large shareof the bandwidthin times of congestion.BecauseRED gateways randomlychoosepackets to bemarked during congestion,RED gateways could easilyidentifywhichconnectionshavereceivedasignificantfrac-tion of therecently-markedpackets.Whenthenumberofmarkedpacketsis sufficiently large,a connectionthathasreceiveda largeshareof themarkedpacketsis alsolikelyto be a connectionthat hasreceiveda large shareof thebandwidth.This informationcouldbeusedby higherpol-icy layersto restrict the bandwidthof thoseconnectionsduringcongestion.

TheRED gateway notifiesconnectionsof congestionat thegateway by markingpackets.With RED gateways,when a packet is marked, the probability of marking apacket from a particular connectionis roughly propor-tional to thatconnection’scurrentshareof thebandwidththroughthegateway. Notethatthispropertydoesnotholdfor Drop-Tail gateways,asdemonstratedin Section9.

For therestof this section,we assumethateachtimethegateway marksa packet, theprobabilitythata packetfrom aparticularconnectionis markedexactlyequalsthatconnection’s fractionof the bandwidththroughthegate-way. Assumethatconnection� hasa fixed fraction � ¡

ofthebandwidththroughthegateway. Let

¡ ! Ä bethenum-berof the � most-recently-markedpacketsthat arefromconnection� . Fromtheassumptionsabove, theexpectedvaluefor

¡ ! Ä is � � ¡.

Fromstandardstatisticalresultsgivenin theappendix, ¡ ! Ä is unlikely to bemuchlargerthanits expectedvaluefor sufficiently large � :¾D¿ 3gÀ & ¡ ! Ä Ø�2 � ¡ � * �PJ ¤ © Ä "$# ¤ £&%(' " ')for 0 �x2 � 0 + � ¡

. The two lines in Figure16 show theupperboundon theprobabilitythata connectionreceivesmorethanC timestheexpectednumberof markedpack-ets,for * t uRÑ,® , andfor �Æt 0 :: ; thex-axisshows � ¡

.

Throughput (%)+

Pro

bability,

0.0 0.05- 0.10. 0.15. 0.20/ 0.25/ 0.3000.0

0.1

01

0.2

02

0.3

03C=2

C=4

Figure16: Upperboundonprobabilitythataconnection’sfraction of markedpacketsis morethanC timesthe ex-pectednumber, given 0 :: totalmarkedpackets.

17

The RED gateway could easily keepa list of the �mostrecently-markedpackets. If someconnectionhasalarge fraction of the marked packets,it is likely that theconnectionalsohada largefractionof theaverageband-width. If someTCPconnectionis receiving a largefrac-tion of thebandwidth,thatconnectioncouldbea misbe-havinghostthatis notfollowingcurrentTCPprotocols,orsimply a connectionwith eithera shorterroundtriptimeor a larger window thanotheractive connections.In ei-thercase,if desired,theRED gateway couldbemodifiedto give lower priority to thoseconnectionsthat receive alarge fraction of the bandwidthduring timesof conges-tion.

11 Implementation

This sectionconsidersefficient implementationsof REDgateways.We show that theRED gateway algorithmcanbe implementedefficiently, with only a small numberofaddandshift instructionsfor eachpacketarrival. In addi-tion, theRED gatewayalgorithmis not tightly coupledtopacket forwardingandits computationsdonot have to bemadein the time-critical packet forwardingpath. Muchof the work of the RED gateway algorithm,suchasthecomputationof theaveragequeuesizeandof thepacket-marking probability � ! , could be performedin parallelwith packetforwarding,orcouldbecomputedby thegate-way asa lower-priority taskastime permits.This meansthattheREDgatewayalgorithmneednot impair thegate-way’sability to processpackets,andtheREDgatewayal-gorithmcanbeadaptedto increasingly-high-speedoutputlines.

If theREDgateway’smethodof markingpacketsis tosetacongestionindicationbit in thepacketheader, ratherthan droppingthe arriving packet, then settingthe con-gestionindicationbit itself addsoverheadto thegatewayalgorithm.However, becauseREDgatewaysaredesignedto mark asfew packetsaspossible,the overheadof set-ting the congestionindicationbit is kept to a minimum.This is unlike DECbit gateways,for example,which setthe congestionindicationbit in every packet that arrivesat the gateway whenthe averagequeuesizeexceedsthethreshold.

Foreverypacketarrivalatthegatewayqueue,theREDgateway calculatesthe averagequeuesize. This canbeimplementedasfollows:���� $ ����DC¼= ? & F 'K���#*As long as =@? is chosenas a (negative) power of two,this canbeimplementedwith oneshift andtwo additions(givenscaledversionsof theparameters)[14].

BecausetheREDgatewaycomputestheaveragequeuesizeat packet arrivals,ratherthanat fixedtime intervals,

thecalculationof theaveragequeuesizeis modifiedwhena packet arrivesat thegateway to anemptyqueue.Afterthe packet arrivesat the gateway to an emptyqueuethegateway calculates� , the numberof packetsthat mighthavebeentransmittedby thegatewayduringthetimethatthe line was free. The gateway calculatesthe averagequeuesizeasif � packetshadarrivedat thegatewaywithaqueuesizeof zero.Thecalculationis asfollows:� $ & 7I� ��J¬')F 7I� ��JL*,+54���� $ &B0 '>=@?4* M ���� Ñwhere F 7I� � J is the start of the queueidle time, and 4is a typical transmissiontime for a small packet. Thisentirecalculationis an approximation,as it is basedonthenumberof packetsthatmighthavearrivedat thegate-way during a certainperiodof time. After the idle time687:9<;>=@?BA 7:9C;D=FE

hasbeencomputedto a roughlevel ofaccuracy, a table lookup could be usedto get the term6&GH?JILKMEONQP(RQSUTWV K P(R$SUT:X8YOZ

, whichcoulditself beanapprox-imationby apowerof two.

Whena packetarrivesat thegatewayandtheaveragequeuesize []\_^ exceedsthethreshold

; []` P8a , thearrivingpacket is marked.Thereis no recalculationof thepacket-markingprobability. However, whena packet arrivesatthe gateway and the averagequeuesize []\_^ is betweenthe two thresholds

;>9<b P8a and; []` P8a , the initial packet-

markingprobability ced is calculatedasfollows:c dgfihUj []\k^ ? hmlfor h jLn ; []`po; []` P8a ?q;>9Cb P8asrhtl n ; []`po ;u9<b P8a; []` P8a ?q;>9Cb P8asvThe parameters

; []`po , ; []` P8a , and;>9Cb P8a arefixed pa-

rametersthat aredeterminedin advance.The valuesfor; []` P8a and;u9<b P8a aredeterminedby thedesiredbounds

on the averagequeuesize,andmight have limited flexi-bility. Thefixedparameter

; []` o , however, couldeasilybesetto a rangeof values.In particular,

; []` o couldbechosenso that h j is a power of two. Thus,the calcula-tion of ced canbeaccomplishedwith oneshift andoneaddinstruction.

In thealgorithmdescribedin Section4,when;>9<b P8axw[]\k^zy ; []` P8a a new pseudo-randomnumber { is com-

putedfor eacharriving packet,where{|n}{@[ b�~k��;�� � r G��is from the uniform distribution on [0,1]. Theserandomnumberscouldbegottenfrom a tableof randomnumbersstoredin memoryor couldbecomputedfairly efficientlyon a 32-bit computer[3]. In the algorithmdescribedinSection4, thearriving packet is markedif

{�yqcedO� 6&GU?q������b�7�� ced E v18

If c�d is approximatedby anegativepowerof two, thenthiscanbeefficiently computed.

It is possibleto implementthe RED gateway algo-rithm to usea new randomnumberonly oncefor everymarkedpacket,insteadof usinganew randomnumberforevery packet that arrivesat the gateway when

;>9<b P8aBw[]\_^�y ; []` P8a . As Section7 explains,whentheaveragequeuesizeis constantthenumberof packet arrivalsaftera markedpacket until thenext packet is markedis a uni-form randomvariablefrom � 1,2, ..., � G �Oc d<�_� . Thus,if theaveragequeuesizewasconstant,thenaftereachpacket ismarked thegateway couldsimply choosea valuefor theuniform randomvariable {�n�{�[ b�~]��;q� � r GM� , andmarkthe

b-th arriving packet if

b�� {���ced . Becausethe av-eragequeuesizechangesover time, we recompute{���cedeachtime that c�d is recomputed.If ced is approximatedbya negativepowerof two, thenthiscanbecomputedusinga shift instructioninsteadof adivide instruction.

Figure17 gives the pseudocodefor an efficient ver-sionof theRED gateway algorithm.This is just onesug-gestionfor anefficient versionof theRED gateway algo-rithm. Themostefficientwayto implementthisalgorithmdepends,of course,on thegateway in question.

Thememoryrequirementsof theRED gateway algo-rithm aremodest.Insteadof keepingstatefor eachactiveconnection,theRED gateway requiresa smallnumberoffixedandvariableparametersfor eachoutputline. This isnotaburdenongatewaymemory.

12 Further work and conclusions

RandomEarlyDetectiongatewaysareaneffectivemech-anismfor congestionavoidanceat the gateway, in coop-erationwith network transportprotocols. If RED gate-waysdrop packetswhenthe averagequeuesizeexceedsthemaximumthreshold,ratherthansimplysettingabit inpacketheaders,thenREDgatewayscontrolthecalculatedaveragequeuesize. This actionprovidesanupperboundon theaveragedelayat thegateway.

Theprobabilitythat theRED gateway choosesa par-ticular connectionto notify duringcongestionis roughlyproportionalto that connection’s shareof the bandwidthat thegateway. Thisapproachavoidsabiasagainstburstytraffic atthegateway. ForREDgateways,therateatwhichthe gateway markspacketsdependson the level of con-gestion,avoiding the global synchronizationthat resultsfrom many connectionsdecreasingtheir windows at thesametime. TheRED gateway is a relatively simplegate-way algorithmthatcouldbe implementedin currentnet-worksor in high-speednetworksof thefuture. TheREDgateway allows consciousdesigndecisionsto be madeabouttheaveragequeuesizeandthemaximumqueuesizeallowedat thegateway.

Initialization:[]\k^ f ��M���eb�7 f ?�Gfor each packet arrival:

calculate the new average queue size []\_^ :if the queue is nonempty[]\_^ f []\_^@� I K 68A�? [k\k^ Eelse using a table lookup:[]\_^ f 6�GU?qILK�EONQP(RQSUTWV K P(R$SUT:X8YOZ []\k^

if;>9Cb P8a�w []\k^�y ; []` P8aincrement

�M���eb�7ced f�h j � []\k^ ? h lif

�M���eb�7g ¡�and

�M���eb�7g�¡¢L£¤£¦¥¨§�©�� {ª��ced �mark the arriving packet�M���eb�7 f �

if�M���eb�7 n � (choosing random number){ f {�[ b�~]��;q� � r GM�

else if; []` P8a�w []\k^

mark the arriving packet�M���eb�7 f ?�Gelse

�M���eb�7 f ?�Gwhen queue becomes emptyA 7:9<;D= f 7:9<;>=Newvariables:{ : a random numberNewfixed parameters:4 : typical transmission time

Figure17: Efficientalgorithmfor REDgateways.

There are many areasfor further researchon REDgateways. The foremostopen questioninvolves deter-mining the optimumaveragequeuesize for maximizingthroughputandminimizingdelayfor variousnetworkcon-figurations.Thisquestionisheavily dependentof thechar-acterizationof thenetwork traffic aswell ason thephys-ical characteristicsof thenetwork. Somework hasbeendonein thisareafor othercongestionavoidancealgorithms[23], but therearestill many openquestions.

Oneareafor furtherresearchconcernstraffic dynam-ics with a mix of Drop Tail andRED gateways,aswouldresult from partial deploymentof RED gateways in thecurrentinternet. Anotherareafor further researchcon-cernsthe behavior of the RED gateway machinerywithtransportprotocolsother than TCP, including open- orclosed-looprate-basedprotocols.

As mentionedin Section10,thelist of packetsmarkedby theREDgatewaycouldbeusedby thegatewayto iden-tify connectionsthat arereceiving a large fractionof the

19

bandwidththroughthe gateway. The gateway could usethis informationto givesuchconnectionslowerpriority atthegateway. We leave thisasanareafor furtherresearch.

Wedonotspecifyin thispaperwhetherthequeuesizeshouldbemeasuredin bytesor in packets. For networkswith a rangeof packet sizesat thecongestedgateway thedifferencecanbesignificant.This includesnetworkswithtwo-waytraffic wherethequeueat thecongestedgatewaycontainslargeFTPpackets,smallTELNET packets,andsmall control packets. For a network wherethe time re-quiredto transmita packet is proportionalto the sizeofthe packet, andthe gateway queueis measuredin bytes,the queuesizereflectsthe delayin secondsfor a packetarriving at thegateway.

TheRED gateway is not constrainedto providestrictFIFOservice.For example,wehaveexperimentedwith aversionof REDgatewaysthatprovidespriority serviceforshortcontrolpackets,reducingproblemswith compressedACKs.

By controllingtheaveragequeuesizebefore thegate-wayqueueoverflows,REDgatewayscouldbeparticularlyusefulin networkswhereit is undesirableto droppacketsat the gateway. This would be the case,for example,inrunningTCPtransportprotocolsovercell-basednetworkssuchasATM. Thereareseriousperformancepenaltiesforcell-basednetworksif a largenumberof cellsaredroppedat thegateway; in this caseit is possiblethatmany of thecellssuccessfullytransmittedbelongto a packet in whichsomecell wasdroppedat a gateway [30]. By providingadvancewarningof incipient congestion,RED gatewayscanbeusefulin avoidingunnecessarypacketor cell dropsat thegateway.

Thesimulationsin thispaperusegatewayswherethereis oneoutputqueuefor eachoutputline, asin mostgate-ways in currentnetworks. RED gatewayscould alsobeusedin routerswith resourcemanagementwheredifferentclassesof traffic aretreateddifferentlyandeachclasshasits own queue[6]. For example,in a routerwhereinter-active(TELNET) traffic andbulk data(FTP)traffic areinseparateclasseswith separatequeues(in orderto givepri-ority to theinteractivetraffic), eachclasscouldhaveasep-arateRandomEarly Detectionqueue.The generalissueof resourcemanagementatgatewayswill beaddressedinfuturepapers.

13 Acknowledgements

We thankAllyn Romanow for discussionson RED gate-ways in ATM networks, andwe thankVern Paxsonandtherefereesfor helpful suggestions.This work couldnothavebeendonewithoutStevenMcCanne,whodevelopedandmaintainedoursimulator.

References

[1] Bacon,D,. Dupuy, A., Schwartz, J., andYemimi,Y., “Nest: a Network SimulationandPrototypingTool”, Proceedingsof Winter 1988USENIXCon-ference, 1988,pp.17-78.

[2] Bala, K., Cidon, I., and Sohraby, K., “Conges-tion Control for High SpeedPacket SwitchedNet-works”, INFOCOM’90, pp.520-526,1990.

[3] Carta,D., “Two FastImplementationsof the‘Mini-mal Standard’RandomNumberGenerator”,Com-municationsof theACM, V.33N.1, January1990,pp.87-88.

[4] Clark,D.D., Shenker, S.,andZhang,L., “Support-ing Real-Time Applicationsin an IntegratedSer-vices Packet Network: Architectureand Mecha-nism”, SIGCOMM’92, August1992,p. 14-26.

[5] Floyd, S., Connectionswith Multiple CongestedGateways in Packet-Switched Networks Part 1:One-wayTraffic, ComputerCommunicationRe-view, V.21N.5,October1991,pp.30-47.

[6] Floyd, S., “Issuesin Flexible ResourceManage-ment for DatagramNetworks”, Proceedingsofthe 3rd Workshopon Very High SpeedNetworks,March,1992.

[7] Floyd, S., andJacobson,V., On Traffic PhaseEf-fectsin Packet-SwitchedGateways, Internetwork-ing: ResearchandExperience,V.3 N.3,September1992,p.115-156.

[8] Floyd, S., and Jacobson,V., TheSynchronizationof Periodic RoutingMessages, to appearin SIG-COMM 93.

[9] Hansen,A Table of Seriesand Products, PrenticeHall, EnglewoodCliffs, NJ,1975.

[10] Harvey, A., Forecasting, structural time seriesmodelsandthe Kalmanfilter, CambridgeUniver-sity Press,1989.

[11] Hashem,E., “Analysisof randomdropfor gatewaycongestioncontrol”, ReportLCSTR-465, Labora-tory for ComputerScience,MIT, Cambridge,MA,1989,p.103.

[12] Hoeffding,W., ProbabilityInequalitiesfor SumsofBoundedRandomVariables, AmericanStatisticalAssociationJournal,Vol. 58, March 1963,p. 13-30.

[13] Hofri, M., ProbabilisticAnalysis of Algorithms,Springer-Verlag,1987.

20

[14] Jacobson,V., CongestionAvoidanceand Control,Proceedingsof SIGCOMM ’88, August1988,pp.314-329.

[15] Jain,R., “A Delay-BasedApproachfor CongestionAvoidancein InterconnectedHeterogeneousCom-puter Networks”, ComputerCommunicationRe-view, V.19N.5,October1989,pp.56-71.

[16] Jain, R., “CongestionControl in ComputerNet-works: Issuesand Trends”, IEEE Network, May1990,pp.24-30.

[17] Jain, R., “Myths About CongestionManagementin High-SpeedNetworks”, Internetworking: Re-searchandExperience,V.3 N.3, September1992,pp.101-114.

[18] Jain, R., and Ramakrishnan,K.K., CongestionAvoidance in Computer Networks with a Con-nectionlessNetwork Layer: Concepts, Goals,and Methodology, Proc.IEEE Comp.NetworkingSymp.,Washington,D.C.,April 1988,pp.134-143.

[19] Keshav, S., “REAL: a Network Simulator”, Re-port 88/472, ComputerScienceDepartment,Uni-versityof CaliforniaatBerkeley, Berkeley, Califor-nia,1988.

[20] Keshav, S., “A Control-TheoreticApproach toFlow Control”, Proceedingsof SIGCOMM ’91,September1991,p.3-16.

[21] Mankin, A. and Ramakrishnan,K. K., editorsfor theIETF PerformanceandCongestionControlWorking Group,“Gateway congestioncontrolsur-vey”, RFC1254,August1991,p.21.

[22] Mishra,P., andKanakia,H,. “A Hopby Hop Rate-basedCongestionControl Scheme”,Proceedingsof SIGCOMM’92, August1992,p.112-123.

[23] Mitra, D. and Seery, J., DynamicAdaptiveWin-dows for High SpeedData Networks: Theoryand Simulations, Proceedingsof SIGCOMM ’90,September1990,p.30-40.

[24] Mitra, D. and Seery, J., DynamicAdaptiveWin-dowsfor High SpeedDataNetworkswith MultiplePathsandPropagationDelays, Proc.IEEE INFO-COM ’91, pp.2B.1.1-2B.1.10.

[25] Pingali,S.,Tipper, D., andHammond,J.,“The Per-formanceof Adaptive Window Flow Controlsin aDynamicLoadEnvironment”,Proc.of IEEE Info-com’90, June1990,pp.55-62.

[26] Postel, J., “Internet Control MessageProtocol”,RFC792,September1981.

[27] Prue,W., andPostel,J., “Somethinga HostCouldDo with SourceQuench”,RFC1016,July1987.

[28] Ramakrishnan,K.K., Chiu,D., andJain,R., “Con-gestionAvoidancein ComputerNetworks with aConnectionlessNetwork Layer, Part IV: A Selec-tive Binary FeedbackSchemefor GeneralTopolo-gies”,DEC-TR-510,November, 1987.

[29] Ramakrishnan,K.K., and Jain, Raj, “A BinaryFeedbackSchemefor CongestionAvoidanceinComputerNetworks”, ACM TransactionsonCom-puterSystems,V. 8, N. 2, pp.158-181,1990.

[30] Romanow, A., “Some PerformanceResults forTCPoverATM with Congestion”,to appearin Sec-ond IEEE Workshopon Architectureand Imple-mentationof High PerformanceCommunicationsSubsystems(HPCS93), Williamsburg, VA, Sept.1-3,1993.

[31] Sanghi,D., andAgrawala,A., “DTP: An EfficientTransportProtocol”, University of Maryland techreportUMIACS-TR-91-133,October1991.

[32] Shenker, S., “Commentson theIETF performanceand congestioncontrol working group draft ongatewaycongestioncontrolpolicies”,unpublished,1989.

[33] Wang,Z., andCrowcroft, J., “A New CongestionControl Scheme:Slow StartandSearch(Tri-S)”,ComputerCommunicationReview, V.21 N.1, Jan-uary1991,pp.32-43.

[34] Wang,Z., andCrowcroft, J.,“Eliminating PeriodicPacketLossesin the4.3-TahoeBSD TCPConges-tion Control Algorithm”, ComputerCommunica-tion Review, V.22N.2,April 1992,pp.9-16.

[35] Young,P., Recursive Estimationand Time-SeriesAnalysis,Springer-Verlag,1984,pp.60-65.

[36] Zhang,L., “A New Architecturefor PacketSwitch-ing Network Protocols”,MIT/LCS/TR-455,Labo-ratory for ComputerScience,MassachusettsInsti-tuteof Technology, August1989.

[37] Zhang,L., andClark,D., “Oscillating Behavior ofNetwork Traffic: A CaseStudy Simulation”, In-ternetworking: Researchand Experience,Vol. 1,1990,pp.101-112.

[38] Zhang,L., Shenker, S., andClark, D., “Observa-tionsontheDynamicsof aCongestionControlAl-gorithm: The Effectsof Two-Way Traffic”, SIG-COMM ’91, September1991,pp.133-148.

21

A Appendix

In thissectionwegivethestatisticalresultusedin Section10on identifyingmisbehaving users.

Let «­¬ , G w�®�w b , beindependentrandomvariables,let ¯ betheir sum,andlet °«±n²¯³� b .

Theorem1 (Hoeffding, 1963) [12, p.15][13, p.104]: Let« j , « l ,..., «�´ be independent,and let� w « ¬ w G

forall « ¬ . Thenfor

� w 7 w GU?�µ�� °« � ,¶@· �5¸F� °« �Bµ�� °« � � 7<� (4)

w ¹»º½¼¼ � 7F¾À¿FÁP º GU? ¼GU? ¼ ?q7F¾ j V ¿ V�PC ´

w = V l ´ P(Ã vÄLet « R8Å ¬ beanindicatorrandomvariablethatis 1 if the® th markedpacket is from connection

9, and0 otherwise.

Then ¯ R(Å ´�n ´Æ¬¨ÇÈj « R8Å ¬ vFromTheorem1,¶@· �5¸F6 ¯ R(Å ´ � c R b � 7�b³E w = V l ´ P(Ãfor

� w 7 w GU? c R . Thus¶@· �5¸F6 ¯ R8Å ´ �B� c R b³E w = V l ´ N$ɨV j X(à o ÃÊfor

G w � w G �Oc R .

22