web page response time 101

ever abandoned a Web site simply becauseit was too slow? I am sure you have. Theconsequences of this abandonment behav-ior can be very serious: In a recent studyby Zona Research, it was estimated thateach year Web sites lose several billions ofdollars due to what they call “unacceptable

download speed and resulting user bailout behavior.” Since the impact ofpoor performance can be so devastating, Web site quality assurance ef-forts have to give performance testing the same level of priority and at-tention they give to functionality testing.

But testing the performance of a Web site can be quite difficult, be-cause the main unit of measurement—page response time—is affected bya lot of different and interacting factors. In this article we’ll look at thebasic concepts and tools necessary for measuring and analyzing Webpage response time. As this article’s title suggests, this is an introductoryarticle on a very complex subject, and I’ve simplified some formulas andexplanations; but I’m quite sure that after reading it you’ll know moreabout Web page response time than most people in your organization,and you will never look at a Web page the same way again.

www.s tqemagaz ine .com STQE Ju ly/Augus t 200148

Measurement & AnalysisMeasurement & Analysis

Web PageResponseTime

Human beings don’t like to wait. We don’t like

waiting in line at a store, we don’t like waiting for our

food at a restaurant, and we definitely don’t like waiting

for Web pages to load. Have you

QUICK LOOK

■ 4 laws of Web site performance

■ Examining the factors behindresponse times

101Understanding and measuring

performance test results

by Alberto Savoia

This article is provided courtesy of STQE, the software testing and quality engineering magazine.

http://www.stqemagazine.com/

Ju ly/Augus t 2001 STQE www.s tqemagaz ine .com49

The Top Four Laws ofWeb Site PerformanceBefore we proceed, I believe it’s im-portant to understand the underlyingbehavioral laws that make Web pageperformance so critical.

1. The Law of StickinessThe first law is that Web site users aresticky, but they are not loyal. Thatmeans that if they find a Web site thatadequately serves their needs theytend to stick with it, but the momentthat site disappoints them (by beingslower than usual, or temporarily un-available), they will move to a com-petitive site and stick with it…untilthat new site gives them reason toswitch again. The implication of thislaw is obvious: do whatever you canto improve and maintain top perfor-mance, or users will abandon yourWeb site and start sticking around acompetitor’s site.

2. The Law of User PerspectiveThe law of user perspective states thatyou should measure your Web site’sperformance from the end-user pointof view. When you pull up your Website and test it from your intranet(running at 100Mbits/sec), perfor-mance might be fantastic—but mostof your users will have to experienceit through much smaller bandwidth.Under those conditions, that homepage animation that looked so cool onyour intranet will take forty secondsto load, and impatient users will notthink it’s very cool at all.

3. The Law ofResponsibility (a.k.a. It’sAlways Your Fault)The next law of Web site performanceis that users don’t really care who’sresponsible for a Web site’s poor re-sponse. If they experience poor per-formance on your Web site, you’re theone who’s going to get the blame—even though the root of the perfor-mance problem may be a temporaryglitch with their Internet ServiceProvider (ISP), or even with the Inter-net as a whole. The implication of thislaw is that you must be aware of allthe factors that can impact your per-formance and take steps to eliminate

or reduce them even if they seem tobe out of your hands.

4. The Law of ExpectationsThe fourth law states that users’ satis-faction with Web site performance isdictated by their expectations, and thattheir expectations are created by theirprevious experiences with other Websites. Let’s assume a user is accus-tomed to home pages that load in, say,six seconds. Given that expectation, ifyour Web site’s home page loads in tenseconds, that user will consider it slow,be dissatisfied, and will—probably—gosomewhere else. The implication of thislaw is that whenever you measure Website performance, you shouldn’t ana-lyze the results against some absolutevalue; instead, analyze them relative tothe performance numbers of other Websites (especially your competitors’).

Now that you understand why youshould care so much about Web perfor-mance, let’s see what the primary caus-es of poor performance are, and howyou can measure, analyze, and improvethem.

Basic Formula for Page Response TimeAt the highest level of abstraction, thetime it takes for a Web page to be com-pletely displayed on a user’s browsercan be roughly described by the equa-tion in Figure 1.

This formula makes several gener-alizations and assumptions, and its ac-curacy varies through the possiblerange of values (it tends to overesti-mate below eight seconds and underes-timate over eight seconds). For thepurposes of this article, however, it willdo just fine, since it introduces the keyvariables that impact page responsetime and shows you how they relate toeach other, without introducing exces-sive complexity. If you want to studythis subject in more detail, a good

starting point is Peter Sevcik and JohnBartlett's presentation, “Understand-ing Web Performance,” from which this basic formula is derived, at http://netforecast.com/Freports/NFR%205055%20Web%20Performance%20FNL.pdf.

Let’s look at each of these fivepage response time variables in moredetail.

Page SizePage size is measured in Kbytes, andon the surface, the impact of this vari-able is pretty obvious: the larger thepage, the longer it takes to download.When estimating page size, however,many people fail to consider all thecomponents that contribute to pagesize—all images, Java and other ap-plets, banners from third sources,etc.—so make sure you don’t overlookanything.

There are many ways to determineWeb page size. A very helpful onlinetool for this task can be found at: http://weboffice-old.web.cern.ch/WebOffice-Old/Tools/pagesize.html. Give this tool a URL

and it will return various statisticsabout the Web page associated with it,as seen in the sample output in Table 1(on the next page).

As you can see, this online toolprovides additional information beyondpage size; it tells you the size and num-ber of the images contained in thepage. As we’ll see a little later, this is es-sential information for estimating pagedownload time.

Minimum Bandwidthbetween Server and ClientMinimum bandwidth is defined as thebandwidth of the smallest pipe be-tween your content and the end user.Just as the strength of a chain is deter-mined by its weakest link, the effectivebandwidth between two end points isdetermined by the smallest bandwidthbetween them.

Page SizeMinimum Bandwidth

+RoundTrip × TurnsTime

+ServerProcessingTime

ClientProcessingTime

( )FIGURE 1 Formula for page response time as experienced by the user

=+PageResponseTime



Typically the most limited band-width is between the end user and itsISP. Since today a large percentage ofpeople are still connecting to the Inter-net using 56.6K (or even 28.8K)modems, a Web site design that doesn’ttake this into account is bound to lose alot of visitors. Keep in mind that mo-dem speeds are typically expressed inbauds, and that the user’s modem baudrating will seldom match the actualbits per seconds (bps) that their mo-dem will typically achieve (for a num-ber of reasons). Your best bet, then, isto convert baud rates to the Kbytes persecond that users are most likely toachieve in their environment. For prac-

tical purposes, assume that 56Kmodems will achieve an average trans-fer rate of 4Kbytes/sec and, knowingthat many users will access your Website at that speed (this is particularlytrue for consumer-oriented sites), usethat as a minimum bandwidth number.

Once you have the page size andthe minimum bandwidth, you can cal-culate the raw download time (seeFigure 2).

Using this formula, we see that itwill take approximately ten seconds forall the bytes in a 40Kbyte page to trav-el over a 56.6K-modem connection.

Unfortunately, the raw downloadtime is not what the end user will expe-

rience—because, as we shallsee, it takes several distinct cy-cles to send the entire contentof a page, and each cycle issubject to unavoidable Internetdelays. Nevertheless, the rawdownload time is a very usefulmetric because it sets a lowerlimit for page response time

based on an end-user connectionspeed. If you have a 120Kbyte page,56.6K modem users will seldom down-load it in less than thirty seconds, andmost 28.8K modem users will have towait at least a minute. Ouch!

Round-Trip TimeIn the context of Web page responsetime, round-trip time (RTT) indicatesthe latency, or time lag, between thesending of a request from the user’sbrowser to the Web server and the re-ceipt of the first few bytes of data fromthe Web server to the user’s computer.RTT is important because every re-quest/response pair (even for a triviallysmall file) has to pay this minimumperformance penalty. As we shall see inthe next section, the typical Web pagerequires several request/response cy-cles.

The simplest way to measure RTTis by using “ping,” a standard trou-bleshooting tool available on most net-work operating systems. On MicrosoftWindows, for example, you can accessping by opening a DOS command win-dow and simply typing ping followed byeither the hostname or the IP addressof the server. (Figure 3 shows what Iwould type to check my company’ssite.)

The first thing you notice is thatbefore it could ping the Web site, thehostname www.keynote.com had to beresolved into its IP address(206.79.179.108) by the DomainName Service (DNS). After resolvingthe IP address, ping sends it four smallpackets of thirty-two bytes each andmeasures the time it takes for the serv-er to respond. Finally, it provides you


S I Z E S I Z E T O TA L O F T H E S I Z E N U M B E R O F O F T H E S I Z E

S I Z E O F D I S P L AY E D O F T H E D I F F E R E N T B I G G E S T O F A L LT H E PA G E T E X T H T M L TA G S I M A G E S I M A G E I M A G E S

U R L ( B Y T E S ) ( B Y T E S ) ( B Y T E S ) C O N TA I N E D ( B Y T E S ) ( B Y T E S )

http://www. newyorktimes.com 57768 10461 47307 39 17978 115656

http://www. morocco-today.com 53748 17628 36120 12 43787 141898

http://www. kansascitystar.com 33645 7271 26374 30 17302 136180

http://www. milwaukeetimes.com 26546 3790 22756 13 40787 88458

http://www. bahraintribune.com/ 20996 7130 13866 17 21678 92599home.asp

TABLE 1 Web page sizes from five newspaper sites

Page Size in KbytesMinimum Bandwidth in Kbytes/sec

DTRAW

=

40 Kbytes4 Kbytes/sec

DTRAW

= = 10sec

FIGURE 2 Calculating raw download time


c> ping www.keynote.comPinging www.keynote.com [206.79.179.108] with 32 bytes of data

Reply from 206.79.179.108: bytes=32 time=103ms TTL=247Reply from 206.79.179.108: bytes=32 time=110ms TTL=247Reply from 206.79.179.108: bytes=32 time=105ms TTL=247Reply from 206.79.179.108: bytes=32 time=104ms TTL=247

Ping statistics for 206.79.179.108:Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),

Approximate round-trip time in milliseconds:Minimum = 103ms, Maximum = 110ms, Average = 105ms

FIGURE 3 Measuring round-trip time using “ping”



with some packet loss data and statis-tics on the round-trip time. In thiscase, the RTT delay seems pretty con-sistent for all measurements (with anaverage of 105ms, a minimum of103ms and a maximum of 110ms), butkeep in mind that this number canchange dramatically at different timesof day, and even within the same ping.A user’s geographical location, theirchoice of ISP, and the current trafficlevels on the Internet can greatly affectthe delay. (A great free tool for gettinga feeling of Internet latency betweendifferent backbones is the “InternetHealth Report,” which can be found atwww.internetpulse.com. I strongly en-courage you to check it out to get areal-time view of delays across theNet.)

TurnsA typical Web page consists of a basepage and several additional objectssuch as graphics or applets. As sug-gested in the previous section, theseobjects are not transmitted along withthe base page; instead, the base pageHTML contains instructions for locat-ing and fetching them. Unfortunatelyfor end-user performance, fetchingeach of these objects requires a fairnumber of additional communicationcycles between the user’s system andthe Web site server—each of which issubject to the RTT delay I just men-tioned.

The term turns defines the num-ber of TCP connections re-quired to completely down-load a page. For HTTP 1.0,the minimum number ofturns required to download atypical base page is four. Oneturn is needed for DNS look-up (to translate a URL into anIP address) and three turns torequest and receive the basepage. In addition to thesefour turns, each distinct ob-ject in a Web page requires aminimum of three turns(more if the object is handledby another server and re-quires an additional DNSlook-up, as is the case withmost banner ad servers).

Since each turn is sub-ject to the round-trip time de-lay, the very verbose HTTP1.0 protocol is not terribly ef-

ficient for the typical modern Webpage, which may have thirty, forty, ormore distinct objects in it. Fortunately,most browsers allow four simultaneousconnections so four turns can takeplace concurrently, which significantlyimproves things. The newest version ofthe HTTP protocol, 1.1, is designed tosignificantly reduce the number of TCPconnections required for a completepage download; but until 1.1 becomesmore widespread, most users and Websites will have to pay a relatively highperformance price for having a lot ofobjects in each page. (For resources onunderstanding the underlying mecha-nisms behind the HTTP protocol, seethe StickyNotes feature at the end ofthis article.)

How do you calculate the numberof objects in a Web page? The simplestway is to use the online page size tool Ipreviously introduced. Alternatively,you can study its HTML source andlook for IMG, APPLET, or OBJECTtags, which instruct the browser tofetch additional objects. Once youknow the number of objects, you cancalculate the number of turns using theformula in Figure 4 where NumObjectsis the number of objects in the page.The constant 4 takes into account thatit takes one turn to do DNS lookup andthree turns to download the base page.NumObjects is multiplied by 3 becauseit takes three turns to download eachobject, and divided by 4 because, asyou might recall, most browsers are set

up to download up to four objects con-currently.

Now that we have a basic under-standing of round-trip time and turns,and a way to measure them, we cancome up with a rough estimate of theportion of the overall Web page down-load time that is attributable to them,as in Figure 5 (in which RTT repre-sents the average round-trip time).

Figure 6 shows what the approxi-mate impact of turns and delay wouldbe on a page with forty objects and aping time of 100ms.

This number is nothing to sneezeat; it means that even if you have a bigpipe between the user and the serverand are able to achieve sub-second rawdownload times, the page will not com-plete downloading in less than 3.4 sec-onds due to the number of turns andthe delays associated with them.

Processing TimeThe last factor in the response timeformula is the processing time re-quired by the server and the client toput together the required page so itcan be viewed by the requester. Thiscan vary dramatically for differenttypes of Web pages.

On the server side, pages withstatic content require minimal pro-cessing time and will cause negligibleadditional delay. Dynamically createdpages (e.g., personalized home pageslike my.yahoo.com) require a bit moreserver effort and computing time, and

will introduce some delay. Fi-nally, pages that involve com-plex transactions (e.g., creditcard verification) may re-quire very significant pro-cessing time and might intro-duce delays of severalseconds.

On the client side, theprocessing time may be triv-ial (for a basic text-onlypage) to moderate (for apage with complex forms andtables) to extreme. If thepage contains a Java applet,for example, the client’sbrowser will have to load andrun the Java interpreter,which can take several sec-onds.

Most Web sites consistof many hardware and soft-ware components from dif-

DTTURNS

= RTT × (4 + )3 × NumObjects4

FIGURE 5 Calculating delay due to turns

3 × NumObjects4

Turns = 4 +

FIGURE 4 Calculating the number of turns

FIGURE 6 Sample calculation with 40 objects and 100 msping time

DTTURNS

= 100ms ×(4 + )3 × 404

= 3,400ms = 3.4sec



ferent vendors, each with a uniqueperformance profile. Because of thisvaried anatomy, there isn’t a simpleformula that you can use to calculateprocessing time. The most practicalway to understand and quantify thiscomponent is to actually measure it;and the simplest way to measure it isto bypass the Internet and to estab-lish a direct connection to the siteunder test (using a fast 100Mbits/secintranet, so bandwidth is maximizedand delays are kept to a minimum).Then, with an old-fashioned stop-watch, measure the time it takes tocompletely load a page. (On Mi-crosoft Explorer, you know a pagehas been completely loaded whenthe word “Done” appears on the bot-tom left corner of the window.)

Some pages will load so fast thatyou will barely have time to press thestart/stop button on the stopwatch.(At 100Mbits/sec, it only takes about1/100 of a second to transfer100Kbytes.) Others will take a def-inite and measurable amount of timeto complete loading. Since you areon a very fast connection with mini-mum delays and huge bandwidth,you can safely attribute any signifi-cant delays to server and client pro-cessing time. If you want to get reallysophisticated, subtract 0.1 to 0.2seconds from the measured time toaccount for delays in your reflexesand make sure you take at least fiveto ten measurements to get a statisti-cally significant sample.

Keep in mind, however, thatsince every time you load a Webpage the browser stores the variouspage components in its local cache,you must clear the cache betweenmeasurements. Otherwise, instead

of fetching these objects from theserver, you will be getting them fromyour local cache and bypassing thevery process you are interested intiming. (Clearing your local cache iseasy. On Microsoft Explorer, for ex-ample, follow the menu: Tools➞ Inter-net Options➞ General, and on the sec-tion “Temporary Internet Files”choose “Delete Files.”)

In addition to page type, anotherfactor that has a significant impact onprocessing time is server load. Theresponse time of a server is inverselyproportional to its load. As the num-ber of users increases, the speed withwhich the server can respond to eachpage request diminishes; first gradu-ally, then dramatically until the servercrashes. Understanding how userload impacts server performance isthe domain of load testing, an activitythat should be part of any Web siteperformance effort.

(Web site load testing is a broadand complex topic, but I’ve writtenarticles on the subject for the last twoissues of STQE magazine, which I be-lieve serve as a good introduction tothe subject. I encourage you to readthem and put them into practice, be-cause any page response time analy-sis that does not factor in server loadlevels will be of limited usefulness inpredicting end-user response time.)

If you are expecting your Website to experience an average trafficlevel of, say, 1,000 sessions/hr, and apeak level of 3,000 sessions/hr, youshould design a test that will driveyour system at several differentloads: the average load, the maxi-mum anticipated load, and one ortwo load levels in between. Mostload testing tools report the page re-

sponse time for different pages, butin order to compare apples to ap-ples, I still recommend taking manu-al measurements with your stop-watch to complement (and doublecheck) the data reported by the loadtesting tool. At the end, you shouldhave a table that looks somethinglike Table 2.

As you can see in this example,the processing time can vary dra-matically from page type to pagetype, and the degradation in perfor-mance varies as the load is in-creased.

Putting It All TogetherAs we have seen, the page responsetimes experienced by end users areaffected by six principal variables:page size, minimum bandwidth,number of turns, round-trip time,server processing time, and clientprocessing time. Let’s look again atthe basic formula correlating allthese variables, shown in Figure 1.

Now that we’ve talked about loadissues, note that this formula’s “PageResponse Time” is really a function ofserver load. Because the impact ofserver load on response time can beso severe, we want to make that con-nection very clear.

By studying this formula you caneasily spot several ways to minimizeyour page response times. Keepingthe total page size small, for exam-ple, can tilt the whole equation inyour favor. Since transfer rates be-tween low- and high-bandwidthclients can be dramatically different,small total page sizes will prevent theloss of low-bandwidth users. Remem-ber that the performance difference


1 , 0 0 0 2 , 0 0 0 3 , 0 0 0N O L O A D S E S S I O N S / H R S E S S I O N S / H R S E S S I O N S / H R

Home Page 0.1 0.3 0.9 2.4

Product Info 0.4 0.5 1.6 5.5

Account Status 3.4 5.6 8.4 16.1

Credit Card Verification 15.5 18.5 22.4 37.7

TABLE 2 Average page completion time (in seconds) at various load levels (measured on intranet at 100Mbits per second)




between a 40K and an 80K page isonly a fraction of a second for aDSL/T1 user, but it’s twenty secondsfor a 28K modem user.

Even if you manage to keep thesize of the page small (say, 40Kbytes),you can undo the benefit of all thatbyte trimming by populating the pagewith too many small objects. Thatkind of clutter means that even endusers with very high bandwidth con-nections will have to wait several sec-onds for all the objects to arrive.

Since the value of the round-trip time delay variable tends to in-crease as the distance between theserver and the client increases, youmight want to consider taking advan-tage of content distribution networks(CDNs) offered by companies suchas Akamai and Speedera. CDNs keepfrequently downloaded pages andpage objects (such as company lo-gos) on multiple servers located ondifferent network backbones. Whena client requests one of those ob-

jects, the request is served by theclosest server to minimize delay.

Even if all your pages are verysmall and contain few objects, yourresponse time will decrease dramati-cally if the server can’t handle theload. Your highest load times are yoursite’s best opportunities for visibilityand business activity, so don’t forgetto make sure that your system canhandle it without crashing and turn-ing that opportunity into a disaster—make sure that you do a load test be-fore a site launch, site redesign, ornew hardware installation.

A final suggestion: minimize theuse of applets and other technologiesthat require a fair amount of client-side processing, since this could havea severe performance impact onslower systems. Just because the de-lay is on the client side you shouldn’tfeel exonerated, because the Law ofResponsibility will come back to biteyou: poor performance is always go-ing to be perceived as your fault.

Happily, the reverse is often trueas well: a positive experience on yourWeb site is something that users willgive you full credit for—and armedwith these common-sense principles,you’re halfway there already. STQE

Alberto Savoia ([email protected]) is Chief Technologistof Keynote’s load testing divisionand has also served as founderand CTO of Velogic, General Man-ager of SunTest, and Director ofSoftware Research at Sun Mi-crosystems Laboratories. His six-teen-year career has been focusedon applying scientific methodologyand rigor to software testing, andhe holds several U.S. patents—in-cluding two on software test au-tomation.


STQE magazine is produced bySTQE Publishing, a division of SoftwareQuality Engineering.


web page response time 101

Documents