Web Stress Testing

Matt OdhnerProgram ManagerApplication Center

Microsoft

Topics to be Covered

� Important web hosting statistics� What stress testing solves & common stress

testing issues� Web Application Stress tool demonstration� Understanding the report data� What a commercial stress tool needs� Application Center Test (ACT) demonstration� Visual Studio integration� The importance of bandwidth testing� Other ACT features� Capacity planning

� Popular web sites receive over 40,000,000 requests per day

� Web hosting is expected to be a $14.6 billion industry by 2003

� Newport Group study revealed 52% of web applications failed to scale, 60% had no performance testing, only 25% used a testing tool

� Zona Research found that $4 billion a year is lost due to slow and failed web pages

Web Hosting Statistics

� Stability issues - unexpected downtime and poorly written COM objects

� Performance problems – locate bottlenecks and whether the application will handle peak loads

� Capacity planning - how many machines are needed to support usage

� Plus, it avoids financial losses and ensures customer satisfaction

Stress Testing Solves …

� Memory� Processor� Network� Hard disk� COM component

Stress Testing Locates Bottlenecks

� Invalid test platform� Invalid script� Thread safety issues with unstable server COM

components� Active Server Page script errors and GLOBAL.ASA

issues� Insufficient processor power and/or scripts are

too complex for the hardware

Common Stress Test Issues

Web serverWeb server

MemberMemberStressStressClientClient

ControllerControllerStressStressClientClient

MemberMemberStressStressClientClient

Traditional Test Bed

Approaches to Stress Testing

� Confirm that the application functions under load� Find the maximum requests per second my

application can handle� Determine the maximum number of concurrent

connections my application can handle� Test the application with x number of unique

users

Scripts view - create, edit, delete, and run stress scripts.

Creating the Script

Creating the Script (cont’d)

Record using your browser – this method is recommended because all page elements are captured.

Concurrent browser connections

Script Configuration - Settings

Users view - manage user names and passwords.

Script configuration - Users

Clients view - add and remove the client machines used to stress the web server(s)

Script Configuration - Clients

Script item details – modify query string, edit POST data, use SSL, query RDS.

Script Configuration - Script Item Details

Performance Monitor Data

Collect performance monitor counters from within the tool

Sub AddNewScript()

Set objWAS = CreateObject("WAS.EngControl.1")

Set objScripts = objWAS.Scripts

Set objScript = objScripts.Add

objScript.sName = "A New Script"

objScript.ScriptItems.sServer = "www.targetserver.com"

MsgBox "The new ScriptID is " & objScript.ScriptID

End Sub

Scriptable Object Model

=h= Selected new user: User2, password,ASPSESSIONIDGQQGQNYG=MPBBLMOAFBPBPDFAJLMCCFCO; Account=6001

=h= closed socket

=h= linger is on, linger timeout is 0

=h= connected to the server. 172.30.170.217:80

=h= sending request to server (470 bytes)

GET /fmstocks/_NewAccount.asp HTTP/1.1

Host: localhost

Session Trace OptionSession Trace Option

ASP Client

Reports view - analyze the results of a test.

Reports – Tabular Report Data

Request

Time To First Byte

Time To Last Byte

Response

0 ms

9 ms6 ms

Time to First and Last Byte

MHz Cost = N * S * avg (Pt) / avg(Rps)

Where:� N = Number of Processors� S = Speed of Processors� Pt = System: % Total Processor Time� Rps = ASP: Requests per secondReports view -

analyze the results of a test.

Calculating ASP Performance

Example: a test using a 2 processor web server achieved 350 requests per second, with the processors 70% utilized.

So, this works out to …

�2 processors * 400 MHz => 800 MHz�70% processor utilization => (800) * (0.70)

=> 560 MHz used�350 ASP requests per second 560/350

=> 1.6 million cycles per ASP request

Calculating ASP Performance

�Designed to meet internal test needs�http://webtool.rte.microsoft.com contains:

–Searchable knowledge base–Tutorial of main features–Latest download–Object model sample scripts

�Peer support on the discussion alias: webtool@microsoft.com

�Online help contains samples�The tool respects ROBOTS.TXT

More About Web Application Stress Tool

All of the features in Web application Stress, Plus:

�Accurate simulation of client connection speeds

�Running a script a specific number of iterations�Testing of servers in multiple domains�Handle complex applications that require

conditional responses�Capable of changing subsequent requests based on the response of a previous request�Functional and performance testing�In-depth reporting and troubleshooting

What is Needed in aCommercial Stress Test Tool?

MicrosoftApplication Center Test

1. Integration with Visual Studio2. IP router technology3. Data collector to replay actual web site traffic4. Completely scriptable, even as the test is running5. Graphical and tabular reporting6. Capacity planning7. Can import Web Application Stress scripts

Significant Features

1. Visual Studio Integration

Visual Studio Integration (cont’d)

Analog60%xDSL

13%

ISDN5%

Cable16%

Wireless6%

� 58% of web site users said download speed was a primary factor that influences them to return to a site (Forrester Research)

� Web surfers give a site about 10 to 15 seconds to display before they hit the stop button and go to a different site

� Bandwidth is not expected to improve soon

Chart shows forecastfor the year 2003

Bandwidth Statistics

Client computers

Web farm

T156 K

T328.8

2. IP Router Technology

ACTACT Local Area Local Area NetworkNetwork

Web Web clustercluster

Delay prior to sending each packet so thatDelay prior to sending each packet so that1,588 bits per second is maintained1,588 bits per second is maintained

IP Router Technology (cont'd)

IP Router Technology (cont'd)

Client computers

Internet traffic

Replays exact traffic

3. Data Collector

If IsObject(Session) = False Then

Set Session = CreateObject("ACT.Session")

End If

Set oRequest = Session.CreateRequest

oRequest.Path = "/samples/cookie.asp"

oRequest.Headers.Add "Connection", "Close"

oRequest.Headers.Add "Cookie", "FirstName=Matt&Lastname=Odhner"

Set oConnection = Session.CreateConnection("MATTOD15")

Set fs = CreateObject("Scripting.FileSystemObject")

Set writeToFile = fs.OpenTextFile("c:\test.txt", 8, TRUE)

thePath = oRequest.Verb & " " & oRequest.Path & " " & _

oConnection.Send(oRequest).ResultCode & chr(13) & chr(10)

writeToFile.Write thePath

writeToFile.Close

4. Completely Scriptable

Reporting - Tabular Reports

Reporting - Connections vs. TTFB & TTLB

Reporting - Comparing Scripts

6. Capacity Planning

1. Manual capacityplanning

Web cluster

Internet

Filter Filter Filter

Capacity Planning (cont'd)

2. Historical capacity planning

Client computers

Web farm

Capacity Planning (cont'd)

3. Calibrated capacity planning

Summary

Web Stress tools give answers to the problems of Web application stress, via:

� Modeling� Prediction� Identification of bottlenecked resources� Provision of performance Metrics to help

developers decide what content to deploy

So - employ one today to improve your web application's performance!

http://www.microsoft.com/TechNet/iis/sol.asp

http://msdn.microsoft.com/library/techart/windnamistakes.htm

http://msdn.microsoft.com/workshop/server/asp/server102599.asp

http://msdn.microsoft.com/workshop/server/asp/server092799.asp

http://msdn.microsoft.com/workshop/server/asp/server122799.asp

http://www.microsoft.com/siteserver/commerce/support/highcapacity.htm

Resources - Further Reading

Resources - Other Web Stress Tools Available

Mercury Interactive, Loadrunner/Winrunner -http://www.merc-int.com/products/

Radview, Webload - http://www.radview.com/webload/

RSW, eLoad - http://www.rswsoftware.com/

Segue, Silkperformer -http://www.segue.com/html/s_solutions/silk/s_performer.htm

Bluecurve, Dynameasure -http://www.bluecurve.com/products/products.htm

Rational, Rational Suite / PerformanceStudio -http://www.rational.com/products/rs/pstudio/index.jtmpl

Questions?

Matt OdhnerProgram ManagerApplication Center

Microsoft