Performance Engineering Methodology

Chapter 4

Performance Engineering

• Performance engineering analyzes the expected performance characteristics of a system during the different phases of its lifecycle.

• Performance engineering – 1) develops practical strategies that help predict

the level of performance a system can achieve and – 2) provides recommendations to realize the

optimal performance level.

Typical PE Questions

• Can the insurance claim system meet its performance requirements of sub-second response time when a natural disaster occurs (e.g., a hurricane). Response Time

• Is the infrastructure of a government agency scalable and can it cope with the computing demands of the new required online security mechanisms? Scalability

• Is the reservation system for cruise lines able to respond to anticipated peak of customer inquiries after a TV ad campaign? Reliability

PE Larger Questions

• How can one plan, design, develop, deploy, and operate IT services that meet ever increasing demands for performance, availability, reliability, and security?

• Is a given IT system properly designed and sized for a given load condition?

PE Activities

• Understand the key factors that affect a system’s performance.

• Measure the system and understand its workload.• Develop and validate a workload model that captures

the key characteristics of the actual workload.• Develop and validate an analytic model that

accurately predicts the system’s performance.• Use the models to predict and optimize the system’s

performance.

Modeling Process

Motivating Example: a Call Center

Call Center• Goals:– Foster better relationships with customers, creating customer

loyalty and ensuring quality service.– Improve efficiency and service performance.– Identify and explore new sales opportunities.

• Main Functions:– Order status inquiry– Shipment tracking– Problem resolution status inquiry

• Requirements: sub-second response time and 24x7 operation.

QoS Questions• Is the system design able to meet the subsecond response time for

all functions? Response Time• What will be the impact of doubling the number of system

representatives in the next year? Scalability• Can acceptable performance levels be maintained after integrating

the system with the mainframe-based inventory application? Scalability

• Is the system capacity adequate to handle up to 1,000 calls in the busiest hour and yet preserve the subsecond response time goal?

• How do failures in the database server affect the 24x7 availability goal?

• What is the impact of starting to offer Web-based self-service to customers?

At the Requirements AnalysisPhase

• Workload definition:– Call center’s view: Arrival rate of phone calls– IT system’s view: Functions received from the representatives.– DB server view: SQL requests from the application server.– LAN view: packet size distribution and interpacket arrival time.

At the System Design Phase

• What should the system throughput be to meet sub-second response times?– 200 customer service representatives and 80% are

working during the peak hour.– Average think time of 30 sec.

Model of the call center system

Call Center Model

• Using the interactive response time law:

• Z: average think time, 30sec• N: number of active

representatives in the system, 200X80% = 160

• X0: system throughput• R: average response time < 1sec

At the System Development Phase

• What should be the capacity of the DB server so that the performance goals are met?– Each submitted functions requires 2.2 SQL calls on

average.– From the Forced Flow Law:

At the Operation Phase

• Assume DB server is a problem. Response times exceed sub-second goal.

• Measurements during peak hour:– 57600 queries/hour– Each query needs 50 msec of CPU, performs 4 I/Os on disk 1

and 2 I/Os on disk 2. Each I/O takes 8 msec on average.– X0 = 57600 / 3600 = 16 queries/sec– Service demands:

• Dcpu = 0.05 sec; • Ddisk1 = 4 x 0.008 = 0.032 sec; • Ddisk2 = 2 x 0.008 = 0.016 sec.

At the Operation Phase (cont’d)

At the Operation Phase (cont’d)

• The residence times at the CPU and disks for open QN model

• Response time of the DB server:• RDB =

At the Evolution Phase

• The company is considering to develop Web applications to allow customers access to the information they need without assistance from a customer representative. Web self-services reduce transaction costs and enhance the customer experience.

• Security requirements mandate that new applications be developed for Web access (authentication, auditing, DB access control mechanisms).

At the Evolution Phase

• Local queries and web queries:

Results for Evolution Scenario

Open Multiclass Queuing Networks - Utilizations

This wokbook comes with the books "Performance by Design," "Capacity Planning for Web Services," and "Scaling for E-Business"

by D. A. Menascé and V. A. F. Almeida, Prentice Hall, 2004, 2002 and 2000.

Classes ®

Queues ¯ 1 2Total

cpu1 0.80000 0.15000 0.95000

d12 0.51200 0.20000 0.71200

d23 0.25600 0.10000 0.35600

iii XSU

Results for Evolution Scenario

Open Multiclass Queuing Networks - Queue Lengths

This wokbook comes with the books "Performance by Design," "Capacity Planning for Web Services," and "Scaling for E-Business"

by D. A. Menascé and V. A. F. Almeida, Prentice Hall, 2004, 2002 and 2000.

Classes ®

Queues ¯ 1 2Total

cpu1 16.00000 3.00000 19.00000

d12 1.77778 0.69444 2.47222

d23 0.39752 0.15528 0.55280

Performance Engineering Methodogy