©2008 Keynote Systems, Inc. Keynote & Client Confidential.

April 21, 2009

How to Ensure High-Performing, Highly

Available Cloud Applications

Ben RushloDirector, Web Performance Consulting, Keynote Systems

April 21, 2009 2©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Cloud Computing

“A form of standardized IT-based capability – such as Intranet-based services, software, IT infrastructure –

offered by a service provider that is accessible via Internet protocols from any computer,

is always available and scales automatically to adjust to demand,

is either per-per-use or advertising-based, has Web- or programmatic-based control interfaces,

and enables full customer self-service.”

Frank Gillett, Forrester Research

April 21, 2009 3©2009 Keynote Systems, Inc. Keynote & Client Confidential.

2008 – Year of Web Glitches

Feb 15

Jun 17

April, May, June. July

Jun 17

Jun 17

Jul 8

Jul 10

Jul 27

Aug 12, Oct 17

Dec 1

Nov 28

April 21, 2009 4©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Cloud Computing’s Reliability & Availability Gap

Enterprises can’t completely rely on cloud infrastructures to run their business

Almost no SLAs provided by cloud providers

Using the Cloud lessens control and increases complexity of managing technology

Who provides you “Cloud Insurance”?

April 21, 2009 5©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Ensure Reliability and Availability in ALL Layers

©2008 Keynote Systems, Inc. Keynote & Client Confidential.

April 21, 2009

Cloud Computing Is Changing Performance Management

April 21, 2009 7©2009 Keynote Systems, Inc. Keynote & Client Confidential.

The “Cloud” – Web services, distributed data centers/infrastructure, content delivery network etc

The Application Lives Everywhere

April 21, 2009 8©2009 Keynote Systems, Inc. Keynote & Client Confidential.

The Browser as a “Platform”

April 21, 2009 9©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Complexity is Increasing

April 21, 2009 10©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Complexity is Increasing

April 21, 2009 11©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Complexity is Increasing

Domains1 -2 5-10

Third Party Content1-2 requests per page 10 30 requests per page

Page Sizes100-300K 300K – 2MB

LanguagesHTML, limited JS and CSS Multiple JS and CSS per page (XML, Flash, Flex etc)

HostingSingle location Hybrid (Cloud)

April 21, 2009 12©2009 Keynote Systems, Inc. Keynote & Client Confidential.

The Application Life Cycle On “Red Bull”

ITOperations

QualityAnalysis

WebDevelopment

April 21, 2009 13©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Increased User Expectations

The Internet has become a utilityAlways on, always responsive

The competition is raising the barEven if it is not your direct competition8s 4s 2s ?

The Cloud model changes expectationsUsers are not simply “viewing content”, instead they are interacting, using applicationsThe user doesn’t care that it is hosted in the cloud and will not lower expecations1s or less might become the new rule

©2008 Keynote Systems, Inc. Keynote & Client Confidential.

April 21, 2009

Performance Management Best Practices – More Important Now Than Ever

April 21, 2009 15©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Performance Management Best Practices

Test at every stage in Application Lifecycle –build with the user in mindTest from the Cloud –where your users areTest with a Browser –what your users useCapture the Right Metrics – what impacts your user

April 21, 2009 16©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Average download times ScalabilityPer element performanceClient side processingVariability – over geographyWeb service performancePre render vs. render timeNetwork vs. Application vs. ContentPage Construction

Key Metrics – Pre Launch

ITOperations

QualityAnalysis

WebDevelopment

Test out the cloud before your users do

April 21, 2009 17©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Key Metrics – In Production

Performance-average versus target

Availability – errors and outagesVariability – throughout day and overallThird-Party QualityTrends and Events

ITOperations

QualityAnalysis

WebDevelopment

Don’t let your users monitor the cloud for you

April 21, 2009 18©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Total Site Quality Framework

Tip: Avoid Pet Metrics

©2008 Keynote Systems, Inc. Keynote & Client Confidential.

April 21, 2009

Key Performance Challenges

April 21, 2009 20©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Asset Count More Important Than Size

Size matters less and lessAsset count primary driver for high speed user’s performance experience

Roundtrips! Roundtrips! Roundtrips!

Most sites have highest asset counts in three areasExternal CSSExternal JSStatic/designer images

Combination of these files is keySprites, packing JS etc.

April 21, 2009 21©2009 Keynote Systems, Inc. Keynote & Client Confidential.

JS and CSS Strategy

External JS and CSS was designed to allow caching across the site

However, many sites are loading unique JS and CSS per pageSome of this related to “silos” within organizations

Developers need to think about the “user flow” not per pageCoordination is keySimilar functionality should be in “Global” JS and CSS and loaded on key entry points in site

April 21, 2009 22©2009 Keynote Systems, Inc. Keynote & Client Confidential.

File Size

Size does matter to poorer connected usersThe “grey area” users – wireless, mobile devices, dial-up, satellite, coffee shops, hotels etc.

JS and CSSNeed to be compressedNeed to be minified (comments removed, whitespace removed)Old code pruned

April 21, 2009 23©2009 Keynote Systems, Inc. Keynote & Client Confidential.

0

1

2

3

4

5

6

7

8

9

UPS

Li

veTr

avel

ocity

Wik

iped

iaSp

rint

Hot

Jobs

Car

eer B

uild

erD

isne

yFi

delity

Yello

w P

ages

Goo

gle

AT&T

Orb

itz

Mer

rill L

ynch

MSN

eBay Ask

CN

NEx

pedi

a

AOL

Bank

Of A

mer

ica

Sym

antic

Face

book

Tick

etm

aste

rN

Y Ti

mes

Appl

e H

ewle

tt-Pa

ckar

dAm

azon

C

BS S

ports

line

Veriz

onYa

hoo

U

SA T

oday Del

lW

alm

art

Pric

elin

e.co

mM

SNBC

Wea

ther

.com

Cha

rles

Schw

abFe

dEx

Mon

ster

Dow

nloa

d Ti

me

Time On Netw ork Client Side Processing

Client Side Processing

40 Typical Home Pages

April 21, 2009 24©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Flash Initialization

April 21, 2009 25©2009 Keynote Systems, Inc. Keynote & Client Confidential.

JS and CSS Rendering

April 21, 2009 26©2009 Keynote Systems, Inc. Keynote & Client Confidential.

JavaScript and The Browser

External JS is not only a roundtrip but is also serialized the browser

Each file much be loaded in serialExponentially extends the downloadWorse in the top of the load (pre render)

Pre render JS files should be limited (ie. few of them)

April 21, 2009 27©2009 Keynote Systems, Inc. Keynote & Client Confidential.

JS Threading Example

None of these images were downloaded to the browser until 2.4 seconds into a 2.8 second page load

Javascript files load one file at a time

April 21, 2009 28©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Geographic Distribution of Content

Performance varies significantly based on geographyKey factors that drive amount of variation are

Element counts (Roundtrips!)User distance from data centerUse of Content Delivery Network

April 21, 2009 29©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Online Tax Application- Canada

0.0

5.0

10.0

15.0

20.0

25.0

Seco

nds

Vancouver Telus Calgary Telus Toronto Bell Montreal Verizon

Vancouver Telus 4.61 1.18 1.37 6.70 0.51

Calgary Telus 6.78 1.64 1.58 8.49 0.57

Toronto Bell 9.27 2.71 2.71 17.90 0.97

Montreal Verizon 11.68 3.48 3.14 21.46 1.09

Home Page Online Online Edition Get Started Validation

April 21, 2009 30©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Third Party Usage and Quality

It is quite common to find pages that have 10+ domains that are third party

Floodlight tags (which dynamically insert new domains)Tracking (Omniture, Google Analytics etc)Media buy/tracking tagsA/B testingSurvey/opinion tags Etc

Third party quality needs to be monitoredThird party tags must be POST render where possible

April 21, 2009 31©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Third Party Call Blocking Visual Content

April 21, 2009 32©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Cache Management

For many sites, visitors return quite frequentlySites need to take into account the impact of caching

Cache needs to be done appropriatelyFar future expire headersEtag configuration

The performance impact of “validating” cache with the server is AS SIGNIFICANT as downloading that content fresh (for high speed user)

April 21, 2009 33©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Automotive Site Example – Return Versus New

0.00

0.50

1.00

1.50

2.00

2.50

3.00

Home Page - New Visitor Home Page - Return Visitor

Seco

nds

Geometric Mean

400K

40K

April 21, 2009 34©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Don’t Forget The Basics

Cloud vendors must provide performance metrics

Measure your cloud application yourself as well

Ensure SLAs in place

Measure…Measure…Measure

April 21, 2009 35©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Keynote’s Cloud: More than 240 locations

April 21, 2009 36©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Test your apps for FREE from Keynote’s Cloud

kite.keynote.com

Download KITE 2.0@

April 21, 2009 37©2009 Keynote Systems, Inc. Keynote & Client Confidential.

Ben Rushlo

How to reach me

(623) 547-7068ben.rushlo@keynote.com

http://www.linkedin.com/in/benrushlo