measuring the performance experienced by real users

• Philip Tellis• @bluesmoon• philip@bluesmoon.info• SOASTA• boomerang

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 1

Measuring the Performance Experienced byReal Users

Philip Tellis / philip@bluesmoon.info

LSPE Meetup / 2013-01-17

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 2

Real Users

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 3

Real Users

• Get delighted• Get frustrated• Have credit cards• ... that they are attached to• Form an impression from experience

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 3

Real Users

• Get delighted• Get frustrated• Have credit cards• ... that they are attached to• Form an impression from experience

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 3

Real Users

• Get delighted• Get frustrated• Have credit cards• ... that they are attached to• Form an impression from experience

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 3

Real Users

• Get delighted• Get frustrated• Have credit cards• ... that they are attached to• Form an impression from experience

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 3

Real Users

• Get delighted• Get frustrated• Have credit cards• ... that they are attached to• Form an impression from experience

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 3

What can we measure?

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 4

Anything that the user does!

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 5

Anything that the user does!(in the browser)

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 5

But we really care about performance here. . .

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 6

So let’s talk about How

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 7

Modern Browsers – no problem

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 8

Modern Browsers – no problem

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 9

Real Users don’t always use the latest browsers.

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 10

What about older browsers?

1 Get a timestamp before they leave a page2 Save it in a cookie3 Compare with timestamp when the page loads

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 11

What about older browsers?

1 Get a timestamp before they leave a page2 Save it in a cookie3 Compare with timestamp when the page loads

Then there’s some magic to handle edge-cases

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 11

The magic. . . ...

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 12

But it doesn’t need to be. . .

http://lognormal.github.com/boomerang/doc/

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 13

boomerang

• BSD licensed• Measure load time, bandwidth, latency• DNS lookup time, NavTiming, Connection Type• Page Parameters• And anything else you can write a plugin for

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 14

And it beacons all of this back to a predefined URL

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 15

So what then?

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 16

Sparse Distribution

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 17

Log-normal distribution

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 18

Bimodal distribution

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 19

Distributions

• Sparse distribution suggests that you don’t have enoughdata points

• Log-normal distribution is typical• Bi-modal distribution suggests two (or more) distributions

combined

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 20

Distributions are interesting, but how does it affectengagement?

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 21

So we compared load time to bounce rate

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 22

And we borrowed a biological term called LD50

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 23

LD50

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 24

This helps you fix only the stuff that’s hurting you.

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 25

Did you know that IE7 seems to be faster than IE8?

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 26

But that’s mostly IE9 users browsing in IE7compatibility mode.

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 27

Did you know that users bounce if a page is toofast?

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 28

But that’s mostly because error pages are the fastest.

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 29

Did you know that user patience wrt performance isa cultural thing?

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 30

It’s tied both to where a user is from, and also thetype of service.

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 31

Demo?

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 32

Summary

1 Measure the load time experienced by real users2 Correlate it with other things that user did3 Profit

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 33

Summary

1 Measure the load time experienced by real users2 Correlate it with other things that user did3 Profit

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 33

Summary

1 Measure the load time experienced by real users2 Correlate it with other things that user did3 Profit

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 33

Thank You!Questions?

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 34

• Philip Tellis• @bluesmoon• philip@bluesmoon.info• www.SOASTA.com• boomerang• LogNormal Blog

LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 35