rum for breakfast
DESCRIPTION
Real user monitoring is one of the best ways of learning “the truth” about what visitors experience on your web site, but it comes at a cost. The real world is messy and noisy making it hard to know exactly what’s going on. Filtering your data, splitting it along multiple dimensions, and determining what to discard are important second steps on the path to insightful RUM analysis, and in this session, we’ll go into some of the details.TRANSCRIPT
![Page 2: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/2.jpg)
RUM for Breakfast
Buddy Brewer, Carlos Bueno, Philip Tellis
Velocity 2012 / 2012-06-26
Velocity 2012 / 2012-06-26 RUM for Breakfast 2
![Page 5: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/5.jpg)
https://github.com/lognormal/boomerang/
Velocity 2012 / 2012-06-26 RUM for Breakfast 5
![Page 7: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/7.jpg)
Buddy built the navtiming plugin
Velocity 2012 / 2012-06-26 RUM for Breakfast 7
![Page 8: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/8.jpg)
tl;dr
1 Measure a bunch of stuff in the browser2 Use high school stats that we vaguely remember3 Randomly invent insights
Velocity 2012 / 2012-06-26 RUM for Breakfast 8
![Page 10: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/10.jpg)
![Page 11: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/11.jpg)
![Page 12: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/12.jpg)
![Page 13: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/13.jpg)
![Page 14: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/14.jpg)
![Page 15: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/15.jpg)
![Page 16: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/16.jpg)
![Page 17: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/17.jpg)
![Page 18: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/18.jpg)
![Page 19: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/19.jpg)
![Page 20: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/20.jpg)
![Page 21: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/21.jpg)
![Page 22: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/22.jpg)
![Page 23: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/23.jpg)
![Page 24: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/24.jpg)
![Page 25: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/25.jpg)
![Page 26: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/26.jpg)
![Page 27: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/27.jpg)
![Page 28: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/28.jpg)
![Page 29: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/29.jpg)
![Page 30: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/30.jpg)
![Page 31: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/31.jpg)
![Page 32: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/32.jpg)
![Page 33: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/33.jpg)
![Page 34: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/34.jpg)
![Page 35: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/35.jpg)
![Page 38: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/38.jpg)
Log-Normal Distribution
The logarithm of the x-axis follows a Normal distribution
Velocity 2012 / 2012-06-26 RUM for Breakfast 11
![Page 39: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/39.jpg)
Log-Normal Distribution
Use the Geometric Mean for pure Log-Normal distributions
Velocity 2012 / 2012-06-26 RUM for Breakfast 12
![Page 40: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/40.jpg)
Log-Normal Distribution
Performance data does not always follow a "pure" Log-Normal
distribution
Velocity 2012 / 2012-06-26 RUM for Breakfast 13
![Page 41: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/41.jpg)
Look at the entire spread
. . .
Velocity 2012 / 2012-06-26 RUM for Breakfast 14
![Page 42: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/42.jpg)
Look at the entire spread
which often approaches an infinite width
Velocity 2012 / 2012-06-26 RUM for Breakfast 14
![Page 44: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/44.jpg)
• 0.8% of hits are fake/abusive• 0.2-0.5% of hits are from a stale cache• 0.1% of hits are absurd• Timestamps in the future (or past depending on how you
interpret it)• Bots ignore robots.txt across domains• "Interesting" caches/copies
Velocity 2012 / 2012-06-26 RUM for Breakfast 16
![Page 45: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/45.jpg)
Even with beacons, you need to sanitize your input
Velocity 2012 / 2012-06-26 RUM for Breakfast 17
![Page 47: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/47.jpg)
Band-pass filtering
• Strip everything outside a reasonable range• Bandwidth range: 4kbps - 4Gbps• Page load time: 0ms - 600s
• You may need to relook at the ranges all the time
Velocity 2012 / 2012-06-26 RUM for Breakfast 18
![Page 49: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/49.jpg)
IQR filtering
Derive the range from the data
Velocity 2012 / 2012-06-26 RUM for Breakfast 19
![Page 52: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/52.jpg)
MoE & Sample size
There is an inverse square root correlation between sample size
and margin of error
Velocity 2012 / 2012-06-26 RUM for Breakfast 22
![Page 53: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/53.jpg)
How big a sample is representative?
Select n such that���1.96 σ√n
��� ≤ 5%µ
Velocity 2012 / 2012-06-26 RUM for Breakfast 23
![Page 54: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/54.jpg)
This needs to be at your lowest drilldown level
Velocity 2012 / 2012-06-26 RUM for Breakfast 24
![Page 56: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/56.jpg)
How does performance impact human behavior?
![Page 57: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/57.jpg)
8 million pages
1.5 million visits
50 different dimensions
![Page 58: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/58.jpg)
0%
17.50%
35.00%
52.50%
70.00%
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
very fast sessions had high bounce rates
![Page 59: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/59.jpg)
0%
17.50%
35.00%
52.50%
70.00%
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
bounce rate vs. load time
![Page 60: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/60.jpg)
0%
17.50%
35.00%
52.50%
70.00%
1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 7 7.5 8 8.5 9 9.5 10 10.5 11 11.5 12 12.5
bounce rate vs. DOM interactive
![Page 61: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/61.jpg)
0%
20.00%
40.00%
60.00%
80.00%
0.5 2 3.5 5 6.5 8 9.5 11 12.5 14 15.5 17 18.5 20 21.5 23 24.5 26 27.5 29
bounce rate vs. front end time
![Page 62: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/62.jpg)
http://www.flickr.com/photos/21560098@N06/3796822070
is my web site performance toxic to my users?
![Page 63: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/63.jpg)
http://www.flickr.com/photos/thecosmopolitan/6117530924
LD50 - when do half the users bounce?
![Page 64: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/64.jpg)
Back end time
DOM Loading
DOM Interactive
Front end time
DOM Complete
Load event
1.7 sec
1.8 sec
2.75 sec
3.5 sec
4.75 sec
5.5 sec
Bounce rate >=50%
![Page 65: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/65.jpg)
What is the LD50 for your site?
Other bounce rates? 40%? 30%?
Other variables? (critical content visible, etc)
Other behaviors? Conversions, revenue, pages per session, actions, when do people make tea?
Future directions
![Page 67: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/67.jpg)
Questions?
Buddy Brewer @bbrewerPhilip Tellis @bluesmoon
Carlos Bueno @archivd
![Page 68: Rum for Breakfast](https://reader034.vdocuments.site/reader034/viewer/2022052307/5559ada2d8b42aa4288b5184/html5/thumbnails/68.jpg)
Photo credits
• Rum on Ice – wiserbailey on flickr
• Laptop-top Cat – wabisabi2015 on flickr
• About NCSA Mosaic – ncsa
• Distilled – Lost Albatross on flickr
• Anthon Berg Chocolates – ulterior epicure on flickr
• KilroySchematic – on WikiPedia
• Boxplot vs PDF – on WikiPedia
Velocity 2012 / 2012-06-26 RUM for Breakfast 27