Web Performance Culture and Tools at Etsy

Download Web Performance Culture and Tools at Etsy

Post on 08-May-2015




1 download

Embed Size (px)


Presented at the NY Web Performance Meetup on Jan 18, 2012.


<ul><li>1.Web PerformanceCulture and Toolsat Etsy Mike Brittain @mikebrittain Director of Eng., Infrastructure Etsy</li></ul> <p>2. All of this started in the last year. 3. Make PerformanceMatter 4. $0.20 per item3.5% transaction fee 5. Focus on the marketplace 6. ecommerce 7. ecommercead supported 8. ecommercead supportedsubscription 9. ecommercead supportedsubscriptionfree 10. $$$ 11. Prove it. 12. Our own years of testinghave conclusively shown that when speed of a feature orproduct improves, usage, quite simply, goes up. http://googleblog.blogspot.com/2009/12/this-week-in-search-121809.html 13. 100+ web, api, jobs40+ databasesMany more doing lots of other stu... 14. $$$ 15. Performance is fun 16. I want you to tell me how you feel... 17. 95th percentilemilliseconds Average 18. ew r!en ste h a 95th percentilew fis go is ith e T dmillisecondsm a Average 19. I am better than you. 20. Make PerformanceMatter 21. PerformanceCulture 22. Our perf team is two people... 23. Measure 24. MeasureBuild tools 25. MeasureBuild toolsTrain &amp; Coordinate 26. MeasureBuild toolsTrain &amp; CoordinateMotivate 27. MeasureBuild toolsTrain &amp; CoordinateMotivateDedicated 28. Our perf team is two people...But every engineer thinks about performance 29. Everything is slow.Please x. 30. page generationtime 31. 2011 Goal800 ms on ve pages 32. http://www.etsy.com/blog/news/2011/november-2011-site-performance-report/ 33. http://www.etsy.com/blog/news/2011/november-2011-site-performance-report/ 34. Performance degradationis a P2 bug 35. WARNING: This part is hardPerformance degradationis a P2 bug 36. 2012 Goal800 ms for all pages 37. EngineeringDenormalize dataOptimize queriesRewrite from scratchCaching (memcache, app code) 38. OperationsNetwork switchesOperating systemsNewer hardwareAdditional hardware capacity 39. ProductIdentify concerns earlyKill lagging features 40. PerformanceCulture 41. Tools 42. How do you makethe graphs? 43. Perf-related data belongs inyour server logs 44. apache_note(custom_eld, $whatever);LogFormat %l %t "%r" %&gt;s %b "%{Referer}i""%{User-Agent}i" %{custom_field}n 45. LogFormat "%{True-Client-IP}i %l %u %t "%r"%&gt;s %b "%{Referer}i" "%{User-Agent}i"%{etsy_user_id}n %{php_bytes}n %{php_microsec}n%D" combined 46. $timer_start = microtime(true); ...$timer_diff = microtime(true) - $timer_start;register_shutdown_function()apache_note(php_microsec, $timer_diff); 47. web0060 - - [24/Aug/2011:04:16:52+0000] "GET /listing/12189259/tropical-etched-pair-of-lampwork-glass HTTP/1.1" 200 11034 "-""Mozilla/5.0 (compatible; Googlebot/2.1;+http://www.google.com/bot.html)"- 13399576 505780 554876 48. Using something else?time, http method, request uri, response code, referer,user-agent, response time, response memory,custom segmentation elds... 49. Quick averagesgrep "GET /listing/" access.log | awk {sum=sum+$(NF-1)} END {print sum/NR} 50. Quick graphsgrep "GET /listing/" access.log | perl -pe "s/.*[.*d{4}:(d{2}):(d{2}):d{2}.*]/1:2/" | awk {print $1, $(NF-1)} &gt; /tmp/pagetimes.dat gives you... 51. Quick graphs# /tmp/pagetimes.dat18:37 251.018:38 252.118:39 253.518:40 251.018:45 250.0and then... 52. Quick graphs# GNUPLOTset terminal pngset output listings.pngset yrange [0:2000]set xdata timeset timefmt "%d/%B/%Y:%H:%M:%S"set format x "%H:%M"plot /tmp/pagetimes.dat using 1:2 with points 53. Quick graphs 54. LogsterRun by cronMaintains a cursor on log lesSimple parsing &amp; aggregationOutput to Ganglia or Graphite github.com/etsy 55. Logster &amp; Graphite Signed-in vs. Signed-out 56. Response timesresponses (pages) by buckettime (ms) 57. Look at your DataJohn Rauser, Velocity 2011 http://www.youtube.com/watch?v=coNDCIMH8bk 58. Idea lifted fromhttp://www.codinghorror.com/blog/2011/06/performance-is-a-feature.html 59. Dont forget the API 60. Measureperformancedata in Production 61. Proling 62. I/O ProlerLightweight, inline prolingStart and end times wrapped around service calls databases, memcache, apc, 3rd party services, etc.Enable with query string 63. I/O Proler 64. I/O Proler 65. XHProfgithub.com/facebook/xhprofDeployed on pre-prod machineDirect access to prolingEnable with query string 66. Prole againstProduction 67. Make performance matter 68. Start small 69. Make graphs and reports 70. Create simple tools 71. Celebrate performance wins 72. Thank you codeascraft.etsy.comMike Brittainmike@etsy.com@mikebrittain 73. Jobs at Etsy We are currently hiring for a number of positions in engineering and operations. http://www.etsy.com/careers </p>