supersize me: making drupal go large
DESCRIPTION
My talk from Drupalcamp London Business Day on 1st March 2013 When building big websites, you're going to face a lot of problems regardless of your technology choice. This talk unveils some of the common problems, and shows how the Drupal community will help you solve these problems.TRANSCRIPT
![Page 1: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/1.jpg)
SUPERSIZE MEMaking Drupal go large
![Page 2: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/2.jpg)
INTRODUCTION
•Who are we?
• Building BIG Drupal
• Three pillars of scale
• Care in the community
•Q&A
![Page 3: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/3.jpg)
WHO ARE WE?
![Page 4: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/4.jpg)
![Page 5: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/5.jpg)
![Page 6: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/6.jpg)
COMMON MISCONCEPTIONS
• “Drupal is slow”
• “Drupal doesn’t scale”
• “It’s only good for a hobby site”
![Page 7: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/7.jpg)
WRONG!
![Page 8: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/8.jpg)
DEVELOPING BIG WEBSITES WITH DRUPAL IS NO DIFFERENT TO
DEVELOPING THEM WITH ANY OTHER SOFTWARE
![Page 9: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/9.jpg)
DEVELOPING BIG WEBSITES WITH DRUPAL IS NO DIFFERENT TO
DEVELOPING THEM WITH ANY OTHER SOFTWARE
![Page 10: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/10.jpg)
DRUPAL IS NO DIFFERENT TO
ANY OTHER SOFTWARE
![Page 11: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/11.jpg)
SCALING PERFORMANCEThe first pillar-box of scale
![Page 12: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/12.jpg)
SCALING COMPLEXITY
• Caching and Reverse proxy caches
•Queues
• Alternative database backends
• Throttling
![Page 13: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/13.jpg)
MONITORING
• All the common open source tools...
•Munin, Nagios, etc
• Plus some new ones
•New Relic, Boundary
• Social media
• saved our site...
![Page 14: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/14.jpg)
SCALING PEOPLEWe’re going to need a bigger boat
![Page 15: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/15.jpg)
COMMUNICATION
![Page 16: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/16.jpg)
PRIORITISATION
![Page 17: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/17.jpg)
DISTRIBUTED WORKING
![Page 18: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/18.jpg)
REDUNDANCY
![Page 19: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/19.jpg)
PROJECTS CONSIST OF CODE
![Page 20: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/20.jpg)
PROJECTS CONSIST OF CODE
CODE IS WRITTEN BY PEOPLE
![Page 21: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/21.jpg)
PEOPLE DELIVER PROJECTS
PROJECTS CONSIST OF CODE
CODE IS WRITTEN BY PEOPLE
![Page 22: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/22.jpg)
SCALING PROCESSES
![Page 23: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/23.jpg)
YOU STILL NEED TO DO ALL THE THINGS YOU WOULD DO
ON ANY BIG SOFTWARE PROJECT
![Page 24: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/24.jpg)
WHAT THINGS?
• Provide clarity of purpose
• Automated and manual testing
• Continuous integration
• Performance testing and benchmarking
• Code review
• Source control
• Release management
• Address technical debt
• Be agile (with a little ‘a’)
![Page 25: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/25.jpg)
CLARITY OF PURPOSE
• Your team must know how to “do”
• The definition of “done”
•What “good” looks like
![Page 26: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/26.jpg)
TESTING
• Know how you’re going to test, before you write a single line of code
•Do it regularly
•Make it repeatable and consistent
• Automate as much as possible
• Test Driven Development (TDD) or Behaviour Driven Development (BDD) is even better
![Page 27: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/27.jpg)
CONTINUOUS INTEGRATION
• Fail fast, Fail early
• Start your project the way you mean to go on
• Provide an environment where developers are allowed to make mistakes (not just dev)
• Automate your environment setup
• Regularly build your environments - including deploying to production
![Page 28: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/28.jpg)
“IT DOESN’T MATTER HOW YOUR BUILD SCRIPT WORKS, IT ONLY MATTERS THAT IT
WORKS”
![Page 29: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/29.jpg)
PERFORMANCE TESTING
• Know what you’re aiming for
• Test early, test regularly
• Benchmark common journeys and monitor changes over time
•Memory usage, Page execution time etc
• Acknowledge that Production is your only real test
![Page 30: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/30.jpg)
KNOW YOUR PEAKS
![Page 31: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/31.jpg)
CODE REVIEW
![Page 32: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/32.jpg)
TECHNICAL DEBT
![Page 33: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/33.jpg)
CARE IN THE COMMUNITYThe good news
![Page 34: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/34.jpg)
YOU DON’T NEED TO REINVENT THE WHEEL
![Page 35: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/35.jpg)
YOU’RE NOT THE FIRST
• Pressflow
• Published approaches to solving performance problems
• Catch’s Github page
•Drupal.org documented performance improvements
•Drupal 7 Fields optimisation (ported from Drupal 8)
• Porting performance improvements from newer versions
![Page 36: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/36.jpg)
EXPERTS ARE OUT THERE
•Drupal issue queues
•Drupal High Performance Group
• IRC
•Meet-ups
•DrupalCamps and DrupalCons
• Large Scale Drupal (LSD) programme
![Page 37: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/37.jpg)
DRUPAL PLAYS WELL WITH OTHERS
![Page 38: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/38.jpg)
CONTRIBUTION
• Contribution encourages good developer practice
•Testing: core changes always accompanied by a test
•Review: minimum of 2 people will review any change to core
•Technical debt: critical issue thresholds block new features
![Page 39: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/39.jpg)
SHARE AND SHARE ALIKE
• Route
• STOMP
• SQL No Revisions
• Drupal 8 Configuration Management Initiative (CMI)
• Test and Target
• Panels Filter Cache
• Various core and contrib patches
![Page 40: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/40.jpg)
CONTRIBUTION CYCLE
![Page 41: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/41.jpg)
WRAPPING UP
![Page 42: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/42.jpg)
DRUPAL IS NO DIFFERENT TO
ANY OTHER SOFTWARE
![Page 43: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/43.jpg)
THE COMMUNITY IS
![Page 44: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/44.jpg)
WRAPPING UP
• If you’re building big, a lot of problems exist regardless of the technology
•Drupal provides ways to solve these problems
• The Drupal community is one of the best communities to have a problem in
•Drupal does scale, in performance, in people, in process
![Page 45: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/45.jpg)
QUESTIONS?
Tweet: @tsphetheanDrupal.org: tsphethean
IRC: tsphethean
Tweet: @alexpottDrupal.org: alexpott
IRC: alexpott
![Page 46: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/46.jpg)
WE’RE HERE ALL WEEKENDCome visit us on our stand
and see our other presentations
• Drupal 8 Configuration system for coders - Alex Pott (Sunday 10:30am)
• Please hold: your call is in a queue - Tom Phethean (Sunday 10:30am)
• zip-BDD-do-dah zip-BDD-ay! - Graham Taylor and Andrew Larcombe (Sunday 11:30am)
![Page 47: Supersize me: Making Drupal go large](https://reader033.vdocuments.site/reader033/viewer/2022052618/554bc8b2b4c90594278b5741/html5/thumbnails/47.jpg)
IMAGE ATTRIBUTIONS
• The Elephants are coming - http://www.fromthewilderness.com/free/ww3/011905_university_washington.shtml
• Pillar Box - http://www.oxfordhistory.org.uk/streets/post_boxes/edward_vii/index.html
• Jaws - http://edge.ascd.org/_Youre-Gonna-Need-a-Bigger-Boat/blog/6287739/127586.html
• Communication - http://familybugs.wordpress.com/2012/04/30/communication-part-1/
• Prioritisation - http://neilkillick.com/2011/12/17/tips-on-speedy-product-backlog-prioritisationordering/
• Remote working - http://www.wolfden.co.uk/RemoteWorking.html
• Redundancy - http://blog.800hosting.com/2011/07/workstation-backup-solutions-pt-3-redundancy/
• Scaling rock face - http://www.flickr.com/photos/alexindigo/2571638877
• Christmas - http://www.tunisia-live.net/2011/12/23/christmas-celebrations-in-tunisia-tis-the-season/
• Volcanic ash cloud - http://www.guardian.co.uk/world/blog/2010/apr/15/volcano-airport-disruption-iceland
• Technical debt - http://agileandbeyond.blogspot.co.uk/2011/05/velocity-handle-with-care.html
• Reinventing the wheel - http://www.sbi-conferences.com/dont-reinvent-the-wheel.html