failing at scale - pnwphp 2016
TRANSCRIPT
![Page 1: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/1.jpg)
Pacific Northwest PHP, September 2016 1
Failing at ScaleChris Tankersley@dragonmantankPacific Northwest PHP, September 2016
![Page 2: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/2.jpg)
Pacific Northwest PHP, September 2016 2
I think something is wrong
![Page 3: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/3.jpg)
Pacific Northwest PHP, September 2016 3
Problems with Distributed Applications• Making sure that configurations are consistent• Making sure that your code gets updated everywhere• Actually finding errors
![Page 4: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/4.jpg)
Pacific Northwest PHP, September 2016 4
I know this is a bug, but where?
![Page 5: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/5.jpg)
Pacific Northwest PHP, September 2016 5
You Can’t Rely on Physical Files• Servers/Containers Disappear• How do you know where something happened?
![Page 6: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/6.jpg)
Pacific Northwest PHP, September 2016 6
syslog
Your Server
PHP Application
Monolog
syslog
Centralized syslog Server
syslog
![Page 7: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/7.jpg)
Pacific Northwest PHP, September 2016 7
Set up the Server# /etc/init.d/syslog.conf, or something similiar
# Make sure the following is setSYSLOGD="-r -m0"
![Page 8: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/8.jpg)
Pacific Northwest PHP, September 2016 8
Set up the server# /etc/syslog.conf, or something similiar
# Set up the log file locations*.emerg /var/log/emergency.log*.alert /var/log/alert.log
# Or lump them together*.emerg /var/log/application.log*.alert /var/log/application.log
![Page 9: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/9.jpg)
Pacific Northwest PHP, September 2016 9
Set up the clients# /etc/syslog.conf, or something similiar
# Send all log types of all log severities to a # single server*.* @<hostname|IP Address>
![Page 10: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/10.jpg)
Pacific Northwest PHP, September 2016 10
Logging SaaS• Same idea as a remote syslog• Usually installs some log watching software• Pretty interface for searching/looking at logs
![Page 11: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/11.jpg)
Pacific Northwest PHP, September 2016 11
Loggly
![Page 12: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/12.jpg)
Pacific Northwest PHP, September 2016 12
ELK Stack• ElasticSearch, LogStash, Kibana• Can run locally or through a SaaS service like logz.io
![Page 13: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/13.jpg)
Pacific Northwest PHP, September 2016 13
Logz.io
![Page 14: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/14.jpg)
Pacific Northwest PHP, September 2016 14
But I swear I fixed that bug!
![Page 15: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/15.jpg)
Pacific Northwest PHP, September 2016 15
Don’t Trust Humans
We Suck
![Page 16: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/16.jpg)
Pacific Northwest PHP, September 2016 16
Anything you do by hand, Automate
![Page 17: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/17.jpg)
Pacific Northwest PHP, September 2016 17
Automating the Build Process• Composer• Grunt/Gulp/npm/asset pipelines• Putting proper config files in place
![Page 18: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/18.jpg)
Pacific Northwest PHP, September 2016 18
Track Everything
![Page 19: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/19.jpg)
Pacific Northwest PHP, September 2016 19
Automating Code Pushes
![Page 20: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/20.jpg)
Pacific Northwest PHP, September 2016 20
I’m pretty sure we have PHP… 5.6 installed?
![Page 21: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/21.jpg)
Pacific Northwest PHP, September 2016 21
Remember, Don’t Trust Humans!
![Page 22: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/22.jpg)
Pacific Northwest PHP, September 2016 22
Ansible• Self contained and can be run from anywhere• Very easy to get set up and running• Lots of “playbooks”
![Page 23: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/23.jpg)
Pacific Northwest PHP, September 2016 23
Puppet• Master-Agent Setup• Constantly checks to make sure machines are compliant• You might be using it already!
![Page 24: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/24.jpg)
Pacific Northwest PHP, September 2016 24
Vagrant Cookbook
![Page 25: Failing at Scale - PNWPHP 2016](https://reader033.vdocuments.site/reader033/viewer/2022052714/5876fdaa1a28abf3398b6b0f/html5/thumbnails/25.jpg)
Pacific Northwest PHP, September 2016 25
Thank You!• https://github.com/dragonmantank• Author of “Docker for Developers”
• https://leanpub.com/dockerfordevs
• http://ctankersley.com• [email protected]• @dragonmantank
• https://joind.in/talk/64ad5