futurestack13: setting up new relic's infrastructure from nic benders & dave peterson, site...
DESCRIPTION
Benefit from our mistakes and hindsight! There's a lot that goes into running a SaaS company - from continuous deployment to DNS to who gets to play with production. We've been running New Relic for over five years now, but if we had to do it over again there are a few things we'd do differently. But there's also a couple things we'd repeat. Come with your questions to this interactive session where we dissect some of the "big decisions" we made in the world of technical operations and how they effected the company and culture.TRANSCRIPT
![Page 1: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/1.jpg)
Setting up New Relic's Infrastructure: A bunch of things we did wrong and a few we did right.DAVE PETERSON & NIC BENDERS2013-10-25
Wednesday, November 6, 13
![Page 3: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/3.jpg)
[“star” graph of code base at launch]
Wednesday, November 6, 13
![Page 4: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/4.jpg)
Wednesday, November 6, 13
![Page 5: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/5.jpg)
100,000req/sec
2Gbps
200TB
Wednesday, November 6, 13
![Page 6: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/6.jpg)
Who Does What?
Wednesday, November 6, 13
![Page 7: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/7.jpg)
Site Operations != IT
Wednesday, November 6, 13
![Page 8: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/8.jpg)
Wednesday, November 6, 13
![Page 9: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/9.jpg)
You are not Batman
Do your work during the dayWednesday, November 6, 13
![Page 10: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/10.jpg)
Lowest hour is still 60% of peak hour!
Wednesday, November 6, 13
![Page 11: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/11.jpg)
Put your serverson the moon
Wednesday, November 6, 13
![Page 12: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/12.jpg)
Put your serverson the moon
Wednesday, November 6, 13
![Page 13: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/13.jpg)
Never open the box
Wednesday, November 6, 13
![Page 14: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/14.jpg)
Domain names are cheap
Wednesday, November 6, 13
![Page 15: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/15.jpg)
10.0.0.0/31 10.0.0.0-10.0.0.1 255.255.255.254 210.0.0.0/30 10.0.0.0-10.0.0.3 255.255.255.252 410.0.0.0/29 10.0.0.0-10.0.0.7 255.255.255.248 810.0.0.0/28 10.0.0.0-10.0.0.15 255.255.255.240 1610.0.0.0/27 10.0.0.0-10.0.0.31 255.255.255.224 3210.0.0.0/26 10.0.0.0-10.0.0.63 255.255.255.192 6410.0.0.0/25 10.0.0.0-10.0.0.127 255.255.255.128 12810.0.0.0/24 10.0.0.0-10.0.0.255 255.255.255.0 25610.0.0.0/23 10.0.0.0-10.0.1.255 255.255.254.0 51210.0.0.0/22 10.0.0.0-10.0.3.255 255.255.252.0 102410.0.0.0/21 10.0.0.0-10.0.7.255 255.255.248.0 204810.0.0.0/20 10.0.0.0-10.0.15.255 255.255.240.0 409610.0.0.0/19 10.0.0.0-10.0.31.255 255.255.224.0 819210.0.0.0/18 10.0.0.0-10.0.63.255 255.255.192.0 1638410.0.0.0/17 10.0.0.0-10.0.127.255 255.255.128.0 3276810.0.0.0/16 10.0.0.0-10.0.255.255 255.255.0.0 65536
Private IPsare FREE
Wednesday, November 6, 13
![Page 16: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/16.jpg)
Hardware
Wednesday, November 6, 13
![Page 17: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/17.jpg)
System Management
Wednesday, November 6, 13
![Page 18: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/18.jpg)
X6550 E5-2450 E5-2643 E5-2690
Average response time
137
ms
167
ms
158
ms
189
ms
Wednesday, November 6, 13
![Page 19: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/19.jpg)
25% performance improvement
2x 6-Core X6550@ 2.67 GHz
2x 8-Core E5-2690@ 2.90 GHz
Wednesday, November 6, 13
![Page 20: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/20.jpg)
Storage
√
Wednesday, November 6, 13
![Page 21: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/21.jpg)
Databases
Wednesday, November 6, 13
![Page 22: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/22.jpg)
32-bits isn't enoughclass CreateTribbles < ActiveRecord::Migration def change create_table :tribbles do |t| t.boolean :hungry t.string :color t.references :parent, index: true end endend
Wednesday, November 6, 13
![Page 23: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/23.jpg)
32-bits isn't enough`id` int(11) NOT NULL AUTO_INCREMENT
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT
`id` SERIAL
Wednesday, November 6, 13
![Page 24: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/24.jpg)
Your database is not a queue
Wednesday, November 6, 13
![Page 25: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/25.jpg)
Long running querieswill kill you
Wednesday, November 6, 13
![Page 26: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/26.jpg)
And now, a thanks to the people who are out there every day fixing the mistakes that we made:
Wednesday, November 6, 13
![Page 27: FUTURESTACK13: Setting up New Relic's Infrastructure from Nic Benders & Dave Peterson, Site Engineering Manager & VP of Operations at New Relic](https://reader033.vdocuments.site/reader033/viewer/2022051313/54813c825806b5c4108b4675/html5/thumbnails/27.jpg)
Jonathan Owens
Jonathan Thurman
Aaron Bento
Jonathan OwensJonathan Owens
Sean Kane
Paul Showalter
Karl Matthias
Wednesday, November 6, 13