setting up new relic's infrastructure

25
Setting up New Relic's Infrastructure: A bunch of things we did wrong and a few we did right. DAVE PETERSON & NIC BENDERS 2013-10-25

Upload: nic-benders

Post on 29-Nov-2014

1.408 views

Category:

Technology


0 download

DESCRIPTION

 

TRANSCRIPT

Page 1: Setting Up New Relic's Infrastructure

Setting up New Relic's Infrastructure: A bunch of things we did wrong and a few we did right.DAVE PETERSON & NIC BENDERS2013-10-25

Page 3: Setting Up New Relic's Infrastructure

[“star” graph of code base at launch]

Page 4: Setting Up New Relic's Infrastructure
Page 5: Setting Up New Relic's Infrastructure

100,000req/sec

2Gbps

200TB

Page 6: Setting Up New Relic's Infrastructure

Who Does What?

Page 7: Setting Up New Relic's Infrastructure

Site Operations != IT

Page 8: Setting Up New Relic's Infrastructure

You are not Batman

Do your work during the day

Page 9: Setting Up New Relic's Infrastructure

Lowest hour is still 60% of peak hour!

Page 10: Setting Up New Relic's Infrastructure

Put your serverson the moon

Page 11: Setting Up New Relic's Infrastructure

Never open the box

Page 12: Setting Up New Relic's Infrastructure

Domain names are cheap

Page 13: Setting Up New Relic's Infrastructure

10.0.0.0/31 10.0.0.0-10.0.0.1 255.255.255.254 2

10.0.0.0/30 10.0.0.0-10.0.0.3 255.255.255.252 4

10.0.0.0/29 10.0.0.0-10.0.0.7 255.255.255.248 8

10.0.0.0/28 10.0.0.0-10.0.0.15 255.255.255.240 16

10.0.0.0/27 10.0.0.0-10.0.0.31 255.255.255.224 32

10.0.0.0/26 10.0.0.0-10.0.0.63 255.255.255.192 64

10.0.0.0/25 10.0.0.0-10.0.0.127 255.255.255.128 128

10.0.0.0/24 10.0.0.0-10.0.0.255 255.255.255.0 256

10.0.0.0/23 10.0.0.0-10.0.1.255 255.255.254.0 512

10.0.0.0/22 10.0.0.0-10.0.3.255 255.255.252.0 1024

10.0.0.0/21 10.0.0.0-10.0.7.255 255.255.248.0 2048

10.0.0.0/20 10.0.0.0-10.0.15.255 255.255.240.0 4096

10.0.0.0/19 10.0.0.0-10.0.31.255 255.255.224.0 8192

10.0.0.0/18 10.0.0.0-10.0.63.255 255.255.192.0 16384

10.0.0.0/17 10.0.0.0-10.0.127.255 255.255.128.0 32768

10.0.0.0/16 10.0.0.0-10.0.255.255 255.255.0.0 65536

Private IPsare FREE

Page 14: Setting Up New Relic's Infrastructure

Hardware

Page 15: Setting Up New Relic's Infrastructure

System Management

Page 16: Setting Up New Relic's Infrastructure

X6550 E5-2450 E5-2643 E5-2690

Average response time

137

ms

167

ms

158

ms

189

ms

Page 17: Setting Up New Relic's Infrastructure

25% performance improvement

2x 6-Core X6550@ 2.67 GHz

2x 8-Core E5-2690@ 2.90 GHz

Page 18: Setting Up New Relic's Infrastructure

Storage

Page 19: Setting Up New Relic's Infrastructure

Databases

Page 20: Setting Up New Relic's Infrastructure

32-bits isn't enoughclass CreateTribbles < ActiveRecord::Migration def change create_table :tribbles do |t| t.boolean :hungry t.string :color t.references :parent, index: true end endend

Page 21: Setting Up New Relic's Infrastructure

32-bits isn't enough`id` int(11) NOT NULL AUTO_INCREMENT

`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT

`id` SERIAL

Page 22: Setting Up New Relic's Infrastructure

Your database is not a queue

Page 23: Setting Up New Relic's Infrastructure

Long running querieswill kill you

Page 24: Setting Up New Relic's Infrastructure

And now, a thanks to the people who are out there every day fixing the mistakes that we made:

Page 25: Setting Up New Relic's Infrastructure

Jonathan Owens

Jonathan Thurman

Aaron Bento

Jonathan OwensJonathan Owens

Sean Kane

Paul Showalter

Karl Matthias