ø downtimeJônatas Davi Paganini

jonatasjonatasdp

8 dev teams

10 deploys / day

RD Station

+2m mail / day

+700 external services

integrations

+N callbacks

Availability is

REQUIRED

99,9%

99,5%

99,0%

98,6%

98,0%

97,0%

99,99%

00:43:00

03:36:00

07:12:00

10:00:00

14:24:00

21:36:00

00:04:32

Availability

99.999???

availability =( total_time - timeout / total_time ) * 100

ø downtime

mindset

NO schedulemaintenance

avoid: all || nothing

build reversible things

build incremental migrations

RubyConf 2012

compatible versions

http://shipit.resultadosdigitais.com.br/blog/migrando-com-zero-downtime

talk is <cheap>!

show me the code!!!

Migration

add_column :people, :full_name, :string

Update data

Person.all.each do |person| person.full_name = "#{person.first_name} #{person.last_name}" person.saveend

350 million updates!!!

WTF time?!

Let’s improve it!

Person.all.each do |person| person.full_name = "#{person.first_name} #{person.last_name}" person.saveend

Select RIGHT attributes!

Update 0.1

Person.select("id,first_name,last_name").each do |person| person.full_name = "#{person.first_name} #{person.last_name}" person.saveend

update_attribute instead of save

Update 0.2

Person.select("id,first_name,last_name").each do |person| person.update_attribute "full_name", "#{person.first_name} #{person.last_name}"end

find in batches

avoid transaction

overhead

Update 0.3

Person.select("id,first_name,last_name").find_in_batches do |people| People.transaction do people.each do |person| person.update_attribute "full_name", "#{person.first_name} #{person.last_name}" end endend

FAILURE resilient

Update 0.4

Person.where(full_name: nil).select("id,first_name,last_name").find_in_batches do |people| People.transaction do people.each do |person| person.update_attribute "full_name", "#{person.first_name} #{person.last_name}" end endend

explore connection pool

Update 0.5

update_sql = "UPDATE people set full_name ="

Person.where(full_name: nil).select("id,first_name,last_name").find_in_batches do |people| People.transaction do people.each(ActiveRecord::Base.connection_config[:pool]) do |person| ActiveRecord::Base.connection_pool.with_connection do |conn| set_full_name = conn.quote("#{person.first_name} #{person.last_name}") conn.execute("#{update_sql} #{set_full_name} where id = #{person.id} ") end end endend

model hook

Model

class Person < ActiveRecord::Base before_save :update_full_name

def update_full_name self.full_name = "#{first_name} #{last_name}" endend

rollout control

Rollout

class Rollout def enabled?(feature, context) redis.sismember(feature, context) rescue Redis::BaseError false end

def enable(feature, context) redis.sadd(feature) end def disable(feature, context) redis.srem(feature) end

private def redis Redis::Namespace.new("rollout", redis: $redis) endend

rollout use

if:

before_save :update_full_name, if: -> { Rollout.enabled? "started_migration", "global" }

deploy steps

smoothlysteps

PR #1

add new column

PR #2

write old & new columns

PR #3

migrate data

PR #4

read from new column

PR #5

remove old column

PR #6

remove rollout code, cache

Conclusion

Embrace migrations ● smoothly● incremental● safe● resilient

Avoid migrations

● all || nothing● ! roll outable features

REFERENCES

blog.codeship.com/rails-migrations-zero-downtime/

shipit.resultadosdigitais.com.br/blog/migrando-com-zero-downtime/

confreaks.tv/videos/railsconf2012-zero-downtime-deploys-for-rails-apps

?

Thanks!jonatasjonatasdp

ideia.meshipit.resultadosdigitais.com.br