淺談Ruby  Process  Fork應用  

大貓    (YC  Ling,  @miaout17)

Applica@on  Concurrency

•  Event-­‐Driven  I/O  (Ex.  Eventmachine,  goliath)  – Everything  need  to  be  non-­‐blocking.    

•  Worker  Threads  (Ex.  Rainbows  with  proper  config)  – Everything  need  to  be  thread-­‐safe.    

•  Worker  Processes  (Ex.  unicorn)  – OS  process  is  expensive  – Reality:  You  are  using  a  lot  of  3rd  party  libraries,  some  of  them  are  blocking  and  not  thread-­‐safe  

– S@ll  mainstream  today  –  I  don’t  like  it  very  much,  but  it  just  works  

 

Worker  Processes  Memory  Usage

Process  1

80MB

Worker  Processes  Memory  Usage

Process  1

80MB

Process  2

80MB

Process  3

80MB

Process  4

80MB

Process  5

80MB

Process  6

80MB

Process  7

80MB

Process  8

80MB

Forked  Worker  Processes  (Ex.  Unicorn)  Memory  Usage:  Copy  on  Write

Process  1

Process  2

Process  3

Process  4

Process  5

Process  6

Process  7

Process  8

Master  Process

Normal  Memory  Page (CoW)Read-­‐Only  Page

God  Monit

Worker  Processes  Startup  Time

Process  1 Process  2 Process  3 Process  4 Process  5 Process  6 Process  7 Process  8 Time

•  Assuming  2  cores  •  The  8  processes  could  be  started  simultaneously  and  the  

startup  will  be  execute  concurrently  with  content  switching.  

Forked  Worker  Processes  (Ex.  Unicorn)  Startup  Time

Process  1 Process  2 Process  3 Process  4 Process  5 Process  6 Process  7 Process  8 Time

Master  Process

•  Load  the  applica@on  ONCE  •  Fork  is  FAST  

Loading  environment

Forking

Resque

Master  Process

Worker  Process

•  The  master  process  loads  the  environment.    •  Whenever  get  a  new  job,  fork  a  worker  process.    •  Whenever  the  job  is  done,  the  worker  process  exits.    •  For  isola@on:  Resque  assumes  chaos

Got  a  job!

Fork  Process

Job  done  Worker  process  terminates  

50  resque  workers?

•  Don’t  do  this:  – COUNT=50  QUEUE=*  rake  resque:workers  

•  80MB*50  =  4GB

Master  Process

80  MB

Master  Process

80  MB

Master  Process

80  MB

Master  Process

80  MB

Master  Process

80  MB

Master  Process

80  MB

…  

Worker  Process

Worker  Process

Worker  Process

resque-­‐pool

•  hgps://github.com/nevans/resque-­‐pool  

Master  Process

Master  Process

Master  Process

Master  Process

Master  Process

Master  Process

Pool  Manager

Fork

Worker  Process

Worker  Process

Worker  Process

Fork

Summary

•  Process  forking  in  Ruby  – For  isola@on  

•  Prevent  process  status  corrup@on  •  See  also:  spork-­‐rails  

– For  concurrency  •  Faster  startup  •  Smaller  memory  footprint