handling long-running tasks in rails
TRANSCRIPT
Long-Running Tasks In RailsWithout Much Effort
Andy Stewart
April 2008
Not me
Still not me
Why?
User startssomething
lengthy
Delayedtask
Regulartask
Example
Synchronous link to Campaign Monitor
10
BackgroundJob (BJ)
Delayed Job (DJ)
BackgrounDRbBeanstalkd &
async-observer
AM4R
Workling
Spawn
BackgroundFu
Sparrow
Conveyor
Factors
DurabilityProgress reporting
SchedulingSerial vs. Parallel
Process managementExecution environment
Error handlingLearning curve
Installation burdenConstraints on task code
Durability
Durable Hopeful
Background JobBackgroundFuDelayed JobWorkling
BackgrounDRbBeanstalk
SpawnSparrowWorkling
50%0% 100%
Tool Method
Background Job Process’s stats
BackgroundFu Incremental
BackgrounDRb Ask workers
Workling DIY “return store”
Scheduling
BackgrounDRb
cron + {rake, script/runner, whatever}
Delaying(on purpose)
Delayed Job
Beanstalk
ProcessManagement
No Problem Hassle
Background JobSpawn
Everything else
Examples
BackgrounDRb
# Lots of configuration...
# Your worker codeclass BillingWorker < BackgrounDRb::MetaWorker set_worker_name :billing_worker
# Called when worker is loaded for the first time def create(args = nil) end
# The lengthy task. def charge_customer(customer_id = nil) # ... do stuff ... endend
# Your invocation codeclass CustomersController < ApplicationController def upgrade_account # ... MiddleMan.worker(:billing_worker).charge_customer(@customer.id) endend
Beanstalk# Some configuration...
# Comments controllerdef create @comment = Comment.new(params[:comment]) if @comment.save BEANSTALK.yput({:type => "comment", :id => @comment.id}) rescue nil # Then redirect and return endend
# Worker - Rake taskloop do job = BEANSTALK.reserve job_hash = job.ybody # ybody deserializes the job case job_hash[:type] when "comment" if Comment.check_for_spam(job_hash[:id]) job.delete else job.bury end else puts "Don't know what type of job this is: #{job_hash.inspect}" endend
Source: nubyonrails.com
Async Observer
# Some configuration...
# Start some workers$ ./vendor/plugins/async_observer/bin/worker
# Your codeclass Person < ActiveRecord::Base async_after_create do |person| SiteStats.increment_members() end
def befriend(other_person) Friendship.async_send(:create, self, other_person) endend
Delayed Job
# Small config - db migration
# A job is a Ruby object with #performclass NewsletterJob < Struct.new(:text, :emails) def perform emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) } endend
# Stick in queueDelayed::Job.enqueue NewsletterJob.new('lorem ipsum...', Customers.find(:all).collect(&:email))
# Delayed jobBatchImporter.new(Shop.find(1)).send_later(:import_massive_csv, massive_csv)
# Running tasks$ rake jobs:work <CTRL-C to cancel>
Background Job
# Command line
$ Bj.submit "./jobs/background_job_to_run"
$ Bj.submit "./script/runner ./jobs/background_job_to_run"
# Within Rails
def upload_to_s3 Bj.submit "./script/runner ./jobs/s3_uploader.rb #{self.id}" end
Source: slackworks.com
1
Background Job
Two-line installationZero configuration
No job-code constraintsAutomatic* process managementDurable, prioritised, taggable jobs
Comprehensive job resultsClustering
* Manual if you want
Feed my spaniel
Buy my PeepCode PDF
Text’s Licence:Creative Commons Attribution-Share Alike 2.0 UK: England & Wales Licence
My Photos’ Licence:Creative Commons Attribution-Noncommercial-No Derivative Works 2.0
UK: England & Wales Licence
Photo Credits:1. BBC2. Forever in Song (CD cover)3. Me4. Me