how test driven development started the robot apocalypse; lessons learned using twilio for telephony

Sunday, December 16, 12

Lessons Learned Using and Testing with TwilioTips, Tricks, and Best Practices


How Test Driven Design started the Robot Apocalypse!... And how it's not my fault!


Customer Call System


Wanted New Efficiencies1. Connecting customers to the same agent;

developing a personal relationship.

2. Automatically popping up the customer record on the agent's browser.

3. Collect call metrics and tie into other datapoints.


Current Provider...1. Offered little to no real-time integration.

2. Was unable or unwilling to customize solution.

3. Was expensive.


Twilio...1. A REST-ful API to make and manipulate calls

and their associated data.

2. Makes real-time callbacks over HTTP to your application about incoming and ongoing calls.

3. Inexpensive: $1 per number, $0.01 per call leg


[censored client]


Call Flows1. A user can click-to-call a target; the user's

phone is called first and is then connected to the target.

2. A user can enter any number and call it; the user's phone is called first and is then connected to the number.

3. If a target calls the mainline, they are immediately connected to a user.

4. Unknown callers to the mainline are placed on a hold queue; any user can handle them.


Demo


Excellent Documentationhttps://twilio.com/docs


https://www.twilio.com/docs

https://www.twilio.com/docs

Productizing Twiliohttp://kalzumeus.com/2011/12/19/productizing-twilio-applications/


http://www.kalzumeus.com/2011/12/19/productizing-twilio-applications/

http://www.kalzumeus.com/2011/12/19/productizing-twilio-applications/

Treat TwiML as Your ViewWe use Builder to generate XML


TWiML Builder Viewxml.instruct!xml.Response do xml.Say 'Hello. Are you' xml.Play 'https://s3.amazonaws.com/CarbonFive/placeholder.wav' xml.Say 'If not, please hold.' xml.Play 'https://s3.amazonaws.com/CarbonFive/sign_off.wav' xml.Enqueue(action: goodbye_twilio_call_path(@call), waitUrl: hold_twilio_call_path(@call)) do xml.text! 'hold' endend


https://s3.amazonaws.com/CarbonFive/sign_off.wav'

https://s3.amazonaws.com/CarbonFive/sign_off.wav'

Port-Forward Callbacks To DevelopmentSet it up yourself or use:localtunnel http://localtunnel.comforward http://forwardhq.com


Model Calls AND ConversationsWe made heavy use of state_machine gemhttps://github.com/pluginaweek/state_machine


https://github.com/pluginaweek/state_machine

https://github.com/pluginaweek/state_machine

Where's the Robopocalypse?


Testing


Deprecated Sandbox


Test AccountLike payment providers, it allows you to make dummy calls, with specific phone numbers resulting in specific responses.


Record responses with VCRSpeeds up test suite. https://github.com/myronmarston/vcr


https://github.com/myronmarston/vcr

https://github.com/myronmarston/vcr

Conversations make for Messy State Machines


On top of all that, we were still figuring it out!


Changes would blow away functionality!


Testing becameManual LaborOur poor PM constantly clicking through scenarios.


PhonioMade use of Twilio JS Client to provide multiple numbers backed by another Twilio account.


How could we automate this?As part of the build and continuous integration.


I didn't know


Gaming BotsScripts that would act as other players to in networked games.


CapybaraRSpec and Cucumber features use it to script a user going through your application.


Remote HostCapybara.run_server = falseCapybara.app_host = 'http://staging.cyberdyne.com'

Alternativelyrequire 'capybara/cucumber'require 'capybara/spec/test_app'

Capybara.app = TestAppCapybara.app_host = 'http://staging.cyberdyne.com'


Supports Multi-"Users"using_session :ahnold do visit '/signin' click 'Terminate', within: '#sarah_connor'end

using_session :robert do visit '/sightings/new' check 'have_you_seen_this_boy' click 'Submit'end


How to do the same for phones?


We're are NOT testing Twilio.


We are testing WITH Twilio.An important distinction!


The Pieces to a Solution were lying around.


What does it look like?

Rspec/Cucumber

CapybaraBrowser

The AppTwilio Account

of the App



Rspec/Cucumber

CapybaraBrowser


of the App

Twilio Accountof the Bots



Rspec/Cucumber

CapybaraBrowser


of the App


Sinatra



Rspec/Cucumber

CapybaraBrowser


of the App


Calls

Sinatra



Rspec/Cucumber

CapybaraBrowser


of the App


CallsBots

Sinatra



Twilio Client

Rspec/Cucumber

CapybaraBrowser


of the App


CallsBots

Sinatra


WOPRhttp://github.com/ZestFinance/woprhttp://github.com/carbonfive/wopr


http://github.com/ZestFinance/wopr

http://github.com/ZestFinance/wopr

http://github.com/carbonfive/wopr

http://github.com/carbonfive/wopr

A @wopr of a Feature@javascript @woprFeature: Outbound Call A user can enter phone number so that they can call it

Scenario: Simple Session Given a user is logged in And the user enters a phone number And the user clicks the Call button Then the user's phone is called And the phone number is called And they are speaking to each other


Setup in Cucumberrequire 'wopr/cucumber'

require File.join(File.dirname(__FILE__), '..', '..', 'staging')

Wopr.configure do |config| config.twilio_server_port = 4000 config.twilio_callback_host = 'http://rudyjahchan.fwd.wf' config.twilio_account_sid = TWILIO_ACCOUNT_SID config.twilio_auth_token = TWILIO_AUTH_TOKENend

require File.join(File.dirname(__FILE__), '..', '..', 'bots')

Wopr::TwilioService.new.update_callbacks([ Wopr::Bot[:ahnold].phone_number, Wopr::Bot[:kyle].phone_number])

Wopr::TwilioCallbackServer.boot


http://rudyjahchan.fwd.wf

http://rudyjahchan.fwd.wf

Creat identified BotsWopr::Bot.create(:ahnold, email: '[email protected]', password: 'n07@r3@1p@$$w0rd', phone_number: '5558675309')

Wopr::Bot.create(:kyle, phone_number: '5557779311')

Wopr::Bot.create(:sarah, phone_number: '9006492568')


mailto:[email protected]


The Goal: Simpler CodeThen /^the user's phone is called$/ do bot(:ahnold).should be_on_a_callend

Then /^the user's phone is not called$/ do bot(:ahnold).should_not be_on_a_callend

Then /^the phone number is called$/ do bot(:kyle).should be_on_a_callend

Then /^they are speaking to each other$/ do bot(:kyle).should be_on_a_call_with(bot(:ahnold))end

Given /^an unknown caller dials the main line$/ do bot(:kyle).make_a_call_to(CYBERDYNE_STAGING_PHONE_NUMBER)end


How does it work?


Stole a LOT from CapybaraParticularly threading code not to block running specs.


Sinatra Appmodule Wopr class TwilioCallbackServer < Sinatra::Base VERIFICATION_PHRASE = 'SHALL WE PLAY A GAME?'

set :views, File.join(File.dirname(__FILE__), 'templates')

get '/__identify__' do [200, {}, VERIFICATION_PHRASE] end

post '/calls' do # ... end

# ...

endend


Mount on Rackdef run_server(port) require 'rack/handler/thin' Thin::Logging.silent = true Rack::Handler::Thin.run(self, Port: port)rescue LoadError require 'rack/handler/webrick' Rack::Handler::WEBrick.run(self, Port: port, AccessLog: [], Logger: WEBrick::Log::new(nil, 0))end


Launch in a threaddef boot(port=Wopr.twilio_server_port) @port = port unless responsive? @server_thread = Thread.new { run_server(@port) } end

Timeout.timeout(60) { @server_thread.join(0.1) until responsive? }end

def responsive? return false if @server_thread && @server_thread.join(0) res = Net::HTTP.start('127.0.0.1', @port) do |http| http.get('/__identify__') end

if res.is_a?(Net::HTTPSuccess) or res.is_a?(Net::HTTPRedirection) return res.body == VERIFICATION_PHRASE endrescue Errno::ECONNREFUSED, Errno::EBADF return falseend


Server manages Callspost '/calls' do if(call = Call.find_by_sid(params[:CallSid])) call.update params else Call.create(params) end

builder :defaultend


<Say /> Keep-alivexml.instruct!xml.Response do xml.Say(loop: 0) do xml.text! <<GIBBERISHYorn desh born, der ritt de gitt der gue, Orn desh, dee born desh, de umn bork! bork! bork!GIBBERISH endend


Bots can examine Callsmodule Wopr class Bot

# ...

def current_call Call.find_all_by_number(phone_number).select{|call| call.status != 'completed'}.last end

def on_a_call? wait_until do current_call end end

# ...

endend


Handle Asynchronicity def eventually(seconds=Wopr.default_wait_time) start_time = Time.now begin yield rescue => e raise e if (Time.now - start_time) >= seconds sleep 1 retry end end

def wait_until(seconds=Wopr.default_wait_time) eventually(seconds) do result = yield return result if result raise ConditionNotMetError end rescue ConditionNotMetError return false end


Bot makes calls w/ Callmodule Wopr class Bot

# ...

def make_a_call_to(phone_number) Call.make(from: self.phone_number, to: phone_number) end

# ... endend

module Wopr class Call class << self def make(options) TwilioService.new.make(options)

end

# ...


TwilioClientServicerequire 'twilio-ruby'

module Wopr class TwilioService def initialize @twilio_client = Twilio::REST::Client.new( Wopr.twilio_account_sid, Wopr.twilio_auth_token ) end

def make(options) calls.create(options.merge( url: "#{Wopr.twilio_callback_host}/calls" )) end

def hangup(sid) call(sid).hangup end # ...


Do we KNOW they're TALKING to each other?It's possible the system made two phone calls, but they’re not with each other.


A Solution: Play and Detect Dial Tones!One bot starts to <Gather> digits, the other <Plays> them, and we confirm they receive it.


Call Gather & Playmodule Wopr class Call

# ...

def play(digits) TwilioService.new.play sid, digits end

def gather TwilioService.new.gather sid end

# ...

endend


Redirect Calls to TWiMLmodule Wopr class TwilioService

# ...

def play(sid, digits) call(sid).redirect_to( "#{Wopr.twilio_callback_host}/calls/#{sid}/play?digits=#{digits}" ) end

def gather(sid) call(sid).redirect_to( "#{Wopr.twilio_callback_host}/calls/#{sid}/gather" ) end

# ...

endend


Prepare to <Gather />

xml.instruct!xml.Response do xml.Gather( timeout: "60", action: "#{Wopr.twilio_callback_host}/calls/#{sid}/gathered", numDigits: "4")end

module Wopr class TwilioCallbackServer < Sinatra::Base

# ...

post '/calls/:sid/gather' do builder :gather, locals: { sid: params[:sid] } end

# ... endend

gather.builder


<Play digits="..." />module Wopr class TwilioCallbackServer < Sinatra::Base

# ...

post '/calls/:sid/play' do builder :play, locals: { sid: params[:sid], digits: params[:digits] } end

# ... endend

xml.instruct!xml.Response do xml.Play(digits: digits) xml.Pause(length: 10)end

play.builder


Gathered Digits Postedmodule Wopr class TwilioCallbackServer < Sinatra::Base

# ...

post '/calls/:sid/gathered' do if(call = Call.find_by_sid(params[:sid])) call.gathered params[:Digits] end

builder :default end

# ...

endend


Again Asynchronous!module Wopr class Bot

# ...

def on_a_call_with?(another_bot) current_call.gather sleep 1 another_bot.current_call.play '6661' wait_until do current_call.gathered_digits.last == '6661' end end

# ...

endend


Dial-Tone not a Perfect Solution.What if tones are used to trigger actions? And how do we confirm audio FILE playback?


<Record> and SOXRetrieve the recording, digest with SOX audio library.


More Capybara Tie-ins?Given /^a user is logged in$/ do bot(:ahnold).log_inend

Given /^the user enters a phone number$/ do bot(:ahnold).within('div#call') do fill_in 'number', with: bot(:kyle).phone_number endend


How far can we take it?


[pic]


Sample Use of woprhttp://github.com/carbonfive/cyberdyne-systems


http://github.com/carbonfive/cyberdyne-systems

http://github.com/carbonfive/cyberdyne-systems

[email protected]@rudy on Twitter




Hasta la vista, baby!


how test driven development started the robot apocalypse; lessons learned using twilio for telephony

Technology

twilio accountcapybara

twilio tips

productizing twilio

use of twilio js client

app browser

forward http

agents browser

localtunnel http