how test driven development started the robot apocalypse; lessons learned using twilio for telephony

Sunday, December 16, 12

Lessons Learned Using and Testing with TwilioTips, Tricks, and Best Practices

How Test Driven Design started the Robot Apocalypse!... And how it's not my fault!

Customer Call System

Wanted New Efficiencies1. Connecting customers to the same agent;

developing a personal relationship.

2. Automatically popping up the customer record on the agent's browser.

3. Collect call metrics and tie into other datapoints.

Current Provider...1. Offered little to no real-time integration.

2. Was unable or unwilling to customize solution.

3. Was expensive.

Twilio...1. A REST-ful API to make and manipulate calls

and their associated data.

2. Makes real-time callbacks over HTTP to your application about incoming and ongoing calls.

3. Inexpensive: $1 per number, $0.01 per call leg

[censored client]

Call Flows1. A user can click-to-call a target; the user's

phone is called first and is then connected to the target.

2. A user can enter any number and call it; the user's phone is called first and is then connected to the number.

3. If a target calls the mainline, they are immediately connected to a user.

4. Unknown callers to the mainline are placed on a hold queue; any user can handle them.

Excellent Documentationhttps://twilio.com/docs

Productizing Twiliohttp://kalzumeus.com/2011/12/19/productizing-twilio-applications/

Treat TwiML as Your ViewWe use Builder to generate XML

TWiML Builder Viewxml.instruct!xml.Response do xml.Say 'Hello. Are you' xml.Play 'https://s3.amazonaws.com/CarbonFive/placeholder.wav' xml.Say 'If not, please hold.' xml.Play 'https://s3.amazonaws.com/CarbonFive/sign_off.wav' xml.Enqueue(action: goodbye_twilio_call_path(@call), waitUrl: hold_twilio_call_path(@call)) do xml.text! 'hold' endend

Port-Forward Callbacks To DevelopmentSet it up yourself or use:localtunnel http://localtunnel.comforward http://forwardhq.com

Model Calls AND ConversationsWe made heavy use of state_machine gemhttps://github.com/pluginaweek/state_machine

Where's the Robopocalypse?

Testing

Deprecated Sandbox

Test AccountLike payment providers, it allows you to make dummy calls, with specific phone numbers resulting in specific responses.

Record responses with VCRSpeeds up test suite. https://github.com/myronmarston/vcr

Conversations make for Messy State Machines

On top of all that, we were still figuring it out!

Changes would blow away functionality!

Testing becameManual LaborOur poor PM constantly clicking through scenarios.

PhonioMade use of Twilio JS Client to provide multiple numbers backed by another Twilio account.

How could we automate this?As part of the build and continuous integration.

I didn't know

Gaming BotsScripts that would act as other players to in networked games.

CapybaraRSpec and Cucumber features use it to script a user going through your application.

Remote HostCapybara.run_server = falseCapybara.app_host = 'http://staging.cyberdyne.com'

Alternativelyrequire 'capybara/cucumber'require 'capybara/spec/test_app'

Capybara.app = TestAppCapybara.app_host = 'http://staging.cyberdyne.com'

Supports Multi-"Users"using_session :ahnold do visit '/signin' click 'Terminate', within: '#sarah_connor'end

using_session :robert do visit '/sightings/new' check 'have_you_seen_this_boy' click 'Submit'end

How to do the same for phones?

We're are NOT testing Twilio.

We are testing WITH Twilio.An important distinction!

The Pieces to a Solution were lying around.

What does it look like?

Rspec/Cucumber

CapybaraBrowser

The AppTwilio Account

of the App

Rspec/Cucumber

CapybaraBrowser

of the App

Twilio Accountof the Bots

Rspec/Cucumber

CapybaraBrowser

of the App

Sinatra

Rspec/Cucumber

CapybaraBrowser

of the App

Sinatra

Rspec/Cucumber

CapybaraBrowser

of the App

Sinatra

Rspec/Cucumber

CapybaraBrowser

of the App

CallsBots

Sinatra

Rspec/Cucumber

CapybaraBrowser

of the App

CallsBots

Sinatra

Twilio Client

Rspec/Cucumber

CapybaraBrowser

of the App

CallsBots

Sinatra

WOPRhttp://github.com/ZestFinance/woprhttp://github.com/carbonfive/wopr

A @wopr of a Feature@javascript @woprFeature: Outbound Call A user can enter phone number so that they can call it

Scenario: Simple Session Given a user is logged in And the user enters a phone number And the user clicks the Call button Then the user's phone is called And the phone number is called And they are speaking to each other

Setup in Cucumberrequire 'wopr/cucumber'

require File.join(File.dirname(__FILE__), '..', '..', 'staging')

Wopr.configure do |config| config.twilio_server_port = 4000 config.twilio_callback_host = 'http://rudyjahchan.fwd.wf' config.twilio_account_sid = TWILIO_ACCOUNT_SID config.twilio_auth_token = TWILIO_AUTH_TOKENend

require File.join(File.dirname(__FILE__), '..', '..', 'bots')

Wopr::TwilioService.new.update_callbacks([ Wopr::Bot[:ahnold].phone_number, Wopr::Bot[:kyle].phone_number])

Wopr::TwilioCallbackServer.boot

Creat identified BotsWopr::Bot.create(:ahnold, email: 'rudy+ahnold@carbonfive.com', password: 'n07@r3@1p@$$w0rd', phone_number: '5558675309')

Wopr::Bot.create(:kyle, phone_number: '5557779311')

Wopr::Bot.create(:sarah, phone_number: '9006492568')

The Goal: Simpler CodeThen /^the user's phone is called$/ do bot(:ahnold).should be_on_a_callend

Then /^the user's phone is not called$/ do bot(:ahnold).should_not be_on_a_callend

Then /^the phone number is called$/ do bot(:kyle).should be_on_a_callend

Then /^they are speaking to each other$/ do bot(:kyle).should be_on_a_call_with(bot(:ahnold))end

Given /^an unknown caller dials the main line$/ do bot(:kyle).make_a_call_to(CYBERDYNE_STAGING_PHONE_NUMBER)end

How does it work?

Stole a LOT from CapybaraParticularly threading code not to block running specs.

Sinatra Appmodule Wopr class TwilioCallbackServer < Sinatra::Base VERIFICATION_PHRASE = 'SHALL WE PLAY A GAME?'

set :views, File.join(File.dirname(__FILE__), 'templates')

get '/__identify__' do [200, {}, VERIFICATION_PHRASE] end

post '/calls' do # ... end

endend

Mount on Rackdef run_server(port) require 'rack/handler/thin' Thin::Logging.silent = true Rack::Handler::Thin.run(self, Port: port)rescue LoadError require 'rack/handler/webrick' Rack::Handler::WEBrick.run(self, Port: port, AccessLog: [], Logger: WEBrick::Log::new(nil, 0))end

Launch in a threaddef boot(port=Wopr.twilio_server_port) @port = port unless responsive? @server_thread = Thread.new { run_server(@port) } end

Timeout.timeout(60) { @server_thread.join(0.1) until responsive? }end

def responsive? return false if @server_thread && @server_thread.join(0) res = Net::HTTP.start('127.0.0.1', @port) do |http| http.get('/__identify__') end

if res.is_a?(Net::HTTPSuccess) or res.is_a?(Net::HTTPRedirection) return res.body == VERIFICATION_PHRASE endrescue Errno::ECONNREFUSED, Errno::EBADF return falseend

Server manages Callspost '/calls' do if(call = Call.find_by_sid(params[:CallSid])) call.update params else Call.create(params) end

builder :defaultend

<Say /> Keep-alivexml.instruct!xml.Response do xml.Say(loop: 0) do xml.text! <<GIBBERISHYorn desh born, der ritt de gitt der gue, Orn desh, dee born desh, de umn bork! bork! bork!GIBBERISH endend

Bots can examine Callsmodule Wopr class Bot

def current_call Call.find_all_by_number(phone_number).select{|call| call.status != 'completed'}.last end

def on_a_call? wait_until do current_call end end

endend

Handle Asynchronicity def eventually(seconds=Wopr.default_wait_time) start_time = Time.now begin yield rescue => e raise e if (Time.now - start_time) >= seconds sleep 1 retry end end

def wait_until(seconds=Wopr.default_wait_time) eventually(seconds) do result = yield return result if result raise ConditionNotMetError end rescue ConditionNotMetError return false end

Bot makes calls w/ Callmodule Wopr class Bot

def make_a_call_to(phone_number) Call.make(from: self.phone_number, to: phone_number) end

# ... endend

module Wopr class Call class << self def make(options) TwilioService.new.make(options)

TwilioClientServicerequire 'twilio-ruby'

module Wopr class TwilioService def initialize @twilio_client = Twilio::REST::Client.new( Wopr.twilio_account_sid, Wopr.twilio_auth_token ) end

def make(options) calls.create(options.merge( url: "#{Wopr.twilio_callback_host}/calls" )) end

def hangup(sid) call(sid).hangup end # ...

Do we KNOW they're TALKING to each other?It's possible the system made two phone calls, but they’re not with each other.

A Solution: Play and Detect Dial Tones!One bot starts to <Gather> digits, the other <Plays> them, and we confirm they receive it.

Call Gather & Playmodule Wopr class Call

def play(digits) TwilioService.new.play sid, digits end

def gather TwilioService.new.gather sid end

endend

Redirect Calls to TWiMLmodule Wopr class TwilioService

def play(sid, digits) call(sid).redirect_to( "#{Wopr.twilio_callback_host}/calls/#{sid}/play?digits=#{digits}" ) end

def gather(sid) call(sid).redirect_to( "#{Wopr.twilio_callback_host}/calls/#{sid}/gather" ) end

endend

Prepare to <Gather />

xml.instruct!xml.Response do xml.Gather( timeout: "60", action: "#{Wopr.twilio_callback_host}/calls/#{sid}/gathered", numDigits: "4")end

module Wopr class TwilioCallbackServer < Sinatra::Base

post '/calls/:sid/gather' do builder :gather, locals: { sid: params[:sid] } end

# ... endend

gather.builder

<Play digits="..." />module Wopr class TwilioCallbackServer < Sinatra::Base

post '/calls/:sid/play' do builder :play, locals: { sid: params[:sid], digits: params[:digits] } end

# ... endend

xml.instruct!xml.Response do xml.Play(digits: digits) xml.Pause(length: 10)end

play.builder

Gathered Digits Postedmodule Wopr class TwilioCallbackServer < Sinatra::Base

post '/calls/:sid/gathered' do if(call = Call.find_by_sid(params[:sid])) call.gathered params[:Digits] end

builder :default end

endend

Again Asynchronous!module Wopr class Bot

def on_a_call_with?(another_bot) current_call.gather sleep 1 another_bot.current_call.play '6661' wait_until do current_call.gathered_digits.last == '6661' end end

endend

Dial-Tone not a Perfect Solution.What if tones are used to trigger actions? And how do we confirm audio FILE playback?

<Record> and SOXRetrieve the recording, digest with SOX audio library.

More Capybara Tie-ins?Given /^a user is logged in$/ do bot(:ahnold).log_inend

Given /^the user enters a phone number$/ do bot(:ahnold).within('div#call') do fill_in 'number', with: bot(:kyle).phone_number endend

How far can we take it?

Sample Use of woprhttp://github.com/carbonfive/cyberdyne-systems

Questions?rudy@carbonfive.com@rudy on Twitter

Hasta la vista, baby!

how test driven development started the robot apocalypse; lessons learned using twilio for telephony

twilio accountcapybara

twilio tips

productizing twilio

use of twilio js client

app browser

forward http

agents browser

localtunnel http

Technology

release 0.1 randall degges - read the docs...django-twilio...

twilio signal 2016 keynote

twilioビジネスセミナー vol.5 -...

twilio con2012 v2

twilio mms deep dive

rapidsms-twilio documentation › pdf › rapidsms-twilio...

twilio mms: taming the complexity

twilio api study vol.1

twilio serveless architecture

twilio using groovy grails

twilio elastic sip trunk provisioning€¦ · twilio...

twilio products from...

twilio case study

twilio japan summit 2014 presentation

twilio intern final presentation

real world twilio

twilio japan summit - twilio for kddi web communications...

twilio transparencyreport 2015

productizing twilio applications

php & twilio