rocket fuelled cucumbers

Rocket FuelledCucumbers

Joseph Wilk

Dealing with an Outbreak of Cucumbers

Joseph Wilk

No Cucumbers where harmed in the making of

this presentation.

Just mildly shaken up

Scenario: Happy Railsconf attendees Given you have a vague idea what Cucumber is When Joseph completes his presentation Then you should understand the scaling choices And you should feel better prepared to face them

Cucumber

• BDD framework

• Plaintext

• Promotes communication

Given /^I have trained monkeys$/ do @test_pilots << Monkeys.newend

Feature: Filling Cucumbers with rocket fuel

Scenario: Rocket fuel Given I have trained monkeys When I click the launch button Then the monkeys should not die

Your code makes it slow

Cucumber provides the venue

Shame on you

What are Acceptance Tests?

• Customer-facing

• Cut through the whole stack

• Examples of how the system is intended to be used

You mademe Cuke

1 minute ~ build

Local builds

1 minute ~ build

AutotestLocal builds

1 minute ~ build

AutotestLocal builds

Don’tPanic

Slow testing problems

Development is slowed down by the time to run

focused Cukes

Running all the Cukes takes too long!

Development

Test Build

Types of Feedback

SpeedConfidence

Happiness is a 10 min build time

Slow builds arethe enemy of continuous integration

Development Feedback

Its too slow to run the cukes.Just push the code and run away

Reuse - Spork$ spork cucumber$ cucumber --drb

require 'rubygems'require 'spork'

Spork.prefork do puts "I'm loading all the heavy stuff..."end

Spork.each_run do puts "I'm loading the stuff just for this run..." # Cucumber hooks must go hereend

git://github.com/timcharper/spork.git

Slow services

• Search

• Solr

• Sphinx

• Databases

• Mongo

• Redis

• Mysql

• Message Queues

• RabbitMQ

Before('@solr') do Solr.boot unless Solr.running?end

Feature: Searching

@solrScenario: Indexed search Given ... When ... Then ...

Run Just Enough Tests

# * Test files must be stored in test/# * Test files names must start with test_# * Test class names must start with Test# * Implementation files must be stored in lib/# * Implementation files must match up with a # test file named# test_.*implementation.rb

@analyzer = Rcov::CodeCoverageAnalyzer.new

Cucover

Autotest

http://vimeo.com/12500864

Slicing Features

Tagging

Filenames

$ cucumber --tags @media

Subsystems

$ cucumber visitor_*

$ cucumber features/admin/*

Profiles $ cucumber --profile admin

@media, @publicFeature: Visitor views artist’s media

@feed, @adminFeature: Logged in user views views their activity feed

Just enough Database

city data

venue data

SELECT * FROM `cities` WHERE (`cities`.`id` = 105838)

INSERT INTO `cities` (`name`) VALUES ('Test name')

INSERT INTO `venues` ...

SELECT * FROM `venues`

WHERE ...

Test Build Feedback

Commit codeHave a 1 hour nap

Options

• Hardware

• Intelligent test selection

• Tighter focused tests

• Divide and Conquer

TestjourSongkick.com

254 Features, 1257 scenarios, 10807 steps

1 build server ~ 4 hours

5 build servers ~ 2 hours

120 sad minutes of my life...

$ git clone git://github.com/brynary/testjour.git$ rake gem install

$ testjour --help

$ git clone git://github.com/brynary/testjour.git$ rake gem install

$ testjour --help

testjour help:

Testjour

MasterWorkQueue

Worker

Testjour

MasterWorkQueue

Slaversync - code

rsync - codeWorker

Worker

Testjour

MasterWorkQueue

Slaversync - code

rsync - codeWorker

Worker

Testjour

MasterWorkQueue

Slaversync - code

rsync - codeWorker

Worker

Looking to the Cloud

EC2, Rackspace Cloud, Windows Azure

20 build servers ~ 11:40 minutes

$2068.99 / $3000

slim-sumo

Disc image

AMILaunch

slim-sumo

Disc image

AMILaunch

ec2 node

slim-sumo

Disc image

AMILaunch

ec2 node

slim-sumo

BuildENV['EC2_URL'] = ec2_urlrequire 'slim-sumo'Sumo.new.running.map{|ec2_node| ec2_node[:hostname]}

Disc image

require 'rubygems'require 'hydra'require 'hydra/tasks'

Hydra::TestTask.new('hydra') do |t| t.add_files 'features/**/*.feature' t.verbose = false t.autosort = falseend

“Buy lots of hardware”

“more hardware please”

“Just a little bit more hardware”

Just run the tests that matter...

Failure probability matters

European Computing manufacturer

Automated test suite ~ 18 hours

Tests that were most ~ 2 hours likely to fail

Automated test suite ~ 18 hours

Tests that never fail

Tests which regularly fail

Daily Build

Daily Build Nightly Build

$cucumber --tags ~@nightly $cucumber --tags @nightly

Flickering Confidence

Main Flicker@flicker

Divide and conquer

• Rails - Engines

• Service Orientated Architecture

1 hour

Divide and Concuquer

Scenario: Related artists Given an artist “SYGC” And “M.Bison” is related to “SYGC” When I visit the artist Then I should see within Similar Artists a link to “M.Bison”

Divide and conquer

RelatedArtistsService

Divide and conquer

related(artist)

Divide and conquer

related(artist)

HTMLRails

Divide and conquer

related(artist)

HTMLRails

Divide and conquer

related(artist)

HTMLRails

unit testunit test

Don’t use Acceptance tests

http://jamesshore.com/Blog/Alternatives-to-Acceptance-Testing.html

Heresy!

PairwiseScenario: Testing Cucumber against different environments Given I have a rails app <Rails version> And I’m using Ruby <Ruby version> And I am using the <Test Framework> And I am using the driver <Driver> Then Cucumber should install and play nicely Examples: | Rails version | Ruby version | test framework | Driver | | 3.0 | 1.8.7 | Rspec | Webrat | | 3.0 | 1.8.7 | Testunit | Capybara | | 3.0 | 1.8.7 | Rspec | Capybara | | 3.0 | 1.8.7 | Testunit | Webrat | | 3.0 | 1.9 | Rspec | Webrat | | 3.0 | 1.9 | Testunit | Webrat | | 2.3.1 | 1.9 | Rspec | Webrat | ....

| Rails version | Ruby version | test framework | Driver | | 2.3.1 | 1.8.7 | Rspec | Webrat | | 2.3.1 | 1.8.7 | Rspec | Capybara | | 2.3.1 | 1.8.7 | Testunit | Webrat | | 2.3.1 | 1.8.7 | Testunit | Capybara | | 2.3.1 | 1.9 | Rspec | Capybara | | 2.3.1 | 1.9 | Rspec | Webrat | | 2.3.1 | 1.9 | Testunit | Capybara | | 2.3.1 | 1.9 | Testunit | Webrat |

“most faults are caused by interactions of at most two factors”

Pairwise

http://www.screencast.com/t/NGI0NjVk

We should add some funky Ajax features

Just when you thought things were going well

JavaScript

In-memoryBrowser basedSelenium, Watir Celerity

Need for speedWebrat

Celerity

Selenium

0 3.75 7.5 11.25 15

start-up time (seconds)

CapybaraLike Webrat but bigger

envjsJohnson

spidermonkey

gem install johnson --prerelease

gem install envjs

sudo gem install capybara

Capybara-envjs

Find the right compromise for you

Slow test builds produce slow release cycles

Find the right compromise for you

Slow test builds produce slow release cycles

Your test builds are a goldmine of data

Thanks

Joseph Wilkhttp://blog.josephwilk.net

@josephwilk

Its been emotional

rocket fuelled cucumbers

filling cucumbers

cucumber iswhen joseph

rocket fuelgiven

vague idea

happy railsconf attendeesgiven

scaling choicesand

making ofthis presentation

Technology

hydrogen fuelled electricity generation fuelled...

sea cucumbers - shcollege.ac.in

open flame propane fuelled preheating · pdf filetitle: open...

client - fuelled

cultivation of cucumbers in green houses

cucumbers have layers - rubyconf 2015

cabbage, cauliflower, turnips, cucumbers, melons, and

water fuelled car

oecd cucumbers explanatory brochure

easytravel by street legal cucumbers

2011, marintek,gas fuelled ships

open flame propane fuelled preheating nozzles · title:...

revised bubble fuelled logo presentation

specifications - fuelled

eat cucumbers

aquablue ventures sea cucumbers & sustainable aquaculture

jelovnik - poklisar.com · beef tartare with pickled...

restaurant scapinosalata verde, spanac, rucola, castraveti,...

cucumbers - wa

growing guide: cucumbers