automated testing with ruby

Automated Testingwith Ruby

Keith Pitty

Open Source Developers’ Conference, Sydney 20084th December

Once upon a time...

way back in 1983...

a young programmer made his wayfrom the world

of Unix at university...

to the world ofIBM mainframe programming.

He learnt to use suchweird and wonderful

technologies as...

PL/I, IMS, CICS, TSO and ISPF/PDF.

Fortunately he had a good mentorwho taught him about

modular programming...

and how to write unit tests with...

Job Control Language

“JCL is the stuff of nightmares for many programmers and operators.”

From editorial review for “MVS JCL in Plain English”

fast forward to 2001

Our programmer was not so young any more...

but he had moved on from mainframe progamming...

to the exciting new world of Java...

and...

as a byproduct ofeXtreme Programming...

he had fallen in love with...

JUnit

He loved using JUnit to do test-driven development

and refactoring...

and encouraged others to use itwhenever he had the opportunity.

fast forward to 2005

By now our hero hadbegun to explore...

and...

He liked the way tests were “baked in” and how you could run them with rake.

Then, one day in 2007,a fellow Rails programmer

enquired...

“Are you using autotest?”

Sheepishly he admittedhe hadn’t even heard of it...

We’ll leave our story there for now.

Autotest

“a continuous testing facility meant to be used during development.”

sudo gem install ZenTest

cd path/to/railsapp

autotest

“As soon as you save a file, autotest will run the

corresponding dependent tests.”

Test::Unit

an implementation of xUnit

the default Ruby testing tool

distributed with Ruby since 2001

also the default testing tool for Rails

assert_kind_of BowlingAnalysis, @bowling_analysisassert_equal "Brett Lee", @bowling_analysis.player.full_nameassert_equal "9.3", @bowling_analysis.overs.to_sassert_equal 1, @bowling_analysis.maidensassert_equal 2, @bowling_analysis.wicketsassert_equal 50, @bowling_analysis.runs

Rails by default provides unit, functional and integration tests

RSpec

RSpec allows you to specify behaviour

Specifying a model

describe Player do before(:each) do @valid_attributes = { :first_name => "value for first_name", :last_name => "value for last_name", :active => true } end

it "should create a new instance given valid attributes" do Player.create!(@valid_attributes) endend

Specifying a controller

describe PlayersController do # missing code here will be revealed soon it "should respond successfully to get 'index' with list of players" do Player.should_receive(:find).and_return(@players) get 'index' response.should be_success assigns[:players].should == @players endend

What makes a good test?

1. Isolated

2. Automated

3. Quick to write

4. Quick to run

5. Unique

Kent Beck suggests:

describe PlayersController do before(:each) do @player_1 = mock(Player, {:first_name => "Greg", :last_name => "Norman", :active => true}) @player_2 = mock(Player, {:first_name => "Adam", :last_name => "Scott", :active => true}) @players = [@player_1, @player_2] end it "should respond successfully to get 'index' with list of players" do Player.should_receive(:find).and_return(@players) get 'index' response.should be_success assigns[:players].should == @players endend

Mocks

Mocks mimic real objects

Mocks allow tests to be isolated

For more see “Mock Dialogue”by Jim Weirich and Joe O’Brien

Other Mock Frameworks

mocha

flexmock

rr

Autospec

Autospec rather than autotestsince RSpec 1.1.5

More on RSpec

Can also specify viewsor

integrate views with controller specs

Integration Testing

Can use Test::Unitbut I prefer...

Cucumber

A replacement for RSpec Story Runner

based on definition of scenarios within features

Feature: Name of feature In order to derive some business value As a user I want to take certain actions Scenario: Name of scenario Given a prerequisite When I take some specific action Then I should notice the expected outcome

cucumber

or

AUTOFEATURE=true autospec

can be used in conjunction with tools likeWebrat and Selenium

Webrat

simulates in-browser testingby leveraging the DOMwithout performance hit

Feature: Player belongs to group In order for a player to be included As a recorder I want to record when a player joins or leaves Scenario: Record when a new player joins Given a player is not in the system When I request a list of active players And I follow "Add Player" And I fill in "player_first_name" with "Greg" And I fill in "player_last_name" with "Norman" And I press "Save Player" Then I should see "Active Players" And I should see "Greg Norman"

makes use of Cucumber step definitions for Webrat, for example...

When /^I follow "(.*)"$/ do |link| clicks_link(link)end

When /^I fill in "(.*)" with "(.*)"$/ do |field, value| fills_in(field, :with => value) end

When /^I press "(.*)"$/ do |button| clicks_button(button)end

Then /^I should see "(.*)"$/ do |text| response.body.should =~ /#{text}/mend

Selenium

• runs tests via a browser

• handles JavaScript

a little more effort thanjust installing the Selenium gem

Feature: Check OSDC Site In order to keep track of conference details A participant Should be able to browse the OSDC site Scenario: Check the home page Given I am on the OSDC home page Then I should see the text "Welcome to the Open Source Developers' Conference"

require 'spec'require 'selenium'

Before do @browser = Selenium::SeleniumDriver.new("localhost", 4444, "*chrome", "http://localhost", 15000) @browser.startend

After do @browser.stopend

Given /I am on the OSDC home page/ do @browser.open 'http://www.osdc.com.au/'end

Then /I should see the text "(.*)"/ do |text| @browser.is_text_present(text).should == trueend

Fixtures

allow data for automated tests to be defined using yaml

can be a pain to maintain

Machinist

Blueprints to generate ActiveRecord objects

Hole.blueprint do number 1end

(1..18).each {|n| Hole.make(:number => n) }

Machinist with Sham

Sham.name { Faker::Name.name }

Sham.date do Date.civil((2005..2009).to_a.rand, (1..12).to_a.rand, (1..28).to_a.rand)end

Player.blueprint do first_name { Sham.name } last_name { Sham.name } active trueend

Round.blueprint do date_played { Sham.date }end

use blueprints in cucumber steps

now available as a gem on github

Other Tools

JavaScript Unit Testing

Rails plugins:

javascript_testjavascript_test_autotest

js_autotest

see Dr Nic’s blog post

Shoulda

Extends Test::Unit with the idea of contexts

Testing Philosophy

How much effort is warranted?

Is it taking too much effort

to learn this new testing tool?

My Opinion

Automated testing can be viewed as a trade-off.

Effort invested in automated tests should result in at least

an equivalent improvement in software quality.

E <= ΔQ

I think it is worth striving to continually improve my command

of automated testing tools...

always remembering the business context.

Quotes

“Producing successful, reliable software involves mixing and matching

an all-too-variable number of error removal approaches,

typically the more of them the better.”

Robert L. GlassFacts and Fallacies of Software Engineering

“Whether you are a hard-core ‘test first’ person is not the point. The point is that everyone

can benefit from tests that are automated, easy to write,

and easy to run.”

Hal FultonThe Ruby Way

References

• Hal Fulton. The Ruby Way. Addison-Wesley, 2007

• Obie Fernandez. The Rails Way. Addison-Wesley, 2008

• David Chelimsky et al. The RSpec Book: Behaviour Driven Development with Ruby. The Pragmatic Programmers, 2009

• Robert L. Glass. Facts and Fallacies of Software Engineering. Addison-Wesley, 2003

• http://zentest.rubyforge.org/ZenTest/

• http://rspec.info

• http://rubyhoedown2008.confreaks.com/02-joe-obrien-and-jim-weirich-mock-dialogue.html

• http://www.infoq.com/news/2008/10/qualities_good_test

• http://github.com/aslakhellesoy/cucumber

• http://github.com/brynary/webrat

• http://github.com/aslakhellesoy/cucumber/wikis/setting-up-selenium

• http://github.com/notahat/machinist

• http://toolmantim.com/article/2008/10/27/fixtureless_datas_with_machinist_and_sham

• http://drnicwilliams.com/2008/01/04/autotesting-javascript-in-rails/

• http://www.thoughtbot.com/projects/shoulda

• http://pragdave.blogs.pragprog.com/pragdave/2008/04/shoulda-used-th.html

Thanks for listening!

http://keithpitty.com