automated testing with ruby

120
Automated Testing with Ruby Keith Pitty Open Source Developers’ Conference, Sydney 2008 4th December

Upload: keith-pitty

Post on 11-May-2015

14.211 views

Category:

Technology


1 download

DESCRIPTION

Slides for talk presented at OSDC 2008 in Sydney.

TRANSCRIPT

Page 1: Automated Testing with Ruby

Automated Testingwith Ruby

Keith Pitty

Open Source Developers’ Conference, Sydney 20084th December

Page 2: Automated Testing with Ruby

Once upon a time...

Page 3: Automated Testing with Ruby

way back in 1983...

Page 4: Automated Testing with Ruby
Page 5: Automated Testing with Ruby

a young programmer made his wayfrom the world

of Unix at university...

Page 6: Automated Testing with Ruby
Page 7: Automated Testing with Ruby

to the world ofIBM mainframe programming.

Page 8: Automated Testing with Ruby
Page 9: Automated Testing with Ruby

He learnt to use suchweird and wonderful

technologies as...

Page 10: Automated Testing with Ruby

PL/I, IMS, CICS, TSO and ISPF/PDF.

Page 11: Automated Testing with Ruby

Fortunately he had a good mentorwho taught him about

modular programming...

Page 12: Automated Testing with Ruby

and how to write unit tests with...

Page 13: Automated Testing with Ruby

Job Control Language

Page 14: Automated Testing with Ruby

“JCL is the stuff of nightmares for many programmers and operators.”

From editorial review for “MVS JCL in Plain English”

Page 15: Automated Testing with Ruby

fast forward to 2001

Page 16: Automated Testing with Ruby

Our programmer was not so young any more...

Page 17: Automated Testing with Ruby

but he had moved on from mainframe progamming...

Page 18: Automated Testing with Ruby

to the exciting new world of Java...

Page 19: Automated Testing with Ruby

and...

Page 20: Automated Testing with Ruby

as a byproduct ofeXtreme Programming...

Page 21: Automated Testing with Ruby

he had fallen in love with...

Page 22: Automated Testing with Ruby

JUnit

Page 23: Automated Testing with Ruby

He loved using JUnit to do test-driven development

and refactoring...

Page 24: Automated Testing with Ruby

and encouraged others to use itwhenever he had the opportunity.

Page 25: Automated Testing with Ruby

fast forward to 2005

Page 26: Automated Testing with Ruby

By now our hero hadbegun to explore...

Page 27: Automated Testing with Ruby
Page 28: Automated Testing with Ruby

and...

Page 29: Automated Testing with Ruby
Page 30: Automated Testing with Ruby

He liked the way tests were “baked in” and how you could run them with rake.

Page 31: Automated Testing with Ruby

Then, one day in 2007,a fellow Rails programmer

enquired...

Page 32: Automated Testing with Ruby

“Are you using autotest?”

Page 33: Automated Testing with Ruby

Sheepishly he admittedhe hadn’t even heard of it...

Page 34: Automated Testing with Ruby

We’ll leave our story there for now.

Page 35: Automated Testing with Ruby

Autotest

Page 36: Automated Testing with Ruby

“a continuous testing facility meant to be used during development.”

Page 37: Automated Testing with Ruby

sudo gem install ZenTest

cd path/to/railsapp

autotest

Page 38: Automated Testing with Ruby

“As soon as you save a file, autotest will run the

corresponding dependent tests.”

Page 39: Automated Testing with Ruby

Test::Unit

Page 40: Automated Testing with Ruby

an implementation of xUnit

Page 41: Automated Testing with Ruby

the default Ruby testing tool

Page 42: Automated Testing with Ruby

distributed with Ruby since 2001

Page 43: Automated Testing with Ruby

also the default testing tool for Rails

Page 44: Automated Testing with Ruby

assert_kind_of BowlingAnalysis, @bowling_analysisassert_equal "Brett Lee", @bowling_analysis.player.full_nameassert_equal "9.3", @bowling_analysis.overs.to_sassert_equal 1, @bowling_analysis.maidensassert_equal 2, @bowling_analysis.wicketsassert_equal 50, @bowling_analysis.runs

Page 45: Automated Testing with Ruby

Rails by default provides unit, functional and integration tests

Page 46: Automated Testing with Ruby

RSpec

Page 47: Automated Testing with Ruby
Page 48: Automated Testing with Ruby

RSpec allows you to specify behaviour

Page 49: Automated Testing with Ruby

Specifying a model

Page 50: Automated Testing with Ruby

describe Player do before(:each) do @valid_attributes = { :first_name => "value for first_name", :last_name => "value for last_name", :active => true } end

it "should create a new instance given valid attributes" do Player.create!(@valid_attributes) endend

Page 51: Automated Testing with Ruby

Specifying a controller

Page 52: Automated Testing with Ruby

describe PlayersController do # missing code here will be revealed soon it "should respond successfully to get 'index' with list of players" do Player.should_receive(:find).and_return(@players) get 'index' response.should be_success assigns[:players].should == @players endend

Page 53: Automated Testing with Ruby

What makes a good test?

Page 54: Automated Testing with Ruby
Page 55: Automated Testing with Ruby

1. Isolated

2. Automated

3. Quick to write

4. Quick to run

5. Unique

Kent Beck suggests:

Page 56: Automated Testing with Ruby

describe PlayersController do before(:each) do @player_1 = mock(Player, {:first_name => "Greg", :last_name => "Norman", :active => true}) @player_2 = mock(Player, {:first_name => "Adam", :last_name => "Scott", :active => true}) @players = [@player_1, @player_2] end it "should respond successfully to get 'index' with list of players" do Player.should_receive(:find).and_return(@players) get 'index' response.should be_success assigns[:players].should == @players endend

Page 57: Automated Testing with Ruby

Mocks

Page 58: Automated Testing with Ruby

Mocks mimic real objects

Page 59: Automated Testing with Ruby

Mocks allow tests to be isolated

Page 60: Automated Testing with Ruby

For more see “Mock Dialogue”by Jim Weirich and Joe O’Brien

Page 61: Automated Testing with Ruby

Other Mock Frameworks

Page 62: Automated Testing with Ruby

mocha

flexmock

rr

Page 63: Automated Testing with Ruby

Autospec

Page 64: Automated Testing with Ruby

Autospec rather than autotestsince RSpec 1.1.5

Page 65: Automated Testing with Ruby

More on RSpec

Page 66: Automated Testing with Ruby

Can also specify viewsor

integrate views with controller specs

Page 67: Automated Testing with Ruby
Page 68: Automated Testing with Ruby

Integration Testing

Page 69: Automated Testing with Ruby

Can use Test::Unitbut I prefer...

Page 70: Automated Testing with Ruby

Cucumber

Page 71: Automated Testing with Ruby
Page 72: Automated Testing with Ruby

A replacement for RSpec Story Runner

Page 73: Automated Testing with Ruby

based on definition of scenarios within features

Page 74: Automated Testing with Ruby

Feature: Name of feature In order to derive some business value As a user I want to take certain actions Scenario: Name of scenario Given a prerequisite When I take some specific action Then I should notice the expected outcome

Page 75: Automated Testing with Ruby

cucumber

or

AUTOFEATURE=true autospec

Page 76: Automated Testing with Ruby

can be used in conjunction with tools likeWebrat and Selenium

Page 77: Automated Testing with Ruby

Webrat

Page 78: Automated Testing with Ruby

simulates in-browser testingby leveraging the DOMwithout performance hit

Page 79: Automated Testing with Ruby

Feature: Player belongs to group In order for a player to be included As a recorder I want to record when a player joins or leaves Scenario: Record when a new player joins Given a player is not in the system When I request a list of active players And I follow "Add Player" And I fill in "player_first_name" with "Greg" And I fill in "player_last_name" with "Norman" And I press "Save Player" Then I should see "Active Players" And I should see "Greg Norman"

Page 80: Automated Testing with Ruby

makes use of Cucumber step definitions for Webrat, for example...

Page 81: Automated Testing with Ruby

When /^I follow "(.*)"$/ do |link| clicks_link(link)end

When /^I fill in "(.*)" with "(.*)"$/ do |field, value| fills_in(field, :with => value) end

When /^I press "(.*)"$/ do |button| clicks_button(button)end

Then /^I should see "(.*)"$/ do |text| response.body.should =~ /#{text}/mend

Page 82: Automated Testing with Ruby

Selenium

Page 83: Automated Testing with Ruby

• runs tests via a browser

• handles JavaScript

Page 84: Automated Testing with Ruby

a little more effort thanjust installing the Selenium gem

Page 85: Automated Testing with Ruby

Feature: Check OSDC Site In order to keep track of conference details A participant Should be able to browse the OSDC site Scenario: Check the home page Given I am on the OSDC home page Then I should see the text "Welcome to the Open Source Developers' Conference"

Page 86: Automated Testing with Ruby

require 'spec'require 'selenium'

Before do @browser = Selenium::SeleniumDriver.new("localhost", 4444, "*chrome", "http://localhost", 15000) @browser.startend

After do @browser.stopend

Given /I am on the OSDC home page/ do @browser.open 'http://www.osdc.com.au/'end

Then /I should see the text "(.*)"/ do |text| @browser.is_text_present(text).should == trueend

Page 87: Automated Testing with Ruby

Fixtures

Page 88: Automated Testing with Ruby

allow data for automated tests to be defined using yaml

Page 89: Automated Testing with Ruby

can be a pain to maintain

Page 90: Automated Testing with Ruby

Machinist

Page 91: Automated Testing with Ruby

Blueprints to generate ActiveRecord objects

Page 92: Automated Testing with Ruby

Hole.blueprint do number 1end

(1..18).each {|n| Hole.make(:number => n) }

Page 93: Automated Testing with Ruby

Machinist with Sham

Page 94: Automated Testing with Ruby

Sham.name { Faker::Name.name }

Sham.date do Date.civil((2005..2009).to_a.rand, (1..12).to_a.rand, (1..28).to_a.rand)end

Player.blueprint do first_name { Sham.name } last_name { Sham.name } active trueend

Round.blueprint do date_played { Sham.date }end

Page 95: Automated Testing with Ruby

use blueprints in cucumber steps

Page 96: Automated Testing with Ruby

now available as a gem on github

Page 97: Automated Testing with Ruby

Other Tools

Page 98: Automated Testing with Ruby

JavaScript Unit Testing

Page 99: Automated Testing with Ruby

Rails plugins:

javascript_testjavascript_test_autotest

Page 100: Automated Testing with Ruby

js_autotest

Page 101: Automated Testing with Ruby

see Dr Nic’s blog post

Page 102: Automated Testing with Ruby

Shoulda

Page 103: Automated Testing with Ruby

Extends Test::Unit with the idea of contexts

Page 104: Automated Testing with Ruby

Testing Philosophy

Page 105: Automated Testing with Ruby

How much effort is warranted?

Page 106: Automated Testing with Ruby

Is it taking too much effort

to learn this new testing tool?

Page 107: Automated Testing with Ruby

My Opinion

Page 108: Automated Testing with Ruby

Automated testing can be viewed as a trade-off.

Page 109: Automated Testing with Ruby

Effort invested in automated tests should result in at least

an equivalent improvement in software quality.

E <= ΔQ

Page 110: Automated Testing with Ruby

I think it is worth striving to continually improve my command

of automated testing tools...

Page 111: Automated Testing with Ruby

always remembering the business context.

Page 112: Automated Testing with Ruby

Quotes

Page 113: Automated Testing with Ruby

“Producing successful, reliable software involves mixing and matching

an all-too-variable number of error removal approaches,

typically the more of them the better.”

Robert L. GlassFacts and Fallacies of Software Engineering

Page 114: Automated Testing with Ruby

“Whether you are a hard-core ‘test first’ person is not the point. The point is that everyone

can benefit from tests that are automated, easy to write,

and easy to run.”

Hal FultonThe Ruby Way

Page 115: Automated Testing with Ruby

References

Page 116: Automated Testing with Ruby

• Hal Fulton. The Ruby Way. Addison-Wesley, 2007

• Obie Fernandez. The Rails Way. Addison-Wesley, 2008

• David Chelimsky et al. The RSpec Book: Behaviour Driven Development with Ruby. The Pragmatic Programmers, 2009

• Robert L. Glass. Facts and Fallacies of Software Engineering. Addison-Wesley, 2003

Page 120: Automated Testing with Ruby

Thanks for listening!

http://keithpitty.com