Testing Tools for Programmers

(twill, scotch, figleaf, wsgi_intercept, and pinocchio)

Testing Tools for Programmers

(twill, scotch, figleaf, wsgi_intercept, and pinocchio)

C. Titus Browntitus@idyll.orgC. Titus Browntitus@idyll.org

Talk slides & code at ‘http://www.idyll.org/pycon07.zip’

This is a concept & demo talk.

I’m not selling anything.

I’m not giving a tutorial.

This stuff is useful and fun.

I have published the slides and demo code if you want to look into

anything.

This is a concept & demo talk.

I’m not selling anything.

I’m not giving a tutorial.

This stuff is useful and fun.

I have published the slides and demo code if you want to look into

anything.

Testing tools for programmers must be

simple, easy to deploy & use, and configurable. Otherwise people won’t

use them.

Testing tools for programmers must be

simple, easy to deploy & use, and configurable. Otherwise people won’t

use them.

Functional Web testing with twillFunctional Web testing with twill

go http://www.google.com/showformsformvalue 1 q “google query statistics”submitshow

forms, cookies, redirects, http basic auth, link following, link checking, http code assertions, test for text presence/nonpresence, tidy checking, etc.

no JavaScript support ;(

go http://www.google.com/showformsformvalue 1 q “google query statistics”submitshow

forms, cookies, redirects, http basic auth, link following, link checking, http code assertions, test for text presence/nonpresence, tidy checking, etc.

no JavaScript support ;(

twill is Pythontwill is Python

from twill.commands import *go(‘http://www.google.com/’)showforms()formvalue(‘1’, ‘q’, “google query statistics”)submit()show()

All base twill commands directly accessible from Python. Additionally, a “nice” wrapper around mechanize is available.

from twill.commands import *go(‘http://www.google.com/’)showforms()formvalue(‘1’, ‘q’, “google query statistics”)submit()show()

All base twill commands directly accessible from Python. Additionally, a “nice” wrapper around mechanize is available.

wsgi_intercept lets twill talk directly to WSGI apps.

wsgi_intercept lets twill talk directly to WSGI apps.

app = get_wsgi_app()

import twilltwill.add_wsgi_intercept(‘localhost’, 80, lambda: app)

twill.commands.go(‘http://localhost/’)…twill.remove_wsgi_intercept(‘http://localhost/’)

This is fairly close in concept to paste.fixture, but you can use the same script for testing a direct WSGI connection as you can for testing your whole “Web stack” (server + middleware + app).

(Will show you demo)

app = get_wsgi_app()

import twilltwill.add_wsgi_intercept(‘localhost’, 80, lambda: app)

twill.commands.go(‘http://localhost/’)…twill.remove_wsgi_intercept(‘http://localhost/’)

This is fairly close in concept to paste.fixture, but you can use the same script for testing a direct WSGI connection as you can for testing your whole “Web stack” (server + middleware + app).

(Will show you demo)

scotch lets you record & replay WSGI data, and generate twill

scripts too.

scotch lets you record & replay WSGI data, and generate twill

scripts too.app = get_wsgi_app()

import scotch.recorderrecorder = scotch.recorder.Recorder(app)serve_wsgi(recorder)…

app = get_wsgi_app()for record in recorder.record_holder:

print record.replay(app)

app = get_wsgi_app()

import scotch.recorderrecorder = scotch.recorder.Recorder(app)serve_wsgi(recorder)…

app = get_wsgi_app()for record in recorder.record_holder:

print record.replay(app)

figleaf is a code coverage recording tool.

figleaf is a code coverage recording tool.

import figleaffigleaf.start()…figleaf.stop()figleaf.write_coverage(‘.figleaf’)

Intended for programmatic use; can take coverage from multiple runs/deployments/platforms, and intersect/union results. Can retrieve from running Web server.

import figleaffigleaf.start()…figleaf.stop()figleaf.write_coverage(‘.figleaf’)

Intended for programmatic use; can take coverage from multiple runs/deployments/platforms, and intersect/union results. Can retrieve from running Web server.

Demos, part IDemos, part I

CherryPy:

1. Unit tests with twill are easy.

2. Unit tests with twill talking directly to WSGI are easier.

3. Some simple code coverage analysis.

4. Some not-so-simple code coverage analysis.

5. A twill extension to do simple “fuzz” testing.

Demos, part IIDemos, part II

Django:

1. You can also test Django apps.

2. Recording a new Django test with scotch.

3. Converting the Django test into a twill script.

4. Playing back the Django test.

Next stepsNext steps

twill/scotch/figleaf/pinocchio/wsgi_intercept release(much of this stuff is in “-latest”, not released)

Improve figleaf dramatically (only minimally usable ;)

Document.Document.Document.Document.

Now y’all test, y’here?

twill/scotch/figleaf/pinocchio/wsgi_intercept release(much of this stuff is in “-latest”, not released)

Improve figleaf dramatically (only minimally usable ;)

Document.Document.Document.Document.

Now y’all test, y’here?

ResourcesResources

Google “python” + package.

http://ivory.idyll.org/blog/

titus@idyll.org

New! ‘testing-in-python’ mailing list.