testing for pragmatic people

Michael DavisDevelopment LeaderPlone Conference5th November 2011

Testing for Pragmatic People

Theory

• Code without tests is broken code• 100% test coverage is a minimum• God kills a kitten every time someone checks in code

without a test

• Testing will make you more attractive to the opposite sex

• There's a special place in hell for people who code without tests

Reality

• Testing is hard• Expensive• Slow to run

• Maintaining

• Test framework keeps changing• Refactoring• Tests wrong

Confused Much?

http://www.fl ickr.com/photos/39585662@N00/5331407243

Realism

• Tests are not mandatory

• However, they are ...• Advocated• Best practice

• Helpful

• Eminently sensible

Testing is atool

Cost/Value

Value of TestCost of Test

Testing Experience

What do we meanby Tests

• Unit tests• Integration tests• Mock tests

• Doc tests

• Regression tests

Unit tests

• Much faster than integration tests• For testing a small element of the code base• Testing individual methods

• getToolByName is always a problem

• Get complex quickly• Expensive to maintain

Doc tests

• Primarily for describing functionality using python calls

• Fails at first failure

• Difficult to write

• Good for documentation

Regression Tests

• Not actually a test• About when and why

Mock Testing

• Using mock objects to run tests• Useful in unit testing to mock required objects• Can mock getToolByName to return the right tool

• Used to test the untestable

Integration Tests

• Often called unit tests• Full Plone instance setup

What about Test Driven

Development

Mindsets

• Writing code - how• Writing test – what• Effects versus Mechanics

Keep It Simple

• Minimise effort• Maximise reuse

Test Framework

class TestCase(PloneSandboxLayer):

defaultBases = (PLONE_FIXTURE,)

def setUpZope(self, app, confgurationContext):

import my.product

self.loadZCML(package=my.product)

z2.installProduct(app, PROJECTNAME)

def setUpPloneSite(self, portal):

self.applyProfle(portal, '%s:default' % PROJECTNAME)

def tearDownZope(self, app):

z2.uninstallProduct(app, PROJECTNAME)

Test Framework

from plone.app.testing import PloneSandboxLayer

from plone.app.testing import PLONE_FIXTURE

from plone.app.testing import IntegrationTesting

from plone.testing import z2

from my.product.confg import PROJECTNAME

class TestCase(PloneSandboxLayer):

....

FIXTURE = TestCase()

INTEGRATION_TESTING = IntegrationTesting(bases=(FIXTURE,), name="fxture:Integration")

Simple Test

class TestReinstall(unittest.TestCase):

"""Ensure product can be reinstalled safely"""

layer = INTEGRATION_TESTING

def setUp(self):

self.portal = self.layer['portal']

def testReinstall(self):

portal_setup = getToolByName(self.portal, 'portal_setup')

portal_setup.runAllImportStepsFromProfle('profle-%s:default' % PROJECTNAME)

Running Tests

bin/test -s my.product

Ran 2 tests with 0 failures and 0 errors in 1.050 seconds.

Tearing down left over layers:

Tear down collective.wfform.tests.base.fixture:Integration in 0.000 seconds.

Tear down collective.wfform.tests.base.TestCase in 0.013 seconds.

Tear down plone.app.testing.layers.PloneFixture in 0.380 seconds.

Tear down plone.testing.z2.Startup in 0.024 seconds.

Tear down plone.testing.zca.LayerCleanup in 0.006 seconds.

Browser Layer

class TestInstallation(unittest.TestCase):

"""Ensure product is properly installed"""


def setUp(self):


def testBrowserLayerRegistered(self):

sm = getSiteManager(self.portal)

layers = [o.__name__ for o in registered_layers()]

assert 'IMyProduct' in layers

Skin Layer




def setUp(self):


def testSkinLayersInstalled(self):

assert 'my_skin' in self.portal.portal_skins.objectIds()

assert 'my_template' in self.portal.portal_skins.my_skin.objectIds()

Portal Type




def setUp(self):


def testTypesInstalled(self):

portal_types = getToolByName(self.portal, 'portal_types')

assert 'MyType' in portal_types.objectIds(), portal_types.objectIds()

Portal Types

class TestContentType(unittest.TestCase):

"""Test content type"""


def setUp(self):


def testAddType(self):

setRoles(self.portal, TEST_USER_ID, ['Manager'])

self.portal.invokeFactory('MyType', 'mt1')

mt1 = getattr(self.portal, 'mt1')

mt1.setTitle('The Title of My Object')

Assert mt1.Title() == 'The Title of My Object', mt1.Title()

Schema

class TestContentSchema(unittest.TestCase):

"""Test content type schema"""


def setUp(self):




self.mt1 = getattr(self.portal, 'mt1')

def testSchema(self):

schema = self.mt1.schema

feld_ids = schema.keys()

assert 'referenceNumber' in feld_ids

Form Validation

class TestValidation(unittest.TestCase):

"""Test validation"""


def setUp(self):




self.mt1 = getattr(self.portal, 'mt1')

app = makerequest(self.app)

app.REQUEST.form['name'] = 'Michael Davis'

Validation Test

def testValidates(self):

dummy_controller_state = ControllerState(

id='my_form',

context=s1,

button='submit',

status='success',

errors={},

next_action=None,)

controller = self.portal.portal_form_controller object

controller_state = controller.validate(dummy_controller_state, app.REQUEST, ['my_validate',])

assert controller_state.getErrors() == {}, controller_state.getErrors()

Test Patterns

def testValidatePrisoner(self):

app.REQUEST.form['name'] = 6

…

assert len(controller_state.getErrors()) == 1

assert controller_state.getErrors().has_key('name')

assert controller_state.getErrors()['name'] == 'You are not a number'

Test Patterns

def testValidateClint(self):

app.REQUEST.form['name'] = ''

…

assert len(controller_state.getErrors()) == 1

assert controller_state.getErrors().has_key('name')

assert controller_state.getErrors()['name'] == 'You are not the man with no name'

CI Server

Use your tools responsibly

http://www.fl ickr.com/photos/12803689@N02/4402962654

testing for pragmatic people

Technology

setup portal

test testing experience

mock testing

unit testing

product self

reality testing

testcase integration

testing import z2