the evolution of development testing

The Evolution of Development Testing

Overview• Automating

• Testing Categories + Scope

• Development Testing

• Principles, Practices + Tool Support

Automating• We automate “stuff” for a living

• Developer/Programmer/Engineer/Automator

• When we’re doing things manually we’re usually missing an opportunity to automate

Automating

• validation of a web form

• reporting of a message parsing error

• calculation of an average execution price

• execution + status reporting of tests for all of the above

• The stuff we automate might be

Automating

• choose validation rules, implement, enforce, report

• log parsing error, automate monitoring/reporting of error

• define calculation rules, implement, provide result

• define inputs, context + expectations, execute scenario, determine + report status

• That automation might look like

Automating

• review user errors later in workflow e.g.“back office” manual activity

• message loss caused an incident, manually track it down

• export raw data to csv, load into Excel, apply formula

• run UI functional scenarios that use/execute the code in mind

• fire up debugger, step through mis-behaving code

• add detailed logging, monitor logs for hints

• Manual solutions might look like

Automating• We automate “stuff” for a living

• Developer/Programmer/Engineer/Automator

• When we’re doing things manually we’re usually missing an opportunity to automate

Test Categorisation• Useful concept - up to a point

• Blurred lines + Overloaded terminology

• Difficulties conversing about testing

Test Categorisation• Unit

• Component

• API

• Acceptance

• Integration

• Functional

• Build Acceptance

• System

• End-to-End

• User Acceptance

• Non-functional

Test Categorisation

Test Categorisation• Useful concept - up to a point

• Blurred lines + Overloaded terminology

• Difficulties conversing about testing

Test Categorisation

Development Testing• What it is

• What it should not be

• How it has evolved

What it is• A Development practice first and foremost

• White box / Black box / Grey box

• Has Quality aims

• Has Efficiency aims

• Test automation framework of choice for most Java developers

• Synonymous with Unit testing

• Widely used for testing beyond the Unit scope

• Ported to many languages - generically known as xUnit

JUnit

JUnit

“Never in the field of software development have so many owed so much to so few lines of code”

Martin Fowler

(Development big-wig)

What it is• A Development practice first and foremost

• White box / Black box / Grey box

• Has Quality aims

• Has Efficiency aims

What it is• Can take a technical perspective on scenarios

• Can take a user functionality perspective

• Exercises precise control over the test context

• Used + extended by Dev during initial and future coding

What it is• Often used to discover and refine solution design

• Often leaves fine-grained regression coverage safety net

• Enables instant feedback on failures due to future enhancements and refactorings

What it is not• Automated full System tests a.k.a E2E, System Integration

• Zero control over the state of the system (the test context)

• Test scope coupled to other external systems, reference data quality, env availability

• Heavily reliant on driving a UI to execute many fine-grained scenarios

Its Evolution• OOP - slow evolution

• Automated Developer Testing around a lot less time

• A lot less mainstream than OOP

• Growing body of knowledge / experience available

Principles, Practices + Tool Support• Structural Principles / Characteristics

• Qualities of a “good” test / spec

• Practices + Tools for improving test qualities

Test structure• Specify Inputs

• Specify the state of the System

• Specify the Event

• Specify the Expectations

• Arrange, Act, Assert

• Given, When, Then

Test structure• Naming conventions + structure

@Test public void methodUnderTest_GivenABC_ThenExpectXYZ() { // given // when // then }

@Test public void methodUnderTest_GivenInputsABC_AndSystemStateDEF_ThenExpectXYZ() { // given - inputs // given - system state! // when // then }

Test structure• IDE assistance for consistency + evolve conventions

“Any fool can write code that a computer can understand”

!

Martin Fowler

Test Qualities• Readability

• What is it? Why does it matter?

Test Qualities• Readability == …?

• Code that you can understand with ease

• Language affects Thought

• Thought affects Action

Test Qualities• Tools can directly address Readability qualities

• Domain Specific Languages = a fancy term

• JUnit, Hamcrest, FEST, JMock, Mockito, Gherkin

• Language -> Thought -> Action

“Test automation is a first class software engineering problem”

!

Brian Marick?

Test Qualities• Maintainability

• What is it? Why does it matter?

Test Qualities• Maintainability == …?

• To change, evolve with ease

• Keeping the cost of change down

Test Qualities• They can be context sensitive. Many are not.

• You may opt to sacrifice:

• DRY Principle for Readability

• Readability for Maintainability

• Maintainability for Obviousness

• Expect debate

Specifying Expectations• Stating Expectations / Asking Questions

• From Computer dialect ——> Human dialect

// then assertTrue(trader.getPermissions().isEmpty()); // Computer dialect // . assertThat(trader.getPermissions().isEmpty(), is(true)); // . // . assertThat(trader.getPermissions()).isEmpty(); // . // . assertThat(trader).hasNoPermissions(); // Human dialect

Specifying Expectations• Stating Expectations / Asking Questions

• From Computer dialect ——> Human dialect // Using Basic JUnit assertEquals(orders.size(), 2); Iterator<Order> itr = orders.iterator(); assertEquals(itr.next(), order3); assertEquals(itr.next(), order4); // Using Hamcrest assertThat(orders, hasItems(order3, order4)); assertThat(orders.size(), is(2)); // Using FEST Assertions assertThat(orders).containsOnly(order3, order4); // Using FEST Assertions (chained assertions) assertThat(orders).containsOnly(order3, order4) .containsSequence(order3, order4);

Specifying Expectations• Easier + Faster to write expressive statements

• Faster to read + review

• Easier to maintain

• Encourages other better practices

Specifying Inputs• Trivial example. Test with very narrow scope

• e.g.Input = Single Object

@Test public void getPermissions_GivenNewUser_ThenReturnsNoPermissions() { // given Trader trader = new Trader(); // when + then assertThat(trader.getPermissions()).isEmpty(); }

Specifying Inputs• Non-trivial example. Test with broader scope

• e.g. Input = Object Graph

• i.e. objects linked to objects

@Test public void getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v1() { // given - inputs Trader trader = new Trader(); TradingAccount tradingAcct = new TradingAccount(new TradingFirm("TraderzRUs", "TRDRZ"), new ClearingFirm("TooBig2Fail", "2BIG2F"), ACTIVE); trader.setPermissions(Arrays.asList(new Permission(tradingAcct))); String goldIsinCode = "COF24680"; // given - system state Future oil3MnthFuture = new Future("OIL.3MNTH", new ISIN("OIL3M0123")); Future oil6MnthFuture = new Future("OIL.6MNTH", new ISIN("OIL6M0456")); Future goldFuture = new Future("GLD.3MNTH", new ISIN(goldIsinCode)); String orderId = "ordId"; Order order1 = new Order(orderId + 1, tradingAcct, oil3MnthFuture, 1000, 2500); Order order2 = new Order(orderId + 2, tradingAcct, oil6MnthFuture, 1500, 2600); Order order3 = new Order(orderId + 3, tradingAcct, goldFuture, 200, 4300); Order order4 = new Order(orderId + 4, tradingAcct, goldFuture, 150, 4300); OrdersDAO orderDAO = new OrdersDAOInMemory(order1, order2, order3, order4); OrderSearchService orderSearchService = new OrderSearchServiceImpl(orderDAO); // when List<Order> orders = orderSearchService.getOrders(trader, tradingAcct, goldIsinCode); // then assertThat(orders).containsOnly(order3, order4); }

Specifying Inputs• Consider many non-trivial tests, with “raw” setup

• Impact of changing one setter or constructor

• Quantity of code there is to read, understand, change

Specifying Inputs - Techniques• Helper methods

• Object Mother Pattern

• Test Data Builder Pattern

@Test public void // Using helpers getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v2() { // given - inputs String accountCode = "TRDRZ"; String goldIsinCode = "GLD24680"; TradingAccount tradingAcct = createTradingAccount(accountCode); Trader trader = createTraderWithPermissionsFor(tradingAcct); // given - system state Future oil3MnthFuture = createFuture("OIL.3MNTH", "OIL3M0123"); Future oil6MnthFuture = createFuture("OIL.6MNTH", "OIL6M0456"); Future goldFuture = createFuture("GLD.3MNTH", goldIsinCode); Order order1 = createOrder(accountCode, oil3MnthFuture, qty(1000), price(2500)); Order order2 = createOrder(accountCode, oil6MnthFuture, qty(1500), price(2600)); Order order3 = createOrder(accountCode, goldFuture, qty(200), price(4300)); Order order4 = createOrder(accountCode, goldFuture, qty(150), price(4300)); OrderSearchService orderSearchService = createOrderService( createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = orderSearchService.getOrders(trader, accountCode, goldIsinCode); // then assertThat(orders).containsOnly(order3, order4); }

@Test public void // Using helpers + hiding irrelevant details getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v3() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", dummyFuture()), order2 = createOrder("TRDRZ", dummyFuture()), order3 = createOrder("TRDRZ", createFuture("GLD24680")), order4 = createOrder("TRDRZ", createFuture("GLD24680")); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order3, order4); }

@Test public void // Using helpers getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v2() { // given - inputs String accountCode = "TRDRZ"; String goldIsinCode = "GLD24680"; TradingAccount tradingAcct = createTradingAccount(accountCode); Trader trader = createTraderWithPermissionsFor(tradingAcct); // given - system state Future oil3MnthFuture = createFuture("OIL.3MNTH", "OIL3M0123"); Future oil6MnthFuture = createFuture("OIL.6MNTH", "OIL6M0456"); Future goldFuture = createFuture("GLD.3MNTH", goldIsinCode); Order order1 = createOrder(accountCode, oil3MnthFuture, qty(1000), price(2500)); Order order2 = createOrder(accountCode, oil6MnthFuture, qty(1500), price(2600)); Order order3 = createOrder(accountCode, goldFuture, qty(200), price(4300)); Order order4 = createOrder(accountCode, goldFuture, qty(150), price(4300)); OrderSearchService orderSearchService = createOrderService( createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = orderSearchService.getOrders(trader, accountCode, goldIsinCode); // then assertThat(orders).containsOnly(order3, order4); }

Specifying Inputs - Techniques• Object Mother Pattern

public class ObjectMother {! public static OrdersDAOInMemory createOrdersDAO(Order order1, Order order2, Order order3, Order order4) { return new OrdersDAOInMemory(order1, order2, order3, order4); }! public static OrderSearchService createOrderService(OrdersDAO ordersDAO) { OrderSearchService orderSearchService = new OrderSearchServiceImpl(ordersDAO); return orderSearchService; } private OrderSearchService prepareOrderService(Order order1, Order order2, Order order3, Order order4) { OrdersDAO orderDAO = createOrdersDAO(order1, order2, order3, order4); OrderSearchService orderSearchService = new OrderSearchServiceImpl(orderDAO); return orderSearchService; }! public static Order createOrder(String accountCode, Future dummyFuture) { return createOrder(accountCode, dummyFuture, qty(0), price(0)); }! public static Order createOrder(String accountCode, Future future, int qty, int price) { return new Order(nextOrderId(), accountCode, future, qty, price); } private Order createOrder(String orderId, TradingAccount tradingAcct, Future future, int qty, int price) { return new Order(orderId, tradingAcct.getTradingFirm().getCode(), future, qty, price); }! public static Future createFuture(String desc, String isinCode) { return new Future(desc, new ISIN(isinCode)); } public static Future createFuture(String isinCode) { return createFuture(DEFAULT_FUTURE_DESC, isinCode); }! public static Future dummyFuture() { return createFuture(DEFAULT_FUTURE_DESC, DEFAULT_ISIN_CODE); }! public static TradingAccount createTradingAccount(String accountCode) {

Specifying Inputs - Techniques• Test Data Builder Pattern

@Test public void // Using Test Data Builders getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v4() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = aTrader().with(aPermission() .with(aTradingAccount() .with(aTradingFirm().withCode("TRDRZ")))).build(); // given - system state OrderBuilder matchingOrder = anOrder().withAccountCode("TRDRZ") .with(aFuture().with(anISIN().withIsinCode("GLD24680")).build()); Order order1 = anOrder().withAccountCode("TRDRZ").build(), order2 = anOrder().withAccountCode("TRDRZ").build(), order3 = matchingOrder.build(), order4 = matchingOrder.build(); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order3, order4); }

public class TraderBuilder { public static String DEFAULT_USERNAME = "trader1"; public static UserDetail DEFAULT_USERDETAIL = new UserDetail(); public static List<Permission> NO_PERMISSIONS = Collections.emptyList(); public static List<Restriction> NO_RESTRICTIONS = Collections.emptyList();! private String userName = DEFAULT_USERNAME; private UserDetail userDetail = DEFAULT_USERDETAIL; private List<Permission> permissions = NO_PERMISSIONS; private List<Restriction> restrictions = NO_RESTRICTIONS; private TraderBuilder() { } public static TraderBuilder aTrader() { return new TraderBuilder(); } public Trader build() { Trader trader = new Trader(); trader.setUserName(userName); trader.setUserDetail(userDetail); trader.setPermissions(permissions); trader.setRestrictions(restrictions); return trader; } public TraderBuilder withUserName(String userName) { this.userName = userName; return this; }! public TraderBuilder with(UserDetail userDetail) { this.userDetail = userDetail; return this; } public TraderBuilder withPermissions(List<Permission> permissions) { this.permissions = new ArrayList<>(permissions); return this; }

public TraderBuilder with(PermissionBuilder permission) { this.permissions = new ArrayList<>(Arrays.asList(permission.build())); return this; } public TraderBuilder withNoPermissions() { this.permissions = Collections.emptyList(); return this; }! public TraderBuilder with(RestrictionBuilder restriction) { this.restrictions = new ArrayList<>(Arrays.asList(restriction.build())); return this; } public TraderBuilder withRestrictions(List<Restriction> restrictions) { this.restrictions = new ArrayList<>(restrictions); return this; } public TraderBuilder but() { return new TraderBuilder() .withUserName(userName) .with(userDetail) .withPermissions(permissions) .withRestrictions(restrictions); }}

@Test public void // Using Test Data Builders getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v4() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = aTrader().with(aPermission() .with(aTradingAccount() .with(aTradingFirm().withCode("TRDRZ")))).build(); // given - system state OrderBuilder matchingOrder = anOrder().withAccountCode("TRDRZ") .with(aFuture().with(anISIN().withIsinCode("GLD24680")).build()); Order order1 = anOrder().withAccountCode("TRDRZ").build(), order2 = anOrder().withAccountCode("TRDRZ").build(), order3 = matchingOrder.build(), order4 = matchingOrder.build(); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order3, order4); }

@Test public void // Using Test Data Builders to create input data variations getOrders_GivenTraderWithVariousPermissions_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v5() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; TradingAccountBuilder tradingAccount = aTradingAccount().with(aTradingFirm().withCode("TRDRZ")); Trader traderWithNoPerms = aTrader().withNoPermissions().build(); Trader traderWithPerms = aTrader().with(aPermission().with(tradingAccount)).build(); Trader traderWithInactivePerms = aTrader().with(aPermission().with( tradingAccount.but().isInActive())).build(); // given - system state OrderBuilder matchingOrder = anOrder().withAccountCode("TRDRZ") .with(aFuture().with(anISIN().withIsinCode("GLD24680")).build()); Order order1 = anOrder().withAccountCode("TRDRZ").build(), order2 = anOrder().withAccountCode("TRDRZ").build(), order3 = matchingOrder.build(), order4 = matchingOrder.build(); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders1 = service.getOrders(traderWithNoPerms, searchAccount, searchIsin); List<Order> orders2 = service.getOrders(traderWithPerms, searchAccount, searchIsin); List<Order> orders3 = service.getOrders(traderWithInactivePerms, searchAccount, searchIsin); // then assertThat(orders1).isEmpty(); assertThat(orders2).containsOnly(order3, order4); assertThat(orders3).isEmpty(); }

Specifying Inputs - Techniques• Test Data Builder Pattern - with tool support

• Make-It-Easy - addresses Builder class boiler-plate

• Model Citizen - annotation-based “Blueprints”

Specifying Inputs - Techniques• Test Data Builder Pattern - Advantages

• Your test / spec documents only the inputs that matter

• Decouples your test from being impacted by a wide range of changes

• Supports “declarative” code style + useful templates

• user = EXPIRED_USER.but( with ( aBalance, 100));

Specifying Inputs - Techniques• Test Data Builder Pattern - Disadvantages

• Requires much new test-supporting code

• Chained methods take getting used to

Specifying the System State• State of the SUT / the Test Context

• Communicating what the context, limitations and boundaries are

• Isolating your test from certain dependencies / interactions

• Use of Test Doubles

• Hand-rolled Test Double

• common misnomer Mocking Frameworks

Specifying the System State• Problems + solutions for Specifying Inputs normally apply

• Unique to System State is your Test Isolation requirements

Specifying the System State - Techniques• Hand-rolling Test Doubles

• Using Test Isolation frameworks to provide Test Doubles

• “Mocking” + “Mocks Aren’t Stubs” http://martinfowler.com/articles/mocksArentStubs.html

• Dummy / Fake / Stubs / Mocks / Spies

Specifying the System State - Techniques• Stub being used in a state-based test

@Test public void // Stubbing system state (Mockito) + using helpers + hiding irrelevant details getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v2() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", createFuture("GLD24680")), order2 = createOrder("TRDRZ", createFuture("GLD24680")); OrdersDAO ordersDAOStub = mock(OrdersDAO.class); given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2)); OrderSearchService service = createOrderService(ordersDAOStub); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order1, order2); }

@Test public void // Stubbing system state + stubbing inputs (Mockito) getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v3() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; // Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); Trader traderStub = mock(Trader.class); Permission permissionStub = mock(Permission.class); given(traderStub.getPermissions()).willReturn(Arrays.asList(permissionStub)); TradingAccount tradingAccountStub = mock(TradingAccount.class); given(permissionStub.getTradingAccount()).willReturn(tradingAccountStub); given(tradingAccountStub.isActive()).willReturn(Boolean.TRUE); given(tradingAccountStub.getTradingFirm()).willReturn(new TradingFirm("", searchAccount)); // given - system state Order order1 = createOrder("TRDRZ", createFuture("GLD24680")), order2 = createOrder("TRDRZ", createFuture("GLD24680")); OrdersDAO ordersDAOStub = mock(OrdersDAO.class); given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2)); OrderSearchService service = createOrderService(ordersDAOStub); // when List<Order> orders = service.getOrders(traderStub, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order1, order2); }

@Test public void // Stubbing system state + stubbing input (Mockito w/ Deep Stubs) getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v3_1() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; // Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); Trader traderStub = mock(Trader.class); Permission permissionStub = mock(Permission.class, RETURNS_DEEP_STUBS); given(traderStub.getPermissions()).willReturn(Arrays.asList(permissionStub)); given(permissionStub.getTradingAccount().isActive()).willReturn(Boolean.TRUE); given(permissionStub.getTradingAccount().getTradingFirm().getCode()).willReturn(searchAccount); // given - system state Order order1 = createOrder("TRDRZ", createFuture("GLD24680")), order2 = createOrder("TRDRZ", createFuture("GLD24680")); OrdersDAO ordersDAOStub = mock(OrdersDAO.class); given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2)); OrderSearchService service = createOrderService(ordersDAOStub); // when List<Order> orders = service.getOrders(traderStub, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order1, order2); }

Specifying the System State - Techniques• A Mock being used in a interaction-based test

@Test public void // Using a Mock to test interactions + Stubbing system state (Mockito) getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v4_1() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", createFuture("GLD24680")), order2 = createOrder("TRDRZ", createFuture("GLD24680")); OrdersDAO ordersDAOStub = mock(OrdersDAO.class); given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2)); PermissionService permissionServiceMock = mock(PermissionService.class); OrderSearchService service = createOrderServiceV2(ordersDAOStub, permissionServiceMock); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then verify(permissionServiceMock).hasViewPermissions(trader, order1); verify(permissionServiceMock).hasViewPermissions(trader, order2); }

@Test public void // Using a Mock to test interactions + Stubbing system state (Mockito) getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v4_2() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", createFuture("GLD24680")), order2 = createOrder("TRDRZ", createFuture("GLD24680")); OrdersDAO ordersDAOStub = mock(OrdersDAO.class); given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2)); PermissionService permissionServiceMock = mock(PermissionService.class); given(permissionServiceMock.hasViewPermissions( any(Trader.class), any(Order.class))).willReturn(Boolean.TRUE); OrderSearchService service = createOrderServiceV2(ordersDAOStub, permissionServiceMock); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then verify(permissionServiceMock).hasViewPermissions(trader, order1); verify(permissionServiceMock).hasViewPermissions(trader, order2); assertThat(orders).containsOnly(order1, order2); }

Specifying the System State - Techniques• Often a test will have

• Zero to many Stubs. Zero or one Mock

• Mocks help you answer questions about what happened

• When a Mock is present, you ask it something at the end

• A Stub should not be asserted against

• There to help create + control the System State for the test

Specifying the Event• Often trivial

• Might be a sequence of events that we want to fire

• Sequence might actually be context / SUT setup

• Practices? Tools?

Specifying it All Together

@Test public void helloWorld() throws Exception { mockMvc.perform(get("/hello").accept(MediaType.TEXT_PLAIN)) .andDo(print()) // print the request/response in the console .andExpect(status().isOk()) .andExpect(content().contentType(MediaType.TEXT_PLAIN)) .andExpect(content().string("Hello World!")); }

• Spring MVC Test support / DSL

• Continuous Integration - development practice

• CI tools - run automated tests frequently

• Continuous Testing - development practice

• CT tools - run automated tests locally, frequently

• e.g. Infinitest, JUnitMax

Continuous Testing

• Tools/APIs just help with heavy lifting

• Sound coding principles are paramount

• Creating unreadable and unmaintainable test, even with “the right” tools

• Unreadable + high maintenance tests will hinder efficiency

Summary

Summary• The power of marginal improvements

• for … each, <> operator, closure support, Guava

• Continuous iterative improvements nudge us forward

• Continuous iterative improvements push down cost of reading, writing, evolving tests

• Drives the cost down to “mass market” level

• Beyond Java

Resources• Books

• GOOS - Growing Object Oriented Software, Guided By Tests

• Effective Unit Testing

• Working Effectively with Legacy Code

• xUnit Test Patterns - Refactoring Test Code

http://www.amazon.co.uk/dp/B00DJG3VW6

http://www.amazon.co.uk/dp/1935182579

http://www.amazon.co.uk/dp/0131177052

http://www.amazon.co.uk/dp/B00132S6V4

Resources• Podcasts

• Sustainable Test Driven Development (Series)

• Hanselminutes - Roy Asherove - The Art of Unit Testing

• Hanselminutes - Quetzal Bradley - Beyond Unit Testing

• All Code Examples are available at

• http://github.com/cathalking/devtestevolution

http://www.sustainabletdd.com/p/podcasts.html

http://www.hanselminutes.com/169/the-art-of-unit-testing-with-roy-osherove

http://www.hanselminutes.com/103/quetzal-bradley-on-testing-after-unit-tests

http://github.com/cathalking/devtestevolution

the evolution of development testing

Technology

test contextused

test qualities readability

test qualities maintainability

test qualities tools

good test specpractices

test contexttest scope

test structure ide assistance

development practice