Download - The Evolution of Development Testing
The Evolution of Development Testing
Overview• Automating
• Testing Categories + Scope
• Development Testing
• Principles, Practices + Tool Support
Overview• Automating
Automating• We automate “stuff” for a living
• Developer/Programmer/Engineer/Automator
• When we’re doing things manually we’re usually missing an opportunity to automate
Automating
• validation of a web form
• reporting of a message parsing error
• calculation of an average execution price
• execution + status reporting of tests for all of the above
• The stuff we automate might be
Automating
• choose validation rules, implement, enforce, report
• log parsing error, automate monitoring/reporting of error
• define calculation rules, implement, provide result
• define inputs, context + expectations, execute scenario, determine + report status
• That automation might look like
Automating
• review user errors later in workflow e.g.“back office” manual activity
• message loss caused an incident, manually track it down
• export raw data to csv, load into Excel, apply formula
• run UI functional scenarios that use/execute the code in mind
• fire up debugger, step through mis-behaving code
• add detailed logging, monitor logs for hints
• Manual solutions might look like
Automating• We automate “stuff” for a living
• Developer/Programmer/Engineer/Automator
• When we’re doing things manually we’re usually missing an opportunity to automate
Overview• Automating
• Testing Categories + Scope
Test Categorisation• Useful concept - up to a point
• Blurred lines + Overloaded terminology
• Difficulties conversing about testing
Test Categorisation• Unit
• Component
• API
• Acceptance
• Integration
• Functional
• Build Acceptance
• System
• End-to-End
• User Acceptance
• Non-functional
Test Categorisation
Test Categorisation
Test Categorisation• Useful concept - up to a point
• Blurred lines + Overloaded terminology
• Difficulties conversing about testing
Test Categorisation
Test Categorisation
Overview• Automating
• Testing Categories + Scope
• Development Testing
Development Testing• What it is
• What it should not be
• How it has evolved
What it is• A Development practice first and foremost
• White box / Black box / Grey box
• Has Quality aims
• Has Efficiency aims
• Test automation framework of choice for most Java developers
• Synonymous with Unit testing
• Widely used for testing beyond the Unit scope
• Ported to many languages - generically known as xUnit
JUnit
JUnit
“Never in the field of software development have so many owed so much to so few lines of code”
Martin Fowler
(Development big-wig)
What it is• A Development practice first and foremost
• White box / Black box / Grey box
• Has Quality aims
• Has Efficiency aims
What it is• Can take a technical perspective on scenarios
• Can take a user functionality perspective
• Exercises precise control over the test context
• Used + extended by Dev during initial and future coding
What it is• Often used to discover and refine solution design
• Often leaves fine-grained regression coverage safety net
• Enables instant feedback on failures due to future enhancements and refactorings
What it is not• Automated full System tests a.k.a E2E, System Integration
• Zero control over the state of the system (the test context)
• Test scope coupled to other external systems, reference data quality, env availability
• Heavily reliant on driving a UI to execute many fine-grained scenarios
Its Evolution• OOP - slow evolution
• Automated Developer Testing around a lot less time
• A lot less mainstream than OOP
• Growing body of knowledge / experience available
Overview• Automating
• Testing Categories + Scope
• Development Testing
• Principles, Practices + Tool Support
Principles, Practices + Tool Support• Structural Principles / Characteristics
• Qualities of a “good” test / spec
• Practices + Tools for improving test qualities
Test structure• Specify Inputs
• Specify the state of the System
• Specify the Event
• Specify the Expectations
• Arrange, Act, Assert
• Given, When, Then
Test structure• Naming conventions + structure
@Test public void methodUnderTest_GivenABC_ThenExpectXYZ() { // given // when // then }
@Test public void methodUnderTest_GivenInputsABC_AndSystemStateDEF_ThenExpectXYZ() { // given - inputs // given - system state! // when // then }
Test structure• IDE assistance for consistency + evolve conventions
Principles, Practices + Tool Support• Structural Principles / Characteristics
• Qualities of a “good” test / spec
“Any fool can write code that a computer can understand”
!
Martin Fowler
Test Qualities• Readability
• What is it? Why does it matter?
Test Qualities• Readability == …?
• Code that you can understand with ease
• Language affects Thought
• Thought affects Action
Test Qualities• Tools can directly address Readability qualities
• Domain Specific Languages = a fancy term
• JUnit, Hamcrest, FEST, JMock, Mockito, Gherkin
• Language -> Thought -> Action
“Test automation is a first class software engineering problem”
!
Brian Marick?
Test Qualities• Maintainability
• What is it? Why does it matter?
Test Qualities• Maintainability == …?
• To change, evolve with ease
• Keeping the cost of change down
Test Qualities• They can be context sensitive. Many are not.
• You may opt to sacrifice:
• DRY Principle for Readability
• Readability for Maintainability
• Maintainability for Obviousness
• Expect debate
Principles, Practices + Tool Support• Structural Principles / Characteristics
• Qualities of a “good” test / spec
• Practices + Tools for improving test qualities
Specifying Expectations• Stating Expectations / Asking Questions
• From Computer dialect ——> Human dialect
// then assertTrue(trader.getPermissions().isEmpty()); // Computer dialect // . assertThat(trader.getPermissions().isEmpty(), is(true)); // . // . assertThat(trader.getPermissions()).isEmpty(); // . // . assertThat(trader).hasNoPermissions(); // Human dialect
Specifying Expectations• Stating Expectations / Asking Questions
• From Computer dialect ——> Human dialect // Using Basic JUnit assertEquals(orders.size(), 2); Iterator<Order> itr = orders.iterator(); assertEquals(itr.next(), order3); assertEquals(itr.next(), order4); // Using Hamcrest assertThat(orders, hasItems(order3, order4)); assertThat(orders.size(), is(2)); // Using FEST Assertions assertThat(orders).containsOnly(order3, order4); // Using FEST Assertions (chained assertions) assertThat(orders).containsOnly(order3, order4) .containsSequence(order3, order4);
Specifying Expectations• Easier + Faster to write expressive statements
• Faster to read + review
• Easier to maintain
• Encourages other better practices
Specifying Inputs• Trivial example. Test with very narrow scope
• e.g.Input = Single Object
@Test public void getPermissions_GivenNewUser_ThenReturnsNoPermissions() { // given Trader trader = new Trader(); // when + then assertThat(trader.getPermissions()).isEmpty(); }
Specifying Inputs• Non-trivial example. Test with broader scope
• e.g. Input = Object Graph
• i.e. objects linked to objects
@Test public void getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v1() { // given - inputs Trader trader = new Trader(); TradingAccount tradingAcct = new TradingAccount(new TradingFirm("TraderzRUs", "TRDRZ"), new ClearingFirm("TooBig2Fail", "2BIG2F"), ACTIVE); trader.setPermissions(Arrays.asList(new Permission(tradingAcct))); String goldIsinCode = "COF24680"; // given - system state Future oil3MnthFuture = new Future("OIL.3MNTH", new ISIN("OIL3M0123")); Future oil6MnthFuture = new Future("OIL.6MNTH", new ISIN("OIL6M0456")); Future goldFuture = new Future("GLD.3MNTH", new ISIN(goldIsinCode)); String orderId = "ordId"; Order order1 = new Order(orderId + 1, tradingAcct, oil3MnthFuture, 1000, 2500); Order order2 = new Order(orderId + 2, tradingAcct, oil6MnthFuture, 1500, 2600); Order order3 = new Order(orderId + 3, tradingAcct, goldFuture, 200, 4300); Order order4 = new Order(orderId + 4, tradingAcct, goldFuture, 150, 4300); OrdersDAO orderDAO = new OrdersDAOInMemory(order1, order2, order3, order4); OrderSearchService orderSearchService = new OrderSearchServiceImpl(orderDAO); // when List<Order> orders = orderSearchService.getOrders(trader, tradingAcct, goldIsinCode); // then assertThat(orders).containsOnly(order3, order4); }
Specifying Inputs• Consider many non-trivial tests, with “raw” setup
• Impact of changing one setter or constructor
• Quantity of code there is to read, understand, change
Specifying Inputs - Techniques• Helper methods
• Object Mother Pattern
• Test Data Builder Pattern
@Test public void getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v1() { // given - inputs Trader trader = new Trader(); TradingAccount tradingAcct = new TradingAccount(new TradingFirm("TraderzRUs", "TRDRZ"), new ClearingFirm("TooBig2Fail", "2BIG2F"), ACTIVE); trader.setPermissions(Arrays.asList(new Permission(tradingAcct))); String goldIsinCode = "COF24680"; // given - system state Future oil3MnthFuture = new Future("OIL.3MNTH", new ISIN("OIL3M0123")); Future oil6MnthFuture = new Future("OIL.6MNTH", new ISIN("OIL6M0456")); Future goldFuture = new Future("GLD.3MNTH", new ISIN(goldIsinCode)); String orderId = "ordId"; Order order1 = new Order(orderId + 1, tradingAcct, oil3MnthFuture, 1000, 2500); Order order2 = new Order(orderId + 2, tradingAcct, oil6MnthFuture, 1500, 2600); Order order3 = new Order(orderId + 3, tradingAcct, goldFuture, 200, 4300); Order order4 = new Order(orderId + 4, tradingAcct, goldFuture, 150, 4300); OrdersDAO orderDAO = new OrdersDAOInMemory(order1, order2, order3, order4); OrderSearchService orderSearchService = new OrderSearchServiceImpl(orderDAO); // when List<Order> orders = orderSearchService.getOrders(trader, tradingAcct, goldIsinCode); // then assertThat(orders).containsOnly(order3, order4); }
@Test public void // Using helpers getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v2() { // given - inputs String accountCode = "TRDRZ"; String goldIsinCode = "GLD24680"; TradingAccount tradingAcct = createTradingAccount(accountCode); Trader trader = createTraderWithPermissionsFor(tradingAcct); // given - system state Future oil3MnthFuture = createFuture("OIL.3MNTH", "OIL3M0123"); Future oil6MnthFuture = createFuture("OIL.6MNTH", "OIL6M0456"); Future goldFuture = createFuture("GLD.3MNTH", goldIsinCode); Order order1 = createOrder(accountCode, oil3MnthFuture, qty(1000), price(2500)); Order order2 = createOrder(accountCode, oil6MnthFuture, qty(1500), price(2600)); Order order3 = createOrder(accountCode, goldFuture, qty(200), price(4300)); Order order4 = createOrder(accountCode, goldFuture, qty(150), price(4300)); OrderSearchService orderSearchService = createOrderService( createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = orderSearchService.getOrders(trader, accountCode, goldIsinCode); // then assertThat(orders).containsOnly(order3, order4); }
@Test public void // Using helpers + hiding irrelevant details getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v3() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", dummyFuture()), order2 = createOrder("TRDRZ", dummyFuture()), order3 = createOrder("TRDRZ", createFuture("GLD24680")), order4 = createOrder("TRDRZ", createFuture("GLD24680")); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order3, order4); }
@Test public void getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v1() { // given - inputs Trader trader = new Trader(); TradingAccount tradingAcct = new TradingAccount(new TradingFirm("TraderzRUs", "TRDRZ"), new ClearingFirm("TooBig2Fail", "2BIG2F"), ACTIVE); trader.setPermissions(Arrays.asList(new Permission(tradingAcct))); String goldIsinCode = "COF24680"; // given - system state Future oil3MnthFuture = new Future("OIL.3MNTH", new ISIN("OIL3M0123")); Future oil6MnthFuture = new Future("OIL.6MNTH", new ISIN("OIL6M0456")); Future goldFuture = new Future("GLD.3MNTH", new ISIN(goldIsinCode)); String orderId = "ordId"; Order order1 = new Order(orderId + 1, tradingAcct, oil3MnthFuture, 1000, 2500); Order order2 = new Order(orderId + 2, tradingAcct, oil6MnthFuture, 1500, 2600); Order order3 = new Order(orderId + 3, tradingAcct, goldFuture, 200, 4300); Order order4 = new Order(orderId + 4, tradingAcct, goldFuture, 150, 4300); OrdersDAO orderDAO = new OrdersDAOInMemory(order1, order2, order3, order4); OrderSearchService orderSearchService = new OrderSearchServiceImpl(orderDAO); // when List<Order> orders = orderSearchService.getOrders(trader, tradingAcct, goldIsinCode); // then assertThat(orders).containsOnly(order3, order4); }
@Test public void // Using helpers getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v2() { // given - inputs String accountCode = "TRDRZ"; String goldIsinCode = "GLD24680"; TradingAccount tradingAcct = createTradingAccount(accountCode); Trader trader = createTraderWithPermissionsFor(tradingAcct); // given - system state Future oil3MnthFuture = createFuture("OIL.3MNTH", "OIL3M0123"); Future oil6MnthFuture = createFuture("OIL.6MNTH", "OIL6M0456"); Future goldFuture = createFuture("GLD.3MNTH", goldIsinCode); Order order1 = createOrder(accountCode, oil3MnthFuture, qty(1000), price(2500)); Order order2 = createOrder(accountCode, oil6MnthFuture, qty(1500), price(2600)); Order order3 = createOrder(accountCode, goldFuture, qty(200), price(4300)); Order order4 = createOrder(accountCode, goldFuture, qty(150), price(4300)); OrderSearchService orderSearchService = createOrderService( createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = orderSearchService.getOrders(trader, accountCode, goldIsinCode); // then assertThat(orders).containsOnly(order3, order4); }
@Test public void // Using helpers + hiding irrelevant details getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v3() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", dummyFuture()), order2 = createOrder("TRDRZ", dummyFuture()), order3 = createOrder("TRDRZ", createFuture("GLD24680")), order4 = createOrder("TRDRZ", createFuture("GLD24680")); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order3, order4); }
Specifying Inputs - Techniques• Object Mother Pattern
public class ObjectMother {! public static OrdersDAOInMemory createOrdersDAO(Order order1, Order order2, Order order3, Order order4) { return new OrdersDAOInMemory(order1, order2, order3, order4); }! public static OrderSearchService createOrderService(OrdersDAO ordersDAO) { OrderSearchService orderSearchService = new OrderSearchServiceImpl(ordersDAO); return orderSearchService; } private OrderSearchService prepareOrderService(Order order1, Order order2, Order order3, Order order4) { OrdersDAO orderDAO = createOrdersDAO(order1, order2, order3, order4); OrderSearchService orderSearchService = new OrderSearchServiceImpl(orderDAO); return orderSearchService; }! public static Order createOrder(String accountCode, Future dummyFuture) { return createOrder(accountCode, dummyFuture, qty(0), price(0)); }! public static Order createOrder(String accountCode, Future future, int qty, int price) { return new Order(nextOrderId(), accountCode, future, qty, price); } private Order createOrder(String orderId, TradingAccount tradingAcct, Future future, int qty, int price) { return new Order(orderId, tradingAcct.getTradingFirm().getCode(), future, qty, price); }! public static Future createFuture(String desc, String isinCode) { return new Future(desc, new ISIN(isinCode)); } public static Future createFuture(String isinCode) { return createFuture(DEFAULT_FUTURE_DESC, isinCode); }! public static Future dummyFuture() { return createFuture(DEFAULT_FUTURE_DESC, DEFAULT_ISIN_CODE); }! public static TradingAccount createTradingAccount(String accountCode) {
Specifying Inputs - Techniques• Test Data Builder Pattern
@Test public void // Using Test Data Builders getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v4() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = aTrader().with(aPermission() .with(aTradingAccount() .with(aTradingFirm().withCode("TRDRZ")))).build(); // given - system state OrderBuilder matchingOrder = anOrder().withAccountCode("TRDRZ") .with(aFuture().with(anISIN().withIsinCode("GLD24680")).build()); Order order1 = anOrder().withAccountCode("TRDRZ").build(), order2 = anOrder().withAccountCode("TRDRZ").build(), order3 = matchingOrder.build(), order4 = matchingOrder.build(); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order3, order4); }
public class TraderBuilder { public static String DEFAULT_USERNAME = "trader1"; public static UserDetail DEFAULT_USERDETAIL = new UserDetail(); public static List<Permission> NO_PERMISSIONS = Collections.emptyList(); public static List<Restriction> NO_RESTRICTIONS = Collections.emptyList();! private String userName = DEFAULT_USERNAME; private UserDetail userDetail = DEFAULT_USERDETAIL; private List<Permission> permissions = NO_PERMISSIONS; private List<Restriction> restrictions = NO_RESTRICTIONS; private TraderBuilder() { } public static TraderBuilder aTrader() { return new TraderBuilder(); } public Trader build() { Trader trader = new Trader(); trader.setUserName(userName); trader.setUserDetail(userDetail); trader.setPermissions(permissions); trader.setRestrictions(restrictions); return trader; } public TraderBuilder withUserName(String userName) { this.userName = userName; return this; }! public TraderBuilder with(UserDetail userDetail) { this.userDetail = userDetail; return this; } public TraderBuilder withPermissions(List<Permission> permissions) { this.permissions = new ArrayList<>(permissions); return this; }
public TraderBuilder with(PermissionBuilder permission) { this.permissions = new ArrayList<>(Arrays.asList(permission.build())); return this; } public TraderBuilder withNoPermissions() { this.permissions = Collections.emptyList(); return this; }! public TraderBuilder with(RestrictionBuilder restriction) { this.restrictions = new ArrayList<>(Arrays.asList(restriction.build())); return this; } public TraderBuilder withRestrictions(List<Restriction> restrictions) { this.restrictions = new ArrayList<>(restrictions); return this; } public TraderBuilder but() { return new TraderBuilder() .withUserName(userName) .with(userDetail) .withPermissions(permissions) .withRestrictions(restrictions); }}
@Test public void // Using Test Data Builders getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v4() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = aTrader().with(aPermission() .with(aTradingAccount() .with(aTradingFirm().withCode("TRDRZ")))).build(); // given - system state OrderBuilder matchingOrder = anOrder().withAccountCode("TRDRZ") .with(aFuture().with(anISIN().withIsinCode("GLD24680")).build()); Order order1 = anOrder().withAccountCode("TRDRZ").build(), order2 = anOrder().withAccountCode("TRDRZ").build(), order3 = matchingOrder.build(), order4 = matchingOrder.build(); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order3, order4); }
@Test public void // Using helpers + hiding irrelevant details getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v3() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", dummyFuture()), order2 = createOrder("TRDRZ", dummyFuture()), order3 = createOrder("TRDRZ", createFuture("GLD24680")), order4 = createOrder("TRDRZ", createFuture("GLD24680")); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order3, order4); }
@Test public void // Using Test Data Builders to create input data variations getOrders_GivenTraderWithVariousPermissions_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v5() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; TradingAccountBuilder tradingAccount = aTradingAccount().with(aTradingFirm().withCode("TRDRZ")); Trader traderWithNoPerms = aTrader().withNoPermissions().build(); Trader traderWithPerms = aTrader().with(aPermission().with(tradingAccount)).build(); Trader traderWithInactivePerms = aTrader().with(aPermission().with( tradingAccount.but().isInActive())).build(); // given - system state OrderBuilder matchingOrder = anOrder().withAccountCode("TRDRZ") .with(aFuture().with(anISIN().withIsinCode("GLD24680")).build()); Order order1 = anOrder().withAccountCode("TRDRZ").build(), order2 = anOrder().withAccountCode("TRDRZ").build(), order3 = matchingOrder.build(), order4 = matchingOrder.build(); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders1 = service.getOrders(traderWithNoPerms, searchAccount, searchIsin); List<Order> orders2 = service.getOrders(traderWithPerms, searchAccount, searchIsin); List<Order> orders3 = service.getOrders(traderWithInactivePerms, searchAccount, searchIsin); // then assertThat(orders1).isEmpty(); assertThat(orders2).containsOnly(order3, order4); assertThat(orders3).isEmpty(); }
Specifying Inputs - Techniques• Test Data Builder Pattern - with tool support
• Make-It-Easy - addresses Builder class boiler-plate
• Model Citizen - annotation-based “Blueprints”
Specifying Inputs - Techniques• Test Data Builder Pattern - Advantages
• Your test / spec documents only the inputs that matter
• Decouples your test from being impacted by a wide range of changes
• Supports “declarative” code style + useful templates
• user = EXPIRED_USER.but( with ( aBalance, 100));
Specifying Inputs - Techniques• Test Data Builder Pattern - Disadvantages
• Requires much new test-supporting code
• Chained methods take getting used to
Specifying the System State• State of the SUT / the Test Context
• Communicating what the context, limitations and boundaries are
• Isolating your test from certain dependencies / interactions
• Use of Test Doubles
• Hand-rolled Test Double
• common misnomer Mocking Frameworks
Specifying the System State• Problems + solutions for Specifying Inputs normally apply
• Unique to System State is your Test Isolation requirements
Specifying the System State - Techniques• Hand-rolling Test Doubles
• Using Test Isolation frameworks to provide Test Doubles
• “Mocking” + “Mocks Aren’t Stubs” http://martinfowler.com/articles/mocksArentStubs.html
• Dummy / Fake / Stubs / Mocks / Spies
Specifying the System State - Techniques• Stub being used in a state-based test
@Test public void // Stubbing system state (Mockito) + using helpers + hiding irrelevant details getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v2() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", createFuture("GLD24680")), order2 = createOrder("TRDRZ", createFuture("GLD24680")); OrdersDAO ordersDAOStub = mock(OrdersDAO.class); given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2)); OrderSearchService service = createOrderService(ordersDAOStub); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order1, order2); }
@Test public void // Stubbing system state + stubbing inputs (Mockito) getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v3() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; // Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); Trader traderStub = mock(Trader.class); Permission permissionStub = mock(Permission.class); given(traderStub.getPermissions()).willReturn(Arrays.asList(permissionStub)); TradingAccount tradingAccountStub = mock(TradingAccount.class); given(permissionStub.getTradingAccount()).willReturn(tradingAccountStub); given(tradingAccountStub.isActive()).willReturn(Boolean.TRUE); given(tradingAccountStub.getTradingFirm()).willReturn(new TradingFirm("", searchAccount)); // given - system state Order order1 = createOrder("TRDRZ", createFuture("GLD24680")), order2 = createOrder("TRDRZ", createFuture("GLD24680")); OrdersDAO ordersDAOStub = mock(OrdersDAO.class); given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2)); OrderSearchService service = createOrderService(ordersDAOStub); // when List<Order> orders = service.getOrders(traderStub, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order1, order2); }
@Test public void // Stubbing system state + stubbing input (Mockito w/ Deep Stubs) getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v3_1() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; // Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); Trader traderStub = mock(Trader.class); Permission permissionStub = mock(Permission.class, RETURNS_DEEP_STUBS); given(traderStub.getPermissions()).willReturn(Arrays.asList(permissionStub)); given(permissionStub.getTradingAccount().isActive()).willReturn(Boolean.TRUE); given(permissionStub.getTradingAccount().getTradingFirm().getCode()).willReturn(searchAccount); // given - system state Order order1 = createOrder("TRDRZ", createFuture("GLD24680")), order2 = createOrder("TRDRZ", createFuture("GLD24680")); OrdersDAO ordersDAOStub = mock(OrdersDAO.class); given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2)); OrderSearchService service = createOrderService(ordersDAOStub); // when List<Order> orders = service.getOrders(traderStub, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order1, order2); }
Specifying the System State - Techniques• A Mock being used in a interaction-based test
@Test public void // Using a Mock to test interactions + Stubbing system state (Mockito) getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v4_1() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", createFuture("GLD24680")), order2 = createOrder("TRDRZ", createFuture("GLD24680")); OrdersDAO ordersDAOStub = mock(OrdersDAO.class); given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2)); PermissionService permissionServiceMock = mock(PermissionService.class); OrderSearchService service = createOrderServiceV2(ordersDAOStub, permissionServiceMock); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then verify(permissionServiceMock).hasViewPermissions(trader, order1); verify(permissionServiceMock).hasViewPermissions(trader, order2); }
@Test public void // Using a Mock to test interactions + Stubbing system state (Mockito) getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v4_2() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", createFuture("GLD24680")), order2 = createOrder("TRDRZ", createFuture("GLD24680")); OrdersDAO ordersDAOStub = mock(OrdersDAO.class); given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2)); PermissionService permissionServiceMock = mock(PermissionService.class); given(permissionServiceMock.hasViewPermissions( any(Trader.class), any(Order.class))).willReturn(Boolean.TRUE); OrderSearchService service = createOrderServiceV2(ordersDAOStub, permissionServiceMock); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then verify(permissionServiceMock).hasViewPermissions(trader, order1); verify(permissionServiceMock).hasViewPermissions(trader, order2); assertThat(orders).containsOnly(order1, order2); }
Specifying the System State - Techniques• Often a test will have
• Zero to many Stubs. Zero or one Mock
• Mocks help you answer questions about what happened
• When a Mock is present, you ask it something at the end
• A Stub should not be asserted against
• There to help create + control the System State for the test
Specifying the Event• Often trivial
• Might be a sequence of events that we want to fire
• Sequence might actually be context / SUT setup
• Practices? Tools?
Specifying it All Together
@Test public void helloWorld() throws Exception { mockMvc.perform(get("/hello").accept(MediaType.TEXT_PLAIN)) .andDo(print()) // print the request/response in the console .andExpect(status().isOk()) .andExpect(content().contentType(MediaType.TEXT_PLAIN)) .andExpect(content().string("Hello World!")); }
• Spring MVC Test support / DSL
Principles, Practices + Tool Support• Structural Principles / Characteristics
• Qualities of a “good” test / spec
• Practices + Tools for improving test qualities
• Continuous Integration - development practice
• CI tools - run automated tests frequently
• Continuous Testing - development practice
• CT tools - run automated tests locally, frequently
• e.g. Infinitest, JUnitMax
Continuous Testing
Overview• Automating
• Testing Categories + Scope
• Development Testing
• Principles, Practices + Tool Support
• Tools/APIs just help with heavy lifting
• Sound coding principles are paramount
• Creating unreadable and unmaintainable test, even with “the right” tools
• Unreadable + high maintenance tests will hinder efficiency
Summary
Summary• The power of marginal improvements
• for … each, <> operator, closure support, Guava
• Continuous iterative improvements nudge us forward
• Continuous iterative improvements push down cost of reading, writing, evolving tests
• Drives the cost down to “mass market” level
• Beyond Java
Resources• Books
• GOOS - Growing Object Oriented Software, Guided By Tests
• Effective Unit Testing
• Working Effectively with Legacy Code
• xUnit Test Patterns - Refactoring Test Code
Resources• Podcasts
• Sustainable Test Driven Development (Series)
• Hanselminutes - Roy Asherove - The Art of Unit Testing
• Hanselminutes - Quetzal Bradley - Beyond Unit Testing
• All Code Examples are available at
• http://github.com/cathalking/devtestevolution