![Page 1: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/1.jpg)
From 3 weeks to 30 minutes – a journey through the ups and downs of test
automation
![Page 2: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/2.jpg)
Who am I?
• Peter Thomas– Chief Software Engineer– Operations IT, UBS Investment Bank
• Developer (mostly)• I do some architecture• I have done Testing• I talk a lot (Mentor/Coach)
• From the dark side (consulting) but getting better
![Page 3: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/3.jpg)
Where did we start?
• Existing mainframe legacy application• 3 week manual PPT testing cycle• 12 week delivery cycle
![Page 4: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/4.jpg)
What did we want to do?
• Belief there was a better way to deliver software
• Incremental development to deliver business value quickly
• Address the rapidly changing business landscape with flexibility in delivery
• Build quality into the solutions• Deliver the software rapidly, but in a cost
effective manner• Put the fun back into software delivery
![Page 5: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/5.jpg)
New York London Kiev Hyderabad Hong Kong
2M trades per day 100 billions settling per day in all major currencies50+ exchanges across EMEA and APAC15 scrum teams/120 people9 applicationsProduction releases every 2 weeks
![Page 6: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/6.jpg)
New York London Kiev Hyderabad Hong Kong
200 commits per day1000 artefacts updated per day1 commit every 5 minutes peak
![Page 7: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/7.jpg)
New York London Kiev Hyderabad Hong Kong
24 Build Targets60+ Test Targets800 Automated Functional Tests10, 000 Unit/Integration Tests7, 000 Behavioural Tests
![Page 8: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/8.jpg)
But……..
![Page 9: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/9.jpg)
Our tests were…..
ComplicatedObscure
Random failuresSlow to run
Difficult to fix
![Page 10: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/10.jpg)
“The TDD rut”
ComplicatedObscure
Random failuresSlow to run
Difficult to fix
![Page 11: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/11.jpg)
Test the Right Thing and
Test the Thing Right
When all you have is a hammer, everything looks like
a nail
![Page 12: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/12.jpg)
Why do you test?
![Page 13: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/13.jpg)
Why do you test?
• Because TDD tells me so?• Because (insert favourite method
here) says I should?• So I meet the 80% coverage metric?
![Page 14: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/14.jpg)
Why do you test?
• To accept the solution• To understand and document the solution• To prove its not broken• To find the unknown unknowns• To help us design and build new features• To help us explore what is really needed• To show it won’t crash under load, to
show it is secure (to test the ‘ilities)…?
![Page 15: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/15.jpg)
Why do you test?
Agile testing Quadrants – Lisa Crispin, Janet Gregory
![Page 16: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/16.jpg)
Testing Purposefully
![Page 17: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/17.jpg)
The Right Thing At The Right Level
UnitComponent
System
![Page 18: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/18.jpg)
The Right Thing At The Right Level
UnitComponent
System
•Tests a single class with no dependencies•If dependencies like Spring Context, Database used then called Unit Integration•Tests technical correctness and robustness•Very specific, a failing test indicates an issue in a specific class•Difficult to perform on poor quality code•Very fast to run, should run on the developer’s desktop in the IDE
![Page 19: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/19.jpg)
The Right Thing At The Right Level
UnitComponent
System
•Tests a group of components which are integrated to perform a business relevant function•Can test technical or business correctness, but should be expressed in Domain concepts•Specific, a failing test indicates problems in that component•Easier to perform on poor quality code, provided component boundaries are clear•Can be quick to run, doesn’t need the full application, should run on developers desktop
![Page 20: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/20.jpg)
The Right Thing At The Right Level
UnitComponent
System
•Tests a system at its boundaries as a ‘black box’•Primarily testing for business correctness•Not Specific, a failing test could be caused anywhere in the system flow•Easy to perform on legacy applications, requires little code modification•Slow to run, can be fragile, may not run on developers desktop
![Page 21: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/21.jpg)
What We Wanted
![Page 22: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/22.jpg)
What We Had
Unit tests which weren’t really Unit TestsEnd to End tests when unit tests would have been sufficientDuplicate and redundant End to End tests
![Page 23: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/23.jpg)
The TDD Cycle
![Page 24: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/24.jpg)
TDD?@Test public void shouldBeEmtpyAfterCreation() { ReportObject aTrm = new
ReportObject(); assertNull(aTrm.getEligibleTrm()); assertNull(aTrm.getProcessedEvent());
assertNull(aTrm.getPayloadXml()); }
@Test public void shouldCorrectlySetAttributesViaConstructor() { ReportObject aTrm = new ReportObject(eligibleObject, REPORTABLE_XML); assertEquals(eligibleObject, reportableTrm.getEligibleTrm()); assertEquals(REPORTABLE_XML, reportableTrm.getPayloadXml()); }
@Test public void shouldCorrectlySetFieldsViaSetters() { ReportObject aTrm = new ReportObject(); aTrm.setEligibleTrm(eligibleObject); aTrm.setProcessedEvent(child); aTrm.setPayloadXml(REPORTABLE_XML);
assertEquals(eligibleObject, aTrm.getEligibleTrm()); assertEquals(child, aTrm.getProcessedEvent()); assertEquals(REPORTABLE_XML, aTrm.getPayloadXml()); }
![Page 25: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/25.jpg)
The Hollow Egg
![Page 26: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/26.jpg)
The Hollow Egg98 Tests2.5K LOC
30 Tests200 LOC
![Page 27: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/27.jpg)
RSpec model
![Page 28: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/28.jpg)
Outside In - The TDD Spiral
![Page 29: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/29.jpg)
Make the Intent Clear
How to achieve acceptance without showing your IDE or
log file to the users
![Page 30: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/30.jpg)
Unit Test Naming?
testProcessError()
whenWorkItemIsManuallyAssignedThenClientRuleShouldBeSetToManualOverride()
shouldAllowAnActioningWorkItemToBeUpdated()
![Page 31: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/31.jpg)
Test Data Nightmare
![Page 32: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/32.jpg)
What Do You Demo?
![Page 33: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/33.jpg)
What Do You Demo?
![Page 34: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/34.jpg)
Executable Specification
![Page 35: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/35.jpg)
Improve Testing Stability
Avoiding the Broken Windows syndrome
![Page 36: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/36.jpg)
Separate Progress & Regression Tests
![Page 37: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/37.jpg)
Speed-up Through Parallelism
![Page 38: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/38.jpg)
Identify Unstable Tests
![Page 39: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/39.jpg)
Quarantine Unstable Tests
![Page 40: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/40.jpg)
Avoid External Dependencies
![Page 41: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/41.jpg)
Introduce Fakes
![Page 42: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/42.jpg)
Avoid Time-Dependent Tests
![Page 43: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/43.jpg)
Test Isolation
![Page 44: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/44.jpg)
Asynchronous Testing Headache
![Page 45: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/45.jpg)
Don’t!
• Does your test need to be asynchronous?
• 80/20 rule?• Create synchronous test runner
harness
![Page 46: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/46.jpg)
Asynchronous Testing using Events
![Page 47: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/47.jpg)
So…?
![Page 48: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/48.jpg)
Treat your Tests Like you Treat your Code
“it’s just a test class” is not an excuse
Clean Code applies to tests too
![Page 49: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/49.jpg)
Think about Why You are Testing
Specification tests for internal quality
Business tests for external quality
![Page 50: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/50.jpg)
Think about Who You are Testing For
More people are interested in your tests than you may think
![Page 51: From 3 weeks to 30 minutes – a journey through the ups and downs of test automation](https://reader036.vdocuments.site/reader036/viewer/2022070410/56814654550346895db36b28/html5/thumbnails/51.jpg)
Zero Tolerance to Instability
“It runs OK on my machine” is not a valid response