automated tests
TRANSCRIPT
PrefaceThere are many types of automatic tests, testing tools, libraries and approaches.
Automatic tests can save you a lot of stress but can also became a kind of a nightmare.
This presentation is an overview of what's available and how to use and not to use them to make them really useful.
Examples taken from PHP world. You might be surprised how many tools is available.
AgendaWhat is a test?
The tools
How to start and progress
Challenges
About meDamian Sromek
PHP Developer since ~2010
BlaBlaCar.com; Weemto.com; Xstream.net; X-Formation.comLinked In Profile
Test? Why?To make the user happy.
To make your boss happy.
To make your company happy.
To make you happy.
TestMake sure software works - quality assurance
Make sure software does what it’s supposed to do
Make sure software stays working
Test - TypesLarge: Acceptance, End to end
User can buy a book using web browser
Medium: Integration, FunctionalInternal API call to shop service works fine when external payment system is mocked.
Small: UnitClass computes the price correctly.
TestManual
Performed by human - usually a student clicking all day long.
AutomatedPerformed by computer - usually PHPUnit clicking all day long.
Manual + AutomatedHuman tester clicking to prepare a new testing scenario that will be automated next.
Manual TestingClicking around (“random”, test plan)
“var_dump”
Debugging using debugger
Automated TestsComputer is much faster at testing than human
Computer tool checks the software. Eg. clicks around
Deliver fast and with less worries - especially when working with many people
Protect against regression
Allows easier testing of edge and hard to reproduce cases
PHP toolshttp://www.phptherightway.com/#testing
Automated Tests = HappyAvoiding for slow, boring “monkey” testing
Avoiding testing again and again, … and again
You can debug easier
You know when you break things
You know when other people break things - they fix it
TDD, BDD, DDDTDD - Test Driven Development
Know your software/class requirements before implementing it.
Behavior Driven Development Know your business by writing down user’s behaviour
DDD - Domain Driven DesignKnow your business first by modeling the domain and events
Development <-> Design
Automated E2E/Acceptance TestsSelenium, Mink
Behat
Codeception
Storyplayer
PHPUnit
Automated Integration TestsTest if multiple classes work together
Test if “classes” work properly with external WebService, Database
PHPUnit
Codeception
Behat (!) - contexts
Automated Unit TestsPHPUnit
PHPSpec (BDD)
Codeception
CT, CI, CDContinouos Testing
Continuous Integration
Continuous Delivery
Automation PlatformsJenkins
Bamboo
Travis-CI
TeamCity
How? When?Know your project, team, goals, definition of done
WordPress website vs banking system
Define risks and make a simple test planText has wrong size vs user unable to pay
Start with the simplest solution and evolve
Other (Un)expected BenefitsBetter understanding of “the purpose” of the software
Safer refactoring
Better code design and quality
Documentation of features and a design
Faster development of a better software!
Is it easy?“It depends”
Need to learn a new things - tools, change the way of thinking.Was learning OOP easy?
Writing Integration and Unit tests can be really hard if you have poor code design
Writing a good test requires you to “become a tester”
LearnBooks
Growing Object-Oriented Software, Guided by TestsTestowanie i jakość oprogramowania. Metody, narzędzia, techniki
YouTube, conferences
Open source projectsSymfony: PHPUnitSylius: Behat, PHPSpec
WebWikipediaTesting on toiletUncle Bob
ExcusesI create a code that works
It works on my computer
I don’t know how to write tests
I don’t like writing tests
I have no time for writing tests
It’s not possible to test this
It’s quite hard to measure the benefits
Good Start - Bugs!1.Get a bug report
2.Reproduce it by writing an automatic test
3.Fix it!
Good Start - High LevelIf you’re responsible for the whole project
Basic Acceptance Tests
then basic Integration Tests
then basic Unit Tests
But What Tool(s)?A tool you or other devs are familiar with
A tool that’s quite popular - big comunity
A tool that fits your needs
As few tools you need
A tool with good support in IDE, automation platforms etc.
Usually PHPUnit would be a good start
Vagrant, Docker
“Great software development is not about the tools you use but about how you use them.” - Me ;D
On the wayUse Continuous Testing (eg. Jenkins)
Keep tests close to the source code
Keep the tests quality at the best possible level
Have as much tests as you need - not more
Keep it simple
How much tests? Acceptance < Integration < Unit
Keep test execution time as low as possibleAcceptance < 30 minUnit < 2 min
Master LevelTest Driven Development / Design
Desing / Test -> Implement -> Refactor
IDE for running tests with debugger
Every feature and bug fix with an automatic test
You shall not merge on failing tests
Test groups, desciptive names and fail messages
The same test suites for many environments
Continuous ImprovementTry the simplest solution for your needs
Learn
Improve
Good Automated TestShows what the software does
Hides how the software works“I should not be able to tell what’s the implementation by reading the test” - Me
Protects from bugs ;)
Should let you replace feature implementation
Should speed up your daily work
Fails with descriptive and easy to understand message
Is independent and repeatable
Must prepare it self. Should clean after it self.
Tests one thing!
ChallengesHaving a lot of poor quality code
Execution takes too longTry to move to Unit tests. Acceptance tests for the simplest scenarios; Database in RAMDisk
Mock-hell Try not to overuse mocks. Some classes are made to integrate things.
Too many testsCheck if the tests are really usefull. If not then remove some.Do they really check something useful?
Randomly failing testsTry not to make tests depend on real time/date. Eg. mock the clock.
Knowing the toolEg. Behat
Quality MetricsNumber of tests and assertions
Execution time
Code coverage
Number of bugs found during development
Number of bugs protected against - regression
Speed of developing new, WORKING features
Certainty of releasing new version of software
Is it worth it?YES!
Automation for the win!
Automated tests for the win!
Testing Legacy Projects- Test only at a high level if you know your internals needs to be
replaced/improved
Testing Large and Complex SystemsOnline Video Platform
Web, Backend, Queue, DistributedFTP upload; Wait for asynchronous actions; Check GUI;
Set Top Box appSelenium; WebSockets
Behat scenarios in Jenkins task config
Exotic TestsMutation testing
Monitor performance (eg. number of SQL querries), profile
Database Migrations
Database (Doctrine) mapping
Questions?Email: [email protected]
Twitter: damiansromek
Blog: damiansromek.pl
Github: paq85
Thank you!