automated tests

Automated TestsWhat? How? When?

Damian Sromek damian.sromek@gmail.com

2016-04

PrefaceThere are many types of automatic tests, testing tools, libraries and approaches.

Automatic tests can save you a lot of stress but can also became a kind of a nightmare.

This presentation is an overview of what's available and how to use and not to use them to make them really useful.

Examples taken from PHP world. You might be surprised how many tools is available.

AgendaWhat is a test?

The tools

How to start and progress

Challenges

About meDamian Sromek

PHP Developer since ~2010

BlaBlaCar.com; Weemto.com; Xstream.net; X-Formation.comLinked In Profile

Test? Why?To make the user happy.

To make your boss happy.

To make your company happy.

To make you happy.

TestMake sure software works - quality assurance

Make sure software does what it’s supposed to do

Make sure software stays working

Test - TypesLarge: Acceptance, End to end

User can buy a book using web browser

Medium: Integration, FunctionalInternal API call to shop service works fine when external payment system is mocked.

Small: UnitClass computes the price correctly.

TestManual

Performed by human - usually a student clicking all day long.

AutomatedPerformed by computer - usually PHPUnit clicking all day long.

Manual + AutomatedHuman tester clicking to prepare a new testing scenario that will be automated next.

Manual TestingClicking around (“random”, test plan)

“var_dump”

Debugging using debugger

Automated TestsComputer is much faster at testing than human

Computer tool checks the software. Eg. clicks around

Deliver fast and with less worries - especially when working with many people

Protect against regression

Allows easier testing of edge and hard to reproduce cases

PHP toolshttp://www.phptherightway.com/#testing

Automated Tests = HappyAvoiding for slow, boring “monkey” testing

Avoiding testing again and again, … and again

You can debug easier

You know when you break things

You know when other people break things - they fix it

TDD, BDD, DDDTDD - Test Driven Development

Know your software/class requirements before implementing it.

Behavior Driven Development Know your business by writing down user’s behaviour

DDD - Domain Driven DesignKnow your business first by modeling the domain and events

Development <-> Design

Automated E2E/Acceptance TestsSelenium, Mink

Behat

Codeception

Storyplayer

PHPUnit

Automated Integration TestsTest if multiple classes work together

Test if “classes” work properly with external WebService, Database

PHPUnit

Codeception

Behat (!) - contexts

Automated Unit TestsPHPUnit

PHPSpec (BDD)

Codeception

CT, CI, CDContinouos Testing

Continuous Integration

Continuous Delivery

Automation PlatformsJenkins

Bamboo

Travis-CI

TeamCity

How? When?Know your project, team, goals, definition of done

WordPress website vs banking system

Define risks and make a simple test planText has wrong size vs user unable to pay

Start with the simplest solution and evolve

Other (Un)expected BenefitsBetter understanding of “the purpose” of the software

Safer refactoring

Better code design and quality

Documentation of features and a design

Faster development of a better software!

Is it easy?“It depends”

Need to learn a new things - tools, change the way of thinking.Was learning OOP easy?

Writing Integration and Unit tests can be really hard if you have poor code design

Writing a good test requires you to “become a tester”

LearnBooks

Growing Object-Oriented Software, Guided by TestsTestowanie i jakość oprogramowania. Metody, narzędzia, techniki

YouTube, conferences

Open source projectsSymfony: PHPUnitSylius: Behat, PHPSpec

WebWikipediaTesting on toiletUncle Bob

ExcusesI create a code that works

It works on my computer

I don’t know how to write tests

I don’t like writing tests

I have no time for writing tests

It’s not possible to test this

It’s quite hard to measure the benefits

Good Start - Bugs!1.Get a bug report

2.Reproduce it by writing an automatic test

3.Fix it!

Good Start - High LevelIf you’re responsible for the whole project

Basic Acceptance Tests

then basic Integration Tests

then basic Unit Tests

But What Tool(s)?A tool you or other devs are familiar with

A tool that’s quite popular - big comunity

A tool that fits your needs

As few tools you need

A tool with good support in IDE, automation platforms etc.

Usually PHPUnit would be a good start

Vagrant, Docker

“Great software development is not about the tools you use but about how you use them.” - Me ;D

On the wayUse Continuous Testing (eg. Jenkins)

Keep tests close to the source code

Keep the tests quality at the best possible level

Have as much tests as you need - not more

Keep it simple

How much tests? Acceptance < Integration < Unit

Keep test execution time as low as possibleAcceptance < 30 minUnit < 2 min

Master LevelTest Driven Development / Design

Desing / Test -> Implement -> Refactor

IDE for running tests with debugger

Every feature and bug fix with an automatic test

You shall not merge on failing tests

Test groups, desciptive names and fail messages

The same test suites for many environments

Continuous ImprovementTry the simplest solution for your needs

Learn

Improve

Good Automated TestShows what the software does

Hides how the software works“I should not be able to tell what’s the implementation by reading the test” - Me

Protects from bugs ;)

Should let you replace feature implementation

Should speed up your daily work

Fails with descriptive and easy to understand message

Is independent and repeatable

Must prepare it self. Should clean after it self.

Tests one thing!

ChallengesHaving a lot of poor quality code

Execution takes too longTry to move to Unit tests. Acceptance tests for the simplest scenarios; Database in RAMDisk

Mock-hell Try not to overuse mocks. Some classes are made to integrate things.

Too many testsCheck if the tests are really usefull. If not then remove some.Do they really check something useful?

Randomly failing testsTry not to make tests depend on real time/date. Eg. mock the clock.

Knowing the toolEg. Behat

Quality MetricsNumber of tests and assertions

Execution time

Code coverage

Number of bugs found during development

Number of bugs protected against - regression

Speed of developing new, WORKING features

Certainty of releasing new version of software

Is it worth it?YES!

Automation for the win!

Automated tests for the win!

Testing Legacy Projects- Test only at a high level if you know your internals needs to be

replaced/improved

Testing Large and Complex SystemsOnline Video Platform

Web, Backend, Queue, DistributedFTP upload; Wait for asynchronous actions; Check GUI;

Set Top Box appSelenium; WebSockets

Behat scenarios in Jenkins task config

Exotic TestsMutation testing

Monitor performance (eg. number of SQL querries), profile

Database Migrations

Database (Doctrine) mapping

Questions?Email: damian.sromek@gmail.com

Twitter: damiansromek

Blog: damiansromek.pl

Github: paq85

Thank you!