Automated Tests in a REST APIby Luís Nóbrega

Regression Tests - benefits

Fast feedback

Discover introduced software bugs (regression)

Verify bug fixes on existing functionality

Validate changes against other components

Include re-running of previous tests

Can be executed on each code change

Ensure product quality

Minimize risks

Regression Tests - when to do it?

Specification has changed

Code has been changed

Add new test cases - increase coverage

New bugs were detected

Create a new test that reproduces the bug.

Regression Tests - State of the Art

Unit Tests - Test Driven Development (TDD) and Behavior Driven Development (BDD)

Tools: PHPUnit and phpspec

Acceptance Tests

Tool: Behat

Unit TestsMust run successfully on each change.

Tests should be written with/before code. We send inputs and verify outputs.

All dependencies are mocked. No hitting the database, file system or network.

Tests written by developers for developers.

Benefits

Allow safe refactoring of code parts.

It can work as documentation.

Gives higher confidence.

Characteristics

Independent

Fast

Repeatable

Up to date

Short

Unit Tests - When to Create?

Always, if it makes sense.

When you are fixing a bug. Reproduce it with a test.

When implementing new features or when specification has changed.

Check code coverage report to add new test cases

Unit Tests - How to execute?

PHPUnit

Execute: bin/phpunit -c app/

phpspec

Execute: bin/phpspec run

Create (describe) a new spec: bin/phpspec desc ‘<namespace>’

Execute with code coverage: bin/phpspec run --config=<config-file>.yml

Code coverage will be exported to build/coverage folder

Acceptance Tests - Behat

Verify the functional requirements

Ensure that the overall application is working

Test specification provided

Prevent regression problems

“Behat is an open source behavior-driven development framework (...) What is behavior-driven development, you ask? It’s the idea that you start by writing human-readable sentences that describe a feature of your application and how it should work, and only then implement this behavior in software.”

Behat - Database for test environment

Two concerns: performance and isolation

Performance

Isolation: each test data is only visible for that test.

How to setup the database before each test? Approaches:

Obelix doctrine commands - 3 minutes

Drop, create and add only structure using entities

MySQL via command line - 1 minute

Creating dump from dev with no data, drop, create and import dump

FASTER - MySQL via command line with only specific tables - 1 second

With three tables, create dump from dev, drop, create and import dump

Behat - Database data fixtures

app/config/automated_tests.yml has data fixtures config with tables and data files

Data files must exist in app/data/test/mysql

In the scenario we should add a tag with the data fixture we need

Basic authentication is always inserted in the database

Behat - Lifecycle request

REST APIin test environment

image: Symfony lifecycle request

Behatsends HTTP request

Behatasserts HTTP response

Feature Scenario

Before Scenario hooksetup database:- create an database dump- drop test database- create test database- import dump- import data

Given - preconditionWhen - action

Then - testable outcome is achieved

runs iterativelyLoop over features -> loop over scenarios of the feature

BehatEvent system

The future - Continuous Integration

Triggers test chain on each pull request

Verifies state before merge

Using the GitHub API to validate pull-request even before they are merged! https://developer.github.com/v3/repos/statuses/

Continuous Integration Best Practices(...)Make the build self-testingOnce the code is built, all tests should run to confirm that it behaves as the developers expect it to behave.

Lessons learned - Benefits and Difficulties

Commitment to TDD needed

Testing influences software design

Testing takes time

Full test coverage is almost unachievable

Testing is needed to ensure software quality

Questions?

Thanks