the test pyramid

How to balance your test suite to have a good feedback, from units to end-to-end tests.

Thiago Ghisi@thiagoghisi

The Test

Pyramid

2002 2007 2010

Tests, seriously? It’s a mobile app...[ ]

Shared codebases108

Devs, QAs and Ops 144

Brian Marick@marick

January, 2009

http://www.slideshare.net/jezhumble/creating-maintainable-automated-acceptance-tests

Mike Cohn@mikewcohnOctober, 2009

http://www.slideshare.net/jezhumble/creating-maintainable-automated-acceptance-tests

http://www.slideshare.net/jezhumble/creating-maintainable-automated-acceptance-tests

Jez Humble@jezhumbleAugust, 2010

Feedback

Perfect!!! Let’s follow the test pyramid guidelines…[ ]

20% of code coverage. :-(

really hard to isolate the dependencies. :-(

easy. :-)

FunctionalTests • UI/End-to-end testing [with full

stack]

• Tests user flow

• BDD, ATDD

• Feedback after a coffee

business likes that ;)

sounds perfect

Cucumber feature file

Honeydew steps

Cucumber

Honeydew

wraps UIAutomator

step definitions in Ruby

+=]

☺

https://github.com/amplify-education/honeydew

why cucumber?https://speakerdeck.com/nicholaspufal/why-bdd-is-misunderstood

Cool.. write as much user-like tests as you can… PERFECT!![ ]

UI test results history (1)

UI test results history (2)

UI test results history (3)

Flaky / Fragile

Slow

Costy

Feedback

the build monkey

http://watirmelon.com/tag/software-testing-pyramid/

Few UI tests

OK… let’s do it right now…[ ]

Unit Test

• Verify small pieces individually

• Run fast

• Stable

• Run a lot of times during development

• Feedback instantly

Unit tests results history

Feedback

Android? It’s Java, let’s TDD.

Android TDD

java.lang.RuntimeException: Stub!

Android SDK classes are not trivial

eyes on docs

huge API

test object

result()

assert(expected, obj.result())

Humm, I’ve seen that..

The ‘2 + 2 = 4’ example!

the object u’re looking at

dependencies

getData()

saveSomething()

getData()

saveSomething()

when(stub.getData()).thenReturn(data)

verify(mock).saveSomething()

http://martinfowler.com/articles/mocksArentStubs.html

mockstubdoublefake

dummy

Unit Testswith

Dependency injection

Roboguice

Robolectric

jUnit

DI

Mocks

Android Shadows

@Test

Lots of Unit tests

Where are you?

http://www.mountaingoatsoftware.com/blog/the-forgotten-layer-of-the-test-automation-pyramid

http://fabiopereira.me/blog/2012/03/18/introducing-depth-of-test-dot/

external services

backend

libs

API contracts validation

PACTO

https://github.com/thoughtworks/pactohttps://speakerdeck.com/marcosccm/evoluindo-ecossistemas-com-consumer-driven-contracts

Service/Integration tests instead of end-to-end tests

Annnnd.. the manual tests?[ ]

Session based exploratory/usability

tests

http://blog.kalistick.com/Agile/harden-testing-pyramid/

http://watirmelon.com/2011/06/10/yet-another-software-testing-pyramid/

http://blog.softed.com/2013/08/26/1883/

UserJourney

UnitTests of single methods without dependencies

IntegrationTests of multiple integrated method

UI

Test automation

Testing on the level of the users - incl. browser, network, databases, external systems, …

Connection of UI tests to a persona like flow through

the system

$$$slowerfewer

$fastermore

closer to the user

closer to the developer

https://my.thoughtworks.com/docs/DOC-21002

http://www.slideshare.net/nashjain/inverting-the-testing-pyramid

http://www.slideshare.net/guestc8adce/pivotal-labs-open-view-presentation-quality-assurance-and-developer-testing-presentation

http://blog.codeclimate.com/blog/2013/10/09/rails-testing-pyramid/

Feedback

Feedback

http://herdingcats.typepad.com/my_weblog/2011/02/a-radical-book.html

Thank you! ;)

THIAGO GHISI@thiagoghisitghisi@thoughtworks.com

Questions?

Feedback?

A beer?