how to test asynchronous code

How to Test Asynchronous Code

by Felix Geisendörfer

05.05.2011 (v1)

@felixge

Twitter / GitHub / IRC

Core Contributor

&

Module Author

node-mysql node-formidable

- Joined the mailing list in June 26, 2009- When I joined there where #24 people- Now mailing list has close to 4000 members members- First patch in September 2009

Story Time

~ June 2009

File uploading & processing as an infrastructure service for web & mobile applications.

- The app we run in production

We found node

We found node

TM

We fell in love

We had fun

We crashed- Or more specifically: node crashed

Transloadit v1

• Complete failure

• Node randomly crashed with:

(evcom) recv() Success

• A mess of code, hard to maintain

~1.5 Years ago

* http://transloadit.com/blog/2010/04/to-launch-or-not

- 0.1.18- New version of node took out promises- So we thought about refactoring, but ...

We had no tests

Time to rewrite- RailsConf 09: Robert Martin, "What Killed Smalltalk Could Kill Ruby, Too

How to Test Asynchronous

Code

- We looked at asynchronous testing frameworks- We really wanted to do TDD- But everything we tried was hard

db.query('SELECT A', function() { db.query('SELECT B', function() { db.query('SELECT C', function() { db.query('SELECT D', function() { // WTF }); }); });});

- Who here is running node in production?- Raise hands if you have a test suite for your node.js projects- Raise hands if you are happy with it- But why is it so difficult?

Some old news

Fold news paper 7 timesAsk audience to guess the numbers of stacked sheets of papersAsk person to guess right (128) to come up front to rip through the folded paper.

What’s this about?

- Newspaper can be thought of as a simple computer program managing 7 booleans- Each boolean has a meaningful impact on the behavior of the Software- This means that ripping through it at once, with a single, becomes very difficult- The problem becomes even worse when your variables have more than 2 states-> the euquation turns from being exponential into combinatoric one -> combinatoric explosion

How to Test Asynchronous

Code

Make it “synchronous”

Version 2

Well, almost : )

Transloadit v2

• Processed many TB of data

• Very low bug rate

• Clean code base, ~99% test coverage

Now

How to MicrotestNode.js Code

Term microtest vs Unit Test

Rule #1: No I/O

Rule #2: TDD

Rule #3: Discipline

Libraries

Numbers

Transloadit

Transloadit

0

3.250

6.500

9.750

13.000

library tests

Lines of code

library vs test code: 1x : 2.04x

Library: 6184 LoCTests: 12622 LoC

Not included: Fixture data, server configuration, customer website, dependencies we developed

Transloadit

0

2.500

5.000

7.500

10.000

integration micro

integration vs. micro tests: 1x : 4.30x

Lines of code

Integration tests: 2254 LoCMicro Tests: 9695 LoC

Transloadit

0

750

1.500

2.250

3.000

integration micro

Assertions

integration vs. micro tests: 1x : 12.30x

Integration tests: 189 assertsMicro Tests: 2324 asserts

node-mysql

node-mysql

0

425

850

1.275

1.700

library tests

Lines of code

library vs test code: 1x : 1.35x

Library: 1240 LoC (+600 LoC MySql constants)Tests:Tests: 1673 LoC

node-mysql

0

325

650

975

1.300

integration micro

integration vs micro tests: 1x : 2.89x

Lines of code

Micro Tests: 1243 LoCIntegration tests: 430 LoC

node-mysql

0

100

200

300

400

integration micro

Assertions

integration vs micro tests: 1x : 8.15x

Micro Tests: 375 assertsIntegration tests: 46 asserts

Why do you have brakes in a car? Because then you can drive faster.

Kevlin Henney

probably Kevlin Henney, http://olvemaudal.wordpress.com/2008/04/14/being-agile-no-speeding-please/

There really are only two acceptable models of development: "think and analyze" or "years and years of testing on thousands of machines".

Linus Torvalds

tl;dr

• Don’t use integration tests to show the basic correctness of your software.

• Write more microtests.

Questions?

Questions?

☝

Thank You

Thank You