ece297 quick start guide unittest++ 1 introductionece297 communicationanddesign winter2019 ece297...

ECE297 Communication and Design Winter 2019

ECE297 Quick Start GuideUnitTest++

“Testing oneself is best when done alone."–Jimmy Carter

1 Introduction

This guide describes how to create tests with the UnitTest++ framework, and provides a tutorialon test-driven-development.

Unit tests offer several significant benefits during code development:

1. They automatically test the low-level functionality of a program.This gives us (the programmers) confidence that each piece of unit tested code is correct. Thisenables us to make changes to existing code without fear of introducing new hard-to-solvebugs. Any new bugs that are introduced will be caught early1, and close to its origin2 —provided we have good unit tests.

2. Writing tests before code allows us to experiment with an interface (function, class etc.) beforeperforming its time-consuming implementation.This process often helps us spot issues in the interface design (e.g. I should really make thistwo functions instead of one), before spending the time performing the detailed implementationand later having to go back and change it.

3. Writing tests forces us to think about and codify the requirements for an interface.This means that when we implement an interface we are coding against a specification (theunit tests) which we must satisfy. This helps to clarify exactly what actions an interfaceshould perform, often making it easier to implement.

Section 2 provides a step-by-step tutorial illustrating the development of a simple class withunit tests. Section 3 describes how unit tests can be run and failures debugged using Netbeans.Finally, Section 4 provides a reference for the different commands included as part of the UnitTest++framework.

2 UnitTest++ Tutorial: Money

This section describes the creation of a simple class, Money, which models both the amount (value)and the type (currency) of the money.

1The next time we run unit tests, which should be very often!2Since each unit test should exercise only a single piece of code the bug is likely in that piece of code.

of 14


2.1 Setup

Download the example project associated with this guide from the course website.It should contain a Makefile, along with the following source files:

1 #include <iostream>2

3 using namespace std;4

5 int main(int argc, char** argv) {6 cout << "This is " << argv[0] << endl;7 return 0;8 }

Listing 1: src/main.cpp

1 #include <iostream>2 #include <unittest++/UnitTest++.h>3

4 using namespace std;5

6 int main(int argc, char** argv) {7 cout << "This is " << argv[0] << endl;8

9 //Runs all defined unit tests10 int num_failures = UnitTest::RunAllTests();11

12 return num_failures;13 }

Listing 2: tests/tester.cpp

1 #include <unittest++/UnitTest++.h>2

3 TEST(HelloUnitTest) {4 CHECK(false);5 }

Listing 3: tests/test_hello.cpp

When we use unit tests, we use a different main() for our program. This different main function(Listing 2) sets up and runs unit tests on the internals (lower level functions) of our program. Whilethis is not very useful for end users it is very useful for us as programmers. Accordingly we canbuild the program in two modes:

• Normal mode with the regular main.cpp (Listing 1), or• Unit Test mode where the alternate main() function in tester.cpp (Listing 2) is used.

To run the unit tests open a terminal in the project directory and run make test on thecommand line:

1 > make test2 ...3 This is ./test_money4 tests/test_hello.cpp:4: error: Failure in HelloUnitTest: false

of 14


5 FAILURE: 1 out of 1 tests failed (1 failures).6 Test time: 0.00 seconds.

Listing 4: Running Unit Tests

We see that there is a failure in the unit test named HelloUnitTest at line 4 of tests/test_hello.cppwhich is shown in Listing 3. The CHECK() function generates a failure whenever its argument evalu-ates to false. Lets change the argument to true:

1 #include <unittest++/UnitTest++.h>2

3 TEST(HelloUnitTest) {4 CHECK(true);5 }

Listing 5: Updated tests/test_hello.cpp

Re-running our unit test we now see that it passes3:1 > make test2 ...3 This is ./test_money4 Success: 1 tests passed.5 Test time: 0.00 seconds.

Listing 6: Re-Running Unit Tests

2.2 Adding our First Test

First, lets create an initial (empty) Money class.1 class Money {2 //Empty3 };

Listing 7: Initial src/Money.h

One of the best practises for writing unit tests is to write the tests before the actual implementa-tion. Listing 8 shows a test for our Money class:

1 #include <unittest++/UnitTest++.h>2 #include "Money.h"3

4 TEST(MoneyConstructorAmount) {5 //Setup6 const std::string currency = "CAD";7 const float amount = 42.00;8

9 //Processing10 Money money(amount, currency);11

12 //Verification13 CHECK_EQUAL(amount, money.get_amount());14 }

3Aside from the final summary (Success: 1 tests passed.) passing unit tests are quiet — only failing testsproduce terminal output.

of 14


Listing 8: Initial tests/TestMoney.cpp

Every unit test performs three main actions:

1. It sets up the input data used in the test,

2. It performs some kind of processing which consumes the input data,

3. And finally, it verifies that the resulting output is correct.

If we now run our unit tests (make test), we see that it (rightly) fails to compile, since wehaven’t yet defined the implementation of the Money class:

1 tests/TestMoney.cpp:10:33: error: no matching function for call to ‘Money::Money(const float&,const string&)’

2 tests/TestMoney.cpp:13:31: error: ‘class Money’ has no member named ‘get_amount’

Listing 9: We will need to define these...

We can fix this by adding the constructor and get_amount() accessor function to Money.h:1 class Money {2 public:3 Money(const float amount, const std::string currency)4 : amount_(amount_) //This is a bug5 {6 }7

8 float get_amount() const {9 return amount_;

10 }11 private:12 float amount_;13 };

Listing 10: Defining constructor and get_amount() in src/Money.h

However, if we re-build and run our unit tests we get a failure!1 tests/TestMoney.cpp:13: error: Failure in MoneyConstructorAmount: Expected 42 but was 8.84939e-392 FAILURE: 1 out of 2 tests failed (1 failures).

Listing 11: The unit test caught the bug! Now we know the test works.

Within TestMoney.cpp (Listing 8) CHECK_EQUAL() verifies whether its two arguments are equal.Inspecting the test indicates that either the amount is not being initialized correctly in the constructor,or the wrong value is being returned. A quick study of the constructor shows we are not initializingamount_ correctly. We can fix that:

1 ...2 Money(const float amount, const std::string currency)3 : amount_(amount)4 {5 }6 ...

Listing 12: Squashing the bug in src/Money.h.

of 14


While it may seem odd that we purposefully inserted a bug in Listing 10, this is actually a goodidea. It allows us to verify that the test we have written will fail if given the wrong result. If thisdidn’t happen our test wouldn’t catch any real bugs and wouldn’t be very useful!

If we re-compile we see that the test now passes:1 Success: 2 tests passed.2 Test time: 0.00 seconds.

Listing 13: All is good.

2.3 Adding More Tests

So far we’ve only tested the amount functionality of our Money class. Lets add a new test for thecurrency functionality:

1 ...2 TEST(MoneyConstructorCurrency) {3 //Setup4 const std::string currency = "CAD";5 const float amount = 42.00;6

7 //Processing8 Money money(amount, currency);9

10 //Verification11 CHECK_EQUAL(currency, money.get_currency());12 }13 ...

Listing 14: Constructor currency test in tests/TestMoney.cpp

As before if we run our unit tests, we get an error that there is no get_currency() memberfunction. Let’s implement it:

1 #include <string>2

3 class Money {4 public:5 Money(const float amount, const std::string currency)6 : amount_(amount_)7 , currency_(currency)8 {9 }

10

11 float get_amount() const {12 return amount_;13 }14

15 std::string get_currency() const {16 return "invalid"; //This is wrong17 }18 private:19 float amount_;20 std::string currency_;21 };

Listing 15: Defining get_currency() in src/Money.h

of 14


If we re-test we should get an error, since get_currency() is returning “invalid” instead ofcurrency_. By making the test fail the first time we now know that the test is correct and doestrigger if given the wrong result. Fix the get_currency() method to return the currency_ memberand the test should pass.

2.4 Using Fixtures

The two tests that we have created so far contain common setup code highlighted in Listing 16:1 ...2 TEST(MoneyConstructorAmount) {3 const std::string currency = "CAD";4 const float amount = 42.00;5

6 Money money(amount, currency);7

8 CHECK_EQUAL(amount, money.get_amount());9 }

10

11 TEST(MoneyConstructorCurrency) {12 const std::string currency = "CAD";13 const float amount = 42.00;14

15 Money money(amount, currency);16

17 CHECK_EQUAL(currency, money.get_currency());18 }19 ...

Listing 16: Common setup code in tests/TestMoney.cpp

Luckily, we can re-factor these test to use a common test fixture class (in this case a struct) todo the initial data setup:

1 ...2 struct ConstructorFixture {3 ConstructorFixture()4 : currency("CAD")5 , amount(42.00) {6 }7 const std::string currency;8 const float amount;9 };

10

11 TEST_FIXTURE(ConstructorFixture, MoneyConstructorAmount) {12 Money money(amount, currency);13

14 CHECK_EQUAL(amount, money.get_amount());15 }16

17 TEST_FIXTURE(ConstructorFixture, MoneyConstructorCurrency) {18 Money money(amount, currency);19

20 CHECK_EQUAL(currency, money.get_currency());21 }22 ...

of 14


Listing 17: Using a fixture to re-use setup code in tests/TestMoney.cpp

In Listing 17 the ConstructorFixture performs the data initialization for each TEST_FIXTURE,since its class name is passed as the first argument. Within the TEST_FIXTUREs we can directlyaccess the fixture class’s member variables. This making it easy to re-use the setup code defined inConstructorFixture with both of our unit tests.

2.5 Money Operators

Lets extend the Money class so we can compare two money objects to see if they are equal. Firstthing we do is define a new test:

1 ...2 struct OperatorFixture {3 OperatorFixture()4 : currencyCAD("CAD")5 , currencyUSD("USD")6 , value42(42)7 , valuePi(3.14) {8 }9

10 const std::string currencyCAD;11 const std::string currencyUSD;12 const float value42;13 const float valuePi;14 };15

16 TEST_FIXTURE(OperatorFixture, TestEqual) {17 Money money42CAD(value42, currencyCAD);18 Money moneyPiUSD(valuePi, currencyUSD);19

20 CHECK(money42CAD == money42CAD);21 CHECK(moneyPiUSD == moneyPiUSD);22 }23 ...

Listing 18: Test for equality operator in tests/TestMoney.cpp

If we run our tests we’ll see it will fail since there is no equality operator defined. We canimplement it in Money.h:

1 ...2 class Money {3 public:4 ...5 friend bool operator==(const Money& lhs, const Money& rhs);6 ...7 };8

9 bool operator==(const Money& lhs, const Money& rhs) {10 return false; //This will fail11 }

Listing 19: Defining operator== in src/Money.h

of 14


Remember that we want the test to fail initially so we know the test will trigger if the codeproduces the wrong result. Building the project and confirming that the test works (i.e. it fails) wecan then go and properly implement operator==:

1 ...2 bool operator==(const Money& lhs, const Money& rhs) {3 return lhs.amount_ == rhs.amount_ && lhs.currency_ == rhs.currency_;4 }5 ...

Listing 20: Correctly implementing operator== in src/Money.h

Running the unit tests should confirm that all tests pass.Having defined an equality operator it is logical to also define inequality. We define the test first:

1 ...2 TEST_FIXTURE(OperatorFixture, TestNotEqual) {3 Money money42CAD(value42, currencyCAD);4 Money moneyPiCAD(valuePi, currencyCAD);5 Money money42USD(value42, currencyUSD);6

7 CHECK_EQUAL(false, money42USD != money42USD); //Equal8 CHECK(money42CAD != moneyPiCAD); //Different value9 CHECK(money42USD != money42CAD); //Different currency

10 CHECK(moneyPiCAD != money42USD); //Different value and currency11 }12 ...

Listing 21: Test for inequality operator in tests/TestMoney.cpp

We run our unit tests and note the build failure (inequality not defined) and add an implemen-tation:

1 class Money {2 public:3 ...4 friend bool operator!=(const Money& lhs, const Money& rhs);5 ...6 };7 ...8 bool operator!=(const Money& lhs, const Money& rhs) {9 return lhs == rhs; //Not going to work!

10 }

Listing 22: Defining operator!= in src/Money.h

Running the tests we should observe 4 failures (so we know each of the 4 CHECKs in Listing 21work correctly). We can then fix the implementation and re-run our tests:

1 ...2 bool operator!=(const Money& lhs, const Money& rhs) {3 return !(lhs == rhs);4 }

Listing 23: Correctly implementing operator!= in src/Money.h

of 14


2.6 Time Constraints

All of the tests we’ve created so far have only considered the functionality of the tested code. Wecan also use unit tests to check the performance of the tested code, by applying time constraints tounit tests:

1 TEST_FIXTURE(OperatorFixture, MoneyPerformance) {2 UNITTEST_TIME_CONSTRAINT(10); //In ms3 Money money42CAD(value42, currencyCAD);4 Money money42USD(value42, currencyUSD);5 Money moneyPiCAD(valuePi, currencyCAD);6 Money moneyPiUSD(valuePi, currencyUSD);7

8 CHECK(money42USD == money42USD);9 CHECK(money42CAD != moneyPiUSD);

10 CHECK(money42CAD != moneyPiCAD);11

12 UnitTest::TimeHelpers::SleepMs(50); //Going to run out of time!13 }

Listing 24: Equality performance test in tests/TestMoney.h

As usual we initially force the test to fail (to confirm it works):1 tests/TestMoney.cpp:81: error: Failure in TestPerformance: Time constraint failed. Expected to run

test under 10ms but took 50ms.2 FAILURE: 1 out of 6 tests failed (1 failures).

Listing 25: Too slow.

Having verified that the time constraint works we can remove the function call to UnitTest::TimeHelpers::SleepMs(50) and all tests should pass.

2.7 Using Suites

It is often useful to group related tests together, this can be done by creating a test suite, usingthe SUITE() command. The first argument of the SUITE() command is the name of the suite. Forinstance we can group our Constructor/Accessor tests into one suite, and the operator tests intoanother as shown in Listing 26.

1 #include <unittest++/UnitTest++.h>2 #include "Money.h"3

4 SUITE(MoneyConstructorAccessor) {5 struct ConstructorFixture {6 ConstructorFixture()7 : currency("CAD")8 , amount(42.00) {9 }

10 const std::string currency;11 const float amount;12 };13

14 TEST_FIXTURE(ConstructorFixture, MoneyConstructorAmount) {15 Money money(amount, currency);16

of 14


17 CHECK_EQUAL(amount, money.get_amount());18 }19

20 TEST_FIXTURE(ConstructorFixture, MoneyConstructorCurrency) {21 Money money(amount, currency);22

23 CHECK_EQUAL(currency, money.get_currency());24 }25 }26

27 SUITE(MoneyOperator) {28 struct OperatorFixture {29 OperatorFixture()30 : currencyCAD("CAD")31 , currencyUSD("USD")32 , value42(42)33 , valuePi(3.14) {34 }35

36 const std::string currencyCAD;37 const std::string currencyUSD;38 const float value42;39 const float valuePi;40 };41

42 TEST_FIXTURE(OperatorFixture, TestEqual) {43 Money money42CAD(value42, currencyCAD);44 Money moneyPiUSD(valuePi, currencyUSD);45

46 CHECK(money42CAD == money42CAD);47 CHECK(moneyPiUSD == moneyPiUSD);48 }49

50 TEST_FIXTURE(OperatorFixture, TestNotEqual) {51 Money money42CAD(value42, currencyCAD);52 Money moneyPiCAD(valuePi, currencyCAD);53 Money money42USD(value42, currencyUSD);54

55 CHECK_EQUAL(false, money42USD != money42USD); //Equal56 CHECK(money42CAD != moneyPiCAD); //Different value57 CHECK(money42USD != money42CAD); //Different currency58 CHECK(moneyPiCAD != money42USD); //Different value and currency59 }60 }

Listing 26: Grouping related tests into suites

3 Running and Debugging Unit Tests in Netbeans

This guide has so far described how to run unit tests from the command line, they can also be runand debugged from within Netbeans.

of 14


Figure 1: Running unit tests in Netbeans.

Figure 2: Place a breakpoint by clicking the associated line number (55 in this case). The linenumber is replaced with a red square to indicate the breakpoint has been set.

3.1 Running Unit Tests in Netbeans

As shown in Figure 1, to run unit tests from within netbeans select ‘Unit_Test’ as the buildconfiguration and then click the ‘Run’ button. This will build and run the project’s unit tests. Anyunit test output will be shown in the netbeans terminal.

Figure 3: Select the highlighted button to run the debugger.

of 14


3.2 Debugging Failed Unit Tests in Netbeans

To debug a failing unit test in Netbeans first configure netbeans to run unit tests (Section 3.1).Next open the source file containing the unit test and create a breakpoint on the offending line (e.g.line 55 in Figure 2). Finally run netbeans in debug mode by clicking the ‘Debug Project’ button(Figure 3)4. The debugger will stop at the breakpoint, and you can then investigate the state ofvariables and step into any function calls.

For more details on using the debugger see the Netbeans Quick Start Guide.

4 UnitTest++ Command Reference

This section describes the commands provided by the UnitTest++ framework.

Table 1: TEST and SUITE reference.

Command Usage Description

1 TEST(test_name) {2 //Test Body3 }

Defines a test named test_name.

The test body (between curly braces) cancontain CHECK calls and C++ code.

1 TEST_FIXTURE(fixture_name, test_name) {2 //Test Body3 }

Defines a test named test_name which usesthe class named fixture_name as a testfixture.

The test body (between curly braces) cancontain CHECK calls and C++ code. Publicmembers of the fixture_name class can beaccessed directly in the test body.

1 SUITE(suite_name) {2 //Suite Body3 }

Defines a test suite named suite_name.

The suite body can contain multiple TESTsand/or TEST_FIXTUREs.

Table 2: CHECK and TIME_CONSTRAINT reference.


4The GDB debugger used by netbeans can also be used from the command-line by running gdb –args exec ,where exec is the name of the unit test executable.

of 14




1 CHECK(expression);

Generates a failure if expression does notevaluate to true.

1 CHECK_EQUAL(expected, actual);

Generates a failure if expected does notequal actual.

Requires equality (operator==) to be de-fined between expected and actual.Requires stream output (operator«) oper-ators to be defined for expected and actual.

1 CHECK_CLOSE(expected, actual, tolerance);

Generates a failure if actual does not equalexpected ± tolerance.

1 CHECK_ARRAY_EQUAL(expected, actual, count);

Generates a failure if any of the count el-ements in the array actual are not equalto the equivalent element in the array ex-pected.

1 CHECK_ARRAY_CLOSE(expected, actual, count,tolerance);

Generates a failure if any of the count ele-ments in the array actual are not equal (±tolerance) to the equivalent element in thearray expected.

1 CHECK_ARRAY2D_CLOSE(expected, actual, rows, columns,tolerance);

Generates a failure if any of therow×columns elements in the 2D array ac-tual are not equal (± tolerance) to the equiv-alent element in the 2D array expected.

1 CHECK_THROW(expression, exception_type);

Generates a failure if expression doesnot generate an exception of type excep-tion_type.

of 14




1 {2 UNITTEST_TIME_CONSTRAINT(time);3 //Timed Code4 }

Generates a failure if the code betweenUNITTEST_TIME_CONSTRAINT and the endof scope (closing curly brace) takes longerthan time milliseconds.

5 Acknowledgements

This document is largely based on “Money, a step by step example" by Matthew Granic, and“UnitTest++ in brief”.

of 14

https://www.ultimatepp.org/srcdoc$UnitTest++$UnitTestpp$en-us.html

ece297 quick start guide unittest++ 1 introductionece297 communicationanddesign winter2019 ece297...

Documents