building block for test-driven development of …...2010/11/18 · workshop tdd4es 18/11/2010 ep 1...
TRANSCRIPT
Workshop TDD4ES
18/11/2010
EP 1
iwt-090191 TDD4ES
Workshop
Building blocks for
Test-Driven Development of Embedded Software
18/11/2010
dr. ing. Jeroen Boydens
ing. Piet Cordemans
ing. Sille Van Landschoot
ing. Wim Catteeuw
@ep_research http://ep.khbo.be
18/11/2010 TDD4ES
Content
Session 1 (Tuesday 16/11/2010)
1. Introduction
2. TDD in practice
3. TDD for embedded software
Session 2 (Thursday 18/11/2010)
1. Test on target (continued)
2. Test on host
3. Remote testing 2
Workshop TDD4ES
18/11/2010
EP 2
18/11/2010 TDD4ES
Session 1 recap
3
Tests
Test on target
Test on host
Remote testing
18/11/2010 TDD4ES
Content: session 2
1. Test on target 1. 3-tier TDD
2. Hardware-aware TDD
3. Hardware-specific TDD
4. Pros-cons
5. Do it yourself
6. Exercise
7. Mocking
4
Workshop TDD4ES
18/11/2010
EP 3
18/11/2010 TDD4ES
Content: session 2 cont’d
2. Test on host 1. Test on host in practice
2. 3-tier TDD
3. Mock verification
4. Embedded TDD cycle
5. Pros-cons
6. Do it yourself
7. Exercise
5
18/11/2010 TDD4ES
Content: session 2 cont’d
3. Remote testing 1. Remoting
2. Remote testing in practice
3. Pros-cons
4. Do it yourself
5. Exercise
6
Workshop TDD4ES
18/11/2010
EP 4
18/11/2010 TDD4ES
1 Test on target
7
Tests
1. 3-Tier TDD
2. Hardware-aware TDD
3. Hardware-specific TDD
4. Pros-cons
5. Do it yourself
6. Exercise
7. Mocking
18/11/2010 TDD4ES
1.1 3-Tier TDD
8
Hardware aware
Hardware specific
• Relative simple unit & integration tests
• Unit test isolation (with stubs)
• Automation of hardware events
Workshop TDD4ES
18/11/2010
EP 5
18/11/2010 TDD4ES
1.2 Hardware-aware TDD
9
1. Identify hardware-aware functionality
2. Write test first
1. Select test values
2. Add stub replacing driver input
3. Write assertions
3. Add minimal implementation
4. Pass test
5. Refactor
18/11/2010 TDD4ES
1.3 Hardware-specific TDD
10
• Follow hardware-aware process
– Writing test-first can be time consuming
– Leads to brittle tests
OR
• Write driver first
Then encapsulate driver behavior with tests
– Treats drivers as third party code
– Changes are quickly notified
Workshop TDD4ES
18/11/2010
EP 6
18/11/2010 TDD4ES
1.4 Pros-cons
11
Advantages
• No cross-compiling
Disadvantages
• Isolating hardware behavior typically enlarges test cases (stubs)
- Processing power
- Memory footprint
• Slow development cycle
- Frequent flash programming
- …
18/11/2010 TDD4ES
1.5 Do it yourself
12
DS1820 - sign - magnitude
+ read() + getTemperature(): int
temperature sensor driver
hardware dependency
DS1820 # sign # magnitude
+ read() + getTemperature(): int
minimal changes quick & dirty
DS1820Mock + read() + setSign(int) + setMagnitude(int)
stub implementation
Workshop TDD4ES
18/11/2010
EP 7
18/11/2010 TDD4ES
1.5 Do it yourself
13
DS1820 + getState() + getTemperature(): int
state pattern: a bit more work & memory
hardware dependency is replaced by a state object
DS1820State + read()
DS1820StateMock + read() + setSign(int) + setMagnitude(int)
DS1820StateReal + read()
18/11/2010 TDD4ES
1.5 Do it yourself
1. Create a new mock class
2. Adapt the interface
3. Implement the mock
4. Write test for class with mock
14
Workshop TDD4ES
18/11/2010
EP 8
18/11/2010 TDD4ES
1.6 Exercise
15
04_TDDonTarget_TempMockStart
Requirements • Button class (in the drivers folder)
• Test the getState method
• Mock the hardware dependent method
Goals • Isolating hardware dependent functionality
• Test automation
18/11/2010 TDD4ES
1.6 Exercise
16
Button # getCurrentState
+ getState : ButtonState
Button driver hardware
dependency
Button # getCurrentState
minimal changes quick & dirty
ButtonMock + getCurrentState + setCurrentState
stub implementation
ButtonStates: RELEASED, PRESSED, REPEAT
Workshop TDD4ES
18/11/2010
EP 9
18/11/2010 TDD4ES
1.7 Mocking
• Software stub replacing the hardware
• Isolates the unit under test
• Mock object -> also validates interaction with the stub
1. Polymorphism
2. Preprocessor directives
3. Function pointers
4. Link-time polymorphism
17
18/11/2010 TDD4ES
1.7 Mocking in C++
1. Polymorphism
Advantages
• Minimal changes to source code
• No code duplication
Disadvantages
• Introduces overhead in production code
18
DS1820 # sign # magnitude
+ read() + getTemperature(): int
DS1820Mock + read() + setSign(int) + setMagnitude(int)
Workshop TDD4ES
18/11/2010
EP 10
18/11/2010 TDD4ES
2. Preprocessor directives
19
1.7 Mocking in C
Conditional compiling per file Conditional compiling per project
Disadvantage • Code duplication ->
unscalable
#ifdef TESTONHOST
#include “mockdriver.c”
#endif
#ifndef TESTONHOST
#include “realdriver.c”
#endif
#ifdef TESTONHOST
#define functionUT(arg1, arg2)
#endif
/* code containing function to be
mocked */
#ifndef TESTONHOST
#include “unittestframework.h”
int main() {
/* run unit tests & report*/
}
#endif
18/11/2010 TDD4ES
1.7 Mocking in C
3. Function pointers
Disadvantages
• Introduces overhead in production code
• Less scalable than polymorphism
20
/* list of functionsToMock with real or
mock address */
struct functionsToMock
{
void (*function1)(arg1);
}
extern struct functionsToMock FTM;
(*FTM.function1)(arg1);
Workshop TDD4ES
18/11/2010
EP 11
18/11/2010 TDD4ES
4. Link-time polymorphism
• Own responsibility to present only one implementation to the linker
21
1.7 Mocking in C
/* Function.h*/
void FunctionName(void);
/* Function.c*/
void FunctionName(void)
{
/*real implementation*/
}
/* FunctionMock.c*/
void FunctionName(void)
{
/*fake implementation*/
}
18/11/2010 TDD4ES
1.7 Mocking in C
4. Link-time polymorphism
Advantages
• No changes to source code
• No overhead in production code
Disadvantages
• Separate build configurations
• Manual selection of build
Demonstration with Eclipse
22
Scripting or IDE
Workshop TDD4ES
18/11/2010
EP 12
18/11/2010 TDD4ES
2 Test on host
23
1. Test on host in practice
2. 3-tier TDD
3. Mock verification
4. Embedded TDD cycle
5. Pros-cons
6. Do it yourself
7. Exercise
18/11/2010 TDD4ES
2.1 Test on host in practice
24
Tests
Program
Test Framework
Interfaces
Mock driverReal driver
Workshop TDD4ES
18/11/2010
EP 13
18/11/2010 TDD4ES
2.2 3-Tier TDD
25
Hardware aware
Hardware specific
• Relative simple unit & integration tests
• Mock hardware drivers
• Mock hardware interaction (per case ROI analysis)
OR
• Encapsulate layer with tests
18/11/2010 TDD4ES
2.3 Mock verification
26
1. Develop test on host
– Mock hardware
– Correspondence with driver on target
2. Migrate test to target
– Mock is replaced with real hardware driver
Test passes: mock & implementation = OK
OR
Test fails: mock & implementation != OK
Workshop TDD4ES
18/11/2010
EP 14
18/11/2010 TDD4ES
2.4 Embedded TDD cycle
27
1. Red bar 2. Green bar 3. Refactor
1. Compile for target
2. Fix
1. Unit tests in target
2. Fix
1. Manual tests
2. Fix
+/- 10 minutes
Host Target
Few hours Few days Daily
© J. Grenning, 2004, Progress before hardware
18/11/2010 TDD4ES
2.5 Pros-cons
28
Advantages
• Fast programming cycle
• Hardware does not need to be available
• Migration supported by test suite
Disadvantages
• Verification on target needed
– Cross-compilation issues
– Mock driver verification
Workshop TDD4ES
18/11/2010
EP 15
18/11/2010 TDD4ES
2.6 Do it yourself
29
1. Develop on host
1. Write a test for the driver
2. Define an interface
3. Implement a mockdriver
4. Use the mockdriver in the program
5. Write a test for program functionality
6. Implement program functionality
7. Run tests on host
2. Verify on target
1. Run tests on target
18/11/2010 TDD4ES
2.7 Exercise
30
05_TDDonHost
Requirements • Test the Button class on host
• Mock the Button class
• Verify on target
• Test the Button in the Program on host
• Verify on target
Goals • TDD on host
• Driver verification
Workshop TDD4ES
18/11/2010
EP 16
18/11/2010 TDD4ES
3 Remote testing
31
1. Remoting
2. Remote testing in practice
3. Pros-cons
4. Do it yourself
5. Exercise
18/11/2010 TDD4ES
3.1 Remoting
32
Background
• Inter Process Communication (IPC)
• Remote procedure call (RPC)
• Common Object Request Broker Architecture (CORBA)
Operation
• Tests / framework on host
• Program / drivers on target
• Management mechanism on both
Workshop TDD4ES
18/11/2010
EP 17
18/11/2010 TDD4ES
SKELETONSSKELETONS
STUBSSTUBS
3.2 Remote testing in practice
33
target host
Program Drivers Tests
Test Framework
TDD
BROKER COMMUNICATION CHANNEL BROKER
STUBSSKELETONS
Architecture
18/11/2010 TDD4ES
3.2 Remote testing in practice
34
Operation
target host
C L A S S
T E S T S
B R O K E R
init broker, init runner
init broker init
setup
run
tear-down
BROKER
LIST
ID Class
Stub
ID
Skeleton
OBJ
Workshop TDD4ES
18/11/2010
EP 18
18/11/2010 TDD4ES
3.3 Pros-cons
35
Advantages
• Scalable alternative for test on target (ROI)
• Fast prototyping
– Stable driver base on target
– On host development of higher level tiers
Disadvantages
• Remote testing system
– Development
– Overhead: memory / processing power
18/11/2010 TDD4ES
3.4 Do it yourself
36
• Hello test framework: (MSUnit)
Visual Studio Unit Testing Framework
using namespace Microsoft::VisualStudio::TestTools::UnitTesting;
[TestClass]
public ref class UnitTest1
{
[TestMethod]
public void MyTest()
{
Assert::IsTrue(true);
}
}
Workshop TDD4ES
18/11/2010
EP 19
18/11/2010 TDD4ES
Assert::IsTrue(true); //succeeds
Assert::AreEqual(1, 2); //fails
Assert::AreEqual(1.25, 2.35, 0.1); //fails
Assert::IsNotNull(object); //succeeds
37
• Grouping
• Setup & Teardown
• Assertions
3.4 MSUnit
[TestClass]
public ref class UnitTest1{}
[ClassInitialize()]
static void MyClassInitialize
(TestContext^ testContext) {/*setup*/};
[TestInitialize()]
void MyTestInitialize() {/*teardown*/};
18/11/2010 TDD4ES 38
3.4 Managed runtime
O/S
.Net Runtime GC
C++/CLI • ref class • ^ heap pointer • gcnew • IDisposable • %
C++ • class • * stack pointer • new • delete • &
Managed Unmanaged
ref class Person{}
Person ^p = gcnew Person();
class Person{}
Person *p = new Person();
Workshop TDD4ES
18/11/2010
EP 20
18/11/2010 TDD4ES
3.4 Do it yourself
39
06_RemoteTesting_DiY • Write new test on host • Run test on target • Review results on host
18/11/2010 TDD4ES
3.5 Exercise
40
06_RemoteTesting
Requirements • Testing target implementation on host
• Led test
• DS1820 test (environment temperature)
Goals • TDD with remote testing
• Testing hardware aware functionality
• Testing hardware independent functionality
Workshop TDD4ES
18/11/2010
EP 21
18/11/2010 TDD4ES
3.5 Exercise
41
06_RemoteTesting
LED + on + off + get : int + set(int)
DS1820 + getTemperature : int + double getTemperature : double
• Led test • DS1820 test (environment temperature)
18/11/2010 TDD4ES
TDD4ES overview
42
Test on target
Test on host
Remote testing
1. Limited number of tests
2. Minimalistic framework
3. Program code
1. Unlimited number of tests
2. Full option framework
3. Program code with hardware virtualization
1. Unlimited number of tests
2. Full option framework with remote testing functionality
3. Program code with remote testing functionality
Workshop TDD4ES
18/11/2010
EP 22
18/11/2010 TDD4ES
TDD4ES overview
43
Memory
Dependency
Slow
Test on host Test on target Remote testing
+++ Test & Program
on host
+++ Real drivers
+++ Real drivers
+++ Development limited to host
+/- Program &
Broker on target
--- Test & Program
on target
--- Only virtual
drivers
--- Frequent flash programming
- Frequent flash programming*
18/11/2010 TDD4ES
Literature
• K. Beck. Test-Driven Development: By Example. Addison-Wesley, 2003.
• K. Beck and C. Andres. Extreme Programming Explained: Embrace Change (2nd ed.). Addison-Wesley, 2004.
• B.W. Boehm. Software Engineering Economics (Prentice-Hall Advances in Computing Science & Technology Series). Prentice Hall PTR, October 1981.
• J. Boydens, P. Cordemans, and E. Steegmans. Test-Driven Development of Embedded Software. In European Conference on the Use of Modern Information and Communication Technologies, 2010.
• E. Dijkstra. The humble programmer. Commun. ACM, 15(10):859866, 1972.
• M. Feathers. Working Effectively with Legacy Code. Prentice Hall PTR, 2005.
• M. Fowler. UML Distilled (3rd ed.). Addison-Wesley, 2004.
• M. Fowler. Refactoring: Improving the design of existing code. Addison-Wesley, 1999.
• E. Gamma, R. Helm, R. Johnson, and J. Vlissides. Design patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley, 1995.
• J. Grenning. Progress before hardware. Agile Alliance Newsletter, 4:74_79, 2004.
44
Workshop TDD4ES
18/11/2010
EP 23
18/11/2010 TDD4ES
Literature continued
• M. Karlesky, W. Bereza, and C. Erickson. Effective test driven development for embedded software. In IEEE 2006 Electro/Information Technology Conference, 2006.
• R. Koss, and J. Langr. Test-Driven Development in C. In C/C++ Users Journal, 2002.
• M. Müller and F. Padberg. About the return on investment of test-driven development. In International Workshop on Economics-Driven Software Engineering Research EDSER-4, 2003.
• R. Osherove. The Art of Unit Testing. Manning, 2009.
45
18/11/2010 TDD4ES
Thank you
Follow-up e-mail
– Publish handouts & code
– List of participants
– Evaluation form
Follow project on http://ep.khbo.be/TDD4ES
46