concjunit: unit testing for concurrent programs pppj 2009 mathias ricken and robert cartwright rice...

ConcJUnit:Unit Testing for Concurrent Programs

PPPJ 2009

Mathias Ricken and Robert Cartwright

Rice University

August 28, 2009

2

Brian Goetz, Java Concurrency in Practice, Addison-Wesley, 2006

3

Concurrency Practiced Badly

1

4

Unit Tests…

Occur early Automate testing Keep the shared repository clean Serve as documentation Prevent bugs from reoccurring Allow safe refactoring

5

Existing Unit Testing Frameworks

JUnit, TestNG

Don’t detect test failures in child threads Don’t ensure that child threads terminate

Tests that should fail may succeed

6

ConcJUnit

Replacement for JUnit Backward compatible, just replace junit.jar file Available at www.concutest.org

1. Detects failures in all threads

2. Warns if child threads outlive main thread

3. Warns if child threads are not joined

7

Sample JUnit Testspublic class Test extends TestCase { public void testException() { throw new RuntimeException("booh!"); } public void testAssertion() { assertEquals(0, 1); }}

if (0!=1) throw new AssertionFailedError();

}} Both tests fail.

Both tests fail.

8

JUnit Test with Child Threadpublic class Test extends TestCase { public void testException() { new Thread() { public void run() { throw new RuntimeException("booh!"); } }.start(); }}

new Thread() { public void run() { throw new RuntimeException("booh!"); }}.start();

throw new RuntimeException("booh!");

Main thread

Child thread

Main thread

Child thread

spawns

uncaught!

end of test

success!

9




Uncaught exception, test should fail but

does not!

By default, no uncaught exception handler installed for child threads

10

Changes to JUnit (1 of 3) Thread group with exception handler

JUnit test runs in a separate thread, not main thread Child threads are created in same thread group When test ends, check if handler was invoked

Reasoning: Uncaught exceptions in all threads must cause

failure

11




Test thread

Child thread

uncaught!

end of test

Main thread

spawns and joins resumes

check group’s handler

invokes group’s handler

failure!

12

Child Thread Outlives Parentpublic class Test extends TestCase { public void testException() { new Thread() { public void run() { throw new RuntimeException("booh!"); } }.start(); }}



Test thread

Child thread

uncaught!end of test

success!


Main thread


Too late!

13

Changes to JUnit (2 of 3)

Check for living child threads after test ends

Reasoning: Uncaught exceptions in all threads must cause

failure If the test is declared a success before all child

threads have ended, failures may go unnoticed Therefore, all child threads must terminate

before test ends

14

Check for Living Child Threadspublic class Test extends TestCase { public void testException() { new Thread() { public void run() { throw new RuntimeException("booh!"); } }.start(); }}



Test thread

Child thread

uncaught!end of test

failure!


Main thread

check for livingchild threads


15

Correctly Written Testpublic class Test extends TestCase { public void testException() { Thread t = new Thread() { public void run() { /* child thread */ } }; t.start(); t.join(); }}

Thread t = new Thread() { public void run() { /* child thread */ }};t.start();t.join(); // wait until child thread has ended

/* child thread */

Test thread

Child thread

end of test

success! Main thread

check for livingchild threads


4

16

Changes to JUnit (3 of 3)

Check if any child threads were not joined

Reasoning: All child threads must terminate before test ends Without join() operation, a test may get “lucky” Require all child threads to be joined

17

Fork/Join Model

Parent thread joins with each of its child threads

May be too limited for a general-purpose programming language

Child thread 1

Child thread 2

Main thread

18

Example of Other Join Models

Chain of child threads guaranteed to outlive parent

Main thread joins with last thread of chain

Child thread 1

Child thread 2

Main thread

Child thread 3

19

Generalize to Join Graph

Threads as nodes; edges to joined thread Test is well-formed as long as all threads

are reachable from main thread

Child thread 1

Child thread 2

Main thread

Child thread 3

MT

CT1

CT2

CT3

20

Child thread 1

Child thread 2

Main thread

MT

CT1

CT2

Unreachable Nodes

An unreachable node has not been joinedChild thread may outlive the test

21

childThread

main Thread

MT

CT

Constructing the Graph: start()

// in mainThreadchildThread.start();

Add node for childThread

22

// in mainThreadchildThread.join();

When leaving join(), add edge from mainThread to childThread

childThread

main Thread

MT

CT

Constructing the Graph : join()

3

23

Modifying the Java Runtime

Changing Thread.start()and join()Need to modify Java Runtime LibraryUtility to process user’s rt.jar filePut new jar file on boot classpath:-Xbootclasspath/p:newrt.jar

Still works without modified Thread classJust does not emit “lucky” warnings

24

Evaluation JFreeChart

All tests passed; tests are not concurrent DrJava: 900 unit tests

Passed: 880No join: 1Lucky: 18Timeout: 1Runtime overhead: ~1 percent

25

Limitations Only checks chosen schedule

A different schedule may still fail

Example:

Thread t = new Thread(…);if (nondeterministic()) t.join();

2

26

Future Work

Insert sleeps or yields in critical code locations Example: If a notify() is delayed, a

wait() may time out. Can detect a number of sample problems

Run tests several times on build server Record schedule, replay if test fails

Makes failures reproducible if found

5

27

ConcJUnit Demo

28

Thank you!

www.concutest.org

29

Notes

30

Notes

1. Probably not caused by concurrency problems; just a metaphor. ←

2. Also cannot detect uncaught exceptions in a program’s uncaught exception handler (JLS limitation) ←

3. Only add edge if joined thread is really dead; do not add if join ended spuriously. ←

31

public class Test extends TestCase { public void testException() { Thread t = new Thread(new Runnable() { public void run() { throw new RuntimeException("booh!"); } }); t.start(); while(t.isAlive()) { try { t.join(); } catch(InterruptedException ie) { } } }}

Thread t = new Thread(new Runnable() { public void run() { throw new RuntimeException("booh!"); }});t.start();while(t.isAlive()) { try { t.join(); } catch(InterruptedException ie) { }}


Loop since join() may end

spuriously

4.

Spurious Wakeup

←

32

Notes

5. Have not studied probabilities or durations for sleeps/yields:

One inserted delay may negatively impact a second inserted delayExample: If both notify() and wait() are delayed. ←

33

Extra Slides

34

JUnit Test with Child Threadpublic class Test extends TestCase { public void testException() { new Thread(new Runnable() { public void run() { throw new RuntimeException("booh!"); } }).start(); }}

new Thread(new Runnable() { public void run() { throw new RuntimeException("booh!"); }}).start();


invokeschecks

TestGroup’s Uncaught Exception Handler

35

Join with All Offspring Threads

Main thread joins with all offspring threads, regardless of what thread spawned them

Child thread 1

Child thread 2

Main thread

36

Child thread 1

Child thread 2

Main thread MT

CT1

CT2

Child thread 1

Child thread 2

Main thread

MT

CT1

CT2

Join Graph Examples

37

Thread Creation Context

In Thread.start()– also record stack trace of Thread.currentThread(Easy to find source code of a child thread that

is not joined

Also available for uncaught exception stack traces

38

Creation Context Example

class Main { void foo() { // which one? new Helper().start(); new Helper().start(); // ... }}

AssertionError:

at Helper.m(Helper.java:2)

at Helper.run(Helper.java:3)

Started at:

at Main.foo(Main.java:4)

at Main.bar(Main.java:15)

at Main.main(Main.java:25)

class Helper extends Thread {

void m() { Assert.fail(); }

public void run() { m(); }

}

39

Many Thanks To…

My advisor Corky Cartwright

My committee members Walid Taha David Scott Bill Scherer

NSF and Texas ATP For providing partial funding

40

Concurrency Problems NotFound During Testing

1

concjunit: unit testing for concurrent programs pppj 2009 mathias ricken and robert cartwright rice...

Documents