testing concurrent programs, a 7-minute jargon-free introduction thesis writing seminar mathias...
TRANSCRIPT
![Page 1: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010](https://reader030.vdocuments.site/reader030/viewer/2022032414/56649ef75503460f94c0aba7/html5/thumbnails/1.jpg)
Testing Concurrent Programs, A 7-Minute Jargon-Free
IntroductionThesis Writing Seminar
Mathias Ricken
Rice University
February 25, 2010
![Page 2: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010](https://reader030.vdocuments.site/reader030/viewer/2022032414/56649ef75503460f94c0aba7/html5/thumbnails/2.jpg)
2
Computer Annoyances
• Program bugs • Slow computers
![Page 3: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010](https://reader030.vdocuments.site/reader030/viewer/2022032414/56649ef75503460f94c0aba7/html5/thumbnails/3.jpg)
3
Unit Testing
• Test program parts, not the whole program– Test parts individually, then together– Smaller parts larger parts
• Unit tests are programs– Compare expected result to actual result:
assertEquals(12, multiply(3,4));
![Page 4: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010](https://reader030.vdocuments.site/reader030/viewer/2022032414/56649ef75503460f94c0aba7/html5/thumbnails/4.jpg)
4
Moore’s Law
Adopted fromSutter 2009
![Page 5: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010](https://reader030.vdocuments.site/reader030/viewer/2022032414/56649ef75503460f94c0aba7/html5/thumbnails/5.jpg)
5
Existing Approaches Fail
• Simple Unit Testing Frameworks– Example: JUnit– Assume program is single-threaded– Many errors go undetected
detected
not detected
Main thread:
Other thread:
![Page 6: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010](https://reader030.vdocuments.site/reader030/viewer/2022032414/56649ef75503460f94c0aba7/html5/thumbnails/6.jpg)
6
Problem: Non-Determinism
• Synchronization points– Information exchanged between threads– Order of threads reaching sync. points is
non-deterministic
![Page 7: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010](https://reader030.vdocuments.site/reader030/viewer/2022032414/56649ef75503460f94c0aba7/html5/thumbnails/7.jpg)
7
Problem: Non-Determinism
• Synchronization points– Information exchanged between threads– Order of threads reaching sync. points is
non-deterministic
– Program could succeed in one order but fail in another
![Page 8: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010](https://reader030.vdocuments.site/reader030/viewer/2022032414/56649ef75503460f94c0aba7/html5/thumbnails/8.jpg)
8
Comprehensive Tools: Costly
• Model checking or schedule-based execution– Examples: JPF, ExitBlock– Test all arrangements– Too costly
t = # of threads
s = # of sync. points
N = # of arrangements
t s N2 1 62 2 202 3 703 2 16803 4 7.6E+053 8 2.3E+114 8 2.1E+19
Longer than the universe exists!
(if N=1 1 minute)
![Page 9: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010](https://reader030.vdocuments.site/reader030/viewer/2022032414/56649ef75503460f94c0aba7/html5/thumbnails/9.jpg)
9
• Insert random delays before synchronization points to modify the order
• Run tests many times with different delays
Probabilistic Approach
![Page 10: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010](https://reader030.vdocuments.site/reader030/viewer/2022032414/56649ef75503460f94c0aba7/html5/thumbnails/10.jpg)
10
Probabilistic Approach
• Insert random delays before synchronization points to modify the order
• Run tests many times with different delays
delay
![Page 11: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010](https://reader030.vdocuments.site/reader030/viewer/2022032414/56649ef75503460f94c0aba7/html5/thumbnails/11.jpg)
11
(Proposed) Contributions
• Probabilistic Approach– Random delays at synchronization points
• Call Graph Analysis– Determine which parts of the program may interact– Only insert delays where necessary
• Improved JUnit Framework– Exception handler for all threads– Ensures that all errors in all threads are detected
![Page 12: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010](https://reader030.vdocuments.site/reader030/viewer/2022032414/56649ef75503460f94c0aba7/html5/thumbnails/12.jpg)
12
Evaluation
• Tested DrJava with Improved JUnit– Number of unit tests: 900– Previously unknown problems: 20– Slowdown: ~1 percent
• Other parts still under development
![Page 13: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010](https://reader030.vdocuments.site/reader030/viewer/2022032414/56649ef75503460f94c0aba7/html5/thumbnails/13.jpg)
13
Conclusion
• Unit testing of concurrent programs will become more important
• Concutest helps…– by improving JUnit’s abilities– by using probabilistic techniques– by using call graph analysis to reduce the
number of synchronization points
![Page 14: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010](https://reader030.vdocuments.site/reader030/viewer/2022032414/56649ef75503460f94c0aba7/html5/thumbnails/14.jpg)
14
Image Attribution1. Left image on Computer Annoyances:
Adapted from Microsoft
2. Graph on Moore’s Law:Adapted from Herb Sutter 2009
3. Top image on Multi-Cores and Concurrency:Adapted from Brian Goetz et al. 2006, Addison Wesley
4. Bottom image on Multi-Cores and Concurrency:Caption Fridays
![Page 15: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010](https://reader030.vdocuments.site/reader030/viewer/2022032414/56649ef75503460f94c0aba7/html5/thumbnails/15.jpg)
15
Extra Slides
![Page 16: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010](https://reader030.vdocuments.site/reader030/viewer/2022032414/56649ef75503460f94c0aba7/html5/thumbnails/16.jpg)
16
Multi-Cores and Concurrency
• Recent trends– Clock speed stagnated– Transistors still increasing– Reason: multi-core CPUs
• Implication– Must use concurrency to
benefit from modern CPUs– Concurrent programming is
difficult
![Page 17: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010](https://reader030.vdocuments.site/reader030/viewer/2022032414/56649ef75503460f94c0aba7/html5/thumbnails/17.jpg)
17
• Number of schedules (N)– t: # of threads, s: # of slices per thread
– Note: s is # of slices (between synchronization points)
Number of Arrangements
![Page 18: Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010](https://reader030.vdocuments.site/reader030/viewer/2022032414/56649ef75503460f94c0aba7/html5/thumbnails/18.jpg)
18
Product of s-combinations
For thread 1: choose s out of ts time slicesFor thread 2: choose s out of ts-s time slices…For thread t-1: choose s out of 2s time slicesFor thread t-1: choose s out of s time slices
Writing s-combinations using factorial
Cancel out terms in denominator and next numerator
Left with (ts)! in numerator and t numerators with s!
Number of Arrangements