Download - What Went Wrong?
![Page 1: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/1.jpg)
What Went Wrong?
Alex GroceCarnegie Mellon University
Willem VisserNASA Ames Research Center
![Page 2: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/2.jpg)
Java PathFinder
void add(Object o) { buffer[head] = o; head = (head+1)%size;}
Object take() { … tail=(tail+1)%size; return buffer[tail];}
Java Code
JAVAC JVM
0: iconst_01: istore_22: goto #395: getstatic 8: aload_09: iload_210: aaload
Bytecode
Special JVM
Model Checker
![Page 3: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/3.jpg)
Counterexamples
• When the model checker finds a bug, it reports a counterexample.
• However, even an exact trace of the program failing is not what we really want for most bugs.
![Page 4: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/4.jpg)
Counterexamples
• How can we get more information from a counterexample?
– Perhaps if we had some successful runs to compare to, or other failing runs in which to look for common elements…
• A common way of debugging by hand
• Except we can automate it using the model checker
![Page 5: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/5.jpg)
Counterexamples
The Error
![Page 6: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/6.jpg)
Counterexamples
“The Error”
Real cause
![Page 7: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/7.jpg)
Counterexamples
“The Error”
Real cause
• Idea: generate other traces that can give you more information about what is going wrong.
• How?
![Page 8: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/8.jpg)
Positives and Negatives
Real cause
assert (x < 5);
(and x = 8)
“The Error”
![Page 9: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/9.jpg)
Positives and Negatives
“The Error”
Real cause
Same location, same error:NEGATIVE (-)
Real cause
assert (x < 5);
(and x = 10)
An Error
assert (x < 5);
(and x = 8)
![Page 10: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/10.jpg)
Positives and Negatives
Real cause
Same control flow location,no error condition:
POSITIVE (+)
Same location, same error:NEGATIVE (-)
Real cause
assert (x < 5);
(and x = 3)assert (x < 5);
(and x = 10)
“The Error” An Error
assert (x < 5);
(and x = 8)
No Error
![Page 11: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/11.jpg)
The Basic Search
“The Error”
Real cause
![Page 12: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/12.jpg)
The Basic Search
Real cause
![Page 13: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/13.jpg)
The Basic Search
Real cause
![Page 14: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/14.jpg)
The Basic Search
Real cause
![Page 15: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/15.jpg)
The Basic Search
Real cause
![Page 16: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/16.jpg)
The Basic Search
Real cause
![Page 17: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/17.jpg)
The Basic Search
Real cause
![Page 18: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/18.jpg)
The Basic Search
![Page 19: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/19.jpg)
The Basic Search
![Page 20: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/20.jpg)
The Basic Search
![Page 21: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/21.jpg)
The Basic Search
![Page 22: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/22.jpg)
The Search in Detail
“The Error”
Real cause
Depth limit
Positives Negatives
![Page 23: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/23.jpg)
The Search
“The Error”
Real cause
Depth limit
Alreadyvisited
Positives Negatives
![Page 24: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/24.jpg)
The Search
“The Error”
Real cause
Depth limit
Positives Negatives
![Page 25: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/25.jpg)
The Search
“The Error”
Real cause
Depth limit
Positives Negatives
“push-through”
![Page 26: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/26.jpg)
The Search
“The Error”
Real cause
Depth limit
Alreadyvisited
Positives Negatives
![Page 27: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/27.jpg)
After the Search
“The Error”
Real cause
• Positives that are prefixes of some negative are removed
• While negatives are a true subset of complete negatives, positives are an approximation based on observed behavior
• This can be useful: in some reactive systems, every trace can be extended into an error trace
![Page 28: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/28.jpg)
Analysis of the Traces
Now what do we do with these?
• Report on code that appears – only in positives/negatives– in all positives/negatives– only and all positives/negatives (causal)
• Transform positives into negatives• Find difference in invariants across positives and negatives
Positives Negatives
![Page 29: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/29.jpg)
Analysis by Code Lines
• For our example trace, the line(s) in the real cause will be:– in all negative traces and – only appear in negative traces– thus will be identified as genuine cause
• Includes which thread executed the code and any nondeterministic choices made
• Can custom define equivalences to allow for looser definition of “the same code”
“The Error”
Real cause
Automaticallyidentified
![Page 30: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/30.jpg)
Common Ground
“The Error”
Real cause
(Can set a minimum shared prefixor suffix size)
Shared prefix
Shared suffix(control flow only)
Transformation“How to make it break”
![Page 31: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/31.jpg)
Transformation Analysis
• Sorted so that minimal way to break (transformation from each positive to nearest negative) is identified
• Rerun code line analysis over transformation code only (very useful in some cases)
• Can identify type of error:– If any positive can be transformed into a negative by
only altering thread scheduling, the error is essentially concurrency based
![Page 32: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/32.jpg)
Invariant Differences
• Pick certain control locations at which to observe data values in our traces
• Run a dynamic invariant detection algorithm to compute invariants across these locations for all negatives and then all positives
• Compare the discovered invariants
![Page 33: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/33.jpg)
A Stack Examplepublic IntStack (int s) { stack = new int[s]; top = 0;}
public void push (int i) { stack[top++] = i;}
public int pop () { top--; return stack[top];}
![Page 34: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/34.jpg)
A Stack Examplepublic IntStack (int s) { stack = new int[s]; top = 0;}
public void push (int i) { stack[top++] = i;}
public int pop () { top--; return stack[top];}
A simple stack with an interfacethat initializes the stack to a randommaximum size then randomlypushes and pops data.
![Page 35: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/35.jpg)
A Stack Examplepublic IntStack (int s) { stack = new int[s]; top = 0;}
public void push (int i) { stack[top++] = i;}
public int pop () { top--; return stack[top];}
A simple stack with an interfacethat initializes the stack to a randommaximum size then randomlypushes and pops data.
ORIGINAL COUNTEREXAMPLE:Stack overflow for stack size 1(array out of bounds on stack)
![Page 36: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/36.jpg)
A Stack Examplepublic IntStack (int s) { stack = new int[s]; top = 0;}
public void push (int i) { stack[top++] = i;}
public int pop () { top--; return stack[top];}
A simple stack with an interfacethat initializes the stack to a randommaximum size then randomlypushes and pops data.
ORIGINAL COUNTEREXAMPLE:Stack overflow for stack size 1(array out of bounds on stack)
Run analysis with a search depth of 50 and compute invariants.
![Page 37: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/37.jpg)
A Stack Example
Code line analysis
Code in the push method appears in all negatives.
public IntStack (int s) { stack = new int[s]; top = 0;}
public void push (int i) { stack[top++] = i;}
public int pop () { top--; return stack[top];}
![Page 38: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/38.jpg)
A Stack Example
Invariant analysis
For positives, on entry to the push method, this invariant holds:
stack.length > top
For negatives, however the invariant is:
stack.length >= top
public IntStack (int s) { stack = new int[s]; top = 0;}
public void push (int i) { stack[top++] = i;}
public int pop () { top--; return stack[top];}
![Page 39: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/39.jpg)
A Stack Example
Transformations from positive to negative
Change stack size from 3 to 1
…
Change stack size from 4 to 2
Insert a push
…
Remove a pop
…
public IntStack (int s) { stack = new int[s]; top = 0;}
public void push (int i) { stack[top++] = i;}
public int pop () { top--; return stack[top];}
![Page 40: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/40.jpg)
DEOS Error Analysis
• DEOS– Real time operating system– Allows threads to be created and deleted– Has time budgets for various threads
![Page 41: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/41.jpg)
DEOS Error Analysis
• DEOS– Real time operating system– Allows threads to be created and deleted– Has time budgets for various threads
• Original counterexample– Assertion about the time budget of a thread fails
![Page 42: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/42.jpg)
DEOS Error Analysis
• DEOS– Real time operating system– Allows threads to be created and deleted– Has time budgets for various threads
• Original counterexample– Assertion about the time budget of a thread fails
• Run analysis with size limit of 10
![Page 43: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/43.jpg)
DEOS Error Analysis
Transformations
Change a wait until next period into a thread deletionInsert an interrupt and system clock ticks (before a deletion already in the positive)Change a wait until next period into an interrupt and system clock ticks and afterwards a deletion…
The DEOS error requires deletion (which appears in the set of code found in all negatives) but also relies upon specific timing, as indicated by the transformations.
![Page 44: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/44.jpg)
Related Work
• “Fate and Free Will in Error Traces” (TACAS 02)
• SLAM group independently investigated very similar concepts this summer (submitted to POPL)– Approach quite similar in spirit, lacks invariants and
transformation approaches, but has some niceties with respect to source code analysis
• Andreas Zeller: work on isolating error-causing thread schedules and cause-effect chains
![Page 45: What Went Wrong?](https://reader036.vdocuments.site/reader036/viewer/2022081419/568152f6550346895dc11246/html5/thumbnails/45.jpg)
Future Work
• More efficient algorithms for generation of the traces (try bounded model checking)
• Different approaches to analysis:– Generation of automata– Characterization of error classes– Checking of causality…