partially disjunctive shape analysis josh berdine byron cook msr cambridge tal lev-ami roman...
TRANSCRIPT
Partially Disjunctive Shape Analysis
Josh BerdineByron CookMSR Cambridge
Tal Lev-AmiRoman ManevichMooly SagivRan ShahamTel Aviv University
Ganesan RamalingamMSR India
Gilad ArnoldUC Berkeley
John FieldIBM Watson
Motivation Scaling shape analysis
Multiple data structures Intricate data structures Concurrency Procedures Complex properties, e.g., linearizability
2
3
Non-blocking stack [Treiber,‘86] void push(Stack *S, data_type v) {[1] Node *x = alloc(sizeof(Node));[2] x->d = v;[3] do {[4] Node *t = S->Top;[5] x->n = t;[6] } while (!CAS(&S->Top,t,x));[7] }
data_type pop(Stack *S){[8] do {[9] Node *t = S->Top;[10] if (t == NULL)[11] return EMPTY;[12] Node *s = t->n;[13] data_type r = t->d;[14] } while (!CAS(&S->Top,t,s));[15] return r;[16] }
benign data races
unbounded number of
threads
x points to valid memory?does list remain acyclic?
stack linearizable?
Automatic proof of linearizabilityfor an unbounded number of
threads
push2(4,5)
pop2():8,5push2(7,8)
4
Non-linearizable pairs stackvoid push2(Stack *S, data_type v1, data_type * v2) { push(s, v1); push(s, v2);}
void pop2(Stack *S, data_type * v1, data_type * v2) { *v2 = pop(s); *v1 = pop(s); }
time
push2(4,5)
pop2():8,5push2(7,8)
illegal sequential execution
push2(4,5)
pop2():8,5push2(7,8)
5
Non-linearizable pairs stackvoid push2(Stack *S, data_type v1, data_type * v2) { push(s, v1); push(s, v2);}
void pop2(Stack *S, data_type * v1, data_type * v2) { *v2 = pop(s); *v1 = pop(s); }
time
push2(4,5)
pop2():8,5push2(7,8)
illegal sequential execution
Motivation Scaling shape analysis
Multiple data structures Intricate data structures Concurrency Procedures Complex properties, e.g., linearizability
Develop abstraction techniques to Handle state space blow-ups Handle unbounded number of threads
More generally, unbounded subheaps Handle states in a more “modular”/”local”
way6
Developed techniques Partial join [SAS’04]
Fuse similar abstract heaps Approximate disjunction abstract heaps
Intersecting heap abstractions [VMCAI’06] Approximate conjunction of abstract heaps
Heap decomposition [TACAS’07, SAS’08] Localize heap abstractions
Universal quantified abstractions [CAV’08] Represent unbounded copies of abstract
(sub)heap
7
8
9
Thread-modular analysis
Single global resource invariant[Flanagan & Qadeer, SPIN 03]
pc=1
pc=1
Separated resource invariants[Gotsman et al., PLDI 07]Coarse-grained concurrency
pc=1
pc=1
Non-disjoint resource invariants[SAS 08, CAV 08]Fine-grained concurrency
pc=1
pc=1
10
Concurrent heaps [Yahav, POPL’01]
Heaps contain both threads and objects Logical structure, or Formula in subset of FOTC [Yorsh et al.,
TOCL‘07]
thread object with
program counter
thread-local variable
list field
list object
pc=6 pc=2
x
n
x
Topt
11
Heaps contain both threads and objects Logical structure, or Formula in subset of FOTC [Yorsh et al., TOCL‘07]
pc=6 pc=2
x
n
x
Topt
pc(tr1)=6 pc(tr2)=2 v1,v2,v3. Top(v1) x(tr1,v2) t(tr1,v1) x(tr2,v3) n(v2,v1) …
v1
v3
v2
tr1tr2
Concurrent heaps [Yahav, POPL’01]
12
Outline Heap decomposition Universally quantified heap
abstractions Checking linearizability for an
unbounded number of threads Experimental results Partial join
Heap Decomposition[R. Manevich, T. Lev-Ami, G.
Ramalingam, M. Sagiv, J. Berdine, SAS’08]
13
14
Select subheaps Parametrically State-sensitive selection
Top
x
t
n
pc=7
pc=14t
pc(tr1)=7 pc(tr2)=14… v1,v2. t(tr1,v1) x(tr1,v2) t(tr2,v1) n(v2,v1)…
v1
v2
tr1
tr2
15
Abstraction by decomposition
Represent a heap as a conjunction of subheaps (may overlap)
Top
x
t
n
pc=7
pc(tr1)=7 v1. t(tr1,v1) x(tr1,v2) n(v2,v1) …
pc(tr2)=14 v2. t(tr2,v1) …
Top
pc=14t
v1
tr2
v1
v2
tr1
tn
pc=??
n t
n
pc=??
n
Universally QuantifiedShape Abstractions
[J. Berdine, T. Lev-Ami, R. Manevich,G. Ramalingam, M. Sagiv, CAV’08]
16
Unbounded concurrent heaps
17
void push(Stack *S, data_type v) {[1] Node *x = alloc(sizeof(Node));[2] x->d = v;[3] do {[4] Node *t = S->Top;[5] x->n = t;[6] } while (!CAS(&S->Top,t,x));[7] }
x
n
x
Top
x x
t
x
t
x
n
t
t
Unbounded parallel composition:push(Top,?) || ... || push(Top,?)
n
n
Local heaps Each local heap
Presents a view of heap relative to one thread Can be instantiated ≥0 times
18
pc=4
t
pc=2
x
xpc=1 Top
Top
pc=6
t
n
x
Top
Top
n
n
n
n
n
n
n
n
Bounded local heaps Each local heap
Presents a view of heap relative to one thread Can be instantiated ≥0 times Bounded by finitary abstraction (Canonical
Abstraction)
19
pc=4
t
pc=2
x
xpc=1 Top
Top
pc=6
t
n
x
Top
Top
n
n
n
n
n
n
n
n
20
Full heap
Top
x
t
n
pc=6
pc=2t
pc(tr1)=7 pc(tr2)=14… v1,v2. t(tr1,v1) x(tr1,v2) t(tr2,v1) n(v2,v1)…
v1
v2
tr1
tr2
Decomposed abstract heap
pc=2
x
Toppc=6
x
n
Topt
tr1 tr2
v1 v1
v2
v3
pc(tr1)=6 v1,v2. Top(v1) x(tr1,v2) t(tr1,v1) n(v2,v1) …
pc(tr2)=2 v1,v3. Top(v1) x(tr2,v3) …
21
pc=2
x
Top
pc(t)=6 v1,v2. Top(v1) x(t,v2) t(t,v1) n(v2,v1) …
t.pc(t)=2 v1,v3. Top(v1) x(t,v3) …
Universally quantifiedlocal heaps
pc=6
x
n
Topt
overlappinglocal heaps
22
t t
v1 v1
v2
v3
symbolicthread
symbolicthread
pc(t)=6 v1,v2. Top(v1) x(t,v2) t(t,v1) n(v2,v1) …
t.pc(t)=2 v1,v3. Top(v1) x(t,v3) …
Meaning of quantified invariant
pc=6
x
n
Topt
x
pc=1
pc=6
pc=2
t
Information maintained (dis)equalities between
local variables of each thread and global variables
Objects reachable from global variables
Information lost Number of threads (dis)equalities between
local variables of different threads
23
pc=2
x
Top
x
pc=1
pc=6
pc=3
t
pc=1
×m n×
Loss of non-aliasing information
pc(t)=6 v1,v2. Top(v1) x(t,v2) t(t,v1) n(v2,v1) …
t.
pc=6
x
n
Top
pc=6
x
n
t
t
pc=6
x
n
t
pc=6
x
t
unwanted aliasingconsider x->n=t
Remedy: record non-
aliasing information
explicitly
24
n
Adding non-aliasing information
pc=6
P
x
n
Top
pc=6
P
x
n
t
t
pc=6
x
n
t
pc=6
x
Referencedby exactlyone thread
pc(t)=6 v1,v2. Top(v1) x(t,v2) t(t,v1) n(v2,v1) Private(v1) Private(v2) …
t.
P
t
25
n
Adding non-aliasing information
pc(t)=6 v1,v2. Top(v1) x(t,v2) t(t,v1) n(v2,v1) Private(v1) Private(v2) …
t.
pc=6
P
x
n
Top
pc=6
P
x
n
t
t
pc=6
x
n
t
pc=6
Px
P
t
Operation on private objects
invisible to other threads
26
n
Recap Decompose heap into conjunction of
subheaps Add universal quantification on top of
finitary heap abstractions Handle unbounded number of threads
Local heaps can overlap Handle fine-grained concurrency
Strengthen local heaps by Private predicate Private objects cannot be affected by actions of
other threads Missing: transformers (see papers)
27
Checking linearizabilityfor an unbounded number
of threads
28
Verification of fixed linearization points
[Amit et al., CAV’07] Compare each concurrent execution to a specific
sequential execution Show that every (terminating) concurrent
operation returns the same result as its sequential counterpart
linearizationpoint
operationConcurrent
Execution
Sequential
Execution
compare results
...
linearizationpoint
Conjoined
Execution
compare
results
29
30
Linearization pointsfor Treiber’s stack
void push(Stack *S, data_type v) {[1] Node *x = alloc(sizeof(Node));[2] x->d = v;[3] do {[4] Node *t = S->Top;[5] x->n = t;[6] } while (!CAS(&S->Top,t,x)); // @LINEARIZE on CAS[7] }
data_type pop(Stack *S){[8] do {[9] Node *t = S->Top; // @LINEARIZE[10] if (t == NULL)[11] return EMPTY; [12] Node *s = t->n;[13] data_type r = t->d;[14] } while (!CAS(&S->Top,t,s)); // @LINEARIZE on CAS[15] return r;[16] }
Shape analysis with delta abstraction [Amit et al.,
CAV’07] Tracks bounded differences between
concurrent and sequential execution Abstracts two heaps together Limited to bounded number of threads
Tracks correlations between all threads Feasible up to 4 threads
31
What about an unbounded
number of threads?
32
Our approach Tracks bounded differences between
concurrent and sequential executionper thread Handles unbounded number of threads
Abstracts correlations between threads Thread-modular characteristics
Top
33
Conjoined execution for push
concurrent state
sequential view
isomorphismrelation
Top
void push(Stack *S, data_type v) {[1] Node *x = alloc(sizeof(Node));[2] x->d = v;[3] do {[4] Node *t = S->Top;[5] x->n = t;[6] } while (!CAS(&S->Top,t,x)); // LINEARIZE on CAS[7] }
Top Top
34
Conjoined execution for push
conjoined state
void push(Stack *S, data_type v) {[1] Node *x = alloc(sizeof(Node));[2] x->d = v;[3] do {[4] Node *t = S->Top;[5] x->n = t;[6] } while (!CAS(&S->Top,t,x)); // LINEARIZE on CAS[7] }
duo-object
35
Conjoined execution for push
Top Top
P
x
delta object
Top Top
void push(Stack *S, data_type v) {[1] Node *x = alloc(sizeof(Node));[2] x->d = v;[3] do {[4] Node *t = S->Top;[5] x->n = t;[6] } while (!CAS(&S->Top,t,x)); // LINEARIZE on CAS[7] }
36
Conjoined execution for push void push(Stack *S, data_type v) {[1] Node *x = alloc(sizeof(Node));[2] x->d = v;[3] do {[4] Node *t = S->Top;[5] x->n = t;[6] } while (!CAS(&S->Top,t,x)); // LINEARIZE on CAS[7] }
Top Top
P
x
Top Top Top Top
P
x t…
Top Top
P
x t
n
if (STop == t) STop = x; evaluate to true;else evaluate to false;
Top
Top
n
37
Run operation sequentially void push(Stack *S, data_type v) {[1] Node *x = alloc(sizeof(Node));[2] x->d = v;[3] do {[4] Node *t = S->Top;[5] x->n = t;[6] } while (!CAS(&S->Top,t,x)); // LINEARIZE on CAS[7] }
Top
Top
n
Top
Top
n
xTop
Top
n
x
t
Top
Top
n
x
t
n
Top Top
n n
38
Run operation sequentially
Top
Top
n
Top
Top
n
xTop
Top
n
x
t
Top
Top
n
x
t
n
TopTop
n
But how do you handleunboundedness due to
recursive data structures?
Employ Canonical
Heap Abstraction
void push(Stack *S, data_type v) {[1] Node *x = alloc(sizeof(Node));[2] x->d = v;[3] do {[4] Node *t = S->Top;[5] x->n = t;[6] } while (!CAS(&S->Top,t,x)); // LINEARIZE on CAS[7] }
39
An unbounded state void push(Stack *S, data_type v) {[1] Node *x = alloc(sizeof(Node));[2] x->d = v;[3] do {[4] Node *t = S->Top;[5] x->n = t;[6] } while (!CAS(&S->Top,t,x)); LINEARIZE on CAS[7] }
Px
n Px
Top
Px
Px
t
Px
t
Px
n
t
t
unboundednumber of
delta objects
n
n
Top
n
n
Top
Px
n
n
Px
Topt
n
n
Px
n
Topt
n
n
40
Bounded local states
number ofdelta objects
per local heapbounded
Observations used Unbounded number of heap objects
Number of delta objects created per thread is bounded
Objects in recursive data structures bounded by known shape abstractions
Delta objects always referenced bylocal variables + global variables Captured by local heaps
Threads mutate data structure near global access points
41
42
HeDec: system for Heap Decomposition
Parametric: allows experimenting with different decompositions Analysis designer specifies decomposition Subheaps not necessarily disjoint Applicable for states with threads
Soundness automatically guaranteed for Any decomposition specification Any transformer specification
43
Verified Programs #states time (sec.)
Treiber’s stack[1986]
764 7
Two-lock queue[Michael & Scott, PODC’96]
3,415 17
Non-blocking queue[Doherty & Groves, FORTE’04]
10,333 252
Experimental results
First automatic verification of linearizability for unbounded number of threads
Partial Join[R. Manevich, M. Sagiv, G.
Ramalingam,J. Field, SAS’04]
44
Similar abstract heaps
pc=6
P
x
n
Topt
45
n
pc=6
P
x
nTop
t
n
Fused heap
pc=6
P
x
n
Topt
46
n
Non-similar abstract heaps
pc=6
P
x
n
Topt
47
n
pc=5
P
x
Topt
n
Principle
48
u,v. 1(u) 1(u,v)
t.u,v. 2(u) 2(u,v)
Principle
49
u,v. (u) (u,v) 1(u,v)
t.u,v. (u) (u,v) 2(u,v)
Principle
50
u,v. (u) (u,v)t.
51
Related work [Yahav, POPL’01]
Shape analysis with counter abstraction [Gotsman et al., PLDI’07]
Thread-modular shape analysis for coarse-grained concurrency
[Amit et al., CAV’07] Linearizability for a bounded number of threads
[Vafeiadis et al.,’06,’07,’08] Linearizability for an unbounded number of threads with
Rely-Guarantee reasoning w. separation logic Requires user annotations
[Gulwani et al., POPL’08] Lifting abstract interpreters to quantified logical domains
[Pnueli et al., TACAS’01] [Clarke et al., TACAS’08][Namjoshi, VMCAI’07]
Model checking concurrent systems
shape analysis
model checking
concurrency
+
52
Conclusion Three abstraction techniques
Heap decomposition Thread quantification Partial join
Parametric system Integrated into TVLA
Handle combination of Unbounded number of threads Dynamically-allocated memory Automatically proves linearizability
Also useful for sequential programs
Thanks!
53
54
i: v: j } i, j (v) } v, w: m,n } i,m (v) i, n (w) i, m, n (v, w) }
Canonical Heaps
55
t: i: v: j {i, j (t, v)} v,w: m,n } i,m (t, v) i, n (t, w) i, m, n (t, v, w) }
“Lifted” Canonical Heaps