modular verification of concurrent assembly code with dynamic thread creation and termination

Modular Verification of Concurrent Assembly Codewith Dynamic Thread Creation and Termination

Xinyu FengYale University

Joint work with Zhong Shao

2005-9-16 NJPLS@Stevens

Motivation Proof-carrying code (PCC)

In principle: verify any property on any code Real binaries & no loss of efficiency Embedded OS, device drivers… All safety & liveness properties… Formal, machine-checkable proofs

In reality: only works for sequential code

Can concurrent code ever be supported by the PCC framework ?

2005-9-16 NJPLS@Stevens

Challenges Challenges for Proof-carrying concur. code

A general framework for concurrent assembly code verification

Lack of structures (e.g. cobegin/coend blocks) Specification/proof generation

Spec inference, proof assistant, theorem prover

Concurrent assembly code verification No directly applicable logic

Traditional Hoare-logic: only sequential code Type Systems: no Concurrent Typed Assembly

Language (TAL)

2005-9-16 NJPLS@Stevens

Previous work Rely-Guarantee (R-G) Method

Shared memory concurrency Thread modular verification Only for higher-level code: cobegin/coend

CCAP[Yu&Shao, ICFP’04] The first PCC framework supporting concurrent

assembly code R-G method Only support static threads

P1 || … || Pn

2005-9-16 NJPLS@Stevens

Concurrency Programming cobegin/coend

S::=…| cobegin P1 || P2 codend | … Higher-level, well-structured Only support properly nested concurrent code

fork/join S::=…| tid := fork f(a) | join tid | … More flexible: improperly nested code OSes/Java/…

2005-9-16 NJPLS@Stevens

Our Contributions

A new PCC framework: CMAP Verification of general properties Dynamic thread creation/termination

Generalize the Rely-Guarantee method Modular verification Realistic features

Multiple instantiations of thread code Thread argument passing, thread-local data

2005-9-16 NJPLS@Stevens

Outline of This Talk Background: the Rely-Guarantee

Method Challenges for Dynamic Thread

Creation/Termination Our Approach The CMAP Framework Conclusion and Future Work

2005-9-16 NJPLS@Stevens

The Rely-Guarantee Method

Thread 1

Thread 2

(A1,G1)

(A2,G2)Shared Memory

S1 S2 S3 S4 S5

A1: S2 – S3, S4 – S5,…

G1: S1 – S2, S3 – S4,…

A2: S1 – S2, S3 – S4,…

G2: S2 – S3, S4 – S5,…

G1 A2

G2 A1

2005-9-16 NJPLS@Stevens

The Rely-Guarantee Method Thread + Thread Environment Rely and Guarantee

A, G: State State Prop Thread Modularity

Non-Interference (interface compatibility): i,j. ij Gi Aj

Safety of each thread Ti: (Ai, Gi)

2005-9-16 NJPLS@Stevens

GCD Example [Yu&Shao’04]

Thread1:

while(a<>b){

if(a > b)

a := a-b;

}

Thread2:

while(a<>b){

if(b > a)

b := b-a;

}

2005-9-16 NJPLS@Stevens

Outline of This Talk Background: the Rely-Guarantee

Method Challenges for Dynamic Thread

Creation/Termination Our Approach The CMAP Framework Conclusion and Future Work

2005-9-16 NJPLS@Stevens

Concurrency Programming cobegin/coend

S::=…| cobegin P1 || P2 codend | … Higher-level, well-structured Only support properly nested concurrent code

fork/join S::=…| tid := fork f(a) | join tid | … More flexible: improperly nested code OSes/Java/…

2005-9-16 NJPLS@Stevens

Static and Dynamic Threads

f(a)...

fork f(a1)

fork f(a2)fork f(an)

…

“Static Threads”

“Dynamic Threads”

2005-9-16 NJPLS@Stevens

Challenges First attempt

Check NI between all static threads Ti: (Ai, Gi) i,j. ij Gi Aj

Too rigid to handle changing env.

2005-9-16 NJPLS@Stevens

Challenges: Changing Env. I A-B: initialize data d

no other threads will change d A : d = d’

B-C: collaborate with T3 to process d

T3 may change d Still do not allow other threads

change d

C-D: T3 terminates No other threads can change d

T1 T2

A

B

T3

C

D

Use pc to mark stages?

2005-9-16 NJPLS@Stevens

Challenges: Changing Env. I

main:

int i:=0;

while (i<100){

data[i]:=f(i);

fork child(i);

i++;

}

Global data: int data[100]T1 T2

A

B

T3

C

D

…

2005-9-16 NJPLS@Stevens

Challenges: Changing Env. II T2 and T3 have no overlap in

their lifetime non-interference between all

threads? Only check those that overlap?

How to specify the overlapping?

T1 T2 T3

2005-9-16 NJPLS@Stevens

Challenges: multiple instantiations

f(a)...

(Aa, Ga)

(Aa1, Ga1)

fork f(a1)

fork f(a2)fork f(an)

(Aa2, Ga2) (Aan, Gan)

GaiAaj

GaAa?

2005-9-16 NJPLS@Stevens

Challenges: Modularity

T1:

.

.

.jmp f

f:...

exit

T2:

.

.

.jmp f

(A1, G1)

(A2, G2)

Certify once,

use everywhere?

2005-9-16 NJPLS@Stevens

Outline of This Talk Background: the Rely-Guarantee

Method Challenges for Dynamic Thread

Creation/Termination Our Approach The CMAP Framework Conclusion and Future Work

2005-9-16 NJPLS@Stevens

Our Approach (1) Problems for checking NI of static threads

Changing environment Multiple instantiations Modularity issues

CMAP: “lazy checking” At each step, all live (dynamic) threads do not

interfere

2005-9-16 NJPLS@Stevens

Our Approach (2)

… t0 tnQ

(A0, G0)… (An, Gn)

How to track the changing thread queue?

jijiji AGttQtt .,WF(Q, ):

each ti satisfies (Ai, Gi)

2005-9-16 NJPLS@Stevens

Our Approach (3)

Q'

'

WF

Q

WF

Initial condition: 0 . WF(Q0, 0)

::= add | sub | jd f |…

| exit | fork | yield

Borrow ideas from typechecking data heaps (as in TAL):

2005-9-16 NJPLS@Stevens

Our Approach (4) Thread Termination: exit

Qt

(A,G) \{(Ai, Gi)}(Ai,G

i)

WF WF!

exit Q\{ti}tiQ

2005-9-16 NJPLS@Stevens

Our Approach (5) Thread Creation: fork f(a)

Qt

(A,G)

t

?

WF WF

fork

(1) t' does not interfere with Q

(2) t does not interfere with the new env.

Q t’

2005-9-16 NJPLS@Stevens

Our Approach (6)

t

(A,G)

Q{t’}tfork

G i Ai i Gi A

?(A’,G’)

{A’’,G’’}

G' (i Ai)A'' (i Gi)G'' A'

WF WF?

Q

G'' i Ai i Gi A''G

G

A

A

2005-9-16 NJPLS@Stevens

Our Approach (7) Queue Extension

WF(Q{t}, {(A, G)})

WF(Q{t',t}, {(A’’, G’’), (AG’’, GA’’)})

fork f(a)A A’’, G’’ G

2005-9-16 NJPLS@Stevens

Our Approach (8) Queue Update

WF(Q{t}, {(A, G)})

WF(Q{t}, {(A’, G’)})

AA’, G’G; t: (A’, G’)

2005-9-16 NJPLS@Stevens

Our Approach (9)

T1:

.

.

.jmp f

f:...

exit

T2:

.

.

.jmp f

(A1, G1)

(A2, G2) Certify once,

use everywhere?

(A, G)

AiA, GGi

2005-9-16 NJPLS@Stevens

Our Approach (10) Check static threads Lazy Check

Changing Env. Changing (A, G) Multiple instantiation Not care Modularity Certify only once

General Enough Language (higher-level/assembly) Thread Model (preemptive/non-preempti

ve)

2005-9-16 NJPLS@Stevens

Example – Unbounded Thread Creation

main:

int i:=0;

while (i<100){

data[i]:=f(i);

fork child(i);

i++;

}

void child(x:int){

data[x] = g(x, data[x])

}

Global data: int data[100]

2005-9-16 NJPLS@Stevens

Example – Unbounded Thread Creation Specification of Child:

Ax: Gx:

Non-interference between children:

2005-9-16 NJPLS@Stevens

Example – Unbounded Thread Creation How to specify the main thread?

main:

int i:=0;

while (i<100){

data[i]:=0;

fork(child, i);

i++

}

Do we need a G such that:

But main cannot satisfy such a G!

2005-9-16 NJPLS@Stevens

main:

int i:=0;

while (i<100){

data[i]:=0;

fork(child, i);

i++

}

(A’, G’)

(A, G)

(A, G)

(A, G)(A, G)

2005-9-16 NJPLS@Stevens

Outline of This Talk Background: the Rely-Guarantee

Method Challenges for Dynamic Thread

Creation/Termination Our Approach The CMAP Framework Conclusion and Future Work

2005-9-16 NJPLS@Stevens

The CMAP Framework The abstract machine The verification logic

Specification language Inference rules Soundness proof

Example programs Unbounded dynamic thread creation Readers/Writers problem Lock-free program

All implemented in Coq!

2005-9-16 NJPLS@Stevens

The CMAP Framework - MachineI1f1

: I2f2

: …

(code heap) C

(program) P::=(C,T,S,Q,I)

0

r1

1 2 …

r2 r3 … rn

(data heap) H

(register file) R

(state) S::=(H,R)

I1h1

: I2h2

: …(thrd entries) T

add…fork hyieldexit

(instr. seq.) I

I

R

I

R

I

R…

(dyn. queue) Q

2005-9-16 NJPLS@Stevens

The CMAP Framework

The paper on CMAP (Feng&Shao ICFP’05):

http://flint.cs.yale.edu/publications/cmap.html

2005-9-16 NJPLS@Stevens

Conclusion Problems for unbounded dynamic thread creation

Changing environment (fork/exit) Multiple instantiation of thread code No previously known modular verification method

Our approach INV: active threads in the system do not interfere

Combine the type-based proof technique with R-G method Unify thread’s assumption/guarantee with env.’s guarant

ee/assumption Thread modularity + code/proof reuse

The CMAP framework and its Coq implementation

2005-9-16 NJPLS@Stevens

Future Work Certified Thread Libraries

fork, yield, exit join, lock, monitors

Surface language Higher-level specifications Partially infer A and G Certifying compilation to CMAP

Where is the threads ?User-level thread + thread lib.

2005-9-16 NJPLS@Stevens

Thank you!