compcert, a coq-certified...

CompCert, a Coq-certified compiler

Sandrine Blazy

IRISA - INRIA Rennes Bretagne Atlantique and Universite Rennes 1

DGA-INRIA seminar, 2010-10-15

joint work with Xavier Leroy (INRIA Paris Rocquencourt)

S. Blazy (INRIA-IRISA) CompCert, a Coq-certified compiler DGA-INRIA seminar 1 / 24

Can you trust your compiler?

Executablemachine code

Sourceprogram

Compiler?

Bugs in the compiler can lead to incorrect machine code being generatedfrom a correct source program.


An example of optimizing compilation

double dotproduct(int n, double * a, double * b){

double dp = 0.0;int i;for (i = 0; i < n; i++) dp += a[i] * b[i];return dp;

}

Compiled for the Alpha processor with all optimizations and manuallydecompiled back to C. . .


double dotproduct(int n, double * a, double * b)

{double dp, a0, a1, a2, a3, b0, b1, b2, b3;

double s0, s1, s2, s3, t0, t1, t2, t3;

int i, k;

dp = 0.0;

if (n <= 0) goto L5;

s0 = s1 = s2 = s3 = 0.0;

i = 0; k = n - 3;

if (k <= 0 || k > n) goto L19;

i = 4; if (k <= i) goto L14;

a0 = a[0]; b0 = b[0]; a1 = a[1]; b1 = b[1];

i = 8; if (k <= i) goto L16;

L17: a2 = a[2]; b2 = b[2]; t0 = a0 * b0;

a3 = a[3]; b3 = b[3]; t1 = a1 * b1;

a0 = a[4]; b0 = b[4]; t2 = a2 * b2; t3 = a3 * b3;

a1 = a[5]; b1 = b[5];

s0 += t0; s1 += t1; s2 += t2; s3 += t3;

a += 4; i += 4; b += 4;

prefetch(a + 20); prefetch(b + 20);

if (i < k) goto L17;

L16: s0 += a0 * b0; s1 += a1 * b1; s2 += a[2] * b[2]; s3 += a[3] * b[3];

a += 4; b += 4;

a0 = a[0]; b0 = b[0]; a1 = a[1]; b1 = b[1];

L18: s0 += a0 * b0; s1 += a1 * b1; s2 += a[2] * b[2]; s3 += a[3] * b[3];

a += 4; b += 4;

dp = s0 + s1 + s2 + s3;

if (i >= n) goto L5;

L19: dp += a[0] * b[0];

i += 1; a += 1; b += 1;

if (i < n) goto L19;

L5: return dp;

L14: a0 = a[0]; b0 = b[0]; a1 = a[1]; b1 = b[1]; goto L18;

}




Sourceprogram

Compiler?

Non-critical sofware:Compiler bugs are negligible compared with those of the program itself.




Sourceprogram

Compiler?

Test

Critical software certified by systematic testingWhat is tested: the executable code generated by the compiler.Compiler bugs are detected along with those of the program.




Sourceprogram

Compiler?

Formal verification

Critical software certified by formal methodsWhat is formally verified: the source code, not the executable code.Compiler bugs can invalidate the guarantees obtained by formal methods.




Sourceprogram

Compilerobservationalequivalence

Formal verification

Formally verified compiler:Guarantees that the generated executable code behaves as prescribed by thesemantics of the source program.


Outline

1 Introduction: Can you trust your compiler?

2 Formally verified compilers

3 The CompCert experiment

4 Perspectives


Formal verification of compilers

Apply formal methods to the compiler itself to prove a semanticpreservation property:

Theorem

For all source codes S,if the compiler generates machine code C from source S,without reporting a compilation error,then C behaves like S.

Note: compilers are allowed to fail (ill-formed source code, or capacityexceeded).


Proof assistants

Implementations of well-defined mathematical logics.

Provide a specification language to write definitions and statetheorems.

Provide ways to build proofs in interaction with the user.(Not fully automated proving.)

Check the proofs for soundness and completeness.

Some mature proof assistants:

ACL2 HOL PVS

Agda Isabelle Twelf

Coq Mizar


Decomposition in multiple compiler passes

Transl 1

Optim 1

Transl 2

Optim 2

Transl 3

Asm

Source

Intermediate 1

Intermediate 2

Assembly

Machine code


Proof patterns (for each compilation pass)

transformation transformation

validator

×

transformation

untrusted solver

×

checker

Verified transformation Verified translation validation

External solver with verified validation

= formally verified

= not verified


Observable behavior

For an imperative language with I/O: add a trace of input-outputoperations performed during execution.

Example

x := 1; x := 2; ≈ x:=2;print(1); print(2); 6≈ print(2);

The observable behavior of a program P is:

terminates(t, n) (P ⇓ terminates(t, n)):termination, with trace t of input-output events, and return code n(return value of the main function).

diverges(T ) (P ⇓ diverges(T )):divergence, with trace T of input-output events (finite or infinite).

goeswrong(t): going wrong (undefined behavior).


Preservation of “not going wrong” behaviors

Compilers are allowed to refine behaviors if the source language is notdeterministic.

Compilers are allowed to generate code that doesn’t go wrong for a sourceprogram that goes wrong.

Theorem (semantic preservation)

for all source codes S,if C is compiled from S without reporting a compilation error,and S ⇓ b,then C ⇓ b.


”Going wrong” behaviorsExample of undefined behavior

Source code: Compiled code:

int f(int x, int n) f:{ slw r3, r3, r4return x << n; blr

}

Clight semantics of the source code:

If 0 ≤ n < 32, then computes x ∗ 2n.

Otherwise, undefined behavior (or: does anything).

Real semantics of the compiled code (PowerPC) :

If n mod 64 < 32, then the result is x ∗ 2(n mod 64).

If n mod 64 ≥ 32, then the result is 0.

Never goes wrong.


Outline




4 Perspectives


The CompCert projectcompcert.inria.fr, conducted by Xavier Leroy

Develop and prove correct a realistic compiler, usable for critical embeddedsoftware.

Source language: a large subset of C.

Target languages: PowerPC and ARM assembly, x86 (experimental)assembly.

Generates reasonably compact and fast code⇒ some optimizations.

This is “software-proof codesign” (as opposed to proving an existingcompiler).

Used Coq to mechanize the proof of semantic preservation and also toimplement (in pure functional style) most of the compiler.


compcert.inria.fr

The subset of C supportedSupported:

Types: integers, floats, arrays, pointers, struct, union.

Operators: arithmetic, pointer arithmetic.

Side-effects within expressions.

Control: if/then/else, loops, goto, switch.

Functions, recursive functions, function pointers.

Not supported at all:

The long long and long double types.

longjmp/setjmp.

Supported through de-sugaring after parsing (not proved !):

Block-scoped variables.

Variable-arity functions.

Assignment & pass-by-value of struct and union.

Bit-fields.S. Blazy (INRIA-IRISA) CompCert, a Coq-certified compiler DGA-INRIA seminar 16 / 24

The formally verified part of the compiler

Clight C#minor Cminor

RTLLTLLTLin

Linear Mach Asm

initial

translation

stack allocation

instruction selection

CFG construction

decomposition of expressions

register

allocation

linearization

of the CFG

spilling, reloading

enforcement of

calling conventions

layout of

of stack frames

PowerPC & ARM

code generation

Optimizations:

constant propagation,

common subexpressions


The whole CompCert compiler

AST Clight

AST Asm

C source AST C

AssemblyExecutable

parsing

construct AST

type-checking

de-sugaring

simplifications

printing of

asm syntax

assembling

linking

Type reconstruction

Graph coloring

Code linearization heuristic

Proved in CoqNot proved yet


Formally verified using Coq

The correctness proof (semantic preservation) for the compiler is entirelymachine-checked, using the Coq proof assistant.(50000 lines of Coq, 4 man.years + experiments.)

Theorem transf_c_program_correct:forall prog tprog behavior,transf_c_program prog = OK tprog ->Clight.exec_program prog behavior ->PPC.exec_program tprog behavior.


Performances of the generated code(On a PowerPC G5 processor)

AE

S

Alm

aben

ch

Bin

aryt

rees

Fan

nku

ch

FF

T

Kn

ucl

eoti

de

Nb

od

y

Qso

rt

Ray

trac

er

Sp

ectr

al

VM

ach

Execution time

gcc -O0Compcertgcc -O1gcc -O3

Compilation times: within a factor of 2 of gcc -O1.S. Blazy (INRIA-IRISA) CompCert, a Coq-certified compiler DGA-INRIA seminar 20 / 24

Outline




4 Perspectives


Enhancements to CompCert

Upstream :

Prove correct some of the C → Clight simplifications.

Is there anything to prove about the C parser? preprocessor??

In the middle :

More optimizations? Possibly using verified translation validation (e.g.instruction scheduling, lazy code motion, and software pipelining) ?

Downstream :

Currently, we stop at assembly language with a C-like memory model.

Refine the memory model to a flat array of bytes.(Issues with bounding the total stack size used by the program.)

Refine to real machine language?(Cf. Moore’s Piton & Gypsy projects circa 1995)


Extensions on the “input” side

Clight

Cminor

RTL

Asm

Compcert C

Lustre

Simulink

Mini-ML

GHC core

Java VM

Gallina

Code generators

Modelchecker

Programprover

Staticanalyzer

Verificationtools

1. Verification of front-ends / code generators for other source languages.2. Formal connections with verification tools.


Connections with verification tools

Consider other C-related tools involved in the production and verificationof critical software: code generators, static analyzers, model checkers,program provers, . . .

(Long term) Formally verify these tools as well?(E.g. verification condition generators, abstract interpreters, abstractdomains, etc)

(Medium term) Validate the operational semantics used in CompCertagainst the other semantics used in these tools?(E.g. axiomatic semantics, collecting semantics, etc)

(More modestly) Agree on a common subset of the C language?


Enforcing Secure Object Initialization

in Java

Laurent Hubert, Thomas Jensen,Vincent Monfort, and David Pichardie

CNRS / IRISAINRIA Rennes

D G A - I N R I A s e m i n a r15 OCTOBER 2010

vendredi 15 octobre 2010

Object Initialization

Initialization is often a critical phasesecurity checks may be performeddefense mechanisms may be installedoperations may be loggedfields are initializedthe object invariant is establishedetc.

Only fully initialized objects should be manipulated by the program

2


Object Initialization in Java

3

Object

Foo

Bar

Timeline of an instance of class BarAn instance of Bar



3

Object

Foo

Bar

Timeline of an instance of class Barallocation of an instance of BarAn instance of Bar



3

Object

Foo

Bar

Timeline of an instance of class Bar

call to the constructor of Barallocation of an instance of BarAn instance of Bar



3

Object

Foo

Bar


call to the constructor of Barcall to the super constructor (Foo)

allocation of an instance of BarAn instance of Bar



3

Object

Foo

Bar


call to the constructor of Barcall to the super constructor (Foo)call to the super constructor (Object)




3

Object

Foo

Bar



initialization of Object




3

Object

Foo

Bar




initialization of Foo




3

Object

Foo

Bar





initialization of Bar




3

Object

Foo

Bar






allocation of an instance of Bar

fully initialized

An instance of Bar


An instance of Bar


4






allocation of an instance of Bar

fully initialized

Enforced by the BCV

Not enforced by the

BCV

Object

Foo

Bar


Outlines

Object initialization and security

Example: the class loader

Oracle's solution

Our type-based solution

Extending our solution

Experimental evaluation

5


ExampleThe Class Loader

6

A security sensitive classMaps permissions to Class objectsDefines namespaces with the VMChecks (with the security manager) that the code has the permission to access or define classesRun with all permissions

Invariant established by the constructorthe caller has the permission CreateClassLoader

Its methods rely on this invariant


ExampleThe OriginaL Class Loader

public abstract class ClassLoader { protected ClassLoader() { SecurityManager sm = System.getSecurityManager(); if (sm != null) {sm.checkCreateClassLoader();} } protected final native void resolveClass(Class c);}

7


the Finalizer attackpublic abstract class ClassLoader { protected ClassLoader() { SecurityManager sm = System.getSecurityManager(); if (sm != null) {sm.checkCreateClassLoader();} } protected final native void resolveClass(Class c);}

class Attacker extends ClassLoader{ Attacker(){super();} // this call will fail public static void main(String args[]){ try{Attacker o = new Attacker();} ... }

}8


the Finalizer attackpublic abstract class ClassLoader { protected ClassLoader() { SecurityManager sm = System.getSecurityManager(); if (sm != null) {sm.checkCreateClassLoader();} } protected final native void resolveClass(Class c);}

class Attacker extends ClassLoader{ Attacker(){super();} // this call will fail public static void main(String args[]){ try{Attacker o = new Attacker();} ... }

}8

protected void finalize(){ this.resolveClass(...); // Attacker exploit !}


A Known Issue

9

Recommendation of the CERT (OBJ04-J) & Joshua Bloch in Effective Java

Source of several security bugs

Solution: proposed in Oracle's Secure Coding Guidelines for Java

Do not allow partially initialized objects to be accessed


Oracle's patch

public abstract class ClassLoader { private volatile boolean initialized;

protected ClassLoader() { SecurityManager sm = System.getSecurityManager(); if (sm != null) {sm.checkCreateClassLoader();} this.initialized = true;}

protected final void resolveClass(Class c){ if (!initialized) { throw new SecurityException( "ClassLoader object not initialized");} ... }}

10

Program your own monitor


Oracle's patch

public abstract class ClassLoader { private volatile boolean initialized;

protected ClassLoader() { SecurityManager sm = System.getSecurityManager(); if (sm != null) {sm.checkCreateClassLoader();} this.initialized = true;}

protected final void resolveClass(Class c){ if (!initialized) { throw new SecurityException( "ClassLoader object not initialized");} ... }}

11

Program your own monitor Lots of coding

Failure to adhere leads to

vulnerability

Not automated


Toward a Solution

Oracle's coding pattern denotes a policyMethods that requires the receiver to be sufficiently initialized test the initialized field

Making the policy explicit using annotationsfor each variable, states if it can hold a partially initialized objectvariables: fields, method parameters, receivers and return valuesformalized as a type system

12


Our Solution

Type based solutionNo additional codeOnly few type annotations are neededFormally definedProvably sound

13


The Type Annotations

14

Example of a class hierarchy Corresponding type lattice structure

A B

Object

Init

Raw

Raw(A) Raw(B)

Raw(Object)

Raw(C) [FähndrichLeino2003]

The constructor of C has terminated successfully

Sub-typing relation

Object

B

Raw(Object)



15

class Bar extends Foo{

private @Raw(Foo) Bar f;

public @Raw(Foo) Bar getF(){ this.f;}

public void setF(@Raw(Foo) Bar f2){ this.f = f2; }}

Example



15

class Bar extends Foo{

private @Raw(Foo) Bar f;

public @Raw(Foo) Bar getF(){ this.f;}

public void setF(@Raw(Foo) Bar f2){ this.f = f2; }}

Example

The receiver is denoted by @Pre and @Post

@Pre(@Init) @Post(@Init)



AnnotatedClass Loader

public abstract class ClassLoader {

protected ClassLoader() { SecurityManager sm = System.getSecurityManager(); if (sm != null) {sm.checkCreateClassLoader();} }

protected final native void resolveClass( Class c);}

16






16

@Pre(@Init)






16







16


@Pre(@Raw)






16


@Pre(@Raw) @Post(@Raw(ClassLoader))






16



@Init






16



@Init

Still too verbose...


Secure by Default Policy

Constructors@Pre(@Raw) @Post(@Raw(C)) C()

De-serialization@Pre(@Raw(C)) C.readObject(...);

Finalization@Pre(@Raw) finalize();

17

All accessible objects are fully initialized except during construction


18







@Init

Default annotations are sufficient for the class loader


18





Default annotations are sufficient for the class loader


Enforcing the Policy

Code generationFailure at run timeDebugging is more difficult

Static type checkingFailure at compile time or load timeDebugging is easier

19


Catching the Attacker

Type checking of the attacker now fails

20

@Pre(@Raw) protected void finalize(){ this.resolveClass(...); // Type error ! }

The annotation on finalize() methods is hard coded


Method Calls in Constructors 1/3

Methods may be called in constructorsIs the object sufficiently initialized?

The developer should answer

public StringBuffer(String str){ ...

this.append(str);}

@Pre(@Raw(StringBuffer))public StringBuffer append(String str) {...}



CT.setInit();

Declare an object as initialized up-to the current class before the end of the constructor

public StringBuffer(String str){ ... CT.setInit(); this.append(str);}

@Pre(@Raw(StringBuffer))public StringBuffer append(String str) {...}


Experimental results

Annotating Oracle's JREstudy on 3 packages: java.lang, java.security and javax.securitywe annotated theses classes to make explicit their policy

Type-checked classeswe have the guarantee that the methods of the class will respect their policyclasses should be checked at load time

23


Experimental results

We verified 377 classes (131 KLoc)+ Few annotations needed (47 @ + 6 setInit)+ Few runtime checks (4 cast operators)- synthetic methods (3)- limitation on arrays (1)

we forbid to store partially initialized objects in arrays

24


Conclusion

Object Initialization is sensitiveInvariants are established during initialization

Only fully initialized objects should be accessed

Not enforced by the ByteCode Verifier

Oracle's coding pattern: not satisfactoryFailure to adhere silently leads to vulnerability

25


Conclusion

A solution to a security weakness of the BCVA type system, formally definedA machine checked correctness proofA prototype implementation of the checker

Future workExtend the type system to arrays and genericsUse similar type-based approach for other secure coding guidelines

26


Thank youW e b D e m o a n d

M a c h i n e C h e c k e d P r o o f shttp://irisa.fr/celtique/ext/rawtypes


http://irisa.fr/celtique/ext/rawtypes

http://irisa.fr/celtique/ext/rawtypes

Cast Operators

28

Static type checking is incomplete

Introducing dynamic checking: cast operators y = CT.assertInit(x); y = CT.assertRaw(x,C.class);check x is sufficiently initialized

Give me back my monitor !


The Type System

Formalization as a type and effect systemType of local variables may evolve (flow sensitive)


The Type System

The typing rule for method calls

Formalization as a type and effect systemType of local variables may evolve (flow sensitive)



30

Typing rule for the setInit annotation


compcert, a coq-certified...

Documents