Putting a Hit on Bugs with Code Contracts

Software needs Reliability• Two Components

Correctness• Does what it’s supposed to do and only that

Robustness• Acts appropriately in cases where it cannot do what it is

supposed to do

But What’s it Supposed to Do?

And how do we normally define that?

Describing Software

• How do developers express what software is supposed to do?A. Write it in English, allowing your users/clients to

approve it beforehand?

B. Write it in the comments?

C. Describe it in a format system based on discrete mathematics

D. Poorly?

E. All of the above?

Correctness of a Routine

• State the conditions that must be true before the routine can work correctlyPre-conditions

• State the conditions that will be true after execution, if the routine has worked correctlyPost-conditions

Correct routine = pre- and post-conditions met

Let’s Consider an Example• Create a class that implements a time of day

Exposes hour, minute, second properties Implementation could be as three separate integers or as

the number of seconds since midnight

• We’re going to look only at the process of assigning the hour

What do we know?

• Document assumptionsPreconditions, postconditions, invariants

• Are executableCan perform checks at run-time

• Help with static verificationAssist with early error detectionCan be used by tools to generate test cases

• Different than assertionsAssertions are not viewed as a contract, they are

a suggestionDifficult to use with test case generation tools

Contracts

• RequiresWhat must be true at method entry

• EnsuresWhat must be true at method exit Includes exits on exceptions

• InvariantsWhat must be true at all method exits

• AssertionsWhat must be true at a particular point

• AssumptionsWhat should be true at a particular point

What Contracts Can I Write?

• Any boolean expression In your favorite programming language! Including method calls (but must

be marked Pure)• Contract.Result

refer to the return value of the method• Contract.OldValue

refer to values at method entry• Quantifiers

Contract.ForAll(0,A.Length, Function(i) A(i) > 0);Contract.Exists(0,A.Length, Function(i) A(i) > 0);

What Can I Put In A Contract?

.method public hidebysig newslot virtual instance int32 Add(object 'value') cil managed{ ldarg.1 ldnull ceq ldc.i4.0 ceq call void [Microsoft.Contracts]Microsoft.Contracts.Contract::Requires(bool) ldarg.0 call instance int32 TabDemo.BaseList::get_Count() ldarg.0 call instance int32 TabDemo.BaseList::get_Count() call !!0 [Microsoft.Contracts]Microsoft.Contracts.Contract::Old<int32>(!!0) ldc.i4.1 add ceq call void [Microsoft.Contracts]Microsoft.Contracts.Contract::Ensures(bool) call !!0 [Microsoft.Contracts]Microsoft.Contracts.Contract::Result<int32>() ldarg.0 call instance int32 TabDemo.BaseList::get_Count() call !!0 [Microsoft.Contracts]Microsoft.Contracts.Contract::Old<int32>(!!0) ceq call void [Microsoft.Contracts]Microsoft.Contracts.Contract::Ensures(bool) ldarg.0 ldfld int32 TabDemo.BaseList::count ldarg.0 ldfld object[] TabDemo.BaseList::items ldlen conv.i4 ceq ldc.i4.0 ceq stloc.1 ldloc.1 brtrue.s IL_0069 ldarg.0 ldarg.0 ldfld int32 TabDemo.BaseList::count ldc.i4.1 add call instance void TabDemo.BaseList::EnsureCapacity(int32) ldarg.0 ldfld object[] TabDemo.BaseList::items ldarg.0 ldfld int32 TabDemo.BaseList::count ldarg.1 stelem.ref ldarg.0 dup ldfld int32 TabDemo.BaseList::count dup stloc.2 ldc.i4.1 add stfld int32 TabDemo.BaseList::count ldloc.2 stloc.0 br.s IL_008b ldloc.0 ret} // end of method BaseList::Add

csc/vbc/…

csc/vbc/…

.method public hidebysig newslot virtual instance int32 Add(object 'value') cil managed{ ldarg.0 ldfld int32 TabDemo.BaseList::count ldarg.0 ldfld object[] TabDemo.BaseList::items ldlen conv.i4 ceq ldc.i4.0 ceq stloc.1 ldloc.1 brtrue.s IL_0029 ldarg.0 ldarg.0 ldfld int32 TabDemo.BaseList::count ldc.i4.1 add call instance void TabDemo.BaseList::EnsureCapacity(int32) ldarg.0 ldfld object[] TabDemo.BaseList::items ldarg.0 ldfld int32 TabDemo.BaseList::count ldarg.1 stelem.ref ldarg.0 dup ldfld int32 TabDemo.BaseList::count dup stloc.2 ldc.i4.1 add stfld int32 TabDemo.BaseList::count ldloc.2 stloc.0 br.s IL_004b ldloc.0 ret}

ReleaseCompile

/d:CONTRACTS_FULL

ccrewrite

Executable Runtime Contract Checking

Public Overridable Function Add(value As Object) As Integer{ Contract.Requires( value IsNot Nothing )

Contract.Ensures( Count = Contract.OldValue(Of Integer)(Count) + 1 ) Contract.Ensures( Contract.Result(Of Integer)() = _ Contract.OldValue(Of Integer)(Count) )

if (_size == _items.Length) EnsureCapacity(_size+1); _items[_size] = value; return _size++;}

.method public hidebysig newslot virtual instance int32 Add(object 'value') cil managed{ .locals init (int32 'Contract.Old(Count)', int32 'Contract.Result<int>()') ldarg.0 call instance int32 TabDemo.BaseList::get_Count() stloc.3 ldarg.1 ldnull ceq ldc.i4.0 ceq ldstr "value != null" call void __RewriterMethods::RewriterRequires$PST06000009(bool, string) ldarg.0 ldfld int32 TabDemo.BaseList::count ldarg.0 ldfld object[] TabDemo.BaseList::items ldlen conv.i4 ceq ldc.i4.0 ceq stloc.1 ldloc.1 brtrue IL_004d nop ldarg.0 ldarg.0 ldfld int32 TabDemo.BaseList::count ldc.i4.1 add call instance void TabDemo.BaseList::EnsureCapacity(int32) nop nop ldarg.0 ldfld object[] TabDemo.BaseList::items ldarg.0 ldfld int32 TabDemo.BaseList::count ldarg.1 stelem.ref ldarg.0 dup ldfld int32 TabDemo.BaseList::count dup stloc.2 ldc.i4.1 add stfld int32 TabDemo.BaseList::count ldloc.2 stloc.0 br IL_0072 ldloc.0 stloc.s 'Contract.Result<int>()' br IL_007a ldarg.0 call instance int32 TabDemo.BaseList::get_Count() ldloc.3 ldc.i4.1 add ceq ldstr "Count == Contract.Old(Count) + 1" call void __RewriterMethods::RewriterEnsures$PST0600000B(bool, string) ldloc.s 'Contract.Result<int>()' ldloc.s V_4 ceq ldstr "Contract.Result<int>() == Contract.Old(Count)" call void __RewriterMethods::RewriterEnsures$PST0600000B(bool, string) ldloc.s 'Contract.Result<int>()' ret}

• No silver bulletBut helps catch errors earliestBest used in a focused manner

• Guides developmentDiscovers implicit assumptionsPropagates assumptions

• Not only explicit contractsDereferencing null Indexing arraysArithmetic exceptions

Static Contract Checking

What Do You Ship?

srcsrc

srcsrc

PowerLib.dll

(minimal runtime checks)

PowerLib.Contracts.dll

All contracts, no code

+

ReleaseAssemblies

Contract ReferenceAssemblies

<ContractClass(GetType(CloneableContract))> _Public Interface ICloneable

Function Clone() As Object

End Interface

Interface Contracts

ContractClassFor(GetType(ICloneable))> _Public Class CloneableContract Implements ICloneable

Public FunctionClone() As Object Implements Icloneable.Clone Contract.Ensures( Contract.Result(Of Object>() IsNot Nothing)

… End Function

End Class

All classes implementing the interface inherit the contract

• Contract library class enables contracts in all .NET languagesNo restrictions on what can be expressed

• Contracts are being used in the BCL todayContract library is a core component of .NET 4.0

• Same contracts used forRuntime checkingStatic checkingDocumentation generation

Code Contracts Summary

And Testing

• Testing is tedious• Too easy to miss cases• Old tests get stale• Too much legacy code

Why People don’t Write Tests

• Pex can be used to generate comprehensive test suite with high code coverage

• Pex finds contract violations and potential error situations

• The generated test suite integrates automatically with Visual Studio Team Test

What The Demo Showed

• Pex does not generate random inputs, enumerate all possible values, or make you write test input generators

• Instead, Pex analyzes your .NET code. Test inputs computed by Z3, Precise inter-procedural, path-sensitive analysis

• As a result, you geta small test suite with high code coverate coverage

Pex Understands The Code

• Pex generates small test suites with high code coverage and bug reports for free

• Reduce test maintenance costsby parameterized unit testing

• Pex has been used in Microsoftto test core .NET componentsAlmost always finds new bug pathways

Pex Summary

• Code Contracts for .NET:http://research.microsoft.com/Contracts/

• Pex: test generation for .NET http://research.microsoft.com/Pex/

Summary

Questions?• My contact information

EMail: bjohnson@objectsharp.comTwitter: LACanuckBlog: http://www.objectsharp.com/blogs/bruceMSN: lacanadians@hotmail.com