doing more with contracts bertrand meyer, ilinca ciupa, andreas leitner, lisa liu
DESCRIPTION
Doing more with contracts Bertrand Meyer, Ilinca Ciupa, Andreas Leitner, Lisa Liu SOFSEM, Harrachov, January 2007 ETH Zurich, Chair of Software Engineering. The three questions. What does it expect? What does it promise? What does it maintain?. Progress in software quality. - PowerPoint PPT PresentationTRANSCRIPT
Chair of Software Engineering1
Doing more with contracts
Bertrand Meyer, Ilinca Ciupa, Andreas Leitner, Lisa LiuSOFSEM, Harrachov, January 2007
ETH Zurich, Chair of Software Engineering
2
The three questions
What does it expect?
What does it promise?
What does it maintain?
3
Progress in software quality
Tools, techniques, methods, languages,software quality in general…
Market forces work not for very high quality but for good enough software
Collective decision to turn IT into a service business
… have been improving steadily…
… but not enough!
4
Where best to apply our efforts?
End-user applications
Compilers & operating systems
General-purpose components
Specialized components
Relies on
5
Components should be contracted
Definition of what each element of the functionality:
Expects (precondition) Promises (postcondition) Maintains (invariant)
Does not have to be complete (but wait)
6
Design by Contract
Every software element is intended to satisfy a certain goal, for the benefit of other software elements (and ultimately of human users).
This goal is the element’s contract.
The contract of any software element should be Explicit Part of the software element itself
7
A class with contracts
classBANK_ACCOUNT
createmake
featuremake (n : STRING)
-- Set up with name n
requiren /= Void
doname := nbalance := 0
ensurename = n
end
name : STRING
balance : INTEGER
deposit ( v : INTEGER)
-- Add amount v
do
balance := balance + v
ensure
balance = old balance + v
end
invariant
name /= Void
balance >= 0
end
ensurename =
n
requiren /=
Void
ensure balance = old balance
+ v
invariantname /=
Voidbalance >= 0
8
The correctness of a class
For every creation procedure cp :
{precp} docp {INV and postcp}
For every exported routine r :
{INV and prer} dor {INV and postr}
a.h (…)
create a.make (…) S1
S2
S3
S4
a.g (…)
a.f (…)
9
What we do with contracts today
Write better software Analysis Design Reuse Implementation Bug avoidanceDocument software automaticallyHelp project managers do their job
Perform systematic testingGuide the debugging process
(with run-time monitoring)
10
deferred class VAT inherit
TANK
feature
in_valve, out_valve: VALVE
fill is-- Fill the vat.
require in_valve.open out_valve.closed
deferred ensure
in_valve.closed out_valve.closed is_full
end
empty, is_full, is_empty, gauge, maximum, ... [Other features] ...
invariant
is_full = (gauge >= 0.97 maximum) and (gauge <= 1.03 maximum)
end
Contracts for analysis, specification
11
Seamless development with Eiffel & contracts
Single notation, tools, concepts, principles
Continuous, incremental development
Keep model, implementation, documentation consistent
Reversibility
Example classes:PLANE, ACCOUNT, TRANSACTION…
STATE, COMMAND…
HASH_TABLE…
TEST_DRIVER…
TABLE…
Analysis
Design
Implemen-
tation
V&V
Generali-zation
12
Documentation: the contract view of a class
classBANK_ACCOUNT
createmake
featuremake (n : STRING)
-- Set up with name n
requiren /= Void
doname := nbalance := 0
ensurename = n
end
name : STRING
balance : INTEGER
deposit ( v : INTEGER)
-- Add amount v
do
balance := balance + v
ensure
balance = old balance + v
end
invariant
name /= Void
balance >= 0
end
ensurename =
n
requiren /=
Void
ensure balance = old balance
+ v
invariantname /=
Voidbalance >= 0
13
Lists with cursors
"Prague"
Cursor
index
count1
forth
afterbefore
14
Command “forth″
15
Where the cursor may go
count+1
Valid cursor positions
item
count1
afterbefore
0
16
From the invariant of class LIST
Valid cursor positions
17
Contracts and inheritance
r isrequire
ensure
r isrequire
ensure
a1: A
a1.r (…)…
Correct call in C:
if a1. then
a1.r (...) -- Here a1. holds end
r ++
C A
D B
ClientInheritance
++ Redefinition
18
Assertion redeclaration rule
When redeclaring a routine, we may only:
Keep or weaken the precondition
Keep or strengthen the postcondition
19
A simple language rule does the trick!
Redefined version may have nothing (assertions kept by default), or
require else new_preensure then new_post
Resulting assertions are: original_precondition or new_pre
original_postcondition and new_post
Assertion redeclaration rule in Eiffel
20
Contracts as a management tool
High-level view of modules for the manager:
Follow what’s going on without reading the code
Enforce strict rules of cooperation between units of the system
Control outsourcing
21
Contracts for testing and debugging
Contracts provide the right basis: Testing is there to find bugs A bug is a discrepancy between intent and reality Contracts describe intent
A contract violation always signals a bug: Precondition violation: bug in client Postcondition violation: bug in routine
In EiffelStudio: select compilation option for run-time contract monitoring at level of class, cluster or system.
22
Anecdotal & non-anecdotal evidence
HP 1: invariant
r = 2 ^ i
HP 2: Eiffel messes up our great system!
Axa Rosenberg: postcondition fails in deep_clone of TWO_WAY_LIST !
Patrice Chalin study (Concordia): Eiffel programmers do use contracts day in, day out.
23
Should we test software?
Some evidence of why we should: Limitations of proof technology Properties not formally specified
Tests as complement of proofs?
From a survey of 240 companies in N. America and Europe:
8% release software to beta sites without testing
83% of their developers don't like to test code.
53% don't like to test their own code because they find it tedious.
30% don't like to test because they find testing tools inadequate.
Testing is tedious
25
“Automated testing”
What can be automated:
Test execution
Robust execution
Regression testing
Test case generation (test suites)
Test result verification (test oracles)
Test scheduling
Test case minimization
Automated vs. manual unit tests
Automatically generated test cases complement manual ones:
Contracts may be “wrong” Implementation is unlikely to establish
postcondition on input satisfying A -> ¬B . Test cases will likely only be written for B (and not
for A -> ¬B ).
27
AutoTest: an automatic test framework
Input is a set of classes AutoTest generates instances and calls features
with automatically selected arguments Automatic tests rely on contracts:
Precondition violations: skip Postcondition/invariant violation: bingo!
Manual tests can be added explicitly:Relevant ones are determined automaticallyAny test (manual or automated) that fails
becomes part of the test suite
28
AutoTest demo
29
Automated testing and slicing
auto_test system.ace BANK_ACCOUNT STRING
create {STRING} v1v1.wipe_out
v1.append_character (’c’)
v1.append_double (2.45)create {STRING} v2v1.append_string (v2)
v2.fill (’g’, 254343)...create {BANK_ACCOUNT}
v3.make (v2)v3.deposit (15)
v3.deposit (100)
v3.deposit (-8901)...
classBANK_ACCOUNT
createmake
featuremake (n : STRING)
requiren /= Void
doname := nbalance := 0
ensurename = n
end
name : STRING
balance : INTEGER
deposit (v : INTEGER)
do
balance := balance + v
ensure
balance =
old balance + v
end
invariant
name /= Void
balance >= 0
end
Robust execution
31
Some AutoTest results (random strategy)
Library Total Failed Total Failed
EiffelBase 40,000 3% 2000 6%
Data Structures 21,000 5% 1400 6%
Gobo Math 1500 1% 140 6%
TESTS ROUTINES
32
Result analysis
(Raluca Borga, U. of Cluj-Napoca,diploma work at ETH)
Fault classification (manual): Specification error Implementation error Inheritance Don’t know
About 50% implementation, 50% specification Results polluted by “void” issue (Will no longer
exist thanks to attached types, see ECOOP 2005 paper)
AutoTest strategies
Random strategy
Use random input
Planning strategy
Employ information from postcondition to satisfy preconditions
...
34
Basic strategy
Object pool For each type, initialize through creation
procedures (constructors) This requires proper ordering of classes Diversify through procedures
Routine arguments
Basic values: heuristics for each type
Objects: get from pool Invoke selected routines AutoTest tests all routines, including inherited ones
(“Fragile base class” issue)
35
Basic idea: random testing may find bugs faster if inputs evenly spread over range of possible values
Example:r (m, n: INTEGER) do … end
m
n
Random inputs
m
n
ART
Adaptive Random Testing (ART, Chen et al.)
36
So far ART only applied to numeric inputs, on which a total order relation exists
Multi-field objects are not members of a totally ordered set
Need to define notion of distance between objects
Extending ART to objects
37
Object distance (Ilinca Ciupa)
p ↔ q
combination (
type_distance (p.type, q.type),
field_distance (p, q),recursive_distance (
{[p.r ↔ q.r] | r Reference_attributes } )
=Δ
38
The role of argument-less boolean queries
Externally, boolean queries often serve as preconditions and appear in invariants
Internally, boolean queries often serve as branch conditions
Lists: before, after, off, is_first, is_last, is_empty…Account: is_overdraft…
"Prague"
Cursor
index
count1
afterbefore
is_first
is_last
39
Boolean query conjecture
The argument-less boolean queries of a well-written class yield a partition of the object state space that helps the testing process.
40
Boolean query coverage (Lisa Liu)
A set of tests for a class satisfies boolean query coverage if and only if the execution of these tests can cover all the reachable abstract object states for that class.
Strategy:
Through constraint solving, generate states that satisfy boolean queries in invariant
(e.g. after implies off)
Through theorem proving, remove states that do not satisfy rest of invariant
(e.g. count < capacity)
Chair of Software Engineering41
"Prague"
Cursor
index
count1
afterbefore
is_first
is_last
42
Experimental results
Class Random Boolean Queries
RoutineCoverage
Bugs Found
Routine Coverage
Bugs Found
INT_STACK 33% 1 100% 2
LINKED_LIST 85% 1 99% 7
BINARY_TREE
88% 5 100% 11
ARRAYED_SET
84% 1 100% 6
FIXED_LIST 93% 12 99% 12
43
AutoTest developments
Large-scale extensive tests (cluster computing)
Comparison with manual efforts
Scaling up (memory, time) (mostly done)
Rigorous assessment of testing criteria & strategies
Complete integration with EiffelStudio environment
Background, unobtrusive, continuous testing
Automatic regression testing
Distributed cooperative testing (“Testi@home”)
44
Towards a real framework
Pluggable strategies Flexible execution mechanism Robust execution mechanism Unified interface Integrate tightly with manual testing Concise output
45
Proving classes
classLINKED_LIST [G]
feature…remove_front
-- Remove first itemrequire
not emptydo
first := first.rightensure
count = old count – 1first = old item (2)
end
…
end
first
right right right
46
Issues in proving classes
1. Providing full contracts
2. Devising models of the underlying world
3. Devising proof rules (i.e. semantics of the programming language)
4. Taking advantage of inheritance on the proof side
5. Doing the proofs, mechanically
47
The contract language
In Eiffel (and other formalisms based on Design by Contract): mostly boolean expressions of the language, plus some extensions such as old.
Seems not expressive enough — but is!
48
Beefing up expressive power
Eiffel Model Library
Components to prove(e.g. EiffelBase)
49
Eiffel Model Library (also known as MML)
(With Bernd Schoeller, Tobias Widmer)Classes correspond to mathematical concepts:
SET [G], FUNCTION [G, H ], TOTAL_FUNCTION [G, H ], RELATION [G, H ], SEQUENCE [G ], …
Completely applicative: no attributes (fields), no implemented routines (all completely deferred)
Specified with contracts (unproven) reflecting mathematical properties
Expressed entirely in Eiffel
50
Example MML class
class SEQUENCE [G] feature
count : NATURAL-- Number of items
last : G-- Last item
extended (x) : SEQUENCE [G]-- Identical sequence except x added at end
ensureResult.count = count + 1
Result.last = x
Result.sub (1, count) ~ Current
mirrored : SEQUENCE [G]-- Same items in reverse order
ensureResult.count = count…
…end
51
Specifying lists
classLINKED_LIST [G]
feature…remove_front
-- Remove first itemrequire
not emptydo
first := first.rightensure
end
…
end
first
right right right
count = old count – 1first = old item (2)
model = old
model.tail
52
Principles
Very simple mathematics only Logic Set theory
53
Proofs: current progress
Basic theory of O-O programming with pointers
Progress towards frame issue
Proof engine: Boogie (Microsoft Research)
Several publications in progress
54
Tests or proofs?
Tests and proofs! Failed proof: proof obligations remain –
candidates for tests Model checking counter-example: try to generate
test data that exercises it
TAPTests And Proofs
ETH Zurich, 11-13 Feb 2007
http://tap.ethz.ch
55
Concurrency
What do contracts become in a concurrent context?
56
put
57
store (b : [G ] ; v : G )
-- Store v into b. require
not b.is_full do
… ensure
not b.is_empty end
QUEUE BUFFER
my_queue : [T ]
…
if not my_queue.is_full then
store (my_queue, t )
end
BUFFER QUEUE
put
58
On separate target, precondition becomes wait condition
From preconditions to wait-conditions
store (buffer : separate BUFFER [INTEGER ] ; v : INTEGER) -- Store v into buffer.
requirenot buffer.is_fullv > 0
dobuffer.put (v)
ensurenot buffer.is_empty
end...store (my_buffer, 10 )
59
SCOOP in a nutshell
No intra-object-concurrency One keyword: separate, indicates thread of control
is “elsewhere” Reserve one or more objects through argument
passing Preconditions become wait conditions Automatic resynchronization on queries (lazy wait) Exception-based mechanism to break lock Asynchronous exceptions
60
Dining philosophers
class PHILOSOPHER inheritPROCESS
renamesetup as getup
redefine step end
feature {BUTLER}step do think ; eat (left, right)
end
eat (l, r : separate FORK) -- Eat, having grabbed l and r.
do … endend
61
Contract-Driven Development: demo
(Andreas Leitner)
62
Introductory programming course
Since 2003200-300 students
Uses object technology, Eiffel, Design by Contract
Outside-in approach (“inverted curriculum”): work from existing software
TRAFFIC library
Textbook in progress, Touch of Class, available onlinegoogle touch class meyer
63
Traffic GUI example
64
Traffic GUI example
65
The Traffic software (Michela Pedroni)
EiffelMedia- OpenGL (3D Drawing)- 2D Drawing Primitives- Sound- Collision detection
Gobo- Data structures- XML parsing- File handling
EWG- Library wrapping
Traffic- 2D and 3D display of a city- Transportation lines, roads, schedules- Random buildings- Vehicles, passengers
EiffelBase- Data structures- Kernel classes- Basic data types
66
The background technology: Eiffel
Method, language, IDE (EiffelStudio), libraries
Constantly refined since 1985
ECMA standard (2005); ISO standard (2006)
67
Some Eiffel projects
Axa RosenbergInvestment management: from $2 billion to >$40 billion 2 million lines of EiffelChicago Board of Trade Price reporting system Eiffel + CORBA + Solaris + Windows + …Boeing (Xontech) Large-scale Eiffel simulations of missile defense systemSwedish social security: accident reporting & management
Chair of Software Engineering68
Open-Source EiffelStudio
69
References
Object-Oriented Software Construction,2nd edition (Prentice Hall)
se.ethz.ch
eiffelstudio.origo.ethz.ch