seefm 07, thessaloniki, nov 20071/ teaching the construction of correct programs using invariant...

SEEFM 07, Thessaloniki, Nov 2007 1/

Teaching the construction of correct programs using

invariant based programming

Ralph-Johan Back Johannes Eriksson Linda Mannila

Åbo Akademi / Dept. of Information TechnologiesTurku, Finland


Formal methods in CS education

Formal methods are perceived as difficult and requiring mathematical sophistication

The CS curriculum is divided into “theory” and “practice”

Formal methods taught independently of programming courses

Students get impression that formal methods are not applicable in practice

Testing and debugging is therefore the main (only) programming method that they learn from CS studies


Overview of talk

A short introduction to invariant based programming

The Socos tool Teaching formal methods at Åbo Akademi using

invariant based programming Experience report on a first year course on



Constructing correct programs

Program code

Contracts

Invariants

Verification conditions

“a posteriori verification”“constructive approach”“invariant based programming”


Example: Sort an array!

A=A0A: Int[N]

Sorted(A,0,N)A: Int[N]

Permutation(A,A0)

Start with a pre-/postcondition specification



A=A0 Sorted(A,0,N)A: Int[N] A: Int[N]

Permutation(A,A0)

Structure according to invariants


Construct a loop


A=A0 Sorted(A,0,N)

A: Int[N]

k: Int0≤k≤NSorted(A,0,k)∀i,j:Int • 0≤i<k ∧ k≤j<N ⇒ A[i]≤A[j]

Permutation(A,A0)

0 k N

sorted

un-sortedless than or

equal to all A[k..N-1] !

LOOP


Add initial transition


A=A0 Sorted(A,0,N)

A: Int[N]

Permutation(A,A0)

k: Int0≤k≤NSorted(A,0,k)∀i,j:Int • 0≤i<k ∧ k≤j<N ⇒ A[i]≤A[j]k:=0

A: Int[N]

⇒

0: Int0≤0≤NSorted(A,0,0)∀i,j:Int • 0≤i<0 ∧ 0≤j<N ⇒ A[i]≤A[j]

A: Int[N]

Permutation(A,A0)

✔

A=A0

✔

✔✔✔✔

What needs to be checked?



A=A0 Sorted(A,0,N)

A: Int[N]

Permutation(A,A0)


[k=N]

Add exit transition

Trivial:Sorted(A,0,k) ∧ k=N

⇒ Sorted(A,0,N)



A=A0 Sorted(A,0,N)

A: Int[N]

Permutation(A,A0)


[k=N]

[k<N]

m:=min(A,k,N);A:=A[ k←A[m], m←A[k] ];k:=k+1

Add loop transition

A: Int[N]Permutation(A,A0)k: Int0≤k≤NSorted(A,0,k)∀i,j:Int • 0≤i<k ∧ k≤j<N ⇒ A[i]≤A[j]

A’: Int[N]Permutation(A’,A0)k+1: Int0≤k+1≤NSorted(A’,0,k+1)∀i,j:Int • 0≤i<k+1 ∧ k+1≤j<N ⇒ A’[i]≤A’[j]

k<Nm=min(A,k,N) ∧ A’= A[ k←A[m], m←A[k] ]

⇒


Example: Sort an Array!

A=A0 Sorted(A,0,N)

A: Int[N]

Permutation(A,A0)


[k=N]

[k<N]

A:=Swap(A,k,min(A,k,N));k:=k+1

0≤N-k

Add a termination function

Variant decreases:N-(k+1) < N-k

Bounded from below:0≤k≤N ⇒ 0≤N-k


Socos: a prototype environment for:

teaching formal methods

state-of-the-art automatic and interactive verification



Invariant Diagrams in SOCOS


Verification in SOCOS

Three types of verification conditions: Consistency (for transitions) Completeness (for situations) Termination (for loops)

Verification conditions are sent to external proof tools

Simplify (automatic proofs), PVS (interactive proof checking)

Prover daemon works in the background Monitors changed files (Re)generates verification conditions and reruns proofs


Backends

Testing

Diagram is converted to a Python program, with run-time evaluation of invariants

Testing

Diagram is converted to a Python program, with run-time evaluation of invariants

Static Checking

Verification conditions are sent to Simplify, a fully automatic prover

Static Checking

Verification conditions are sent to Simplify, a fully automatic prover

Full Verification

PVS is used for full verification of the final components

Full Verification

PVS is used for full verification of the final components

Higher assurance→


Teaching invariant based programming at Abo

Akademi Spring 2005: Ph.D. course on invariant

based programming -- testing out the idea

Spring 2007: a course on IBP for first year students

2008 -- : IBP now part of standard CS curriculum

Planned next step: teaching IBP as a special math course in high school


New first year CS curriculum

Mathematics courses algebra probability theory

Computer Science courses Introduction to CS Python programming (to show that

programming is fun) Structured derivations (to teach

mathematical and logical reasoning) Invariant based programming (to teach how

to construct programs that are correct) Java programming Systems design course

Formalmethods bundle


Invariant based programming course (spring 2007)

aimed at first or second year students

interactive, emphasizing student participation

17 sessions a 90 min 11 lectures 6 practical excercises

Socos tool support only used in 4 last sessions only automatic proofs (Simplify), no PVS proofs

16 active participants half with no background in formal methods


Collecting data about the course

pre- and postcourse questionairs

observations

hand-in assignments

final exam

8 students selected for semi-structured interview


Main results - 1

The students found the course useful, interesting, somewhat fun and of medium difficulty level.

On average, students found invariant based programming

rather easy to learn, useful in practice and made the general structure of the program more

comprehensible


Main results - 2

Difficulties were mainly in constructing proofs and finding the invariant for more complex programs

The programs written by the students show that they had understood the idea behind IBP, and were able to construct and prove simple

invariant based programs.


Main results - 3

The students appreciated the diagrammatic notation of IBP

most students are visual learners, textual programming languages or pseudocode may not be

the best way for expressing algorithms to these students

We had expected that identifying the invariants would be the most difficult task, but this was not the case

writing proofs by hand seemed to be most problematic, as they required much time and effort

formulating postconditions was also sometimes problematic

Students found that IBP provides good support for finding bugs during the program construction

instead of after the program is ready


Main results - 4 Starting with informal reasoning in the course before introducing the

formal framework was not appreciated the students would have wanted the formal proof obligations to be introduced

earlier it seems that students who are not mathematically mature do not know how

to reason ”informally” but first need to learn a formal approach with a fixed set of rules

Socos supporting a formal method with a computer based tool in the course was

very well received the students preferred SOCOS over pen and paper, as the automation

increased productivity. unfamiliarity with the SOCOS syntax was the main cause of difficulty


Thank You

http://mde.abo.fi/SOCOS

seefm 07, thessaloniki, nov 20071/ teaching the construction of correct programs using invariant...

Documents

programming method

practice formal methods

finland slide

cs education formal

a0 sorteda

intn sorteda

programming courses

intn permutationa