python-csp: bringing occam to python

Ancient historypython-csp: features and idioms

Using built-in processesFuture challenges

python-csp: bringing OCCAM to Python

Sarah Mount

Europython 2010

Sarah Mount python-csp: bringing OCCAM to Python



1 Ancient history

2 python-csp: features and idiomsProcess creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models

3 Using built-in processes

4 Future challenges




This talk. . .

is all about the python-csp project. There is a similar projectcalled PyCSP, these will merge over the next few months. Currentcode base is here: http://code.google.com/p/python-csp

Please do contribute bug fixes / issues / code / docs / rants / . . .


http://code.google.com/p/python-csp



INMOS B0042 Board c© David May

The Transputer was the original “multicore” machine and was builtto run the OCCAM language – a realisation of CSP.http://www.cs.bris.ac.uk/~dave/transputer.html


http://www.cs.bris.ac.uk/~dave/transputer.html



Tony Hoare

Invented CSP, OCCAM, Quicksort and other baddass stuff. Youmay remember him from such talks as his Europython 2009Keynote.




OCCAM

OCCAM lives on in it’s current implementation as OCCAM-πhttp://www.cs.kent.ac.uk/projects/ofa/kroc/

It can run on Lego bricks, Arduino boards and other things. LikePython it uses indentation to demark scope. Unlike PythonOCCAM is MOSTLY IN UPPERCASE.


http://www.cs.kent.ac.uk/projects/ofa/kroc/



CSP and message passingThis next bit will be revision for many of you!




Most of us think of programs like this

http://xkcd.com/205/


http://xkcd.com/205/



This is the old “standard” model!

Multicore will soon take over the world (if it hasn’t already).

Shared memory concurrency / parallelism is hard:

Race hazardsDeadlockLivelockTracking which lock goes with which data and what orderlocks should be used.Operating Systems are old news!




Message passing

Definition

In message passing concurrency (or parallelism) “everything” is aprocess and no data is shared between processes. Instead, data is“passed” between channels which connect processes together.Think: OS processes and UNIX pipes.

The actor model (similar to Kamaelia) uses asynchronouschannels. Processes write data to a channel and continueexecution.

CSP (and π-calculus) use synchronous channels, orrendezvous, where a process writing to a channel has to waituntil the value has been read by another process beforeproceeding.




Process diagram




Why synchronous channels?

Leslie Lamport (1978). Time, clocks, and the ordering of events ina distributed system”. Communications of the ACM 21 (7)




Process creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models

Hello World!

from csp . c s p p r o c e s s i m p o r t ∗

@ p r o c e s sd e f h e l l o ( ) :

p r i n t ’ H e l l o CSP w o r l d ! ’

h e l l o ( ) . s t a r t ( )





Example of process creation: web server

@ p r o c e s sd e f s e r v e r ( host , p o r t ) :

sock = s o c k e t . s o c k e t ( . . . )# Set up s o c k e t sw h i l e True :

conn , addr = sock . a c c e p t ( )r e q u e s t = conn . r e c v (4096) . s t r i p ( )i f r e q u e s t . s t a r t s w i t h ( ’GET ’ ) :

ok ( r e q u e s t , conn ) . s t a r t ( )e l s e :

n o t f o u n d ( r e q u e s t , conn ) . s t a r t ( )





Example of process creation: web server

@ p r o c e s sd e f n o t f o u n d ( r e q u e s t , conn ) :

page = ’<h1>F i l e not found</h1> ’conn . send ( r e s p o n s e (404 ,

’ Not Found ’ ,page ) )

conn . shutdown ( s o c k e t .SHUT RDWR)conn . c l o s e ( )r e t u r n





Combining processes: in parallel

# With a Par o b j e c t :Par ( p1 , p2 , p3 , . . . pn ) . s t a r t ( )# With s y n t a c t i c s u g a r :p1 // ( p2 , p3 , . . . pn )# RHS must be a s e q u e n c e t y p e .





Combining processes: repeating a process sequentially

@ p r o c e s sd e f h e l l o ( ) :

p r i n t ’ H e l l o CSP w o r l d ! ’i f n a m e == ’ m a i n ’ :

h e l l o ( ) ∗ 32 ∗ h e l l o ( )





Channels

@ p r o c e s sd e f c l i e n t ( chan ) :

p r i n t chan . r e a d ( )@ p r o c e s sd e f s e r v e r ( chan ) :

chan . w r i t e ( ’ H e l l o CSP wo r l d ! ’ )

i f n a m e == ’ m a i n ’ :ch = Channel ( )Par ( c l i e n t ( ch ) , s e r v e r ( ch ) ) . s t a r t ( )





What you need to know about channels

Channels are currently UNIX pipes

This will likely change

Channel rendezvous is “slow” compared with JCSP:200µ s vs 16µ s

This will be fixed by having channels written in C

Because channels are, at the OS level, open “files”, there canonly be a limited number of them (around 300 on mymachine)

This makes me sad :-(





Message passing an responsiveness

This isn’t about speed.

Sometimes, you have several channels and reading from themall round-robin may reduce responsiveness if one channel readblocks for a long time.

This is BAD:

@ p r o c e s sd e f f o o ( c h a n n e l s ) :

f o r chan i n c h a n n e l s :p r i n t chan . r e a d ( )





ALTing and choice

Definition

ALTing, or non-deterministic choice, selects a channel to read fromfrom those channels which are ready to read from.

You can do this with (only) two channels:

@ p r o c e s sd e f f o o ( c1 , c2 ) :

p r i n t c1 | c2p r i n t c1 | c2





ALTing proper (many channels)

@ p r o c e s sd e f f o o ( c1 , c2 , c3 , c4 , c5 ) :

a l t = A l t ( c1 , c2 , c3 , c4 , c5 )# Choose random a v a i l a b l e o f f e rp r i n t a l t . s e l e c t ( )

# Avoid l a s t s e l e c t e d c h a n n e lp r i n t a l t . f a i r s e l e c t ( )

# Choose c h a n n e l w i t h l o w e s t i n d e xp r i n t a l t . p r i s e l e c t ( )





Syntactic sugar for ALTing

@ p r o c e s sd e f f o o ( c1 , c2 , c3 , c4 , c5 ) :

gen = A l t ( c1 , c2 , c3 , c4 , c5 )p r i n t gen . n e x t ( )p r i n t gen . n e x t ( )p r i n t gen . n e x t ( )p r i n t gen . n e x t ( )p r i n t gen . n e x t ( )





ALTing: when you really, really must return right now

@ p r o c e s sd e f f o o ( c1 , c2 , c3 ) :

# S k i p ( ) w i l l / a l w a y s / complete# a r e a d i m m e d i a t e l ygen = A l t ( c1 , c2 , c3 , S k i p ( ) )p r i n t gen . n e x t ( ). . .





Channel poisoning

Definition

Idiomatically in this style of programming most processes containinfinite loops. Poisoning a channel asks all processes connected tothat channel to shut down automatically. Each process which hasa poisoned channel will automatically poison any other channels itis connected to. This way, a whole graph of connected processescan be terminated, avoiding race conditions.

@ p r o c e s sd e f f o o ( c h a n n e l ) :

. . .c h a n n e l . p o i s o n ( )





A bigger example: Mandelbrot generator





@ p r o c e s sd e f main ( IMSIZE ) :

c h a n n e l s = [ ]# One p r o d u c e r + c h a n n e l f o r each

image column .f o r x i n r an ge ( IMSIZE [ 0 ] ) :

c h a n n e l s . append ( Channel ( ) )p r o c e s s e s . append ( mande lbrot ( x ,

. . . c h a n n e l s [ x ] ) )p r o c e s s e s . i n s e r t ( 0 , consume ( IMSIZE ,

c h a n n e l s ) )mandel = Par (∗ p r o c e s s e s )mandel . s t a r t ( )





@ p r o c e s sd e f mande lbrot ( xcoord , cout ) :

# Do some maths h e r ecout . w r i t e ( xcoord , co lumn data )





@ p r o c e s sd e f consumer ( c i n s ) :

# I n i t i a l i s e PyGame . . .gen = l e n ( c i n s ) ∗ A l t (∗ c i n s )f o r i i n ran ge ( l e n ( c i n s ) ) :

xcoord , column = gen . n e x t ( )# B l i t t h a t column to s c r e e n

f o r e v e n t i n pygame . e v e n t . g e t ( ) :i f e v e n t . t y p e == pygame . QUIT :

f o r c h a n n e l i n c i n s :c h a n n e l . p o i s o n ( )

pygame . q u i t ( )




Oscilloscope




@ p r o c e s sd e f O s c i l l o s c o p e ( i n c h a n ) :

# I n i t i a l i s e PyGameydata = [ ]w h i l e not QUIT :

ydata . append ( i n c h a n . r e a d ( ) )ydata . pop ( 0 )# B l i t data to s c r e e n . . .

# Deal w i t h e v e n t s




from csp . b u i l t i n s i m p o r t S i nd e f t r a c e s i n ( ) :

chans = Channel ( ) , Channel ( )Par ( G e n e r a t e F l o a t s ( chans [ 0 ] ) ,

S i n ( chans [ 0 ] , chans [ 1 ] ) ,O s c i l l o s c o p e ( chans [ 1 ] ) ) . s t a r t ( )

r e t u r n




Process diagram




d e f t r ac e m ux ( ) :chans = [ Channel ( ) f o r i i n ra ng e ( 6 ) ]par = Par ( G e n e r a t e F l o a t s ( chans [ 0 ] ) ,

D e l t a 2 ( chans [ 0 ] , chans [ 1 ] ,chans [ 2 ] ) ,

Cos ( chans [ 1 ] , chans [ 3 ] ) ,S i n ( chans [ 2 ] , chans [ 4 ] ) ,Mux2( chans [ 3 ] , chans [ 4 ] ,

chans [ 5 ] ) ,O s c i l l o s c o p e ( chans [ 5 ] ) )

par . s t a r t ( )




Wiring


python-csp: bringing occam to python

Technology

pythoncsp project

r t sarah mount pythoncsp

ancient history pythoncsp

com205sarah mount pythoncsp

ancient history2 pythoncsp

occam t

ancient historypythoncsp

realisation of csp