Deferred Cancellation of Units of Execution

Philipp Bachmann

Institute for Medical Informatics and Biostatistics

Clarastrasse 12

CH–4058 Basel, BS

Switzerland

Revision: 1.7 ∗

October 4, 2008

Abstract

The pattern proposed helps to implement a portable termination andload adaptation mechanism for Active Objects [SSRB02a] and similar de-signs that delegate work to a pool of Units of Execution [MSM04, pp217–221] to execute service requests asynchronously from their actual in-vocation.

For the pattern proposed we identified usage examples in popular ex-isting applications or libraries.

An example accompanies the pattern presentation. This exampleshows sample code in C++.

A behavioral pattern

1 Intent

Safely shut down reactive systems without resource leakages.

2 Example

A webserver can deliver content to many clients in parallel. One implementationoption to accomplish this is a pool of Units of Execution [MSM04, pp 217–221]—processes or threads—, each serving at most a single client at any one time. AllUnits of Execution run the same code, it’s the data that differs. The codebasically consists of an infinite loop that contains a blocking system call to

∗Copyright c© 2008 Philipp Bachmann <bachlipp@web.de> and the Institute for MedicalInformatics and Biostatistics. Permission is granted to copy for the 15th Pattern Languagesof Programs (PLoP) conference 2008. All other rights reserved.

1

5 FORCES

temporarily yield execution of a Unit until a client requests another webpage.During the implementation of a webserver it will turn out, that the provisionfor the startup of the Units is the least problem. It is much more difficult todevelop a correct way to shut down the server again or to dynamically adaptthe number of workers to varying load.

3 Context

Reactive systems, e.g. UNIX daemons or MS Windows services, are imple-mented as Active Objects [SSRB02a] or use the Half–Sync / Half–Async [SC96,SSRB02b] or the Leader / Followers pattern [SSRB02c], [Ste98, pp 754–756].These approaches are subsumed under the Worker Threads pattern [Lea99, pp290–298], which in turn is a realization of the pattern Share the Load [Mes96,pp 588–589].

Monitor Objects [SSRB02d] and Reactors [SP, Sch95, SSRB02f] presentblocking system calls to the Units of Execution within those reactive systems.

4 Problem

How to destroy an Active Object?

Active Objects and the similar concurrency mechanisms in Half–Sync / Half–Async or Leader / Followers implementations need to be shut down when theapplication receives a termination request. Additionally there might be the needto dynamically adapt the degree of concurrency within the reactive system tovarying load, which means both increasing and decreasing the number of Unitsof Execution, the latter being a similar case to application termination.

Some programming languages and operating systems assist with the solutionof the problem more than others (see e.g. Sections 10.3 and 10.4)—so particu-larly it is a challenge to portably destroy Active Object.

5 Forces

• Resource leaks are evil at runtime, because they accumulate. They arestill ugly at termination, consider dangling lockfiles, files with the id ofthe (now gone) process, or still open connections to other systems, forexample.

• The higher the level of abstraction, the better the portability.

• While blocking in a system call a Unit of Execution can not even terminate.

• Especially Units of Execution blocking in certain system calls are goodcandidates to cancel.

2

Table 1: Class–Responsibility–Collaboration Cards

ActivationListHands service re-quests from

ActiveObject

over to PoolOfUnitsOf-Execution

Throws ActivationList-Disabled

(a) Activation List

ActivationListDisabledThrown by ActivationList

(b) Activation List Disabled

ActiveObjectDelegates servicerequests from

RemoteClients

to PoolOfUnitsOf-Execution

by means of ActivationList

(c) Active Object

LocalClientCreates ActiveObjectDestroys ActiveObject

(d) Local Client

PoolOfUnitsOfExecutionTakes service re-quests from

ActivationList

and executesthem.Reacts upon ActivationList-

Disabled

(e) Pool of Units of Execution

RemoteClientsRequest servicesfrom

ActiveObject

React upon ActivationList-Disabled

(f) Remote Clients

6 Solution

Add a special exception class to your code. Identify those calls the concurrentUnits of Execution may infinitely block in. Change the implementation of thesecalls to throw an instance of the exception class if and only if a certain flagis set. On termination set this flag and unblock the Units of Execution, bothwithin the same Thread Safe Interface [SSRB02h] member function.

A first sketch of the solution is shown in Table 1. Figure 1 sketches theactivity of Active Objects and their destruction. The design of the life cyclephases prior to destruction is not discussed in this pattern, so there are noactivities related to these phases in the diagram. Section 11 refers to patternsthat are concerned with such activities.

6.1 Participants

ActivationList Queues service requests to ActiveObject until there is a Unitof Execution from PoolOfUnitsOfExecution ready to finally execute thecall. ActivationList implements the Monitor Object pattern, becauseit is shared among the Units of Execution of LocalClient and PoolO-fUnitsOfExecution, and it uses synchronization depending on its State

3

6 SOLUTION

Figure 1: Activity diagram illustrating Active Object with Deferred Cancella-tion. For the sake of readability the concurrency inside was limited to one Unitof Execution.

[GHJV96c]. ActivationList implements a scheduling policy. Note thatthis participant does need to be explicit in case of Active Object andsome Half–Sync / Half–Async server designs, but can be left implicit incase of Leader / Followers designs. ActivationLists can be disabled andindicate this by means of an ActivationListDisabled exception.

ActivationListDisabled Exception indicating not to expect any services fromActivationList.

ActiveObject Decouples a service request from its actual execution. So servicerequests are handled asynchronously. ActiveObjects delegate work to aPoolOfUnitsOfExecution it owns.1 It indirectly hands work over to thepool by means of the intermediate ActivationList.

LocalClient The LocalClient owns ActiveObject. First it creates an ActiveOb-ject, later it destroys it.

PoolOfUnitsOfExecution Each Unit takes member function requests out ofActivationList to execute them on behalf of the ActiveObject. The Unitshandle ActivationListDisabled exceptions. A Unit is called Servant in theoriginal Active Object pattern.

1For the sake of this paper ActiveObjects include implementations of the Half–Sync /Half–Async and the Leader / Followers pattern, because regarding to cancellation all threeare very similar to each other.

4

6.2 Dynamics

Figure 2: Class diagram illustrating Deferred Cancellation

RemoteClients RemoteClients request services from ActiveObject.

Figure 2 sketches the participants and their relations to each other.

6.2 Dynamics

At startup LocalClient creates ActiveObject, which in turn creates Activation-List and PoolOfUnitsOfExecution. The details were described in [SSRB02a, p425] and [Bac05, pp 24–25].

This mechanism can be adapted to the case that by means of some metricLocalClient gets forced to increase the size of PoolOfUnitsOfExecution.

During the lifetime of ActiveObject RemoteClients send service requests toActiveObject. ActiveObject reifies the requests as Commands [GHJV96a] andhands the Commands over to ActivationList, its Command Processor [BMR+00a].This also was described in detail before. Here the patterns Active Objects, Half–Sync / Half–Async, and Leader / Followers differ from each other.

The termination of ActiveObject starts with a person or a parent processsending a termination signal to LocalClient. LocalClient in turn disables Ac-tivationList. ActivationList sets a flag and wakes up all Units from PoolOf-UnitsOfExecution blocking in a member function of ActivationList to receivenew work. Before returning from the member function each Unit checks the flagset by the LocalClient Unit of Execution before. Because it is set, the mem-ber function is left now by means of throwing ActivationListDisabled. EachUnit is given the chance to release resources and then terminates. LocalClientthen joins the Units of Execution, i.e. it waits for all of them to terminate.Now LocalClient releases its own resources including resources shared amongPoolOfUnitsOfExecution and terminates.

The latter mechanism can be adapted to the case that by means of somemetric LocalClient gets forced to reduce the size of PoolOfUnitsOfExecution.

The dynamics of Deferred Cancellation is shown in Figure 3.

6.3 Rationale

This pattern is about controlled cancellation of Units of Execution by anotherUnit of Execution.

5

7 RESULTING CONTEXT

Figure 3: Sequence diagram illustrating Deferred Cancellation

The solution proposed provides for well defined cancellation points: Themember functions of ActivationList throw a special exception if and only if itis disabled.

Each Unit from PoolOfUnitsOfExecution is not simply being forced to im-mediately die. It is given the chance to release resources.

The design proposed consists of joinable Units of Execution, not of detachedones. So LocalClient can wait until the last Unit from PoolOfUnitsOfExecutionhas terminated, before resources shared among the Units and itself are goingto be released. If you go for detached Units of Execution instead, combineThread Safe Interface with one of the following smart pointer patterns: Countedor Detached Counted Body idiom [Cop00, pp 173–179], [Lak96, pp 429–434],[Str98, pp 841–845] or Shared Ownership [Car96] to protect ActivationList andother shared resources from premature release (see Section 8.2 for an example).

7 Resulting Context

Blocking calls in PoolOfUnitsOfExecution were identified. There is a mechanismin place to transmit cancellation requests from LocalClient to the members ofPoolOfUnitsOfExecution. All Units of Execution implement means to reactupon cancellation requests by releasing resources and finally quitting.

7.1 Pros and Cons

The Asynchronous Cancellation pattern has the following benefits:

1. Well–defined cancellation points. The control flows of the working Unitsof Execution get interrupted at clearly defined calls.

2. Opportunity granted to release resources. After receiving a cancellationrequest the workers can release resources acquired before, be it memory,be it open files, be it any other resource.

3. Portability. This pattern works on all platforms that allow for handcraft-ing the Monitor Object pattern, the core of ActivationList.

6

4. Cooperation. As pointed out by Anthony Williams, one of the authorsof the current Boost.Thread implementation, using exceptions to forcetermination gives Units from PoolOfUnitsOfExecution great flexibility toreact upon such requests [Wil08].

The Asynchronous Cancellation pattern has the following liability:

1. Cooperation. Benefit 4 can also turn into a liability. Each Unit fromPoolOfUnitsOfExecution can react upon ActivationListDisabled by ignor-ing it. As LocalClient only terminates after all Units have terminated, amalicious Unit has the power to veto against termination.

Additionally to these general pros and cons we identified the following im-plementation specific ones.

The implementation technique of the Asynchronous Cancellation patternshown has the following liability:

2. There might be too few cancellation points. In case of some Half–Sync /Half–Async or all Leader / Followers server designs the blocking systemcalls are select() (a Reactor) or accept(), which you have less con-trol over than regarding to the condition variables aggregated by MonitorObjects, the core of ActivationList. A workaround is as follows: Thecontrolling Unit of Execution sends a network packet to the port the ap-plication listens to. This unblocks the Leader. Then both the Leader andthe Followers may be treated as described above.

8 Implementation

Depending on the platforms the application is required to work on an implemen-tation might rely on features of the operating system to shut down (see Section10). If this is not possible, you need a way to intercept blocking calls. Thefollowing two Sections conform to the description of the pattern in that it stepsinto blocking calls not at Wrapper Facades [SSRB02j] of condition variables, thelowest possible layer of abstraction, but at the higher–level Monitor Object.

The code is presented as self–contained as possible. Therefore we had todecide on the nature of the Units of Execution: Here we consider threads. Toallow for a quick identification of the core of this pattern the most importantentities have been underlined, and a comparison between code artifacts and theparticipants is given in Section 8.3.

8.1 Example Resolved

The code in Listing 1 shows a very simple Active Object without sophisticatedscheduling: Every service request gets executed as soon as there is a free Unitof Execution available. Only the cancellation aspect is shown in detail. Otheraspects are discussed in [SSRB02a, p 425].

7

8 IMPLEMENTATION

Listing 1: Cancellability added to scheduler from [SSRB02a, p 425]

1 extern "C" {

2 void *svc_run (void *);

3 }

4

5 struct Method_Request {

6 virtual ~Method_Request ();

7 virtual void call() =0;

8 };

9

10 struct Message_Queue_Disabled {};

11

12 class Message_Queue {

13 ...

14 std :: size_t max_messages_;

15 mutable Thread_Mutex monitor_lock_;

16 Thread_Condition not_empty_ ;

17 Thread_Condition not_full_ ;

18 volatile bool isActive_ ;

19 public :

20 enum { MAX_MESSAGES = ... };

21 explicit Message_Queue(std :: size_t max_messages =MAX_MESSAGES)

22 : max_messages_(max_messages),

23 not_empty_ (monitor_lock_),

24 not_full_ (monitor_lock_),

25 isActive_ (true ),... {

26 ...

27 }

28 Message_Queue(const Message_Queue &rhs)

29 : not_empty_ (monitor_lock_), // Condition variables can’t

30 not_full_ (monitor_lock_), // be copied

31 isActive_ (rhs.isActive ),... {

32 ...

33 }

34 ...

35 bool empty_i () const;

36 bool emptyAndEnabled_i () const {

37 return empty_i () && isActive_ ;

38 }

39 bool full_i () const ;

40 bool fullAndEnabled_i() const {

41 return full_i () && isActive_ ;

42 }

43 void disable () {

44 Thread_Mutex_Guard guard(monitor_lock_);

45 isActive_ =false;

46 not_empty_ .notify_all ();

47 not_full_ .notify_all ();

48 }

49 // Transfers ownership

50 Method_Request *get () {

51 Method_Request *result =0;

52 {

53 Thread_Mutex_Guard

54 guard(std :: move( not_empty_ .wait_if (

55 boost ::bind(

56 &Message_Queue:: emptyAndEnabled_i ,

8

8.1 Example Resolved

57 this ,

58 _1

59 )

60 )));

61 if(! isActive_ )

62 throw Message_Queue_Disabled ;

63 const bool wasFull (full_i ());

64 result =get_i ();

65 if(wasFull )

66 not_full_ .notify_all ();

67 }

68 return result ;

69 }

70 // Transfers ownership

71 void put (Method_Request *msg) {

72 Thread_Mutex_Guard

73 guard(std :: move(not_full_ .wait_if (

74 boost ::bind(

75 &Message_Queue:: fullAndEnabled_i ,

76 this ,

77 _1

78 )

79 )));

80 if(! isActive_ )

81 throw Message_Queue_Disabled ;

82 const bool wasEmpty (empty_i ());

83 put_i(msg );

84 if(wasEmpty )

85 not_empty_ .notify_all ();

86 }

87 };

88

89 class MQ_Scheduler {

90 typedef std ::vector < thread_type > pool_type ;

91 Message_Queue act_queue_ ;

92 pool_type pool_;

93 void joinPool_ () {

94 for (pool_type :: reverse_iterator in(pool_.rbegin ());

95 pool_ .rend ()!= in;

96 ++in)

97 joinThread (*in);

98 }

99 public :

100 MQ_Scheduler(std :: size_t high_water_mark ,

101 std :: size_t number_of_threads)

102 : act_queue_ (high_water_mark) {

103 pool_.reserve (number_of_threads );

104 try {

105 for(std :: size_t i(0); number_of_threads >i;++i)

106 pool_ .push_back (createThread(svc_run ,& act_queue_ ));

107 }

108 catch (...) {

109 act_queue_ .disable ();

110 joinPool_ ();

111 throw;

112 }

113 }

9

8 IMPLEMENTATION

114 ~MQ_Scheduler() {

115 act_queue_ .disable ();

116 joinPool_ ();

117 }

118 // Transfers ownership

119 void insert (Method_Request * method_request) {

120 act_queue_ .put (method_request);

121 }

122 };

123

124 void *svc_run (void *arg) {

125 // Set all signals blocked in the current thread ’s signal mask

126 ...

127 assert (arg );

128 Message_Queue *act_queue

129 =static_cast < Message_Queue * >(arg );

130 while (true)

131 try {

132 // Block until the queue is not empty

133 std ::auto_ptr < Method_Request > mr(act_queue ->get ());

134 mr ->call ();

135 }

136 catch (const Message_Queue_Disabled &) {

137 break;

138 }

139 catch (...) {

140 }

141 return 0;

142 }

Note some details of this implementation:

• Message_Queue::disable() uses Scoped Locking [SSRB02g] to acquireand unconditionally release the lock again, an application of ResourceAcquisition is Initialization [Str98, pp 388–393], [Str94, pp 495–497].

• Thread_Condition and Thread_Mutex are Wrapper Facades that turnoperating system specific imperative interfaces into object oriented oneswithout impacting performance. Wrapper Facades still give the compilerthe opportunity to inline the respective member functions.

• Message_Queue::{get(),put()}make use of condition variables equippedwith move semantics proposed for future revisions of the C++ standard[HSK06] to become able to return Scoped Locking guards by value fromThread_Condition::wait_if<>(), a Convenience Method [Hir97] similarto what Boost.Thread offers. Move semantics can be implemented todaywith help of the Change of Authority idiom [Bac05].

• MQ_Scheduler::joinPool_() is incomplete in that it does not call pool_.clear(). It is an implementation helper only and is therefore declaredprivate.

• The templated constructor calls std::vector<>::reserve() to preventstd::vector<>::push_back() to throw an exception.

10

8.1 Example Resolved

• thread_type, createThread(), and joinThread() wrap the respectiveoperating system specific types and functions. createThread() translateserror codes into exceptions.

From a design point of view thread_type is a Future type, and the roleof the Rendezvous function is taken by joinThread() [SSRB02a, pp 413,417,423–430,435–436]. MQ_Scheduler::pool_ is an instance of a Futurecontainer, MQ_Scheduler::joinPool_() being its Rendezvous function.

• The main loop of the threads in svc_run() is in fact a simple exampleof the Leader / Followers design pattern: The threads line up to becomea Leader, i.e. they are Followers. Only one thread at any one time cantake new work out of the queue, i.e. it takes the role of the Leader. Onreturning from Message_Queue::get() Message_Queue::monitor_lock_

is being released, thus implicitly a new Leader gets designated. The formerLeaders can process their work packages concurrently then.

Listing 2 shows how the parts proposed above work together. No emphasiswas put on where the service requests actually originate from, i.e. on an usefulinterface to RemoteClient.

Listing 2: How to use MQ_Scheduler

1 namespace {

2 std :: jmp_buf env_;

3 } /* (anonymous) namespace */

4

5 extern "C" {

6 static void SIGTERMDisposition_(int sig) {

7 longjmp (env_ ,1);

8 }

9 }

10

11 int main() {

12 signal (SIGTERM ,SIGTERMDisposition_ );

13 MQ_Scheduler sched;

14 std :: auto_ptr < Method_Request > mr;

15 if(! setjmp (env_)) {

16 while(true) {

17 // Very rough sketch ...

18 mr.reset(new Concrete_Method_Request );

19 sched.insert (mr.release ());

20 }

21 }

22 return EXIT_SUCCESS;

23 } // On automatic destruction of "sched " first its workers

24 // will terminate after having caught a "Message_Queue_Disabled "

25 // exception. Then they are going to be joined before

26 // " Message_Queue_Disabled :: act_queue_" is destructed.

27 // After all , "MQ_Scheduler" vanishes.

In an even simpler setting the main thread can simply block in pause() until asignal comes in as described e.g. in [Ste98] instead of the above signal handlerand its interaction via longjmp() and setjmp() that have to be used with

11

8 IMPLEMENTATION

Figure 4: Class diagram matching Listings 1 and 2.

caution to avoid resource leaks. mr was defined outside the setjmp() block toavoid it not to be destructed in case of termination.

Figure 4 sketches the participants and their relations to each other. Thedynamics of the entities in this example is shown in Figure 5. Note thatThread_Condition automatically deactivates the associated Thread_Mutex onblocking and activates it again before returning in Thread_Condition::wait().

8.2 Dynamic Adaptation to Varying Load

This section extends the example with a mechanism to dynamically shut downsome Units of Execution to adapt their number to decreasing load. Reasons forthe need for this dynamic adaptation are e.g.

• Threads consume resources even if they block, so they should be limitedin number to the expected load.

• Many reactive systems are I/O bound, so the performance can benefitfrom more threads than there are processor cores.

• It is hard to estimate the load up–front. Furthermore, the load likelyvaries.

From a more abstract point of view Half–Sync / Half–Async server designs turnasynchrony into synchrony, which on the one hand yields a programming modeleasier to deal with than event–driven I/O strategies, but on the other handresults in the need for dynamic load adaptation.

Several aspects need to be refined in order to do so: We want to cancelsome, but not all threads of PoolOfUnitsOfExecution. As all of these threadsexecute the same code, it is reasonable to assume that the code does not needto cancel a particular thread—all we need is a way to cancel a certain number

12

8.2 Dynamic Adaptation to Varying Load

Figure 5: Sequence diagram matching Listings 1 and 2. For the sake of read-ability the concurrency inside was limited to one thread.

13

8 IMPLEMENTATION

of threads from the pool regardless of their identity2. So the cancellability ofMessage_Queue::get() got changed in a way that the number of threads tosend the exception to can be specified.

Furthermore, we can now cancel getting work from Message_Queue inde-pendently from putting new work to the queue—only the first operation needsto be cancelled during dynamic adaptation to decreasing load, because the Re-moteClients must not be affected by this internal operation.

The code in Listing 3 shows the modified Message_Queue.

Listing 3: Message queue with refined cancellability

1 class Message_Queue {

2 ...

3 volatile std :: size_t consumers2Cancel_;

4 volatile bool isProducerActive_;

5 public :

6 explicit Message_Queue(std :: size_t max_messages =MAX_MESSAGES)

7 : ...,

8 consumers2Cancel_(0),

9 isProducerActive_(true ),... {

10 ...

11 }

12 ...

13 bool emptyAndEnabled_i () const {

14 return empty_i () && 0== consumers2Cancel_;

15 }

16 bool fullAndEnabled_i() const {

17 return full_i () && isProducerActive_;

18 }

19 void disableGet (std :: size_t consumers2Cancel) {

20 Thread_Mutex_Guard guard(monitor_lock_);

21 consumers2Cancel_=consumers2Cancel;

22 not_empty_ .notify_all ();

23 }

24 void disablePut () {

25 Thread_Mutex_Guard guard(monitor_lock_);

26 isProducerActive_=false ;

27 not_full_ .notify_all ();

28 }

29 // Transfers ownership

30 Method_Request *get () {

31 Method_Request *result =0;

32 {

33 Thread_Mutex_Guard

34 guard(std :: move( not_empty_ .wait_if (

35 boost ::bind(

36 &Message_Queue:: emptyAndEnabled_i ,

37 this ,

38 _1

39 )

40 )));

41 if(0!= consumers2Cancel_ --)

2It might be wise to refine this strategy to improve CPU cache affinity. This is beyondthe scope of this example. See the original Leader / Followers paper for details on the LIFOfollower promotion protocol.

14

8.2 Dynamic Adaptation to Varying Load

42 throw Message_Queue_Disabled ;

43 const bool wasFull (full_i ());

44 result =get_i ();

45 if(wasFull )

46 not_full_ .notify_all ();

47 }

48 return result ;

49 }

50 // Transfers ownership

51 void put (Method_Request *msg) {

52 Thread_Mutex_Guard

53 guard(std :: move(not_full_ .wait_if (

54 boost ::bind(

55 &Message_Queue:: fullAndEnabled_i ,

56 this ,

57 _1

58 )

59 )));

60 if(! isProducerActive_)

61 throw Message_Queue_Disabled ;

62 const bool wasEmpty (empty_i ());

63 put_i(msg );

64 if(wasEmpty )

65 not_empty_ .notify_all ();

66 }

67 };

Simply cancelling some of the threads from the pool regardless of their iden-tity also means that it becomes difficult to both join the threads that have goneand to maintain a list of ids of still active threads later to join. Therefore weuse detached threads here and emulate joinThread() by our own mechanism.The heart of this is a threadsafe incarnation of the Detached Counted Bodyidiom [Cop00, pp 176–179], [Lak96, pp 429–434], [Str98, pp 841–845] or SharedOwnership [Car96] (see Page 6), shown in Listing 4.

Listing 4: Smart pointer that both protects shared data and works as a Future

1 class Message_QueueHandle {

2 class Count {

3 // No copy allowed , therefore private and declared only

4 Count(const Count &);

5 // No assignment allowed , therefore private and declared only

6 Count &operator =( const Count &);

7 public :

8 std :: size_t count_ ;

9 mutable Thread_Mutex lock_;

10 Thread_Condition unique_ ;

11 Count () : count_ (1), unique_ (lock_) {}

12 };

13 Message_Queue *rep_;

14 Count *count_ ;

15 public :

16 explicit Message_QueueHandle( Message_Queue *rep)

17 : rep_(rep), count_ (new Count) {}

18 Message_QueueHandle(const Message_QueueHandle &rhs)

19 : rep_(rhs.rep_), count_ (rhs.count_ ) {

20 atomicIncrement();

15

8 IMPLEMENTATION

21 }

22 ~ Message_QueueHandle () {

23 if (0== atomicDecrement ()) {

24 delete count_ ;

25 delete rep_;

26 }

27 }

28 Message_QueueHandle &operator =( const Message_QueueHandle &);

29 Message_Queue *operator ->() const {

30 return rep_;

31 }

32 std :: size_t atomicIncrement () {

33 Thread_Mutex_Guard guard(count_ ->lock_ );

34 return ++ count_ ->count_ ;

35 }

36 std :: size_t atomicDecrement () {

37 Thread_Mutex_Guard guard(count_ ->lock_ );

38 if(1==-- count_ ->count_ )

39 count_ ->unique_ .notify ();

40 return count_ ->count_ ;

41 }

42 void waitUnlessUnique() const {

43 Thread_Mutex_Guard guard(count_ ->lock_ );

44 while (1!= count_ ->count_ )

45 count_ ->unique_ .wait ();

46 }

47 };

Instances of this class protect an instance of Message_Queue from prematuredestruction. As shown below, Message_QueueHandle::waitUnlessUnique()

makes instances of this class Futures.The scheduler only needs slight adaptations, whereas the main thread loop

now periodically tests whether to cancel or start threads—the respective metricis only sketched in the following code. Listing 5 shows the remaining codefraction.

Listing 5: Scheduler and thread main loop

1 class MQ_Scheduler {

2 Message_QueueHandle act_queue_ ;

3 void joinPool_ () {

4 act_queue_ . waitUnlessUnique ();

5 }

6 public :

7 MQ_Scheduler(std :: size_t high_water_mark ,

8 std :: size_t number_of_threads)

9 : act_queue_ (new Message_Queue(high_water_mark )) {

10 std :: size_t i(0);

11 try {

12 for (; number_of_threads >i;++ i)

13 {

14 act_queue_ .atomicIncrement ();

15 createDetachedThread (svc_run ,& act_queue_ );

16 }

17 }

18 catch (...) {

19 // Creation of (i+1)th thread failed to happen .

16

8.2 Dynamic Adaptation to Varying Load

20 act_queue_ .atomicDecrement ();

21 act_queue_ .disableGet (i);

22 joinPool_ ();

23 throw;

24 }

25 }

26 ~MQ_Scheduler() {

27 act_queue_ .disablePut ();

28 act_queue_ .disableGet (std :: numeric_limits < std :: size_t >

29 ::max ());

30 joinPool_ ();

31 }

32 // Transfers ownership

33 void insert (Method_Request *method_request) {

34 act_queue_ .put( method_request);

35 }

36 };

37

38 void *svc_run (void *arg) {

39 // Set all signals blocked in the current thread ’s signal mask

40 ...

41 assert (arg );

42 // Copy the smart pointer

43 Message_QueueHandle act_queue (

44 *static_cast < Message_QueueHandle * >(arg )

45 );

46 act_queue .atomicDecrement ();

47 while (true)

48 try {

49 if(tooManySpareThreads )

50 act_queue ->disableGet ( superfluousSpares);

51 // Block until the queue is not empty

52 std :: auto_ptr < Method_Request > mr(act_queue ->get ());

53 if(tooFewSpareThreads)

54 // Increase number of Units of Execution

55 // (beyond the scope of this paper )

56 mr ->call ();

57 }

58 catch(const Message_Queue_Disabled &) {

59 break;

60 }

61 catch (...) {

62 }

63 return 0;

64 } // On automatic destruction of "act_queue" its reference

65 // counter will be reliably decremented. After all worker

66 // threads terminated , the main thread holds the only

67 // reference to the Activation List and therefore

68 // knows that all other threads were gone.

Additionally to the comments on the code in the section before note somedetails of this implementation:

• As the instance of Message_QueueHandle must be passed by pointer tosvc_run() it is important to temporarily manipulate its reference counter.

• The main thread waits until it is the only one that holds a reference to

17

10 KNOWN USES

the contents of Message_QueueHandle. To implement waiting, a conditionvariable is used within Message_QueueHandle.

The usage of the modified scheduler does not differ from the first example.

8.3 Relationship of Examples and Participants

The code examples shown above map to the participants defined in Section 6.1as follows.

Code Participant

Message Queue ActivationListMessage Queue Disabled ActivationListDisabledMQ Scheduler ActiveObjectmain() LocalClientSection 8.1:

MQ Scheduler::pool ; Sec-tion 8.2: Implicit.

PoolOfUnitsOfExecution

main() RemoteClients

9 Variants

The level of abstraction chosen for the interception of blocking calls can vary.So a variant is to equip condition variables with an operation to disable theiroperation and to let all their potentially blocking operations throw if and only ifthey have been disabled. If portability is an issue, then choosing a high a levelas suggested can make life easier.

Another variant is to make cancellability a property of a Unit of Executioninstead of a class with blocking operations.

10 Known Uses

Examples of Static Strategy can be found in existing software.

10.1 ACE

The ADAPTIVE Communication Environment (ACE) provides ACE_Message_Queue<>, a synchronized queue, that can be disabled similar to the one proposedabove. Its blocking member functions do not throw if the queue has beendisabled. Instead they return the special value ESHUTDOWN then.

18

10.2 Boost.Thread

10.2 Boost.Thread

Deferred cancellation was recently added to the Boost.Thread [Kem02, Wil]C++ library. Cancellation is referred to as “interruption”. Different from thepattern description above it’s not ActivationList that can be disabled, but it isthe thread itself. This is similar to Java (see Section 10.4). Interruption takesplace by means of an exception thrown from well–defined interruption points,e.g. boost::condition_variable. The user can define additional interruptionpoints, if necessary. The set of pre–defined interruption points is still limited toentities within Boost.Thread—there are many more blocking calls, however, e.g.in std::fstream or during network communications, because cancellability isa crosscutting concern [KLM+97].

Internally, Thread–Specific Storage [SSRB02i] is used to manage the flag thatindicates interruption of a thread for it to be accessible from the interruptionpoints.

10.3 POSIX Threads

POSIX 1003.1c compliant systems provide pthread_cancel() that cancels athread specified as an argument. Threads can be configured dynamically toreact differently upon cancellation requests: They can igore them, they can becancellable at well–defined cancellation points only (“deferred cancellation”),or they can immediately exit (“asynchronous cancellation”). In both of thelast two cases during the cancellation procedure so–called cleanup handlers aregoing to be executed, that have been registered by the user before. The defaultbehavior is cancellation enabled at well–defined cancellation points only. Theset of POSIX Threads cancellation points is extensible. The list of cancellationpoints defined in UNIX98 lists accept() as a cancellation point, POSIX 1003.1cdoes not. For details see e.g. [But97].

Some C++ compilers automatically register destructors as cleanup handlers.

10.4 Java

Similar to Boost.Thread (see Section 10.2) instances of the class java.lang.

Thread can be set interrupted. In this case some blocking operations are leftwith an instance of java.lang.InterruptedException thrown, some with aninstance of java.nio.channels.ClosedByInterruptException thrown, whilecalls to java.nio.channels.Selectors are simply waked up. For details seee.g. [Lea99, pp 169–177,294].

10.5 MS Windows

MS Windows is listed here as a counterexample. TerminateThread() simplykills a thread immediately without giving it a chance to clean up.

Implementing Active Objects and thread pools with a mechanism that auto-matically adapts the size to varying load on MS Windows requires code similar

19

REFERENCES

to that of Section 8.1 that does not rely on safe cancellation provided by theoperating system.

11 Related Idioms and Patterns

The context of Deferred Cancellation is formed by a pool of Units of Execution,e.g. Worker Threads. This is the basis of Active Objects, Half–Sync / Half–Async or Leader / Followers designs.

Deferred Cancellation shows how to safely clean up resources specific tomembers of PoolOfUnitsOfExecution and shared among many Units. Theseresources must have been initialized before. Kircher / Jain have a compre-hensive collection of initialization patterns [KJ04, pp 19–79].

The mechanism to force cancellation of the members of the pool steps in atWrapper Facades or higher level abstractions like Monitor Objects.

If static configuration of the platforms is required or there is the needto choose among different higher–level architectures, e.g. between the designsLeader / Followers, One Child per Client [Ste98, pp 732–736] and One Threadper Client [Ste98, pp 752–753], then static and metaprogramming patterns—especially Static Adapter and Static Abstract Type Factory—can be appliedadditionally to build such Static Frameworks [Bac06].

There are many more patterns and idioms that assist in the developmentof concurrent applications. Schmidt et al. [SSRB02e] and Mattson et al.[MSM04] are comprehensive collections or pattern languages in this area.

Acknowledgements

Without the invaluable feedback of Berna L. Massingill, who was the PLoPshepherd of this work, this paper would not have been the way it is now.

Last but not least my thanks and love go to Cornelia Kneser, my wife, forher constant support throughout the writing of the paper.

This work was supported by the Institute for Medical Informatics and Bio-statistics, Basel, Switzerland.

References

[Bac05] Bachmann, Philipp: Change of authority and thread safe inter-face goes synchronized. In Proceedings of the 12th Pattern Lan-guages of Programs (PLoP) conference 2005, Allerton Park, Mon-ticello, IL, USA, December 2005. http://hillside.net/plop/

2005/proceedings/PLoP2005_pbachmann1_0.pdf, visited on Jan-uary 9, 2007.

[Bac06] Bachmann, Philipp: Static and metaprogramming patterns to facil-itate portable code. In Proceedings of the 13th Pattern Languages of

20

References

Programs (PLoP) conference 2006, Portland, OR, USA, Septem-ber 2006. http://hillside.net/plop/2006/Papers/Library/

portableProgrammingPL.pdf, visited on May 2, 2008.

[BMR+00a] Buschmann, Frank, Regine Meunier, Hans Rohnert, Peter Som-merlad und Michael Stal: Command–Processor, Kapitel 3: Ent-wurfsmuster, Seiten 277–291. In: [BMR+00b], 1998, 1. korr. Nach-druck Auflage, 2000, ISBN 3-8273-1282-5. German translation of“Command Processor”.

[BMR+00b] Buschmann, Frank, Regine Meunier, Hans Rohnert, Peter Som-merlad und Michael Stal: Pattern–orientierte Softwarearchitek-tur. Ein Pattern–System, deutsche Ubersetzung von Christiane

Lockenhoff. Addison–Wesley. An Imprint of Pearson Educa-tion, Munchen · Boston · San Francisco · Harlow, England · DonMills, Ontario · Sydney · Mexico City · Madrid · Amsterdam, 1998,1. korr. Nachdruck Auflage, 2000, ISBN 3-8273-1282-5. Germantranslation of “Pattern–Oriented Software Architecture. A Systemof Patterns”.

[But97] Butenhof, David R.: Programming with POSIX Threads. Addison–Wesley Professional Computing Series; ed. by Brian W.

Kernighan. Addison–Wesley Longman Publishing Co., Inc.,Boston, Massachusetts, 1997, ISBN 0-201-63392-2.

[Car96] Cargill, Tom: Localized ownership: Managing dynamic objects inC++. In Vlissides, John M. et al. [VCK96], chapter 1, pages 5–18,ISBN 0-201-60734-4 / 0-201-89527-7.

[Cop00] Coplien, James: C++ idioms patterns. In Harrison, Neil et al.[HFR00], chapter 10, pages 167–197, ISBN 0-201-43304-4.

[CS95] Coplien, James O. and Douglas C. Schmidt (editors): Pattern Lan-guages of Program Design, volume 1 of The Software Patterns Se-ries; ed. by John Vlissides. Addison–Wesley, Boston · San Fran-cisco · New York · Toronto · Montreal · London · Munich · Paris ·

Madrid · Capetown · Sydney · Tokyo · Singapore · Mexico City,1995, ISBN 0-201-60734-4.

[GHJV96a] Gamma, Erich, Richard Helm, Ralph Johnson und John Vlissi-des: Befehl, Kapitel 5: Verhaltensmuster, Seiten 273–286. In: Pro-fessionelle Softwareentwicklung [GHJV96b], dritter, unveranderterNachdruck Auflage, 1996. German translation of “Command”.

[GHJV96b] Gamma, Erich, Richard Helm, Ralph Johnson und John Vlissides:Entwurfsmuster. Elemente wiederverwendbarer objektorientierterSoftware, deutsche Ubersetzung von Dirk Riehle. Professionel-le Softwareentwicklung. Addison–Wesley–Longman, Bonn · Rea-ding, Massachusetts · Menlo Park, California · New York · Harlow,

21

REFERENCES

England · Don Mills, Ontario · Sydney · Mexico City · Madrid ·

Amsterdam, dritter, unveranderter Nachdruck Auflage, 1996. Ger-man translation of “Design Patterns. Elements of Reusable Object–Oriented Software”.

[GHJV96c] Gamma, Erich, Richard Helm, Ralph Johnson und John Vlissides:Zustand, Kapitel 5: Verhaltensmuster, Seiten 398–409. In: Pro-fessionelle Softwareentwicklung [GHJV96b], dritter, unveranderterNachdruck Auflage, 1996. German translation of “State”.

[HFR00] Harrison, Neil, Brian Foote, and Hans Rohnert (editors): PatternLanguages of Program Design, volume 4 of The Software PatternsSeries; ed. by John M. Vlissides. Addison–Wesley. An imprint ofAddison Wesley Longman, Inc., Reading, Massachusetts · Harlow,England · Menlo Park, California · Berkeley, California · Don Mills,Ontario · Sydney · Bonn · Amsterdam · Tokyo · Mexico City, 2000,ISBN 0-201-43304-4.

[Hir97] Hirschfeld, Robert: Convenience method. In Preliminary Confer-ence Proceedings of EuroPLoP ’96 [wuc97].

[HSK06] Hinnant, Howard E., Bjarne Stroustrup, and Bronek Koz-icki: A brief introduction to rvalue references. Technical Re-port WG21/N2027 = J16/06-0097, ISO IEC JTC1 / SC22 /WG21—The C++ Standards Committee, Herb Sutter · Mi-crosoft Corp. · 1 Microsoft Way · Redmond WA USA 98052-6399,June 2006. http://www.open-std.org/jtc1/sc22/wg21/docs/

papers/2006/n2027.html, visited on April 25, 2008.

[Kem02] Kempf, Bill: The Boost.Threads library. The C/C++ Users Jour-nal. Advanced Solutions for C/C++ Programmers, May 2002.http://www.ddj.com/cpp/184401518, visited on May 2, 2008.

[KJ04] Kircher, Michael and Prashant Jain: Pattern–Oriented SoftwareArchitecture. Patterns for Resource Management. Wiley Series inSoftware Design Patterns. John Wiley & Sons, Ltd, Chichester,2004, ISBN 0-470-84525-2.

[KLM+97] Kiczales, Gregor, John Lamping, Anurag Mendhekar, ChrisMaeda, Cristina Videira Lopes, Jean-Marc Loingtier, and John Ir-win: Aspect–oriented programming. In Proceedings of the EuropeanConference on Object–Oriented Programming (ECOOP), Finland,number 1241 in Lecture Notes in Computer Science. Springer, June1997.

[Lak96] Lakos, John: Large–Scale C++ Software Design. Addison–WesleyProfessional Computing Series; ed. by Brian W. Kernighan.Addison–Wesley, Boston · San Francisco · New York · Toronto ·

Montreal · London · Munich · Paris · Madrid · Capetown · Sydney ·

22

References

Tokyo · Singapore · Mexico City, 11th printing may 2002 edition,1996, ISBN 0-201-63362-0.

[Lea99] Lea, Doug: Concurrent Programming in Java. Design Principlesand Patterns. The Java Series; Lisa Friendly, Series Editor,Tim Lindholm, Technical Editor. Addison–Wesley. An imprintof Addison Wesley Longman, Inc., Reading, Massachusetts · Har-low, England · Menlo Park, California · Berkeley, California · DonMills, Ontario · Sydney · Bonn · Amsterdam · Tokyo · Mexico City,second edition, November 1999, ISBN 0-201-31009-0.

[Mes96] Meszaros, Gerard: A pattern language for improving the capacity ofreactive systems. In Vlissides, John M. et al. [VCK96], chapter 35,pages 575–591, ISBN 0-201-60734-4 / 0-201-89527-7.

[MSM04] Mattson, Timothy G., Beverly A. Sanders, and Berna L.Massingill: Patterns for Parallel Programming. The Software Pat-terns Series; ed. by John Vlissides. Addison–Wesley, Boston ·

San Francisco · New York · Toronto · Montreal · London · Mu-nich · Paris · Madrid · Capetown · Sydney · Tokyo · Singapore ·

Mexico City, September 2004, ISBN 0-321-22811-1.

[SC96] Schmidt, Douglas C. and Charles D. Cranor: Half–sync / half–async. an architectural pattern for efficient and well–structuredconcurrent I/O. In Vlissides, John M. et al. [VCK96], chapter 27,pages 437–459, ISBN 0-201-60734-4 / 0-201-89527-7.

[Sch95] Schmidt, Douglas C.: Reactor: An object behavioral pattern forconcurrent event demultiplexing and event handler dispatching. InCoplien, James O. and Douglas C. Schmidt [CS95], chapter 29,pages 529–545, ISBN 0-201-60734-4.

[SP] Schmidt, Douglas C. and Irfan Pyarali: The design and use of theACE reactor. an object–oriented framework for event demultiplex-ing. http://www.cs.wustl.edu/~schmidt/PDF/reactor-rules.pdf, visited on January 9, 2007.

[SSRB02a] Schmidt, Douglas, Michael Stal, Hans Rohnert und Frank Busch-mann: Active Object, Kapitel 5: Nebenlaufigkeit, Seiten 411–443.In: [SSRB02e], 2002, ISBN 3-89864-142-2. German translation of“Active Object”.

[SSRB02b] Schmidt, Douglas, Michael Stal, Hans Rohnert und Frank Busch-mann: Half–Sync / Half–Async, Kapitel 5: Nebenlaufigkeit, Seiten473–497. In: [SSRB02e], 2002, ISBN 3-89864-142-2. German trans-lation of “Half–Sync / Half–Async”.

23

REFERENCES

[SSRB02c] Schmidt, Douglas, Michael Stal, Hans Rohnert und Frank Busch-mann: Leader / Followers, Kapitel 5: Nebenlaufigkeit, Seiten 499–528. In: [SSRB02e], 2002, ISBN 3-89864-142-2. German translationof “Leader / Followers”.

[SSRB02d] Schmidt, Douglas, Michael Stal, Hans Rohnert und Frank Busch-mann: Monitor Object, Kapitel 5: Nebenlaufigkeit, Seiten 445–471.In: [SSRB02e], 2002, ISBN 3-89864-142-2. German translation of“Monitor Object”.

[SSRB02e] Schmidt, Douglas, Michael Stal, Hans Rohnert und Frank Busch-mann: Pattern–orientierte Software-Architektur. Muster fur ne-benlaufige und vernetzte Objekte, ubersetzt aus dem Amerikani-schen von Martina Buschmann. dpunkt.verlag, Heidelberg,2002, ISBN 3-89864-142-2. German translation of “Pattern–Oriented Software Architecture. Volume 2: Patterns for Concur-rent and Networked Objects”.

[SSRB02f] Schmidt, Douglas, Michael Stal, Hans Rohnert und Frank Busch-mann: Reactor, Kapitel 3: Ereignisverarbeitung, Seiten 197–236.In: [SSRB02e], 2002, ISBN 3-89864-142-2. German translation of“Reactor”.

[SSRB02g] Schmidt, Douglas, Michael Stal, Hans Rohnert und Frank Busch-mann: Scoped Locking, Kapitel 4: Synchronisation, Seiten 359–367.In: [SSRB02e], 2002, ISBN 3-89864-142-2. German translation of“Scoped Locking”.

[SSRB02h] Schmidt, Douglas, Michael Stal, Hans Rohnert und Frank Busch-mann: Thread–Safe Interface, Kapitel 4: Synchronisation, Seiten383–391. In: [SSRB02e], 2002, ISBN 3-89864-142-2. German trans-lation of “Thread Safe Interface”.

[SSRB02i] Schmidt, Douglas, Michael Stal, Hans Rohnert und Frank Busch-mann: Thread–Specific Storage, Kapitel 5: Nebenlaufigkeit, Seiten529–561. In: [SSRB02e], 2002, ISBN 3-89864-142-2. German trans-lation of “Thread–Specific Storage”.

[SSRB02j] Schmidt, Douglas, Michael Stal, Hans Rohnert und Frank Busch-mann: Wrapper–Facade, Kapitel 2: Dienstzugriff und Konfigurati-on, Seiten 53–84. In: [SSRB02e], 2002, ISBN 3-89864-142-2. Ger-man translation of “Wrapper Facade”.

[Ste98] Stevens, W. Richard: UNIX Network Programming, volume 1. Net-working APIs: Sockets and XTI. Prentice Hall PTR, Upper SaddleRiver, NJ, second edition, 1998, ISBN 0-13-490012-X.

[Str94] Stroustrup, Bjarne: Design und Entwicklung von C++. Addison–Wesley, Bonn · Paris · Reading, Massachusetts · Menlo Park, Ca-lifornia · New York · Don Mills, Ontario · Workingham, England ·

24

References

Amsterdam · Milan · Sydney · Tokyo Singapore · Madrid · SanJuan · Seoul · Mexico City · Taipei, Taiwan, 1994, ISBN 3-89319-755-9. German translation of “The Design and Evolution of C++”.

[Str98] Stroustrup, Bjarne: Die C++–Programmiersprache. DeutscheUbersetzung von Nicolai Josuttis und Achim Lorke. Addison–Wesley–Longman, Bonn · Reading, Massachusetts · Menlo Park,California · New York · Harlow, England · Don Mills, Ontario ·

Sydney · Mexico City · Madrid · Amsterdam, dritte, aktualisierteund erweiterte Auflage, 1998, ISBN 3-8273-1296-5. German trans-lation of “The C++ Programming Language, Third Edition”.

[VCK96] Vlissides, John M., James O. Coplien, and Norman L. Kerth (edi-tors): Pattern Languages of Program Design, volume 2. Addison–Wesley. An imprint of Addison Wesley Longman, Inc., Reading,Massachusetts · Harlow, England · Menlo Park, California · Berke-ley, California · Don Mills, Ontario · Sydney · Bonn · Amsterdam ·

Tokyo · Mexico City, 1996, ISBN 0-201-60734-4 / 0-201-89527-7.

[Wil] Williams, Anthony: [Boost.]Thread. published online. http://

www.boost.org/doc/libs/1_35_0/doc/html/thread.html, vis-ited on May 2, 2008.

[Wil08] Williams, Anthony: Thread interruption in the Boost Thread li-brary, March 2008. http://www.justsoftwaresolutions.co.uk/threading/thread-interruption-in-boost-thread-library.

html, visited on April 27, 2008.

[wuc97] Preliminary conference proceedings of EuroPLoP ’96. TechnicalReport wucs-97-07, Washington University, Department of Com-puter Science, 1997.

25