lecture 6-2 : concurrent queues and stacks companion slides for the art of multiprocessor...

Lecture 6-2 :Concurrent Queues and

Stacks

Companion slides forThe Art of Multiprocessor

Programmingby Maurice Herlihy & Nir Shavit

pool

• Data Structure similar to Set– Does not necessarily provide contains()

method– Allows the same item to appear more than

once– get() and set()

Art of Multiprocessor Programming© Herlihy-Shavit 2007

2

public interface Pool<T> {

void put(T item);

T get();

}


3

Queues & Stacks

• Both: pool of items• Queue

– enq() & deq()– First-in-first-out (FIFO) order

• Stack– push() & pop()– Last-in-first-out (LIFO) order


4

Bounded vs Unbounded

• Bounded– Fixed capacity– Good when resources an issue

• Unbounded– Holds any number of objects


5

Blocking vs Non-Blocking

• Problem cases:– Removing from empty pool– Adding to full (bounded) pool

• Blocking– Caller waits until state changes

• Non-Blocking– Method throws exception


6

Queue: Concurrency

enq(x) y=deq()

enq() and deq() work at

different ends of the object

tail head


7

Concurrency

enq(x)

Challenge: what if the queue is empty or full?

y=deq()

tailhead


8

A Bounded Lock-Based Queue

public class BoundedQueue<T> {

ReentrantLock enqLock, deqLock;

Condition notEmptyCondition, notFullCondition;

AtomicInteger size;

Node head;

Node tail;

int capacity;

public BoundedQueue(int capacity) {

this.capacity = capacity;

this.head = new Node(null);

this.tail = head;

this.size = new AtomicInteger(0);

this.enqLock = new ReentrantLock();

this.notFullCondition = enqLock.newCondition();

this.deqLock = new ReentrantLock();

this.notEmptyCondition = deqLock.newCondition();

}

protected class Node {

public T value;

public Node next;

public Node(T x) {

value = x;

next = null;

}

}

9

public void enq(T x) {

boolean mustWakeDequeuers = false;

enqLock.lock();

try {

while (size.get() == capacity)

notFullCondition.await();

Node e = new Node(x);

tail.next = e;

tail = e;

if (size.getAndIncrement() == 0) {

mustWakeDequeuers = true;

}

} finally {

enqLock.unlock();

}

if (mustWakeDequeuers) {

deqLock.lock();

try {

notEmptyCondition.signalAll();

} finally {

deqLock.unlock();

}

}

}

public T deq() {

T result;

boolean mustWakeEnqueuers = false;

deqLock.lock();

try {

while (size.get() == 0) {

notEmptyCondition.await();

result = head.next.value;

head = head.next;

if (size.getAndDecrement() == capacity) {

mustWakeEnqueuers = true;

}

} finally {

deqLock.unlock();

}

if (mustWakeEnqueuers) {

enqLock.lock();

try {

notFullCondition.signalAll();

} finally {

enqLock.unlock();

}

}

return result;

}

lock

• enqLock/deqLock– At most one enqueuer/dequeuer at a time can

manipulate the queue’s fields

• Two locks– Enqueuer does not lock out dequeuer – vice versa

• Association– enqLock associated with notFullCondition– deqLock associated with notEmptyCondition


10

enqueue1. Acquires enqLock2. Reads the size field3. If full, enqueuer must wait until dequeuer makes room4. enqueuer waits on notFullCondition field, releasing

enqLock temporarily, and blocking until that condition is signaled.

5. Each time the thread awakens, it checks whether there is a room, and if not, goes back to sleep

6. Insert new item into tail7. Release enqLock8. If queue was empty, notify/signal waiting dequeuers


11

dequeue1. Acquires deqLock2. Reads the size field3. If empty, dequeuer must wait until item is enqueued4. dequeuer waits on notEmptyCondition field, releasing

deqLock temporarily, and blocking until that condition is signaled.

5. Each time the thread awakens, it checks whether item was enqueued, and if not, goes back to sleep

6. Assigne the value of head’s next node to “result” and reset head to head’s next node

7. Release deqLock8. If queue was full, notify/signal waiting enqueuers9. Return “result”


12

Art of Multiprocessor Programming 1313

Bounded Queue

Sentinel

head

tail


Bounded Queue

head

tail

First actual item


Bounded Queue

head

tail

Lock out other deq()

calls

deqLock


Bounded Queue

head

tail

Lock out other enq()

calls

deqLock

enqLock


Not Done Yet

head

tail

deqLock

enqLock

Need to tell whether queue is full or

empty


Not Done Yet

head

tail

deqLock

enqLock

Max size is 8 items

size

1


Not Done Yet

head

tail

deqLock

enqLock

Incremented by enq()Decremented by deq()

size

1


Enqueuer

head

tail

deqLock

enqLock

size

1

Lock enqLock


Enqueuer

head

tail

deqLock

enqLock

size

1

Read size

OK


Enqueuer

head

tail

deqLock

enqLock

size

1

No need to lock tail


Enqueuer

head

tail

deqLock

enqLock

size

1

Enqueue Node


Enqueuer

head

tail

deqLock

enqLock

size

12

getAndincrement()


Enqueuer

head

tail

deqLock

enqLock

size

8 Release lock2


Enqueuer

head

tail

deqLock

enqLock

size

2

If queue was empty, notify

waiting dequeuers


Unsuccesful Enqueuer

head

tail

deqLock

enqLock

size

8

Uh-oh

Read size

…


Dequeuer

head

tail

deqLock

enqLock

size

2

Lock deqLock


Dequeuer

head

tail

deqLock

enqLock

size

2

Read sentinel’s next field

OK


Dequeuer

head

tail

deqLock

enqLock

size

2

Read value


Dequeuer

head

tail

deqLock

enqLock

size

2

Make first Node new sentinel


Dequeuer

head

tail

deqLock

enqLock

size

1

Decrement size


Dequeuer

head

tail

deqLock

enqLock

size

1Release deqLock


34

Monitor Locks

• Java ReentrantLocks are monitors• Allow blocking on a condition

rather than spinning• Threads:

– acquire and release lock– wait on a condition


35

public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock;}

The Java Lock Interface

Create condition to wait on


36

Lock Conditions

public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); }


37


Lock Conditions

Release lock and wait on condition


38


Lock Conditions

Wake up one waiting thread


39


Lock Conditions

Wake up all waiting threads


40

Await

• Releases lock associated with q• Sleeps (gives up processor)• Awakens (resumes running)• Reacquires lock & returns

q.await()


41

Signal

• Awakens one waiting thread– Which will reacquire lock

q.signal();


42

Signal All

• Awakens all waiting threads– Which will each reacquire lock

q.signalAll();

Unbounded Lock-Free Queue (Nonblocking)

• Unbounded– No need to count the number of items

• Lock-free– Use AtomicReference<V>

• An object reference that may be updated atomically.

– boolean compareAndSet(V expect, V update)• Atomically sets the value to the given updated value if

the current value == the expected value.

• Nonblocking– No need to provide conditions on which to wait

43


A Lock-Free Queue

Sentinel

head

tail


Enqueue

head

tail

Enq( )


Enqueue

head

tail


Logical Enqueue

head

tail

CAS


Physical Enqueue

head

tailCAS


Enqueue

• These two steps are not atomic• The tail field refers to either

– Actual last Node (good)– Penultimate Node (not so good)

• Be prepared!


Enqueue

• What do you do if you find– A trailing tail?

• Stop and help fix it– If tail node has non-null next field– CAS the queue’s tail field to tail.next

• As in the universal construction


When CASs Fail

• During logical enqueue– Abandon hope, restart– Still lock-free (why?)

• During physical enqueue– Ignore it (why?)


Dequeuer

head

tail

Read value


Dequeuer

head

tail

Make first Node new sentinel

CAS


54

Unbounded Lock-Free Queue(Nonblocking)

public class LockFreeQueue<T> {

private AtomicReference<Node> head;

private AtomicReference<Node> tail;

public LockFreeQueue() {

this.head = new AtomicReference<Node>(null);

this.tail = new AtomicReference<Node>(null);

}

public class Node {

public T value;

public AtomicReference<Node> next;

public Node(T value) {

this.value = value;

this.next = new AtomicReference<Node>(null);

}

}

55

public void enq(T item) {

Node node = new Node(item);

while (true) {

Node last = tail.get();

Node next = last.next.get();

if (last == tail.get()) {

if (next == null) {

if (last.next.compareAndSet(next, node)) {

tail.compareAndSet(last, node);

return;

}

} else {

tail.compareAndSet(last, next);

}

}

}

}

public T deq() throws EmptyException {

while (true) {

Node first = head.get();

Node last = tail.get();

Node next = first.next.get();

if (first == head.get()) {

if (first == last) {

if (next = null) {

throw new EmptyException();

}

tail.compareAndSet(last, next);

} else {

T value = next.value;

if (head.compareAndSet(first,next))

return value;

}

}

}

}

Unbounded Lock-Free Queue(Nonblocking)


Concurrent Stack

• Methods– push(x) – pop()

• Last-in, First-out (LIFO) order• Lock-Free!


Empty Stack

Top


Push

Top


Push

TopCAS


Push

Top


Push

TopCAS


Push

Top


Pop

Top


Pop

TopCAS


Pop

TopCAS

mine!


Pop

TopCAS


Pop

Top


public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff(); }}

Lock-free Stack


public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public Boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) }public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff()}}

Lock-free Stack

tryPush attempts to push a node


public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff()}}

Lock-free Stack

Read top value



Lock-free Stack

current top will be new node’s successor



Lock-free Stack

Try to swing top, return success or failure



Lock-free Stack

Push calls tryPush



Lock-free Stack

Create new node



Lock-free Stack

If tryPush() fails,back off before retrying

78

protected boolean tryPush(Node node)

{

Node oldTop = top.get();

node.next = oldTop;

return (top.compareAndSet(oldTop, node));

}

public void push( T value )

{

Node node = new Node( value );

while (true) {

if (tryPush(node)) { return; }

else { backoff.backoff( ); }

}

}

protected Node tryPop( ) throws EmptyException

{

Node oldTop = top.get();

if ( oldTop == null ) {

throw new EmptyException( );

}

Node newTop = oldTop.next;

if ( top.compareAndSet( oldTop, newTop ) ) {

return oldTop;

} else { return null; }

}

public T pop() throws EmptyException {

while (true) {

Node returnNode = tryPop( );

if ( returnNode != null ) {

return returnNode.value;

} else { backoff.backoff( ); }

}

}

Unbounded Lock-Free Stack


Lock-free Stack

• Good– No locking

• Bad– Without GC, fear ABA– Without backoff, huge contention at

top – In any case, no parallelism


Question

• Are stacks inherently sequential?• Reasons why

– Every pop() call fights for top item

• Reasons why not– Think about it!

lecture 6-2 : concurrent queues and stacks companion slides for the art of multiprocessor...

Documents