Java ThreadsJava Threads

IntroductionIntroduction

Process 1Process 1

Processes and ThreadsProcesses and Threads

Environment

Stack

CodeHeap

CPUState

Process 2Process 2

Environment

Stack

CodeHeap

CPUState

Process 1Process 1

EnvironmentCodeHeap

Stack

CPUState

Thread 2Thread 2

Stack

CPUState

Thread 1Thread 1

Stack

CPUState

Thread 3Thread 3

Processes and ThreadsProcesses and Threads

TerminologyTerminology

• A thread is a single sequence of execution within a program

• Multiprogramming:

- Running multiple processes concurrently

• Multithreading:

- Running multiple threads within one process concurrently

- That is, having several code executions within a single process

• Concurrency is achieved either by using several

processors or by time-slicing over one processor

Time Slicing vs. ParallelismTime Slicing vs. Parallelism

CPU CPU1 CPU2

Multiple Threads in an ApplicationMultiple Threads in an Application

• Each thread has its own run-time stack and CPU state

(i.e., register content, next instruction, etc. )

• If two threads execute the same method, each will have

its own copy of the local variables the methods uses

• However, all threads see the same dynamic memory,

i.e., heap

- which variables are stored on the heap?

• Two different threads can act on the same object and

same static fields concurrently

Why Threads?Why Threads?

• Improve the responsiveness of applications

- i.e., users wait less time

• Improve utilization of resources

- e.g., one thread can run while the other waits for I/O

• Provide a convenient programming technique

- run resource cleaners in the background

- graphical applications

- monitor resource status

Java ThreadsJava Threads

Class ThreadClass Thread

• We use the class Thread in order to create and

run threads

• Two ways to create Thread objects:

- Implementing Runnable and wrapping with Thread

- Extending Thread

• Thread itself also implements Runnable, so in

both ways you implement the method run()

Implementing RunnableImplementing Runnable

public class MinusPrinter implements Runnable { public void run() { for(int i=0; i<1000; ++i) { System.out.print("-"); } }}

Runnable r = new MinusPrinter();Thread t1 = new Thread(r);

Extending ThreadExtending Thread

public class PlusPrinter extends Thread { public void run() { for(int i=0; i<1000; ++i) { System.out.print("+"); } }}

Thread t2 = new PlusPrinter();

Running ThreadsRunning Threads

• A Thread's execution starts by invoking its method

start()

• This method invokes its method run() in a new thread

public static void main(String argv[]) { Runnable r = new MinusPrinter(); Thread t1 = new Thread(r); Thread t2 = new PlusPrinter(); t1.start(); t2.start();}

The OutputThe Output

Java Thread SchedulingJava Thread Scheduling

Thread SchedulingThread Scheduling

• Usually, threads run one at a time

- Unless several processors are being used

• Thread execution is merged in order to provide

concurrency

Scheduling ThreadsScheduling Threads

I/O operation completes

start()

Currently executedthread

Ready queue

•Waiting for I/O operation to be completed•Waiting to be notified•Sleeping•Waiting to enter a synchronized section

Newly createdthreads

Blocked queue

Alive

Thread State DiagramThread State Diagram

New Thread Dead Thread

Running

Runnable

new PlusPrinter();

run() method returns

for (…) { … }

BlockedObject.wait()Thread.sleep()blocking IO callwaiting on a monitor

thread.start();

Thread SchedulingThread Scheduling

• Thread scheduling is the mechanism used to determine

how Runnable threads (in the ready queue) are allocated

CPU time

• Java's scheduling is based on priorities

• Each thread has a priority - an integer number

- Use thread.getPriority()/setPriority() to control priorities

• In principle, the runtime system chooses the thread that

has the highest priority

Thread Scheduling (cont)Thread Scheduling (cont)

• If several threads have the same priority, an arbitrary

one is chosen

• Scheduling may violate the priority-based policy to

avoid starvation of low-priority threads

• Java's scheduling also uses time slicing, when it is

supported by the operating system

- mainly used for sharing CPU among equal highest-priority

threads

Thread Scheduling (cont)Thread Scheduling (cont)

A thread runs until one of the following occurs:

• The thread dies (e.g., run() completes)

• The thread becomes not Runnable (e.g., sleeps or waits

for an IO operation to complete)

• A higher-priority thread becomes Runnable

• On systems that support time-slicing, its time allotment

has expired

• The thread yields (discussed later)

Scheduling is OS Dependant!Scheduling is OS Dependant!

• Java maps Java threads to OS threads

• In particular, Java relies on the operating system for

- Time slicing

- Priorities

• Thus, scheduling differs from one system to another

• Do not count on scheduling and priorities for algorithm

correctness!

Relinquishing the CPURelinquishing the CPU

• A running thread can explicitly relinquish the

CPU for other threads to use

• The static method Thread.yield() temporarily

pauses the currently executing thread and allows

other threads to execute

• A thread can block for a while using the method

thread.sleep(milliseconds)

Changing the Old ExampleChanging the Old Example

public class MinusPrinter implements Runnable { public void run() { for(int i=0; i<1000; ++i) { System.out.print("-"); Thread.yield(); } }}

public class PlusPrinter extends Thread { public void run() { for(int i=0; i<1000; ++i) { System.out.print("+"); Thread.yield(); } }}

The OutputThe Output

Daemon ThreadsDaemon Threads

• There are two types of threads:

- daemon threads (like the garbage-collection thread)

- non-daemon threads (like the thread running main())

• JVM will let all non-daemon threads complete (i.e.,

complete the execution of run())

• When only daemon threads stay alive, they are killed

and JVM exits

• Controlled by thread.isDaemon and thread.setDeamon

NotesNotes

• Threads inherit their priority and daemon properties

from their creating threads

• The method thread.join() blocks and waits until the

thread completes running

• A thread can have a name for identification

• Stopping a running thread was possible in old versions

of Java, but it is now deprecated

- Instead, interruption mechanisms should be used

Garbage CollectionGarbage Collection

• The garbage collector of Java runs on a separate

(daemon) thread

• An object is a candidate for garbage collection if

this object can no longer be accessed by any

living thread

Synchronizing ThreadsSynchronizing Threads

Thread SynchronizationThread Synchronization

• Consider the following consumer-producer

scenario:

The Simpson SimulationThe Simpson Simulationpublic class CookieJar { int contents; boolean hasCookie = false;

public void putCookie(String who, int value) { while (hasCookie) {} contents = value; hasCookie = true; System.out.println(who + " put cookie " + value); }

public int getCookie(String who) { while (!hasCookie) {} hasCookie = false; System.out.println(who + " got cookie " + contents); return contents; }}

horribly horribly inefficient!!inefficient!!

try {Thread.sleep(1000);}catch (InterruptedException e) {}

The Simpson Simulation (cont)The Simpson Simulation (cont)

public class Homer implements Runnable { CookieJar jar; public Homer(CookieJar jar) { this.jar = jar; } public void eat() { int cookie = jar.getCookie("Homer"); } public void run() { for (int i = 1 ; i <= 10 ; i++) eat(); } }

The Simpson Simulation (cont)The Simpson Simulation (cont)

public class Marge implements Runnable { CookieJar jar; public Marge(CookieJar jar) { this.jar = jar; } public void bake(int cookie) { jar.putCookie("Marge", cookie); } public void run() { for (int i = 0 ; i < 10 ; i++) bake(i); } }

The Simpson Simulation (cont)The Simpson Simulation (cont)

public class RunSimpsons {

public static void main(String[] args) {

CookieJar jar = new CookieJar();

Homer homer = new Homer(jar); Marge marge = new Marge(jar);

new Thread(homer).start(); new Thread(marge).start(); } }

Oops! Missed a Cookie!Oops! Missed a Cookie!

public class CookieJar { int contents; boolean hasCookie = false;

public void putCookie(String who, int value) { while (hasCookie) { sleep(); } contents = value; hasCookie = true; }

public int getCookie(String who) { while (!hasCookie) { sleep();} hasCookie = false; return contents;}}

HomerHomer

MargeMarge

Race Condition ExampleRace Condition Example

Put green pieces Put red piecesHow can we havealternating colors?

Race ConditionRace Condition

• Race condition –

- Two threads are simultaneously reading or modifying

some shared data

- The outcome of a program is affected by the order in

which the program's threads are allocated CPU time

• Both threads “race” for accessing shared data

• When undesired, synchronization is required

Monitors and LocksMonitors and Locks

• Monitors are key elements in Java's thread

synchronization

• Every object has a monitor

• An object's monitor is used as a guardian that watches a

block of code (called a critical section) and enables only

one thread to enter that code

• To enter a critical section, a thread must first acquire an

ownership over the corresponding monitor

Unique Lock OwnershipsUnique Lock Ownerships

• Only one thread can own a specific monitor

• If a thread A tries to enter a block under a monitor and a

different thread B has already entered that block, A will wait

until B releases the monitor and (hopefully) that monitor

will be passed to A

- Hence, monitors are related to as locks

• When a thread leaves the critical section, the monitor is

automatically released

• Threads awaiting a monitor are blocked and queued

The The synchronizedsynchronized keyword keyword

• To monitor a block code using the monitor of

Object o, use the synchronized keyword as

follows:

synchronized(o) { critical-section }

• synchronized method() {critical-section} is a

shorthand for

method() {synchronized(this) {critical-section} } –

An ExampleAn Example public class BankAccount {

private float balance;

public synchronized void deposit(float amount) { balance += amount; } public synchronized void withdraw(float amount) { balance -= amount; } public synchronized void transfer (float amount, BankAccount target) { withdraw(amount); target.deposit(amount);}}

Critical SectionsCritical Sections

Bank Account

deposit()

t1t2t3

Synchronization ScopesSynchronization Scopes

private String a = "hello";private Date b = new Date();

void a() { synchronized(a) { System.out.println("In A 1"); System.out.println("In A 2"); }}void b() { synchronized(b) { System.out.println("In B 1"); System.out.println("In B 2"); }}

synchronized void a() { System.out.println("In A 1"); System.out.println("In A 2");}

synchronized void b() { System.out.println("In B 1"); System.out.println("In B 2");}

Synchronization ScopesSynchronization Scopes

static String c = "world";

void a() { synchronized (c) { System.out.println("In A 1"); System.out.println("In A 2"); }}void b() { synchronized (getClass()) { System.out.println("In B 1"); System.out.println("In B 2"); }}

static synchronized void a() { System.out.println("In A 1"); System.out.println("In A 2");}

static synchronized void b() { System.out.println("In B 1"); System.out.println("In B 2");}

Uses the monitor of the Class object

Synchronization ScopesSynchronization Scopes

What will happen here?

void a() { Date d = new Date(); synchronized (d) { System.out.println("In A 1"); System.out.println("In A 2"); }}

Back to the SimpsonsBack to the Simpsons

public synchronized void putCookie(String who, int value) { while (hasCookie) { sleep(); } contents = value; hasCookie = true;}

public synchronized int getCookie(String who) { while (!hasCookie) { sleep(); } hasCookie = false; return contents;}

HomerHomer

MargeMarge

deadlock!deadlock!

Another Deadlock ExampleAnother Deadlock Example public class BankAccount {

private float balance;

public synchronized void deposit(float amount) { balance += amount; } public synchronized void withdraw(float amount) { balance -= amount; } public synchronized void transfer (float amount, BankAccount target) { withdraw(amount); target.deposit(amount);}}

DeadlocksDeadlocks

deposit()

Alice's Account Bob's Account

t1 t2

deposit() ?transfer()

withdraw()

transfer()

withdraw()

wait() wait() and and notify()notify()

• Suppose that an object has some monitor, but

conditions disable it from completing the critical

section

• The wait/notify mechanism enables that object to

release the monitor and wait until conditions are

changed

wait()wait()

• The method Object.wait() requires the current

thread to own the monitor of the object

• When called, the current thread

- releases ownership on the object's monitor

- stops and waits until some other thread will wake it

up and the monitor will be re-obtained

notify()notify()

• Like wait, requires the object to own the monitor

• The method Object.notify() wakes up an arbitrary

thread that waits on the monitor of the object

• Object.notifyAll() wakes all such threads

• When a thread is waken up, it regularly waits for the

monitor to be available

• The thread calling notify should release the monitor for

the waiting thread to continue

Waiting and NotifyingWaiting and Notifying

synchronized (lock) { while (!resourceAvailable()) { lock.wait(); } cosumeResource();}

produceResource();synchronized (lock) { lock.notifyAll(); }

Wait/Notify SequenceWait/Notify Sequence

Lock Object

ConsumerThread

ProducerThread

1. synchronized(lock){

2. lock.wait();

3. produceResource()4. synchronized(lock) {5. lock.notify();6.}

7. Reacquire lock

8.Return from wait()

9. consumeResource();10. }

Wait/Notify SequenceWait/Notify Sequence

Lock Object

ConsumerThread

ProducerThread

1. synchronized(lock){

2. lock.wait();

3. produceResource()4. synchronized(lock) {5. lock.notify();6.}

7. Reacquire lock

8. Return from wait()

9. consumeResource();10. }

Wait/Notify SequenceWait/Notify Sequence

Lock Object

ConsumerThread

ProducerThread

1. synchronized(lock){

2. lock.wait();

3. produceResource()4. synchronized(lock) {5. lock.notify();6.}

7. Reacquire lock

8. Return from wait()

9. consumeResource();10. }

Wait/Notify SequenceWait/Notify Sequence

Lock Object

ConsumerThread

ProducerThread

1. synchronized(lock){

2. lock.wait();

3. produceResource()4. synchronized(lock) {5. lock.notify();6.}

7. Reacquire lock

8. Return from wait()

9. consumeResource();10. }

Wait/Notify SequenceWait/Notify Sequence

Lock Object

ConsumerThread

ProducerThread

1. synchronized(lock){

2. lock.wait();

3. produceResource()4. synchronized(lock) {5. lock.notify();6.}

7. Reacquire lock

8. Return from wait()

9. consumeResource();10. }

Wait/Notify SequenceWait/Notify Sequence

Lock Object

ConsumerThread

ProducerThread

1. synchronized(lock){

2. lock.wait();

3. produceResource()4. synchronized(lock) {5. lock.notify();6.}

7. Reacquire lock

8. Return from wait()

9. consumeResource();10. }

Wait/Notify SequenceWait/Notify Sequence

Lock Object

ConsumerThread

ProducerThread

1. synchronized(lock){

2. lock.wait();

3. produceResource()4. synchronized(lock) {5. lock.notify();6.}

7. Reacquire lock

8. Return from wait()

9. consumeResource();10. }

Wait/Notify SequenceWait/Notify Sequence

Lock Object

ConsumerThread

ProducerThread

1. synchronized(lock){

2. lock.wait();

3. produceResource()4. synchronized(lock) {5. lock.notify();6.}

7. Reacquire lock

8. Return from wait()

9. consumeResource();10. }

Wait/Notify SequenceWait/Notify Sequence

Lock Object

ConsumerThread

ProducerThread

1. synchronized(lock){

2. lock.wait();

3. produceResource()4. synchronized(lock) {5. lock.notify();6.}

7. Reacquire lock

8. Return from wait()

9. consumeResource();10. }

Wait/Notify SequenceWait/Notify Sequence

Lock Object

ConsumerThread

ProducerThread

1. synchronized(lock){

2. lock.wait();

3. produceResource()4. synchronized(lock) {5. lock.notify();6.}

7. Reacquire lock

8. Return from wait()

9. consumeResource();10. }

Wait/Notify SequenceWait/Notify Sequence

Lock Object

ConsumerThread

ProducerThread

1. synchronized(lock){

2. lock.wait();

3. produceResource()4. synchronized(lock) {5. lock.notify();6.}

7. Reacquire lock

8. Return from wait()

9. consumeResource();10. }

Fixed Simpson ExampleFixed Simpson Example

public synchronized void putCookie(String who, int value) { while (hasCookie) { try { wait(); } catch(InterruptedException e){} } contents = value; hasCookie = true; System.out.println(who + " put cookie " + value); notifyAll(); }

public synchronized int getCookie(String who) { while (!hasCookie) { try{ wait(); } catch(InterruptedException e){} } hasCookie = false; System.out.println(who + " got cookie " + contents); notifyAll(); return contents; }

HomerHomer

MargeMarge

Multithreading Client-Multithreading Client-ServerServer

Server

Client

Request Handler

Client

Request Handler …

Multithreading Client-ServerMultithreading Client-Server

• When a new request arrives, it is served in a new

thread

• The server thread continues to listen

• Next, we will show how the EchoServer example

of last week should be fixed to handle concurrent

requests

ServerServer

import java.net.*; import java.io.*;

public class EchoServer { public static void main(String[] args) throws IOException {

ServerSocket serverSocket = new ServerSocket(8000);

while (true) { try { Socket socket = serverSocket.accept(); new EchoRequestHandler(socket).start(); } catch (Exception e) { System.err.println("Error: " + e.getMessage()); }}}}

Request HandlerRequest Handler

public class EchoRequestHandler extends Thread { Socket socket = null;

public EchoRequestHandler(Socket socket) { this.socket = socket; }

public void run() {... next slide ... }

}

Request HandlerRequest Handler

public void run() { try { BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream())); PrintStream writer = new PrintStream(socket.getOutputStream()); String lineRead = null; while ((lineRead = reader.readLine()) != null) { writer.println("You wrote: " + lineRead); writer.flush(); } } catch (IOException exp) { System.err.println("Error: " + exp.getMessage());} finally { try { if (!socket.isClosed()) socket.close(); } }}

Thread PoolsThread Pools

Why Thread Pools?Why Thread Pools?

• Thread pools improve resource utilization

- The overhead of creating a new thread is

significant

• Thread pools enable applications to control

and bound their thread usage

- Creating too many threads in one JVM can

cause the system to run out of memory and

even crash

Thread Pools in ServersThread Pools in Servers

• Thread pools are especially important in client-server

applications

- The processing of each individual task is short-lived and the

number of requests is large

- Servers should not spend more time and consume more

system resources creating and destroying threads than

processing actual user requests

• When too many requests arrive, thread pools enable the

server to force clients to wait threads are available

The “Obvious” ImplementationThe “Obvious” Implementation

• There is a pool of threads

• Each task asks for a thread when starting and

returns the thread to the pool after finishing

• When there are no available threads in the pool

the thread that initiates the task waits till the pool

is not empty

• What is the problem here?

The “Obvious” Implementation is The “Obvious” Implementation is ProblematicProblematic

• When the pool is empty, the submitting thread

has to wait for a thread to be available

- We usually want to avoid blocking that thread

- A server may want to perform some actions when too

many requests arrive

• Technically, Java threads that finished running

cannot run again

Task Queue

Worker Threads

wait()

Q is Empty

All the worker threads wait for tasks

A Possible SolutionA Possible Solution

Task Queue

Worker Threads

Task

The number of worker threads is fixed. When a task is inserted to the queue, notify is called.

A Possible SolutionA Possible Solution

Task Queue

Worker Threads

notify()

The number of worker threads is fixed. When a task is inserted to the queue, notify is called.

A Possible SolutionA Possible Solution

Task Queue

Worker Threads

The task is executed by the tread

A Possible SolutionA Possible Solution

Task Queue

Worker Threads

The task is executed by the tread

Task

notify()

A Possible SolutionA Possible Solution

Task Queue

Worker Threads

The remaining tasks are executed by the

tread

A Possible SolutionA Possible Solution

Task Queue

Worker Threads

When a task ends the tread is released

While the Q is not empty, take the task from the Q and run it (if the Q was empty, wait() would have been called)

A Possible SolutionA Possible Solution

Task Queue

Worker Threads

A new task is executed by the released tread

A Possible SolutionA Possible Solution

Thread Pool ImplementationThread Pool Implementation

public class TaskManager { LinkedList taskQueue = new LinkedList(); List threads = new LinkedList(); public TaskManager(int numThreads) { for(int i=0; i<numThreads; ++i) { Thread worker = new Worker(taskQueue); threads.add(worker); worker.start(); }} public void execute(Runnable task) { synchronized(taskQueue) { taskQueue.addLast(task); taskQueue.notify(); } } }

Thread Pool ImplementationThread Pool Implementation

public class Worker extends Thread { LinkedList taskQueue = null; public Worker(LinkedList queue) { taskQueue = queue; } public void run() { Runnable task = null; while (true) { synchronized (taskQueue) { while (taskQueue.isEmpty()) { try {taskQueue.wait();} catch (InterruptedException ignored) {}} task = (Runnable) taskQueue.removeFirst(); } task.run();}}}

Risks in Using Thread PoolsRisks in Using Thread Pools

• Threads can leak

- A thread can endlessly wait for an I/O operation to complete

- For example, the client may stop the interaction with the

socket without closing it properly

- What if task.run() throws a runtime exception?

• Solutions:

- Bound I/O operations by timeouts

- Catch possible runtime exceptions

Pool SizePool Size

• Each thread consumes resources

- memory, management overhead, etc.

• Therefore, you have to Tune the thread pool size

according to the number and characterizations of

expected tasks

• There should also be a limit on the size of the task queue

- What should we do when too many requests arrive?