1

A Portable Mechanism for Thread Persistence and Migration

Wei TaoThe School of Computing

University of Utah

Doctoral Dissertation Defense

January 3, 2001

2

Outline

Research objectives Problem statement Survey of related work Our solution

– saving control state with an example

– synchronization

Prototype implementation– Performance

Discussion Research contribution Future work

3

Research objectives

To investigate a general approach for thread persistence and migration which is

– portable: applicable to any platforms and environments

– transparent: hiding most code from application users

– adaptable: general enough to be used by most frameworks

To show effectiveness and viability of this mechanism by

– prototype implementation

To evaluate this mechanism by – performance measurement

– definition of constraints

4

The Java programming language

Java supports basic forms of distribution– Architecture-independent byte code with compact format

– Dynamic loading and linking

– Safety and security models enforcing type and access rules

– Synchronized multiple threads of control providing concurrency

Thread migration requires externalizing and moving execution states

– Java provides no mechanism for migrating execution states

» Java’s RMI is based on the RPC model

» Existing Java mobile agent systems transfer agent objects without their associated control state

– Need a strategy to externalize thread’s state

5

Externalizing Java threads

A thread’s execution state includes– Data component

» Values of local variables in the activation records

» In Java, it contains thread’s control stack and related data

– Code component

» Program’s executable code

» Pointers into this code (program counter values)

» In Java, it contains the byte code and a pointer into the code

Object sharing and multithreading– Deep or shallow copy

– Synchronization of multiple threads

6

Possible solutions Checkpoint Java Virtual Machine

– Saves a memory image of JVM process in a flat file

– Has platform specific data representation

– Is inefficient and inflexible

– Works only for complete programs

Modify Java Virtual Machine implementation– Takes architecture-independent checkpoints in JVM

– Makes JVM save or restore the execution state when program shuts down or restarts

– Negates the ubiquity benefit of standard JVM

Define thread saving in application code– Insert state saving code into either source code or byte code

– Platform independence

7

State of the art

Persistence in Java– Persistent thread is a feature missing in existing persistence

systems in Java

» Object serialization

» Persistent Storage Engine (PSE)

» The JDBC interface

» Orthogonal persistence: PJAMA

Java mobile agent systems– Strong mobility is not supported in most Java mobile agent

systems

» IBM’s AGLETS

» ObjectSpace’s VOYAGER

» General Magic’s ODYSSEY

8

State of the art (continued)

Thread migration in Java– Specialized thread migration or mobility systems

» DELTA EXECUTION: integrated with JESSICA

» MOBA: designed for Grid-based thread migration

» SUMATRA: integrated in its mobile agent system

– Extended JVM

» Provides new services in JDK for saving threads

» Not portable to standard JVMs

» No support for synchronization

– Source code level transformation

» Big performance overhead

» Cannot save complete execution state

» May extend the language

9

Our approach

Byte code level transformation– Inserting byte code to save and rebuild control state by a class

file processor

Special features– Portability

» No modification to JVM or Java language

– Transparency

» Code are inserted automatically by the post-processor

– Adaptability

» Is a general strategy to be used with any persistence or migration systems

– Low overheads

– Ability to save programs with complex synchronization states

10

Design and implementation

New APIs for thread externalization– New exception classes: ShutdownException, MoveException

– New methods initiate externalization

» checkStop: throws ShutdownException if shutdown flag is set

» moveTo(host): starts to externalize by throwing MoveException

ShutdownManager– Provides public method to initiate shutdown: checkStop

– Provides methods to be inserted by the processor

– Manages the shutdown and restart process

MigrationManager– Provides public method to start migration: moveTo(host)

– Manages migration process when necessary

11

Two exceptions

public class ShutdownException extends Exception {

private Stack record;

public void push(Object item) { … }

public Object pop() { … }

public Stack getRecord() { … }

}

public class MoveException extends ShutdownException {

private String host;

public MoveException(String host) { … }

public String getHost() { … }

}

12

Saving control state Collapsing a control stack

– Wraps each statement that throws either exception with handlers

– In the handlers, saves local data and byte code label into the exception and re-throws it

– Externalized thread state is saved or transmitted at top level

Rebuilding a control stack – Check the saved state for current thread at the beginning of each

processed method

– Execution jumps to the start of shutdown code if the saved state is null (not a restart) or empty (restart process is complete)

– Otherwise the saved label is popped from the saved state record

– A switch instruction is used to dispatch to the proper code to restore local data

– Execution jumps to the instruction with the saved label after restoring local data

13

A simple example: merge sort

public void mergeSort( int[ ] A, int p, int r )

throws ShutdownException {

if (p >= r) return;

int q = (p + r)/2;

mergeSort(A, p, q);

mergeSort(A, q+1, r);

merge(A, p, q, r);

}

public void merge( int[ ] A, int p, int q, int r )

throws ShutdownException {

ShutdownManager.checkStop( );

...

}

14

Merge sort: transformed code

public void merge( int[ ] A, int p, int q, int r )

throws ShutdownException {

… // retrieve the saved record of current thread to se

if( se != null && !se.isEmpty()) {

goto r

}

try {

ShutdownManager.checkStop( );

} catch (ShutdownException se) {

se.pushInt( r ); se.pushInt( q );

se.pushInt( p ); se.pushObject( A );

throw se;

}

r : ...

}

15

Merge sort: transformed code

public void mergeSort(int[ ] A, int p, int r) throws ShutdownException {

… // retrieve the saved record of current thread to se

if( se != null && !se.isEmpty( ) ) {

int pc = se.popInt( );

switch( pc ) {

case 0: q = se.popInt( );

t1 = (int[])se.popObject( );

t2 = se.popInt( );

t3 = se.popInt( ); goto r1

case 1: q = se.popInt( );

t1 = (int[])se.popObject( );

t2 = se.popInt( );

t3 = se.popInt( ); goto r2

case 2: q = se.popInt( );

t1 = (int[])se.popObject( );

t2 = se.popInt( );

t3 = se.popInt( ); t4 = se.popInt( ); goto r3

}

16

Transformed code (continued) if (p >= r) return;

int q = (p + r)/2;

t1 = A; t2 = p; t3 = q;

r1: try {

mergeSort( t1, t2, t3 );

} catch (ShutdownException se) {

se.pushInt( q ); se.pushInt( 0 ); se.pushInt( r ); se.pushInt( p ); se.pushObject( A ); throw se;

}

t1 = A; t2 = q + 1; t3 = r;

r2: try {

mergeSort( t1, t2, t3 );

} catch (ShutdownException se) {

se.pushInt( q ); se.pushInt( 1 ); se.pushInt( r ); se.pushInt( p ); se.pushObject( A ); throw se;

}

t1 = A; t2 = p; t3 = q; t4 = r;

r3: try {

merge( t1, t2, t3, t4 );

} catch (ShutdownException se) {

se.pushInt( q ); se.pushInt( 2 ); se.pushInt( r ); se.pushInt( p ); se.pushObject( A ); throw se;

}

}

17

Problems with multi-threading

Inconsistent state– A thread releases all locks it holds after shutdown, possibly

leaving the unlocked objects in inconsistent states

– Threads should be prevented from accessing those resources with inconsistent states

Indefinite blocking– No threads should be blocked indefinitely because of other

threads’ shutdown

Missed signals– A thread should not send signals to threads that have already

shut down

Nested monitor lockouts– Threads should be restarted in a correct order

18

Synchronizing shutdown

Saving synchronization state– Locking information is collected during shutdown process

– Object wait set information is collected at execution time

Before a thread releases a lock during shutdown process, it

– sets the locked object to be dirty

» A thread starts shutdown when trying to access a dirty object

– interrupts all threads that are waiting for the lock

» All waiting threads start shutdown after receiving interruption

Notify signal is never missed due to shutdown– Wait threads will not start shutdown until the owner thread starts

shutdown and interrupts them

19

Synchronizing restart

Threads should be restarted in a specific order– The threads in wait set of a lock should restart before the thread

who owned the lock at shutdown time

» A wait thread acquires the lock and jumps to execute wait()

» It releases the lock when entering wait state, allowing other threads to acquire the lock

A central barrier object is used to block new computation

– A thread should not start new computation until all other threads have finished the restart process

– Threads in wait state when shutdown occurred would wait again

– Other threads are blocked by a central barrier controlled by shutdown manager

20

Prototype implementation

Thread persistence– All active threads in a program are saved and restarted

– A shutdown manager is created when a program starts

– Experimental frameworks

» Java serialization

» Persistent storage engine

Thread migration– A thread migrates transparently with control state

» resumes execution from exactly where it left

– Experimental frameworks

» Java sockets

» ObjectSpace’s VOYAGER

21

Using serialization

Persistent types– Classes implement java.io.Serializable or java.io.Externalizable

Object sharing– A complete representation of the entire object graph is serialized

– Objects inside the object graph maintain their sharing status

Non-serializable types– Declaring a field to be transient prevents it from being serialized

» Transient objects become null after serialization

» They may need to be newly created after deserialization

– Local variables with non-serializable types are saved as null

» Define a corresponding wrapper class for each non-serializable types that need to be recreated

22

Using persistent storage engine

PSE has the following advantages over serialization– Better performance and reliability

– Database management features such as transactions

Same APIs and byte code transformations as used in the object serialization

Two implementation differences– The captured states are saved with the objects to a persistent

store in PSE instead of a file

– The object of a class implements java.lang.Runnable is saved

» New thread objects are created using saved Runnable objects

» A look-up table is used to swizzle old references to the corresponding new thread objects

23

Using Java sockets

Class MobileThread– serves as a super class for all mobile threads

– has a local field keeping the thread’s control state

– provides method send( ) to send out the thread

Each server which may host mobile threads– maintains a server socket with specified address and port.

– creates a migration manager for each client socket

– manages a local name service to identify local resources

A migration manager– receives object streams and recreates mobile thread objects

– starts the received mobile threads

24

Integrating with VOYAGER

Voyager– Voyager is a Java distributed object system which supports

object mobility and mobile agents

– Objects are transferred, but not the control states

– Programmer must manually encode control states

» forced to adopt event style programming

» migration cannot occur deep in recursion, iteration …

Supporting control migration in Voyager agents– Use the additional APIs to initiate externalization

– Insert externalization code via post-processor

– Enable agents to migrate to a new host in any control context

25

Performance measurements

Four benchmarked applications– Bank: average time on 10 threads with 30,000 transactions

– Crew: total time on two readers and two writers

– Buyer agent: total time of 50 repetitions of the bid cycle

– Tower of Hanoi: total time of 20 disks

Experiment setup– Environment:

» Java SDK 2 on Microsoft Windows98 (PII 300MHz)

– Measurement:

» Run time: Averaging 10 executions of each program

» Space: total file size in bytes for all processed class files

26

Normalized run time

0

50

100

150

200

250

300

%

Bank Crew Buyer Hanoi

Programs

Original

Transformed

State Saving

27

Normalized class file sizes

0

50

100

150

200

250

%

Bank Crew Buyer Hanoi

Programs

Original

Without Synch

With Synch

28

Performance analysis

Run time overhead for normal execution depends on– percentage of processed methods

– percentage of instructions which may initiate shutdown

– total run time of original code

Run time overhead for state saving depends on– how deep in the call chain the shutdown instruction is

– how many local data at each level needs to be saved

Space overhead depends on– percentage of processed methods

– percentage of instructions which may initiate shutdown

– how many local data needs to be saved at each instruction

– how intense synchronization is used in the program

29

Discussion

Limitations of language level transformation– System level thread migration

– Class and interface initializers

– Finally clauses

Constraints for class files– Unsafe type

– Unstructured lock

– Other restrictions

» Requesting shutdown at prohibited spots

» Call back of native methods

30

Research contributions

Designed a language level rewriting approach to externalize running threads

Analyzed the correctness of code transformation using denotational semantics

Implemented a Java byte code processor for thread persistence and migration on different persistence and migration frameworks

Presented a mechanism to preserve synchronization when saving multiple threads

Measured and analyzed performance overheads Discussed the limitations of language level approach

31

Future work

Formal analysis– Byte code transformation

– Synchronization

Framework enhancements– Saving a group of related threads and synchronization

– Processing byte code during load time

– Enabling state saving and restoration for native methods

More applications– Fast activation of applications or applets

– Recoverable exceptions

– Distributed transactions

– Load balancing among heterogeneous platforms

32

The full dissertation is available at– http://www.cs.utah.edu/~tao/research/index.html

The prototype JTHREAD is available at– http://www.cs.utah.edu/~tao/research/Jthread.html