comp superscalar: bringing grid superscalar and gcm together enric tejedor universitat politècnica...
TRANSCRIPT
COMP Superscalar:Bringing GRID superscalar
and GCM together
Enric TejedorUniversitat Politècnica de
V ProActive and GCM User Group, GridCOMP conferenceOctober 21, 2008. INRIA, Sophia Antipolis, France
Rosa M. BadiaBarcelona Supercomputing
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
Outline
• Introduction
• Grid Component Model
• GRID superscalar
• Programming model
• Runtime operation
• Initialisation
• Task processing
• Finalisation and error handling
• Conclusions
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
1. Introduction
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
Basic concepts
•The Grid Component Model (GCM)
• CoreGRID NoE, GridCOMP
• Distributed, based on Fractal, intended for the Grid
•GRID superscalar
• Framework to ease the development of Grid-unaware applications
• Simple programming model: Grid as transparent as possible
• Runtime that optimises the performance of the application (exploiting
possible concurrency)
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
A componentised Grid framework
• GRID superscalar is suitable to benefit from the GCM features
• Componentisation process: identify the inner functionalities of the runtime,
and assign each one to a separate component
• Result: COMP Superscalar, whose runtime gains in:
• Reusability
• Flexibility
• Deployability
• Separation of concerns
• Ease of development
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
Componentised runtime
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
2. Programming model
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
...for (i=0; i<N; i++){ T1 (data1, data2); T2 (data4, data5); T3 (data2, data5, data6); T4 (data7, data8); T5 (data6, data8, data9);}...
Sequential Application
T10 T20
T30
T40
T50
T11 T21
T31
T41
T51
T12
…
Resource 1
Resource 2
Resource 3
Resource N
.
.
.
Task graph creation
based on data
precedence
Task selection +
parameters direction
(input, output, inout)
Scheduling,
data transfer,
task execution
Synchronization,
results transfer
Parallel Resources(multicore,SMP, cluster, grid)
Programming model
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
initialize(f1);
for (int i = 0; i < 2; i++) {
genRandom(f2);
add(f1, f2);
}
print(f2);
Java application
COMPSs programming model
public interface SumItf {
@ClassName(“example.Sum")
@MethodConstraints(OSType = "Linux")
void genRandom(
@ParamMetadata(type = Type.FILE, direction = Direction.OUT)
String f
);
@ClassName(“example.Sum")
...
}
Task constraints
Parametermetadata
Implementation
Java interface
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
Custom Java Class Loader
Java app code
COMPSs runtime
Annotated interface
Javassist
insertscalls to
Custom Loader
uses
input
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
3. Runtime operation
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
Implementation details
•Base technologies:
• Java as programming language
• ProActive: implementation of the GCM model, used to build the components
• JavaGAT: used for job submission and file transfer
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
Initialization
•Multicast invocation
• Forwarded to all subcomponents
TA TS JM
FM
FIP FTM
init()
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
Initialization
•Multicast invocation
• Reduction of return values
TA TS JM
FM
FIP FTMReduction
Reduction
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
Task processing (1)
•Application submits task
• Task Analyser receives the request
TA TS JM
FM
FIP FTM
executeTask(…)
T1genRd
T3genRd
T2add
T4add
f1 f1
f2
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
Task processing (2)
TA JM
FM
FIP FTM
TS
•File Information Provider is contacted
• File accesses of the task are registered
executeTask(…)
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
Task processing (3)
•Task Analyser discovers dependencies
• Dependency-free tasks are sent for scheduling
TA TS JM
FM
FIP FTM
executeTask(…)
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
Task processing (4)
•Task Scheduler decides where to execute tasks
• Job Manager is informed of this decision
TA TS JM
FM
FIP FTM
executeTask(…) Scheduling
algorithm
Resourcecapabilitie
s
Taskconstraint
s
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
Task processing (5)
• Job Manager checks the necessary file transfers
• File Transfer Manager actually performs them
TA TS JM
FM
FIP FTM
executeTask(…)
GAT
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
Task processing (6)
•FTM informs of the end of transfers for a task
• Task Grid job, submitted for execution
TA TS JM
FM
FIP FTM
executeTask(…)
GAT
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
Task processing (7)
•Callback for the end of a job is received
• The notification is forwarded to the Task Analyser
TA TS JM
FM
FIP FTM
executeTask(…)
GAT
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
Safe stopping process for the runtime
•COMPSs subcomponents have data dependencies
•Modification of the Life Cycle Controller
• Components stopped in a safe order: TS, TA, JM, FTM, FIP
• Otherwise, a deadlock could happen
TATA TS JM
FM
FIP FTM
LCC
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
4. Conclusions
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
Conclusions
•COMP Superscalar: componentised version of GRID superscalar
• GCM-based
• Hierarchical runtime: separation of concerns
• Task scheduling, file and resource management
•Straightforward programming model for Grid-unaware Java
applications
•Feedback about GCM-ProActive
• Component distribution
• Performance: no noticeable overhead for a medium number of tasks
• Component composition and dynamic reconfiguration: Grid IDE
V ProActive and GCM User Group. October 21st 2008, Sophia Antipolis (France)
Thank you!
Questions?