advance reservations of grid resources for gemss applications
DESCRIPTION
Advance Reservations of Grid Resources for GEMSS Applications. Junwei Cao Falk Zimmermann Guy Lonsdale C&C Research Laboratories NEC Europe Ltd., Sankt Augustin, Germany http://www.ccrl-nece.de/. Outline. GEMSS Overview GEMSS Applications GEMSS Architecture COSY Resource Management - PowerPoint PPT PresentationTRANSCRIPT
PSNC POZNAŃ 2003
Advance Reservations of Grid Resources for GEMSS Applications
Junwei CaoFalk Zimmermann
Guy Lonsdale
C&C Research LaboratoriesNEC Europe Ltd., Sankt Augustin, Germany
http://www.ccrl-nece.de/
PSNC POZNAŃ 2003
Outline
GEMSS Overview GEMSS Applications GEMSS Architecture COSY Resource Management COSY Queue Scheduling COSY Advance Reservations Future Work
PSNC POZNAŃ 2003
GEMSS Project GEMSS: Grid-Enabled Medical Simulation
Services EC/IST FP5 Grid Project (1.9.2002-28.2.2005) Grid middleware initiative within medical
application setting. http://www.gemss.de/. GEMSS Consortium: 10 partners from industry
& academia including University clinics:NEC Europe Ltd, MPI Leipzig, ISS Vienna, CFX Ltd, CRID FUNDP, IT Innovation, USFD, IDAC Ireland Ltd., ASD, IBMTP Vienna.
PSNC POZNAŃ 2003
GEMSS Objectives Demonstrate that the grid can improve pre-operative planning
& near- real-time surgical support by providing access to advanced simulation and image-processing services.
Build middleware on existing or developing grid technology standards to provide support for authorization, workflow, security and Quality of Service aspects.
Develop, evaluate and validate a test-bed for the GEMSS system, including its deployment in the end-user’s working environment.
Anticipate privacy, security and other legal concerns by examining and incorporating into its grid services the latest laws and EU regulations related to providing medical services over the Internet.
PSNC POZNAŃ 2003
GEMSS ApplicationsName Domain Class UsersMaxillo-facial surgery simulation
Medicine – pre-surgical planning
Distributed supercomputing / On demand
Medical doctors, researchers
Neurosurgery support Medicine – intra-operative planning
On demand Medical doctors, researchers
Radiotherapy planning
Medicine – Monte Carlo treatment simulation
On demand / distributed supercomputing
Medical end-users;
Doctors, researchers
Inhaled drud delivery planning
Medicine – air flow dynamics
On demand / distributed supercomputing
Medical end-users;
Doctors, researchers
Cardio-vascular system simulation
Medicine – blood flow dynamics
On demand Medical end-users;
Doctors, researchers
Advanced image reconstruction
Medicine – nuclear / in vivo diagnostics
On demand Medical end-users;
Doctors, researchers
PSNC POZNAŃ 2003
Maxillo-facial Surgery Simulation
Pre- and post surgeryCourtesy Dr. Dr. Th. Hierl, University Clinic Leipzig.
Modeling the distraction procedure
Provide a virtual try-out space for the pre-operative planning of maxillo-facial surgery.
Based on image processing, meshing & HPC Finite Element simulation.
Grid scenario: client-server, doctor uses a client machine to access remote HPC services.
PSNC POZNAŃ 2003
An Example Implementation
PSNC POZNAŃ 2003
GEMSS Architecture Service-oriented architecture Client-server topology Based on Web Services technologies Client-side pluggable component
framework Protocol-independent client-side APIs for
service invocation Support for complex process models:
Business process workflows QoS negotiation workflows Application workflows
PSNC POZNAŃ 2003
Client-side ArchitectureGEMSS client application
instantiate invokeGEMSS client application
instantiate invoke
API
ReqDescServiceDesc
Proxyinterface
Datatransfer
workflow enactor client pluggablecomponentinfrastructure
service discovery
security encoding
selection of service
accounting attrib.
QoS requirements
security credentials
login / logouttransportinterface
InternetLocal Registry Local Authorisation
PSNC POZNAŃ 2003
Server-side Architecture
Service provider firewallService provider firewall
Web server
WS security, WS policy and secure transfer
Services
Accountservice
QoSnegotiation
Applicationservices
…
Conversationauthorisation
Cert. /key
store
LoggerResourcemanager
Errorrecovery
Service staterepository
Intrusiondetection ...
GEMSS hosting environment HPC hardware
PSNC POZNAŃ 2003
Resource Management Requirements
From application services: Start a batch job Kill a job Job status queryFrom QoS negotiation services: Book a reservation Confirm a reservation Release a reservation Reservation status query
PSNC POZNAŃ 2003
COSY Structure
Operating Systems, MPI Environments (NEC, SCore …), …
Users / Applications
Resources
Generic User Interfaces (submit, release, queue, confirm,…)
JobScheduling
Log Management
Resource Monitoring Database Management
Configurations
Job Execution
Plugin Scripts
Specific User Interfaces (mpirun …)
PSNC POZNAŃ 2003
COSY Job Scheduling Queue Scheduling:
First-come-first-served algorithm Aggressive backfilling Multiple queue support Queue and user priority Resource limitation Access control list Batch and interactive support
Advance Reservations: Exact start time Latest end time Two phase commitment
PSNC POZNAŃ 2003
When Scheduling with ARs …
Mean queue wait time increases and resource utilization decreases. Schedulable only with limit percentage of ARs. How to prevent ARs from taking start time advantages over queued jobs? How to satisfy ARs as far as possible without sacrificing queue efficiency?
PSNC POZNAŃ 2003
COSY Policies for ARs …
A mandatory shortest notice time for ARs is defined Using current mean wait time of queued jobs Using times of mean wait time of queued jobs
PSNC POZNAŃ 2003
Experimental Results
p
0
0. 5
1
1. 5
2
2. 5
0 1 2 3 4 5 6
n
q
5% 10% 15% 20% p
0
20
40
60
80
0 1 2 3 4 5 6
n
u(%)
5% 10% 15% 20%
Queue wait time Resource utilization
PSNC POZNAŃ 2003
Summary COSY takes the current mean wait time of
queued jobs as the mandatory AR shortest notice time so that ARs cannot take start time advantages.
The mandatory AR shortest notice time is applied in COSY as 1 time of current mean wait time of queued jobs and increases to 4 linearly as the AR percentage increases from 0% (not inclusive) to 15% so that ARs can be satisfied as far as possible without increasing the queue wait time.
When the AR percentage is more than 15%, COSY will stop accepting ARs temporarily in order to guarantee a proper queue scheduling.
Advantage: adaptive to queue workload. Disadvantage: additional calculation of statistics.
PSNC POZNAŃ 2003
Future Work
Implementation of GEMSS grid architecture (9.2003 – 11.2003 – 2.2004 – 8.2004)
Integration of GEMSS applications with GEMSS grid infrastructure (11.2003 – 8.2004)
Extension of COSY with standard languages and protocols (e.g. GRAAP)
Cost estimation of ARs over queue jobs Co-reservation and co-allocation