ischia, italy - 9-21 july 20061 session 1 second part day 1 monday 10 th july malcolm atkinson

Ischia, Italy - 9-21 July 2006 1

Session 1 Second Part

Day 1

Monday 10th July

Malcolm Atkinson


Introduction to GridsIntroduction to Grids

History of Distributed Computing

Challenges of Distributed Computing

Driving Forces & Motivation for Grids Collaboration

Data Deluge

Computational Roadmap

A history of Grids


What is e-Science?What is e-Science?

• Goal: to enable better research in all disciplines• Method: Develop collaboration supported by

advanced distributed computation– to generate, curate and analyse rich data resources

• From experiments, observations and simulations• Quality management, preservation and reliable evidence

– to develop and explore models and simulations• Computation and data at all scales• Trustworthy, economic, timely and relevant results

– to enable dynamic distributed collaboration• Facilitating collaboration with information and resource sharing• Security, trust, reliability, accountability, manageability and agility

Our challenge is to develop an integrated approach to all three


Distributed Systems HistoryDistributed Systems History

1960 1970 1980 1990 2000

Bespoke pioneering distributed systems, e.g. SABRE & SAGE

IBM CICS

Linklider proposes shared multi-site computing

Dozens of academic networks Many incompatible protocols

ARPA net

CORBA & DCOM

IP-based InternetAcademic & Research

WWW

Ethernet

TCP


Distributed Systems to GridsDistributed Systems to Grids

1960 1970 1980 1990 2000

Bespoke pioneering distributed systems

IBM CICS

Linklater proposes shared multi-site computing

Dozens of academic networks Two dominant protocols

ARPA net

CORBA & DCOM

IP-based InternetAcademic & Research

WWWCondor

I-way

GlobusStarts

Unicore

Web Services

Many research gridsUsing many protocols & M/W stacks

D-Gridstarts

Collaboration via shared bio/chem/medical “DBs”

EDG EGEEEGEE II


A Grid Computing Timeline

I-Way

: Sup

erCom

putin

g ‘95

1995 ’96 ’97 ’98 ’99 2000 ’01 ’02 ’03 ’04 ’05 2006US G

rid F

orum

form

s at S

C ‘98

Grid F

orum

s mer

ge, fo

rm G

GF

OGSA-WG fo

rmed

OGSA v1.0

“Ana

tomy”

pape

r

“Phy

siolog

y” pa

per

Europ

ean

& AP G

rid F

orum

s

• UK e-Science programme starts• UK e-Science programme starts

• DARPA funds Globus Toolkit & Legion• EU funds UNICORE project• US DoE pioneers grids for scientific research• NSF funds National Technology Grid• NASA starts Information Power Grid

• DARPA funds Globus Toolkit & Legion• EU funds UNICORE project• US DoE pioneers grids for scientific research• NSF funds National Technology Grid• NASA starts Information Power Grid

Today:• Grid solutions are common for HPC• Grid-based business solutions are

becoming common• Required technologies & standards are

evolving

Today:• Grid solutions are common for HPC• Grid-based business solutions are

becoming common• Required technologies & standards are

evolving

Japan government funds:• Business Grid project • NAREGI project

Japan government funds:• Business Grid project • NAREGI project

Source: Hiro Kishimoto GGF17 Keynote May 2006

GGF & E

GA

form

OGF


What is a Grid?

LicenseLicense

PrinterPrinter

A grid is a system consisting of− Distributed but connected resources and − Software and/or hardware that provides and manages logically

seamless access to those resources to meet desired objectives

A grid is a system consisting of− Distributed but connected resources and − Software and/or hardware that provides and manages logically

seamless access to those resources to meet desired objectives

R2AD

DatabaseDatabase

Webserver

Webserver

Data CenterCluster

Handheld Supercomputer

Workstation

Server


Ischia, Italy - 9-21 July 2006 10Source: Hiro Kishimoto GGF17 Keynote May 2006

Grid & Related Paradigms

Utility Computing• Computing “services”• No knowledge of provider• Enabled by grid technology

Utility Computing• Computing “services”• No knowledge of provider• Enabled by grid technology

Distributed Computing• Loosely coupled• Heterogeneous• Single Administration

Distributed Computing• Loosely coupled• Heterogeneous• Single Administration

Cluster• Tightly coupled• Homogeneous• Cooperative working

Cluster• Tightly coupled• Homogeneous• Cooperative working

Grid Computing• Large scale• Cross-organizational• Geographical distribution• Distributed Management

Grid Computing• Large scale• Cross-organizational• Geographical distribution• Distributed Management


How Are Grids Used?

High-performance computing

Collaborative data-sharing

Collaborative design

Drug discovery

Financial modeling

Data center automation

High-energy physics

Life sciences

E-Business

E-Science



Grids In Use: E-Science Examples

• Data sharing and integration− Life sciences, sharing standard data-sets,

combining collaborative data-sets− Medical informatics, integrating hospital information

systems for better care and better science− Sciences, high-energy physics

• Data sharing and integration− Life sciences, sharing standard data-sets,

combining collaborative data-sets− Medical informatics, integrating hospital information

systems for better care and better science− Sciences, high-energy physics

• Capability computing− Life sciences, molecular modeling, tomography− Engineering, materials science− Sciences, astronomy, physics

• Capability computing− Life sciences, molecular modeling, tomography− Engineering, materials science− Sciences, astronomy, physics

• High-throughput, capacity computing for − Life sciences: BLAST, CHARMM, drug screening− Engineering: aircraft design, materials, biomedical− Sciences: high-energy physics, economic modeling

• High-throughput, capacity computing for − Life sciences: BLAST, CHARMM, drug screening− Engineering: aircraft design, materials, biomedical− Sciences: high-energy physics, economic modeling

• Simulation-based science and engineering− Earthquake simulation

• Simulation-based science and engineering− Earthquake simulation



Grids: integrating & providing homogeneityGrids: integrating & providing homogeneity

• (Potentially) Generic & Industry Supported• Mechanism for combining many heterogeneous

resources so they can be used remotely– Data resources– Computation resources– Instruments– Research processes & procedures

• Restricting choices as to how it may be done– Harder for provider to make localised decisions– Deployment can be challenging

• Providing more homogeneity through virtualisation– Should be easier to compose services– More opportunity to amortise costs

• A component of e-Infrastructure

Deliberately choosing consistent interfaces, protocols

& management controls across a set of compatible

services


Grids as a Foundation for SolutionsGrids as a Foundation for Solutions

• The grid per se doesn’t provide– Supported e-Science methods– Supported data & information resources– Computations – Convenient access

• Grids help providers of these– International & national secure e-Infrastructure– Standards for interoperation– Standard APIs to promote re-use

• But Research Support must be built– What is needed?– Who should do it?

Much to be done by

developers of

applications & services

and by resource providers


Grids as a Foundation for SolutionsGrids as a Foundation for Solutions

• The grid per se doesn’t provide– Supported e-Science methods– Supported data & information resources– Computations – Convenient access

• Grids help providers of these– International & national secure e-Infrastructure– Standards for interoperation– Standard APIs to promote re-use

• But Research Support must be built– What is needed?– Who should do it?

Much to be done by

developers of

applications & services

and by resource providers

Therefore must support many categories of user:

•Application & Service Developers

•Deployers & Operations teams

•Higher-level abstraction builders & gateway providers


Why use / build Grids?Why use / build Grids?

• Research Arguments– Enables new ways of working– New distributed & collaborative research– Unprecedented scale and resources

• Economic Arguments– Reduced system management costs– Shared resources better utilisation– Pooled resources increased capacity– Load sharing & utility computing – Cheaper disaster recovery



• Computer Science Arguments– New attempt at an old hard problem– Frustrating ignorance existing results– New scale, new dynamics, new scope

• Engineering Arguments– Enable autonomous organisations to

• Write complementary software components• Set up run & use complementary services• Share operational responsibility• General & consistent environment for

Abstraction, Automation, Optimisation & Tools• Generally available code mobility



• Political & Management Arguments– Stimulate innovation– Promote intra-organisation collaboration– Promote inter-enterprise collaboration


Drivers for GridsDrivers for Grids

• Collaboration– Expertise is distributed– Necessary to achieve critical mass of effort– Necessary to raise sufficient resources

• Computational Power– Rapid growth in number of processors– Powered by Moore’s law + device roadmap– Challenge to transform models to exploit this

• Deluge of Data– Growth in scale: Number and Size of resources– Growth in complexity– Policy drives greater availability

Biomedical Research Informatics Delivered by Grid Enabled Services

Glasgow Edinburgh

Leicester Oxford

London

Netherlands

Publically Curated Data

Private data

Private data

Private data

Private data

Private data

Private data

CFG Virtual Organisation Ensembl

MGI

HUGO

OMIM

SWISS-PROT

… DATA HUB

RGD

SyntenyGrid

Service

blast

+

Portal

http://www.brc.dcs.gla.ac.uk/projects/bridges/

Overview of e-Infrastructure 25

eDiaMoND: Screening for Breast Cancer

1 Trust Many TrustsCollaborative WorkingAudit capabilityEpidemiology

Other Modalities-MRI-PET-Ultrasound

Better access toCase informationAnd digital tools

Supplement MentoringWith access to digitalTraining cases and sharingOf information acrossclinics

LettersRadiology reportingsystems

eDiaMoNDGrid

2ndary CaptureOr FFD

Case Information

X-Rays andCase Information

DigitalReading

SMF

Case andReading Information

CAD Temporal Comparison

Screening

ElectronicPatient Records

Assessment/ SymptomaticBiopsy

Case andReading Information

Symptomatic/AssessmentInformation

Training

Manage Training Cases

Perform Training

SMF CAD 3D Images

Patients

Provided by eDiamond project: Prof. Sir Mike Brady et al.

climateprediction.net and GENIE

• Largest climate model ensemble

• >45,000 users, >1,000,000 model years

10K2K

Response of Atlantic circulation to freshwater forcing

27Courtesy of David Gavaghan & IB Team

Integrative Biology

Tackling two Grand Challenge research questions:

• What causes heart disease?• How does a cancer form and grow?

Together these diseases cause 61% of all UK deaths

Will build a powerful, fault-tolerant Grid infrastructure for biomedical science

Enabling biomedical researchers to use distributed resources such as high-performance computers, databases and visualisation tools to develop complex models of how these killer diseases develop.

28Courtesy of David Gavaghan & IB Team

IB Partners


Foundations of CollaborationFoundations of Collaboration

• Strong commitment by individuals– To work together– To take on communication challenges– Mutual respect & mutual trust

• Distributed technology– To support information interchange– To support resource sharing– To support data integration– To support trust building

• Sufficient time• Common goals• Complementary knowledge, skills & data

Can we predictwhen it will work?Can we findremedies when itdoesn’t?


Grid & Collaboration QuestionsGrid & Collaboration Questions

• Without collaboration little is achievable• But must collaboration precede successful

grid applications?• Or will persistently and pervasively available

grids stimulate collaborations?

• If we deliver support for collaborative teams, will we also support the individual researcher?

• Can we use grids to democratise computation?

CARMEN - Scales of Integration

resolving the ‘neural code’ from the timing of action potential activity

determining ion channel contribution to the timing of action potentials

examining integration within networks of differing dimensions

Understanding the brain may be the greatest

informatics challenge of the 21st century

CARMEN Consortium

Leadership & Infrastructure

Colin Ingram

Paul Watson

Leslie Smith Jim Austin

CARMEN Consortium

International Partners

Daniel Gardner(Cornell)

Lead for the US NIH, Neuroscience Information Framework and Brain ML

Shiro Usui(RIKEN Brain Science Institute)

Lead for the Japan Node of the International Neuroinformatics Coordinating Facility

Sten Grillner(Karolinska Institute)

Chairman of the OECD,International Neuroinformatics Coordinating Facility

Ad Aertsen(Freiburg)

Neural network modelling and large-scale simulations

George Gerstein(Pennsylvania)

Analysis of spike pattern trains

CARMEN Consortium

Commercial Partners

- applications in the pharmaceutical sector

- interfacing of data acquisition software

- application of infrastructure

- commercialisation of tools

16th March 2006

NanoCMOSgriD Meeting the Design Challenges of Nano-CMOS Electronics

The ChallengeThe Challenge

Ischia, Italy - 9-21 July 2006

International Tech nology Roadmap for Semiconductors Year 2005 2010 2015 2020

MPU Half Pitch (nm) 90 45 25 14

MPU Gate Length (nm) 32 18 10 6

2005 edition Toshiba 04

Device diversification

90nm: HP, LOP, LSTP

45nm: UTB SOI

32nm: Double gate

25 nm

Bulk MOSFET

FD SOI

UTB SOI

FinFET

HP(MPU)

LOP

LSTP

Stat.Sets

230 nm

Bulk MOSFET

Standard

SingleSet

16th March 2006


Advanced Processor Technologies Group (APTGUM)

Device Modelling Group (DMGUG)

Electronic Systems Design Group (ESDGUS)

Intelligent Systems Group (ISGUY)

National e-Science Centre (NeSC)

Microsystems Technology Group (MSTGUG)

Mixed-Mode Design Group in IMNS (MMDGUE)

e-Science NorthWest Centre (eSNW)

University PartnersUniversity Partners

Ischia, Italy - 9-21 July 2006

16th March 2006


Industrial PartnersIndustrial PartnersGlobal EDS vendor and world TCAD leader600 licences of grid implementation, model implementation

UK fabless design company and world microprocessor leaderCore IP, simulation tools, staff time

UK fabless design company and world mixed mode leaderAdditional PhD studentship for mixed mode design

Global semiconductor player with strong UK presenceAccess to technology, device data, processing

Global semiconductor player with strong UK presenceAccess to technology, device data, processing

Global semiconductor player with UK presenceCASE studentship, interconnects

Trade association of the microelectronics industry in the UKRecruiting new industrial partners and dissemination


Grids in contextGrids in context

• Part of a long-term drive for distributed computing– A new and ambitious form

• Search for trade-offs & multiple uses– Leads to many varieties– Multiple stake holders

• Many good reasons for building & using grids• Question

– Will we have many grids or a consistent general purpose “foundation” grid?

• What are the minimum standards across the grids

• Collaboration is a key driver & enabler


Minimum Grid FunctionalitiesMinimum Grid Functionalities

• Supports distributed computation– Data and computation– Over a variety of

• hardware components (servers, data stores, …)• Software components (services: resource managers,

computation and data services)

– With regularity that can be exploited• By applications• By other middleware & tools• By providers and operations

– It will normally have security mechanisms • To develop and sustain trust regimes


After the breakAfter the break

• Distributed system challenges

• Service Oriented Architectures

• Examples of the basic architecture

ischia, italy - 9-21 july 20061 session 1 second part day 1 monday 10 th july malcolm atkinson

Documents

grid solutions

grid computing timeline

grid forums merge

research grids

us grid forum forms

distributed systems

history of grids

protocols mw stacks