click to edit master subtitle style 8/13/10 future of the ngs: virtual facilities sharing services...

Click to edit Master subtitle style

8/13/10

Future of the NGS:Virtual FacilitiesSharing Services

Dr Andrew RichardsSTFC Rutherford Appleton Laboratory, UK


UK e

-Infrastructure

LHC

ISIS TS2

HPCx+

HECtoR

Users

get common access, tools, information, nationally supported services, through NGS

Integratedinternationally

VRE, VLE, IE

Regional and Campus grids

Community Grids

HEIs

Collaborations

SAGA

To date

• Start: 4 core site to learn how to run grids in production.

• Now: 24 member sites, mixed resources.

• Future: – Virtual Facility, Shared Resources– national and international gateway to resource

providers for researchers.

8/13/10

NGS Member Institutions, February 2010

Specialist servicesWestminster • Operates and supports P-GRADE portal and GEMLCA legacy application support services

Belfast e-Science Centre Web Service Hosting Container Service

• Web service containers into which projects or VOs can deploy their own Web or Grid services, using automatic deployment technology

Oxford e-Research Centre• OMII-UK GridSAM

• OGF HPC-Basic Profile compliant Job submission• Promoting interoperability with international grids

• Eucalyptus Cloud system• Exposing AWS compatible interfaces and functionality to NGS users

Edinburgh• Eucalyptus Cloud system

STFC Rutherford Appleton Laboratory• Visualisation using specialised cluster from within the STFC e-Science Viz group

0

20

40

60

80

100

>1000 cpu, ~500 db~200 MyProxy users

25 VOs Supported

25 member institutes 33 heterogeneous resources15,000 processing cores

In the last 12 months• 4,629,127 CPU hrs used • 888,862 jobs ran

2nd largest e-Science CA• 22,121 certificates issued• 4,911 active currently

www.ngs.ac.uk

> 75 applications

Funding Source

Diverse User Community

Key areas

• Adoption of open standards

• Provide access to wide range of resources

• Integrating HEI resources and researchers

• Community-led support programmes

• Promote and facilitate training

Organisational Membership

• Campus Champion– Appointed of an institutional Campus Champion– Liaison between HEI/research organisation and NGS

• Infrastructure Members– NGS Interfaces – monitored and certified

– Affiliate• Maintains control over permitted users

– Partner• Supporting access by a significant body of NGS users• Publish a service level description (SLD) detailing the services

offered

Membership Programme

Goals:

1. Increase the range and depth of services and resources that NGS can offer to its users

1. Provide leadership and sources of best practice to sites needing to put their resources “on the Grid”

1. Create communities able to exploit the connected resources for interdisciplinary research

NGS Membership programme

Aiming to integrate services to access a growing number, scale and variety of resources spanning the complete space from advanced real time facilities such as synchrotrons and telescopes through to complex queries of historical data stored in national or institutional data centres.

Why join?

• Institutions have a mission to support their own users

• Increasing dependence on computation & data, and growing need to collaborate beyond the institution, make “getting on the Grid” a necessity for an institution to remain competitive

• Growing need to illustrate ‘green’ credentials which are easily demonstrated by commitment to efficient usage of currently owned resources through ‘grid’

Impact

• Improve accessibility to local resources

• Widen accessibility to local resources

• Use once Use anywhere

• Share/Trade/Buy/Sell resources

• Facilitate collaboration nationally and internationally

Benefits to the research community

– Access by local researchers to local resources through standard interfaces• Application level integration possible, the command line is no longer the only

option!• Support development of compliant access methods either available locally or

through a central service provider– Access to resources of a type that may not be locally available

• Software unlicensed in the university?• Data is generated at an international facility and needed on local resources?

– Usage of resources independent of location insulating them from local downtime and periods of resource instability

User access

• SARoNGS• WMS• HERMES Data Client• Application Hosting Environment (AHE)

Direct access GSI-SSH terminal

MEG

NGS Portal/ApplicationsRepository

Service Development (Filling the gaps)

Core Developments• NGS Portal: Applications Repository

– NGS, NSCCS• HERMES data client / DataMinx project• SARoNGS – Shibboleth Access• NGS Accounting : UAS : RUS• MEG – MyProxy enabled GSISSH access• Monitoring: INCA

Associated developments (EGEE)• APEL• GOCDB

Benefits to the institution

• Links into national e-infrastructure framework– Provision of value added services that a single institution wouldn’t want to run!– How large is the nationally collaborative research body?– Removing the possibility of needing to arrange NxN custom agreements with

different collaborating institutions• Allows for creation of inter-institutional economy

– smooth usage of resources allowing increased utilization of resource, increasing efficiency

– external accounting & quality assurance frameworks– standard interfaces allowing users to see no change

Support and Training

Local & national training events

NGS Road ShowsNational helpdeskCampus

Champion

Online tutorials and practicals

Training

• Direct support– Roadshows– Training for users (communities)– Train the trainers

• Opportunities– Shared workshops and schools– Influence the training agenda– Exploit expertise

Case Studies and Documentation

• Specific + “Generic”– Site neutral, topical, expert– Freely available, professionally produced– Exploit NGS interfaces and services (it works)

• Opportunities– Support local outreach– Exploit NGS documentation– Publicise your/our achievements

Events

• Community Conferences– CCPb, Bioinformatics,

NGS UF/IF, road shows


8/13/10

Best Practice

Best Practice• NGS well placed for this role

– Supported role to enable National Engagement– Collaborative activity not tied to 1 institution

• Coordination connected to National Facilities– Push an integration agenda, not specific field– Track record (RCUK international review)

• Opportunities– Growing recognition of role

• NGI, BBSRC + The Genome Analysis Centre, GridPP, BADC ...– Even better with your experience/expertise/services

• Application expertise

What does the NGS offer?

• Access to central support services for e-infrastructure– Helpdesk– Training– Application support– Central services e.g. CA, MyProxy, WMS etc

• Certification process to ensure resource providers deliver service against SLD/standard interfaces

• Common interfaces enable brokering of resources– Ease of access for end users– Ability to trade: buy/sell compute time across institute resource– Ability to link to international resources: International project participation– Ability to share data/migrate data/buy data services at other institutes for

resilience/backup/

Focus

• We Don’t do everything– Focus on

• Access to computational resources• Access to data storage• Data movement and management

– Emphasis on• Open standards• International standards• Collaboration

Supporting Collaboration

• “Grid” accounting– Support a person, group, project, institution ...

• A “Virtual Organisation”– Share or exchange resources

• NGS acts as “honest broker” • Standard interfaces give some future-proofing

• Opportunities– Support research collaborations (proven tools)– Load balancing– Value added tools with broad application– see next page ...

Outsourcing

• Standard interfaces – Simplify and reduce barrier to moving work – Avoid lock in and reduce barriers to moving– Allow 3rd part providers to offer services

• NGS “honest broker” role• Opportunities

– Sharing resources through common interfaces– Trading/buying/selling via NGS mediation– “commercial” services through standard interfaces

• E.g. cloud , data storage, training...


8/13/10

Impact on the Community

Research to Service

Research Group NGS University


8/13/10

International

ESFRI Projects

• European Strategic Forum for Research Infrastructure – “bottom up” list of (~30) pan-European RI– Strong support from EC

• Opportunities– Signposts for the future– Ensure UK access to key RI

• ELIXIR, BBMRI, ICOS, HiPER, XFEL, CLARIN, DARIAH ...– EC support – in connection with EC e-Infrastructure

Click to edit the outline text format Second Outline

Level Third Outline

Level Fourth Outline

Level Fifth

Outline Level

Sixth Outline Level

Seventh Outline Level

Eighth Outline Level

• Ninth Outline LevelClick to edit Master text styles– Second level

• Third level– Fourth level

» Fifth level

8/13/10


Level Third Outline

Level Fourth Outline

Level Fifth

Outline Level

Sixth Outline Level





» Fifth level

8/13/10

Grid: EGI

EGI.eu Office in Amsterdam (March)Information catalogs, AAA, Metadata/data catalogs,File replication, file transferJob brokeringInterfaces and portals ...

EGI and NGIs

• 34 Countries signed MOU• EGI.eu established in Amsterdam• Steven Newhouse interim director• 4 year project• Subscription based model• NGS (JISC signed MOU)

Intercontinental

• Existing NGS links– User led

• TeraGrid– Technical connections through e.g. OGF/GIN

• Security, Identity, data management, SRB/IRODs• MoU with OSG

• Opportunities– We know who to talk to


8/13/10

www.ngs.ac.uk

Dr Andrew [email protected]

Questions?


8/13/10

Case Studies

Note (to delete): CD suggests case studies highlighting different benefits of NGS, e.g. increased speed (burglary), more complex calculations/simulations (dinosaurs), novel technologies and interoperability (GENIUS), variety of services and interoperability with other resources (GENIE)… They all have applied benefits and impact on people/UK


Level Third Outline

LevelFourth

Outline Level Fifth

Outline Level

Sixth Outline Level





» Fifth level


Level Third Outline

LevelFourth

Outline Level Fifth

Outline Level

Sixth Outline Level





» Fifth level

Predicting CrimeNick Malleston, Leeds University

• burglary rates• agent-based predictive models• vary environmental factors, predicts

burglar’s behaviour

adapted Java program to run across Grid

NGS speeds things up - 2.5 years of results in under a week


Level Third Outline

LevelFourth

Outline Level Fifth

Outline Level

Sixth Outline Level





» Fifth level


Level Third Outline

LevelFourth

Outline Level Fifth

Outline Level

Sixth Outline Level





» Fifth level

Dinosaur locomotionKarl Bates, University of Manchester

• muscle activation patterns in dinosaurs

• NGS allows increased model sophistication

Oceans and ClimateAndrew Price, GENIE project, Southampton

• thermohaline circulation in oceans• integrates component earth models• future climate prediction

• 5 yrs computations in 3 months• integrates NGS and other resources• NGS hosts database• users share simulations, metadata

Cerebral blood flow

http://wiki.realitygrid.org/wiki/GENIUS

• processes 2D MRI images, recreates 3D vasculature map• visualise and steer the model in real time• advanced resource reservation• utilize international federated grid of supercomputers

GENIUS project

Simulation performed on the NGS of a

drug permeating through a membrane

Name: Dr Brian CheneyInstitution: University of SouthamptonResearch: Membrane Permeation

Drs Brian Cheney and Jonathan Essex research membrane permeation of small molecules at the University of Southampton. They’re interested in learning what physical and chemical features make a molecule a good or bad permeant, and in developing ways to quantify and estimate a molecule’s permeability.

Using the NGS to access geographically distributed astronomy databases

Name: Helen XiangInstitution: University of PortsmouthResearch: Astronomy databases

Helen Xiang and Professor Robert Nicol at the University of Portsmouth have been working on Astronomy databases. They use software called OGSA-DAI to link up astronomy data that is stored on the NGS and at Portsmouth. This way they can retrieve the data from two places with one command.


Level Third Outline

LevelFourth

Outline Level Fifth

Outline Level

Sixth Outline Level





» Fifth level


Level Third Outline

LevelFourth

Outline Level Fifth

Outline Level

Sixth Outline Level





» Fifth level

Simulating DNA Mechanics

Overtwisted DNA circles

Simulating DNA stretching

Work on the NGS done by Sarah Harris and Jon Mitchell, University of Leeds

GENIUS projectGrid Enabled Neurosurgical

Imaging Using Simulation

Demonstrations atAHM 2007, SC2007http://wiki.realitygrid.org/wiki/GENIUS

Working closely with researchers at University College London to get NGS supporting the Genius project this year

Real-time visualisation of blood flow through the brain. The simulation can be steered by input data and viewing angle.

Real-time visualisation done using ray-tracer in HemeLB algorithm to avoid data transmission and pre/post processing stages

Outreach

click to edit master subtitle style 8/13/10 future of the ngs: virtual facilities sharing services...

Documents

connected resources

mixed resources

member sites

depth of services

support programmespromote

grid services

stfc escience

service level description