Technology on the NGSPete Oliver

NGS Operations Manager

Technology• NGS Offers a diverse set of Technology– Very acronym rich!– Answer Questions such as:• Who am I and what can I do?• What resources do I have?• How do I run Jobs• How do I get access to my data?• What does my Institution need to get connected?• What’s the next new thing?

Who and What• Who am I and What can I do?

– Authentication• UK Certification Authority

– https://ca.grid-support.ac.uk/– Recognised World Wide

• Access Management Federation (SARoNGS)– UK

– Authorisation• Virtual Organisations (VO) ie communities of users• Partners Support All VOs• Affiliates Choose who they support• NGS offers VOM to manage the VO membership• https://voms.ngs.ac.uk/

– Supports established groups– AND– Adhoc communities that may be short lived

Tools

• Who am I and What can I do?– Tools

• Certificate management tool– JAVA based– Sets up authentication and authorisation– Certificate Wizard

» Uploads credentials the NGS Myproxy Service

• VOMS aware SSH terminals– MEG

» Allows users to use any ssh terminal tool if MEG installed on site– GSISSH-TERM

• NGS Myproxy Service – Acts on your behalf for authentication to resources

Resources

• How much CPU time have I got?– Resource providers upload to NGS RUS system• Keeps track of CPU time used on a user, site and VO

level• Web based interface for Users, VOs and Institutions

– https://uas.ngs.ac.uk/– Institutions can then manage there own collaborations

» Adhoc communities» Formal arrangements

• Support each other during downtime etc

Applications

• Finding applications and running jobs– Resource Providers publish applications they have

via to the NGS BDII using• Universal Execution Environment Scheme (UEE)

– All Application launch scripts are in /usr/ngs/app_name» Can be links to existing software» Launch script is customised by the Resource provider to

make sure the app runs on their site– Integration with GridPP

– Resource Providers Publish which VOs can use their resource

UEE and BDII

NGS BDIINAMDCASTEP

User Specifies UEE name “NAMD” in

WMS JDL and submits

HEI publish using has CASTEP using UEE to NGS BDII

HEI Compute

HEI publish using has NAMD using UEE to NGS BDII

HEI Compute

NGS WMS

NGS WMS Query

NGS WMS Launch

• How UEE and BDII are used

HEI publish using has NAMD using UEE to NGS BDII

HEI Compute

freeBusy

Freebusy

Portals

• Web Based Portal– Don’t need to login to UI• Login to portal instead• http://portal.ngs.ac.uk

– Configure jobs using Portal interfaces– Upload download files– Track jobs– Integration with WMS (to do)

Information

• What’s out there?– All NGS Partners and Affiliates publish

Information to NGS BDII• CPUs available• Applications available• Plus a lot of other info used by WMS • NGS Website uses this info to construct

– Usage meters– Applications lists

Monitoring

• How do I know my job will work– Monitoring

• Tests infrastructure as a user to ensure it works!

– All Partners and Affiliates are monitored• Site alerted when tests fail

– Reviewed by NGS Operations

• INCA– http://inca2.ngs.ac.uk

• Downtime in GOCDB (EGEE activity)– https://goc.gridops.org/

– When will it run?• WMS makes decision on load of sites• Very difficult to give exact timings

Data and Storage• Storing Data

– NGS does not provide storage • Except to the life time of running jobs

– Allows Resource Providers to make available resources• Gridftp servers, SRB Vaults

• Moving data– Small amounts can be done with the job

• The WMS manages the transfers

– SRB• Infrastructure (MCAT databases)• Partners and Affiliates can supply Vaults which use the central infrastructure

– Gui’s to help• HERMES

Institutional Resources

• How to setup your Institution’s resources– Access to NGS experts and guides• Tested installations

– Various OS’s » RedHat, SL, SUSE etc

– Software Stacks» VDT, Glite etc

• Knowledge will increase the more we do together

Resources out there

• The NGS itself does not provide any resources it allows HEIs to make their resources available to who they want to!– Eg STFC-RAL• 256 Core Cluster to all NGS users

– 30+ applications– Oracle

• 1800 Core Cluster to STFC staff, Facilities and Facility users (ISIS, CFL, DLS)– 30+ applications

Research & Development

• NGS R&D– Advanced Reservation– Novel environments• Virtualisation• Clouds

– Agile Deployments– Very fast Networking • OPN

Summary

• I

Technology User needs HEI need to run Run by NGS For all

UI * *

Myproxyuploadtool * *

Portal * *

MEG/GSISSH-TERM * *

Hermes * *

UAS * *

INCA Monitoring * *

Compute and Data Resources * *

End User Applications * *

RUS Clients * *

UEE * *

RA * *

Middle ware stack Globus/gLite/other * *

BDII Server *

Myproxy Server *

RUS Server *

SARoNGS *

VOM Server *

WMS *

CA and RA Network *

INCA Server *

Questions?

• Questions?

• http://www.ngs.ac.uk• support@grid-support.ac.uk