what’s missing from hpc?

Update: What’s Missing from HPC?

John West DoD HPC Modernization Program

HPCC Newport 2013 Page-2

If you’re in this room, you probably think HPC is a great idea.


Peak Computational Capability by Country

And as we heard last year countries all over the world are racing to close the leadership gap in capabilities once held by just a small number of nations.

2012

1993

Data : top500.org


Few users

Supercomputing

High Performance Computing

Individual Computing

Few users

Many users

Technical Computing

“Missing Middle”


So…where are they?


The HPC and supercomputing users we have today didn’t have a choice.


The choice was obvious for individual computer users. And the middle, stayed at the bottom.


The journey from the bottom.


Individual Computing User

HPC User

• Primitive interfaces

• Incomplete toolchain

• Little expertise and no social support

• Complex management

• Expensive hardware and software


They already have something that works,

Why do they stay at the bottom?

and it’s too hard to just “take ‘er for a spin.”


How do we increase the reach of HPC?

Hardware has gotten cheaper and better There are system management options that

reduce deployment complexity Interfaces are primitive (some work needed) Incomplete toolchain (hand to hand combat) Little expertise and no social support “I don’t know how, and there’s no one around here

I can ask!”


Makers and Takers

HPC Consumer – Use high performance computers – Run applications – Understand computing principles

HPC Provider – Run and design high performance computers – Write and extend applications – Master computing principles

In practice this is a continuous spectrum, and workers may move in either direction during their career.


An interagency (NITRD) position

The NITRD High End Computing Interagency Working Group (HEC-IWG) position on education and workforce development (Mar 2013)

Articulates foundational principles

Starting place for coordinated agency programs that will build a workforce

NITRD is not a funding agency

The Networking and Information Technology Research and Development Program, www.nitrd.gov


NITRD Position Overview

Affirms the importance of HPC/HEC in national security and competitiveness terms

Reviews DOE/NNSA-funded survey on characteristics of the HEC workforce – Statistics remain a problem for this segment of the workforce

Articulates foundational principles that must be addressed for success


DoE HPC Provider Study IDC HPC User Forum: Special Study (July 2010). A Study of the Talent

and Skill Set Issues Impacting HPC Data Centers.

Staffing is hard – 93% of HPC centers surveyed said that hiring qualified staff is

“somewhat hard” or “very hard” with the majority reporting that it is “very hard” to find qualified staff.

Where do staff come from? – STEM grads – Other HPC centers – HPC vendors

What skills are needed on the provider side? – Combined understanding of a scientific discipline and computational

science and/or computer science; parallel programming and code optimization, especially for scaling to large processor/core counts; algorithm development; HEC system administration; and understanding of parallel file systems.


The NITRD Principles An effective program

– Increases the impact of HPC/HEC – Must address the entire spectrum consumer provider




Career transition for those already in the workforce just as important as increasing STEM grads – Many of us came to HPC after practicing in a discipline that uses it – Steal whenever possible (executive MBAs, certificates, …)




Career transition for those already in the workforce just as important as increasing STEM grads

If we want to teach it we have to define it – Enumerate the skill vectors that span our space (steal when possible):

admins, architects, developers, … – Then work with traditional and non-traditional education partners on

curricula




Career transition for those already in the workforce just as important as increasing STEM grads

If we want to teach it we have to define it …and reinforce it

– (Continue to) fund research that gives academic community experience with real world (ish) problems

– Internships, fellowships, awards, etc. – Establishing and illuminating HPC career paths will help with recruitment

and retention (certifications? Maybe eventually…)


Next Steps for NITRD HEC Members Define a set of career paths and skillsets Map the union of current efforts, identify gaps With educators, describe and develop curricula that

will produce new Providers and Consumers Pilot new, more flexible methods of education and

workforce development that enable in-career transitions

Continue to fund relevant academic research problems, internships, graduate and post-doctoral fellowships, and partnerships with industry and academia.

...and share, share, share


Read it at goo.gl/e03fU Comment at [email protected]