hpc trends for 2013
DESCRIPTION
In this presentation from the 2013 National HPCC Conference, Addison Snell from Intersect360 Research presents: HPC Trends for 2013. Addison Snell will present some of the top insights from recent market intelligence studies from Intersect360 Research, including forward-looking views of the vertical markets, new applications, and technologies with the best prospects for growth in 2012 and beyond. The view from Intersect360 Research will include applications in both High Performance Technical Computing (HPTC) and High Performance Business Computing (HPBC), with an emphasis on the opportunities for HPC technologies in emerging Big Data applications. The evolving industry dynamics around accelerators, file systems, and InfiniBand will also be discussed.” Watch the full presentation at: http://insidehpc.com/2013/04/03/video-hpc-trends-for-2013/TRANSCRIPT
New at Intersect360 Research • HPC500 user organization, www.hpc500.com
– Goal: 500 users worldwide, demographically representative of industry (verticals, geos, budgets)
– Over 100 end-user organizations signed up so far – Free access to research – First member call scheduled for April 4
• Hired Michael Feldman to analyst team • Report highlight overviews in slide format • Newsletter announcing research, articles,
podcasts, etc.: Sign up at intersect360.com
Topics for Discussion • Shifts in budget allocations • Processor architectures and their implications
– Memory – Programming models – Efficiency
• Big Data and its potential for HPC • What’s cloud got to do with it
HPC Budget Distribution by Year • ~$29B total market implies
~$44B total budget • Hardware declined every
year until sudden rebound in 2012
• Facilities increased every year until reversing in 2012
• Public cloud is a small part of the market
From HPC User Site Census The primary challenges for users are: • How to plan the balance between processors per node,
cores per processor, memory per node, I/O and interconnect on node, total nodes, etc.
• How to adapt applications for node parallelism and on-chip (i.e., multi-core) parallelism.
• How to organize the overall job mix. Smaller nodes may be a better fit for processing large numbers of small jobs or large set of jobs with a broad range of requirements. Larger nodes may work best with a job mix skewed to larger problems.
Memory Configuration
Accelerators (Mostly NVIDIA GPUs)
Challenges of Architecture Trends • Power consumption • Cost of memory • New models of parallelization • Languages and programming models, especially
for accelerated solutions • System efficiency • Personnel for administration, optimization,
programming services, etc.
Technical vs. Enterprise Computing
Technical Computing • Top-line missions:
– Find the oil – Design the minivan – Cure the disease
• Driven by price/performance
• Fast adoption of new technologies, algorithms, and approaches
Enterprise Computing • Keeps business running
– Communicate/collaborate – Market and sell the product – Accounting, HR, finance, …
• Driven by RAS: reliability, availability, serviceability
• Slow adoption of new technologies, algorithms, and approaches
Where Big Data Comes From • “Big Data” is not a specific application type, but
rather a trend – or even a collection of trends – spanning multiple application types
• Data growing in multiple ways: – More data (volume of data) – More types of data (variety of data) – Faster ingest of data (velocity of data) – Accessibility of data (internet, instrumentation, …)
• Exceeds organizational ability to manage data or make competitive decisions based on it
Different Types of Big Data • “Big” in Big Data is a relative term, like “High” in
High Performance Computing, not absolute TB or IOPS
• Different types of challenges: – Large files – Large numbers of files – Many users of files (concurrent access, copies) – Fast rate of ingest – Long or short lifespan of data
• Implication: If data, then value.
Scaling an Enterprise Data Infrastructure
Parallel file systems
I/O interconnects
Operating systems
Cloud
Important Insights on Big Data 1. It is much broader than Hadoop – many
different types of users and applications. 2. Money is being spent on it now – often 25% of
the annual IT budget. 3. Performance matters – even enterprise users
are buying based on performance.
Big Data trends will lead to the adoption of HPC technologies in more areas.
A Digression on Public HPC Clouds
• Cost Models • Barriers
Percent Rating Cloud Barrier “Significant”
Data movement and security are “top” barriers,
but there is a long list
• From special study on HPC and cloud, 2011
• N = 139 – 144
Cloud + Big Data: When Trends Collide
• Cloud is a major business computing trend. Big Data is a major business computing trend. Therefore …
• But the barriers to Big Data in cloud are the same as HPC in cloud (security, data movement, etc.)
• Not as simple as offloading everything to Amazon • If cloud is a priority, invest in management software to
coordinate workloads across public and private
I like sushi. I like ice cream. Therefore I like sushi-flavored ice cream.
Conclusions for HPC in 2013 • Users have options for breakout performance,
but they all require effort – Users are evaluating multi-core, accelerators, cloud,
programming models, etc., but not committing yet. – What solutions will be efficient? – What are the software and human skills required?
• There is an opportunity to expand the use of HPC technology
• Big Data is a bigger near-term opportunity than “Missing Middle” for most technologies
