Download - HPC Trends for 2013
New at Intersect360 Research • HPC500 user organization, www.hpc500.com
– Goal: 500 users worldwide, demographically representative of industry (verticals, geos, budgets)
– Over 100 end-user organizations signed up so far – Free access to research – First member call scheduled for April 4
• Hired Michael Feldman to analyst team • Report highlight overviews in slide format • Newsletter announcing research, articles,
podcasts, etc.: Sign up at intersect360.com
Topics for Discussion • Shifts in budget allocations • Processor architectures and their implications
– Memory – Programming models – Efficiency
• Big Data and its potential for HPC • What’s cloud got to do with it
HPC Budget Distribution by Year • ~$29B total market implies
~$44B total budget • Hardware declined every
year until sudden rebound in 2012
• Facilities increased every year until reversing in 2012
• Public cloud is a small part of the market
From HPC User Site Census The primary challenges for users are: • How to plan the balance between processors per node,
cores per processor, memory per node, I/O and interconnect on node, total nodes, etc.
• How to adapt applications for node parallelism and on-chip (i.e., multi-core) parallelism.
• How to organize the overall job mix. Smaller nodes may be a better fit for processing large numbers of small jobs or large set of jobs with a broad range of requirements. Larger nodes may work best with a job mix skewed to larger problems.
Memory Configuration
Accelerators (Mostly NVIDIA GPUs)
Challenges of Architecture Trends • Power consumption • Cost of memory • New models of parallelization • Languages and programming models, especially
for accelerated solutions • System efficiency • Personnel for administration, optimization,
programming services, etc.
Technical vs. Enterprise Computing
Technical Computing • Top-line missions:
– Find the oil – Design the minivan – Cure the disease
• Driven by price/performance
• Fast adoption of new technologies, algorithms, and approaches
Enterprise Computing • Keeps business running
– Communicate/collaborate – Market and sell the product – Accounting, HR, finance, …
• Driven by RAS: reliability, availability, serviceability
• Slow adoption of new technologies, algorithms, and approaches
Where Big Data Comes From • “Big Data” is not a specific application type, but
rather a trend – or even a collection of trends – spanning multiple application types
• Data growing in multiple ways: – More data (volume of data) – More types of data (variety of data) – Faster ingest of data (velocity of data) – Accessibility of data (internet, instrumentation, …)
• Exceeds organizational ability to manage data or make competitive decisions based on it
Different Types of Big Data • “Big” in Big Data is a relative term, like “High” in
High Performance Computing, not absolute TB or IOPS
• Different types of challenges: – Large files – Large numbers of files – Many users of files (concurrent access, copies) – Fast rate of ingest – Long or short lifespan of data
• Implication: If data, then value.
Scaling an Enterprise Data Infrastructure
Parallel file systems
I/O interconnects
Operating systems
Cloud
Important Insights on Big Data 1. It is much broader than Hadoop – many
different types of users and applications. 2. Money is being spent on it now – often 25% of
the annual IT budget. 3. Performance matters – even enterprise users
are buying based on performance.
Big Data trends will lead to the adoption of HPC technologies in more areas.
A Digression on Public HPC Clouds
• Cost Models • Barriers
Percent Rating Cloud Barrier “Significant”
Data movement and security are “top” barriers,
but there is a long list
• From special study on HPC and cloud, 2011
• N = 139 – 144
Cloud + Big Data: When Trends Collide
• Cloud is a major business computing trend. Big Data is a major business computing trend. Therefore …
• But the barriers to Big Data in cloud are the same as HPC in cloud (security, data movement, etc.)
• Not as simple as offloading everything to Amazon • If cloud is a priority, invest in management software to
coordinate workloads across public and private
I like sushi. I like ice cream. Therefore I like sushi-flavored ice cream.
Conclusions for HPC in 2013 • Users have options for breakout performance,
but they all require effort – Users are evaluating multi-core, accelerators, cloud,
programming models, etc., but not committing yet. – What solutions will be efficient? – What are the software and human skills required?
• There is an opportunity to expand the use of HPC technology
• Big Data is a bigger near-term opportunity than “Missing Middle” for most technologies
