the future is heterogeneous computing · the future is heterogeneous computing | oct 27, 2010. the...

| The Future Is Heterogeneous Computing | Oct 27, 2010

The Future Is Heterogeneous Computing

Mike Houston

Principal Architect, Accelerated Parallel ProcessingAdvanced Micro Devices

October 27th, 2010

Valencia, SpainSeptember 7, 2010


Workload Example: Changing Consumer Behavior

2

20 hoursof video

uploaded to YouTube

every minute

50 million +digital media files

added to personal content libraries

every day

Approximately

9 billionvideo files owned are

high-definition

1000 images

are uploaded to Facebook

every second


Challenges for Next Generation Systems

• The Power Wall Even more broadly constraining in the future!

• Complexity Management – HW and SW Principles for managing exponential growth

• Parallelism, Programmability and Efficiency Optimized SW for System-level Solutions

• System balance Memory Technologies and System Design

Interconnect Design


The Power Wall

• Easy prediction: Power will continue to be the #1 design constraint for Computer Systems design.

• Why? Vmin will not continue tracking Moore’s Law

Integration of system-level components consume chip power

– A well utilized 100GB/sec DDR memory interface consumes ~15W for the I/O alone!

2nd Order Effects of Power

– Thermal, packaging & cooling (node-level & datacenter-level)

– Electrical stability in the face of rising variablity

Thermal Design Points (TDPs) in all market segments continue to drop

Lightly loaded and idle power characteristics are key parameters in the Operational Expense (OpEx) equation

Percent of total world energy consumed by computing devices continues to grow year-on-year


Optimized SW for System-level Solutions

Long history of SW optimizations for HW “characteristics”• Optimizing compilers• Cache / TLB blocking• Multi-processor coordination: communication & synchronization• Non-uniform memory characteristics: Process and memory affinity

Scarcity/Abundance principle favors increased use of Abstractions

• Abstraction leads to Increased productivity but costs performance

• Still allow experts burrow down into lower level “on the metal” details

System-level Integration Era will demand even more• Many Core: user mode and/or managed runtime scheduling?• Heterogeneous Many Core: capability aware scheduling?

SW productivity versus optimization dichotomy• Exposed HW leads to better performance but requires a “platform

characteristics aware programming model”


The Memory Wall – getting thicker

There has always been a Critical Balance between Data Availability and Processing

Situation When? Implication Industry Solutions

DRAM vs CPU Cycle Time Gap Early 1990s

Memory wait time dominates computing

Non-blocking caches

O-o-O Machines

SW Productivity Crisis

Object oriented languages; Managed runtime environments

Mid1990s

Larger working setsMore diverse data types

Larger CachesCache HierarchiesElaborate prefetch

Single Thread CMP Focus 2005 and beyond

Multiple working sets! Virtual Machines!

More memory accesses

Huge CachesMultiple Memory

ControllersExtreme PHYs

New & Emerging Abstractions

Browser-based RuntimesImage/Video as basic data types

Throughput-based designs

2009 and beyond

Even larger working setsLarger data types

Accelerated Parallel Processing

Chip StackingTBD


Interconnect Challenges

• Coherence domain – knowing when to stop

Interesting implications for on-chip interconnect networks

• Industry Mantra: “Never bet against Ethernet”

But, current Ethernet not well suited for lossless transmission

Troublesome for storage, messaging and more

• The more subtle and trickier problems

Adaptive routing, congestion management, QOS, End-to-end characteristics, and more

• Data centers of tomorrow are going to take great interest in this area


Single-thread PerformanceIP

C

Issue Width

The IPC Complexity Wall

o

we arehere

Inte

gra

tion

(log s

cale

)

Time

Moore’s Law !

we arehere

o

Pow

er B

udget (T

DP)

Time

The Power Wall

we arehere

o

Frequency

Time

The Frequency Wall

we arehere

o

Sin

gle

-thre

ad P

erf

?

Time

we arehere

o

Single thread Perf (!)

- DFM- Variability- Reliability- Wire delay

Server: power=$$DT: eliminate fansMobile: battery

Perf

orm

ance

Cache Size

Locality

we arehere

o


0

20

40

60

80

100

120

140

1 2 4 8 16 32 64 128

Sp

ee

d-u

p

Number of CPU Cores

0% Serial

100% Serial

0

20

40

60

80

100

120

140

1 2 4 8 16 32 64 128

Sp

ee

d-u

p

Number of CPU Cores

0% Serial

10% Serial

35% Serial

100% Serial

Parallel Programs and Amdahl’s Law

Speed-up =1

SW + (1 – SW) / N

SW: % Serial WorkN: Number of processors

Assume 100W TDP Socket10W for global clocking20W for on-chip network/caches15W for I/O (memory, PCIe, etc)

This leaves 55W for all the cores 850mW per Core !


Transistors(thousands)

Single-threadPerformance(SpecINT)

Frequency(MHz)

Typical Power(Watts)

Number ofCores

Original data collected and plotted by M. Horowitz, F. Labonte, O. Shacham, K. Olukotun, L. Hammond and C. BattenDotted line extrapolations by C. Moore

35 Years of Microprocessor Trend Data


The Power Wall – Again!

• Escalating multi-core designs will crash into the power wall just like single cores did due to escalating frequency

• Why? In order to maintain a reasonable balance, core additions must be

accompanied by increases in other resources that consume power (on-chip network, caches, memory and I/O BW, …)

– Spiral upwards effect on power

The use of multiple cores forces each core to actually slow down– At some point, the power limits will not even allow you to activate all

of the cores at the same time

Small, low-power cores tend to be very weak on single-threaded general purpose workloads

– Customer value proposition will continue to demand excellent performance on general purpose workloads

– The transition to compelling general purpose parallel workloads will not be a fast one


What about Throughput Computing?

• Works around Amdahl’s law by focusing on throughput of multiple independent tasks Servers: Transaction Processing; Web Clicks; Search Queries

Clients: Graphics; Multimedia; Sensory Inputs (future)

HPC: Data-level parallelism

• New bottlenecks start to appear As some point, the OS itself becomes the “serial component”

User mode scheduling and task-stealing runtimes

Memory BW – Goal is to saturate the pipeline to memory

Large number of outstanding references

Large number of active and/or standby threads

Power – Overall utilization goes up, so does power consumption

Still the #1 constraint in modern computer design


Three Eras of Processor Performance

Single-Core Era

Sin

gle

-thre

ad Perf

orm

ance

?

Time

we arehere

o

Enabled by: Moore’s Law Voltage Scaling MicroArchitecture

Constrained by:PowerComplexity

Multi-Core Era

Thro

ughput P

erf

orm

ance

Time(# of Processors)

we arehere

o

Enabled by: Moore’s Law Desire for Throughput 20 years of SMP arch

Constrained by:PowerParallel SW availabilityScalability

HeterogeneousSystems Era

Targ

ete

d A

pplic

ation

Perf

orm

ance

Time(Data-parallel exploitation)

we arehere

o

Enabled by: Moore’s Law Abundant data parallelism Power efficient GPUs

Currently constrained by:Programming modelsCommunication overheads


2003

AMD x86 64-bit CMP Evolution

2005 2007 2008 2009 2010

AMD Opteron™Dual-Core

AMD OpteronQuad-Core

AMD Opteron

45nm Quad-Core

AMD Opteron

Six-Core AMD Opteron

AMD Opteron 6100 Series

Mfg. Process 90nm SOI 90nm SOI 65nm SOI 45nm SOI 45nm SOI 45nm SOI

CPU Core

K8 K8 Greyhound Greyhound+ Greyhound+ Greyhound+

L2/L3 1MB/0 1MB/0 512kB/2MB 512kB/6MB 512kB/6MB 512kB/12MB

HyperTransport™Technology

3x 1.6GT/.s 3x 1.6GT/.s 3x 2GT/s 3x 4.0GT/s 3x 4.8GT/s 4x 6.4GT/s

Memory 2x DDR1 300 2x DDR1 400 2x DDR2 667 2x DDR2 800 2x DDR2 1066 4x DDR3 1333

Max Power Budget Remains Consistent


L3 CACHE

AMD Opteron™ 6100 SeriesSilicon and Package

L3 CACHE

Core 2Core 1 Core 3

Core 4 Core 5 Core 6

12 AMD64 x86 Cores18 MB on-chip cache4 Memory Channels @ 1333 MHz4 HT Links @ 6.4 GT/sec


AMD Radeon HD5870 GPU Architecture

17

GPU Processing Performance Trend

0

500

1000

1500

2000

2500

3000

Sep-0

5

Mar

-06

Oct-0

6

Apr-07

Nov-0

7

Jun-

08

Dec-0

8Ju

l-09

GigaFLOPS

RV770ATI RADEON™HD 4800

ATI FirePro™V8700

AM D FireStream™92509270

RV670ATI RADEON™HD 3800ATI FireGL™

V7700AM D FireStream™

9170

R600ATI RADEON™HD 2900ATI FireGL™

V7600V8600V8650

R580(+)ATI RADEON™

X19xxATI FireStream™R520

ATI RADEON™X1800

ATI FireGL™ V7200V7300V7350

Unified Shaders

Double-precisionfloating pointGPGPU

via CTM

Stream SDKCAL+IL/Brook+

2.5x ALUincrease

* Peak single-precision performance;For RV670, RV770 & Cypress divide by 5 for peak double-precision performance

* CypressATI RADEON™HD 5870

OpenCL 1.1+DirectX 112.25x Perf.

18

0

2

4

6

8

10

12

14

16

Nov-05 Jan-06 Sep-07 Nov-07 Jun-08 Oct-09

ATI Radeon™ X1800 XT

ATI Radeon™ X1900 XTX

ATI Radeon™ HD 2900 PRO

ATI Radeon™ HD 3870



GPU Efficiency

7.50

4.56

4.50

2.24

2.21

0.92

2.01

1.061.07

0.42

GFLOPS/WGFLOPS/mm2

14.47GFLOPS/W

7.90GFLOPS/mm2

19

Digital Content Creation

AMD Accelerated Parallel Processing (APP) Technology is…

EngineeringSciences Government

Gaming Productivity

Heterogeneous: Developers leverage AMD GPUs and CPUs for optimal application performance and user experience

High performance: Massively parallel, programmable GPU architecture delivers unprecedented performance and power efficiency

Industry Standards: OpenCL™ enables cross-platform development

20

Moving Past Proprietary Solutions for Ease of Cross-Platform Programming

Open and Custom Tools

High Level Language Compilers

High Level Tools

Application Specific Libraries

OpenCL -

• Cross-platform development

• Interoperability with OpenGL and DX

• CPU/GPU backends enable balanced platform approach

Industry Standard Interfaces

OpenCL™DirectX® OpenGL®

AMD GPUs

Other CPUs/GPUs

AMD CPUs

21

Heterogeneous Computing:Next-Generation Software Ecosystem

Hardware & Drivers: AMD Fusion™,Discrete CPUs/GPUs

OpenCL & Direct Compute

Tools: HLLcompilers,Debuggers,

ProfilersMiddleware/Libraries: Video, Imaging, Math/Sciences,

Physics

High LevelFrameworks

End-user ApplicationsAd

vanc

ed O

ptim

izat

ions

& L

oad

Bal

anci

ngLoad balance across CPUs and GPUs; leverage AMD Fusion™ performance advantages Drive new

features into industry standards

Increase ease of application

development

22

AMD Balanced Platform Advantage

Delivers advanced performance for a wide range of platform configurations

Other Highly Parallel Workloads

Graphics Workloads

Serial/Task-Parallel Workloads

CPU is excellent for running some algorithms

Ideal place to process if GPU is fully loaded

Great use for additional CPU cores

GPU is ideal for data parallel algorithms like image processing, CAE, etc

Great use for AMD Accelerated Parallel Processing (APP) technology

Great use for additional GPUs


Challenges: Extracting Parallelism

i=0i++

load x(i)fmulstore

cmp i (1000000)bc

……

……

i,j=0i++j++

load x(i,j)fmulstore

cmp j (100000)bc

cmp i (100000)bc

2D array representingvery large

dataset

Loop 1M times for 1M pieces

of data

Coarse-grain dataparallel Code

Maps very well toThroughput-orienteddata parallel engines

i=0i++

load x(i)fmulstore

cmp i (16)bc

……

Loop 16 times for 16pieces of data

Fine-grain dataparallel Code

Maps very well tointegrated SIMD

dataflow (ie: SSE)

Nested dataparallel Code

Lots of conditional data parallelism. Benefits from closer coupling between CPU & GPU

24

A New Era of Processor Performance

Throughput Performance GPU

HomogeneousComputing

System-levelprogrammable

Multi-CoreEra

HeterogeneousSystems Era

Single-CoreEra

HeterogeneousComputing

Graphicsdriver-based

programs

OpenCL/DXdriver-based

programs

Prog

ram

mab

ility

CPU

Microprocessor Advancement

GPU

Advance

ment

25

Now the AMD Fusion Era of Computing Begins

26

DISCLAIMER

The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions and typographical errors.

The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product dif ferences between dif fering manufacturers, software changes, BIOS f lashes, f irmware upgrades, or the like. AMD assumes no obligation to update or otherwise correct or revise this information. However, AMD reserves the right to revise this information and to make changes from time to time to the content hereof without obligation of AMD to notify any person of such revisions or changes.

AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION.

AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE LIABLE TO ANY PERSON FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATIONCONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

This presentation contains forward-looking s tatements concerning AMD and technology partner produc t offerings which are made pursuant to the safe harbor provis ions of the P rivate Securities Litigation Reform A ct of 1995. Forward-looking s tatements are commonly identified by words such as "would," "may," "expects," "believes," "plans," "intends," “s trategy,” “roadmaps ,” "projects" and other terms with s imilar meaning. Investors are cautioned that the forward-looking s tatements in this presentation are based on current beliefs , assumptions and expectations, speak only as of the date of this presentation and involve risks and uncertainties that could cause ac tual results to differ materially from current expectations.

A TTRIBUTION

© 2010 Advanced M icro Devices, Inc. A ll rights reserved. A MD, the A MD A rrow logo, A MD O pteron, A TI, the A TI logo, Radeon and combinations thereof are trademarks of A dvanced M ic ro Devices, Inc. M icrosoft, Windows, and Windows V ista are registered trademarks of M icrosoft Corporation in the United States and/or other jurisdictions. O penCL is trademark of A pple Inc. used under license to the Khronos Group Inc. O ther names are for informational purposes only and may be trademarks of their respective owners .

the future is heterogeneous computing · the future is heterogeneous computing | oct 27, 2010. the...

Documents