smart data slides: emerging hardware choices for modern ai data management

33
November 10, 2016 Adrian Bowles, PhD Founder, STORM Insights, Inc. [email protected] Emerging Hardware Choices for #ModernAI

Upload: dataversity

Post on 16-Apr-2017

370 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

November 10, 2016Adrian Bowles, PhD

Founder, STORM Insights, [email protected]

Emerging Hardware Choices for #ModernAI

Page 2: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

Hardware - The Final Frontier for Workload OptimizationPerformance Challenges for #ModernAIOptimizing Workloads Through Parallel ExecutionThree Architectural Paths

NeuromorphicGPU/Advanced Memory Quantum

Market Overview & Recommendations

Agenda

Page 3: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

Value Migrates to Hardware

OptimizeCommoditizeStandardize

Page 4: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

ConventionalAI

MachineLearning

BigData

#ModernAI Scope

Page 5: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

Emerging AI Hardware Trends and OptionsA Role for Hardware Optimization

CognitiveMachine LearningReasoningUnderstandingPlanning

Human InputLanguageVisionAural

Human-Oriented OutputMachine Input

IOTMachine-Oriented Output

Emerging AI Hardware Trends and Options

Page 6: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Human

Machine

Input OutputNarrative Generation

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

Data Mgmt

Learn Model

Reason

Understand

Plan

TasteSmell

Touch

Hear

See

Gestures

Emotions

Language

Visualization

Reports

Haptics

IoT IoT

Cognitive Systems: Communication & Control

Sensors

SystemsControls

Page 7: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

Hearing (audioception)~12,000 outer hair cells/ear

~3,500 inner hair cells Vision (ophthalmoception)Photoreceptors - Per Eye~120,000,000 rod cells

(triggered by single photon)~6,000,000 cone cells

(require more photons to trigger)~ 60,000 photosensitive ganglion cells

Touch (tactioception)Thermoreceptors, mechanoreceptors, chemoreceptors and nociceptors for touch, pressure, pain, temperature, vibration

Smell (olfacoception)Chemoreception

Taste (gustaoception)Chemoreception

Neurosynaptic Problem Solving Scope

Page 8: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

Hearing (audioception)~12,000 outer hair cells/ear

~3,500 inner hair cells Vision (ophthalmoception)Photoreceptors - Per Eye~120,000,000 rod cells

(triggered by single photon)~6,000,000 cone cells

(require more photons to trigger)~ 60,000 photosensitive ganglion cells

Touch (tactioception)Thermoreceptors, mechanoreceptors, chemoreceptors and nociceptors for touch, pressure, pain, temperature, vibration

Smell (olfacoception)Chemoreception

Taste (gustaoception)Chemoreception

Human Cognition~100,000,000,000 (100B) Neurons

~100-500,000,000,000,000 (100-500T) Synapses

Neurosynaptic Problem Solving Scope

Learn

ModelReason

Understand

Plan

Page 9: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2015 by STORM Insights Inc. All Rights reserved.

deeplearning

Deep learning refers to a biologically-inspired approach to machine learning that leverages a collection of simple processing units - analogous to neurosynaptic elements - that collaborate to solve complex problems at multiple levels of abstraction. These modern neural networks can support supervised, reinforcement, or unsupervised learning systems. In general, deep learning solutions require a high degree of parallelism, which may be implemented in hardware and/or software.

Deep Learning is Inherently Parallel

Page 10: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

Memory(Instructions & Data)

Central Processing Unit(CPU)

Control Unit

Arithmetic/Logic Unit(ALU)

InputDevice(s)

Output Device(s)

Operating System

The von Neumann Architecture

Page 11: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

Memory(Instructions & Data)

Central Processing Unit(CPU)

Control Unit

Arithmetic/Logic Unit(ALU)

InputDevice(s)

Output Device(s)

Operating System

“Speed”/Throughput Constraints

Page 12: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

Memory(Instructions & Data)

Central Processing Unit(CPU)

Control Unit

Arithmetic/Logic Unit(ALU)

InputDevice(s)

Output Device(s)

Operating System

Control Unit

Arithmetic/Logic Unit(ALU)

Parallelism With Multi-Cores

Page 13: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights Reserved. 9/28/2011

IBM Power 75090 servers, 32 cores/server, 2880 Cores in 10 racks

16Tb RAM

~80TeraFLOPS

80,000,000,000,000FLOPS

IBM Watson - Parallelism for Deep QA

Page 14: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Source: https://www.top500.org/system/177999

Amdahl’s Law: The theoretical performance improvement resulting from a resource improvement for a fixed workload is limited by that part of the workload that cannot benefit from the resource improvement.

Limits to Parallelism

Page 15: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2015 by STORM Insights Inc. All Rights reserved.

Research Examples:The European Commission FACETS (Fast Analog Computing with Emergent Transient States)

and BrainScaleS (Brain-inspired multi scale computation in neuromorphic hybrid systems)UK SpiNNaker (Spiking Neural Network Architecture)DARPA - SyNAPSE (Systems of Neuromorphic Adaptive Plastic Scalable Electronics)

Computer, device/component -level systems modeled after biological systems or components, such as neurons and synapses. These may be implemented in analog, digital or hybrid hardware. Typically designed to learn by experience over time, rather than by programming.

Neuromorphic Architectures (“Brain-Inspired”)

Massively interconnected networks of very simple processors.

Page 16: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

Synapse 16 chip board

Neuromorphic Architectures

IBM - SyNAPSE board

“TrueNorth chips can be seamlessly tiled to create vast, scalable neuromorphic systems.” Already demonstrated 16 million neurons and 4 billion synapses. Goal is to integrate 4,096 chips in a single rack with 4 billion neurons and 1 trillion synapses while consuming ~4kW of power.

Page 17: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Source: Qualcomm

Copyright (c) 2015 by STORM Insights Inc. All Rights reserved.

Neuromorphic Architectures

MAY 2, 2016: Qualcomm Incorporated (NASDAQ: QCOM) today announced at the Embedded Vision Summit in Santa Clara, Calif., that its subsidiary, Qualcomm Technologies, Inc., is offering the first deep learning software development kit (SDK) for devices powered by Qualcomm® Snapdragon™ 820 processors. The SDK, called the Qualcomm Snapdragon Neural Processing Engine, is powered by the Qualcomm® Zeroth™ Machine Intelligence Platform

Page 18: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

The Nvidia M40 processor for training neural networks.Nvidia

NVIDIA Maxwell™ architectureUp to 7 Teraflops of single-precision performance with NVIDIA GPU Boost™3072 NVIDIA CUDA® cores24 GB of GDDR5 memory288 GB/sec memory bandwidthQualified to deliver maximum uptime in the datacenter

GPU/Advanced Memory Architectures

Page 19: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

GPU/Advanced Memory Architectures

Page 20: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

Server racks with TPUs used in the AlphaGo matches with Lee Sedol

GPU/Advanced Memory Architectures

Page 21: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

At Facebook, we've made great progress thus far with off-the-shelf infrastructure components and design. We've developed software that can read stories, answer questions about scenes, play games and even learn unspecified tasks through observing some examples. But we realized that truly tackling these problems at scale would require us to design our own systems. Today, we're unveiling our next-generation GPU-based systems for training neural networks, which we've code-named “Big Sur.”

• FAIR is more than tripling its investment in GPU hardware as we focus even more on research and enable other teams across the company to use neural networks in our products and services.

• As part of our ongoing commitment to open source and open standards, we plan to contribute our innovations in GPU hardware to the Open Compute Project so others can benefit from them.

Facebook Open-source AI hardware design

https://code.facebook.com/posts/1687861518126048/facebook-to-open-source-ai-hardware-design/

GPU/Advanced Memory Architectures

Page 22: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

Source: https://www.micron.com/about/emerging-technologies/automata-processing

GPU/Advanced Memory Architectures

Page 23: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

GPU/Advanced Memory Architectures

Page 24: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

http://www.research.ibm.com/quantum/

Quantum Architectures

Page 25: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

Source: https://arxiv.org/abs/1608.00263

Quantum Architectures

Page 26: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

Probabalistic Architecture?

Page 27: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

NeuromorphicGPU/Memory Acceleration Quantum

Market/Technology Positions & Maturity

Ready NowMuch More in the Pipeline

Promising -Ready Now At Handset Level

Promising -Watch But Don’t Wait

Proven approach for ||ismEasy interoperability with conventional systems

+Natural behavioral process model+Lower power requirements- Requires new software model

& skills

+Incredible compute power potential- Requires new software model

& skills- Requires interface to

conventional system for pre-processing

- Requires extremely cold (big, expensive) environment

Page 28: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

IBMQualcomm

Brain Corporation (hosted by Qualcomm)KnupathTenstorrentCirrascaleNeurogrid (Stanford)Tensilica - Cadence1026 LabsCerebrasArtificial LearningHRL LaboratoriesIsocline

NvidiaIntelAMD

Facebook (FAIR)

Nervana Systems/IntelMovidius - Intel (Vision processing)Google TPU

IBMD-WaveGoogle

NeuromorphicGPU/Memory Acceleration Quantum

Ones to Watch

On the Horizon

Ready NowMuch More in the Pipeline

Promising -Ready Now At Handset Level

Promising -Watch But Don’t Wait

Page 29: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

[email protected] @ajbowlesSkype ajbowles

Upcoming Webinar Dates & Topics

December 8 Leverage the IOT to Build a Smart Data Ecosystem

January #Modern AI and Cognitive Computing: Boundaries and OpportunitiesFebruary Artificial General Intelligence: When I Can I Get It?March Data Science and Business Analysis: A Look at Best Practices for Roles, Skills, and ProcessesApril Machine Learning: Moving Beyond Discovery to UnderstandingMay Streaming Analytics for Agile IoT-Oriented ApplicationsJune Machine Learning Case StudiesJuly Advances in Natural Language Processing I: UnderstandingAugust Organizing Data and Knowledge: The Role of Taxonomies and OntologiesSeptember Advances in Natural Language Processing II: NL GenerationOctober Choosing the Right Data Management Architecture for Cognitive ComputingNovember See Me, Feel Me, Touch Me, Heal Me: The Rise of the Cognitive InterfaceDecember The Road to Autonomous Applications

For More Information…

Page 30: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

Basilar membrane. (2016, October 28). In Wikipedia, The Free Encyclopedia. Retrieved 01:58, October 28, 2016, from https://en.wikipedia.org/w/index.php?title=Basilar_membrane&oldid=746543229

Somatosensory system. (2016, October 9). In Wikipedia, The Free Encyclopedia. Retrieved 04:59, October 9, 2016, from https://en.wikipedia.org/w/index.php?title=Somatosensory_system&oldid=743336883

Photoreceptor cell. (2016, September 19). In Wikipedia, The Free Encyclopedia. Retrieved 03:07, September 19, 2016, from https://en.wikipedia.org/w/index.php?title=Photoreceptor_cell&oldid=740108113

Page 31: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

Hardware - The Final Frontier for Workload Optimization#ModernAI DefinedPerformance Challenges Optimizing Workloads Through Parallel ExecutionThree Architecture Paths

NeuromorphicGPU/Advanced Memory Quantum

Agenda

A Role for HardwareCognitive

Machine LearningReasoningUnderstandingPlanning

Human InputLanguageVisionAural

Human-Oriented OutputMachine Input

IOTMachine-Oriented Output

Page 32: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.

Page 33: Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management

Copyright (c) 2015 by STORM Insights Inc. All Rights reserved.