Edited byShih-Chii LiuTobi DelbruckGiacomo Indiveri Adrian WhatleyRodney Douglas

1 2 1 4 3

1

2

3

4 address events

Vtun

CC

Vtun

Iin Iout

X

Y

Time

Space

5 m

s

Space-Timeaxondendrite

soma

Event-Based Neuromorphic Systems

AE

R en

cod

er

AE

R d

ecod

er

EVENT-BASEDNEUROMORPHICSYSTEMS

EVENT-BASEDNEUROMORPHICSYSTEMS

Edited by

Shih-Chii LiuTobi DelbruckGiacomo IndiveriAdrian WhatleyRodney DouglasUniversity of Zurich and ETH ZurichSwitzerland

This edition first published 2015© 2015 John Wiley & Sons, Ltd

Registered officeJohn Wiley & Sons Ltd,The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, United Kingdom

For details of our global editorial offices, for customer services and for information about how to apply forpermission to reuse the copyright material in this book please see our website at www.wiley.com.

The right of the author to be identified as the author of this work has been asserted in accordance with the Copyright,Designs and Patents Act 1988.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in anyform or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UKCopyright, Designs and Patents Act 1988, without the prior permission of the publisher.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not beavailable in electronic books.

Designations used by companies to distinguish their products are often claimed as trademarks. All brand names andproduct names used in this book are trade names, service marks, trademarks or registered trademarks of theirrespective owners. The publisher is not associated with any product or vendor mentioned in this book.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparingthis book, they make no representations or warranties with respect to the accuracy or completeness of the contents ofthis book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. It issold on the understanding that the publisher is not engaged in rendering professional services and neither thepublisher nor the author shall be liable for damages arising herefrom. If professional advice or other expertassistance is required, the services of a competent professional should be sought.

Library of Congress Cataloging-in-Publication Data applied for.

ISBN: 9780470018491

Set in 10/12pt Times by Aptara Inc., New Delhi, India

1 2015

This book is dedicated to the memories of Misha Mahowald,Jorg Kramer, and Paul Mueller.

Contents

List of Contributors xv

Foreword xvii

Acknowledgments xix

List of Abbreviations and Acronyms xxi

1 Introduction 11.1 Origins and Historical Context 31.2 Building Useful Neuromorphic Systems 5

References 5

Part I UNDERSTANDING NEUROMORPHIC SYSTEMS 7

2 Communication 92.1 Introduction 92.2 Address-Event Representation 12

2.2.1 AER Encoders 132.2.2 Arbitration Mechanisms 132.2.3 Encoding Mechanisms 172.2.4 Multiple AER Endpoints 192.2.5 Address Mapping 192.2.6 Routing 19

2.3 Considerations for AER Link Design 202.3.1 Trade-off: Dynamic or Static Allocation 212.3.2 Trade-off: Arbitered Access or Collisions? 232.3.3 Trade-off: Queueing versus Dropping Spikes 242.3.4 Predicting Throughput Requirements 252.3.5 Design Trade-offs 27

2.4 The Evolution of AER Links 282.4.1 Single Sender, Single Receiver 282.4.2 Multiple Senders, Multiple Receivers 302.4.3 Parallel Signal Protocol 31

viii Contents

2.4.4 Word-Serial Addressing 322.4.5 Serial Differential Signaling 33

2.5 Discussion 34References 35

3 Silicon Retinas 373.1 Introduction 373.2 Biological Retinas 383.3 Silicon Retinas with Serial Analog Output 393.4 Asynchronous Event-Based Pixel Output Versus Synchronous Frames 403.5 AER Retinas 40

3.5.1 Dynamic Vision Sensor 413.5.2 Asynchronous Time-Based Image Sensor 463.5.3 Asynchronous Parvo–Magno Retina Model 463.5.4 Event-Based Intensity-Coding Imagers (Octopus and TTFS) 483.5.5 Spatial Contrast and Orientation Vision Sensor (VISe) 50

3.6 Silicon Retina Pixels 543.6.1 DVS Pixel 543.6.2 ATIS Pixel 563.6.3 VISe Pixel 583.6.4 Octopus Pixel 59

3.7 New Specifications for Silicon Retinas 603.7.1 DVS Response Uniformity 603.7.2 DVS Background Activity 623.7.3 DVS Dynamic Range 623.7.4 DVS Latency and Jitter 63

3.8 Discussion 64References 67

4 Silicon Cochleas 714.1 Introduction 724.2 Cochlea Architectures 75

4.2.1 Cascaded 1D 764.2.2 Basic 1D Silicon Cochlea 774.2.3 2D Architecture 784.2.4 The Resistive (Conductive) Network 794.2.5 The BM Resonators 804.2.6 The 2D Silicon Cochlea Model 804.2.7 Adding the Active Nonlinear Behavior of the OHCs 82

4.3 Spike-Based Cochleas 834.3.1 Q-control of AEREAR2 Filters 854.3.2 Applications: Spike-Based Auditory Processing 86

4.4 Tree Diagram 874.5 Discussion 87

References 89

Contents ix

5 Locomotion Motor Control 915.1 Introduction 92

5.1.1 Determining Functional Biological Elements 925.1.2 Rhythmic Motor Patterns 93

5.2 Modeling Neural Circuits in Locomotor Control 955.2.1 Describing Locomotor Behavior 965.2.2 Fictive Analysis 975.2.3 Connection Models 995.2.4 Basic CPG Construction 1005.2.5 Neuromorphic Architectures 102

5.3 Neuromorphic CPGs at Work 1085.3.1 A Neuroprosthesis: Control of Locomotion in Vivo 1095.3.2 Walking Robots 1115.3.3 Modeling Intersegmental Coordination 112

5.4 Discussion 113References 115

6 Learning in Neuromorphic Systems 1196.1 Introduction: Synaptic Connections, Memory, and Learning 1206.2 Retaining Memories in Neuromorphic Hardware 121

6.2.1 The Problem of Memory Maintenance: Intuition 1216.2.2 The Problem of Memory Maintenance: Quantitative Analysis 1226.2.3 Solving the Problem of Memory Maintenance 124

6.3 Storing Memories in Neuromorphic Hardware 1286.3.1 Synaptic Models for Learning 1286.3.2 Implementing a Synaptic Model in Neuromorphic Hardware 132

6.4 Toward Associative Memories in Neuromorphic Hardware 1366.4.1 Memory Retrieval in Attractor Neural Networks 1376.4.2 Issues 142

6.5 Attractor States in a Neuromorphic Chip 1436.5.1 Memory Retrieval 1436.5.2 Learning Visual Stimuli in Real Time 145

6.6 Discussion 148References 149

Part II BUILDING NEUROMORPHIC SYSTEMS 153

7 Silicon Neurons 1557.1 Introduction 1567.2 Silicon Neuron Circuit Blocks 158

7.2.1 Conductance Dynamics 1587.2.2 Spike-Event Generation 1597.2.3 Spiking Thresholds and Refractory Periods 1617.2.4 Spike-Frequency Adaptation and Adaptive Thresholds 162

x Contents

7.2.5 Axons and Dendritic Trees 1647.2.6 Additional Useful Building Blocks 165

7.3 Silicon Neuron Implementations 1667.3.1 Subthreshold Biophysically Realistic Models 1667.3.2 Compact I&F Circuits for Event-Based Systems 1697.3.3 Generalized I&F Neuron Circuits 1707.3.4 Above Threshold, Accelerated-Time, and Switched-Capacitor

Designs 1747.4 Discussion 176

References 180

8 Silicon Synapses 1858.1 Introduction 1868.2 Silicon Synapse Implementations 188

8.2.1 Non Conductance-Based Circuits 1888.2.2 Conductance-Based Circuits 1988.2.3 NMDA Synapse 200

8.3 Dynamic Plastic Synapses 2018.3.1 Short-Term Plasticity 2018.3.2 Long-Term Plasticity 203

8.4 Discussion 213References 215

9 Silicon Cochlea Building Blocks 2199.1 Introduction 2199.2 Voltage-Domain Second-Order Filter 220

9.2.1 Transconductance Amplifier 2209.2.2 Second-Order Low-Pass Filter 2229.2.3 Stability of the Filter 2239.2.4 Stabilised Second-Order Low-Pass Filter 2259.2.5 Differentiation 225

9.3 Current-Domain Second-Order Filter 2279.3.1 The Translinear Loop 2279.3.2 Second-Order Tau Cell Log-Domain Filter 229

9.4 Exponential Bias Generation 2309.5 The Inner Hair Cell Model 2339.6 Discussion 234

References 234

10 Programmable and Configurable Analog Neuromorphic ICs 23710.1 Introduction 23810.2 Floating-Gate Circuit Basics 23810.3 Floating-Gate Circuits Enabling Capacitive Circuits 23810.4 Modifying Floating-Gate Charge 242

10.4.1 Electron Tunneling 24210.4.2 pFET Hot-Electron Injection 242

Contents xi

10.5 Accurate Programming of Programmable Analog Devices 24410.6 Scaling of Programmable Analog Approaches 24610.7 Low-Power Analog Signal Processing 24710.8 Low-Power Comparisons to Digital Approaches: Analog Computing in

Memory 24910.9 Analog Programming at Digital Complexity: Large-Scale Field

Programmable Analog Arrays 25110.10 Applications of Complex Analog Signal Processing 253

10.10.1 Analog Transform Imagers 25310.10.2 Adaptive Filters and Classifiers 253

10.11 Discussion 256References 257

11 Bias Generator Circuits 26111.1 Introduction 26111.2 Bias Generator Circuits 263

11.2.1 Bootstrapped Current Mirror Master Bias Current Reference 26311.2.2 Master Bias Power Supply Rejection Ratio (PSRR) 26511.2.3 Stability of the Master Bias 26511.2.4 Master Bias Startup and Power Control 26611.2.5 Current Splitters: Obtaining a Digitally Controlled Fraction of the

Master Current 26711.2.6 Achieving Fine Monotonic Resolution of Bias Currents 27111.2.7 Using Coarse–Fine Range Selection 27311.2.8 Shifted-Source Biasing for Small Currents 27411.2.9 Buffering and Bypass Decoupling of Individual Biases 27511.2.10 A General Purpose Bias Buffer Circuit 27811.2.11 Protecting Bias Splitter Currents from Parasitic Photocurrents 279

11.3 Overall Bias Generator Architecture Including External Controller 27911.4 Typical Characteristics 28011.5 Design Kits 28111.6 Discussion 282

References 282

12 On-Chip AER Communication Circuits 28512.1 Introduction 286

12.1.1 Communication Cycle 28612.1.2 Speedup in Communication 287

12.2 AER Transmitter Blocks 28912.2.1 AER Circuits within a Pixel 28912.2.2 Arbiter 29012.2.3 Other AER Blocks 29512.2.4 Combined Operation 297

12.3 AER Receiver Blocks 29812.3.1 Chip-Level Handshaking Block 29812.3.2 Decoder 299

xii Contents

12.3.3 Handshaking Circuits in Receiver Pixel 30012.3.4 Pulse Extender Circuits 30112.3.5 Receiver Array Peripheral Handshaking Circuits 301

12.4 Discussion 302References 303

13 Hardware Infrastructure 30513.1 Introduction 306

13.1.1 Monitoring AER Events 30713.1.2 Sequencing AER Events 31113.1.3 Mapping AER Events 313

13.2 Hardware Infrastructure Boards for Small Systems 31613.2.1 Silicon Cortex 31613.2.2 Centralized Communication 31713.2.3 Composable Architecture Solution 31813.2.4 Daisy-Chain Architecture 32413.2.5 Interfacing Boards using Serial AER 32413.2.6 Reconfigurable Mesh-Grid Architecture 328

13.3 Medium-Scale Multichip Systems 32913.3.1 Octopus Retina + IFAT 32913.3.2 Multichip Orientation System 33213.3.3 CAVIAR 335

13.4 FPGAs 34013.5 Discussion 342

References 345

14 Software Infrastructure 34914.1 Introduction 349

14.1.1 Importance of Cross-Community Commonality 35014.2 Chip and System Description Software 350

14.2.1 Extensible Markup Language 35114.2.2 NeuroML 351

14.3 Configuration Software 35214.4 Address Event Stream Handling Software 352

14.4.1 Field-Programmable Gate Arrays 35314.4.2 Structure of AE Stream Handling Software 35314.4.3 Bandwidth and Latency 35314.4.4 Optimization 35414.4.5 Application Programming Interface 35514.4.6 Network Transport of AE Streams 355

14.5 Mapping Software 35614.6 Software Examples 357

14.6.1 ChipDatabase – A System for Tuning Neuromorphic aVLSI Chips 35714.6.2 Spike Toolbox 35914.6.3 jAER 35914.6.4 Python and PyNN 360

Contents xiii

14.7 Discussion 363References 363

15 Algorithmic Processing of Event Streams 36515.1 Introduction 36515.2 Requirements for Software Infrastructure 367

15.2.1 Processing Latency 36915.3 Embedded Implementations 36915.4 Examples of Algorithms 370

15.4.1 Noise Reduction Filters 37015.4.2 Time-Stamp Maps and Subsampling by Bit-Shifting Addresses 37215.4.3 Event Labelers as Low-Level Feature Detectors 37215.4.4 Visual Trackers 37415.4.5 Event-Based Audio Processing 378

15.5 Discussion 379References 379

16 Towards Large-Scale Neuromorphic Systems 38116.1 Introduction 38116.2 Large-Scale System Examples 382

16.2.1 Spiking Neural Network Architecture 38216.2.2 Hierarchical AER 38416.2.3 Neurogrid 38616.2.4 High Input Count Analog Neural Network System 388

16.3 Discussion 390References 391

17 The Brain as Potential Technology 39317.1 Introduction 39317.2 The Nature of Neuronal Computation: Principles of Brain Technology 39517.3 Approaches to Understanding Brains 39617.4 Some Principles of Brain Construction and Function 39817.5 An Example Model of Neural Circuit Processing 40017.6 Toward Neuromorphic Cognition 402

References 404

Index 407

List of Contributors

Editors:Shih-Chii LiuTobi DelbruckGiacomo IndiveriAdrian WhatleyRodney DouglasInstitute of NeuroinformaticsUniversity of Zurich and ETH ZurichZurich, Switzerland

Contributors:Corey AshbyJohns Hopkins UniversityBaltimore, MD, USA

Ralph Etienne-CummingsJohns Hopkins UniversityBaltimore, MD, USA

Paolo Del GiudiceDepartment of Technologies and HealthIstituto Superiore di SanitaRome, Italy

Stefano FusiCenter for Theoretical NeuroscienceColumbia UniversityNew York, NY, USA

Tara HamiltonThe MARCS InstituteUniversity of Western SydneySydney, Australia

xvi List of Contributors

Jennifer HaslerGeorgia TechAtlanta, GA, USA

Alejandro Linares-BarrancoUniversidad de SevillaSevilla, Spain

Bernabe Linares-BarrancoNational Microelectronics Center(IMSE-CNM-CSIC)Sevilla, Spain

Rajit ManoharCornell TechNew York, NY, USA

Kevan MartinInstitute of NeuroinformaticsUniversity of Zurich and ETH ZurichZurich, Switzerland

Andre van SchaikThe MARCS InstituteUniversity of Western SydneySydney, Australia

Jacob VogelsteinJohns Hopkins UniversityBaltimore, MD, USA

Contributors by Chapter:Chapter 1: Tobi DelbruckChapter 2: Rajit Manohar, Adrian Whatley, Shih-Chii LiuChapter 3: Tobi Delbruck, Bernabe Linares-BarrancoChapter 4: Andre van Schaik, Tara Hamilton, Shih-Chii LiuChapter 5: Corey Ashby, Ralph Etienne-Cummings, Jacob VogelsteinChapter 6: Stefano Fusi, Paolo del GiudiceChapter 7: Giacomo IndiveriChapter 8: Shih-Chii Liu, Giacomo IndiveriChapter 9: Andre van Schaik, Tara HamiltonChapter 10: Jennifer HaslerChapter 11: Tobi Delbruck, Bernabe Linares-BarrancoChapter 12: Shih-Chii LiuChapter 13: Adrian Whatley, Alejandro Linares-Barranco, Shih-Chii LiuChapter 14: Adrian WhatleyChapter 15: Tobi DelbruckChapter 16: Rajit ManoharChapter 17: Rodney Douglas, Kevan Martin

Foreword

The motivation for building neuromorphic systems has roots in engineering and neuroscience.On the engineering side, inspiration from how the brain solves complex problems has led tonew computing algorithms; however, the goal of reverse engineering the brain is a difficultone because the brain is based on a biological technology that was evolved and not designedby human engineers. On the neuroscience side, the goal is to understand brain function,which is still at an early stage owing the extremely heterogeneous and compact nature ofneural circuits. Neuromorphic systems are a bridge between these two ambitious enterprises.The lessons learned from building devices based on neural architectures are providing newengineering capabilities and new biological insights.

Building neuromorphic VLSI chips and perfecting asynchronous event-based communica-tion between them has required a generation of talented engineering scientists. These peoplewere inspired by Carver Mead and his 1989 landmark book on Analog VLSI and NeuralSystems. I was a Wiersma Visiting Professor of Neurobiology at the California Institute ofTechnology in 1987 and attended ‘Carverland’ group meetings. Neuromorphic engineeringwas still in its infancy, but the strengths and weaknesses of the technology were alreadyapparent. The promise of a new massively parallel, low-power, and inexpensive computingarchitecture was balanced by the technical challenges of working with the transistor mismatchand noise that plagued analog VLSI chips. The brain was an existence proof that these prob-lems could be overcome, but it took much longer time than expected to find the practicalsolutions which are discussed in detail in Event-Based Neuromorphic Systems.

At about the same time that neuromorphic systems were introduced, the neural networkrevolution was getting underway based on simulations of simplified models of neurons. Thetwo-volume 1986 book on Parallel Distributed Processing1 had chapters on two new learningalgorithms for multilayer network models, the backpropagation of errors and the Boltzmannmachine. These networks were learned from examples, in contrast to engineered systems thatwere handcrafted. The increase in overall computing power by a factor of a million over thelast 25 years and the large sizes of data sets now available on the Internet have made deeplearning in hierarchies of simple model neurons both powerful and practical, at least whenpower is unlimited. The Neural Information Processing Systems (NIPS) meeting in 2013had 2000 attendees and the applications of machine learning ranged from vision systems toadvertisement recommender systems.

1 Rumelhart DE and McClelland JL. 1986. Parallel Distributed Processing. MIT Press, Cambridge, MA.

xviii Foreword

Systems neuroscience has made progress one neuron at a time since single cortical neuronswere first recorded in 1959. In the last 10 years, new optical techniques have made it possible torecord simultaneously from hundreds of neurons and allowed researchers to both selectivelystimulate and suppress the spikes in subtypes of neurons. Analytic tools are being devel-oped to explore the statistical structure of brain activity in large populations of neurons, andreconstruction of the connectivity of the brain from electron micrographs, aided by machinelearning, is producing intricate wiring diagrams. The goal, which is far from yet realized, is touse these new tools to understand how activity in neural circuits generates behavior.

The next 25 years could be a golden period as these three interacting areas of research inneuromorphic electronic systems, artificial neural networks, and systems neuroscience reachmaturity and fulfill their potential. Each has an important role to play in achieving the ultimategoal, to understand how the properties of neurons and communications systems in brains giverise to our ability to see, hear, plan, decide, and take action. Reaching this goal would give us abetter understanding of who we are and create a new neurotechnology sector of the economywith far-reaching impact on our everyday lives. Event-Based Neuromorphic Systems is anessential resource for neuromorphic electrical engineers pursuing this goal.

Terrence SejnowskiLa Jolla, CaliforniaDecember 15, 2013

Acknowledgments

This book would never taken form without the consistent support of the US National ScienceFoundation (NSF) in funding the Telluride Neuromorphic Cognition Engineering Workshopand the help of the EU FET (Future and Emerging Technologies) program in supportingthe European CapoCaccia Cognitive Neuromorphic Engineering Workshop. These unique,hands-on workshops were the places where many of the ideas developed in this book werefirst discussed and prototyped. For many neuromorphic engineers, these workshops are ahighlight of the year and a kind of working holiday time that other venues, such as IEEEconferences, cannot replace.

The editors and contributors to Event-Based Neuromorphic Systems acknowledge the fol-lowing people for reading and commenting on various chapters in the book: Luca Longinotti,Bjorn Beyer, Michael Pfeiffer, Josep Maria Margarit Taule, Diederik Moeys, Bradley Minch,Sim Bamford, Min-Hao Yang, and Christoph Posch. They acknowledge Srinjoy Mitra for hiscontribution to the on-chip AER circuit chapter, Philipp Hafliger for his contribution to thesynapse chapter, Raphael Berner for his contribution to the retina chapter, and Anton Civit forhis contribution to the hardware infrastructure chapter. They acknowledge Kwabena Boahenfor use of the material in the communications chapter; Diana Kasabov for proofreading andcorrections; and Daniel Fasnacht for setting up the original DokuWiki at the start of the bookproject. The editors also acknowledge the students of the Neuromorphic Engineering I courseat the Institute of Neuroinformatics, University of Zurich and ETH Zurich who gave feedbackon Chapters 7 and 8.

They further acknowledge the Institute of Neuroinformatics, University of Zurich andETH Zurich; the NSF Telluride Neuromorphic Cognition Engineering Workshop; and theCapoCaccia Cognitive Neuromorphic Engineering Workshop.

The figure at the head of the Silicon Cochleas chapter and reproduced on the cover iscourtesy of Eric Fragniere. The figure at the head of the Learning in Neuromorphic Systemschapter is courtesy of Valentin Nagerl and Kevan Martin. The figures at the head of the SiliconNeurons and Silicon Synapses chapters are courtesy of Nuno Miguel Macarico Amorim daCosta, John Anderson, and Kevan Martin.

List of Abbreviationsand Acronyms

1D one dimensional2D two dimensional3D three dimensionalACA analog computing arraysACK acknowledgeA/D analog–digital (converter)ADC analog–digital converterAdEx Adaptive exponential integrate-and-fire modelAE address eventAEB address-event busAER address-event representationAEX AER extension boardAFGA autozeroing floating-gate amplifierAGC automatic gain controlALOHA Not actually an abbreviation, ALOHA refers to a network media access

protocol originally developed at the University of HawaiiANN artificial neural networkANNCORE analog neural network coreAPI Application Programming InterfaceAPS active pixel sensorAQC automatic Q (quality factor) controlARM Acorn RISC MachineASIC application-specific integrated circuitASIMO Advanced Step in Innovative MObility (robot)ASP analog signal processor/processingATA AT Attachment (also PATA: Parallel ATA); an interface standard for connect-

ing mass storage devices (e.g., hard disks) in computersATIS asynchronous time-based image sensorATLUM Automatic Tape-collecting Lathe Ultra-MicrotomeaVLSI Analog very large scale integrationBB bias bufferBGA ball grid array

xxii List of Abbreviations and Acronyms

BJT bipolar junction transistorBM basilar membraneBPF band-pass filterbps bits per secondBps bytes per secondBSI back-side illuminationC4 capacitively coupled current conveyorCAB computational analog blockCADSP cooperative analog–digital signal processingCAVIAR Convolution AER Vision Architecture for Real-timeCCD charge-coupled deviceCCN cooperative and competitive networkCCW counter clockwiseCDS correlated double samplingCIS CMOS image sensorCLBT compatible lateral bipolar transistorCMI current-mirror integratorCMOS complementary metal oxide semiconductorCoP center of pressureCPG central pattern generatorCPLD complex programmable logic deviceCPU central processing unitCSMA carrier sense multiple accessCV coefficient of variationCW clockwiseDAC digital-to-analog converterDAEB domain address -event busDAVIS Dynamic and Active-Pixel Vision SensorDC direct currentDCT discrete cosine transformDDS differential double samplingDFA deterministic finite automatonDIY do it yourselfDMA direct memory accessDNC digital network chipDOF degree(s) of freedomDPE dynamic parameter estimationDPI differential pair integratorDPRAM dual-ported RAMDRAM dynamic random access memoryDSP digital signal processor/processingDVS dynamic vision sensorEEPROM electrically erasable programmable read only memoryEPSC excitatory post-synaptic currentEPSP excitatory post-synaptic potentialESD electrostatic discharge

List of Abbreviations and Acronyms xxiii

ETH Eidgenossische Technische HochschuleEU European UnionFACETS Fast Analog Computing with Emergent Transient StatesFE frame eventsFET field effect transistorFET also Future and Emerging TechnologiesFG floating gateFIFO First-In First-Out (memory)fMRI functional magnetic resonance imagingFPAA field-programmable analog arrayFPGA field-programmable gate arrayFPN fixed pattern noiseFPS frames per secondFSI front side illuminationFSM finite state machineFX2LP A highly integrated USB 2.0 microcontroller from Cypress Semiconductor

CorporationGALS globally asynchronous, locally synchronousGB gigabyte, 230 bytesGbps gigabits per secondGeps giga events per secondGPL general public licenseGPS global positioning systemGPU graphics processing unitGUI graphical user interfaceHCO half-center oscillatorHDL Hardware Description LanguageHEI hot electron injectionHH Hodgkin–HuxleyHiAER hierarchical AERHICANN high input count analog neural networkHMAX Hierarchical Model and XHMM Hidden Markov ModelHTML Hyper-Text Markup LanguageHW hardwareHWR half-wave rectifierhWTA hard winner-take-allI&F integrate-and-fireIC integrated circuitIDC insulation displacement connectorIEEE Institute of Electrical and Electronics EngineersIFAT integrate-and-fire array transceiverIHC inner hair cellIMS intramuscular stimulationIMU inertial or intensity measurement unitINCF International Neuroinformatics Coordinating Facility

xxiv List of Abbreviations and Acronyms

INE Institute of Neuromorphic EngineeringI/O input/outputIP intellectual propertyIPSC inhibitory post-synaptic currentISI inter-spike intervalISMS intraspinal micro stimulationITD interaural time differenceJPEG Joint Photographic Experts GroupKB kilobyte, 210 byteskeps kilo events per secondLAEB local address-event busLFSR linear feedback shift registerLIF leaky integrate-and-fireLLN log-domain LPF neuronLMS least mean squaresLPF low-pass filterLSM liquid-state machineLTD long-term depressionLTI linear time-invariantLTN linear threshold neuronLTP long-term potentiationLTU linear threshold unitLUT look-up tableLVDS low voltage differential signalingMACs multiplyand accumulate operationsMB megabyte, 220 bytesMEMs microelectromechanical systemsMeps mega events per secondMIM metal insulator metal (capacitor)MIPS microprocessor without interlocked pipeline stages (a microprocessor archi-

tecture)MIPS also millions of instructions per secondMLR mesencephalic locomotor regionMMAC millions of multiply accumulate operationsMMC/SD Multimedia card/secure digitalMNC multi-neuron chipMOSFET metal oxide semiconductor field effect transistorMUX multiplex; multiplexerNE neuromorphic engineeringNEF neural engineering frameworknFET n-channel FETNMDA N-Methyl-d-AspartateNoC Network on ChipNSF National Science FoundationOHC outer hair cellOR Octopus Retina

List of Abbreviations and Acronyms xxv

ORISYS orientation systemOS operating systemOTA operational transconductance amplifierPC personal computerPCB printed circuit boardPCI Peripheral Component InterconnectPCIe Peripheral Component Interconnect ExpressPDR phase dependent responsepFET p-channel FETPFM pulse frequency modulationPLD programmable logic devicePRNG pseudo-random number generatorPSC post-synaptic currentPSRR power supply rejection ratioPSTH peri-stimulus time histogramPTAT proportional to absolute temperaturePVT process, voltage, temperaturePWM pulse width modulationPyNN Python for Neural NetworksQ quality factor of filterQE quantum efficiencyQIF quadratic integrate-and-fireQVGA Quarter Video Graphics Array; 320 ×240 pixel arrayRAM random access memoryREQ requestRF radio frequencyRF also receptive fieldRFC Request for Comments(a publication of the Internet Engineering Task Force

and the Internet SocietyRISC reduced instruction set computingRMS root mean squareRNN recurrent neural networkROI region of interestSAC selective attention chipSAER serial AERSAM spatial acuity modulationSATA serial ATA (an interface standard for connecting mass storage devices (e.g.,

hard disks) to computers, designed to replace ATA)SATD sum of absolute timestamp differencesSC spatial contrastS-C switched-capacitorSCX Silicon CortexSDRAM synchronous dynamic random access memorySerDes serializer/deserializerSFA spike-frequency adaptationSiCPG silicon CPG

xxvi List of Abbreviations and Acronyms

SIE serial interface engineSiN silicon neuronSNR signal-to-noise ratioSOS second-order sectionSpiNNaker spiking neural network architectureSRAM static random access memorySS shifted sourceSSI stacked silicon interconnectSTD short-term depressionSTDP spike timing-dependent plasticitySTRF spatiotemporal receptive fieldSW softwaresWTA soft winner-take-allTC temporal contrastTCAM ternary content-addressable memoryTCDS time correlated double samplingTCP Transport Control ProtocolTN TrueNorthTTFS time to first spikeUCSD University of California at San DiegoUDP User Datagram ProtocolUSB universal serial busUSO unit segmental oscillatorsV1 primary visual cortexVGA Video Graphics Array; 640×480 pixel arrayVHDL Verilog Hardware Description LanguageVISe VIsion SensorVLSI very large scale integrationVME VERSAbus Eurocard bus standardVMM vector-matrix multiplication/multiplierWABIAN-2R WAseda BIpedal humANoid No. 2 Refined (robot)WKB Wentzel–Kramers–BrillouinWR_OTA wide-linear range OTAWTA winner-take-allXML Extensible Markup LanguageZMP zero moment point

1Introduction

The effortless ability of animal brains to engage with their world provides a constant challengefor technology. Despite vast progress in digital computer hardware, software, and systemconcepts, it remains true that brains far outperform technological computers across a widespectrum of tasks, particularly when these are considered in the light of power consumption.For example, the honeybee demonstrates remarkable task, navigational, and social intelligencewhile foraging for nectar, and achieves this performance using less than a million neurons,burning less than a milliwatt, using ionic device physics with a bulk mobility that is about10 million times lower than that of electronics. This performance is many orders of magnitudemore task-competent and power-efficient than current neuronal simulations or autonomousrobots. For example, a 2009 ‘cat-scale’ neural simulation on a supercomputer simulated1013 synaptic connections at 700 times slower than real time, while burning about 2 MW(Ananthanarayanan et al. 2009); and the DARPA Grand Challenge robotic cars drove along adensely GPS-defined path, carrying over a kilowatt of sensing and computing power (Thrunet al. 2007).

Although we do not yet grasp completely nature’s principles for generating intelligentbehavior at such low cost, neuroscience has made substantial progress toward describing thecomponents, connection architectures, and computational processes of brains. All of theseare remarkably different from current technology. Processing is distributed across billions ofelementary units, the neurons. Each neuron is wired to thousands of others, receiving inputthrough specialized modifiable connections, the synapses. The neuron collects and transformsthis input via its tree-like dendrites, and distributes its output via tree-like axons. Memoryinstantiated through the synaptic connections between neurons is co-localized with processingthrough their spatial arrangements and analog interactions on the neurons’ input dendritic trees.Synaptic plasticity is wonderfully complex, yet allows animals to retain important memoriesover a lifetime while learning on the time scale of milliseconds. The output axons conveyasynchronous spike events to their many targets via complex arborizations. In the neocortexthe majority of the targets are close to the source neuron, indicating that network processingis strongly localized, with relatively smaller bandwidth devoted to long-range integration.

The various perceptual, cognitive, and behavioral functions of the brain are systematicallyorganized across the space of the brain. Nevertheless at least some aspects of these various

Event-Based Neuromorphic Systems, First Edition.Edited by Shih-Chii Liu, Tobi Delbruck, Giacomo Indiveri, Adrian Whatley, and Rodney Douglas.© 2015 John Wiley & Sons, Ltd. Published 2015 by John Wiley & Sons, Ltd.

2 Event-Based Neuromorphic Systems

processes can be discerned within each specialized area, and their organization suggests acoalition of richly intercommunicating specialists. Overall then, the brain is characterized byvast numbers of processors, with asynchronous message passing on a vast point-to-point wiredcommunication infrastructure. Constraints on the construction and maintenance of this wiringenforce a strategy of local collective specialization, with longer range coordination.

For the past two decades neuromorphic engineers have grappled with the implementationof these principles in integrated circuits and systems. The opportunity of this challenge isthe realization of a technology for computing that combines the organizing principles of thenervous system with the superior charge carrier mobility of electronics. This book providessome insights and many practical details into the ongoing work toward this goal. These resultsbecome ever more important for more mainstream computing, as limits on component densityforce ever more distributed processing models.

The origin of this neuromorphic approach dates from the 1980s, when Carver Mead’sgroup at Caltech came to understand that they would have to emulate the brain’s style ofcommunication if they were to emulate its style of computation. These early developmentscontinued in a handful of laboratories around the world, but more recently there has beenan increase of development both in academic and industrial labs across North America,Europe, and Asia. The relevance of the neuromorphic approach to the broader challengesof computation is now clearly recognized (Hof 2014). Progress in neuromorphic methodshas been facilitated by the strongly cooperative community of neuroscientists and engineersinterested in this field. That cooperation has been promoted by practical workshops such asthe Telluride Neuromorphic Cognition Engineering Workshop in the United States, and theCapoCaccia Cognitive Neuromorphic Engineering Workshop in Europe.

Event-Based Neuromorphic Systems arose from this community’s wish to disseminate state-of-the-art techniques for building neuromorphic electronic systems that sense, communicate,compute, and learn using asynchronous event-based communication. This book complementsthe introductory textbook (Liu et al. 2002) that explained the basic circuit building blocks forneuromorphic engineering systems. Event-Based Neuromorphic Systems now shows how thosebuilding blocks can be used to construct complete systems, with a primary focus on the hot fieldof event-based neuromorphic systems. The systems described in this book include sensors andneuronal processing circuits that implement models of the nervous systems. Communicationbetween the modules is based on the crucial asynchronous event-driven protocol called theaddress-event representation (AER), which transposes the communication of spike events onslow point-to-point axons, into digital communication of small data packets on fast buses (see,for example, Chapter 2). The book as a whole describes the state of the art in the field ofneuromorphic engineering, including the building blocks necessary for constructing completeneuromorphic chips and for solving the technological challenges necessary to make multi-chipscalable systems. A glance at the index shows the wide breadth of topics, for example, next to‘Moore’s law’ is ‘motion artifact’ and next to ‘bistable synapse’ is ‘bootstrapped mirror.’

The book is organized into two parts: Part I (Chapters 2–6) is accessible to readers from awider range of backgrounds. It describes the range of AER communication architectures, AERsensors, and electronic neural models that are being constructed without delving exhaustivelyinto the underlying technological details. Several of these chapters also include a historicaltree that helps relate the architectures and circuits to each other, and that guides readers to theextensive literature. It also includes the largely theoretical Chapter 6 on learning in event-basedsystems.