PowerPoint Presentation

A cortical neural network model of visual motion perception for reactive navigation

Cognitive Anteater Robotics Lab (CARL)University of California, IrvineMichael Beyeler

1

December 1, 20152Projects Overview

Understand the neural correlates of visually guided navigationCapture important network dynamicsLink network dynamics to behavior

Build more powerful artificial vision systems and cognitive agentsBehavior and intelligence of organisms with nervous systems surpass any state of the art, man-made systemIncorporate on-line learningIncorporate predictive capabilitiesDecember 1, 20153Long-Term Goal and MotivationPerceptual variablesPhysical variablesSteering system

Reverse engineering the brainUse brain as a working modelUnderstanding through modelingCurrently: Spiking neural networks on GPUsLow-cost yet high-performanceEnables real-time processing of large-scale neural network modelsNear-future: Dedicated neuromorphic hardwareDecember 1, 20154Approach

Spiking Neural Network (SNN) models describe key aspects of neural function and network dynamics

SNN models are composed of:spiking point-neurons for computationIzhikevich spiking neurons: 20 different neuron typesdynamic synapses for learning and memory storageSynaptic receptors for AMPA, NMDA, and GABAvariable-delay axons for communicationneuromodulatory systems to control action selection and learning

December 1, 20155Spiking Neural Networks

(Izhikevich, 2003, 2004)

December 1, 20156Constructing Functional SNN ModelsCARLsim 3

neural circuits:createtune/optimizeexplore

TheoreticalmodelExperimentaldataFunctionalapplication

TheoreticalneuroscienceReal-timeapplicationsNeuromorphicengineering(Nageswaran et al., 2009; Richert et al., 2011, Beyeler et al, 2015)

stop to think about what people are looking for in a simulator: needs to fit your purposes. neuron model, connectivity, plasticity, visualization, tuning.6

GPU-accelerated spiking neural network simulator

User-friendly, well documented.Runs on Linux, Mac OS X, Windows systems with CUDA SDK.Scalable, extendable.PyNN-like interface.Highly optimized for NVIDIA GPUs.Capable of simulating biological detailed neural models.

Freely available at:http://www.socsci.uci.edu/~jkrichma/CARLsim/

December 1, 20157CARLsim in a Nutshell

7

Provides a PyNN-like user interfaceLets you configure networks, apply input stimuli, and monitor network activityDecember 1, 20158CARLsim API

8

Easily create complex network topographies:December 1, 20159Building Biologically Detailed SNNs

Short-term and long-term plasticity:

Versatile toolbox for the visualization and analysis of neuronal, synaptic, and network information.Generates raster plots, histograms, heat maps, flow fields.Plot, record, load, save.December 1, 201510MATLAB Offline Analysis Toolbox

December 1, 201511capable of quickly and efficiently tuning large-scale SNNsarbitrary fitness functionmulti-objective optimization

utilizes Evolutionary Computations in Java (ECJ)fully integrated with CARLsim

transparent use of a GPUup to 60x speedups compared to single-threaded CPU simulationAutomated Parameter Tuning Interface

Benchmark: 80-20 network with E-STDP (Vogels & Abbott, 2005)CPU: Intel Core i7 CPU 920 @ 2.67 GHzGPU: NVIDIA GTX 780 (3 GB of GDDR5, 2304 CUDA cores)

December 1, 201512Performance Benchmarks

Often informed by visual motion cues

Typically requires quick and precise timing

Involves large-scale brain processingReactive Navigation Through a Cluttered Scene

December 1, 201514Primary visual cortex (V1)tuned to simple attributes of shape, motion, color, texture, depth

Middle temporal (MT) areatuned to coherent local motion (retinal flow)

Posterior parietal cortex (PPC)Polysensory areas (VIP, MST, 7a, etc.)Tuned to global, complex motionSelf-motion and object motionSpatial reference framesPath integration (?)Visual Motion Pathway

(Britten, 2008)

December 1, 201515Problem: Local-velocity sample is different from object velocityGoal: Disambiguate local-velocity samples and integrate them into an accurate estimate of the global (object) velocity

Intersection-of-constraints (IOC):Each local velocity sample constrains the global object velocityFind object velocity by integrating local samplesThere is evidence that MT firing rates represent the velocity of moving objects using IOC

Aperture Problem(Bradley & Goyal, 2008)

Motion is an orientation in space-timeV1 simple cells: space-time oriented receptive fields

Spatiotemporal energy model:Linear filtering / motion energy / opponent energyAdelson & Bergen (1985), Simoncelli & Heeger (1998)December 1, 201516Primary Visual Cortex (V1)

(Simoncelli & Heeger, 1998)(Bradley & Goyal, 2008)

December 1, 201517Middle Temporal Area (MT)ExcitationFeedforward and localSpecific InhibitionCross-direction inhibitionUnspecific InhibitionDivisive normalization

MT CDSMT PDS(Beyeler et al., 2015)Based on Rust et al. (2006)

Direction Tuning

Component-direction-selectivePattern-direction-selective

Direction Tuning (2)

V1MT CDSMT PDS(Beyeler et al., 2014)(CDS: component-direction-selective, PDS: pattern-direction-selective)December 1, 201519

Three speed tuning classes observed in MT (Rodman & Albright, 1987)band-pass or speed-tunedlow-passhigh-pass

Stimulus: single bar drifting over entire visual field to the right (preferred) or left (anti-preferred) at different speeds12/1/2015

20Speed Tuning

12/1/2015

21Computational Performance

CPU: single-core Intel Xeon X5675 @ 3.07 GHz (24 GB of RAM)GPU: single NVIDIA Tesla M2090 512 cores @ 1.3 GHz (6 GB of G-DDR5)(Beyeler et al., 2014)

December 1, 201522Android Based Robotics (ABR)

Introducing Le Carl: The French Robot

December 1, 201523Visually Guided Navigation

Behavioral paradigm:Analogous to Fajen & Warren (2003)Navigate towards a distant, visually salient targetAvoid obstacles along the wayOnly sensory input: VisionVisual input:320x240px snapshots~ 20 frames / secondPosition tracking:overhead cameraperspective transform (OpenCV)

December 1, 201524Setup and Network Architecture(Beyeler et al., 2015)40,000 neurons1.7 million synapses

December 1, 201525Inspired by the Balance Strategy:Simple control law for obstacle avoidanceUsed by honeybees for visual control of flight (Srinivasan & Zhang, 1997)Humans might make use of similar strategy (Kountouriotis et al., 2013)

Important difference:Obstacles are located via motion discontinuitiesPosterior Parietal Cortex (PPC)

December 1, 201526ABR Cockpit View

December 1, 201527Simulated Neuronal Responses(Beyeler et al., 2015)

December 1, 201528Behavioral Results

Comparison to Psychophysical Data

December 1, 201529

CARLsim highlights:Runs on Linux, Mac OS X, Windows.Highly optimized for NVIDIA GPUs: CUDA 5.0, device capability 2.0User-friendly, scalable, extendableDownload: www.socsci.uci.edu/~jkrichma/CARLsim

Presented a large-scale spiking neural network thatis biologically inspiredsolves the aperture problem via cortical mechanismsis integrated with a real-time, real-world robotics platformAndroid Based Robotics combined with CARLsim is the first step toward a complete robot navigation system.December 1, 201530Conclusions

December 1, 201531Other Projects

(Beyeler et al., 2013)

Add MNIST, add backup slides31

December 1, 201532Team CARL (2015)Front row Emily Rounds, Steve Doubleday, Jeff Krichmar, Ting-Shuo Chou, Nikil DuttBack row Alexis Craig, Alex Wang, Michael Beyeler, Feng Rong, Timo Oess, Saideep GuptaSupported by the National Science Foundation and Qualcomm Technologies Incorporated.

Update picture32

December 1, 201533

12/1/2015

34Izhikevich Spiking Neurons

(Izhikevich, 2003)

12/1/2015

35Spiking Neural Network Models

(Izhikevich, 2004)

December 1, 201536Motion Energy Model

(Adelson & Bergen, 1985;Simoncelli & Heeger, 1998)

December 1, 201537Solving the Aperture Problem

(Bradley & Goyal, 2008)

Vector average (VA)Advantage: simpleVA is a poor predictor of object direction and speedVA speed almost always less than object speedVA tends to depend on object shapeAssumes symmetrical distribution of local-velocity samplesDecember 1, 201538Solving the Aperture Problem

(Bradley & Goyal, 2008)

Lateral intraparietal (LIP) areaintegrates sensory evidence over time until a threshold is reachedopponent inhibitionused to make perceptual decisions about direction of motion

Race modelFirst pool to reach the threshold wins the race (Shadlen & Newsome, 2001; Smith & Ratcliff, 2004)

Reaction TimeTime it takes the winning pool to reach criterion12/1/2015

39Spiking Layer of LIP Decision Neurons

LIPMT PDSInput

Total of 80 trials (fixed motion strength, ten repetitions per motion direction)12/1/2015

40Perceptual Decision-Making

SNN model:71,026 neurons133 million synapsesLearning rule:Ca-based STDP (Brader et al., 2007)Categorization:Accumulator model (Smith & Ratcliff, 2004)

Achieved 92% accuracyReproduced RT distributions from human psychophysicsDecember 1, 201541Handwritten Digit Classification