information-theoretic stimulus design for neuroscience...
TRANSCRIPT
![Page 1: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/1.jpg)
Information-theoretic stimulus design
for neurophysiology & psychophysics
Christopher DiMattina, PhD
Assistant Professor of Psychology
Florida Gulf Coast University
![Page 2: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/2.jpg)
Optimal experimental design
Part 1
2 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 3: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/3.jpg)
Consider a simple problem
• Estimate the slope of a line through the origin from noisy
input-output data {(xi, yi)}i = 1:N
yi = a∙xi + (noise)i
x in [-2 , 2]
3 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 4: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/4.jpg)
System identification
a∙x x + y
noise
system
observations inputs
4 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 5: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/5.jpg)
Standard approach
• Choose N inputs xi uniformly from [-2, 2] , observe yi
• Obtain maximum a posteriori (MAP) estimate for slope
parameter a
5 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 6: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/6.jpg)
Accuracy & time tradeoff
• More data confidence intervals get tighter
• Experiment takes longer
6 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 7: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/7.jpg)
Efficient stimulus selection
• How can we efficiently choose our inputs x to get the most
accurate estimates for a fixed number of observations?
• This question can be re-phrased using information theory
high accuracy = low posterior entropy
Claude Shannon
7 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 8: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/8.jpg)
Solution
• For linear regression with Gaussian noise, posterior is a Gaussian with µa = y/x, σ2
a = (σn/x) 2
• entropy = C + ln (σa) = C + ln |σn/x|
Posterior entropy is minimized
at the endpoints
singularity at x = 0
8 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 9: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/9.jpg)
Put all stimuli at the endpoints
less entropy
more entropy
9 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 10: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/10.jpg)
Optimal experimental design
• This simple example shows how optimal experimental
design (OED) can greatly reduce the number of stimuli
needed to estimate model parameters
• How can this be applied in sensory neuroscience?
10 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 11: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/11.jpg)
Sensory neuroscience
Part 2
11 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 12: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/12.jpg)
Sensory neuroscience
• Psychophysics
• Neurophysiology (single-unit, fMRI)
F(x,θ)
x inputs
y observations
goal: estimate θ
Reviews: Wu, David & Gallant (2006), Sharpee (2014)
12 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 13: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/13.jpg)
Psychometric functions
• F(x, θ) relates stimulus parameter(s) to probability correct
• Model parameters θ are slope and threshold
[from Palamedes website: http://www.palamedestoolbox.org/ ]
13 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 14: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/14.jpg)
Tuning curves
[from David Heeger’s website: http://www.cns.nyu.edu/~david/ ]
• F(x, θ) relates stimulus parameter(s) to neural response
• Model parameters θ are peak and tuning width
14 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 15: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/15.jpg)
Neural models
• F(x, θ) relates stimulus parameters to neural responses
• Model parameters θ are weights, thresholds, etc…
(Simoncelli et al., 2004)
(Riesenhuber & Poggio , 2000)
15 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 16: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/16.jpg)
Non-adaptive stimulus generation
• Traditionally investigators attempt to identify models
F(x, θ) using fixed stimulus ensembles
• This approach is non-adaptive (open-loop)
Simoncelli et al. (2004)
16 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 17: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/17.jpg)
Active data collection
• Recently, in sensory neuroscience there has been a
great interest in closed-loop data collection
(DiMattina & Zhang, 2013)
Reviews: Benda et al. (2007), Paninski et al. (2007), DiMattina & Zhang (2013)
17 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 18: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/18.jpg)
Firing rate optimization
(Yamane et al., 2008)
Perhaps the most popular application of adaptive
stimulus design is firing rate optimization
18 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 19: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/19.jpg)
Model estimation & comparison
• Active data collection can help to more efficiently estimate
and compare models
(DiMattina, 2009)
19 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 20: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/20.jpg)
Old news in Statistics & Machine Learning
• Lindley (1956) first showed that information theory could
be applied to compare experimental designs
• MacKay (1992) showed that training of neural networks
could be speeded up with stimuli maximizing mutual info
20 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 21: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/21.jpg)
Old news in Psychophysics
• Staircase method (Cornsweet, 1962)
• PSI Method – Adaptive information-theoretic approach
(Kontsevich & Tyler, 1999) – 230 citations and counting!
21 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 22: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/22.jpg)
News in Neuroscience
• Lewi, Butera & Paninski (2009) developed a fast
implementation of information-theoretic stimulus design for
the Generalized Linear Model (GLM)
• Used Laplace approximation of the posterior density
Lewi et al. (2009)
22 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 23: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/23.jpg)
Generalized Linear Model
• Estimate receptive fields with fewer trials
Lewi et al. (2009)
23 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 24: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/24.jpg)
Limitations of the GLM
• GLM is essentially a single-layer perceptron
• Cannot model many nonlinear neurons like those found in
the auditory or higher visual systems
Frank Rosenblatt
24 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 25: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/25.jpg)
Nonlinear auditory neurons
• GLM cannot model non-monotonic rate-level tuning seen
in auditory neurons
• Cannot model complex non-linear properties like
harmonic combination sensitivity
Kadia & Wang (2003)
25 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 26: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/26.jpg)
Nonlinear visual neurons
• Neurons in IT can be
modeled as combining
inputs from subunits
tuned to shape features
• One does not know the
subunit parameters – a
“hidden unit” problem
Brincat & Connor (2004)
26 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 27: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/27.jpg)
OED for nonlinear models
Part 3
27 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 28: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/28.jpg)
Work at Johns Hopkins
• Goal was to develop methods for on-line estimation and
comparison of generic nonlinear neural models
28 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 29: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/29.jpg)
Neural networks
• A reasonable starting point because of their universal
approximation properties and large body of work
• Method is applicable to arbitrary firing rate models F(x,θ)
29 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 30: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/30.jpg)
Representing the posterior
• Evolving posterior pn(θ) is a Gaussian mixture
• After each observation, we update each peak recursively using Extended Kalman Filter (EKF) equations (Alspach & Sorenson, 1972)
30 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 31: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/31.jpg)
Choosing the next stimulus
• We chose the peak with the most weight and found the best stimulus for reducing the entropy of that Gaussian
• Quite often most of probability mass was on only a few bumps, so this approach is reasonable
DiMattina & Zhang (2011)
31 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 32: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/32.jpg)
Not just a good idea
• For nonlinear models with hidden units, it may not be
possible to recover the true model parameters with white
noise stimuli (DiMattina & Zhang, 2010, 2011)
DiMattina & Zhang (2011)
32 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 33: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/33.jpg)
Estimating network structure
• Nonlinear network model (nearly 300 parameters total)
• Want to recover network structure using input-output data
DiMattina & Zhang (2011)
33 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 34: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/34.jpg)
Estimating network structure
• Much more effective at recovering input filters and
network structure than IID (white-noise) stimuli
DiMattina & Zhang (2011)
34 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 35: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/35.jpg)
Multiple models
• The correct nonlinear model is often unknown
• Might want to estimate several models and generate
critical stimuli to compare models
DiMattina & Zhang (2011)
35 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 36: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/36.jpg)
Two phase experiment
DiMattina & Zhang (2011)
36 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 37: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/37.jpg)
Comparison criterion
• Bayes Information Criterion (Swartz, 1978; Bishop, 2006)
ln 𝑃(𝐷) = ln 𝑝 𝐷 𝜃𝑀𝐴𝑃 −𝑀
2ln𝑁
Other good criteria: Minimize model space entropy (Cavagnaro et al. 2010)
rewards good fit penalizes model complexity
37 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 38: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/38.jpg)
Optimal stimuli for model comparison
• Both models fit data
about equally well
• Stimuli optimized for
increasing the
expected BIC
increment did a good
job of discriminating
the models
• IID stimuli and stimuli
optimized for model
estimation did poorly
DiMattina & Zhang (2011)
38 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 39: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/39.jpg)
Modeling nonlinear neurons
Part 4
39 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 40: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/40.jpg)
Collaborative effort
• Wanted to test this approach in experiments
• Collaborated with Eric Young, William Tam and Eyal Dekel
William Tam Chris DiMattina
Eric Young Kechen Zhang
Eyal Dekel
40 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 41: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/41.jpg)
Test bed
• Inferior colliculus of the awake marmoset monkey
41 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 42: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/42.jpg)
Stimuli
• Wide-band, steady-state acoustic spectra
Yu & Young (2000)
(Wolfe et al., 2012)
42 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 43: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/43.jpg)
Underlying circuitry
• There are theories of the underlying functional circuitry of
its main input, the Dorsal Cochlear Nucleus (Young, 1998)
• Can use a model of this circuitry as a candidate model for
the neurons in the IC, which have similar properties
auditory nerve inputs
43 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 44: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/44.jpg)
Experimental set-up
Tam et al. (2011)
44 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 45: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/45.jpg)
Facts
• Searched over a pre-computed set of stimuli (~ 6000)
• Auditory nerve model front-end (Bruce et al. 2003)
• Took only about 300 stimuli (~ 5 minutes) to estimate
model parameters
45 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 46: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/46.jpg)
Could characterize nonlinear neurons
Tam et al. (2011)
46 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 47: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/47.jpg)
More neurons
Tam et al. (2011)
47 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 48: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/48.jpg)
Predicting effective, ineffective stimuli
Tam et al. (2011)
48 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 49: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/49.jpg)
Comparing models
Tam et al. (2011)
49 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 50: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/50.jpg)
Largest and smallest difference
Tam et al. (2011)
50 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 51: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/51.jpg)
Cumulative difference
Tam et al. (2011)
51 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 52: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/52.jpg)
Conclusions
• Demonstrates that optimally designed stimuli may be effectively used in neurophysiology experiments to estimate models
• Very helpful for comparing nonlinear models
• Hope to extend implementation to more complex and generic receptive field models for vision science
52 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 53: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/53.jpg)
High-dimensional
psychophysics
Part 5
53 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 54: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/54.jpg)
Standard PSI
• Represent posterior density using a 2-D grid of particles,
search a 1-D grid of stimuli to minimize expected entropy
DiMattina & Zhang (2014), in preparation
54 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 55: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/55.jpg)
Breaks down in higher dimensions
55 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 56: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/56.jpg)
High-dimensional questions
• Many people in psychophysics are interested how observers combine multiple cues (Knill & Saunders 2003)
• How do we combine multiple cues to detect edges (DiMattina, Fox & Lewicki 2012)?
• We need methods for efficiently estimating high-dimensional psychometric models
56 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 57: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/57.jpg)
Faster implementation
• Applied to 2-D and 3-D examples of nonlinear cue combination
• All three implementations are tractable + give same results as Grid-Psi
57 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 58: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/58.jpg)
Future goals
Part 6
58 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 59: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/59.jpg)
Future work
• Higher-dimensional models with multiple subunits
• For instance, complex cells integrate inputs from many
Gabor-like subunits (Chen et al. 2007)
(Chen et al. 2007)
59 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 60: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/60.jpg)
Psychophysical studies
• How do subjects combine information from multiple neurons
responding to a stimulus to make perceptual decisions?
(DiMattina, Fox & Lewicki 2012)
60 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 61: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/61.jpg)
Software toolbox
• MATLAB toolbox containing various methods for optimal
experimental design for psychophysics and neuroscience
61 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 62: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/62.jpg)
Book
• As adaptive stimulus generation methods are becoming more prevalent in brain and cognitive sciences, it may be time for a multi-method, multi-disciplinary edited volume
62 OCNS 2014 Info Theory Workshop - Quebec City, CA
![Page 63: Information-theoretic stimulus design for neuroscience ...itech.fgcu.edu/faculty/cdimattina/papers/Info... · Modeling nonlinear neurons Part 4 OCNS 2014 Info Theory Workshop - Quebec](https://reader034.vdocuments.site/reader034/viewer/2022051913/6003d7f3e1bc867c913514a1/html5/thumbnails/63.jpg)
Thank You
• OCNS & Information theory workshop
• Alex Dimitrov
• Colleagues at Johns Hopkins
• Kechen Zhang, Eric Young, Eyal Dekel, William Tam
• Florida Gulf Coast University
63 OCNS 2014 Info Theory Workshop - Quebec City, CA