Download - Lecture 7
Lecture 7
The Eye and Neuromorphic Vision
1
Outline
• The eye and the retina• Artificial Vs biological vision systems• Fundamentals of photo receptors • Read-out strategies• Examples of neuromorphic vision systems• Further processing
2
• Eyeball Cross Section and Retina
3
Retina Cells
4
Schematized Retinal Cells
5
Outline
• The eye and the retina• Artificial Vs biological vision systems• Fundamentals of photo receptors • Read-out strategies• Examples of neuromorphic vision systems• Further processing
6
Frame-based Sensors Vs Biological Vision
Artificial Vision Biological Systems
Frame-based Continuous Detection
Global AGC Local AGC
Redundancy Low redundancy
High power consumption Low power consumption
7
Notion of Frame
Tframe
TintegrationTime
x
y
Tintegration<Tframe
8
Dynamic Range
Rüedi et al, JSSC 2003 ”A 128x128 Pixel 120dB Dynamic-Range Vision-Sensor Chip for Image Contrast and Orientation Extraction”
Bad choice of the integration time
The human eye has a dynamic range higher than 10 decades
9
Outline
• The eye and the retina• Artificial Vs biological vision systems• Fundamentals of photo receptors • Read-out strategies• Examples of neuromorphic vision systems• Further processing
10
Depletion Region and Phototransduction
11
Sili
con
Electromagnetic Spectrum
12
Some Possible Photodetectors in CMOS Technology
13
A. Moini. ‘Vision Chips’. Kluwer Academic Publishers, 2000.
Photogates
14
Logarithmic Photoreceptors
(a) (b)
15
Photoreceptors with Negative Feedback
(a)
(b)16
Classic Active Pixel Sensor (APS)
17
Outline
• The eye and the retina• Artificial vision Vs biological systems• Fundamentals of photo receptors • Read-out strategies• Examples of neuromorphic vision systems• Further processing
18
Read-out Strategies:
Addressing/Scanning Charge Coupled Devices (CCD) Address Event Representation (AER)
19
Addressing/Scanning
Token System20
Charge Coupled Devices (CCD)
21
Address Event Representation (AER) (I)
22
Address Event Representation (AER) (II)
23
Outline
• The eye and the retina• Artificial Vs vision biological systems• Fundamentals of photo receptors • Read-out strategies• Examples of neuromorphic vision systems• Further processing
24
Delbrück’s Adaptative Photo Cell (I)
25
Delbrück’s Adaptative Photo Cell (II)
Non-linear element26
Mahowald’s Pixel
27
Retina Pixel Boahen
28
Dynamic Vision Sensor (DVS)
P. Lichtsteiner, C. Posch, and T. Delbrück, “A 128 128 120 dB 15 µs latency asynchronous temporal contrast vision sensor,” IEEE J. Solid-State Circuits, vol. 43, no. 2, pp. 566–576, Feb. 2008.
29
Common Source Amplifier
30
Can used as comparator
Examples of Biomorphic Vision Sensors
Spatio-Temporal Contrast Detector:http://folk.uio.no/juanle/Projects/Spatial_contrast_retina.html
Dynamic Vision Sensor:http://folk.uio.no/juanle/Projects/DVS.html
31
Outline
• The eye and the retina• Artificial vision Vs biological systems• Fundamentals of photo receptors • Read-out strategies• Examples of neuromorphic vision systems• Further processing
32
Further Image Processing
• Motion Detection• Features extraction
33
Token Based Motion Processing: Reichardt Detector
34
Intensity Based
35
Features Extraction
36
’Difference of Gaussians’ Kernel (I)
37
’Difference of Gaussians’ Kernel (II)
38
Result of Edge Detection
39
45 Degrees Edge Kernel (I)
40
45 Degrees Edge Kernel (II)
41