Download - Lecture 7
![Page 1: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/1.jpg)
Lecture 7
The Eye and Neuromorphic Vision
1
![Page 2: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/2.jpg)
Outline
• The eye and the retina• Artificial Vs biological vision systems• Fundamentals of photo receptors • Read-out strategies• Examples of neuromorphic vision systems• Further processing
2
![Page 3: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/3.jpg)
• Eyeball Cross Section and Retina
3
![Page 4: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/4.jpg)
Retina Cells
4
![Page 5: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/5.jpg)
Schematized Retinal Cells
5
![Page 6: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/6.jpg)
Outline
• The eye and the retina• Artificial Vs biological vision systems• Fundamentals of photo receptors • Read-out strategies• Examples of neuromorphic vision systems• Further processing
6
![Page 7: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/7.jpg)
Frame-based Sensors Vs Biological Vision
Artificial Vision Biological Systems
Frame-based Continuous Detection
Global AGC Local AGC
Redundancy Low redundancy
High power consumption Low power consumption
7
![Page 8: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/8.jpg)
Notion of Frame
Tframe
TintegrationTime
x
y
Tintegration<Tframe
8
![Page 9: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/9.jpg)
Dynamic Range
Rüedi et al, JSSC 2003 ”A 128x128 Pixel 120dB Dynamic-Range Vision-Sensor Chip for Image Contrast and Orientation Extraction”
Bad choice of the integration time
The human eye has a dynamic range higher than 10 decades
9
![Page 10: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/10.jpg)
Outline
• The eye and the retina• Artificial Vs biological vision systems• Fundamentals of photo receptors • Read-out strategies• Examples of neuromorphic vision systems• Further processing
10
![Page 11: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/11.jpg)
Depletion Region and Phototransduction
11
![Page 12: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/12.jpg)
Sili
con
Electromagnetic Spectrum
12
![Page 13: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/13.jpg)
Some Possible Photodetectors in CMOS Technology
13
A. Moini. ‘Vision Chips’. Kluwer Academic Publishers, 2000.
![Page 14: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/14.jpg)
Photogates
14
![Page 15: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/15.jpg)
Logarithmic Photoreceptors
(a) (b)
15
![Page 16: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/16.jpg)
Photoreceptors with Negative Feedback
(a)
(b)16
![Page 17: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/17.jpg)
Classic Active Pixel Sensor (APS)
17
![Page 18: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/18.jpg)
Outline
• The eye and the retina• Artificial vision Vs biological systems• Fundamentals of photo receptors • Read-out strategies• Examples of neuromorphic vision systems• Further processing
18
![Page 19: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/19.jpg)
Read-out Strategies:
Addressing/Scanning Charge Coupled Devices (CCD) Address Event Representation (AER)
19
![Page 20: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/20.jpg)
Addressing/Scanning
Token System20
![Page 21: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/21.jpg)
Charge Coupled Devices (CCD)
21
![Page 22: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/22.jpg)
Address Event Representation (AER) (I)
22
![Page 23: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/23.jpg)
Address Event Representation (AER) (II)
23
![Page 24: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/24.jpg)
Outline
• The eye and the retina• Artificial Vs vision biological systems• Fundamentals of photo receptors • Read-out strategies• Examples of neuromorphic vision systems• Further processing
24
![Page 25: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/25.jpg)
Delbrück’s Adaptative Photo Cell (I)
25
![Page 26: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/26.jpg)
Delbrück’s Adaptative Photo Cell (II)
Non-linear element26
![Page 27: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/27.jpg)
Mahowald’s Pixel
27
![Page 28: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/28.jpg)
Retina Pixel Boahen
28
![Page 29: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/29.jpg)
Dynamic Vision Sensor (DVS)
P. Lichtsteiner, C. Posch, and T. Delbrück, “A 128 128 120 dB 15 µs latency asynchronous temporal contrast vision sensor,” IEEE J. Solid-State Circuits, vol. 43, no. 2, pp. 566–576, Feb. 2008.
29
![Page 30: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/30.jpg)
Common Source Amplifier
30
Can used as comparator
![Page 31: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/31.jpg)
Examples of Biomorphic Vision Sensors
Spatio-Temporal Contrast Detector:http://folk.uio.no/juanle/Projects/Spatial_contrast_retina.html
Dynamic Vision Sensor:http://folk.uio.no/juanle/Projects/DVS.html
31
![Page 32: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/32.jpg)
Outline
• The eye and the retina• Artificial vision Vs biological systems• Fundamentals of photo receptors • Read-out strategies• Examples of neuromorphic vision systems• Further processing
32
![Page 33: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/33.jpg)
Further Image Processing
• Motion Detection• Features extraction
33
![Page 34: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/34.jpg)
Token Based Motion Processing: Reichardt Detector
34
![Page 35: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/35.jpg)
Intensity Based
35
![Page 36: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/36.jpg)
Features Extraction
36
![Page 37: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/37.jpg)
’Difference of Gaussians’ Kernel (I)
37
![Page 38: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/38.jpg)
’Difference of Gaussians’ Kernel (II)
38
![Page 39: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/39.jpg)
Result of Edge Detection
39
![Page 40: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/40.jpg)
45 Degrees Edge Kernel (I)
40
![Page 41: Lecture 7](https://reader035.vdocuments.site/reader035/viewer/2022081604/56816384550346895dd46974/html5/thumbnails/41.jpg)
45 Degrees Edge Kernel (II)
41