eric dubois visual communications from broadcast tv to ...edubois/glinski/glinski...eric dubois,...
TRANSCRIPT
![Page 1: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/1.jpg)
Visual Communications from
Broadcast TV to Telepresence
Eric Dubois
![Page 2: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/2.jpg)
Electronic Visual Communications
• From
Eric Dubois, June 26, 2013
![Page 3: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/3.jpg)
Electronic Visual Communications
• To
Eric Dubois, June 26, 2013
![Page 4: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/4.jpg)
My trajectory
• 1972-1974 M.Eng. Electrical Engineering, McGill University
• 1974-1977 Ph.D. Electrical Engineering, University of Toronto
• 1977-1998 Professeur, Institut national de la recherche scientifique (INRS), INRS-Télécommunications.
• 1998-2013 Professor, School of Electrical Engineering and Computer Science, University of Ottawa.
Eric Dubois, June 26, 2013
![Page 5: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/5.jpg)
What have I worked on?
Eric Dubois, June 26, 2013
![Page 6: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/6.jpg)
What have I worked on?
Eric Dubois, June 26, 2013
![Page 7: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/7.jpg)
Who have I worked with?
Eric Dubois, June 26, 2013
![Page 8: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/8.jpg)
Some topics I have worked on
• Two-dimensional filtering, multidimensional filtering
• Video sampling theory
• Spatiotemporal NTSC video analysis
• Motion estimation, motion compensated processing
• Anaglyph, stereoscopic imaging
• Color spaces
• Color filter arrays, demosaicking for digital cameras
• Omnidirectional imaging, panoramas
Eric Dubois, June 26, 2013
![Page 9: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/9.jpg)
My Introduction to Two-Dimensional Signal Processing
Eric Dubois, June 26, 2013
From J.V. Hu and L.R. Rabiner, “Design techniques for two-dimensional filters,”
IEEE Trans. Audio and Electroacoustics, vol. AU-20, June 1972.
![Page 10: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/10.jpg)
Society of Motion Picture and Television Engineers – Best paper award 1988
Eric Dubois, June 26, 2013
![Page 11: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/11.jpg)
Still doing that
Eric Dubois, June 26, 2013
From G. Jeon and E. Dubois, “Demosaicking of noisy Bayer-sampled color images
with least-squares luma-chroma demultiplexing and noise level estimation,” IEEE
Trans. Image Process., vol. 22, Jan. 2013
![Page 12: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/12.jpg)
Fourier Analysis 101
Eric Dubois, June 26, 2013
Jean-Baptiste Joseph Fourier (21 March 1768 – 16 May 1830)
![Page 13: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/13.jpg)
Sine waves and Frequency
• A signal is an entity that carries information such as speech, audio, images, video and so on.
• Signal processing is a major branch of electrical engineering and computer science.
• Fourier found that any signal can be created by adding up sine waves of different frequencies and amplitudes.
• The frequency of a sine wave is the number of oscillations per second, measured in Hertz (Hz).
• Fourier analysis refers to finding out the frequencies and amplitudes needed to construct a particular signal.
• Fourier synthesis refers to actually constructing a signal from these sine waves.
Eric Dubois, June 26, 2013
![Page 14: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/14.jpg)
Low frequency sine wave (70 Hz)
Eric Dubois, June 26, 2013
0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
time (seconds)
am
plit
ude
0 500 1000 1500 2000 2500 30000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
frequency (Hz)
am
plit
ude
![Page 15: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/15.jpg)
Medium frequency sine wave (500 Hz)
Eric Dubois, June 26, 2013
0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
time (seconds)
am
plit
ude
0 500 1000 1500 2000 2500 30000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
frequency (Hz)
am
plit
ude
![Page 16: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/16.jpg)
High frequency sine wave (2000 Hz)
Eric Dubois, June 26, 2013
0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
time (seconds)
am
plit
ude
0 500 1000 1500 2000 2500 30000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
frequency (Hz)
am
plit
ude
![Page 17: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/17.jpg)
More complex tone (C chord)
Eric Dubois, June 26, 2013
0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
time (seconds)
am
plit
ude
0 50 100 150 200 250 300 350 400 450 5000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
frequency (Hz)
am
plit
ude
![Page 18: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/18.jpg)
Musical phrase (name that tune)
Eric Dubois, June 26, 2013
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
time (seconds)
am
plit
ude
0 2 4 6 8 10 12 14 16 18-80
-60
-40
-20
0
20
40
60
80
Frequency in kHz
dB
![Page 19: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/19.jpg)
What is a filter?
• A filter is a device that typically removes some of the frequencies in a signal, and lets others pass through.
• For example, a filter that lets the low frequencies through and removes the high frequencies is called a low-pass filter.
• Other examples are high-pass filters, band-pass filters and band-stop filters.
• The filter is described by its frequency response.
• Let’s see some examples for the audio signals.
Eric Dubois, June 26, 2013
![Page 20: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/20.jpg)
More complex tone (C chord)
Eric Dubois, June 26, 2013
0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
time (seconds)
am
plit
ude
0 50 100 150 200 250 300 350 400 450 5000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
frequency (Hz)
am
plit
ude
![Page 21: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/21.jpg)
Low-pass filter
Eric Dubois, June 26, 2013
0 50 100 150 200 250 300 350 400 450 5000
0.2
0.4
0.6
0.8
1
frequency (Hz)
magnitude r
esponse
0 50 100 150 200 250 300 350 400 450 5000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
frequency (Hz)
am
plit
ude
Frequency response Filter output
![Page 22: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/22.jpg)
High-pass filter
Eric Dubois, June 26, 2013
Frequency response Filter output
0 50 100 150 200 250 300 350 400 450 5000
0.2
0.4
0.6
0.8
1
frequency (Hz)
magnitude r
esponse
0 50 100 150 200 250 300 350 400 450 5000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
frequency (Hz)
am
plit
ude
![Page 23: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/23.jpg)
Band-pass filter
Eric Dubois, June 26, 2013
Frequency response Filter output
0 50 100 150 200 250 300 350 400 450 5000
0.2
0.4
0.6
0.8
1
frequency (Hz)
magnitude r
esponse
0 50 100 150 200 250 300 350 400 450 5000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
frequency (Hz)
am
plit
ude
![Page 24: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/24.jpg)
Band-stop filter
Eric Dubois, June 26, 2013
Frequency response Filter output
0 50 100 150 200 250 300 350 400 450 5000
0.2
0.4
0.6
0.8
1
frequency (Hz)
magnitude r
esponse
0 50 100 150 200 250 300 350 400 450 5000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
frequency (Hz)
am
plit
ude
![Page 25: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/25.jpg)
Spatial frequencies
Eric Dubois, June 26, 2013
horizontal
vertical
low medium high
![Page 26: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/26.jpg)
Spatial frequencies
Eric Dubois, June 26, 2013
diagonal
arbitrary
low medium high
![Page 27: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/27.jpg)
Frequency Spectrum
Eric Dubois, June 26, 2013
Horizontal frequency
Ver
tica
l fr
equen
cy
![Page 28: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/28.jpg)
First Application: NTSC decoding
• NTSC: National Television System Committee
• Extended black and white to color in the 1950s in a compatible fashion.
• The video signal was viewed as a one-dimensional signal.
• The main problem was separating the color from the black and white information.
• Using a two –dimensional representation, better solutions could be found.
• Three-dimensional representations were even better.
Eric Dubois, June 26, 2013
![Page 29: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/29.jpg)
NTSC One-dimensional spectrum
Eric Dubois, June 26, 2013
From the Television Engineering Handbook, 1986.
![Page 30: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/30.jpg)
Two-dimensional NTSC spectrum
Eric Dubois, June 26, 2013
From E. Dubois, M.S. Sabri, and J.-Y. Ouellet, “Three-dimensional spectrum and
processing of digital NTSC color signals,” SMPTE J., vol. 91, pp. 372-378, April 1982.
![Page 31: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/31.jpg)
Measured Two-dimensional NTSC spectrum
Eric Dubois, June 26, 2013
From E. Dubois and W.F. Schreiber, “Improvements to NTSC by multidimensional
filtering,”SMPTE J., vol. 97, pp. 446-463, June 1988.
![Page 32: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/32.jpg)
Chrominance band-pass filter
Eric Dubois, June 26, 2013
From E. Dubois and W.F. Schreiber, “Improvements to NTSC by multidimensional
filtering,”SMPTE J., vol. 97, pp. 446-463, June 1988.
![Page 33: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/33.jpg)
Two-dimensional NTSC prefilters
Eric Dubois, June 26, 2013
From E. Dubois and W.F. Schreiber, “Improvements to NTSC by multidimensional
filtering,”SMPTE J., vol. 97, pp. 446-463, June 1988.
![Page 34: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/34.jpg)
Test image
Eric Dubois, June 26, 2013
![Page 35: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/35.jpg)
Conventional NTSC
Eric Dubois, June 26, 2013
![Page 36: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/36.jpg)
Enhanced NTSC
Eric Dubois, June 26, 2013
![Page 37: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/37.jpg)
Three-dimensional NTSC spectrum
Eric Dubois, June 26, 2013
From E. Dubois, M.S. Sabri, and J.-Y. Ouellet, “Three-dimensional spectrum and
processing of digital NTSC color signals,” SMPTE J., vol. 91, pp. 372-378, April 1982.
![Page 38: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/38.jpg)
Second application: Demosaicking for digital cameras
Eric Dubois, June 26, 2013
![Page 39: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/39.jpg)
Problem Statement
• Problem: Most digital color cameras capture only one
color component at each spatial location. The
remaining components must be reconstructed by
interpolation from the captured samples. Cameras
provide hardware or software to do this, but the quality
may be inadequate.
• Objective: Develop new algorithms to interpolate each
color plane (called demosaicking) with better quality
reconstruction, and with minimal computational
complexity.
Eric Dubois, June 26, 2013
![Page 40: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/40.jpg)
Construction of color image from color planes
+
Eric Dubois, June 26, 2013
![Page 41: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/41.jpg)
Lighthouse original
Eric Dubois, June 26, 2013
![Page 42: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/42.jpg)
Lighthouse Bayer CFA image
Eric Dubois, June 26, 2013
![Page 43: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/43.jpg)
Formation of Color planes
Eric Dubois, June 26, 2013
![Page 44: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/44.jpg)
Color plane interpolation
GA
GB
GL GR
)(4
1ABRLI GGGGG
GI
Green channel: bilinear interpolation
Eric Dubois, June 26, 2013
![Page 45: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/45.jpg)
Color plane interpolation
)(4
1SESWNENWC RRRRR
RC
Red channel: bilinear interpolation
RNW RNE
RSW RSE RS
SESWS RRR 2
1
Eric Dubois, June 26, 2013
![Page 46: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/46.jpg)
Lighthouse Interpolated color image
![Page 47: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/47.jpg)
Lighthouse original
![Page 48: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/48.jpg)
Spectrum of CFA signal
a b
Eric Dubois, June 26, 2013
![Page 49: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/49.jpg)
Using C2a only Using C2b only Eric Dubois, June 26, 2013
![Page 50: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/50.jpg)
Original From C2a only From C2b only
Demosaicking using C2a only or C2b only -- details
Eric Dubois, June 26, 2013
![Page 51: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/51.jpg)
My Contributions
• I recognized that the C2 component appears twice.
• We can reconstruct the signal with either of them.
• Typically at each location at least one of them will give a good reconstruction.
• We need a genie to tell us which to use at each location.
• I presented an algorithm that works almost as well as the genie most of the time and gives state-of-the-art performance with relatively low computational complexity.
Eric Dubois, June 26, 2013
![Page 52: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/52.jpg)
Eric Dubois, June 26, 2013
![Page 53: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/53.jpg)
Third Application: Stereoscopic Imaging
Development of a new approach for creating stereoscopic images for colored glasses, called the anaglyph method.
Eric Dubois, June 26, 2013
![Page 54: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/54.jpg)
Binocular Vision
We see the world with two eyes. Each eye sees a slightly different view of the scene we’re looking at. The brain interprets the differences and provides us with the 3D perception of depth.
Eric Dubois, June 26, 2013
![Page 55: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/55.jpg)
Stereoscopic Imaging
• Form two views of the scene from slightly different points of view -- either with a camera or by computer graphics
• Display the two views with some apparatus that forces the left eye to only see the left view and the right eye to only see the right view
Eric Dubois, June 26, 2013
![Page 56: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/56.jpg)
What is ‘anaglyph’?
Anaglyph is a method to view stereoscopic images using coloured spectacles. The method was patented in 1891 by Louis Ducos du Hauron, but similar methods had been demonstrated previously by W. Rollmann in 1853 and J.C. D’Almeida in 1858.
Eric Dubois, June 26, 2013
![Page 57: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/57.jpg)
Classical method
For monochrome (no color) stereo images, the left view in blue (or green) is superimposed on the same image with the right view in red. When viewed through spectacles of corresponding colors but reversed, the three-dimensional effect is perceived.
Eric Dubois, June 26, 2013
![Page 58: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/58.jpg)
The Anaglyph stereoscopic images in this presentation require the red/blue glasses available in this room to perceive the 3D effect The red filter goes over your LEFT eye.
LEFT RIGHT Eric Dubois, June 26, 2013
![Page 59: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/59.jpg)
CAUTION
• It is said that about 10% of people don’t perceive the stereoscopic 3D effect.
• Some people may feel queasy when viewing 3D images.
• If you’re in the first group, you may find the images in the presentation rather boring.
• If you’re in the 2nd group please don’t feel
obliged to look at the images with the glasses!
Eric Dubois, June 26, 2013
![Page 60: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/60.jpg)
Grayscale anaglyph
Eric Dubois, June 26, 2013
![Page 61: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/61.jpg)
New Method to Make Anaglyph Images
• Uses the properties of the glasses --- how the two filters affect the wavelengths in the light
• Uses the characteristics of the Human Visual System
• Uses the characteristics of the display device
Eric Dubois, June 26, 2013
![Page 62: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/62.jpg)
Eric Dubois, June 26, 2013
![Page 63: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/63.jpg)
Transmission of red and blue filters
Eric Dubois, June 26, 2013
![Page 64: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/64.jpg)
Anaglyph viewing scenario
Eric Dubois, June 26, 2013
Light from screen Glasses (filters) Human visual system
Create the best anaglyph image, such that when passed through
the glasses, it gives the viewer the closest rendition to the
true stereo pair.
![Page 65: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/65.jpg)
![Page 66: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/66.jpg)
Youtube 3d video
Eric Dubois, June 26, 2013
![Page 67: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/67.jpg)
Star polyhedron
Eric Dubois, June 26, 2013
![Page 68: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/68.jpg)
Eric Dubois, June 26, 2013
![Page 69: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/69.jpg)
Stereoscopic panoramas
• Telepresence aims to provide an electronic experience as close as possible to being there.
• We have been working on techniques for virtual navigation in remote real-world environments (Navire project) using omnidirectional images (similar to Google Street View).
• One aspect is stereoscopic omnidirectional imaging.
• Ph.D. student Luis Gurrieri is currently working on this, and I will show some of his results.
Eric Dubois, June 26, 2013
![Page 70: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/70.jpg)
Eric Dubois, June 26, 2013
Stereoscopic Panoramas
• Living Room
• Lamoureux
• Tabaret
![Page 71: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/71.jpg)
Concluding Thoughts
• Spend a lot of time to choose the best notation to describe the problem.
• Formulate the problem carefully trying to address the fundamental principles.
• Get a bright idea.
• Express your solution in terms of your great notation and your solid formulation.
Eric Dubois, June 26, 2013
![Page 72: Eric Dubois Visual Communications from Broadcast TV to ...edubois/glinski/Glinski...Eric Dubois, June 26, 2013 Eric Dubois, June 26, 2013 Third Application: Stereoscopic Imaging Development](https://reader036.vdocuments.site/reader036/viewer/2022071502/6121b930fb676a2515660785/html5/thumbnails/72.jpg)
Acknowledgements
• I would like to thank my many students and collaborators over the past 35 years; I have touched on some of their work here, but of course I could not mention most of it. I am happy to be in contact with most of them on LinkedIn.
• Thanks to NSERC for 35 years of Operating/Research/ Discovery Grant funding. I have had grant #00022 since 1978. Also thanks to BNR and CBC.
• Thanks to Kent Walker of the University’s Multimedia Distribution department for replacing the projector in this room within one day.
• And thanks to you for your attention!
Eric Dubois, June 26, 2013