introduction and course overview - carnegie mellon school ...16385/s18/lectures/lecture15.pdf ·...
TRANSCRIPT
![Page 1: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/1.jpg)
Color
16-385 Computer VisionSpring 2018, Lecture 15http://www.cs.cmu.edu/~16385/
![Page 2: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/2.jpg)
Course announcements
• Homework 4 has been posted.- Due Friday March 23rd (one-week homework!)- Any questions about the homework?- How many of you have looked at/started/finished homework 4?
• Talk this week: Katie Bouman, “Imaging the Invisible”.- Wednesday, March 21st 10:00 AM GHC6115.
![Page 3: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/3.jpg)
Overview of today’s lecture
• Color and human color perception.
• Retinal color space.
• Color matching.
• Linear color spaces.
• Chromaticity.
• Non-linear color spaces.
• Example computer vision application using color.
![Page 4: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/4.jpg)
Slide credits
Many of these slides were inspired or adapted from:
• Todd Zickler (Harvard).• Fredo Durand (MIT).
![Page 5: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/5.jpg)
Color and human color perception
![Page 6: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/6.jpg)
Color is an artifact of human perception
• “Color” is not an objective physical property of light (electromagnetic radiation).• Instead, light is characterized by its wavelength.
What we call “color” is how we subjectively perceive a very small
range of these wavelengths.
electromagnetic spectrum
![Page 7: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/7.jpg)
spectral reflectance
illuminant spectrum
Light-material interaction
spectral radiance
![Page 8: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/8.jpg)
spectral reflectance
illuminant spectrum
Light-material interaction
spectral radiance
![Page 9: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/9.jpg)
Illuminant Spectral Power Distribution (SPD)
• Most types of light “contain” more than one wavelengths.• We can describe light based on the distribution of power over different wavelengths.
We call our sensation of all of these
distributions “white”.
![Page 10: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/10.jpg)
spectral reflectance
illuminant spectrum
Light-material interaction
spectral radiance
![Page 11: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/11.jpg)
Spectral reflectance
• Most materials absorb and reflect light differently at different wavelengths.• We can describe this as a ratio of reflected vs incident light over different wavelengths.
![Page 12: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/12.jpg)
spectral reflectance
illuminant spectrumspectral radiance
Light-material interaction
![Page 13: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/13.jpg)
Human color vision
perceived color
object color
color names
retinal color
spectral radiance
![Page 14: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/14.jpg)
Retinal vs perceived color
Retinal vs perceived color.
![Page 15: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/15.jpg)
Retinal vs perceived color
• Our visual system tries to “adapt” to illuminant.
• We may interpret the same retinal color very differently.
![Page 16: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/16.jpg)
Human color vision
perceived color
object color
color names
retinal color
spectral radiance
We will exclusively discuss retinal color in this course
![Page 17: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/17.jpg)
Retinal color space
![Page 18: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/18.jpg)
Spectral Sensitivity Function (SSF)
• Any light sensor (digital or not) has different sensitivity to different wavelengths.
• This is described by the sensor’s spectral sensitivity function .
• When measuring light of a some SPD , the sensor produces a scalar response:
sensor response
light SPD sensor SSF
Weighted combination of light’s SPD: light contributes more at wavelengths where the sensor has higher sensitivity.
![Page 19: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/19.jpg)
Spectral Sensitivity Function of Human Eye
• The human eye is a collection of light sensors called cone cells.
• There are three types of cells with different spectral sensitivity functions.
• Human color perception is three-dimensional (tristimulus color).
“short”
“medium”
“long”
cone distribution for normal vision (64% L, 32% M)
![Page 20: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/20.jpg)
The retinal color space
“pure beam” (laser)
![Page 21: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/21.jpg)
The retinal color space
• “lasso curve”• contained in positive octant• parameterized by wavelength• starts and ends at origin• never comes close to M axis“pure beam” (laser)
why?why?
![Page 22: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/22.jpg)
The retinal color space
“pure beam” (laser)
if we also consider variations in the strengthof the laser this “lasso” turns into (convex!) radial cone with a “horse-shoe shaped” radial cross-section
![Page 23: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/23.jpg)
The retinal color space
“mixed beam”= positive combination of pure colors
colors of mixed beams areinside of convex cone
![Page 24: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/24.jpg)
The retinal color space
“mixed beam”= positive combination of pure colors
• distinct mixed beams can produce the same retinal color
• These beams are called metamers
![Page 25: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/25.jpg)
There is an infinity of metamers
![Page 26: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/26.jpg)
Example: illuminant metamerism
day light scanned copy hallogen light
![Page 27: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/27.jpg)
Color matching
![Page 28: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/28.jpg)
“primaries”
“test light”
Adjust the strengths of the primaries until they re-produce the test color. Then:
equality symbol means “has the same retinal color as” or “is metameric to”
CIE color matching
![Page 29: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/29.jpg)
“primaries”
“test light”
CIE color matching
To match some test colors, you need to add some primary beam on the left (same as “subtracting light” from the right)
![Page 30: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/30.jpg)
http://graphics.stanford.edu/courses/cs178/applets/colormatching.html
Color matching demo
![Page 31: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/31.jpg)
“primaries”
CIE color matching
Repeat this matching experiments for pure test beams at wavelengths λi and keep track of
the coefficients (negative or positive) required to reproduce each pure test beam.
![Page 32: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/32.jpg)
“primaries”
CIE color matching
Repeat this matching experiments for pure test beams at wavelengths λi and keep track of
the coefficients (negative or positive) required to reproduce each pure test beam.
note the negative values
![Page 33: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/33.jpg)
“primaries”
CIE color matching
What about “mixed beams”?
![Page 34: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/34.jpg)
Two views of retinal color
Analytic: Retinal color is produced by analyzing spectral power distributions using the color sensitivity functions.
Synthetic: Retinal color is produced by synthesizing color primaries using the color matching functions.
?
![Page 35: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/35.jpg)
Two views of retinal color
Analytic: Retinal color is produced by analyzing spectral power distributions using the color sensitivity functions.
Synthetic: Retinal color is produced by synthesizing color primaries using the color matching functions.
The two views are equivalent: Color matching functions are also color sensitivity functions. For each set of color sensitivity functions, there are corresponding color primaries.
![Page 36: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/36.jpg)
Linear color spaces
![Page 37: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/37.jpg)
1) Color matching experimental outcome:
Linear color spaces
same in matrix form:
how is this matrix formed?
![Page 38: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/38.jpg)
1) Color matching experimental outcome:
2) Implication for arbitrary mixed beams:
Linear color spaces
same in matrix form:
where do these terms come from?
![Page 39: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/39.jpg)
1) Color matching experimental outcome:
2) Implication for arbitrary mixed beams:
Linear color spaces
same in matrix form:
what is this similar to?
![Page 40: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/40.jpg)
1) Color matching experimental outcome:
2) Implication for arbitrary mixed beams:
change of basis matrix
Linear color spaces
same in matrix form:
representation of retinal color in LMS space
representation of retinal color in space of primaries
![Page 41: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/41.jpg)
1) Color matching experimental outcome:
2) Implication for arbitrary mixed beams:
change of basis matrix
Linear color spaces
same in matrix form:
representation of retinal color in LMS space
representation of retinal color in space of primaries
![Page 42: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/42.jpg)
Linear color spaces
can insert any invertible M
basis for retinal color ⇔ color matching functions⇔ primary colors⇔ color space
change of basis matrixrepresentation of retinal
color in LMS spacerepresentation of retinal
color in space of primaries
![Page 43: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/43.jpg)
A few important color spaces
LMS color space CIE RGB color space
not the “usual” RGB color space encountered in practice
![Page 44: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/44.jpg)
Two views of retinal color
Analytic: Retinal color is three numbers formed by taking the dot product of a power spectral distribution with three color matching/sensitivity functions.
Synthetic: Retinal color is three numbers formed by assigning weights to three color primaries to match the perception of a power spectral distribution.
How would you make a color measurement device?
![Page 45: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/45.jpg)
How would you make a color measurement device?
Do what the eye does:• Select three spectral filters (i.e.,
three color matching functions.).• Capture three measurements.
CIE RGB color space
Can we use the CIE RGB color matching functions?
![Page 46: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/46.jpg)
How would you make a color measurement device?
Do what the eye does:• Select three spectral filters (i.e.,
three color matching functions.).• Capture three measurements.
CIE RGB color space
Can we use the CIE RGB color matching functions?
Negative values are an issue (we can’t “subtract” light at a sensor)
![Page 47: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/47.jpg)
How would you make a color measurement device?
Do what the eye does:• Select three spectral filters (i.e.,
three color matching functions.).• Capture three measurements.
LMS color space
Can we use the LMS color matching functions?
![Page 48: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/48.jpg)
How would you make a color measurement device?
Do what the eye does:• Select three spectral filters (i.e.,
three color matching functions.).• Capture three measurements.
LMS color space
Can we use the LMS color matching functions?
• They weren’t known when CIE was doing their color matching experiments.
• We’ll see later they create other issues.
![Page 49: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/49.jpg)
How would you make a color measurement device?
Do what the eye does:• Select three spectral filters (i.e.,
three color matching functions).• Capture three measurements.
LMS color space
Can we use the LMS color matching functions?
• They weren’t known when CIE was doing their color matching experiments.
• We’ll see later they create other issues.
![Page 50: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/50.jpg)
The CIE XYZ color space
CIE XYZ color space
• Derived from CIE RGB by adding enough blue and green to make the red positive.
• Probably the most important reference(i.e., device independent) color space.
Remarkable and/or scary: 80+ years of CIE XYZ is all down to color matching experiments done with 12 “standard observers”.
![Page 51: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/51.jpg)
The CIE XYZ color space
CIE XYZ color space
Y corresponds to luminance (“brightness”)
How would you convert a color image to grayscale?
• Derived from CIE RGB by adding enough blue and green to make the red positive.
• Probably the most important reference(i.e., device independent) color space.
X and Z correspond to chromaticity
![Page 52: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/52.jpg)
A few important color spaces
LMS color space CIE RGB color space
CIE XYZ color space
![Page 53: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/53.jpg)
Two views of retinal color
Analytic: Retinal color is three numbers formed by taking the dot product of a power spectral distribution with three color matching/sensitivity functions.
Synthetic: Retinal color is three numbers formed by assigning weights to three color primaries to match the perception of a power spectral distribution.
How would you make a color reproduction device?
![Page 54: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/54.jpg)
How would you make a color reproduction device?
Do what color matching does:• Select three color primaries.• Represent all colors as mixtures of
these three primaries.
Can we use the XYZ color primaries?
CIE XYZ color space
![Page 55: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/55.jpg)
How would you make a color reproduction device?
Do what color matching does:• Select three color primaries.• Represent all colors as mixtures of
these three primaries.
Can we use the XYZ color primaries?
CIE XYZ color space• No, because they are not “real” colors (they require an SPD with negative values).
• Same goes for LMS color primaries.
![Page 56: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/56.jpg)
The Standard RGB (sRGB) color space
sRGB color space
• Derived by Microsoft and HP in 1996, based on CRT displays used at the time.
• Similar but not equivalent to CIE RGB.
While it is called “standard”, when you grab an “RGB” image, it is highly likely it is in a different RGB color space…
Note the negative values
![Page 57: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/57.jpg)
A few important color spaces
LMS color space CIE RGB color space
CIE XYZ color space sRGB color space
![Page 58: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/58.jpg)
A few important color spaces
LMS color space CIE RGB color space
CIE XYZ color space sRGB color space
Is there a way to “compare” all these color
spaces?
![Page 59: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/59.jpg)
Chromaticity
![Page 60: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/60.jpg)
chromaticity
luminance/brightness
CIE xy (chromaticity)
Perspective projection of 3D retinal color space to two dimensions.
![Page 61: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/61.jpg)
CIE xy (chromaticity)
Note: These colors can be extremely misleading depending on the file origin and the display you are using
![Page 62: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/62.jpg)
CIE xy (chromaticity)
What does the boundary of the chromaticity diagram correspond to?
![Page 63: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/63.jpg)
Color gamuts
We can compare color spaces by looking at what parts of the chromaticity space they can reproduce with their primaries.
But why would a color space not be able to reproduce all of the chromaticity space?
![Page 64: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/64.jpg)
Color gamuts
We can compare color spaces by looking at what parts of the chromaticity space they can reproduce with their primaries.
But why would a color space not be able to reproduce all of the chromaticity space?
• Many colors require negative weights to be reproduced, which are not realizable.
![Page 65: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/65.jpg)
Color gamuts
sRGB color gamut:• What are the three triangle corners?• What is the interior of the triangle?• What is the exterior of the triangle?
![Page 66: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/66.jpg)
Color gamuts
sRGB color gamut
sRGB color primaries
sRGBrealizable
colors
sRGBimpossible
colors
![Page 67: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/67.jpg)
Color gamuts
Gamuts of various common industrial RGB spaces
What is this?
![Page 68: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/68.jpg)
The problem with RGBs visualized in chromaticity space
RGB values have no meaning if the primaries between devices are not the same!
![Page 69: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/69.jpg)
Color gamuts
• Can we create an RGB color space that reproduces the entire chromaticity diagram?
• What would be the pros and cons of such a color space?
• What devices would you use it for?
![Page 70: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/70.jpg)
Chromaticity diagrams can be misleading
Different gamuts may compare very differently when seen in full 3D retinal color space.
![Page 71: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/71.jpg)
Two views of retinal color
Analytic: Retinal color is three numbers formed by taking the dot product of a power spectral distribution with three color matching/sensitivity functions.
Synthetic: Retinal color is three numbers formed by assigning weights to three color primaries to match the perception of a power spectral distribution.
How would you make a color reproduction device?
![Page 72: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/72.jpg)
Non-linear color spaces
![Page 73: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/73.jpg)
A few important linear color spaces
LMS color space CIE RGB color space
CIE XYZ color space sRGB color space
What about non-linear color spaces?
![Page 74: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/74.jpg)
chromaticity
luminance/brightness
CIE xy (chromaticity)
CIE xyY is a non-linear color space.
![Page 75: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/75.jpg)
Uniform color spaces
![Page 76: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/76.jpg)
MacAdam ellipses
Areas in chromaticity space of imperceptible change:• They are ellipses instead of circles.• They change scale and direction in different parts
of the chromaticity space.
![Page 77: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/77.jpg)
MacAdam ellipsesNote: MacAdam ellipses are almost always shown at 10x scale for visualization. In reality, the areas of imperceptible difference are much smaller.
![Page 78: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/78.jpg)
The Lab (aka L*ab, aka L*a*b*) color space
![Page 79: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/79.jpg)
The Lab (aka L*ab, aka L*a*b*) color space
![Page 80: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/80.jpg)
Hue, saturation, and value
Do not use color space HSV! Use LCh:• L* for “value”.• C = sqrt(a2 + b2) for “saturation” (chroma).• h = atan(b / a) for “hue”.
![Page 81: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/81.jpg)
![Page 82: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/82.jpg)
LCh
![Page 83: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/83.jpg)
Chromaticity: Human skin
![Page 84: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/84.jpg)
Useful for detecting faces
“How OpenCV's Face Tracker Works”-SERVO Magazine, March 2007
![Page 85: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/85.jpg)
Application: Shadow removal
![Page 86: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/86.jpg)
Application: Shadow removal
![Page 87: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/87.jpg)
Application: Shadow removal
![Page 88: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/88.jpg)
Application: Shadow removal
B
W R
YG
P
Narrow-band (delta-function sensitivities)
Log-opponent chromaticities for 6 surfaces under 9 lights
![Page 89: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/89.jpg)
Application: Shadow removal
Log-opponent chromaticities for 6 surfaces under 9 lights
This axis is invariant to illuminant colour
Rotate chromaticities
![Page 90: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/90.jpg)
Normalized sensitivities of a SONY DXC-930 video camera
Application: Shadow removal
Log-opponent chromaticities for 6 surfaces under 9 different lights
![Page 91: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/91.jpg)
Application: Shadow removal
Log-opponent chromaticities for 6 surfaces under 9 different lights
The invariant axis is now only approximately illuminant invariant (but hopefully good enough)
Rotate chromaticities
![Page 92: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/92.jpg)
Application: Shadow removal
![Page 93: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/93.jpg)
Application: Invariance for material segmentation
Input image Hue
![Page 94: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/94.jpg)
DIFFUSE
= +
SPECULAR
Application: highlight removal
Problem: This is hard when the diffuse color is spatially-varying
![Page 95: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/95.jpg)
Teaser for Homework 4
![Page 96: Introduction and course overview - Carnegie Mellon School ...16385/s18/lectures/lecture15.pdf · Course announcements • Homework 4 has been posted. - Due Friday March 23rd (one-week](https://reader036.vdocuments.site/reader036/viewer/2022071213/60419ba6d316944c1161e0d9/html5/thumbnails/96.jpg)
References
Basic reading:• Szeliski textbook, Section 2.3.2, 3.1.2.• Gortler textbook, Chapter 19.• Michael Brown, “Understanding the In-Camera Image Processing Pipeline for Computer Vision,” CVPR 2016,
very detailed discussion of issues relating to color photography and management, slides available at: http://www.comp.nus.edu.sg/~brown/CVPR2016_Brown.html
Additional reading:• Reinhard et al., “Color Imaging: Fundamentals and Applications,” A.K Peters/CRC Press 2008.• Koenderink, “Color Imaging: Fundamentals and Applications,” MIT Press 2010.• Fairchild, “Color Appearance Models,” Wiley 2013.
all of the above books are great references on color photography, reproduction, and management.