![Page 1: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/1.jpg)
Computational Photography and Capture:
(Re)Coloring
Gabriel Brostow & Tim Weyrich
TA: Frederic Besse
![Page 2: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/2.jpg)
Week Date Topic Hours
1 12-Jan Introduction to Computational Photography and Capture 1
1 14-Jan Intro + More on Cameras, Sensors and Color 2
2 19-Jan No lecture! (Go capture bracketed photos?) -
2 21-Jan Blending, Compositing, Poisson Editing 2
3 26-Jan Time-Lapse 1
3 28-Jan Carving, Warping, and Morphing 2
4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1
4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2
5 09-Feb Colorization and Color Transfer 1
5 11-Feb Image Inpainting and Texture Synthesis 2
7 23-Feb Video Based Rendering of Scenes I 1
7 25-Feb Video Based Rendering of Scenes II 2
8 02-Mar Video Texture Synthesis 1
8 04-Mar Video Sprites 2
9 09-Mar Deblurring/Dehazing and Coded Aperture Imaging 1
9 11-Mar Image-based Rendering 2
10 16-Mar Motion Capture guest lecture by Doug Griffin
10 18-Mar Capturing Geometry with Active Lighting 2
11 23-Mar Intrinsic Images 1
11 25-Mar Dual Photography and Reflectance Analysis 2
![Page 3: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/3.jpg)
Today’s Lecture
• Colorization using Optimization– Levin, Lischinski, Weiss, Siggraph 2004
• Color Transfer Between Images– Reinhard, Ashikhmin, Gooch, Shirley, CG&A 2001
• N-Dimensional Probability Density Function Transfer and its Application to Colour Transfer– Pitié, Kokaram, Dahyot, ICCV 2005
(see also Interactive Local Adjustment of Tonal Values’06)
![Page 4: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/4.jpg)
Colorization Using OptimizationAnat Levin Dani Lischinski Yair Weiss
SIGGRAPH 2004
Project web page
Note:
Using Anat Levin’s slides for structure,
with the other papers appearing throughout
![Page 5: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/5.jpg)
Colorization
Colorization: a computer-assisted process of
adding color to a monochrome image or movie.
(Invented by Wilson Markle, 1970)
![Page 6: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/6.jpg)
Motivation
• Colorizing black and white movies and TV shows
Earl Glick (Chairman, Hal Roach Studios), 1984:
“You couldn't make Wyatt Earp today for $1 million an
episode. But for $50,000 a segment, you can turn it into
color and have a brand new series with no residuals to pay”
Hugh O'Brien as Wyatt Earp, 1957
![Page 7: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/7.jpg)
Motivation
• Colorizing black and white movies and TV shows
• Recoloring color images for special effects
![Page 8: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/8.jpg)
Motivation
![Page 9: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/9.jpg)
Typical Colorization Process
Images from:
“Yet Another Colorization Tutorial”
http://www.worth1000.com/tutorial.asp
?sid=161018
![Page 10: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/10.jpg)
• Delineate region boundary
Typical Colorization Process
Images from:
“Yet Another Colorization Tutorial”
http://www.worth1000.com/tutorial.asp
?sid=161018
![Page 11: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/11.jpg)
Typical Colorization Process
Images from:
“Yet Another Colorization Tutorial”
http://www.worth1000.com/tutorial.asp
?sid=161018
• Delineate region boundary
• Choose region color from palette.
![Page 12: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/12.jpg)
• Delineate region boundary
• Choose region color from palette.
Typical Colorization Process
Images from:
“Yet Another Colorization Tutorial”
http://www.worth1000.com/tutorial.asp
?sid=161018
![Page 13: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/13.jpg)
• Delineate region boundary
• Choose region color from palette.
Images from:
“Yet Another Colorization Tutorial”
http://www.worth1000.com/tutorial.asp
?sid=161018
Typical Colorization Process
![Page 14: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/14.jpg)
• Delineate region boundary
• Choose region color from palette.
• Track regions across video frames
Video Colorization Process
![Page 15: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/15.jpg)
Colorization Process Discussion
Time consuming and labor intensive
• Fine boundaries.
• Failures in tracking.
![Page 16: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/16.jpg)
Hertzmann et al. 2001, Welsh et al. 2002
Colorization by Analogy
A : A’
B : B’
?
![Page 17: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/17.jpg)
Hertzmann et al. 2001, Welsh et al. 2002
Colorization by Analogy
A : A’
B : B’
![Page 19: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/19.jpg)
Colorization by Analogy - Discussion
• Indirect artistic control
• No spatial continuity constraint
![Page 20: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/20.jpg)
Levin et al. Approach
![Page 21: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/21.jpg)
Levin et al. Approach
Artist scribbles desired colors inside regions
![Page 22: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/22.jpg)
Levin et al. Approach
Colors are propagated to all pixels
“Nearby pixels with similar intensities should have the same color”
![Page 23: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/23.jpg)
VUY ,Intensity channel Color channels
“Neighboring pixels with similar intensities should have
similar colors”
Propagation using Optimization
![Page 24: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/24.jpg)
VUY ,Intensity channel Color channels
“Neighboring pixels with similar intensities should have
similar colors”
![Page 25: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/25.jpg)
VUY ,Intensity channel Color channels
2
)(
)()()(r rNs
rs sUwrUUJ
• Minimize difference between color at a pixel and an affinity-
weighted average of the neighbors
Propagation using Optimization
![Page 26: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/26.jpg)
Affinity Functions
22 /))()(( rsYrY
rs ew rproportional to local variance
![Page 27: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/27.jpg)
Affinity Functions in Space-Time
22 2/))()(( rsYrY
rs ew
i
1i
1i
![Page 28: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/28.jpg)
2
)(
)()()(r rNs
rs sUwrUUJ
Since cost is quadratic, minimum can be found by
solving sparse system of linear equations.
Minimizing cost function
Minimize:
Subject to labeling constraints
Using Matlab’s least-squares solver for sparse linear systems (code online)
![Page 29: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/29.jpg)
Color Interpolation
![Page 30: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/30.jpg)
Coloring Stills
![Page 31: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/31.jpg)
![Page 32: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/32.jpg)
Coloring Stills
Original Colorized
![Page 33: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/33.jpg)
Progressive Colorization
![Page 34: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/34.jpg)
Progressive Colorization
![Page 35: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/35.jpg)
Progressive Colorization
![Page 36: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/36.jpg)
Coloring Stills
![Page 37: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/37.jpg)
Coloring Stills
![Page 38: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/38.jpg)
Colorization Challenges
![Page 39: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/39.jpg)
Segmentation?
Our result
Segmentation aided
colorization
NCuts Segmentation
(Shi & Malik 97)
![Page 40: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/40.jpg)
Recoloring
Affinity between pixels – based on intensity AND color similarities.
![Page 41: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/41.jpg)
Recoloring
![Page 42: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/42.jpg)
Recoloring
c.f. “Poisson image editing” Perez et al. SIGGRAPH 2003
![Page 43: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/43.jpg)
13 out of 92 frames
Colorizing Video
![Page 44: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/44.jpg)
Colorizing Video
16 out of 101 frames
![Page 47: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/47.jpg)
Still Needed:
• Import image segmentation advantages:• affinity functions, optimization techniques.
• Alternative color spaces, propagating hue and saturation differently
![Page 48: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/48.jpg)
Other Approaches?
• Color space was YUV
• Small amount of user effort needed
• For film/color grading, can this be automated?
![Page 49: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/49.jpg)
Linear AlignmentReinhard, Ashikhmin, Gooch, Shirley, CG&A 2001
• (R, G, B) is rubbish: all channels are correlated
• (L*, a*, b*) is good
• Algorithm (per channel):
– Align mean
– Rescale standard deviation
![Page 50: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/50.jpg)
![Page 51: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/51.jpg)
![Page 52: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/52.jpg)
Linear AlignmentReinhard, Ashikhmin, Gooch, Shirley, CG&A 2001
• RGB is rubbish: all channels are correlated
• L* a* b* is good
• Algorithm (per channel):
– Align mean
– Rescale standard deviation
![Page 53: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/53.jpg)
![Page 54: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/54.jpg)
![Page 55: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/55.jpg)
![Page 56: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/56.jpg)
But Needs Some Guidance…
![Page 57: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/57.jpg)
Non-Linear AlignmentPitié, Kokaram, Dahyot, ICCV 2005
See their project web-page for code + papersSee Lab page for code + paper
![Page 58: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/58.jpg)
Iterate 1D Solution at Different Rotations
• 1D solution uses cumulative PDFs:
• ND solution:
– Pick a rotation matrix R, apply to both 3D distribs.
– Project both distribs. onto each axis in turn
– Apply 1D solution
– Unproject, unrotate
– <repeat>
![Page 59: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/59.jpg)
![Page 60: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/60.jpg)
![Page 61: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/61.jpg)
![Page 62: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/62.jpg)
Don’t Forget Other Problems
• (Really) automatic colorization
• Flicker removal
• Dirt removal
• Noise removal
• Supersampling…
• Texture (aren’t histograms enough?)– Two-scale Tone Management for Photographic Look, by
Bae, Paris, Durand, Siggraph 2006
![Page 63: Computational Photography and Capture: (Re)Coloring€¦ · 4 02-Feb High-Dynamic-Range Imaging and Tone Mapping 1 4 04-Feb Hybrid Images, Flash and Multi-Flash Photography 2 5 09-Feb](https://reader036.vdocuments.site/reader036/viewer/2022081406/5f0f65ed7e708231d443f65a/html5/thumbnails/63.jpg)
Don’t Forget Other Problems
• (Really) automatic colorization
• Flicker removal
• Dirt removal
• Noise removal
• Supersampling…
• Texture– Two-scale Tone Management for Photographic Look, by
Bae, Paris, Durand, Siggraph 2006