contrast preserving decolorization cewu lu, li xu, jiaya jia, the chinese university of hong kong
TRANSCRIPT
Contrast Preserving Decolorization
Cewu Lu, Li Xu, Jiaya Jia, The Chinese University of Hong Kong
Mono printers are still the majority
• Fast• Economic• Environmental friendly
Documents generally have color figures
The printing problem
The printing problem
The printing problem
The printing problem
HP printer
The printing problem
Our Result
The printing problem
Decolorization
Mapping
Single Channel
Applications
Color Blindness
Applications
Color Blindness
Decolorization could lose contrast Mapping( )
Mapping( ) =
=
Mapping
Decolorization could lose contrast
• Bala and Eschbach 2004
• Neumann et al. 2007
• Smith et al. 2008
Pervious Work (Local methods)
Pervious Work (Local methods)
Naive Mapping
Color Contrast
Result
• Gooch et al. 2004
• Rasche et al. 2005
• Kim et al. 2009
Pervious Work (Global methods)
Pervious Work (Global methods)
Color feature preserving optimization mapping function
( )g f c
Pervious Work (Global methods)
In most global methods, color order is strictly satisfied
( )g f c
Color order could be ambiguous
Can you tell the order?
brightness( ) < brightness ( ) YUV space
Lightness( ) > Lightness ( ) LAB space
Color order could be ambiguous
People with different culture and language background have different senses of brightness with respect to color.
E. Ozgen et al., Current Directions in Psychological Science, 2004
K. Zhou et al., National Academy of Sciences, 2010
The order of different colors cannot be defined uniquely by people
B. Wong et al., Nature Methods, 2010
Color order could be ambiguous
If we enforce the color order constraint, contrast loss could happen
Input Ours[Rasche et al. 2005] [Kim et al. 2009]
Color order could be ambiguous
Our Contribution
Weak Color Order
Bimodal Contrast-PreservingRelax the color order constraint
Unambiguous color pairs
Global Mapping Polynomial Mapping
The Framework
• Objective Function Bimodal Contrast-Preserving Weak Color Order
• Finite Multivariate Polynomial Mapping Function
• Numerical Solution
Bimodal Contrast-Preserving
• Color pixel , grayscale contrast , color contrast (CIELab distance) • follows a Gaussian distribution with mean
{ , }x y xy x yg g g
xyg xy
xy
2
22
, exp2xy xy
xy
gG
Bimodal Contrast-Preserving
• Color pixel , grayscale contrast , color contrast (CIELab distance) • follows a Gaussian distribution with mean .
{ , }x y xy x yg g g
xyg xy
xy
2
22
, exp2xy xy
xy
gG
xy xyg
Bimodal Contrast-Preserving
• Tradition methods (order preserving):
2
( , )
max sign( , ) ,xyg
x y N
G x y
N : neighborhood pixel set
• Our bimodal contrast-preserving for ambiguous color pairs:
2 2
( , )
max , ,xy xyg
x y N
G G
Bimodal Contrast-Preserving
2
( , )
max sign( , ) ,xyg
x y N
G x y
2 2
( , )
max , ,xy xyg
x y N
G G
xyxy
xy xyg
xyg
sign(x,y) 1=
Bimodal Contrast-Preserving
2
( , )
max sign( , ) ,xyg
x y N
G x y
2 2
( , )
max , ,xy xyg
x y N
G G
xyxy
xyg
xyg
xy
sign(x,y) 1=
Our Work
• Objective Function Bimodal Contrast-Preserving Weak Color Order
• Finite Multivariate Polynomial Mapping Function
• Numerical Solution
Weak Color Order
• Unambiguous color pairs: or & &x y x y x yr r g g b b & &x y x y x yr r g g b b
Weak Color Order
• Unambiguous color pairs: or & &x y x y x yr r g g b b & &x y x y x yr r g g b b
,
1.0 unambiguous color pair
0.5 ambiguous color pairx y
• Our model thus becomes
2 2, ,
( , )
max , 1 ,x y xy x y xyg
x y N
G G
Our Work
• Objective Function Bimodal Contrast-Preserving Weak Color Order
• Finite Multivariate Polynomial Mapping Function
• Numerical Solution
Multivariate Polynomial Mapping Function
2 2, ,
( , )
max , 1 ,x y xy x y xyg
x y N
G G
Solve for grayscale image: g
Variables (pixels): 400x250 = 100,000
Example
Too many (easily produce unnatural structures)
Multivariate Polynomial Mapping Function
2 2, ,
( , )
max , 1 ,x y xy x y xyg
x y N
G G
• Parametric global color-to-grayscale mapping
grayscale value (color vector, )f
Small Scale
Multivariate Polynomial Mapping Function
31 21 2 3span{ : =0, 1, 2, ... n}dd d
n ir g b d d d d
• Parametric color-to-grayscale( , ) i i
i
f c m n
When n = 2, a grayscale is a linear combination of elements
imthiis the monomial basis of , .
2 2 2{ , , , , , , , , }r g b rg gb rb r g b
{ , , }c r g b
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale( , ) i i
i
f c m
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale( , ) i i
i
f c m
2b2g2r
gbrbrg
bgr
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale( , ) i i
i
f c m
2b2g2r
gbrbrg
bg
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale( , ) i i
i
f c m
2b2g2r
gbrbrg
b
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale( , ) i i
i
f c m
2b2g2r
gbrbrg
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale( , ) i i
i
f c m
2b2g2r
gbrb
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale( , ) i i
i
f c m
2b2g2r
gb
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale( , ) i i
i
f c m
2b2g2r
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale( , ) i i
i
f c m
2b2g
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale( , ) i i
i
f c m
2b
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale( , ) i i
i
f c m
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale( , ) i i
i
f c m
0.1550 0.8835 0.3693
0.1817 0.4977 -1.7275
-0.4479 0.6417 0.6234
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale( , ) i i
i
f c m
0.1550 0.8835 0.3693
0.1817 0.4977 -1.7275
-0.4479 0.6417 0.6234
Our Model
, ,xy x y x y i ix iyi
g g g f c f c m m
• Objective function:
2 2, ,
( , )
max , 1 ,x y xy x y xyx y N
G G
Numerical Solution
2 2, ,
( , )
max , 1 ,x y xy x y xyx y N
G G
2 2, ,
( , )
min In , 1 ,x y xy x y xyx y N
G G
2 2, ,
( , )
In , 1 ,x y xy x y xyx y N
E G G
Define:
Numerical Solution
0
E
2, ,
, 2 2, , , ,
,
, 1 ,
x y x y
x y
x y x y x y x y
G
G G
min E
, ,( , )
1 2 0i xi yi xj yj x y xj yj x yx y N ij
Em m m m m m
Numerical Solution
, ,( , )
2 1i xi yi xj yj x y xj yj x yx y N i
m m m m m m
Initialize :
Numerical Solution
2, ,
, 2 2, , , ,
,
, 1 ,
x y x y
x y
x y x y x y x y
G
G G
, ,( , )
2 1i xi yi xj yj x y xj yj x yx y N i
m m m m m m
obtain
Numerical Solution
2, ,
, 2 2, , , ,
,
, 1 ,
x y x y
x y
x y x y x y x y
G
G G
, ,( , )
2 1i xi yi xj yj x y xj yj x yx y N i
m m m m m m
obtainobtain ,x y
Numerical Solution
2, ,
, 2 2, , , ,
,
, 1 ,
x y x y
x y
x y x y x y x y
G
G G
, ,( , )
2 1i xi yi xj yj x y xj yj x yx y N i
m m m m m m
obtainobtain ,x y
Numerical Solution
2, ,
, 2 2, , , ,
,
, 1 ,
x y x y
x y
x y x y x y x y
G
G G
, ,( , )
2 1i xi yi xj yj x y xj yj x yx y N i
m m m m m m
obtainobtain ,x y
Numerical Solution
2, ,
, 2 2, , , ,
,
, 1 ,
x y x y
x y
x y x y x y x y
G
G G
, ,( , )
2 1i xi yi xj yj x y xj yj x yx y N i
m m m m m m
obtainobtain ,x y
Numerical Solution (Example)
2 2 2 r g b rg rb gb r g bIter 1
0.33 0.33 0.33 0.00 0.00 0.00 0.00 0.00 0.00
Numerical Solution (Example)
2 2 2 r g b rg rb gb r g bIter 2
0.97 0.91 0.38 -3.71 2.46 -4.01 -4.02 4.00 0.79
Numerical Solution (Example)
2 2 2 r g b rg rb gb r g bIter 3
1.14 -0.25 1.22 -1.55 -1.53 -3.51 -1.18 3.32 0.69
Numerical Solution (Example)
2 2 2 r g b rg rb gb r g bIter 4
1.33 -1.61 2.10 1.35 -0.36 -1.61 -1.69 1.70 0.29
Numerical Solution (Example)
2 2 2 r g b rg rb gb r g bIter 5
1.52 -2.25 2.46 2.69 -1.38 -0.30 -1.95 0.79 -0.02
Numerical Solution (Example)
2 2 2 r g b rg rb gb r g bIter 13
1.98 -3.29 3.02 5.94 -3.38 2.81 -2.91 -1.56 -0.96
Numerical Solution (Example)
2 2 2 r g b rg rb gb r g bIter 14
1.99 -3.31 3.03 6.03 -3.42 2.89 -2.95 -1.62 -0.98
Numerical Solution (Example)
2 2 2 r g b rg rb gb r g bIter 15
2.00 -3.32 3.04 6.10 -3.45 2.94 -2.98 -1.67 -1.00
Results
Input
Ours [Rasche et al. 2005] [Kim et al. 2009]
Results
Input
Ours [Rasche et al. 2005] [Kim et al. 2009]
Results
Input
Ours [Rasche et al. 2005] [Kim et al. 2009]
Results
Input
Ours [Rasche et al. 2005] [Kim et al. 2009]
Results (Quantitative Evaluation)
• color contrast preserving ratio (CCPR)
# ( , ) | ( , ) ,| |CCPR=
| |x yx y x y g g
the set containing all neighboring pixel pairs with the original color difference .
,x y
Results (Quantitative Evaluation)
10
Our Results (Quantitative Evaluation)
10
,x y
Results (Quantitative Evaluation)
10
,x y , ,&x y x yg
Results (Quantitative Evaluation)
10
,x y , ,&x y x yg
Number: 38740 Number: 24853
Results (Quantitative Evaluation)
Number: 38740 Number: 24853
24853CCPR= 64.2%
38740
Results (Quantitative Evaluation)
Results (contrast boosting)
substituting our grayscale image for the L channel in the Lab space
Results (contrast boosting)
substituting our grayscale image for the L channel in the Lab space
Conclusion
• A new color-to-grayscale method that can well maintain the color contrast.
• Weak color constraint.
• Polynomial Mapping Function for global mapping.
The End
Limitations
• Color2gray is very subjective visual experience. Contrast enhancement may not be acceptable for everyone.
• Compared to the naive color2grayscale mapping, our method is less efficient due to the extra operations.
An arguable result
Running Time
• For a 600 × 600 color input, our Matlab implementation takes 0.8s
• A C-language implementation can be 10 times faster at least.