EE 113D Final Project: ColorizationSpring, 2006

Group Members:Johnny ChengBrian CheungAustin Wong

Professor: R. JainTA: Rick Lan

Motivation

• Enhances Visual Appeal of the Image– Photoshop Application– Colorize old photographs

• Perceptually Enhance the Information Content of the Image– Eg. X-ray imaging, scientific illustration

• Minimize Human Labor– No longer need to choose colors from a palette

to color individual components

Visual Idea Representation

+

=

Method

1. Convert 3 dimensional RGB pixel values to 2 dimensional YCbCr color space.

2. Luminance comparison between source (colored) and destination (grayscale) image.

3. Transfer of CbCr chromaticity values.

4. Convert YCbCr color space back into RGB color space.

YCbCr RGB

R´ = 1.164(Y - 16) + 1.596(Cr - 128)

G´ = 1.164(Y - 16) - 0.813(Cr - 128) - 0.392(Cb - 128)

B´ = 1.164(Y - 16) + 2.017(Cb - 128)

Procedures: Preparing the Images

• We took 2 similar 162 x 210 images, one in RGB format and the other in grayscale, and decreased the resolutions to two 8x10 images using Photoshop.

• Although the pictures are not discernable, colorization can still be seen

Procedures: Preparing the Images

• Using Matlab, we used the imread command to read in the coefficients of each image

• The RGB image created an 8x10x3 matrix of coefficients and the grayscale image created an 8x10 matrix of coefficients.

• We used the Matlab program to perform a luminance comparison between the two image files thus creating a new YCbCr image

• The RGB image, for consistent dimensions, was converted into a YCbCrformat with dimensions 8x10.

Procedures: Assembly Programming

• The equations that we implemented in assembly language are:

R´ = 116(Y - 16) + 160(Cr - 128)G´ = 116(Y - 16) - 81(Cr - 128) - 39(Cb - 128)B´ = 116(Y - 16) + 202(Cb - 128).

• We used the MACP function from the DSP chip, which multiplied two numbers and added it to the accumulator

• Syntax for a specific block:

A = #0B = *AR1+B = B - #16AR7 = #operand1*AR7 = BMACP(*AR7,const1,A)

Procedures: Assembly Programming

loop1 ;R´ = 1.164(Y - 16) + 1.596(Cr - 128) and store it into output1A = #0 ;Clears the accumulator B = *AR1+ ;This section will perform the expression, (Y-16)B = B - #16AR7 = #operand1 ;AR7 will hold the address location of operand1nop*AR7 = BnopnopMACP(*AR7,const1,A) ;This will multiply B to const1 (116) and add it the value

;stored in the accumulator

B = *AR3+ ;This will increment the Cr conversion constant matrixB = B - #128 ;This section will perform the expression, 1.596(Cr - 128)AR7 = #operand2 ;AR7 will hold the address location of operand2nop*AR7 = BnopMACP(*AR7,const2,A) ;This will multiply B to const2 (160) and add it the value

;stored in the accumulator

*AR4+ = A ;Set accumulator A to output1and then increment AR4 if (*AR0- != 0) goto loop1 ;If the count is not at 0, go back to loop1

Procedures: Output

• Once we obtained the coefficient values inside the memory, we exported the data onto a dat file, where the values were in hexadecimal format

• The results would be compared numerically and visually

• The values were then copied onto an excel spreadsheet and were converted back into decimal form.

Photoshop YCbCr and luminance comparison Convert to RGB

• Understanding the Matlab Code

• Processing time and memory constraints

• Choosing the suitable Assembly Code

• Writing the Assembly Code

• Trying to figure out what lines of code meant what and trying to figure out why the author chose to write that particular code was a challenge for us.

• For example, we needed to understand which lines of code converted the images into its 3-dimensional RGB form and which lines of code did the luminance comparison.

• Once we got a handle on the matlab code, we realized it was impossible for us to complete the whole process given the amount of time. We decided to cut down on our specifications and only chose a part of the Matlab Code that we were going to imitate on the DSP chip. We decided to focus on changing the image back into its 3-dimensional RGB form from the newly colored YCbCr grayscale image.

• Since the amount of time it took to execute the Matlab code on our images exceeded one minute, we didn’t know how long it would take the DSP to perform similar operations.

• We decided to shrink the image to a more workable size from the 162x210 to a 8x10. It greatly reduced the Matlab processing time to about 10 seconds, which would correlate to a more reasonable job for the DSP chip to carry out.

• We had a difficult time finding out which Assembly code to base our project on.

• We realized that we were basically going to have to write our own code using the similar

format and structure as the assembly code from Experiment A (page 78)

EXP 1EXP 2

COL. TO GRAY

EXP A_______________

__________________________

• Not being fully adept in Assembly, we had a hard time writing some of our code.

• For instance, we needed to perform a subtraction in some of our RGB conversion equations. There were no arithmetic instructions for doing subtraction, multiplication, and then addition.

• After solving our syntax issues, the DSP chip still wasn’t performing any of the algebraic operations correctly. The values obtained from the RGB equations were doubled what was expected.

• We had initialization problems that occurred with the .data and .output.

• We copied over the wrong coefficients from Matlab, and thus we were processing the wrong values on the DSP.

Results - Red

Scaled DSP HEX Values

Scaled DSP Dec Values

Un-scaled DSP Dec Values

Un-Scaled Matlab Values

Scaled Matlab values Difference relative difference % relative difference

1614 5652 56.52 57 5700 48 0.008421053 0.842105263

1CB4 7348 73.48 74 7400 52 0.007027027 0.702702703

2F74 12148 121.48 122 12200 52 0.004262295 0.426229508

39A8 14760 147.6 148 14800 40 0.002702703 0.27027027

2BFC 11260 112.6 113 11300 40 0.003539823 0.353982301

39A8 14760 147.6 148 14800 40 0.002702703 0.27027027

2BB8 11192 111.92 112 11200 8 0.000714286 0.071428571

2478 9336 93.36 94 9400 64 0.006808511 0.680851064

1EDC 7900 79 79 7900 0 0 0

19B4 6580 65.8 66 6600 20 0.003030303 0.303030303

Results - Green

Scaled DSP HEX Values

Scaled DSP Dec Values

Un-scaled DSP Dec Values

Un-Scaled Matlab Values

Scaled Matlab values Difference relative difference % relative difference

1DDC 7644 76.44 77 7700 56 0.007272727 0.727272727

2546 9542 95.42 96 9600 58 0.006041667 0.604166667

378A 14218 142.18 143 14300 82 0.005734266 0.573426573

3.00E+48 15944 159.44 160 16000 56 0.0035 0.35

3C08 15368 153.68 154 15400 32 0.002077922 0.207792208

3.00E+48 15944 159.44 160 16000 56 0.0035 0.35

308D 12429 124.29 125 12500 71 0.00568 0.568

2974 10612 106.12 107 10700 88 0.008224299 0.822429907

24A2 9378 93.78 94 9400 22 0.002340426 0.234042553

21CA 8650 86.5 87 8700 50 0.005747126 0.574712644

Results - Blue

Scaled DSP HEX Values

Scaled DSP Dec Values

Un-scaled DSP Dec Values

Un-Scaled Matlab Values

Scaled Matlab values Difference relative difference % relative difference

1614 5652 56.52 57 5700 48 0.008421053 0.842105263

1CB4 7348 73.48 74 7400 52 0.007027027 0.702702703

2F74 12148 121.48 122 12200 52 0.004262295 0.426229508

39A8 14760 147.6 148 14800 40 0.002702703 0.27027027

2BFC 11260 112.6 113 11300 40 0.003539823 0.353982301

39A8 14760 147.6 148 14800 40 0.002702703 0.27027027

2BB8 11192 111.92 112 11200 8 0.000714286 0.071428571

2478 9336 93.36 94 9400 64 0.006808511 0.680851064

1EDC 7900 79 79 7900 0 0 0

19B4 6580 65.8 66 6600 20 0.003030303 0.303030303

Total Average Error: 0.516981113%

From Matlab:

A Closer Look

Matlab:

DSP:

Future Developments

1. Convert 3 dimensional RGB pixel values to 2 dimensional YCbCr color space.

2. Luminance comparison between source (colored) and destination (grayscale) image.

3. Transfer of CbCr chromaticity values.

4. Convert YCbCr color space back into RGB color space.