The Art of Digital Image processing

C. S. TongDepartment of MathematicsHong Kong Baptist University

No, they're both the same size

Is the left center circle bigger?

It's a spiral, right?

No, these are a bunch of independent circles

Keep staring at the black dot. After a whilethe gray haze around it will appear to shrink.

Can you find the dog?

How many colors do you see?

There are only 3 colors: White, green, and pink.There seem to be two different shades of pink,but there is only one pink.

Count the black dots! :o)

Are the horizontal lines parallel or do they slope?

Do you see a musician or a girl's face?

Do you see the face? Or an Eskimo?

Do you see a cube missing a corner?Or do you see a small cube in a big one?

Is the blue on the inner left back or the outer left front?

What is a digital image?• A digital image is just a 2D array of

picture elements (pixels)

What is a digital image?

• Each pixel is associated with a number which represents its intensity or brightness

• Usually allow up to 256 levels of brightness (so called 8-bit images)– how many levels do you think you can

distinguish?

Effects of Quantization

Effects of changing intensity resolution

8-Bit image7-Bit image6-Bit image5-Bit image4-Bit image3-Bit image2-Bit image1-Bit image

Effects of Quantization

The demo showed that the human eye can only resolve about 20-30

grey levels

What is a digital image?

• The density of pixels significantly affect the quality of the image

• A typical scanner or digital camera has a resolution of about 600 dpi (or about 1 million pixels per picture)

• By comparison, the human eye has a resolution of about 10,000 dpi (or 100 million cone cells)

Effects of Quantization

Effects of changing spatial resolution

Effects of Quantization

Can be used for concealing identify

What is a digital image?

• Colour can be represented by three primary colour components: Red, Green and Blue 24-bit RGB images

• For special editing effects such as transparency, some image formats support 32-bit RGB- , the additional 8-bit describes the channel

• Video is just a sequence of images. Frame rate of over 24 pictures per second is often sufficient

What is a digital image?

• A more efficient image format for representing colours is the Index Image Format

• All the distinct colours that appear in an image are stored in a file called the colormap

• The colour image is now an array of indices, each of which specify the color of that pixel as the corresponding colour in the colormap

Editing Colormap

X=[1 2 1, 2 2 2, 1 2 1];Map=[0.9 0.1 0.1, 0.1 0.0 0.8];

Change color 1 from redto green, i.e.Map=[0.1 0.9 0.0, 0.1 0.0 0.8];

Editing Colormap

Chroma-keying

• The idea of editing the colormap can be used for many movie effects

• Take pictures of an actor in front of a blue screen

• Edit the colormap and make the blue color transparent

• Overlay the pictures to a desired background

Chroma-keying

Map the black background to the Tsing Ma Bridge

Editing Colormap

Convert image to black and white imageIncrease intensity in

the Blue component

Increase intensity in the Red component

Digital Negative

Histogram

Contrast Stretching

Original image Contrast adjustedContrast adjusted plus cropping

Histogram Equalization

Original image Histogram EqualizedContrast Adjusted

Median Filtering

Original Image 5% Binary NoiseMF (3-by-3)

Median Filtering

20% Binary Noise 50% Binary NoiseMF (3-by-3)MF (3-by-3)

Independent Component Analysis

Original Image Noisy ImageDenoising using ICA

Edge Detection

Original Noise (0.05)Sobel Noise (0.01)Sobel SobelLaplacian Laplacian Laplacian

High-boost Filter

Original Low-passed High-passedHigh-boost

Fourier Transform

Spatial Domain

FlogF

Frequency Domain

Ghost-buster

Ghost appears Ghost removed

Image Degradation

Blurred PhotoPerfect Photo

Image Restoration

Original Image • Motion Blurred ImageBlur removed using Wiener Filter (nsr=0.05)

Image Restoration

• Blur removed using nsr=0.1

WF restorations CST restorations Original

• Blur removed using nsr=0.05• Blur removed using nsr=0.01• Blur removed using nsr=0.005Blur removed using nsr=0.001

Result

Blurred ImageBlurred Image Blurred ImageBlurred ImageRestored ImageRestored ImageRestored ImageRestored Image

Other Blurring Function

Horizontal BlurredHorizontal Blurred Vertical BlurredVertical BlurredRestored ImageRestored ImageRestored ImageRestored Image

How to Recognize Shapes?

After appropriate translation, rotation , and scaling, we can now see the two shapes are the same!

How to Recognize Shapes?

After all possible translation, rotation , and scaling, we can now see the two shapes are not the same!

Pattern Recognition: Overview

• Each pattern to be related to a set of features (feature vector)

• Distinguish a set of patterns by some measure of distance between feature vectors

Feature Extraction• This is the most crucial part of a

recognition system • Usually prefer features which are

invariant to translation, rotation and scaling

• Standard approach include: statistical moments and PCA

• Very much context-dependent

A small scale illustration

Patterns Features

Apples, Lemons Colour

+ Melons

+ Bananas

+ Oranges, Grape Fruits... Texture...

Shape

Size

Complexity

• Clearly, as the set of patterns grows, the

number and complexity of the features

grow

• There may not be any suitable

distinguishing featuresSometimes I can’t even read my own hand writing!

Chinese Character Recognition

• There are over 20,000 Chinese Characters• Although not all are in common usage, at

least 5,000 are needed in most applications

• Chinese Characters come in many font types

Chinese Character Recognition

• For fixed font character recognition, each character is represented by a N-by-M binary matrix (typically 24-by-24)

• Or equivalently, a character is a 576 dimensional vector

• Noise in scanning is modelled by bit-reversal (so called binary noise)

Chinese Character Recognition

• Handwriting is much more difficult: no natural representation available

• A character involves combining a number of elementary strokes in two spatial dimension

• Large variation in writing styles

Effects of Noise

• Binary noise at 0, 2, 4, 6, 8, 10% level

Regional decomposition

• Partition the character into 9 sub-regions and extract the mean intensity of each sub-region

• This yields a 9-component feature vector describing the local distribution of “ink” or weight of the character

Projection Codes

• Stroke information is obtained by projecting the character onto the horizontal and vertical axes

• Each projection profile is divided into 3 parts; and the maximal projection value in each part is extracted to give a 6-component feature vector describing the (global) distribution of strokes

Projection Codes

Local & Global Features

• The local and global features are combined into a 15-component code vector

• Further transformed to enhance the entropy of the code so as to improve the discriminating power of the codes

Why Max-Entropy?

• A code value that occurs frequently is useless as it does not discriminate

• A code value that occurs infrequently is highly discriminative but only when it occurs (which is rare!)

• Thus the most discriminative code is one with a uniform distribution

Results (Local & Global Features)

• Order-N recognition means the character is correctly identified as one of the top N matches

Size ofcharacter Set

Order Recognitionrate

1000 10 99.80%

2000 20 99.90%

5000 1 83.42%

5000 50 99.84%

10% Binary Noise

Results (Local & Global Features)• Not good for outright recognition (low

order-1 recognition rate)• Quite good for classifying the

character as belonging to a small group of characters (because the higher order recognition rates rapidly converge to 1 as order increases)

ICA: Character Recognition

• Use ICA to remove noise from noisy input

• Compare processed character from characters in the dictionary

• Identify the character as the one with the best match (1-norm)

ICA: Character Recognition

• Good recognition even for very high noise level

• The Bell & Sejnowski implementation is too slow as it involves the inverses of large matrices, especially when the dictionary is large

Two-Stage Approach

• Use local & global features in stage 1 to reduce the effective dictionary to a much smaller set

• Use ICA in stage 2 to complete the identification of the character

• Identify noisy input with the character in the dictionary with the best match (1-norm)

Results (2-Stage Approach)

Noise LevelTest No. 3% 7% 10% 15%

1 99.5 99.5 99.5 99.52 99.5 100 99.5 97.53 99.5 98.5 100 99.54 100 99.5 98.5 99.55 98.5 99.5 99.5 98.56 99.5 99.5 99 99.57 99.5 100 99 998 99.5 98.5 98.5 98.59 100 98.5 97.5 99

10 98.5 99.5 99.5 97.5

Mean 99.4 99.3 99.05 98.8StandardDeviation

0.5164 0.5869 0.7246 0.7888

Results (benchmark)

Noise LevelTest No. 3% 7% 10% 15%

1 98.5 98 99 972 98 97.5 98.5 98.53 98.5 98.5 99 994 97.5 98.5 99 98.55 99 98.5 96.5 1006 99.5 98 97.5 98.57 98.5 98.5 100 98.58 98 96.5 98 98.59 99 98.5 98 9810 97.5 98.5 99 96.5

Mean 98.4 98.1 98.45 98.3StandardDeviation

0.6583 0.6583 0.9846 0.9775

Results

97

97. 5

98

98. 5

99

99. 5

3 7 10 15Noi se Level / %

Reco

gnit

ion

Rate

/%

Norm-1 DP- I CA

MORPHING EFFECTS

Start Middle: interpolation

End

MORPHING EFFECTS

Start: Human Head

Middle: morph End: Orangutan

MORPHING EFFECTS

Start: Lion Middle: morph End: Horse

See website http://graphics.stanford.edu/cgi-bin/alumni/tolis/personal/getpage.cgi?

morph.html

MORPHING EFFECTS

See website http://www.cs.wisc.edu/computer-vision/projects/interp/interp.html

View morphing

End of Presentation