engg1015 digital!images! - university of hong kongengg1015/fa11/handouts/04...• basic circuit...

ENGG1015 Digital Images

1st Semester, 2011

Dr Edmund Lam

Department of Electrical and Electronic Engineering

The content in this lecture is based substan1ally on last year’s from Dr Hayden So, but all errors should be blamed on me…

Back to top-‐level

1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 2

Applications

Systems

Digital Logic

Circuits

Electrical Signals

High Level

Low Level

•  Computer & Embedded Systems •  Computer Network •  Mobile Network

•  Image & Video Processing

•  Combinational Logic •  Boolean Algebra

•  Basic Circuit Theory

•  Voltage, Current •  Power & Energy

This week

Back to top-‐level


Applications

Systems

Digital Logic

Circuits

Electrical Signals

High Level

Low Level

•  Computer & Embedded Systems •  Computer Network •  Mobile Network

•  Image & Video Processing

•  Combinational Logic •  Boolean Algebra

•  Basic Circuit Theory

•  Voltage, Current •  Power & Energy

This week

Digital Images


Representation

Processing Hardware

Note: The three parts are mostly independent (even with different “language”), but they do intersect.

Representing Images

bitmap

R G B

pixel

An image is broken down into small regions called picture elements (pixels)

Digital image (bitmap): A pixel-by-pixel representation of an image. Implications?

1st semester, 2011 6 Digital Images - ENGG1015 - Dr. E. Lam

Image Dimensions   Image Size

•  The number of pixel in X-Y direction •  Sometimes quoted using the total number of pixels in a

picture (N megapixels)

  Image Resolution •  The density of pixels •  Measured by pixel-per-inch (PPI) •  NOT the number of pixels

15

14


Representing Pixels   Each pixel is represented by one or more values   Black & white images (binary images):

•  Each pixel is represented by exactly 1 value (B or W) •  1 bit is enough to represent 2 possible values

  Grayscale images: •  Each pixel is usually a byte (8 bits), keeping the brightness

or gray levels   Color images:

•  Each pixel represented a group of color components of that location… often three colors

•  Different color systems: RGB, CMYK, YCbCr, etc   Hyperspectral images:

•  Many values per pixel location, corresponding to different frequencies


Binary and Grayscale Images

  Binary Image

  Each pixel is 1 bit, either 0 or 1

  Dithering is used to produce (fake) different intensities

  Grayscale Image

  Each pixel is usually a byte (8-bit), keeping the brightness or gray levels


B&W B&W (w/ dither) Grayscale

Color Images

  indexed color image   # of color support

depends on the # of bits for each pixel •  4 bits 24=16 colors •  8 bits 28=256 colors

  Color Look-Up Tables (LUTs)

  Color palette

  24-bit color image   Each pixel is

represented by 3 bytes using a certain color model

  Supports 256x256x256 colors •  16 million colors


16 colors 256 colors 16M colors

RGB Color Model   Additive color model   Primary colors: Red, Green,

and Blue   Secondary colors obtained by

additive mixing of primary colors: Cyan, Magenta, Yellow

  Commission Internationale d'Eclairage (CIE) in 1931 specifies red to be 700nm, green to be 546.1nm and blue to be 435.8nm

  Used in media that transmit light (e.g. TV)


CMY Color Model   Subtractive color model   Subtractive primaries:

Cyan, magenta, and yellow

  A subtractive primary absorbs a primary color and reflects the other two •  E.g. Cyan absorbs red and

reflect blues and green

  Used in printing device


Colors that Can be Reproduced


Printing an Image   Print Size

•  Depends on the mapping between printer’s resolution, image resolution & image size

•  A Printer’s printing resolution is usually higher than an image’s resolution because multiple dots of ink are needed to created color of an image pixel

  Color Space •  On screen display: (additive) •  Printing devices: (subtractive)

  Color Production •  Each pixel may have different color •  Each ink drop has only on-off (one bit!)


Dithering   Create the illusion of new colors and shades by

varying the pattern of dots. •  E.g. Newspaper photographs are dithered. If you

look closely, you can see that different shades of gray are produced by varying the patterns of black and white dots. There are no gray dots at all.


Dither, Halftone, Grayscale

original dither halftone


RGB Color Space

  The RGB model describes the formation of color by linearly mixing different portion of “Red”, “Blue” and “Green” light.

 Color is represented by a triplet {r,g,b}, which indicates the weighting coefficients

 We often normalize the coefficients to be between 0 and 1 (inclusive), or integers between 0 and 255 (8-bit).


More Color Models   Both RGB and CMY(K) model specify linear

combinations of the “primaries”   But they have little resemblance to how human

beings reason about colors   E.g. How do you get the RGB values of the

pale orange color on the right?


[R G B] = [204 131 42]

[R G B] = [? ? ?]

[248 215 152]

HS(B/V), HSL, HSI Color Model   The family of HSx models describe colors

similar to how human perceives colors •  Also similar to how painters create colors

 HSB: Hue Saturation Brightness  HSV: Hue Saturation Value  HSL: Hue Saturation Lightness  HSI: Hue Saturation Intensity  Similar, but often comes with confusing (or

even contradicting) definitions


Cylindrical-‐Coordination   Hue:

•  The dominant color •  The angle away

from red   Saturation

•  The amount away from the center

•  How “full” the color is

  Lightness/Brightness/Value •  The amount of

white/black added


Luminance-‐chrominance  Another common alternative: a luminance-

chrominance representation  One value for “luminance” (Y): the

“brightness”, or achromatic image •  Y = 0.2126 R + 0.7152 G + 0.0722 B

 Need two more numbers for the “chrominance” •  YUV and YCbCr

 Why? TV broadcast and digital picture compression…


More Image Representations?  Raster image (bitmap image) - Raster

graphics uses pixel values to describe an image. The file size is independent of the image complexity. For higher resolution, the file size increases dramatically

 Vector graphics (draw graphics) - An alternate approach is to use only instructions for drawing lines, circles, ellipses, curves, and other shapes.


Vector Graphics  Vector-based images are composed of

key points and paths which define shapes, and coloring instructions, such as line and fill colors.

 Example:


Vector Graphics Advantages   Vector graphics can be scaled up and down

easily and quickly while retaining the quality of the picture. Raster images scale poorly and display poorly at resolutions other than that for which the image was originally created.

  Vector graphics require less bandwidth and can be accessed and viewed faster than raster graphics.

  Vector graphics can be edited and manipulated far easier than raster images.


(Partial) Summary   Many decisions to make

  No universally “best” options •  Depends on the physical system, e.g. monitor vs

printer •  Depends on the requirement, e.g. color vs

grayscale

  Intersects with other fields •  e.g. Psychology (visual science)

  How about our video chats problem?


Image Processing

 Used in digital camera, TV, cell phones…

 Used in all kinds of photo editing SW •  e.g. Photoshop, GIMP…


Image Processing -‐ Examples

Original Grayscale

Blur Edge Detection


RGB to Grayscale Conversion  Each pixel of a grayscale image has only one

intensity value, V

 High V: white, Low V: black

 Easiest conversion:

 Produce better result if you weight G and R more than B •  Human eyes are more sensitive to green and red €

V =R +G + B

3


Basic Filtering: Windowing   Filters are building blocks of image processing

systems   One of the most basic filtering method is by

windowing

€

y[r,c] =1h[i, j]

i, j∑

h[i, j]x[r + i,c + j]j=−1

1

∑i=−1

1

∑

r

c 1st semester, 2011 30 Digital Images - ENGG1015 - Dr. E. Lam

Windowing in Action

12 8 27 26 54 48 14 9 16 8 29 9 3 11 10 15 50 60 8 12 34 2 29 52

17 2 44 35 56 72 22 39 43 34 63 77

1 2 1 2 4 2 1 2 1

9 16 8 11 10 15 12 34 2

1 × + 2 × + 1 × 2 × + 4 × + 2 × 1 × + 2 × + 1 ×

+ + = 14

14 19

16 8 29 10 15 50 34 2 29

19

X H Y

34

34 8 29 9 15 50 60 2 29 52

16


Gaussian Blur   A simple but effective way to blur a picture   Each pixel is replaced with a weighted sum of the

values of its surrounding pixels   The weighting factors have a Gaussian distribution,

thereby the name   Intuitively: each pixel is mixed to certain extent with

its neighbors


2 4 5 4 2

4 9 12 9 4

5 12 15 12 5

4 9 12 9 4

2 4 5 4 2

“Generalizing” Windowing   So far we are only doing (weighted) average of pixel

values within a window •  A linear technique •  How about a nonlinear technique, e.g. taking median?

Actually that’s very useful •  Which is faster, mean or median?

  People often “flip” the filters by 180 degrees:

•  We “cheated”, because our filters were symmetric

  Why? Because it links us to a signal processing technique called convolution •  Extensive body of knowledge, allowing us to know and

compare the effects of these filters for different weights


€

y[r,c] =1h[i, j]

i, j∑

h[i, j]x[r − i,c − j]j=−1

1

∑i=−1

1

∑

Edge Detection   Useful in understanding an image

•  For robot, face recognition, medical imaging etc

  In a smooth contour, the pixel values usually do not change rapidly

  However, the pixel exhibit sudden jump in values near an edge •  E.g. jump from 1 to 130

  Sobel edge detection is one of the simplest algorithms that makes use of this observation to find edges •  Compares values of the neighbors of pixel


Sobel Tilter

  A Sobel filter combines the results of the two weight matrices

  Each filter kernel estimates gradient in x and y direction from the input image.

  Magnitude of the resulting pixel in matrix D is:


+1 0 -1

+2 0 -2

+1 0 -1

+1 +2 +1

0 0 0

-1 -2 -1

€

D[r,c] = Dx2[r,c]+ Dy

2[r,c]

From convolving with Hx From convolving with Hy

Hx Hy

Sobel Filter Example – x dir

3 3 3 39 39 39 39 3 3 3 40 40 40 40 3 3 3 41 41 41 41 3 3 3 42 42 42 42 3 3 3 41 41 41 41 3 3 3 40 40 40 40 3 3 3 39 39 39 39

-1 0 +1 -2 0 +2 -1 0 +1

3 3 3 3 3 3 3 3 3

-1 × + 0 × + 1 × -2 × + 0 × + 2 × -1 × + 0 × + 1 ×

+ + = 0

0 152

3 3 40 3 3 41 3 3 42

152

S Gx D

152

152 3 40 40 3 41 41 3 42 42

flipped

0

0 40 40 40 41 41 41 42 42 42


Sobel Filter Example – x dir

  Result in D shows a clear line at the edge   Note that Gx is a flipped version of Hx

  Some more normalization has to be done in actual implementation

3 3 3 39 39 39 39 3 3 3 40 40 40 40 3 3 3 41 41 41 41 3 3 3 42 42 42 42 3 3 3 41 41 41 41 3 3 3 40 40 40 40 3 3 3 39 39 39 39

-1 0 +1 -2 0 +2 -1 0 +1

0 0 152 152 0 0 0

0 0 152 152 0 0 0

0 0 152 152 0 0 0

0 0 152 152 0 0 0

0 0 152 152 0 0 0

0 0 152 152 0 0 0

0 0 152 152 0 0 0

S Gx D flipped


Image Processing Summary   Image processing is the task of manipulating

the image by mathematical means to achieve high level requirements

  Common operations: filtering   Many other operations:   E.g. Image forensic, Lithography, medical

imaging, automatic image diagnosis, robot control, etc…

  What’s the “(computational) cost” of various image processing algorithms?

  What sort of image processing operations do we need in our video chats?


Digital Cameras   Resolution measured in pixels H x V

  Image sensing: charge coupled device (CCD) or complementary metal-oxide semiconductor (CMOS)

  Megapixels is used to denote the total max pixels in the image •  E.g. 5 Megapixel - in the 2520 by 1890 and higher pixel

range. Photo quality 11 x 14 prints from this class of camera.

  Comparing film cameras to digital cameras is difficult since resolution is measured differently


Taking Pictures 1. Image focused by lens 2. Image captured on CCD 3. CCD generates analog

representation of image 4. Analog signal converts to

digital 5. Digital signal processing

(DSP) adjust quality, etc Step 5

Step 4

Step 3

Step 1

Step 2


Marketing Caveats  Q: For digital cameras, higher

“megapixel” value always produce better photos?

 A: Not really. If you will only look at the photos on websites, or will only print them on 3R papers, you don’t need all the pixels from a 10M pixels camera.


Area You Ready?


Flat Panel TVs and Monitors   Pictures displayed as matrix of pixels on screen   Two major technologies for generating picture

•  Plasma •  Liquid Crystal Display (LCD)

  Plasma •  Neon-Xenon gas trapped between two glasses •  When electrically charged, each pixel display red,

blue or green color.   LCD

•  Liquid crystal between glasses pass/block light depending on electrical signal

•  Pass corresponding backlight


LED TVs?  Misleading term

 Proper name: LED-backlight LCD TVs

 Use the same LCD display technology as all other “LCD displays”.

 Most other standard “LCD displays” use cold cathode fluorescent light (CCFL) for backlight


3 Characteristic Dimensions  Panel Size •  The physical dimension of the panel •  A 42” panel has a diagonal measurement of

42”  Display Resolution •  The number of picture-elements (pixels) along

each X-Y direction  Dot Pitch •  The distance between two pixel of the screen

Panel Size = Display Resolution * Dot Pitch


Standard Display Resolutions


Marketing Caveats  Q: For flat panel TVs, a bigger screen

always produce better display than a smaller screen?

 A: Not really. It depends on the distance you will be watching the TV and the TV source signal.


More Pixel = Good?   Human eye can identify 120 pixels per degree

of visual arc •  i.e. if 2 dots are closer than 1/120 degree, then our

eyes cannot tell the difference   At a distance of 2m (normal distance to a TV)

our eyes cannot differentiate 2 dots 0.4mm apart.

  Closer to TV => easier to differentiate pixels   Far away => cannot tell the difference

screen

Minimum: 2 arc minute 1st semester, 2011 49 Digital Images - ENGG1015 - Dr. E. Lam

Image courtesy of www.carltonbale.com 1st semester, 2011 50 Digital Images - ENGG1015 - Dr. E. Lam

Source: http://www.diamond-vision.com/quad_dot_pattern.asp

True LED displays  Each pixel is a

LED

 Used mostly in outdoor, large- scale displays


Dallas Cowboys Stadium Sideline Display 48.64m x 21.76m Pixel Pitch: 20mm Displays World’s Largest High-Definition Video Display

Hong Kong Shatin Racecourse 70.4m x 8m World’s Longest TV screen


In Conclusion…  Digital signal processing is a very broad

field within EEE  The processing of digital image is a

good example of high-level applications that run on digital signal processing systems.

 To display and process digital images correctly, you need the right combination of image representation, hardware, and processing power.


Homework 1  Homework 1 is out •  Due 14 Oct, 2011 •  5pm •  Turn in physical copy of your answer •  Homework boxes near Room 712, Chow

Yei Ching Building

  Individual homework  Good way to study for final  Zero tolerance on plagiarism


engg1015 digital!images! - university of hong kongengg1015/fa11/handouts/04...• basic circuit...

Documents