Introduction to Image Processing

CS474/674 – Prof. Bebis

What is Image Processing?

• Manipulation of digital images by computer.

• Image processing focuses on two major tasks:– Improvement of pictorial information for human

interpretation and high level processing.

– Processing of image data for storage and transmission.

Related Areas

• Image Processing

• Computer Vision

• Computer Graphics

Image Processing

Image Processing

• Image Enhancement

Image Processing (cont’d)

• Image Restoration

Image Processing (cont’d)

• Image Compression

Computer Graphics

Computer Graphics

Image

Output:

Geometric ModelsSyntheticSyntheticCameraCamera

Projection, shading, lighting models

Computer Vision

Computer Vision

Model

Output:

Real Scene

Cameras Images

Applications: Image Enhancement

• One of the most common uses of IP techniques: improve quality, remove noise etc

Applications: Space

• Launched in 1990 the Hubble telescope can take images of very distant objects

• An incorrect mirror made many of Hubble’s images useless

• Image processing techniques were used to fix this!

Applications: Medicine

• Take slice from MRI scan of a dog’s heart, and find boundaries between different types of tissue

– Image with gray levels representing tissue density

– Use a suitable filter to highlight edges

Original MRI image of a dog’s heart Edge detection image

Applications: GIS

• Geographic Information Systems– Digital image processing techniques are used extensively to

manipulate satellite imagery.

meteorologyterrain classification

Applications: Industrial Inspection

• Human operators are expensive, slow and unreliable

• Make machines do thejob instead!

• Industrial vision systems are used in all kinds of industries

Applications: Law Enforcement

• Image processing techniques are used extensively by law enforcers

Fingerprint recognitionNumber plate recognition for speed cameras or automated toll systems

Examples: HCI

• Make Human Computer Interaction (HCI) more natural– Face recognition

– Gesture recognition

Key Stages in Digital Image Processing

Image Acquisition

Image RestorationMorphological

Processing

Segmentation

Object Recognition

Image Enhancement

Representation & Description

Problem Domain

Colour Image Processing

Image Compression

Image Acquisition

Image Acquisition

Image RestorationMorphological

Processing

Segmentation

Object Recognition

Image Enhancement

Representation & Description

Problem Domain

Colour Image Processing

Image Compression

Image Enhancement

Image Acquisition

Image RestorationMorphological

Processing

Segmentation

Object Recognition

Image Enhancement

Representation & Description

Problem Domain

Colour Image Processing

Image Compression

Image Restoration

Image Acquisition

Image RestorationMorphological

Processing

Segmentation

Object Recognition

Image Enhancement

Representation & Description

Problem Domain

Colour Image Processing

Image Compression

Morphological Processing

Image Acquisition

Image RestorationMorphological

Processing

Segmentation

Object Recognition

Image Enhancement

Representation & Description

Problem Domain

Colour Image Processing

Image Compression

Segmentation

Image Acquisition

Image RestorationMorphological

Processing

Segmentation

Object Recognition

Image Enhancement

Representation & Description

Problem Domain

Colour Image Processing

Image Compression

Representation & Description

Image Acquisition

Image RestorationMorphological

Processing

Segmentation

Object Recognition

Image Enhancement

Representation & Description

Problem Domain

Colour Image Processing

Image Compression

Object Recognition

Image Acquisition

Image RestorationMorphological

Processing

Segmentation

Object Recognition

Image Enhancement

Representation & Description

Problem Domain

Colour Image Processing

Image Compression

Image Compression

Image Acquisition

Image RestorationMorphological

Processing

Segmentation

Representation & Description

Image Enhancement

Object Recognition

Problem Domain

Colour Image Processing

Image Compression

Color Image Processing

Image Acquisition

Image RestorationMorphological

Processing

Segmentation

Representation & Description

Image Enhancement

Object Recognition

Problem Domain

Color Image Processing

Image Compression

How are images represented in the computer?

Color images

A Simple model of image formation

What is (visible) light?

• The visible portion of the electromagnetic (EM) spectrum.– Approximately between 400 and 700 nanometers.

Examples: Gama-Ray Imaging

Gamma-ray imaging: nuclear medicine and astronomical observations

Examples: X-Ray Imaging

X-rays: medical diagnostics, industry, and astronomy, etc.

Examples: Ultraviolet Imaging

Ultraviolet: industrial inspection, microscopy, lasers, biological imaging,and astronomical observations

Examples: Infrared Imaging

Infrared bands: light microscopy, astronomy, remote sensing, industry, and law enforcement.

Sonic images

• Produced by the reflection of sound waves off an object.

• High sound frequencies are used to improve resolution.

Range images

• Can be produced by using laser range-finders.

• An array of distances to the objects in the scene.

Image formation

• There are two parts to the image formation process:

– The geometry of image formation, which determines where in the image plane the projection of a point in the scene will be located.

– The physics of light, which determines the brightness of a point in the image plane as a function of illumination and surface properties.

Pinhole camera• This is the simplest device to form an image of a 3D scene

on a 2D surface.

• Straight rays of light pass through a “pinhole” and form an inverted image of the object on the image plane.

fXx

Z

fYy

Z

Camera optics

• In practice, the aperture must be larger to admit more light.

• Lenses are placed in the aperture to focus the bundle of rays from each scene point onto the corresponding point in the image plane

Physics of Light

f(x,y)=i(x,y)r(x,y)

where

1) i(x,y) the amount of illumination

incident to the scene

2) r(x,y) the reflectance from the object 1),(0

),(0

yxr

yxi

CCD (Charged-Coupled Device) cameras• Tiny solid state cells convert light energy into electrical

charge.

• The image plane acts as a digital memory that can be read row by row by a computer.

Frame grabber• Usually, a CCD camera plugs into a computer board (frame

grabber).

• The frame grabber digitizes the signal and stores it in its memory (frame buffer).

Image digitization

• Sampling means measuring the value of an image at a finite number of points.

• Quantization is the representation of the measured value at the sampled point by an integer.

Image digitization (cont’d)

0

255

Image digitization (cont’d)

2D example

Effect of Image Sampling original image sampled by a factor of 2

sampled by a factor of 4 sampled by a factor of 8

Effect of Image Quantization 256 gray levels (8bits/pixel) 32 gray levels (5 bits/pixel) 16 gray levels (4 bits/pixel)

8 gray levels (3 bits/pixel) 4 gray levels (2 bits/pixel) 2 gray levels (1 bit/pixel)

Representing Digital Images

The result of sampling and quantization is a matrix of integer numbers.

Here we have an image f(x,y) that was sampled to produce M rows and N columns.

)1,1(

...

...

)1,0(

...)0,1(

......

...)1,1(...

...)1,0()0,0(

),(

NMf

Nf

Mf

f

ff

yxf

Representing Digital Images (cont’d)

• There is no requirements about M and N

• Usually L= 2k

• Dynamic Range : [0, L-1]

The number of bits b required to store an image:

b = M x N x k where k is the number of bits/pixel

Image file formats

• Many image formats adhere to the following simple model:– Header– Data (line by line, no breaks between lines).

Image file formats (cont.)

• Header contains at least:– A signature or “magic number” (i.e., a short sequence

of bytes for identifying the file format).– The width and height of the image.

Common image file formats

• PGM (Portable Gray Map)• PNG (Portable Network Graphics)• GIF (Graphic Interchange Format) – • JPEG (Joint Photographic Experts Group)• TIFF (Tagged Image File Format)• FITS (Flexible Image Transport System)

PGM format

• A popular format for grayscale images (8 bits/pixel)• Closely-related formats are:

– PBM (Portable Bitmap), for binary images (1 bit/pixel)

– PPM (Portable Pixelmap), for color images (24 bits/pixel)

• ASCII or binary (raw) storage

ASCII

Raw

Image Class class ImageType { public: ImageType(); // constructor ~ImageType(); // destructor void getImageInfo(int&, int&, int&); void setImageInfo(int, int, int); void setVal(int, int, int); void getVal(int, int, int&); // more functions ...

private: int N, M, Q; //N: # rows, M: # columns int **pixelValue; };

Input / Output Functions

• C++ routine to read the header of a PGM image:

ReadImageHeader.cpp

• C++ routine to read a PGM image:

ReadImage.cpp

• C++ routine to write a PGM image:

WriteImage.cpp

An example - Threshold.cpp void readImageHeader(char[], int&, int&, int&, bool&); void readImage(char[], ImageType&); void writeImage(char[], ImageType&);

void main(int argc, char *argv[]) { int i, j, M, N, Q; bool type; int val, thresh;

// read image header readImageHeader(argv[1], N, M, Q, type);

// allocate memory for the image array ImageType image(N, M, Q);

Threshold.cpp (cont’d) // read image readImage(argv[1], image);

cout << "Enter threshold: "; cin >> thresh;

// threshold imagefor(i=0; i<N; i++)

for(j=0; j<M; j++) { image.getVal(i, j, val); if(val < thresh) image.setVal(i, j, 0; else image.setVal(i, j, 255); }

// write image writeImage(argv[2], image); }

Reading/Writing PGM images

(1D array of unsigned char)

(1D array of unsigned char)

(2D array of int)

(2D array of int)

Use “write”

Use “read”

Writing a PGM image to a file void writeImage(char fname[], ImageType& image) int N, M, Q; unsigned char *charImage; ofstream ofp;

image.getImageInfo(N, M, Q); charImage = (unsigned char *) new unsigned char [M*N];

// convert integer values to unsigned char int val; for(i=0; i<N; i++) for(j=0; j<M; j++) image.getVal(i, j, val); charImage[i*M+j]=(unsigned char)val; }

Writing a PGM image... (cont’d) ofp.open(fname, ios::out | ios::binary); if (!ofp) { cout << "Can't open file: " << fname << endl; exit(1); } ofp << "P5" << endl; ofp << M << " " << N << endl; ofp << Q << endl; ofp.write( reinterpret_cast<char *>(charImage), (M*N)*sizeof(unsigned char));

if (ofp.fail()) { cout << "Can't write image " << fname << endl; exit(0); } ofp.close();

}

Reading a PGM image from a file void readImage(char fname[], ImageType& image) { int i, j; int N, M, Q; unsigned char *charImage; char header [100], *ptr; ifstream ifp;

ifp.open(fname, ios::in | ios::binary); if (!ifp) { cout << "Can't read image: " << fname << endl; exit(1); }

Reading a PGM image from a file

// read header ifp.getline(header,100,'\n'); if ( (header[0]!=80) || // 'P' (header[1]!=53) ) { // '5' cout << "Image " << fname << " is not PGM" << endl; exit(1); }

ifp.getline(header,100,'\n'); // skip comments while(header[0]=='#') ifp.getline(header,100,'\n');

M=strtol(header,&ptr,0); // read M, N N=atoi(ptr);

Reading a PGM image …. (cont’d)

ifp.getline(header,100,'\n'); Q=strtol(header,&ptr,0);

charImage = (unsigned char *) new unsigned char [M*N];

ifp.read( reinterpret_cast<char *>(charImage), (M*N)*sizeof(unsigned char));

if (ifp.fail()) { cout << "Image " << fname << " has wrong size" << endl; exit(1); }

ifp.close();

Reading a PGM image…(cont’d)

// Convert unsigned characters to integers int val;

for(i=0; i<N; i++) for(j=0; j<M; j++) { val = (int)charImage[i*M+j]; image.setVal(i, j, val); }

}

How do I “see” images on my computer?

• Unix/Linux: xv, gimp

• Windows: Photoshop

Irfanview

How do I convert an image from one format to another?

• Use “Save As” option

More Information on Image Processing and Computer Vision

• Computer Vision Home Page http://www.cs.cmu.edu/afs/cs/project/cil/ftp/html/vision.html

• UNR Computer Vision Laboratory http://www.cs.unr.edu/CVL