image analysis and computer vision. orientation basis definition and history image processing basics...

90
IMAGE ANALYSIS AND COMPUTER VISION

Upload: mervyn-pitts

Post on 12-Jan-2016

230 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

IMAGE ANALYSIS AND COMPUTER VISION

Page 2: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Image processing basic steps

1. Image Enhancement

2. Image Restoration

3. Image Analysis

4. Image compression

5. Image Synthesis

Page 3: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Goal in image analysis• Image analysis operations are used in applications, that

require the measurement and classification of image information

• Examples:• Cell recognition from tissue sample• Object recognition from conveyor belt• Zip code reading from envelope

Page 4: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing
Page 5: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Group discussion• List application possibilities for image analysis!

Page 6: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Image analysis

• Basis is visual image, whose content should be interpreted

• As a result mostly non-image data• As a goal is to understand images content classifying its content

Page 7: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Example: Robot vision

Page 8: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Example: Robot vehicle

Page 9: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Example: Traffic analysis

Page 10: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Image analysis operations

Page 11: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Image analysis operations• Segmentation

– operation that highlights individual objects within an image.

• Feature Extraction– after segmentation->measure the individual features of each

object

• Object Classification– classify the object to particular category

feature extraction

classification

segmentspace

distinctobjects

featurespace

feature

classificationspace

class

Segmentation

image

Page 12: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Segmentation operations

• Image Preprocessing• Initial Object Discrimination• Image Morphological Operations

feature extraction

classification

segmentspace

distinctobjects

featurespace

feature

classificationspace

class

Segmentation

image

Page 13: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Image Preprocessing

• In preprocessing eg. change images contrast, filter noise and remove distracting image background

• Image enhancement operations is used

Page 14: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing
Page 15: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Initial Object Discrimination

• Separates image objects into rough groups with like characteristics using image enhancement operations

• Outlining and contrast enhancement often used

Page 16: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Initial object discrimination - example

Original Binary contrast enhanced

Page 17: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Initial object discrimination - example• Sobel edge-enhancement

1 Filter with horizontal mask2 Filter with vertical mask 3 Add

-1 0 1 -1 -2 -1

-2 0 2 0 0 0

-1 0 1 1 2 1

Page 18: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Morphological processing

• In preprocessed image the boundaries are very rough-> need to “clean up”

• Morphological operations

Page 19: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Morphological operations• Binary operations

–Erosion and dilation –Opening and Closing–Outlining–Skeletonization

• Gray-scale operations–Top-Hat and Well transformations–Morphological gradient–Watershed edge detection

Page 20: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Binary morphology• Focus on two brightness values. black=0, white=255• Technically same as spatial convolution• combines pixel brightness with a structuring element,

looking for specific pattern• Array of logical values• (cut=AND, union=OR, complement=NOT)

Page 21: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing
Page 22: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Binary morphology - equation

O(x,y) = 0 or 1 (predefined) if X =I (x,y) ANDX0 =I (x+1,y) ANDX1 =I (x+1,y-1) ANDX2 =I (x,y-1) ANDX3 =I (x-1,y-1) ANDX4 =I (x-1,y) ANDX5 =I (x-1,y+1) ANDX6 =I (x,y+1) ANDX7 =I (x+1,y+1)otherwise, O(x,y) = opposite state

Page 23: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Erosion• Reduces the size of the objects in relation to their

background

Mask 1 1 1 O(x,y) = 1 if “Hit”

1 1 1

1 1 1 = 0 if “Miss”

1 1 1

Page 24: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Example (1/2)

Original Binary contrast enhanced

Page 25: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Example (2/2)

Eroded imageBinary contrast enhanced

Page 26: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Dilation• Uniformly expands the size of object

Mask 0 0 0 O(x,y) = 0 if “Hit”

0 0 0 = 1 if “Miss”

0 0 0

Page 27: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Example

Binary contrast enhanced

Dilated image

Page 28: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Opening• Erosion then Dilation• Removes one pixel mistakes like erosion• Object size remains

Binary contrast enhanced

Erosion Dilation

Page 29: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Closing• Dilation + erosion• Fills pixel wide holes• Object size remains

Binary contrast enhanced

Dilation Erosion

Page 30: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Cleaning

Original binary contrast enhanced image

Opening Cleaned = Opened and Closed

Page 31: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Outlining• Forms one-pixel-wide outlines and tends to be more

immune to image noise than most edge enhancement operations

• Implementation: • Eroded image subtract from original

Page 32: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Outlining

Original Binary contrast enhanced

Page 33: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Outlining

Eroded image Original - Eroded image

Page 34: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Skeletonizing• Make “wireframe” model from image • Uses different erosion masks• Analogy: fire, which burns object from each side

Page 35: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Gray-scale operations

• Used when binary operations degrade an image• Gray-scale operation can be followed binary operation • Mask terms -255 ... 255 or “Don’t care”

Page 36: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing
Page 37: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Erosion and Dilation• Erosion reduces the size of objects by darken the bright

areas in image• Dilation is inverse operation

Page 38: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Erosion and Dilation example

Erosion

Dilation

Original

Page 39: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Opening and Closing• Opening = erosion + dilation• Opening reduces noise pixels• Closing = dilation + erosion• Closing fills one-pixel-wide holes

Page 40: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Opening example

Page 41: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Opening - example 2

Page 42: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Morphological gradient

Images outlines as a result Make copy from image. Erosion to other

image and dilation to other. Then images subtract from each other using a dual-image point process.

Original

Eroded

Dilated

Eroded - dilated =Gradient

Image

Page 43: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Morphological gradient

Original Erosion

Dilation Gradient=Erosion-Dilation

Page 44: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Feature Extraction

• Operation followed by segmentation• Choose essential features and measure them from

objects• Goal is to find features, which help find out object’s

class easier

feature extraction

classification

segmentspace

distinctobjects

featurespace

feature

classificationspace

class

Segmentation

image

Page 45: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Features• Brightness and color• Texture• Shape• Spatial moments• Edge shape

Page 46: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Pornographic image analysis

Page 47: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing
Page 48: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Feature: Brightness and color

Histogram can show

• Color (sorting by colors)• Brightness

• average brightness• Standard deviation brightness• mode brightness• sum of all pixel brightnesses<-> energy (zero-order spatial

moment)

Page 49: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Example: Fruit sorting

Problem:• Boxes goes on conveyor belt, which has green apples

(Granny Smith) and red apples (Red Delicious) and also oranges

• Sort boxes to the correct follow on conveyor belts automatically

Page 50: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Example: Fruit sorting

Solution:• Capture image with camera to RGB images• Convert RGB to HSL• Explore Hue color component which fruit it is

Page 51: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

RGB HSL

Page 52: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Feature: Texture• Images spatial frequency shows is the one question

smooth texture (low frequency) or coarse texture (high frequency)

Page 53: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Texture features - meters• High-pass filter• Fourier -transform• Standard deviation of the brightnesses (big deviation =

coarse texture, small deviation = smooth texture)

Page 54: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Feature: Shape• Most common object measurement method• Objects physical measures • The goal is to use fewest necessary measure methods• Often standard shapes, square, circular, elliptical• Even the weirdest shapes can be measured

Page 55: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Shape measures

• Perimeter• Area (pixels)• Area to perimeter ratio

Roundness=(4*Area)/Perimeter²

Value between 0 and 1.

Page 56: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Shape measures• Major axis (x1,y1 and x2,y2)• Major axis length sqr((x2-x1)²+(y2-y1)²)• Major axis angle

Page 57: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Shape measures• Minor axis• Minor axis width• Minor axis width to Major axis length ratio

Value between 0 (elongated) and 1 (circle)

Page 58: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Shape measures• Bounding box area

= Major axis length * Minor axis length

• Number of holes• Total hole area• Total hole area to object area ratio

Value between 0 (no holes) and 1 (entirely a hole)

Page 59: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Feature: Spatial moment• Statistical shape measures• Zero-order spatial moment

Sum of object’s pixel brightness.

Same as object’s area in binary image and

object’s ‘energy’ in gray-scale images

Page 60: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Spatial moment• First-order spatial moment

m

xsum = i * xi i=0

where

i = pixel’s x-coordinate xi = pixel’s value at im= image’s horizontal dimension

y-moment n

Ysum = j * yj j=0

Page 61: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Center of mass (=Centroid)

Center of mass= first-order / zero-order

Binary imagessum of x-pixel coordinates

CoMx= number of pixels in object

sum of y-coordinatesCoMy=

number of pixel in image

Page 62: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Feature: Boundary Descriptions• Explicit description• Chain codes• Line segment representation• Fourier descriptors

Page 63: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Boundary shape, accuracy

Sequential list of the boundary pixels

• save the boundary (x,y)-values• No change of location, orientation or size

Page 64: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Boundary shape, Chain code

• Boundary can be followed and recorded using chain code

• pick a starting pixel and detect direction to the next boundary pixel (values 0..7)

Page 65: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Chain codes• Absolute chain code

• Each code represents absolute direction, so can’t handle geometric change

• Relative chain code• Direction values are relative to earlier values. Can handle

geometric change.

Page 66: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing
Page 67: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Relative chain code

Page 68: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Line segment representation

• Some cases reduced form of the outline may be sufficient

• Example replacing individual boundary (x,y) locations with line segments.

Page 69: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Line segment representation -generally form

• Like chain code, but segment lengths and angles can be value free

Page 70: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Variable line segment boundary representation• Combines adjoining line segments with similar direction

angles

Page 71: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Object Classification• Compare the measurements of a new object with those of

a known object or other known criteria • In recognition the objects found in image ( example letters

and numbers) gets labels.

feature extraction

classification

segmentspace

distinctobjects

featurespace

feature

classificationspace

class

Segmentation

image

Page 72: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Knowledge base

• In image analysis you need pre-knowledge from problem area. This is saved in knowledge base. It can contain example different kind of letter and number features, where the wanted objects may have locate

• Knowledge base also controls different module’s operation.

Page 73: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Object classification• Comparing measures with known features

Classification:

• pick feature measures

• pick tolerances

• create classes

Page 74: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Object classification - example• Recognize ring and spade from conveyor belt

Features: Hole or no holeTolerance: Hole size 5mm ± 5%

Classes: Good, if in tolerances

Otherwise reject

Page 75: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Other classification methods• Gray-level classification

• Image information is divided in classes by gray level; text, logo, background ...

• Fuzzy logic• If several features which depend on each other

• Neural networks• Unique ability to be “trained”

Page 76: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Application level• Basic description of application• Example:

1. Capture image from register plate

2. Process and interpret objects

3. Search register database, is there offenses

Page 77: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Object detection system - planning• Object localization• Feature choosing• Classifiers planning• Classifiers teaching• Capacity evaluating

Page 78: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Pattern recognition

Page 79: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Applications

Page 80: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing
Page 81: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Cell analysis

Page 82: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Chromosome analysis

Page 83: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Thermo analysis

Page 84: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Micro hardness analysis

Page 85: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

DNA analysis

Page 86: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Motion analysis

Page 87: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

• Counting trees QR Code

108/11

Computer vision examples

Page 88: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

109/11

Facial recognition (=there is a face)

Page 89: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Facial recognition principle

Page 90: IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing

Face features