pattern recognition in image analysis

Pattern Recognition in Image AnalysisI A l i E t ti f k l d f i d t• Image Analysis: Extraction of knowledge from image data.

• Pattern Recognition: Detection and extraction of patterns from data.• Pattern: A subset of data that may be described by some well-defined set of rules.y yPatterns may constitute the smallest entity in the data that represent knowledge.

pixel

this is a pattern

PRIA01: Introduction, Klaus D. Toennies 1

Applications of Pattern Recognition• automatic analysis of medical images• automatic inspection of parts on an assembly line

h h iti b t• human speech recognition by computers• classification of seismic signals (e.g., for oil and mineral exploration)• selection of tax returns to audit, stocks to buy, people to insure, y, p p• identification of people from fingerprints, retinal scans, handwriting, ...• automatic inspection of printed circuits, printed characters, handwriting recognition• automatic analysis of satellite pictures (e.g., weather condition, water reserves,

mineral prospects,...)…

Pattern recognition (in general as well as applied to images) is mainly a classification task.Images often constitute the data source for pattern recognition.


g p g

Pattern Recognition and Objects

• Patterns in images represent (attributes of) objects

• Pattern recognition tasks are– Object detection tasksObject detection tasks

to find an unknown number of instances of a known kind of an objectin the imageg

– Object recognition tasksto recognise a detected object as one of a specific kindto recognise a detected object as one of a specific kind


Detection and Recognition

D t ti t h i• Detection techniques: segmentation, objectmatching, searchingtechniques

• Recognition techn• Recognition techn-iques: feature comput-ation/ reduction, l ifi ticlassification,

clustering


Detection and Recognition• May be used independently of each other

I bi ti h i• In combination as process chain– Detection (and segmentation) separates object detail of objects of some broad

class (e.g. „cars“)( g „ )

– Recognition separates detected objects in sub-classes (e.g. „sports cars“, „limousines“,…)

• Some problems need an integrated solution (e.g. „separate different car typesdirectly in an image“)

I t t d l ti i h l k i i “ b t h diffi lt• Integrated solution is more „human-loke vision“ but much more difficult– Potentially every pixel contributes to the solution

Features are not guaranteed to belong to an object to be classified– Features are not guaranteed to belong to an object to be classified


Classical Solution

• Pattern recognition as a processing pipeline– (Segmentation)– Feature computationFeature computation– Classification


Pattern Recognition as a ClassificationPattern Recognition as a Classification Task

Features: (f1, f2, ..., fn)

Objects to be classified are described by features.Features are evaluated to separate objects into different classes.

Features: (f1, f2, ..., fn)


Feature DetectionAn important prerequisite for feature detection in images is the extraction of structures that have common feature values Segmentation.Why?

Often, features are not computed from single pixels but from pixel sets. Their computation is erroneous if feature values change over the set.

Image

Segmentation Segmentation and Pattern Recognition:g

FeatureComputation

Segmentation and Pattern Recognition:Knowledge on the features to be evaluated greatly enhances the segmentation success.

Computation

Classification Pattern Recognition


ClassificationClassification: grouping patterns (samples) according to their features into different classes.How do I decide this?How do I decide this?• Decide which features are relevant to the problem and decide on a way to

compute them.id l l ifi i h i (b d h f f )• Decide on a general classification technique (based on the type of features)

• Train a classifier based on samples of the images to be analysed:- Find out a differentiation between different meanings based on featureFind out a differentiation between different meanings based on feature

characteristics.- Find a suitable generalisation of the feature values based on the training set

E i h f h l ifi i i d d i d• Estimate the error of the classifier using an independent, representative test data set.

• Apply the classifier to images of the problem domain.


Classification Techniques• Statistical pattern recognition: Assume that the pattern is a sample of from a

number of known distributions and assign it to the one class to which it most likely belongs to.

Requires a feature vector as input information.

Solutions are, e.g., kNN-classifier, backpropagation networks, support ector machinesvector machines

• Semantical pattern recognition: Assume that the features of a pattern follows known laws / rules for constructing the pattern and assign it to the class whose rules are most closely followed.

Requires a feature structure to reflect the rules

S l ti ( t ft d i i l i ) d hSolutions are grammars (not often used in image analysis) and graph matching techniques


Is Pattern Recognition Equivalent toIs Pattern Recognition Equivalent to Image Analysis?g y

Task: Detecting windows and doors in image

• Doors are rotated with respect to camera• Windows are partially occluded

image.

Windows are partially occludedScene is 3-d but features are from 2-d picture.

Feature-based pattern recognition from 3-d scenes:• Features are those of the object and not that of its projection!j p j• Pattern recognition may require 3-d surface reconstruction prior to analysis.


Useful and not so UsefulUseful and not so Useful Features This should have similar

appearances!pp

Useful features pertain to objectUseful features pertain to objectUseless features are those pertaining to imaging the scene (e.g., highlights ...)


Finding the Features• Reconstruct 3-d surfaces (= 3-d computer vision)• Induce locations of light sources

S th i i fl f li ht f t iti• Synthesise influences from light sources, surface curvature, camera position• Extract object features after accounting for synthesised influences

This is much too complicated in real-world applications!

Solutions:• looking at 2-d scenes only for feature-based pattern recognition• using reconstructed 3-d geometric features (fitting methods)


2-d Scenes and Almost 2-d Scenes• Medical images such as CT, MRI, etc.• satellite images and aerial photos

d i d t t i• drawings and text images• some images from computer aided manufacturing


What if it is truly 3 d?What, if it is truly 3-d?

These cars belong to the same class while this one does not


The not-so-classical way:

G d l f h l d d id li

Fitting a ModelGenerate models of the class and decide on quality of fit.

M d lModel:• geometrical, 3-d model: fit edges to model

edges.• Picture 2 d model (set of 2 d pictures): fit to• Picture 2-d model (set of 2-d pictures): fit to

interpolation from pictures.

Wh ?Why?Human Vision: There must be some fast classification that may be later refined


be later refined.

3-d Model Fitting

• Rough classification of an object• 3-d model may aid to segmentation in 3-d being pre-requisite to PR

techniques• 2 d model may allow for segmentation free classification• 2-d model may allow for segmentation free classification

Problem:• scene may contain more than just the object (i.e., segmentation before

segmentation) and in may be distorted by artefacts / noise.Find the “important” features in an image (saliency of features)Find the important features in an image (saliency of features)


Fitting 2-d to 2-d

• Top-Down-Approach

• Alternative to segmentation-to-features-to-objectj

• Does not requires segmentation Simple road model

• Requires appropriate model– Capturing the essence of membersp g

of an object class

– Capturing acceptable variabilityp g p yPRIA01: Introduction, Klaus D. Toennies 18

Top-Down vs. Bottom-Up• Bottom-Up = Classical Pattern Recognition

G ti f ( t ti ll ) i f l f t– Generation of (potentially) meaningful features

– Conceptual distance between features and semantics is large (shape features, texture features)

– Requires elaborated classification techniques but feature generation is (largely) domain independent

• Top-Down = Model Fitting– Generation of a meaning model of object‘s appearance in an image

C l di b d l d i i ll– Conceptual distance between model and semantics is small

– Simple classification in low-dimensional feature space but domain-dependent


What this course will be about...Segmentation• Review segmentation techniques, deformable templates, texture segmentationg q , p , g

Features• Feature detection, feature representation, feature reduction

Pattern Recognition Techniques• statistical, semantic, and neural network pattern recognition

... combined with a project on texture segmentation

Aim of the course is to give an overview on techniques and applications in pattern recognition as applied to image analysis.


Background / LiteratureRequirements

Basic knowledge of image processing (what is a filter, a gradient, what is segmentation, etc.)segmentation, etc.)

Literature• R.O.Duda, P.E.Hart, D.G.Stork, Pattern Classification, 2nd edition, 2001, Wiley and

Sons.• E. Gose, R. Johnsbaugh, S. Jost, Pattern Recognition and Image Analysis, Prentice , g , , g g y ,

Hall, 1996• R.Jain, R.Kasturi, B.G.Schunck, Machine Vision, McGraw-Hill, 1995

E R D i M hi Vi i Th Al ith P ti liti 3 d diti• E.R.Davies, Machine Vision – Theory, Algorithms, Practicalities, 3nd edition, Academic Press, 2002.

• Powerpoint slides of the course (on the net)


pattern recognition in image analysis

Documents