CSSE463: Image Recognition CSSE463: Image Recognition Day 11Day 11

Lab 4 (shape) tomorrow: feel free to start in Lab 4 (shape) tomorrow: feel free to start in advanceadvance

Test Monday through shape (no pattern rec.)Test Monday through shape (no pattern rec.) Start thinking about term project ideas.Start thinking about term project ideas. Questions?Questions?

Next 1.5 weeks: Pattern recognitionNext 1.5 weeks: Pattern recognition Concepts, error types (today)Concepts, error types (today) Basic theory and how to use classifiers in MATLAB:Basic theory and how to use classifiers in MATLAB:

Support vector machines (SVM).Support vector machines (SVM). Neural networksNeural networks

Pattern recognitionPattern recognition Making a decision from Making a decision from

datadata A classification problem: A classification problem:

assign a single class label assign a single class label to a datum pointto a datum point

Can include a special class, Can include a special class, rejectreject, ,

if a sample (a single datum if a sample (a single datum point) appears not to belong to point) appears not to belong to any known classany known class

If it is on the boundary If it is on the boundary between classesbetween classes

Else Else forced classificationforced classification Boundaries between Boundaries between

classes-how?classes-how? There’s tons of theory, There’s tons of theory,

can be applied to many can be applied to many areas. We focus on small areas. We focus on small subset of those used for subset of those used for visionvision Q1

Baseline: Hand-tuned decision Baseline: Hand-tuned decision boundariesboundaries

You did this based on observations for fruit You did this based on observations for fruit classificationclassification

You’ll do the same thing in Lab 4 for You’ll do the same thing in Lab 4 for shapesshapes

But what if the features were much more But what if the features were much more complex?complex?We now discuss classifiers that learn class We now discuss classifiers that learn class

boundaries based on boundaries based on exemplarsexemplars (e.g., labeled (e.g., labeled training examples)training examples)

Ex: Nearest neighbor classifierEx: Nearest neighbor classifier Assumes we have a feature Assumes we have a feature

vector for each imagevector for each image Calculate distance from Calculate distance from new new

test sampletest sample to to each labeled each labeled training sampletraining sample. .

Assign label as closest training Assign label as closest training samplesample

Generalize by assigning same Generalize by assigning same label as the majority of the k label as the majority of the k nearest neighbors. No majority?nearest neighbors. No majority?

d

i

ipipppdDIn

ypypxpxpppDIn

1

22121

221

22121

))()((,

))()(())()((,2

Nearest class meanNearest class mean

Find class means and Find class means and calculate distance to calculate distance to each meaneach mean Pro?Pro? Con?Con?

Partial solution: Partial solution: clusteringclustering Learning vector Learning vector

quantization (LVQ): quantization (LVQ): tries to find optimal tries to find optimal clustersclusters

LVQ

Test point

Q2

Common model of learning Common model of learning machinesmachines

StatisticalLearning

Labeled TrainingImages

Extract Features(color, texture)

TestImage

Summary

LabelClassifierExtract Features

(color, texture)

Focus on testingFocus on testingLet m = the number of possible class labelsLet m = the number of possible class labelsConsider m==2. Consider m==2. Example: Calculate distance to cluster Example: Calculate distance to cluster

means for 2 classes.means for 2 classes.

Dist/prob1

Dist/prob2

TestImage

Extract Features(color, texture)

Decide Class:1, 2

How good is your classifier?How good is your classifier?

Example from medicine: Example from medicine: Disease detectionDisease detection

Consider costs of false Consider costs of false neg. vs. false pos.neg. vs. false pos.

Lots Lots of different error of different error measuresmeasures Accuracy = 10500/10800 = Accuracy = 10500/10800 =

97%. Is 97% accuracy OK?97%. Is 97% accuracy OK? Recall (or true positive Recall (or true positive

rate) = 500/600=83%rate) = 500/600=83% Precision = 500/700=71%Precision = 500/700=71% False pos rate = 200/10200 False pos rate = 200/10200

= 2%= 2%

DetectedDetected

TrueTrue

YesYes NoNo

YesYes 500500

(true (true pos.)pos.)

100100

(false (false neg.)neg.)

NoNo 200200

(false (false pos.)pos.)

1000010000

(true (true neg.)neg.)

10200Total actualnegative

600Total actualpositive

700Total det. as pos.

10100Total det. as neg.

How good is your classifier?How good is your classifier?

Write out definitions of Write out definitions of each measure noweach measure now

ExamplesExamples Accuracy = 10500/10800 = Accuracy = 10500/10800 =

97%. 97%. Recall (or true positive Recall (or true positive

rate) = 500/600=83%rate) = 500/600=83% Precision = 500/700=71%Precision = 500/700=71% False pos rate = 200/10200 False pos rate = 200/10200

= 2%= 2%

Detected:Detected:

Has:Has:

YesYes NoNo

YesYes 500500

(true (true pos.)pos.)

100100

(false (false neg.)neg.)

NoNo 200200

(false (false pos.)pos.)

1000010000

(true (true neg.)neg.)

Q3a-d

Thresholding real-valued output allows you to Thresholding real-valued output allows you to tradeoff TPR and FPRtradeoff TPR and FPR

If t == 0: TPR = ___, FPR = ___If t == 0: TPR = ___, FPR = ___

9/129/12 2/82/8

If t == 1: TPR = ___, FPR = ___If t == 1: TPR = ___, FPR = ___

Repeat for many values of tRepeat for many values of t

-3 -2 -1 0 1 2 3

NN N P N P N N P N PP N PP PP PPP

Simple example: Classes P = positive, N = negative, and single Simple example: Classes P = positive, N = negative, and single real-valued output.real-valued output.

False Pos Rate

Tru

e P

os R

ate

True class:

Output:

Threshold output to get class. label = output > t ? P : NChoice of threshold a

ROC curve ROC curve

Receiver-operating Receiver-operating characteristiccharacteristic

Useful when you Useful when you can change a can change a threshold to get threshold to get different true and different true and false positive ratesfalse positive rates

Consider extremesConsider extremes Much more Much more

information information recorded here!recorded here!

Q3

Multiclass problems Multiclass problems

Consider m>2. Consider m>2. Example: Calculate distance to cluster Example: Calculate distance to cluster

means for 10 classes.means for 10 classes.

Dist/prob1

Dist/prob2

Dist/probN

TestImage

Extract Features(color, texture)

Decide Class:1, 2,…N

Confusion matrices for m>2 Confusion matrices for m>2 (outdoor image example)(outdoor image example)

Beach recall: 169/(169+0+2+3+12+14)=84.5%Beach recall: 169/(169+0+2+3+12+14)=84.5% Note confusion between mountain and urban Note confusion between mountain and urban

classes due to featuresclasses due to features Similar colors and spatial layoutSimilar colors and spatial layout

Bch Sun FF Fld Mtn UrbBch 169 0 2 3 12 14Sun 2 183 5 0 5 5FF 3 6 176 6 4 5Fld 15 0 1 173 11 0Mtn 11 0 2 21 142 24Urb 16 4 8 5 27 140

Detected

Tru

e

Q4

Why do we need separate training Why do we need separate training and test sets?and test sets?

Exam analogyExam analogy

But working on practice questions is helpful…get But working on practice questions is helpful…get the analogy? the analogy? We hope our ability to do well on We hope our ability to do well on practice questions helps us on the actual exampractice questions helps us on the actual exam

Application to nearest-neighbor classifiersApplication to nearest-neighbor classifiers

Often reserve a 3Often reserve a 3rdrd set for validation as well (to set for validation as well (to tune parameters of training set)tune parameters of training set)

Q5-8

Look at this to understand Look at this to understand nearest neighbor nearest neighbor

http://ai6034.mit.edu/fall12/index.php?title=Demonstrations

Shows Voronai diagrams for nearest Shows Voronai diagrams for nearest neighbor classifiersneighbor classifiers

Nice intro to SVMs alsoNice intro to SVMs also