visuelle perzeption für mensch- maschine...
TRANSCRIPT
![Page 1: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/1.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Dr. Edgar Seemann, 30.10.09 1
Visuelle Perzeption für Mensch-Maschine Schnittstellen
Vorlesung, WS 2009
Prof. Dr. Rainer StiefelhagenDr. Edgar Seemann
Institut für AnthropomatikUniversität Karlsruhe (TH)
http://[email protected]
![Page 2: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/2.jpg)
Dr. Edgar Seemann, 30.10.09 2
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Basics:
Pattern Recognition
WS 2009/10
Dr. Edgar Seemann
![Page 3: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/3.jpg)
Dr. Edgar Seemann, 30.10.09 6
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Termine (1)
6
TBA21.12.2009
Stereo and Optical Flow18.12.2009
Termine Thema19.10.2009 Introduction, Applications
23.10.2009 Basics: Cameras, Transformations, Color
26.10.2009 Basics: Image Processing
30.10.2009 Basics: Pattern recognition
02.11.2009 Computer Vision: Tasks, Challenges, Learning, Performance measures
06.11.2009 Face Detection I: Color, Edges (Birchfield)
09.11.2009 Project 1: Intro + Programming tips13.11.2009 Face Detection II: ANNs, SVM, Viola & Jones
16.11.2009 Project 1: Questions
20.11.2009 Face Recognition I: Traditional Approaches, Eigenfaces, Fisherfaces, EBGM
23.12.2009 Face Recognition II
27.11.2009 Head Pose Estimation: Model-based, NN, Texture Mapping, Focus of Attention
30.11.2009 People Detection I
03.12.2009 People Detection II
07.12.2009 Project 1: Student Presentations, Project 2: Intro11.12.2009 People Detection III (Part-Based Models)
14.12.2009 Scene Context and Geometry
![Page 4: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/4.jpg)
Dr. Edgar Seemann, 30.10.09 7
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Today
� Why pattern recognition and what is it?
� Dimensionality Reduction� Principle Component Analysis
� Classification� Bayes Decision Theory
� k-NN� Kd-trees
� Ball trees
![Page 5: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/5.jpg)
Dr. Edgar Seemann, 30.10.09 8
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Overview
� Learn common patterns based on either � a priori knowledge or
� on statistical information
� Why statistics?� Manual definition of patterns often tedious or not
obvious
� Important for the adaptability to different tasks/domains
� Finally: Try to mimic human learning / better understand human learning
![Page 6: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/6.jpg)
Dr. Edgar Seemann, 30.10.09 9
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Pattern Recognition
� Supervised� Data samples with associated class labels
� Unsupervised� Data samples WITHOUT any labels
� Semi-Supervised / Weakly-Supervised Learning� Not a topic in this lecture
SensorFeature Selection/
ExtractionClassifierRepresentation
pattern
Feature
pattern
Decision
![Page 7: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/7.jpg)
Dr. Edgar Seemann, 30.10.09 10
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Pattern Recognition Stages
1. Feature Extraction� Which part of the data is most important
2. Classification/Learning� How can we map the extracted features to our desired
output (supervised learning)?
= x y in [a,b,c,…]w
![Page 8: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/8.jpg)
Dr. Edgar Seemann, 30.10.09 11
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Examples
� Speech recognition
� Computer Vision� You will see many examples in the course of this
semester …
w
![Page 9: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/9.jpg)
Dr. Edgar Seemann, 30.10.09 12
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
� Parametric/Non-parametric distributions� Naïve Bayes Classifier� Support Vector Machines� Neural Networks� Decision Trees� Boosting� ….
What’s in the box?
![Page 10: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/10.jpg)
Dr. Edgar Seemann, 30.10.09 13
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Classification vs. Regression
� Classification� Decide to which class a feature vector belongs� Typically a small number of classes� Many basic methods are restricted to binary
classification
� Regression� Function approximation� Multi-class problems can
be addressed by defininga desired function valuee.g. yi=[0,..1,0]
![Page 11: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/11.jpg)
Dr. Edgar Seemann, 30.10.09 14
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Problems and Considerations
� Features� How do I encode domain knowledge?
� Allow invariance (e.g. rotation in the case of images)
� Which part of the data can be discarded as it represents redundant information?
� How can we reduce the dimensionality?� i.e. how can we make the problem as simple as possible, but as
complex as necessary
![Page 12: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/12.jpg)
Dr. Edgar Seemann, 30.10.09 15
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Curse of Dimensionality
� Generally: the more dimensions, the more difficult for a learning algorithm to extract patterns
� More dimensions = more degrees of freedom
Decision boundary
Class A
Class B
Defining a linear decision
boundary:
2-dim: 3 degrees of freedom
3-dim: 5 degrees of freedom
4-dim: 7 degrees of freedom
…
![Page 13: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/13.jpg)
Dr. Edgar Seemann, 30.10.09 16
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Dimensionality Reduction
� Reduce the dimensionality of the data, while retaining relevant information
� Popular techniques:� Principal Component Analysis (PCA)
� Linear Discriminant Analysis (LDA)
� Multidimensional Scaling (MDS)
� …
� Many of these techniques are readily available in MATLAB or Octave
![Page 14: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/14.jpg)
Dr. Edgar Seemann, 30.10.09 17
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
PCA
![Page 15: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/15.jpg)
Dr. Edgar Seemann, 30.10.09 18
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
First Steps: Sample Variance
� Given a set of samples
with mean
� An unbiased variance estimator is defined as
� with the sequence with zero mean
� written in vectors
),...,( 1 naa
'a
∑ −−
= 2)'(1
1)var( aa
na i ∑−
= 2
1
1iz
n
Tzzn 1
1
−
),...,( 1 nzz
![Page 16: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/16.jpg)
Dr. Edgar Seemann, 30.10.09 19
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
First Steps: Covariance
� Given two sets of samples
with means
� A covariance estimator is defined as
� with sequences with zero mean
� written in vectors
),...,( 1 naa
',' ba
∑ −−−
= )')('(1
1),cov( bbaa
nba ii ∑−
= iidcn 1
1
Tcdn 1
1
−
),...,(),,...,( 11 nn ddcc
),...,( 1 nbb
![Page 17: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/17.jpg)
Dr. Edgar Seemann, 30.10.09 20
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
First Steps: Covariance Matrix
� Given n sets samples
with means and
d being the dimensionality
� A multi-dimensional covariance estimator is defined as
� Written in vectors
nvv ,...,1
'iv
Tjjiiji vvvv
nV )')('(
1
1)cov( , −−
−=
),...,( 1 di aav =
TVVn 1
1
−
![Page 18: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/18.jpg)
Dr. Edgar Seemann, 30.10.09 21
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Variance Properties
� Properties:� var(a)=cov(a,a)
� cov(a,b)=0 iff a,b are completely uncorrelated
� cov(V) is a square-symmetric matrix (containing variances on the diagonal and covariances on the off-diagonals)
)var(),cov(
),cov()var(
1
11
nn
n
vvv
vvv
…..
…..…
……
.…..
…..
![Page 19: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/19.jpg)
Dr. Edgar Seemann, 30.10.09 22
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
PCA: Toy example
� Toy example:� Try to understand the motion of a spring
� Data from three camera sensors (x,y-position foreach camera, i.e. 6-dim. data)
![Page 20: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/20.jpg)
Dr. Edgar Seemann, 30.10.09 23
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Noise and Redundancy
� Data contains noise
� Data is redundant� 1-dimensional motion vs. 6-dimensional sample data
� Sensor data from the three cameras are highly correlated
![Page 21: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/21.jpg)
Dr. Edgar Seemann, 30.10.09 24
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Change of basis
� By changing the basis, we may represent the data almost perfectly with a lower number of dimensions
� Remember that the data has a mean of zero
r2’
r1’
Basis Change
![Page 22: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/22.jpg)
Dr. Edgar Seemann, 30.10.09 25
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Change of Basis
� Basis change can be written as matrix multiplication
� Given the new basis vectorswe can transform data samples xi in the following manner
� i.e. we are projecting xi onto the new basis vectors
npp ,...,1
![Page 23: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/23.jpg)
Dr. Edgar Seemann, 30.10.09 26
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Reducing the co-variances
� Remember the covariancematrix:
� We have seen that a basis change may reduce the correlation between the different dimensions
� Goal of PCA� Make covariance matrix as diagonal as possible
)var(),cov(
),cov()var(
1
11
nn
n
vvv
vvv
…..
…..…
……
.
…..
…..
![Page 24: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/24.jpg)
Dr. Edgar Seemann, 30.10.09 27
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
PCA Assumptions
� Basis (principal components) is orthogonal
� Linearity� Change of basis is a linear operation
� For non-linear problems: kernel PCA
� Mean and variance are sufficient statistics� This holds if data is distributed according to a Gaussian
distribution
� Large variances have important dynamics
![Page 25: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/25.jpg)
Dr. Edgar Seemann, 30.10.09 28
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Finally solving it
� Theorem:The covariance matrix is diagonalized by an orthogonal matrix of its eigenvectors
� That is the principal components of the underlying data are the eigenvectors
� The higher the eigenvalue the more variance is captured along the dimension
![Page 26: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/26.jpg)
Dr. Edgar Seemann, 30.10.09 29
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Eigenspectrum and Energy
![Page 27: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/27.jpg)
Dr. Edgar Seemann, 30.10.09 30
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Computing the eigen values and vectors
� Eigenvectors can be computed by solving the linear equation system
� This can be done with the well-known Gauss-Algorithm
� In practice, we most often use a singular value decomposition (SVD)
xAx λ=
0)( =− xIA λ
![Page 28: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/28.jpg)
Dr. Edgar Seemann, 30.10.09 31
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
PCA in practice
� The covariance of a set of images can be computed by writing the image data as columns into a matrix and multiplying it with it’s transposed matrix
� If the data samples are high dimensional, the covariance matrix can get extremely large� Let the sample xi be images with a resolution of 1600x1200
then the covariance would be a 1920000x1920000 matrix
� Computationally, computing the eigenvectors would be very costly
TAAN
Cov1
1
−=
![Page 29: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/29.jpg)
Dr. Edgar Seemann, 30.10.09 32
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
� Fortunately, it ain’t that bad� If we have a maximum of n data samples, we can also obtain only
a maximum of n eigenvectors (one dimension for each sample point)
� i.e. the number of eigenvectors is restricted by both the sampledimension and the sample number
� Let’s have a look at the following equations:
� λ is eigenvalue of ATA
� λ is eigenvalue of AAT
� We can compute the eigenvectors of ATA and multiply them by A
Algebra to the rescue
)()( AxAxAA
xAAxAA
xAxA
T
T
T
λλ
λ
=
=
=
![Page 30: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/30.jpg)
Dr. Edgar Seemann, 30.10.09 34
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Summary PCA
� can be used to reduce the number of dimensions
� Find the principal dimensions� Principal dimensions are assumed to have a high
variance
� discard non-informative dimensions (redundancy and noise)
� Extensions:� Robust PCA (deal with occlusion)
� Incremental PCA
� Kernel PCA (non-linear)
![Page 31: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/31.jpg)
Dr. Edgar Seemann, 30.10.09 35
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hciMotivation: Fisher linear discriminant
� more on this in one of Prof. Stiefelhagen’s lectures
![Page 32: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/32.jpg)
Dr. Edgar Seemann, 30.10.09 36
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Pattern Classification
![Page 33: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/33.jpg)
Dr. Edgar Seemann, 30.10.09 37
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Bayes Decision Theory
� Example: Character Recognition� Classify characters into the classes C1=a and C2=b
minimizing the probability of an incorrect classification
![Page 34: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/34.jpg)
Dr. Edgar Seemann, 30.10.09 38
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Priors
� Prior probabilities represent the probability of a class to occur
� Often used to encode domain knowledge
![Page 35: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/35.jpg)
Dr. Edgar Seemann, 30.10.09 39
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Conditional Probability
� Given a feature vector x� x measuring e.g. the width or height of the character
� The conditional probability measures the probability of observing x when the character is of class Ck
)|( kCxp
![Page 36: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/36.jpg)
Dr. Edgar Seemann, 30.10.09 40
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Example
� Say x = 15
� What decision should be made?
x = 15
![Page 37: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/37.jpg)
Dr. Edgar Seemann, 30.10.09 41
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Example
� Say x = 25
� What decision should be made?
x = 25
![Page 38: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/38.jpg)
Dr. Edgar Seemann, 30.10.09 42
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Example
� Say x = 20
� What decision should be made?
x = 20
![Page 39: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/39.jpg)
Dr. Edgar Seemann, 30.10.09 43
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Bayes’ Theorem
� The a-posteriori probabilitydefines the probability of a class given a specific feature vector x
)|( xCp k
![Page 40: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/40.jpg)
Dr. Edgar Seemann, 30.10.09 44
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Decision boundary
Decision boundary
![Page 41: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/41.jpg)
Dr. Edgar Seemann, 30.10.09 45
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Decision Rule
� Choose C1 iff
� this is equivalent to
� this is equivalent to
� Special cases: and
)|()|( 21 xCpxCp >
)()|()()|( 2211 CpCxpCpCxp >
)(
)(
)|(
)|(
1
2
2
1
Cp
Cp
Cxp
Cxp >
)()( 21 CpCp = )|()|( 21 CxpCxp =
![Page 42: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/42.jpg)
Dr. Edgar Seemann, 30.10.09 46
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
More Classes
� We can generalize this rule to multi-class problems (e.g. all letters from the alphabet)� Choose the class with the highest a-posteriori
probability
![Page 43: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/43.jpg)
Dr. Edgar Seemann, 30.10.09 47
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Example
![Page 44: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/44.jpg)
Dr. Edgar Seemann, 30.10.09 48
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Generalization with Loss-Function
� In some applications we may consider the loss of a mis-classification� E.g. medical applications
� Loss if classified as healthy despite a disease:
� Loss if classified as ill despite being healthy:
� We assume: >>
)|( illhealthyλ)|( healthyillλ
)|( illhealthyλ )|( healthyillλ
![Page 45: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/45.jpg)
Dr. Edgar Seemann, 30.10.09 49
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Expected Loss
� We have an observation x, but do not know the loss function λ for all x:
� with αi the possible decision and Ck the possible classes
� The expected loss can be computed in the following manner:
?)|( =xiαλ ikki C λαλ =)|(
![Page 46: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/46.jpg)
Dr. Edgar Seemann, 30.10.09 50
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Example
� Let’s look at a two class example:
� We want to minimize the expected loss, i.e. we choose αi when
>
![Page 47: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/47.jpg)
Dr. Edgar Seemann, 30.10.09 51
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Some simple math
![Page 48: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/48.jpg)
Dr. Edgar Seemann, 30.10.09 52
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Linear Discriminant FunctionsPerceptron Algorithm
![Page 49: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/49.jpg)
Dr. Edgar Seemann, 30.10.09 53
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Linear Discriminant Functions
� Separate two classes with a linear hyper plane
� With the normal vector of the hyper plane
� Examples:� Perceptron
� Linear SVM
0)( wxwxy T +=
0w
Tw
Tw
![Page 50: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/50.jpg)
Dr. Edgar Seemann, 30.10.09 54
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Perceptron Algorithmus[Rosenblatt ’58]
� Preconditions:� Data set is linear separable
� Goal� Find a separating hyper plane
� Idea� Iteratively improve solution
� Only update solution if the data sample of interest is classified incorrectly
![Page 51: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/51.jpg)
Dr. Edgar Seemann, 30.10.09 55
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Perceptron Algorithm
1. Initialize w = 0
2. Classify a new data sample with the following inner product y(x) = sign(wTx)
3. If correct, then goto step 2else and w = w - y(x)x
4. If no errors left, then doneelse goto step 2
![Page 52: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/52.jpg)
Dr. Edgar Seemann, 30.10.09 56
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Algorithm Visualized
![Page 53: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/53.jpg)
Dr. Edgar Seemann, 30.10.09 57
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Why does this work?
� Given misclassified sample x with y(x)=1
� Then: w = w - x
� If we now classify the sample again:� y(x) = (w-x)Tx = wTx – xTx
� xTx is positive and therefore the next prediction for this data sample will be closer to the correct value
� Convergence:� Does not converge if data is not separable
![Page 54: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/54.jpg)
Dr. Edgar Seemann, 30.10.09 58
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Wait
� We did not consider w0� What happens if separating hyper plane does not pass
through (0,0)
� Small trick:� Choose x to be n+1 dimensional with n+1 dimension always
being 1, then
� Becomes xwxy T=)(
0)( wxwxy T +=
![Page 55: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/55.jpg)
Dr. Edgar Seemann, 30.10.09 59
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Instance-Based Learning
![Page 56: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/56.jpg)
Dr. Edgar Seemann, 30.10.09 60
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Instance-based Learning
� Learning=storing all training instances
� Classification=assigning target function to a new instance
� Referred to as “Lazy” learning Instance
� Examples:� Template-Matching
� K-Nearest Neighbor
![Page 57: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/57.jpg)
Dr. Edgar Seemann, 30.10.09 61
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
1-Nearest-Neighbor
� Features� All instances correspond to points in an n-dimensional
Euclidean space
� Classification done by comparing feature vectors of the different points
![Page 58: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/58.jpg)
Dr. Edgar Seemann, 30.10.09 62
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
3-Nearest-Neighbor
� Search for the 3 nearest neighbors� Typically majority vote, when not all neighbors are
from the same class
![Page 59: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/59.jpg)
Dr. Edgar Seemann, 30.10.09 63
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
3-Nearest-Neighbor
� Decision surface� Described by the Voronoi diagram
� Voronoi diagram is the dual of the Delaunaytriangulation (computable in O(nlogn))
![Page 60: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/60.jpg)
Dr. Edgar Seemann, 30.10.09 64
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
When to Consider Nearest Neighbor ?
� Lots of training data
� Less than 20 attributes (dimensions) per example
� Advantages:� Training is very fast
� Learn complex target functions
� Don’t lose information
� Disadvantages:� Slow at query time
� Easily fooled by irrelevant attributes
![Page 61: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/61.jpg)
Dr. Edgar Seemann, 30.10.09 65
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
K-Nearest Neighbors
� Typically decision surface is not computed, but for each new test sample, we compute the nearest neighbors on the fly
� Probabilistic interpretation: estimate density in a local neighborhood
� There are efficient memory indexing techniques in order to retrieve the stored training examples� kd-tree [Friedman et al. 1977]
� Ball-tree
![Page 62: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/62.jpg)
Dr. Edgar Seemann, 30.10.09 66
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
KD Tree for NN Search
� Each node contains� Children information� The tightest box that bounds all the data points within the node.
![Page 63: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/63.jpg)
Dr. Edgar Seemann, 30.10.09 67
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
NN Search by KD Tree
![Page 64: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/64.jpg)
Dr. Edgar Seemann, 30.10.09 68
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
NN Search by KD Tree
![Page 65: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/65.jpg)
Dr. Edgar Seemann, 30.10.09 69
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
NN Search by KD Tree
![Page 66: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/66.jpg)
Dr. Edgar Seemann, 30.10.09 70
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
NN Search by KD Tree
![Page 67: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/67.jpg)
Dr. Edgar Seemann, 30.10.09 71
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
NN Search by KD Tree
![Page 68: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/68.jpg)
Dr. Edgar Seemann, 30.10.09 72
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
NN Search by KD Tree
![Page 69: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/69.jpg)
Dr. Edgar Seemann, 30.10.09 73
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
NN Search by KD Tree
![Page 70: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/70.jpg)
Dr. Edgar Seemann, 30.10.09 74
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Standard Kd-tree construction
� Choose splitting planes by cycling through the dimensions� Example:
� Root node: split in x-direction
� Next level: split in y-direction
� Next level: split in z-direction
� The position of the splitting plane is chosen to be the median of the points (with respect to their coordinates in the axis being used)
� Typically generates a quite balanced tree
![Page 71: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/71.jpg)
Dr. Edgar Seemann, 30.10.09 75
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Kd-tree construction
� There exist many (sometimes application specific) heuristics that try to speed up kd-tree creation� E.g. split in the dimension of the highest variance
� Kd-trees can be built incrementally� Useful for incremental learning (for example when
exploring a new territory with a robot)
� Existing libraries:� C++: libkdtree++
![Page 72: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/72.jpg)
Dr. Edgar Seemann, 30.10.09 76
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
A kd-tree: level 1
![Page 73: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/73.jpg)
Dr. Edgar Seemann, 30.10.09 77
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
A kd-tree: level 2
![Page 74: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/74.jpg)
Dr. Edgar Seemann, 30.10.09 78
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
A kd-tree: level 3
![Page 75: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/75.jpg)
Dr. Edgar Seemann, 30.10.09 79
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
A kd-tree: level 4
![Page 76: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/76.jpg)
Dr. Edgar Seemann, 30.10.09 80
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
A kd-tree: level 5
![Page 77: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/77.jpg)
Dr. Edgar Seemann, 30.10.09 81
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
A kd-tree: level 6
![Page 78: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/78.jpg)
Dr. Edgar Seemann, 30.10.09 82
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Complexity
� Building a static kd-tree from n points: O(n log n)� O(log n) tree levels, O(n) median search
� Insertion into a balanced kd-tree: O(logn)
� Removal from a balanced kd-tree O(logn)
� Query of an axis-parallel range in a balanced kd-tree: O(n1-1/d +k)� with k the number of the reported points, and d the
dimension of the kd-tree.
![Page 79: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/79.jpg)
Dr. Edgar Seemann, 30.10.09 83
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Problem of Dimensionality
� Imagine instances described by 20 attributes, but only 2 are relevant to target function
� Curse of dimensionality: nearest neighbor is easily mislead when high dimensional X
![Page 80: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/80.jpg)
Dr. Edgar Seemann, 30.10.09 85
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Ball Trees
� Can be constructed in a similar fashion as kd-trees
� Ball trees have shown to be superior to kd-trees in many applications (though there is high variance and dataset dependence)
� The Proximity Project [Gray, Lee, Rotella, Moore 2005]
![Page 81: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/81.jpg)
Dr. Edgar Seemann, 30.10.09 86
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
A ball-tree: level 1
![Page 82: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/82.jpg)
Dr. Edgar Seemann, 30.10.09 87
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
A ball-tree: level 2
![Page 83: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/83.jpg)
Dr. Edgar Seemann, 30.10.09 88
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
A ball-tree: level 3
![Page 84: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/84.jpg)
Dr. Edgar Seemann, 30.10.09 89
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
A ball-tree: level 4
![Page 85: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/85.jpg)
Dr. Edgar Seemann, 30.10.09 90
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
A ball-tree: level 5
![Page 86: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/86.jpg)
Dr. Edgar Seemann, 30.10.09 91
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
kNN Discussion
� Highly effective inductive inference method for noisy training data and complex target functions
� Target function for a whole space may be described as a combination of less complex local approximations
� Learning is very simple
� Classification can be time consuming, if training data set is large: O(logn)
![Page 87: Visuelle Perzeption für Mensch- Maschine Schnittstellencvhci.anthropomatik.kit.edu/download/visionhci09/2009-WS_V04_Basics-Pattern...Dr. Edgar Seemann, 30.10.09 6 Computer Vision](https://reader034.vdocuments.site/reader034/viewer/2022050420/5f8f95964f37463680217545/html5/thumbnails/87.jpg)
Dr. Edgar Seemann, 30.10.09 92
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
End of Lecture