visuelle perzeption für mensch- maschine schnittstellen · edgar seemann, 08.12.08 4 computer...
TRANSCRIPT
![Page 1: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/1.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 1
Visuelle Perzeption für Mensch-Maschine Schnittstellen
Vorlesung, WS 2008
Dr. Rainer StiefelhagenDr. Edgar Seemann
Interactive Systems LaboratoriesUniversität Karlsruhe (TH)
http://isl.ira.uka.de/msmmi/teaching/[email protected]
![Page 2: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/2.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 2
Programming
Assignments
WS 2008/09
Dr. Edgar Seemann
![Page 3: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/3.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 3
Organisatorisches
� There will be no lecture on Friday, January 23rd
![Page 4: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/4.jpg)
Edgar Seemann, 08.12.08 4
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Assignment 1 Results
� Gruppe 1:Christian JohnerMike MorantePatrick Mehl
� Gruppe 2:Steffen Braun
� Gruppe 10:Thomas Stephan
� Gruppe 11:Igor Plotkin
� Gruppe 3:Martin WagnerHilke KieritzJan Hendrik Hammer
� Gruppe 4:Wenlei WuChengchao Qu
� Gruppe 5:Michael WeberTomas SemelaDennis Kopcan
� Gruppe 6:Johann KorndoerferDaniel KoesterDaniel Putsch
� Gruppe 8Mathias Luedtke(Florian Krupicka)
� Gruppe 9Felix ReuterElke Mueller
� Gruppe 7Benjamin BartoschThomas Lichtenstein
![Page 5: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/5.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 5
This Lecture
� Student presentations
� Short Intro into Assignment 3� Data Set
� Choice of Parameters
� Non-Maxiumum Suppression
![Page 6: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/6.jpg)
Edgar Seemann, 08.12.08 7
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
It’s your turn
![Page 7: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/7.jpg)
Edgar Seemann, 08.12.08 8
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Assignment 3
![Page 8: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/8.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 9
Assignment 3
� People Detection� Detect for the given images, where and at which scale
the image contains people
� That is:1. We have to implement a sliding window search
2. At each location we classify the window with the SVM from assignment 2
3. We have to fuse the detections with a non-maxiumsuppression approach
![Page 9: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/9.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 10
The Same Training Data
� Training Set (PersonTrain.tar.bz2): � 2418 positive examples� 2436 negative examples� 96x160 pixels (64x128 + larger border)
� Idl-files:� Pos.idl� Neg.idl
![Page 10: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/10.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 11
Test Set
� Test set (PersonTestDetection.tar.bz2):� 41 positive images
� 0 negative images
� Ground-truth defined in testset.idl
![Page 11: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/11.jpg)
Edgar Seemann, 08.12.08 12
Com
pute
r V
isio
n fo
r H
uman
-Com
pute
r In
tera
ctio
n
Res
earc
h G
roup
, Uni
vers
itätK
arls
ruhe
(TH
)cv
:hci
Quantitative Evaluation
� We use a precision-recall c++ binary (“precisionrecall”)
� A python script just sets some default command-line parameters
� ./doROC.py groundtruth.idl result.idl� Produces a text file result.txt containing the plotting
data
� plotSimple.py can be used to display the results
![Page 12: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/12.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 13
Your Task
� Compute an .idl file, which specifies for each test image a set of detection hypotheses
� Annotool helps to display results at different confidence levels
![Page 13: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/13.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 14
Sliding Window Technique
� We obtain for each position/scale a recognition score
� Parameters: scale range, scale steps, x/y-steps
� Positions with low scores can be discarded
Red: score>0.8
![Page 14: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/14.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 15
Sliding Window Parameters
� Use 64x128 windows
� Start at original resolution
� Shift windows� 4 pixels in x-direction
� 4 pixels in y-direction
� You free to experiment with these values
� Change scale� Shrink image with a factor of 1.2
� Other common choice is sqrt(2) as shrinking factor
![Page 15: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/15.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 16
Discard bad hypotheses
� We have to find a reasonable threshold for the SVM score
� Suggestion:� Accept everything with score >0� Display results in “AnnoTool”� Decrease/Increase threshold according to visual results
� Rules:� Allow enough hypotheses to have a recall of 1� We should have multiple hypotheses (shifted, scaled)
around each person
![Page 16: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/16.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 17
Non-Maximum Suppression
� A good detector will generally not only fire on the exact position
� Need to reduce the number of detections, since every additional detection (even on the object) will count as false positive detection
![Page 17: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/17.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 18
Naïve Approach
� Pick hypotheses in a greedy fashion� Accept the strongest hypothesis
� Remove all other hypotheses, which strongly overlap
� libAnnotation contains methods to compute the cover/overlap between two hypotheses
![Page 18: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/18.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 19
Non-Maxiumum suppression approaches
� Finding modes in a non-parametric distribution� Kernel Density Estimation
� Mean-Shift Mode Estimation (MSME) (e.g. [Dalal’05])
� Pixel-based reasoning (e.g. [Leibe et al. 2004])� Infer an object segmentation
� Use segmentation to determine, which pixels belong to which object
![Page 19: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/19.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 20
Mean Shift Theory
![Page 20: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/20.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 21
Kernel Density Estimation
� Which probability at position x should be higher?
� Single peak could result from noise, image artifacts etc.
x x
![Page 21: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/21.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 22
Intuitive Description
Distribution of identical billiard balls
Region ofinterest
Center ofmass
Mean Shiftvector
Objective : Find the densest region
![Page 22: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/22.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 23
Intuitive Description
Distribution of identical billiard balls
Region ofinterest
Center ofmass
Mean Shiftvector
Objective : Find the densest region
![Page 23: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/23.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 24
Intuitive Description
Distribution of identical billiard balls
Region ofinterest
Center ofmass
Mean Shiftvector
Objective : Find the densest region
![Page 24: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/24.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 25
Intuitive Description
Distribution of identical billiard balls
Region ofinterest
Center ofmass
Mean Shiftvector
Objective : Find the densest region
![Page 25: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/25.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 26
Intuitive Description
Distribution of identical billiard balls
Region ofinterest
Center ofmass
Mean Shiftvector
Objective : Find the densest region
![Page 26: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/26.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 27
Intuitive Description
Distribution of identical billiard balls
Region ofinterest
Center ofmass
Mean Shiftvector
Objective : Find the densest region
![Page 27: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/27.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 28
Intuitive Description
Distribution of identical billiard balls
Region ofinterest
Center ofmass
Objective : Find the densest region
![Page 28: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/28.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 29
AdaptiveGradient Ascent
Mean Shift Properties
• Automatic convergence speed – the mean shift vector size depends on the gradient itself.
• Near maxima, the steps are small and refined
• Convergence is guaranteed for infinitesimal steps only � infinitely convergent, (therefore set a lower bound)
• For Uniform Kernel ( ), convergence is achieved ina finite number of steps
• Normal Kernel ( ) exhibits a smooth trajectory, but is slower than Uniform Kernel ( ).
![Page 29: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/29.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 30
Real Modality Analysis
Tessellate the space with windows
Run the procedure in parallel
![Page 30: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/30.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 31
Real Modality AnalysisAn example
Window tracks signify the steepest ascent directions
![Page 31: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/31.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 32
Mean Shift Strengths & Weaknesses
Strengths :
• Application independent tool
• Suitable for real data analysis
• Does not assume any prior shape(e.g. elliptical) on data clusters
• Can handle arbitrary featurespaces
• Only ONE parameter to choose
• h (window size) has a physicalmeaning, unlike K-Means
Weaknesses :
• The window size (bandwidth selection) is not trivial
• Inappropriate window size cancause modes to be merged, or generate additional “shallow”modes � Use adaptive windowsize
![Page 32: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/32.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 33
Presentation
� Shortly present what exactly you have implemented (maybe a visualization of your features)
� What were the lessons learned?
� Please prepare a couple of slides
� Try to finish within the given 8 minutes
� Send me your PPT (<=PPT2003) or PDF files till February 6th, 8 am
![Page 33: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/33.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 34
Send me your results
� Please send me your results:� Group members (Name, MatrikelNr.)� Presentation file� Source code
� I should be able to run the code and reproduce the results
� E-Mail: [email protected]� If the e-mail is larger than 10mb, please try to split it
� I will try to give some feedback
![Page 34: Visuelle Perzeption für Mensch- Maschine Schnittstellen · Edgar Seemann, 08.12.08 4 Computer Vision for Human-Computer Interaction Research Group, Universität Karlsruhe (TH) cv:hci](https://reader034.vdocuments.site/reader034/viewer/2022050300/5f6967980658436bce5a217b/html5/thumbnails/34.jpg)
Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 35
End of Lecture