iccv2009 recognition and learning object categories p1 c02 - detecting single objects in context
DESCRIPTION
TRANSCRIPT
Context for single object classes
Who needs context anyway?We can recognize objects even out of context
Banksy
Why is context important?• Changes the interpretation of an object (or its function)
• Context defines what an unexpected event is
Look-Alikes by Joan Steiner
Even in high resolution, we can not shut down contextual processing and it is hard to recognize the true identities of the elements that compose this scene.
The importance of context• Cognitive psychology
– Palmer 1975 – Biederman 1981– …
• Computer vision– Noton and Stark (1971)– Hanson and Riseman (1978)– Barrow & Tenenbaum (1978) – Ohta, kanade, Skai (1978)– Haralick (1983)– Strat and Fischler (1991)– Bobick and Pinhanez (1995)– Campbell et al (1997)
What is the context for a single object category?
The influence of an object extends beyond its physical boundaries
Global and local representationsbuilding
car
sidewalk
Urban street scene
Global and local representations
Image index: Summary statistics, configuration of textures
Urban street scene
features
histogram
building
car
sidewalk
Urban street scene
Global scene representations
Spatial structure is important in order to provide context for object localization
Sivic et. al., ICCV 2005Fei-Fei and Perona, CVPR 2005
Bag of words Spatially organized textures
Non localized textons
S. Lazebnik, et al, CVPR 2006Walker, Malik. Vision Research 2004 …
M. Gorkani, R. Picard, ICPR 1994A. Oliva, A. Torralba, IJCV 2001
…
An integrated model of Scenes, Objects, and Parts
Ncar
S
g
Scene
Scenegist
features
0
0
1
1
5
5
N
P(Ncar | S = street)
P(Ncar | S = park)
0 5 10 150
0.05
0.1
0.15
0.2
0 5 10 150
0.2
0.4
0.6
0.8
N
Context driven object detection
ZcarNcar
S
g
Scene
Scenegist
features
01 5
P(Ncar | S = street)
0 5 10 150
0.05
0.1
0.15
0.2
N
An integrated model of Scenes, Objects, and Parts
p(d | F=1) = N(d | m1, s1)p(d | F=0) = N(d | m0, s0)
We train a multiview car detector.
xcari
dcari
carFi
N=4
An integrated model of Scenes, Objects, and Parts
ZcarNcar
S
g
Scene
Scenegist
features
xcari
dcari
carFi
M=4
P(F,S | x,d,g) a p(F | S)p(S | g) p(xi | g) PN(xi; mb, sb2) PN(di; mtp, stp
2) PN(di; mtn, stn2)
i:Fi=0 i:Fi=1 i:Fi=0
A car out of context …
We are wired for 3D~6cm
We can not shut down 3D perception
(c) 2006 Walt Anthony
Scenes rule over objects
3D percept is driven by the scene, which imposes its ruling to the objects
3D from pixel valuesD. Hoiem, A.A. Efros, and M. Hebert, "Automatic Photo Pop-up”. SIGGRAPH 2005.
A. Saxena, M. Sun, A. Y. Ng. "Learning 3-D Scene Structure from a Single Still Image"In ICCV workshop on 3D Representation for Recognition (3dRR-07), 2007.
Surface EstimationImage Support Vertical Sky
V-Left V-Center V-Right V-Porous V-Solid
[Hoiem, Efros, Hebert ICCV 2005]
ObjectSurface?
Support?Slide by Derek Hoiem
Object Support
Slide by Derek Hoiem
Slide by James Coughlan
Slide by James Coughlan
3d Scene Context
Image World
Hoiem, Efros, Hebert ICCV 2005
3D scene context
meters
met
ers
Ped
Ped
Car
Hoiem, Efros, Hebert ICCV 2005
Qualitative Results
Initial: 2 TP / 3 FP Final: 7 TP / 4 FP
Local Detector from [Murphy-Torralba-Freeman 2003]
Car: TP / FP Ped: TP / FP
Slide by Derek Hoiem
3D City Modeling using Cognitive Loops
N. Cornelis, B. Leibe, K. Cornelis, L. Van Gool. CVPR'06
Single view metrologyCriminisi, et al. 1999
Need to recover:• Ground plane• Reference height• Horizon line• Where objects contact the ground