optical flow 10-24-2005. problem problems in motion estimation –noise, –color (intensity)...
Post on 21-Dec-2015
216 views
TRANSCRIPT
![Page 1: Optical Flow 10-24-2005. Problem Problems in motion estimation –Noise, –color (intensity) smoothness, –lighting (shadowing effects), –occlusion, –abrupt](https://reader030.vdocuments.site/reader030/viewer/2022032521/56649d575503460f94a35b2a/html5/thumbnails/1.jpg)
Optical Flow
10-24-2005
![Page 2: Optical Flow 10-24-2005. Problem Problems in motion estimation –Noise, –color (intensity) smoothness, –lighting (shadowing effects), –occlusion, –abrupt](https://reader030.vdocuments.site/reader030/viewer/2022032521/56649d575503460f94a35b2a/html5/thumbnails/2.jpg)
Problem
• Problems in motion estimation– Noise, – color (intensity) smoothness, – lighting (shadowing effects), – occlusion, – abrupt movements, etc
• Approaches:– Block matching,– Generalized block matching,– Optical flow (block-based, Horn-Schunck etc)– Bayesian, etc.
• Applications– Video coding and compression,– Segmentation– Object reconstruction (structure-from-motion)– Detection and tracking, etc.
![Page 3: Optical Flow 10-24-2005. Problem Problems in motion estimation –Noise, –color (intensity) smoothness, –lighting (shadowing effects), –occlusion, –abrupt](https://reader030.vdocuments.site/reader030/viewer/2022032521/56649d575503460f94a35b2a/html5/thumbnails/3.jpg)
Motion description• 2D motion:
p = [x(t),y(t)] p’= [x(t+ t0), y(t+t0)]
d(t) = [x(t+ t0)-x(t),y(t+t0)-y(t)]
• 3D motion:
Α = [ Χ1, Υ1, Ζ1 ]Τ Β = [ Χ2, Υ2, Ζ2 ]
Τ
= R + T
• Basic projection models: Orthographic
Perspective
x x(t) d1(t)
y y(t) d2(t)
X
Y
Z
2
2
2
X
Y
Z
1
1
1
YyXx
Z
Yy
Z
Xx
l
ll
l
![Page 4: Optical Flow 10-24-2005. Problem Problems in motion estimation –Noise, –color (intensity) smoothness, –lighting (shadowing effects), –occlusion, –abrupt](https://reader030.vdocuments.site/reader030/viewer/2022032521/56649d575503460f94a35b2a/html5/thumbnails/4.jpg)
Optical Flow
• Basic assumptions: – Image is smooth locally– Pixel intensity does not change over time (no lighting changes)
• Normal flow:
• Second order differential equation:
![Page 5: Optical Flow 10-24-2005. Problem Problems in motion estimation –Noise, –color (intensity) smoothness, –lighting (shadowing effects), –occlusion, –abrupt](https://reader030.vdocuments.site/reader030/viewer/2022032521/56649d575503460f94a35b2a/html5/thumbnails/5.jpg)
Block-based Optical Flow Estimation
• Optical flow estimation within a block (smoothness assumption): all pixels of the block have the same motion
• Error:
• Motion equation:
and
![Page 6: Optical Flow 10-24-2005. Problem Problems in motion estimation –Noise, –color (intensity) smoothness, –lighting (shadowing effects), –occlusion, –abrupt](https://reader030.vdocuments.site/reader030/viewer/2022032521/56649d575503460f94a35b2a/html5/thumbnails/6.jpg)
Horn-Schunck
• We want an optical flow field that satisfies the Optical Flow Equation with the minimum variance between the vectors (smoothness)
Gauss-Seidel
![Page 7: Optical Flow 10-24-2005. Problem Problems in motion estimation –Noise, –color (intensity) smoothness, –lighting (shadowing effects), –occlusion, –abrupt](https://reader030.vdocuments.site/reader030/viewer/2022032521/56649d575503460f94a35b2a/html5/thumbnails/7.jpg)
Derivative Estimation with Finite differences
![Page 8: Optical Flow 10-24-2005. Problem Problems in motion estimation –Noise, –color (intensity) smoothness, –lighting (shadowing effects), –occlusion, –abrupt](https://reader030.vdocuments.site/reader030/viewer/2022032521/56649d575503460f94a35b2a/html5/thumbnails/8.jpg)
Example 1
![Page 9: Optical Flow 10-24-2005. Problem Problems in motion estimation –Noise, –color (intensity) smoothness, –lighting (shadowing effects), –occlusion, –abrupt](https://reader030.vdocuments.site/reader030/viewer/2022032521/56649d575503460f94a35b2a/html5/thumbnails/9.jpg)
Example 2
![Page 10: Optical Flow 10-24-2005. Problem Problems in motion estimation –Noise, –color (intensity) smoothness, –lighting (shadowing effects), –occlusion, –abrupt](https://reader030.vdocuments.site/reader030/viewer/2022032521/56649d575503460f94a35b2a/html5/thumbnails/10.jpg)
Example 3: frame reconstruction
Reconstructed I2 (second) frame
Reconstructed I2 (second) frame
![Page 11: Optical Flow 10-24-2005. Problem Problems in motion estimation –Noise, –color (intensity) smoothness, –lighting (shadowing effects), –occlusion, –abrupt](https://reader030.vdocuments.site/reader030/viewer/2022032521/56649d575503460f94a35b2a/html5/thumbnails/11.jpg)
Application Examples