multiple light source optical flow multiple light source optical flow robert j. woodham iccv’90
TRANSCRIPT
Multiple Light Source Multiple Light Source
Optical FlowOptical Flow
Multiple Light Source Multiple Light Source
Optical FlowOptical Flow
Robert J. Woodham
ICCV’90
Introduction
Optical Flow Definition
Is a vector field that shows the direction and magnitude of the intensity changes from one image to the other
Is a vector field that shows the direction and magnitude of the intensity changes from one image to the other
Main Idea
Use the intensity color values recorded from multiple Images of moving objects acquired simultaneously
under different illumination conditions to calculate optical flow
Some considerations
Object Motion vs. brightness change
Not purely geometricDepends on radiometric factors
(illumination, reflectance)
Not purely geometricDepends on radiometric factors
(illumination, reflectance)
The idea is based on. . .
Photometric stereo uses multiple conditions of illumination to determine shape from shading
Theory
Optical Flow Constraint Equation
dE/dt=Exu + Eyv + Et
where
E = E(x,y,t) be the image brightness at point (x,y) as a function of time t
Ex = E/x, Ey = E/y, Et = E/t (partial derivatives of E with respect to x, y and t)
u =dx/dt and v= dy/dt (instantaneous flow in the point (x,y).
Theory (2)
If the brightness does not change as consequence of motion . . .
Exu + Eyv + Et = 0
Validity conditions
Purely translational motion,Orthographic projection
Uniform incident illumination
Theory (3)
Equation properties
•Cannot be solved locally – 1 equation with 2 unknowns
•Variation in scene illumination cause dE/dt0
•Objects acts as indirect sources of illumination (inter-reflection)
•Locations of brightness discontinuity – undefined points.
Exu + Eyv + Et = 0
Using Multiple Light Sources
t
t
yx
yx
E
E
EE
EE
v
u
2
1
1
22
11
E1xu + E1yv + E1t = 0E1xu + E1yv + E1t = 0
E2xu + E2yv + E2t = 0E2xu + E2yv + E2t = 0
::
For 2 light sources
3 Light Sources
E1xu + E1yv + E1t = 0E1xu + E1yv + E1t = 0
E2xu + E2yv + E2t = 0E2xu + E2yv + E2t = 0
E3xu + E3yv + E3t = 0E3xu + E3yv + E3t = 0
Ax = b
x = [u,v]T
b = -[E1t,, E2t ,E3t]T
yx
yx
yx
EE
EE
EE
A
33
22
11
x = (ATA)-1ATbStandard Least Square solution
Overdetermined Problem
Implementation
3 under different illumination condition at time t0
3 same illumination as time t0, with same background but different object position
6 images
3 images taken under different illumination condition in t0
Implementation (2)
Multiple light source optical flow computation at one point
u
v
3 Flow constraint lines
Results
Optical Flow vectors
•Estimation is good where the surface is smoothly shaded
•In the collar dark points degenerate the results
•In the discontinuities, due change of image brightness the estimates is also inaccurate
•Vector in the background due the shadows and inter-reflection
Practical Implementation
Can be used 3 light sources (red, green and blue) continuously illuminating a workspace
The capture can be made using cameras to capture different spectral channels
Conclusion
•The method works better in smoothly curves (not distinct surface markings and the local brightness depends on local shading)
•Restrictions in surface discontinuities and surface markings because local brightness change is dominated by scene features (largely independent of the illumination direction)