© 2005 martin bujňák, [email protected], dataexpert.sk martin bujňák [email protected]...

© 2005 Martin Bujňák, [email protected], http://www.DataExpert.sk/Research

Martin Bujňá[email protected]

Supervisor : RNDr. Martin Samuelčík

On-line Structure from Motion


Aim

• Reconstruction of sparse 3D model of the scene using video sequence

• Input : video sequence captured with standard hand-held camera

• As small as possible user interactions

• Output :– Sparse 3D model– Camera calibration parameters

• Intrinsic

• Extrinsic


Feature detection


Feature detection

• Harris based detector– Removing features that can be

interchanged with its neighborhood

– Introducing feature orientation


Features matching

• Find the nearest (distance) similar (correlation) feature in the next frame

• Matching ambiguities– two neighboring frames don’t

differ a lot – „outliers“ are effectively

removed in further processes (up to 30% of bad pairs is expected)

??


Guided searching

• F-matrix : two view geometry– Restricts searching for

matching pair to 1D search region in the second frame

– Using “stereo” principle• Epipolar line ordering constrain• Ratio of corresponding

segments is near 1 for small motion between two frames

?x


Guided searching


Projective reconstruction• Canonical camera pair from

F-matrix– F-matrix calculated using

RANSAC paradigm– 4 free parameters calculated

so that camera pair satisfies calibration conditions

• Principal point at image center• Aspect ration of 1• Zero skew• Unknown varying focal length


Projective reconstruction

• sparse 3D reconstruction obtained from camera pair– Intersection of sightlines passing

through known feature pairs

• Quasi calibration– Angles are near to “calibrated”

case• Estimation of precision of the 3D

point (size of area where 3D point can be put)


Structure from motion (SfM)

• Merging camera pairs– Merge using common 2D-

3D feature points– Search for projective

transformation H • Find H or H-1 depending

on quality of 3D points• All measurements are

performed in 2D• RANSAC paradigm

– Linear solution• New more precise SfM

– Recalculating structure– Recalculating camera

motionNew pair

Scene

4x4 homography H

H-1

3px noise added to images


Self-Calibration

**

projection

constraints

• Pinhole camera :– Expressed by 3x4 matrix– Can be decomposed into calibration

3x3 (K), rotation 3x3 (R) and translation (T) 3x1 matrix

• Self-calibration– Means searching for calibration

parameters – only from input images– Math : Dual Absolute Quadrics(Ω*)

• KKT = P(H Ω* HT)PT

KKT = (PH) Ω* (HTPT)

– Linear solution with one cubical constrain


Experiment

• Detection and correction of lens radial distortion– Polynomial model :

X1 = (1 + a(x02+y0

2))* x0

Y1 = (1 + a(x02+y0

2))* y0

– Search for ‘a’ using F-matrix

• minimizing distance of matching feature points from their corresponding epipolar lines

– Numerical minimization • simulated annealing


Conclusion

• Contribution– New feature tracker– Enhanced guided matching – Sequential SfM without need of searching for initial

frame– Lens radial distortion correction

• Results– Video

• Current state– video

© 2005 martin bujňák, [email protected], dataexpert.sk martin bujňák [email protected]...

Documents