multiview geometry and stereopsis. inputs: two images of a scene (taken from 2 viewpoints). output:...

Multiview Geometry and Stereopsis

Inputs: two images of a scene (taken from 2 viewpoints) . Output: Depth map.

Inputs: multiple images of a scene. Output: 3D model.

Stereopsis

Parallax

Wikipedia

Compute Correspondences

Compute Correspondences

Correspondences

Geometric Constraints (given by epipolar geometry) make this 1D search problem instead of 2D.

Reconstruction / Triangulation

Reconstruction

• Linear Method: find P such that

• Non-Linear Method: find Q minimizing

Geometry of Two-View

• Epipolar geometry– Essential matrix– Fundamental matrix

Stereopsis Define a (linear) function F(p1, p2) such that it is zero if p1 and p2 are corresponding points in two images.

• Epipolar plane defined by P, O, O’, p and p’

• Epipoles e, e’

• Epipolar lines l, l’

• Baseline OO’

Epipolar geometry

p’ lies on l’ where the epipolar plane intersects with image plane π’

l’ is epipolar line associated with p and intersects baseline OO’ on e’

e’ is the projection of O observed from O’

• Potential matches for p have to lie on the corresponding epipolar line l’

• Potential matches for p’ have to lie on the corresponding epipolar line l

Epipolar constraint

Epipolar Constraint: Calibrated Case

Essential Matrix(Longuet-Higgins, 1981)3 ×3 skew-symmetric

matrix: rank=2

O’P’ = t + R p’

• E is defined by 5 parameters (scaling not relevant)

• E p’ is the epipolar line associated with p’

• E T p is the epipolar line associated with p

• Can write as l .p = 0

• The point p lies on the epipolar line associated with the vectorE p’

Properties of essential matrix

Properties of essential matrix

• E is defined by 5 parameters (scaling not relevant)

• E e’=0 and ETe=0 (E e’=-RT[tx]e=0 )

• E is singular• E has two equal non-zero singular values

Epipolar Constraint: Uncalibrated Case

Fundamental Matrix(Faugeras and Luong, 1992)are normalized image coordinate pp ˆ,ˆ

• F has rank 2 and is defined by 7 parameters

• F p’ is the epipolar line associated with p’ in the 1st image

• F T p is the epipolar line associated with p in the 2nd image

• F e’=0 and F T e=0

• F is singular

Properties of fundamental matrix

Weak calibration

• In theory: – E can be estimated with 5 point

correspondences– F can be estimated with 7 point

correspondences– Some methods estimate E and F matrices from

a minimal number of parameters

• Estimating epipolar geometry from a redundant set of point correspondences with unknown intrinsic parameters

The Eight-Point Algorithm (Longuet-Higgins, 1981)

|F | =1.

Minimize:

under the constraint2

Homogenous system, set F33=1

Enforcing Rank 2 Constraint

• Singular Value Decomposition of F (svd in MATLAB)

Sigma is diagonal (its (non-negative) values are called singular values of F). U, V are orthogonal.

Set the smallest singular value to zero.

The Normalized Eight-Point Algorithm (Hartley, 1995)

• Estimation of transformation parameters suffer from poor numerical condition problem

• Center the image data at the origin, and scale it so themean squared distance between the origin and the data points is 2 pixels: q = T p , q’ = T’ p’

• Use the eight-point algorithm to compute F from thepoints q and q’

• Enforce the rank-2 constraint (use singular value decomposition)

• Output T F T’

T

Weak calibration experiment

a) Linear Least Squares

b) Hartley