going back a little

Computer Vision : CISC 4/689

Going Back a little

• Cameras.ppt


Applications of RANSAC: Solution for affine parameters

• Affine transform of [x,y] to [u,v]:

• Rewrite to solve for transform parameters:


Assignment

• Program-1

• info-Link

• DataNote: You can generate, bring-in, your own images from www, as long as:For n+1 levels, image must be Mr£2n+1 rows by Mc£2n+1 colsMr and Mc are any +ve integers

Sunday 10pm


Another app. : Automatic Homography H Estimation

– Homographies describe image transformation of...• General scene when camera motion is rotation about camera center• Planar surfaces under general camera motion

• How to get correct correspondences without human intervention?from Hartley & Zisserman


Computing a Homography

• 8 degrees of freedom in 3 x 3 matrix H, so at least n = 4 pairs of 2-D points are sufficient to determine it

• Use same basic algorithm for P (aka Direct Linear Transformation, or DLT) to compute H

– Now stacked matrix A is 2n x 9 vs. 2n x 12 for camera matrix P estimation because all points are 2-D

• 3 collinear points in either image is a degenerate configuration preventing a unique solution

Lets Side-track


Estimating H: DLT Algorithm

• x0i = Hxi is an equation involving homogeneous vectors, so Hxi

and x0i need only be in the same direction, not strictly equal

• We can specify “same directionality” by using a cross product formulation:

• See Hartley & Zisserman, Chapter 3.1-3.1.1 (linked on course page) for details


Texture Mapping• Needed for nice display when applying transformations (like a homography H) to

a whole image• Simple approach: Iterate over source image coordinates and apply x0 = H x to

get destination pixel location– Problem: Some destination pixels may not be “hit”, leaving holes

• Easy solution: Iterate over destination image and apply inverse transform x = H-1

x0 – Round off H-1

x0 to address “nearest” source pixel value– This ensures every destination pixel is filled in


Automatic H Estimation: Feature Extraction

• Find features in pair of images using corner detection—e.g., eigenvalue threshold of:

from Hartley & Zisserman

~500 features found


Automatic H Estimation: Finding Feature Matches

• Best match over threshold within square search window (here §300 pixels) using SSD or normalized cross-correlation



Automatic H Estimation: Initial Match Hypotheses

268 matched features (over SSD threshold) in left image pointing to locations of corresponding right image features



Automatic H Estimation: Applying RANSAC

• Sampling– Size: Recall that 4 correspondences suffice to define homography, so sample size

s = 4– Choice

• Pick SSD threshold conservatively to minimize bad matches• Disregard degenerate configurations• Ensure points have good spatial distribution over image

• Distance measure– Obvious choice is symmetric transfer error:


Automatic H Estimation: Outliers & Inliers after RANSAC

• 43 samples used with t = 1.25 pixels

117 outliers (² = 0.44) 151 inliersfrom Hartley & Zisserman


A Short Review of Camera Calibration


Pinhole Camera Terminology

Camera center/ pinhole

Principal point/image center

Image pointCamera point

Focal length

Optical axis

Image plane


Calibration

• Slides (calibration.ppt)


Calibration and Pose estimation example

• Recover intrinsic and extrinsic parameters of camera by using calibration board.

• 3D points are given, can find 2D image coordinates for the corresponding 3D points.

• Assume world is located at the folded lower corner, principal point is center of the image, fold is 90 degrees,

• Total length and width of board is 9in by 9in.

Next 8 slides, courtesy UCF.


Matlab code

• Matlab• fx = 1.5031• fy =1.2773

• Rc = -0.0201 -0.2000 -0.9796 0.2198 0.9588 -0.1797 0.9752 -0.2189 0.0247

• Tc = (29.0725, -2.8850, 53.4196)

• camera position:( -51.0289, 19.7118 26.6985)


Multi-View GeometryRelates

• 3D World Points

• Camera Centers

• Camera Orientations


Multi-View GeometryRelates

• 3D World Points

• Camera Centers

• Camera Orientations

• Camera Intrinsic Parameters

• Image Points


Stereo

scene pointscene point

optical centeroptical center

image planeimage plane


Stereo

• Basic Principle: Triangulation– Gives reconstruction as intersection of two rays– Requires

• calibration• point correspondence


Stereo Constraints

p p’?

Given p in left image, where can the corresponding point p’in right image be?


Stereo Constraints

X1

Y1

Z1O1

Image plane

Focal plane

M

p p’Y2

X2

Z2O2

Epipolar Line

Epipole


Stereo

• The geometric information that relates two different viewpoints of the same scene is entirely contained in a mathematical construct known as fundamental matrix.

• The geometry of two different images of the same scene is called the epipolar geometry.


Stereo/Two-View Geometry

• The relationship of two views of a scene taken from different camera positions to one another

• Interpretations– “Stereo vision” generally means

two synchronized cameras or eyes capturing images

– Could also be two sequential views from the same camera in motion

• Assuming a static scene

http://www-sop.inria.fr/robotvis/personnel/sbougnou/Meta3DViewer/EpipolarGeo

http://www-sop.inria.fr/robotvis/personnel/sbougnou/Meta3DViewer/EpipolarGeo


3D from two-views

There are two ways of extracting 3D from a pair of images. • Classical method, called Calibrated route, we need to calibrate both

cameras (or viewpoints) w.r.t some world coordinate system. i.e, calculate the so-called epipolar geometry by extracting the essential matrix of the system.

• Second method, called uncalibrated route, a quantity known as fundamental matrix is calculated from image correspondences, and this is then used to determine the 3D.

Either way, principle of binocular vision is triangulation. Given a single image, the 3D location of any visible object point must lie on the straight line that passes through COP and image point (see fig.). Intersection of two such lines from two views is triangulation.


Mapping Points between Images

• What is the relationship between the images x, x’ of the scene point X in two views?

• Intuitively, it depends on:– The rigid transformation between cameras (derivable from the

camera matrices P, P’)– The scene structure (i.e., the depth of X)

• Parallax: Closer points appear to move more


Example: Two-View Geometry

courtesy of F. Dellaert

x1 x’1

x2x’2

x3 x’3

Is there a transformation relating the points xi to x’i ?


Epipolar Geometry

• Baseline: Line joining camera centers C, C’• Epipolar plane ¦: Defined by baseline and scene point X

from Hartley& Zisserman

baseline


Epipolar Lines

• Epipolar lines l, l’: Intersection of epipolar plane ¦ with image planes• Epipoles e, e’: Where baseline intersects image planes

– Equivalently, the image in one view of the other camera center.

C C’from Hartley& Zisserman


Epipolar Pencil

• As position of X varies, epipolar planes “rotate” about the baseline (like a book with pages)

– This set of planes is called the epipolar pencil• Epipolar lines “radiate” from epipole—this is the pencil of epipolar lines



Epipolar Constraint• Camera center C and image point define ray in 3-D space that projects to epipolar line l’ in other

view (since it’s on the epipolar plane)• 3-D point X on this ray, so image of X in other view x’ must be on l’• In other words, the epipolar geometry defines a mapping x ! l’, of points in one image to

lines in the other


C C’

x’


Example: Epipolar Lines for Converging Cameras

from Hartley & ZissermanLeft view Right view

Intersection of epipolar lines = Epipole ! Indicates direction of other camera


Special Case: Translation Parallel to Image Plane

Note that epipolar lines are parallel and corresponding points lie on correspond-ing epipolar lines (the latter is true for all kinds of camera motions)


From Geometry to Algebra

O O’

P

pp’

Courtesy, UCF


From Geometry to Algebra

O O’

P

pp’


Linear Constraint:Should be able to express as matrix multiplication.

going back a little

Documents