computing f class 8 read notes section 4.2. c1c1 c2c2 l2l2 l1l1 e1e1 e2e2 fundamental matrix (3x3...
Post on 20-Dec-2015
219 views
TRANSCRIPT
Computing FClass 8
Read notes Section 4.2
C1
C2
l2
l1
e1
e20m m 1
T2 F
Fundamental matrix (3x3 rank 2
matrix)
1. Computable from corresponding points
2. Simplifies matching3. Allows to detect wrong
matches4. Related to calibration
Underlying structure in set of matches for rigid scenes
l2
C1m1
L1
m2
L2
M
C2
m1
m2
C1
C2
l2
l1
e1
e2
m1
L1
m2
L2
M
l2lT1
Epipolar geometry
Canonical representation:
]λe'|ve'F][[e'P' 0]|[IP T
Other entities besides points?
Lines give no constraint for two view geometry(but will for three and more views)
Curves and surfaces yield some constraints related to tangency
(e.g. Sinha et al. CVPR’04)
The projective reconstruction theorem
If a set of point correspondences in two views determine the fundamental matrix uniquely, then the scene and cameras may be reconstructed from these correspondences alone, and any two such reconstructions from these correspondences are projectively equivalent
allows reconstruction from pair of uncalibrated images!
Today’s menu
Computation of F• Linear (8-point)• Minimal (7-point)• Robust (RANSAC)• Non-linear refinement (MLE, …)
• Practical approach
Epipolar geometry: basic equation
0Fxx'T
separate known from unknown
0'''''' 333231232221131211 fyfxffyyfyxfyfxyfxxfx
0,,,,,,,,1,,,',',',',',' T333231232221131211 fffffffffyxyyyxyxyxxx
(data) (unknowns)(linear)
0Af
0f1''''''
1'''''' 111111111111
nnnnnnnnnnnn yxyyyxyxyxxx
yxyyyxyxyxxx
the singularity constraint
0Fe'T 0Fe 0detF 2Frank
T333
T222
T111
T
3
2
1
VσUVσUVσUVσ
σσ
UF
SVD from linearly computed F matrix (rank 3)
T222
T111
T2
1
VσUVσUV0
σσ
UF'
FF'-FminCompute closest rank-2 approximation
the minimum case – 7 point correspondences
0f1''''''
1''''''
777777777777
111111111111
yxyyyxyxyxxx
yxyyyxyxyxxx
T9x9717x7 V0,0,σ,...,σdiagUA
9x298 0]VA[V T8
T ] 000000010[Ve.g.V
1...70,)xλFF(x 21T iii
one parameter family of solutions
but F1+F2 not automatically rank 2
F1 F2
F
3
F7pts
0λλλ)λFFdet( 012
23
321 aaaa
(obtain 1 or 3 solutions)
(cubic equation)
0)λIFFdet(Fdet)λFFdet( 1-12221
the minimum case – impose rank 2
Compute possible as eigenvalues of (only real solutions are potential solutions)
1-12 FF
B.detAdetABdet
0
1´´´´´´
1´´´´´´
1´´´´´´
33
32
31
23
22
21
13
12
11
222222222222
111111111111
f
f
f
f
f
f
f
f
f
yxyyyyxxxyxx
yxyyyyxxxyxx
yxyyyyxxxyxx
nnnnnnnnnnnn
~10000 ~10000 ~10000 ~10000~100 ~100 1~100 ~100
!Orders of magnitude differencebetween column of data matrix least-squares yields poor results
the NOT normalized 8-point algorithm
Transform image to ~[-1,1]x[-1,1]
(0,0)
(700,500)
(700,0)
(0,500)
(1,-1)
(0,0)
(1,1)(-1,1)
(-1,-1)
1
1500
2
10700
2
normalized least squares yields good results (Hartley, PAMI´97)
the normalized 8-point algorithm
Robust estimation
• What if set of matches contains gross outliers?(to keep things simple let’s consider line fitting first)
RANSACObjective
Robust fit of model to data set S which contains outliers
Algorithm
(i) Randomly select a sample of s data points from S and instantiate the model from this subset.
(ii) Determine the set of data points Si which are within a distance threshold t of the model. The set Si is the consensus set of samples and defines the inliers of S.
(iii) If the subset of Si is greater than some threshold T, re-estimate the model using all the points in Si and terminate
(iv) If the size of Si is less than T, select a new subset and repeat the above.
(v) After N trials the largest consensus set Si is selected, and the model is re-estimated using all the points in the subset Si
Distance threshold
Choose t so probability for inlier is α (e.g. 0.95)
• Often empirically• Zero-mean Gaussian noise σ then
follows distribution with m=codimension of model
2d2
m(dimension+codimension=dimension space)
Codimension
Model t 2
1 line,F 3.84σ2
2 H,P 5.99σ2
3 T 7.81σ2
How many samples?Choose N so that, with probability p, at least one random sample is free from outliers. e.g. p=0.99
sepN 11log/1log
peNs 111
proportion of outliers es 5% 10% 20% 25% 30% 40% 50%2 2 3 5 6 7 11 173 3 4 7 9 11 19 354 3 5 9 13 17 34 725 4 6 12 17 26 57 1466 4 7 16 24 37 97 2937 4 8 20 33 54 163 5888 5 9 26 44 78 272 117
7Note: Assumes that inliers allow to identify other inliers
Acceptable consensus set?
• Typically, terminate when inlier ratio reaches expected ratio of inliers
neT 1
Adaptively determining the number of samples
e is often unknown a priori, so pick worst case, i.e. 0, and adapt if more inliers are found, e.g. 80% would yield e=0.2
• N=∞, sample_count =0• While N >sample_count repeat
• Choose a sample and count the number of inliers• Set e=1-(number of inliers)/(total number of
points)• Recompute N from e• Increment the sample_count by 1
• Terminate
sepN 11log/1log
Other robust algorithms
• RANSAC maximizes number of inliers
• LMedS minimizes median error
• Not recommended: case deletion, iterative least-squares, etc.
100%
50%
residual(pixels)
inlier percentile
1.25
Non-linear refinment
algebraic minimization
possible to iteratively minimize algebraic distance subject to det F=0 (see H&Z if interested)
Geometric distance
Gold standard
Symmetric epipolar distance
Gold standard
Maximum Likelihood Estimation
i
iiii dd 22 'x̂,x'x̂,x
(= least-squares for Gaussian noise)
0x̂F'x̂ subject to T
iXt],|[MP'0],|[IP Parameterize:
Initialize: normalized 8-point, (P,P‘) from F, reconstruct Xi
iiii XP'x̂,PXx̂
Minimize cost using Levenberg-Marquardt(preferably sparse LM, e.g. see H&Z)
(overparametrized)
Gold standard
Alternative, minimal parametrization (with a=1)
(note (x,y,1) and (x‘,y‘,1) are epipoles)
problems: • a=0 pick largest of a,b,c,d to fix to 1
• epipole at infinity
pick largest of x,y,w and of x’,y’,w’
4x3x3=36 parametrizations!
reparametrize at every iteration, to be sure
Symmetric epipolar error
2
221
2
2T2
1T
T
FxFx
1
Fx'Fx'
1Fxx'
i
iiii dFd2T2 x'F,xx,x'
Some experiments:
Some experiments:
Some experiments:
Some experiments:
i
iiii dFd2T2 x'F,xx,x'
(for all points!)
Residual error:
Recommendations:
1. Do not use unnormalized algorithms
2. Quick and easy to implement: 8-point normalized
3. Better: enforce rank-2 constraint during minimization
4. Best: Maximum Likelihood Estimation (minimal parameterization, sparse implementation)
Special case:
Enforce constraints for optimal results:Pure translation (2dof), Planar motion (6dof), Calibrated case (5dof)
Step 1. Extract featuresStep 2. Compute a set of potential matchesStep 3. do
Step 3.1 select minimal sample (i.e. 7 matches)
Step 3.2 compute solution(s) for F
Step 3.3 determine inliers
until (#inliers,#samples)<95%
samples#7)1(1
matches#inliers#
#inliers 90%
80%
70% 60%
50%
#samples
5 13 35 106 382
Step 4. Compute F based on all inliersStep 5. Look for additional matchesStep 6. Refine F based on all correct matches
(generate hypothesis)
(verify hypothesis)
Automatic computation of F
Abort verification early
Assume percentage of inliers p, what is the probability P to pick n or more wrong samples out of m?stop if P<0.05 (sample most probably contains outlier) (P=cum. binomial distr. funct. (m-n,m,p ))
To avoid problems this requires to also verify at random!(but we already have a random sampler anyway)
(inspired from Chum and Matas BMVC2002)
O O O O O I O O I O O O OI I I O I I I I O O I O I I I I I I O I I I I I I O I O O I I I I…
O O O O O O O
O O O O I O O O O
restrict search range to neighborhood of epipolar line (e.g. 1.5 pixels)
relax disparity restriction (along epipolar line)
Finding more matches
• Degenerate cases• Planar scene• Pure rotation
• No unique solution• Remaining DOF filled by noise• Use simpler model (e.g. homography)
• Solution 1: Model selection (Torr et al., ICCV´98, Kanatani, Akaike)
• Compare H and F according to expected residual error (compensate for model complexity)
• Solution 2: RANSAC• Compare H and F according to inlier count
(see next slide)
Degenerate cases:
RANSAC for quasi-degenerate cases
• Full model (8pts, 1D solution)
• Planar model (6pts, 3D solution)
Accept if large number of remaining inliers
• Plane+parallax model (plane+2pts)
Sample for out of plane points among outliers
closest rank-6 of Anx9 for all plane inliers
(accept inliers to solution F)
(accept inliers to solution F1,F2&F3)
• Absence of sufficient features (no texture)• Repeated structure ambiguity
(Schaffalitzky and Zisserman, BMVC‘98)
• Robust matcher also finds Robust matcher also finds support for wrong hypothesissupport for wrong hypothesis• solution: detect repetition solution: detect repetition
More problems:
RANSAC for ambiguous matching
• Include multiple candidate matches in set of potential matches
• Select according to matching probability (~ matching score)
• Helps for repeated structures or scenes with similar features as it avoids an early commitment
(Tordoff and Murray ECCV02)
geometric relations between two views is fully
described by recovered 3x3 matrix F
two-view geometry
Next class: rectification and stereo
image I(x,y) image I´(x´,y´)Disparity map D(x,y)
(x´,y´)=(x+D(x,y),y)