note on coupled line cameras for rectangle reconstruction (acdde 2012)

Note on Coupled Line Cameras for Rectangle Reconstruction

Joo-Haeng Lee

Robot & Cognitive Systems Dept.ETRI

Asian Conference on Design and Digital Engineering 2012 (ACDDE 2012)Geometric Computing and CAD Workshop

Dec.6-8, 2012, Niseko, Hokkaido, Japan

Joo-Haeng Lee ([email protected])

Outline

• Problem definition

• Outline of proposed solution

• Illustrative example

• Theory: coupled line cameras

• Experimental results

• Q&A

mailto:[email protected]


QUIZ 1. You have some image quadrilaterals taken from a camera. Which of the following is the image of any rectangle?


(a) Rhombus (b) Parallelogram

(c) Trapezoid___ (d) IsoscelesTrapezoid


Parallelogram

(d) IsoscelesTrapezoid


Parallelogram

(d) IsoscelesTrapezoid

u0

u1 u2

u3l0

l1l2

l3r

v0

v1 v2

v3

f

Reconstructed RectangleGiven Image Quadrilateral

ReconstructedProjective Structure



(c) (d) IsoscelesTrapezoid


Problem Definition

• Given: (1) a single image of a scene rectangle of an unknown aspect ratio; (2) a simple camera model with unknown parameter values




Problem Definition

• Given: (1) a single image of a scene rectangle of an unknown aspect ratio; (2) a simple camera model with unknown parameter values

• Problem: (1) to reconstruct the projective structure including the scene rectangle; (2) to calibrate unknown camera parameters




Proposed Solution




Proposed Solution1. An analytic solution based on coupled

line cameras is provided for the constrained case where the center of a scene rectangle is projected to the image center.




Proposed Solution1. An analytic solution based on coupled

line cameras is provided for the constrained case where the center of a scene rectangle is projected to the image center.

2. By prefixing a simple pre-processing step, we can solve the general cases without the centering constraint above.




Proposed Solution

3. We also provide a determinant to tell if an image quadrilateral is a projection of any scene rectangle.




Proposed Solution

3. We also provide a determinant to tell if an image quadrilateral is a projection of any scene rectangle.

4. We present the experimental results of the proposed method with synthetic and real data.




Illustrative Example




Illustrative Example1. Assume a simple camera model with

unknown parameters: (ex) pinhole camera

(1) Assume a simple camera model with unknown parameters.

•  Square pixel: fx= fy

•  No skew: s = 0 •  Image center on the

principal axis

(2) When an image quadrilateral Qg is given,

(3) Find a centered quad Q using the vanishing points of Qg.

(4) We can determine if the the centered quad Q is the image of a scene rectangle.!

•  Determinant: D !

(5) If so, we can reconstruct the centered scene rectangle Gg in a metric sense before camera calibration.

(6) Finally, we can calibrate camera parameters:

•  focal length: f • external params: [R|T]

Given: (1) an image of a scene rectangle of an unknown aspect ratio; (2) a simple camera model with unknown parameter values: focal length, position, and orientation

Problem: (1) to reconstruct the projective structure including the scene rectangle; (2) to calibrate unknown camera parameters

Proposed Solution:

1.  Analytic solution based on coupled line cameras is provided when the center of a scene rectangle is projected to the image center.

2.  By prefixing a simple pre-processing step, we can solve the general cases without the centering constraint.

3.  We also provide a determinant to tell if an image quadrilateral is a projection of a scene rectangle.

4.  We demonstrate the performance of the proposed method with synthetic and real data.

Summary!

Illustrative Example what we can do!

Camera Calibration from a Single Image based on !Coupled Line Cameras and Rectangle Constraint!

Joo-Haeng Lee [email protected] !Robot & Cognitive Systems Dept., ETRI, KOREA!

Poster #5, Session TuPSAT2, ICPR 2012!

D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0

Line Camera a special linear camera model!

Given: (1) 1D image of a scene line denoted by l0 and l2; (2) the principal axis passes through the center of a scene line.

Solution: an analytic solution to the pose estimation of a line camera

cosθ

0= d

l0− l

2

l0+ l

2

= dα0

Coupled Line Cameras a special pin-hole camera model!

Given: (1) a centered quad Q; (2) the principal axis passes through the center of a scene rectangle G; (3) a diagonal angle of G:

Constraint: (1) for each diagonal of Q, a line camera can be defined; (2) these two line cameras should share the principal axis.

Solution: an analytic solution to the pose estimation of coupled line cameras

d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc

Synthetic: (1) generated 100 random rectangles: Gref; (2) added noises within dmax pixels to the vertices of Gref: Gerr; (3) relative errors: |vi-vm|, , pc, and f.

Real: (1) a rectangle with a known aspect ratio is moving on the desk: A4 paper (2) independently reconstructed and calibrated for 9 cases.

Experiments performance of the proposed method!

Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ

|vi-vm| pc f φ1 2 3

dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw

Reconstructed aspect ratio: φ Merged frustums

A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




Illustrative Example2. Given an image quadrilateral Qg,




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




Illustrative Example3. Find a centered quad Q using the

vanishing points of Qg




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




Illustrative Example3. Find a centered quad Q using the

vanishing points of Qg

a positive real number, the expression inside the outersquare root of (5) also should be a positive real number:

D± =A0 +A1 ± 2

!A0A1

A1 "A0

> 0 (6)

The above condition guarantees that a quadrilateral His the projection of a scene rectangle G, which will befully reconstructed in Section 3.1.

3. Reconstruction and Calibration

3.1. Reconstructing Projective Structure

We define a projective structure as a frustum with arectangular base G and an apex in the center of projec-tion pc. (See Fig 2.) Since G has a canonical form andparameterized by !, its vertices can be represented asv0 = (1, 0, 0), v1 = (cos!, sin!, 0), v2 = "v0, andv3 = "v1 where ! is the crossing angle of two diag-onals. Since #vi# = 1, the crossing angle ! is repre-sented as

cos! = $v0, v1% (7)

Two diagonals of the quadrilateral H intersect at theimage center um on the principal axis with the crossingangle ".

To compute !, we denote the center of projection aspcc = (0, 0, 0), the image center as uc

m = (0, 0,"1),the first two vertices of H as uc

0 = (tan#0, 0,"1)and uc

1 = (cos " tan#1, sin " tan#1,"1), the centerof G as vcm = (0, 0,"d), and the vertices of G asvci = siuc

i/#uci#. Since vi = vci " vcm, ! can be com-

puted using (7) with known values of ", d, si, and #i.The coordinates of pc = (x, y, z) can be found by

solving following equations: d cos $0 = x, d cos $1 =x cos!+ y sin!, and x2+ y2+ z2 = d2, which are de-rived from the projective structure. Now, the projectivestructure has been reconstructed as a frustum with thebase G(!) and the apex pc.

Note that the reconstructed G is guaranteed to be arectangle satisfying geometric constraints such as or-thogonality since the proposed method is based on acanonical configuration of coupled line camera of Sec-tion 2.

3.2. Camera Calibration

Since our reconstruction method is formulated usinga canonical configuration of coupled line cameras, thederivation based on H can be substituted with a givenimage quadrilateral Q. Now we can find a homographyH using four point correspondences between G(!) andQ. A homography is defined as H = sKW where s, K

w0w1

u3

g

u3

u1

u2

u0

g! u0

um

Qg

Q

Figure 3: Finding a centered quad Q (in blue) from aoff-centered quad Qg (in red) using vanishing points.

and W denote a scalar, a camera and a transformationmatrices, respectively. Note that we assume a simplecamera model K: the only unknown internal parameteris a focal length f . Since the model plane is at z = 0, itis straightforward to derive K and W. See [2] for details.

3.3. Off-Centered Quadrilateral

In a centered case above, the centers of a scene rect-angle G and an image quadrilateral Q are both alignedat the principal axis, which is exceptional in reality. Theproposed method, however, can be readily applied tothe off-centered case by prefixing a simple preprocess-ing step.

Let Qg and Gg denote an off-centered image quadri-lateral and a corresponding scene rectangle, respec-tively. In the preprocessing step, we will find a cen-tered quadrilateral Q which will reconstruct the projec-tive structure where the projective correspondences be-tween Qg and Gg are preserved. We can show that suchQ can be geometrically derived from Qg using the prop-erties of parallel lines and vanishing points. (See Fig 3.)

We choose one vertex ugi of Qg as the initial vertex

u0 of Q, say u0 = ug0. First, we compute two vanishing

points wi from Qg: w0 as intersection of ug0u

g1 and ug

2ug3

and w1 as intersection of ug0u

g3 and ug

1ug2. Let u2 be the

opposite vertex of u0 on the diagonal passing throughu0 and um: u2 = a(um " u0) + u0 where a is an un-known scalar defining u2. Then, we can make symbolicdefinitions of the other vertices of Q: u1 as intersectionof two lines u0w1 and u2w0, and u3 as intersection ofu0w0 and u2w1.

Note that the above definitions of ui guarantees thateach pair of opposite edges are parallel before projec-tive distortion. The symbolic definitions of ui abovecan be combined in one constraint: the intersection ofu0u2 and u1u3 equals to the image center um, whichis formulated as a single equation of one unknown a.Since we can find the value of a, all the vertices ui ofthe centered image quadrilateral Q can be computed ac-

760




Illustrative Example4. We can determine if the the centered

quad Q is the image of a scene rectangle.




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




Illustrative Example5. If so, we can reconstruct the scene

rectangles, G and Gg, in a metric sense before camera calibration.




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




Illustrative Example6. Finally, we can calibrate camera

parameters: (1) focal length f, (2) external params: [R|T]




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




Line Camera




Line Camera• Given: (1) 1D image of a scene line

denoted by l0 and l2; (2) the principal axis passes through the center m of a scene line.




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




Line Camera• Solution: an analytic solution to the pose

estimation of a line camera




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




Line Camera• Solution: an analytic solution to the pose

estimation of a line camera




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




Coupled Line Cameras




Coupled Line Cameras• Given: (1) a centered quad Q; (2) the

principal axis passes through the center of a scene rectangle G; (3) a diagonal angle φ"of G




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




Coupled Line Cameras• Constraint: (1) for each diagonal of Q, a

line camera can be defined; (2) these two line cameras share a principal axis.




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




Coupled Line Cameras• Solution: an analytic solution to the pose

estimation of coupled line cameras




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg









principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg






Q

t0

t1

k

!

"0

"1

# $0

d

$1

%0

%1

s0

s1

& G

x

y z

pc



d s0s2!0

"2 "0

v0v2 m

pc

l0l2

(a) Line camera

cv0v2 m

(b) Trajectory of the center of projection

Figure 1: A configuration of a line camera

scene, which will be projected as a line u0u2 in the linecamera C0. Especially, we are interested in the posi-tion pc and the orientation !0 of C0 when the principalaxis passes through the center vm of v0v2 and the centerum = (0, 0, 1) of image line.

To simplify the formulation, we assume a canon-

ical configuration where !vmvi! = 1, and vm isplaced at the origin of the world coordinate system:vm = (0, 0, 0). For derivation, we define followings:d = !pcvm!, li = !umui!, "i = !vmpcvi, andsi = !pcvi!.

In this configuration, we can derive the following re-lation:

l2l0

=d" cos !0d+ cos !0

=d0d1

(1)

where d0 = d " cos !0 = s0 cos"0 and d1 = d +cos !0 = s2 cos"2. We can derive the relation between!0 and d from (1):

cos !0 = d (l0 " l2)/(l0 + l2) = d #0 (2)

where #i = (li " li+2)/(li + li+2), which is solelyderived from a image line uiui+2. Note that !0 and dare sufficient parameters to determine the exact positionof pc in 2D. When #0 is fixed, pc is defined on a certainsphere as in Fig 1b. Once !i and d are known, additionalparameters can be also determined: tan"i = sin !i/dand si = sin !i/ sin"i.

2.2. Coupled Line Cameras

A standard pin-hole camera can be represented withtwo line cameras coupled by sharing the center of pro-jection. (See Fig 2.) Let a scene rectangle G havetwo diagonals v0v2 and v1v3, each of which followsthe canonical configuration in Section 2.1: !vmvi! = 1where vm = (0, 0, 0). Each diagonal vivi+2 is pro-jected to an image line uiui+2 in a line camera Ci.When two line cameras Ci share the center of projec-tion pc, two image lines uiui+2 intersect at um on thecommon principal axis, say pcvm. The four vertices ui

form a quadrilateral H , which is the projection of the

(a) Pin-hole Camera (b) Line Camera C0 (c) Line Camera C1

Figure 2: A pin-hole camera and its decomposition intocoupled line cameras.

rectangle G in a pin-hole camera with the center of pro-jection at pc. Note that the principal axis passes throughvm, um and pc.

Using this configuration of coupled line cameras, wefind the orientation !i of each line camera Ci and thelength d of the common principal axis from a givenquadrilateral H . Using the lengths of partial diagonals,li = !umui!, we can find the relation between the cou-pled cameras Ci from (1):

tan"1

tan"0

=l1l0

=sin !1(d" cos !0)

sin !0(d" cos !1)(3)

Manipulation of (2) and (3) leads to the system of non-linear equations:

d =$ sin !0 cos !1 " cos !0 sin !1

$ sin !0 " sin !1=

cos !0#0

=cos !1#1

(4)where $ = l1/l0. Using (4), the orientation !0 can berepresented with coefficients, #0, #1, and $, that aresolely derived from a quadrilateral H:

tan!02

=

!

A0 +A1 ± 2#A0A1

A1 "A0

="

D± (5)

where

A0 = B0 + 2B1, A1 = B0 " 2B1

B0 = 2(#0 " 1)2(#20 + #2

1)" 4#20(#1 " 1)2$2

B1 = (#0 " 1)2(#0 " #1)(#0 + #1)

The actual value of !0 should be chosen to make d > 0using (2). With known !0, we can compute values of!1 and d using (4). Based on the result of this section,the explicit coordinates of pc and the shape of G arereconstructed in Section 3.1.

2.3. Rectangle Determination

In our configuration of coupled line cameras, the ex-istence of !i implies that H is the projection of a canon-ical rectangle G. As the orientation angle !0 should be

759

d s0s2!0

"2 "0

v0v2 m

pc

l0l2

(a) Line camera

cv0v2 m

(b) Trajectory of the center of projection

Figure 1: A configuration of a line camera

scene, which will be projected as a line u0u2 in the linecamera C0. Especially, we are interested in the posi-tion pc and the orientation !0 of C0 when the principalaxis passes through the center vm of v0v2 and the centerum = (0, 0, 1) of image line.

To simplify the formulation, we assume a canon-

ical configuration where !vmvi! = 1, and vm isplaced at the origin of the world coordinate system:vm = (0, 0, 0). For derivation, we define followings:d = !pcvm!, li = !umui!, "i = !vmpcvi, andsi = !pcvi!.

In this configuration, we can derive the following re-lation:

l2l0

=d" cos !0d+ cos !0

=d0d1

(1)

where d0 = d " cos !0 = s0 cos"0 and d1 = d +cos !0 = s2 cos"2. We can derive the relation between!0 and d from (1):

cos !0 = d (l0 " l2)/(l0 + l2) = d #0 (2)

where #i = (li " li+2)/(li + li+2), which is solelyderived from a image line uiui+2. Note that !0 and dare sufficient parameters to determine the exact positionof pc in 2D. When #0 is fixed, pc is defined on a certainsphere as in Fig 1b. Once !i and d are known, additionalparameters can be also determined: tan"i = sin !i/dand si = sin !i/ sin"i.

2.2. Coupled Line Cameras

A standard pin-hole camera can be represented withtwo line cameras coupled by sharing the center of pro-jection. (See Fig 2.) Let a scene rectangle G havetwo diagonals v0v2 and v1v3, each of which followsthe canonical configuration in Section 2.1: !vmvi! = 1where vm = (0, 0, 0). Each diagonal vivi+2 is pro-jected to an image line uiui+2 in a line camera Ci.When two line cameras Ci share the center of projec-tion pc, two image lines uiui+2 intersect at um on thecommon principal axis, say pcvm. The four vertices ui

form a quadrilateral H , which is the projection of the

(a) Pin-hole Camera (b) Line Camera C0 (c) Line Camera C1

Figure 2: A pin-hole camera and its decomposition intocoupled line cameras.

rectangle G in a pin-hole camera with the center of pro-jection at pc. Note that the principal axis passes throughvm, um and pc.

Using this configuration of coupled line cameras, wefind the orientation !i of each line camera Ci and thelength d of the common principal axis from a givenquadrilateral H . Using the lengths of partial diagonals,li = !umui!, we can find the relation between the cou-pled cameras Ci from (1):

tan"1

tan"0

=l1l0

=sin !1(d" cos !0)

sin !0(d" cos !1)(3)

Manipulation of (2) and (3) leads to the system of non-linear equations:

d =$ sin !0 cos !1 " cos !0 sin !1

$ sin !0 " sin !1=

cos !0#0

=cos !1#1

(4)where $ = l1/l0. Using (4), the orientation !0 can berepresented with coefficients, #0, #1, and $, that aresolely derived from a quadrilateral H:

tan!02

=

!

A0 +A1 ± 2#A0A1

A1 "A0

="

D± (5)

where

A0 = B0 + 2B1, A1 = B0 " 2B1

B0 = 2(#0 " 1)2(#20 + #2

1)" 4#20(#1 " 1)2$2

B1 = (#0 " 1)2(#0 " #1)(#0 + #1)

The actual value of !0 should be chosen to make d > 0using (2). With known !0, we can compute values of!1 and d using (4). Based on the result of this section,the explicit coordinates of pc and the shape of G arereconstructed in Section 3.1.

2.3. Rectangle Determination

In our configuration of coupled line cameras, the ex-istence of !i implies that H is the projection of a canon-ical rectangle G. As the orientation angle !0 should be

759







principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




D>0 D<0

D<0D<0


(a) Rhombus Parallelogram

IsoscelesTrapezoid




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg

D>0


(a) Rhombus Parallelogram

IsoscelesTrapezoid

v0

v1 v2

v3

f

D>0


Experimental Results

• Synthetic Data

• Real Data




Real Data

•




Real Data




Real Data

•




Real Data




Synthetic Data1. Generated 100 random rectangles Gref and

corresponding image quads Qref;

2. Get image quad Qg by adding noises to Qref

within dmax pixels;

3. Reconstruct Gg from Qg;

4. Measured errors between Gref and Gg.




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




Synthetic Data




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




Real Data1. A rectangle with a known aspect ratio is

moving on a desk: (ex) A4-sized paper;

2. Take pictures to get 9 image quads;

3. Reconstructed and calibrated for each case.




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




Real Data




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0




cosθ

0= d

l0− l

2

l0+ l

2

= dα0





d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc




Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ


dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg




Summary• We proposed an analytic solution to

reconstruct a scene rectangle of an unknown aspect ratio from a single image quadrilateral.

• Our method is based on novel formulation of coupled line cameras and rectangle constraint.




Acknowledgement

This research has been partially supported by KMKE & KRC 2010-ZC1140 and KMKE ISTDP No.10041743




This research has been first presented at ICPR 2012 (Int. Conf. Pattern Recognition), Tsukuba, Japan. Nov., 2012.

Acknowledgement




principal axis










Proposed Solution:





Summary!





D

±= F

1(l

i) =

A0+ A

1± 2 A

0A

1

A1− A

0

> 0


Given: (1) 1D image of a scene line denoted by l0 and l2; (2) the principal axis passes through the center m of a scene line v0v2.


cosθ

0= d

l0− l

2

l0+ l

2

= dα0


Given: (1) a centered quad Q; (2) the principal axis passes through the center m of an unknown scene rectangle G. (Diag. angle= )



d =

cosθ0

α0

=cosθ

1

α1

= F2(θ

0,θ

1, l

i)

tan

θ0

2= F

1(l

i) = D

±

θ0→ d →θ

1→ψ

i→ s

i→φ

→ G → pc

Synthetic: (1) generated 100 random rectangles: Gref; (2) added noises within dmax pixels to the vertices of Gref; (3) relative errors between Gref and reconstructed Gg: |vi-m|, , pc, and f.



Qg

Q

Gg G

cv0v2 m

u0

u1

u2

u3

l0

l1

l2l3

r

φ

φ = 1.414;

d s0s2q0

y2 y0

v0v2 m

pc

l0l2

φQ

|vi-m| pc f φ1 2 3

dmax

123456

Error H%L

v0

v2

u0

u2 θ0 v1

v3

u1 u3

θ1

v0 v1

v2 v3

pc

Q

G m

Ê

Ê

Ê

Ê

Ê

Ê Ê

ÊÊ

‡

‡

‡

‡

‡

‡

‡

‡ ‡

1 2 3 4 5 6 7 8 9

Rect

ID

1.40

1.41

1.42

1.43

1.44

1.45

1.46

Aspect Ratio

‡ Compensated

Ê Raw


A moving A4 paper

1 2 3 4

5 6 7 8 9

Gref Q

Qg

G

Gg

Camera Calibration from a Single Image based on

Coupled Line Cameras and Rectangle Constraint

Joo-Haeng LeeRobot and Cognitive Systems Research Dept., ETRI

[email protected]

Abstract

Given a single image of a scene rectangle of an un-

known aspect ratio and size, we present a method to

reconstruct the projective structure and to find camera

parameters including focal length, position, and orien-

tation. First, we solve the special case when the center

of a scene rectangle is projected to the image center. We

formulate this problem with coupled line cameras and

present the analytic solution for it. Then, by prefixing

a simple preprocessing step, we solve the general case

without the centering constraint. We also provides a

determinant to tell if an image quadrilateral is a pro-

jection of a rectangle. We demonstrate the performance

of the proposed method with synthetic and real data.

1. Introduction

Camera calibration is one of the most classical topicsin computer vision research. We have an extensive listof related works providing mature solutions. In this pa-per, we are interested in a special problem to calibrate acamera from a single image of an unknown scene rect-angle. We do not assume any prior knowledge on cor-respondences between scene and image points. Due tothe limited information, a simple camera model is used:the focal length is the only unknown internal parameter.

The problem in this paper has a different nature withclassical computer vision problems. In the PnP prob-lem, we find transformation matrix between the sceneand the camera frames with prior knowledge on thecorrespondences between the scene and image pointsas well as the internal camera parameters [1]. In cam-era resection, we find the projection matrix from knowncorrespondences between the scene and image pointswithout prior knowledge of camera parameters [2].Camera self-calibration does not rely on a known Eu-clidean structure, but requires multiple images fromcamera motion [3].

Several approaches are based on geometric prop-erties of a rectangle or a parallelogram. Wu et al.

proposed a calibration method based on rectangles ofknown aspect ratio [4]. Li et al. designed a rectanglelandmark to localize a mobile robot with an approxi-mate rectangle constraint, which does not give an ana-lytic solution [5]. Kim and Kweon propose a method toestimate intrinsic camera parameters from two or morerectangle of unknown aspect ratio [6]. A parallelogramconstraint can be used for calibration, which generallyrequires more than two scene parallelograms projectedin multiple-view images as in [7, 8, 9].

Our contribution can be summarized as follows.Based on a geometric configuration of coupled linecameras, which models a simple camera of an unknownfocal length, we give an analytic solution to reconstructa complete projective structure from a single image ofan unknown rectangle in the scene: no prior knowledgeis required on the aspect ratio and correspondences.Then, the reconstruction result can be utilized in findingthe internal and external parameters of a camera: focallength, rotation and translation. The proposed solutionalso provides a simple determinant to tell if an imagequadrilateral is a projection of a scene rectangle.

In a general framework for plane-based camera cal-ibration, camera parameters can be found first usingthe image of the absolute conic (IAC) and its relationwith projective features such as vanishing points [2, 10].Then, a scene geometry can be reconstructed using anon-linear optimization on geometric constrains suchorthogonality, which cannot be formulated as a closed-form in general.

2. Problem Formulation

2.1. Line Camera

A line camera is a conceptual camera, which followsthe same projection model of a standard pin-hole cam-era. (See Fig 1a.) Let v0v2 be a line segment in the

21st International Conference on Pattern Recognition (ICPR 2012)November 11-15, 2012. Tsukuba, Japan

978-4-9906441-1-6 ©2012 IAPR 758




Q & A

joohaeng at etri dot re dot kr



note on coupled line cameras for rectangle reconstruction (acdde 2012)

Technology

d image

image quadrilateralstaken

single image

rt joohaeng lee

image quadrilateral

ca image center

proposed solutionjoohaeng

line camerajoohaeng