defending thesis (english)

59
3-D VISUAL RECONSTRUCTION : A SYSTEM PERSPECTIVE Student: Guillermo Enrique Medina Zegarra Advisor: PhD. Edgar Lobaton, USA Co-Advisor PhD. Nestor Calvo, Argentina Arequipa, Per´ u May 07, 2012

Upload: guillermo-medina-zegarra

Post on 05-Dec-2014

385 views

Category:

Technology


2 download

DESCRIPTION

You can see the slides of my defending thesis, title: "3-D Visual Reconstruction: A System Perspective"

TRANSCRIPT

Page 1: Defending thesis (english)

3-D VISUAL RECONSTRUCTION : ASYSTEM PERSPECTIVE

Student:

Guillermo Enrique Medina Zegarra

Advisor:PhD. Edgar Lobaton, USA

Co-AdvisorPhD. Nestor Calvo, Argentina

Arequipa, Peru

May 07, 2012

Page 2: Defending thesis (english)

1

Index

Index

1 Introduction

2 Image formation

3 Geometry from two views

4 Proposal

5 Results

6 Limitations and problems founded

7 Conclusions and future work

Page 3: Defending thesis (english)

2

Index

1 IntroductionMotivation and contextDefinition the problemGeneral objectiveSpecific objectives

2 Image formation

3 Geometry from two views

4 Proposal

5 Results

6 Limitations and problems founded

7 Conclusions and future work

Page 4: Defending thesis (english)

3

Motivation and context

Limitations on pre-Renaissance to create 3D.

The artists during the Renaissance and the depth.

The vanishing points and three dimensions.

(a) Jesus intoJerusalem

(b) The School of Athens

Figura: Painting pre-Renaissance and Renaissance [Ma et al., 2004].

Page 5: Defending thesis (english)

3

Motivation and context

Limitations on pre-Renaissance to create 3D.

The artists during the Renaissance and the depth.

The vanishing points and three dimensions.

(a) Jesus intoJerusalem

(b) The School of Athens

Figura: Painting pre-Renaissance and Renaissance [Ma et al., 2004].

Page 6: Defending thesis (english)

3

Motivation and context

Limitations on pre-Renaissance to create 3D.

The artists during the Renaissance and the depth.

The vanishing points and three dimensions.

(a) Jesus intoJerusalem

(b) The School of Athens

Figura: Painting pre-Renaissance and Renaissance [Ma et al., 2004].

Page 7: Defending thesis (english)

4

Definition the problem

Physical architecture, location, distribution and lighting.

(a) One camara[Cipolla et al., 2010]

(b) Artificial lighting[VISGRAF., 2012]

Page 8: Defending thesis (english)

5

Definition the problem (cont...)

Figura: How to get the parameters to map an object to the image plane? [Faugeras, 1993].

Page 9: Defending thesis (english)

6

Definition the problem (cont...)

Figura: How to find corresponding points ? [Szeliski, 2011].

Page 10: Defending thesis (english)

7

Definition the problem (cont...)

Figura: How to find a 3D point of each pair of corresponding points ?[Szeliski, 2011].

Page 11: Defending thesis (english)

8

Definition the problem (cont...)

Figura: How to reconstruction and smooth a surface from a cloud ofpoints ? [Hartley and Zisserman, 2004].

Page 12: Defending thesis (english)

9

General objective

Objetivo general

Propose a model for the reconstruction of a 3D image of an objectfrom two images captured by two cameras located adequately.

Page 13: Defending thesis (english)

10

Specific objectives

Specific objectives

Position two digital cameras on a physical architecture for imageacquisition and calibration.

Rectification of the stereo image pair and calculate the disparitymap through the normalized cross-correlation.

Create the object’s surface from the Delaunay triangulation of thedisparity map.

Page 14: Defending thesis (english)

11

Index

1 Introduction

2 Image formation

3 Geometry from two views

4 Proposal

5 Results

6 Limitations and problems founded

7 Conclusions and future work

Page 15: Defending thesis (english)

12

Pinhole camera model

Help to understand the image formation from geometric point ofview.

Parts of the pinhole camera model: optical center (o), focaldistance (f ) and image plane (I ).

x = op ∩ I x ∈ R2 , p ∈ R3

Figura: Pinhole camera model [Ma et al., 2004].

Page 16: Defending thesis (english)

13

Pinhole camera model (cont...)

Figura: Example of the projection of an object on image plane .

Page 17: Defending thesis (english)

14

Index

1 Introduction

2 Image formation

3 Geometry from two views

4 Proposal

5 Results

6 Limitations and problems founded

7 Conclusions and future work

Page 18: Defending thesis (english)

15

Epipolar geometry

Study the geometric relationship and mathematical analysis of a3-D p point in their image planes.

Figura: Two projections x1, x2 ∈ R2 of a 3-D point p from two vantagepoints [Ma et al., 2004].

Page 19: Defending thesis (english)

16

Epipolar geometry (cont...)

Figura: Example of the projection of a cube image on two image planes.

Page 20: Defending thesis (english)

17

Rectification

Figura: Rectification of the stereo image pair [Fusiello et al., 2000].

Page 21: Defending thesis (english)

18

Disparity calculation

(a) (b) Disparity map

(a - b) Tsukuba image pair [Scharstein and Szeliski, 2002].

Page 22: Defending thesis (english)

19

Index

1 Introduction

2 Image formation

3 Geometry from two views

4 Proposal

5 Results

6 Limitations and problems founded

7 Conclusions and future work

Page 23: Defending thesis (english)

20

Pipeline of the proposal

Page 24: Defending thesis (english)

21

Description of the proposed pipeline

Physical architecture

Image acquisition

Calibration

Canon SD1200 Sony DSC-S750

Page 25: Defending thesis (english)

21

Description of the proposed pipeline

Physical architecture

Image acquisition

Calibration

features Sony DSC-S750 Canon SD1200 ISSensor type CCD CCDImage size 640 × 480 640 × 480ISO 100 100Flash off off

Technical characteristics of the two digital cameras

Page 26: Defending thesis (english)

21

Description of the proposed pipeline

Physical architecture

Image acquisition

Calibration

Chessboard (7 × 10)

Page 27: Defending thesis (english)

22

Description of the proposed pipeline

Rectification

Linear search

The correspondence of points isin the same horizontal line

Original images Rectified images

Page 28: Defending thesis (english)

23

Description of the proposed pipeline

Pre-processing

Manual segmentation

Gaussian filter

Rectified images Pre-processed images

Page 29: Defending thesis (english)

24

Description of the proposed pipeline

Disparity map

Normalized Cross-Correlation

Median filter

Left image pre-processed Right image pre-processed Disparity map

Page 30: Defending thesis (english)

25

Description of the proposed pipeline

3-D mesh

Delaunay triangulation

Intersection of lines

3-D mesh

Disparity map Point Cloud 3-D mesh

Page 31: Defending thesis (english)

26

Description of the proposed pipeline

Reconstructed model

Smoothing the surface

Texturing of the right image

Creation of the surface Smoothing of the surface Texturing of the model

Page 32: Defending thesis (english)

27

“Cubo magico”

Original images Rectified images Pre-processed images

Page 33: Defending thesis (english)

28

“Cubo magico” (cont...)

Disparity map Point Cloud 3-D mesh

Creation of the surface Smoothing of the surface Texturing of the model

Page 34: Defending thesis (english)

29

Multiple views of the “Magic Cube”

Page 35: Defending thesis (english)

29

Multiple views of the “Magic Cube”

Page 36: Defending thesis (english)

29

Multiple views of the “Magic Cube”

Page 37: Defending thesis (english)

29

Multiple views of the “Magic Cube”

Page 38: Defending thesis (english)

29

Multiple views of the “Magic Cube”

Page 39: Defending thesis (english)

29

Multiple views of the “Magic Cube”

Page 40: Defending thesis (english)

29

Multiple views of the “Magic Cube”

Page 41: Defending thesis (english)

29

Multiple views of the “Magic Cube”

Page 42: Defending thesis (english)

30

Index

1 Introduction

2 Image formation

3 Geometry from two views

4 Proposal

5 ResultsTeddy bearHuman face

6 Limitations and problems founded

7 Conclusions and future work

Page 43: Defending thesis (english)

31

Teddy bear

Original images Rectified imagenes Pre-processed imagenes

Page 44: Defending thesis (english)

32

Teddy bear (cont...)

Disparity map Cloud of points 3-D mesh

Creation of the surface Smoothing of the surface Texturing of the model

Page 45: Defending thesis (english)

33

Human face

Original images Rectified imagenes Pre-processed imagenes

Page 46: Defending thesis (english)

34

Human face (cont...)

Cloud of points Model without smoothing Smoothing model “Transformed” model

3-D mesh Model without smoothing Smoothing model “Transformed” model

Page 47: Defending thesis (english)

35

Index

1 Introduction

2 Image formation

3 Geometry from two views

4 Proposal

5 Results

6 Limitations and problems founded

7 Conclusions and future work

Page 48: Defending thesis (english)

36

Limitations and problems founded

neighbourhood size Imperfections of the created model

Page 49: Defending thesis (english)

37

Limitations and problems founded (cont...)

Imperfect original image Imperfect original image

Wrong disparity map “Amorphous” 3-D reconstruction

Page 50: Defending thesis (english)

38

Index

1 Introduction

2 Image formation

3 Geometry from two views

4 Proposal

5 Results

6 Limitations and problems founded

7 Conclusions and future work

Page 51: Defending thesis (english)

39

Conclusions

Physic architecture was designed simple and profit.

Lighting conditions must be adequate.

A pipeline was proposed with a sequence of steps needed to get a3-D reconstruction of a stereo image pair.

The method for the disparity calculation is simple and no robust.

There is a strong dependency between each step of thereconstruction.

Page 52: Defending thesis (english)

40

Future work

Create an environment withappropriate conditions for calibration,lighting and image acquisition.

Physical architecture and artificial lighting [Bradley et al., 2008]

Use multiple cameras.

Multiple views [Hartley and Zisserman, 2004]

Page 53: Defending thesis (english)

40

Future work

Create an environment withappropriate conditions for calibration,lighting and image acquisition.

Physical architecture and artificial lighting [Bradley et al., 2008]

Use multiple cameras.

Multiple views [Hartley and Zisserman, 2004]

Page 54: Defending thesis (english)

41

Future work (cont...)

Use robust methods.

Page 55: Defending thesis (english)

42

Publicated on

Symposium article“3-D visual reconstruction : a system perspective.”G. Medina-Zegarra y E. Lobaton2nd International Symposium on Innovation andTechnology (2011)pag. 102-107, November 28-30, Lima, PeruISBN: 978-612-45917-1-6Place: Technological University of Peru

Editor: International Institute of Innovation andTechnology (IIITEC)Chair: Mario Chauca Saavedra

Page 56: Defending thesis (english)

43

Acknowledgements

X PhD. Alex CuadrosX PhD. Alfedro MirandaX Mag. Alfedro PazX PhD. Carlos LeytonX PhD(c). Christian Lopez del AlamoX PhD. Eduardo TejadaX PhD. Jesus MenaX PhD. Jose Corrales-NievesX PhD(c). Juan Carlos GutierrezX Lic. Luıs ParejaX Family Barrios Neyra

Page 57: Defending thesis (english)

44

References

Bradley, D., Popa, T., Sheffer, A., Heidrich, W., and Boubekeur, T. (2008).

Markerless garment capture.ACM Transactions on Graphics (TOG), 27:99:1–99:9.

Cipolla, R., Battiato, S., and Farinella, G. M. (2010).

Computer Vision: Detection, Recognition and Reconstruction.Springer.

Faugeras, O. (1993).

Three-dimensional Computer Vision: A Geometric Viewpoint.The MIT Press. ISBN: 0262061589.

Fusiello, A., Trucco, E., and Verri, A. (2000).

A compact algorithm for rectification of stereo pairs.Machine Vision and Applications, 12:16–22.

Hartley, R. and Zisserman, A. (2004).

Multiple View Geometry in Computer Vision. Second Edition.Cambridge University Press. ISBN: 0521540518.

Ma, Y., Soatto, S., Kosecka, J., and Sastry, S. S. (2004).

An Invitation to 3D Vision from Images to Geometric Models.Springer. ISBN: 0387008934.

Scharstein, D. and Szeliski, R. (2002).

A taxonomy and evaluation of dense two-frame stereo correspondence algorithms.International Journal of Computer Vision, 47:7–42.

Szeliski, R. (2011).

Computer Vision: Algorithms and Applications.Springer. ISBN: 9781848829343.

VISGRAF. (2012).

Vision and graphics laboratory.Institute of Pure and Applied Mathematics (IMPA) http: // w3. impa. br/ ~ anafucs/ 3d_ museum/ 14Enero.

Page 58: Defending thesis (english)

3-D VISUAL RECONSTRUCTION : ASYSTEM PERSPECTIVE

Student:

Guillermo Enrique Medina Zegarra

Advisor:PhD. Edgar Lobaton, USA

Co-AdvisorPhD. Nestor Calvo, Argentina

Arequipa, Peru

May 07, 2012

44

Page 59: Defending thesis (english)

45

View points (perception)

(a) The glass is half fullor half empty ?

(b) is it a duck or a rabbit ?