segmentation based environment modeling using a single image
DESCRIPTION
Segmentation Based Environment Modeling Using a Single Image. SeungTaek Ryoo Department of Software, HanShin University, Korea [email protected] 2004. 9. 29. 3D Reconstruction Planar Objects Single View Metrology (Criminisi1999) Piecewise Planar Objects from Single Images (Sturm1999) - PowerPoint PPT PresentationTRANSCRIPT
Segmentation Based Environment ModelingUsing a Single Image
SeungTaek Ryoo
Department of Software, HanShin University, Korea
2004. 9. 29
Previous Works
3D Reconstruction Planar Objects
• Single View Metrology (Criminisi1999)
• Piecewise Planar Objects from Single Images (Sturm1999)
• Automatic 3D Modeling of Architecture (Dick2000)
• Image Based Modeling and Photo-Editing (Oh2001)
Criminis1999
Sturm1999
Oh2001 Dick2000
Objective
Environment Mapping
Desired Image
Input Image (Regular Image, OCC Map)
Segmented Image
Image Segmentation
Depth Calculation
Depth Image
Environment Model
Environment Modeling
Objective Realistic & Real-time View Rendering with Full-View
Panorama• Image-Segmentation Based Environment Modeling
The Structure of Segmentation Based Modeling
Image Segmentation
Segmentation indoor scene consists of floor and ceiling. outdoor scene consists of ground and sky. the environment map is divided into
• a floor (ground), ceiling (sky), and surrounding objects.
Depth Calculation (1/4)
Assumption View direction of the camera must be parallel to the ground The floor (ground) and the ceiling must consist of a plane
that is parallel to the XZ plane in the world coordinate. The sky must consist of a hemi-sphere with an infinite radius The surrounding objects must be perpendicular to the
ground
mo
Image plane
Z
R
WC
Depth Calculation (2/4)
Extracting the depth value from Regular Image The depth value can be obtained by testing for the
intersection points between the floor and the ray, and between the Image plane and the ray
The equation can be obtained with the law of the triangle
b
drDepthZ )(
XY CylinderXZ Cylinder
Depth Calculation (3/4)
Extracting the depth value from OCC map depth value of the ceiling (floor) by testing the relationship
b/w XZ cylinder and ray, and b/w XY cylinder and ray
b
drDepthZ )(
Y = dY = d
rb
dZ
R
r Zb
d
R
V
Uo
Depth Calculation (4/4)
How to get the depth value of the surrounding objects Set the depth value of the floor(ceiling) close to
the surrounding objects. Some other assumptions should be made.
• the relationship between the surrounding objects.
• the relationship between the floor(ceiling) and the surrounding objects.
• the relationship between the image plane and the surrounding objects.
Color Image Segmented Image Depth Image
Environment Modeling
Feature Points
TriangulationRectangular Grid
Texture mapping & Point sample rendering
Wireframe Texture Mapping Point Sample Rendering
Texture mapping : the stretching effect of the 3D model occurs by the occlusion Point Sample Rendering : holes occurs by the lack of the information
Regular Image
Segmentation
Depth Calculation
Regular Image Desired Image
EnvironmentModeling
EnvironmentMapping
Regular Image
Seokguram (Stone Cave)
Gyeongbokgung (Royal Palaces)
OCC Map (Indoor Scene)
Segmentation
Depth Calculation
Environment Map (Octahedral Map)
Environment Modeling
Orthogonal Cross Cylinder
Environment Mapping
OCC Map (Outdoor Scene)
Segmentation
Depth Calculation
Orthogonal Cross Cylinder
Segmentation Based Environment Model
Desired Image
Result (Art Gallery)
Environment Map (sidestep)
Segmented Based Environment Model (sidestep)
Conclusion & Future Works
Conclusion can be used for environment modeling by extracting the depth
value by the image segmentation• navigate through the virtual environment when the viewpoint has
been changed• easy to implement on the environment map
Future Works Multiple Resolution
• the problem of the blurring-image effect caused by the fixed resolution
Multiple Images (Registration & Integration)• the problem of the stretching effect of the 3D model caused by the
occlusion Generalization to apply any type of images
• only images whose viewpoint and the floor are perpendicular may be used.