1/20

Relative Magnitude of Gaussian Curvature from Shading Images

Using Neural Network

Yuji Iwahori1, Shinji Fukui2 ,Chie Fujitani1, Yoshinori Adachi1

and Robert J. Woodham3

1 Chubu University, Japan2 Aichi University of Education, Japan

3 University of British Columbia, Canada

2/20

Introduction

Surface Curvature is the invariant feature for the viewing direction. It can be used to many applications in the field of computer vision.Using a fixed camera, multiple light sources, that is, multiple shading images are used as input.Empirical implementation with the neural network has been performed to obtain the relative magnitude of Gaussian Curvature.

2D Shading Images

Recovering Gaussian Curvat

urefor 3D Shape des

cription

3/20

Previous Approaches (1)

[Woodham, 1994] Look Up Table (LUT) based Photometric Stereo: Empirical estimation of surface gradient and sur

face curvature under the different light source directions.[Iwahori, Woodham, 1995]

Neural network implementation of Photometric Stereo (to obtain the surface orientation)

with PCA (principal component analysis) to the co-linear light sources to remove the correlation

4/20

Previous Approaches (2)

[Angelopoulou and Wolff, 1998] Recovering the sign of Gaussian curvature without knowing the surface gradient

[Okatani and Deguchi, 1998] also formulated the method to recover the sig

n of Gaussian curvature.These approaches are applicable for the diffuse

reflectance. Also, the sign of GC is simple but limited information.

5/20

New Proposed Approach

Extension of [Iwahori, Fukui, Woodham, 1998] which proposed the classification of surface c

urvature using neural network.1. Empirical approach using Radial Basis Functio

n neural network (RBF-NN) to perform the non-parametric functional approximation.

2. In addition, the relative magnitude of Gaussian curvature could be obtained for the general surface reflectance (not limited to the diffuse reflectance).

6/20

Principle (Empirical Constraint)

Orthographic projection is assumed.

Test object and a calibration sphere with the same reflectance property, are observed. As far as the observed image irradiance (intensity) vector is the same for both objects, the corresponding surface normal vector should be the same.

Three shading images with different light sources are used to determine the curvature sign and the relative magnitude.

7/20

RBF neural network

The corresponding coordinate (x,y) on a sphere which has the same triple of the image irradiance for a test object and a sphere can used to obtain the information of the surface curvature.

RBF neural network is used to realize this purpose.

Features:

Non-parametric functional approximation

(interpolation) in multidimensional spaces

8/20

RBF neural network

E1

E2

E3

xsph

ysph

E - cm )

E - c1 )

E - c2 )

……

…

RBF NN learns the mapping of (E1,E2,E3) to (x,y) of each point on a sphere. After the learning, given the triple of (E1,E2,E3) of local five points on a test object, we get the corresponding (x,y) onto a sphere.

9/20

Local Surface Curvature

(a) Convex (b) Concave

(d) plane (e) convex parabolic

(f) concave parabolic

(c) hyperbolic

G>0 G>0 G<0

G=0 G=0 G=0

10/20

Mapping onto Sphere by Neural Net

(1) Local convex surface

(2) Local concave surface

11/20

Mapping onto Sphere by Neural Net

(4) Local plane(3) Local hyperbolic surface

12/20

Mapping onto Sphere by Neural Net

(5) Convex parabolic surface

(6) Concave parabolic surface

13/20

Relative Magnitude of Gaussian Curvature G

It can be estimated from the area value surrounded by four mapped points onto a sphere. Let be the height,

)2()1(

1222

yyxy

xyxx

yxff

ff

ffG

yxfz ,

This is the definition of G.

14/20

Relative Magnitude of Gaussian Curvature G

The area value S surrounded by four mapped points is

sin2

1baS

ba

2

1))(4(

2

1 2yyxxxy fffyx

15/20

Experiments (Procedures)

Three shading images are used for a test object and a sphere with the same reflectance property,Input images are taken under the same illuminating conditions for both a test object and a sphere.Neural network learning is done for a sphere.Generalization for a test object is done..Classification and the Encoding the relative magnitude are done.

16/20

Experiments (Results)

17/20

Evaluation (Simulation)

Normalized true distribution of Gaussian curvature from Hessian matrix of 2-D Sinc Function

Normalized estimated distribution of Gaussian curvature based on the proposed approach

18/20

Evaluation (Accuracy)

Red curve is true, while blue curve is estimated curve of the average cross section.

19/20

Conclusion

A new method is proposed to recover the relative magnitude of Gaussian curvature with the neural network implementation.Entire approach is empirical under the condition that no explicit assumptions are not used for the surface reflectance function nor the illuminating directions.Robust results are directly obtained in addition to the classification of local surfaces.

20/20

Further works

Extension without using any calibration object.

For a test object with multiple color textures.

For cast-shadow or inter-reflection.

Thank you