Online Submission ID: papers 0056

Reflectance Sharing: Image-based Rendering from a Sparse Set of Images

Figure 1: Image-based rendering in the most extreme case of sparse input data. The face is rendered using a non-parametric, spatially varying reflectance function that is recoveredfrom a single image (shown in Fig. 9) of known geometry under directional illumination. The reflectance function is estimated by sharing reflectance information spatially across thesurface, allowing one to render novel images with variable lighting (shown here) and viewpoint.

AbstractWhen the shape of an object is known, its appearance is determinedby the spatially-varying reflectance function defined on its surface.Image-based rendering methods that use geometry seek to estimatethis function from image data. Most existing methods recover aunique angular reflectance function (e.g., BRDF) at each surfacepoint and provide reflectance estimates with high spatial resolution.Their angular accuracy is limited by the number of available im-ages, and as a result, most of these methods focus on capturingparametric or low-frequency angular reflectance effects, or allow-ing only one of lighting or viewpoint variation.

We present an alternative approach that enables an increase in theangular accuracy of a spatially-varying reflectance function in ex-change for a decrease in spatial resolution. By framing the problemas scattered-data interpolation in a mixed spatial and angular do-main, reflectance information is shared across the surface, exploit-ing the high spatial resolution that images provide to fill the holesbetween sparsely observed view and lighting directions. Since theBRDF typically varies slowly from point to point over much of anobject’s surface, this method can provide accurate image-based ren-dering from a sparse set of images without assuming a parametricreflectance model. In fact, the method obtains good results even inthe limiting case of a single input image.

1 IntroductionGiven a set of images of a scene, image-based rendering (IBR)methods strive to build a representation for synthesizing new im-ages of that scene under arbitrary illumination and viewpoint. Oneeffective IBR representation consists of the scene geometry coupledwith a reflectance function defined on that geometry. At a givenpoint under given illumination conditions, the reflectance functionassigns a radiance value to each exitant ray, so once the geometryand reflectance function are known, realistic images can be synthe-sized under arbitrary viewpoint and (possibly complex and near-field) illumination.

This approach to IBR involves two stages: recovery of both ge-

ometry and reflectance. Yet, while great strides have been made atrecovering object shape (e.g., laser-scanners and computer visiontechniques), less progress has been made at recovering reflectanceproperties. Recovering reflectance is difficult because that is wherethe high-dimensionality of IBR is rooted. At each surface point, thereflectance is described by the bi-directional reflectance distributionfunction (BRDF). The BRDF is a four dimensional function of theview and lighting directions, and it can vary sharply in these angulardimensions, especially when a surface is specular. In addition, theBRDF generally changes spatially over an object’s surface. With-out further assumptions, recovering the spatially-varying BRDF (or6D SBRDF) requires a set of images large enough to observe high-frequency radiometric events, such as sharp specular highlights, ateach point on the surface. This set consists of a near exhaustivesampling of images of the scene from all viewpoints and lightingdirections, which can be tens-of-thousands of images or more.

In previous work, recovering spatial reflectance has been madetractable in two different ways. The first is to approximate re-flectance using an analytic BRDF model, thereby simplifying theproblem from that of recovering a 4D function at each point to thatof estimating of a handful of parameters. Examples include thework of McAllister [2002] who represents the SBRDF using the (2-lobe) Lafortune model, Sato et al. [1997] and Georghiades [2003]who use a simplified Torrance-Sparrow model, and Yu et al. [1999]who use the Ward model. Since they only recover a few parametersat each point, these methods are able to provide reflectance esti-mates from a small number of images. They require the selectionof a parametric BRDF model a priori, however, which limits theiraccuracy and generality.

The second category of methods avoids the restrictions of para-metric BRDF models by: i) using a much larger number of inputimages, and ii) recovering only a subset of the SBRDF. For exam-ple, Wood et al. [2000] use over 600 images of an object underfixed illumination to estimate the 2D view-dependent reflectancevariation, and Debevec et al. [2000] use 2048 images1 to measure

1View variation is simulated by a special-purpose parametric model.

1

Online Submission ID: papers 0056

the 2D lighting-dependent variation with fixed viewpoint. A full4D (view×illumination) reflectance function is measured by Ma-tusik et al. [2002], who use more than 12,000 images of an objectwith known visual hull; but even this large number of images pro-vides only a sparse sampling of the reflectance at each point, andas a result, images of the object can be synthesized using only low-frequency illumination environments.

This paper presents an alternative approach to estimating spatialreflectance—one that aims to combine the benefits of both para-metric methods (i.e., sparse images) and non-parametric methods(i.e., arbitrary reflectance functions.) In developing this approach,it makes the following technical contributions.

• SBRDF estimation is posed as a scattered-data interpolationproblem, with images providing dense 2D slices of data em-bedded in the mixed spatial and angular SBRDF domain.

• This high-dimensional interpolation problem is solved by in-troducing: i) a new parameterization of the BRDF domain,and ii) a non-parametric representation of reflectance basedon radial basis functions.

Unlike parametric approaches, the proposed method does notassume a priori knowledge of the SBRDF and is flexible enoughto represent arbitrary reflectance functions, including those withhigh-frequency specular effects. This approach is also very dif-ferent from previous non-parametric techniques (e.g., [Wood et al.2000; Debevec et al. 2000; Matusik et al. 2002]) that interpolatereflectance only in the angular dimensions, estimating a unique re-flectance function at each point, and providing an SBRDF with veryhigh spatial resolution. Instead, we simultaneously interpolate inboth the spatial and angular dimensions. This enables a controlledexchange between spatial and angular information, effectively giv-ing up some of the spatial resolution in order to fill the holes be-tween sparsely observed view and illumination conditions. Sincereflectance typically varies slowly from point to point over muchof an object’s surface, this means that we can often estimate an ac-curate SBRDF from a drastically reduced set of images. In fact,as shown in Fig. 1, for some surfaces a reasonable estimate can beobtained even in the extreme case of a single input image.

The remainder of this paper is organized as follows. After adiscussion of related work, we introduce a new BRDF parameter-ization in Sect. 3. Then, in Sects. 4–6, we present our reflectancerepresentation based on radial basis functions in three parts. Sect. 4discusses why RBFs are suitable for representing reflectance, andSect. 5 introduces our representation for the special case of sur-faces with spatially-invariant reflectance. (Figures 4 and 5 in thissection show how this representation can accurately represent bothanalytic and measured BRDF data.) In Sect. 6, we describe howthis representation is used in the spatially-varying case, which isthe principal contribution of the paper. An example in which ourmethod reduces the number of input images by more than an orderof magnitude is shown in Fig. 7. Finally, in Sect. 7 we broadenthe approach to allow generalized spatial variation with texture andapply it to image-based rendering of a human face (Figs. 10–12),including the extreme case of a single input image (Fig. 1).

2 Main Ideas and Related WorkThe proposed method for SBRDF estimation builds on three prin-cipal observations.

Smooth spatial variation. Most existing methods recover a uniqueBRDF at each point and thereby provide an SBRDF with very highspatial resolution. Many parametric methods have demonstrated,however, that the number of input images can be reduced if one iswilling to accept a decrease in spatial resolution. This has been ex-ploited, for example, by Yu et al. [1999] and Georghiades [2003],who assume that the specular BRDF parameters are constant across

a surface. Similarly, Sato et al. [1997] estimate the specular pa-rameters at only a small set of points, later interpolating these pa-rameters across the surface. Lensch et al. [2001] present a noveltechnique in which reflectance samples at clusters of surface pointsare used to estimate a basis of (1-lobe) Lafortune models. The re-flectance at each point is then uniquely expressed as a linear com-bination of these basis BRDFs.

Similar to these approaches, our method trades spatial resolutionfor an increase in angular resolution. The difference, however, isthat we implement this exchange using a non-parametric represen-tation. We begin by assuming that the SBRDF varies smoothly inthe spatial dimensions, but we also demonstrate how this can berelaxed to handle rapid spatial variation in terms of a multiplica-tive texture. (In cases where the shape of the BRDF itself changesrapidly, we currently assume that discontinuities are given as input.)

Curved surfaces. Techniques for image-based BRDF measure-ment ([Lu et al. 1998; Marschner et al. 1999; Matusik et al. 2003b])exploit the fact that a single image of a curved, homogeneous sur-face represents a very dense sampling of a 2D slice of the 4DBRDF. In this paper, we extend this idea to the spatially-varyingcase, where an image provides a 2D slice in the higher-dimensionalSBRDF domain. Our results demonstrate that, like the homoge-neous case, surface curvature (along with smooth spatial variation)can be exploited to increase the angular resolution of the SBRDF.2

Angular Compressibility. While it is a high dimensional function,a typical BRDF varies slowly over much of its angular domain. Thisproperty has been exploited for 3D shape reconstruction [Hertz-mann and Seitz 2003], efficient BRDF acquisition [Matusik et al.2003b], efficient rendering of BRDFs [Kautz and McCool 1999;McCool et al. 2001; Jaroszkiewicz and McCool 2003], and efficientevaluation of environment maps [Cabral et al. 1999; Ramamoorthiand Hanrahan 2002; Lawrence et al. 2004]. Here, we exploit com-pressibility by assuming that the BRDF typically varies rapidly onlyin certain dimensions, such as the half-angle.

The three ideas of this section have been developed in very dif-ferent contexts, and this paper combines and expands them to solvea novel problem: estimating non-parametric SBRDFs from sparseimages. The fusion of these ideas is enabled by a new BRDF param-eterization (see Sect. 3), and an interpolation approach that unifiesthe treatment of spatial and angular dimensions (see Sects. 4–6).

2.1 Assumptions

We exploit scene geometry to reduce the number of input images re-quired to accurately represent appearance. Thus, unlike pure lightfield techniques [Gortler et al. 1996; Levoy and Hanrahan 1996],the method requires a set of images of an object with known geom-etry, viewpoint, and either point-source or directional illumination.A number of suitable acquisition systems have been described inthe literature. (See, e.g., [Sato et al. 1997; Debevec et al. 2000;McAllister 2002; Matusik et al. 2002].)

In addition, global effects such as sub-surface scattering and in-terreflection are not explicitly considered in our formulation. Fordirectional illumination and orthographic views, however, some ofthese effects will be absorbed into our representation and can be re-produced when rendered under the same conditions. (See Sect. 7.)In this case, our use of the term SBRDF is synonymous with thenon-local reflectance field defined by Debevec et al. [2000].

Finally, in this paper we restrict our attention to isotropicBRDFs. While the ideas of exploiting spatial coherence and usingRBFs to interpolate scattered reflectance samples can be applied tothe anisotropic case, this would require a parameterization which isdifferent from that presented in Sect. 3 and is left for future work.

2For near-planar surfaces where curvature is not available, more angularreflectance information is obtained using near-field illumination and per-spective views [Karner et al. 1996].

2

Online Submission ID: papers 0056

h d

d

h

h

lv

t

n

φ

θθ φ θ

2φd θhsin

dπ

2θ

v

w

u(a) (b)

Figure 2: (a) The halfway/difference parameterization of Rusinkiewicz. In theisotropic case, the BRDF domain is parametrized by (θh,φd ,θd ). (b) The modifiedparameterization defined by Eq. (1) that is suitable for interpolation.

3 Notation and BRDF ParameterizationAt the core of our approach is the interpolation of scattered data inmany (3-6) dimensions. The success of any interpolation techniquedepends heavily on how the SBRDF is parameterized. This sectionintroduces some notation and presents one possible parameteriza-tion. Based on this parameterization, our interpolation technique isdiscussed in Sects. 5 and 6.

The SBRDF is a function of six dimensions written f (~x,~θ ),where ~x = (x,y) ⊂ R

2 is the pair of spatial coordinates that pa-rameterize the surface geometry (a surface point is written ~s(x,y)),and ~θ ∈ Ω×Ω are the angular coordinates that parameterize thedouble-hemisphere of view/illumination directions in a local coor-dinate frame defined on the tangent plane at a surface point (i.e., theBRDF domain.) A common parameterization of the BRDF domainis ~θ = (θi,φi,θo,φo) which represent the spherical coordinates ofthe light and view directions in the local frame. When the BRDF isisotropic, the angular variation reduces to a function of three dimen-sions, commonly parameterized by (θi,θo,φo − φi). In this work,we restrict ourselves to this isotropic case and consider the SBRDFto be a function defined on a 5D domain. In the special case whenthe SBRDF is a constant function of the spatial dimensions (i.e.,f (~x,~θ ) = f (~θ )) we say that the surface is homogeneous and is de-scribed by a 4D (or isotropic 3D) function.

The BRDF domain can be parameterized in a number ofways, and as discussed below, one good choice is Rusinkiewicz’shalfway/difference parameterization [Rusinkiewicz 1998], shownin Fig. 2(a). Using this parameterization in the isotropic case, theBRDF is written as ~θ = (θh,φd ,θd)⊂ [0, π

2 )× [0,π)× [0, π2 ). (Note

that φd is restricted to [0,π) since φd 7−→ φd +π by reciprocity.)The existence of singularities at θh = 0 and θd = 0 and

the required periodicity (φd 7−→ φd + π) make the standardhalfway/difference parameterization unsuitable for most interpola-tion techniques. Instead, we define the mapping (θh,φd ,θd) 7−→(u,v,w), as

(u,v,w) =

(

sinθh cos2φd ,sinθh sin2φd ,2θd

π

)

. (1)

This modified parameterization is shown in Fig. 2(b). It eliminatesthe singularity at θh = 0 and ensures that the BRDF f (u,v,w) sat-isfies reciprocity. In addition, the mapping is such that the one re-maining singularity occurs at θd = 0 (i.e., where the light and viewdirections are equivalent). This configuration is difficult to createin practice, making it unlikely to occur during acquisition. Duringsynthesis, it must be handled with care.

3.1 Considerations for Image-based Acquisition

The halfway/difference parameterization has been shown to in-crease compression rates since common features such as specu-lar and retro-reflective peaks are aligned with the coordinate axes[Rusinkiewicz 1998]. The modified parameterization of Eq. (1)maintains this property, since specular events cluster along the w-axis, and retro-reflective peaks occur in the plane w = 0.

These parameterizations are useful in IBR for an additional rea-son: for image-based data, they separate the sparsely-sampled anddensely-sampled dimensions of the BRDF. To see this, note that fororthographic projection3 and distant illumination—or more gen-erally, when scene relief is relatively small—a single image of acurved, homogeneous surface provides BRDF samples lying in aplane of constant θd , since this angle is independent of the surfacenormal. The image represents a nearly continuous sampling of θhand φd in this plane. Thus, a set of images provides dense samplingof (θh,φd) but only as many samples of θd as there are images.

Conveniently, the irregular sampling obtained from image-baseddata corresponds well with the behavior of general BRDFs, whichvary slowly in the sparsely sampled θd-dimension, especially whenθd is small. At the same time, by imaging curved surfaces, weensure that the sampling of the half-angle θh is high enough to ac-curately recover the high-frequency variation (e.g., specular high-lights) that is generally observed in that dimension.

4 Scattered Data Interpolation

Recall that our goal is to estimate a continuous SBRDF f (~x,~θ) froma set of samples fi ∈R

5 drawn from images of a surface with knowngeometry. Our task is complicated by the fact that, as discussed inthe previous section, the input samples are very non-uniformly dis-tributed. This means, for example, that we cannot generally solvethis problem using a Fourier basis.

There are a number of methods for interpolating scattered datain this relatively high-dimensional space. Finite element methods,for example, approximate a function using a multivariate piece-wise polynomial spline [Zienkiewicz 1979; Terzopoulos 1983].Since they require a triangulation of the domain, however, theseapproaches become computationally prohibitive as the dimensionincreases.4 Local methods such as local polynomial regression(used by Marschner [1999]) and the push/pull algorithm of Gortleret al. [1996] generally produce only tabulated values of the functionwhich is also cumbersome for high dimensions.

Interpolation using radial basis functions (RBFs), on the otherhand, is quite efficient for scattered data in many dimensions. For afixed number of samples, the cost of computing an RBF interpolantis dimension-independent, and the size of the representation doesnot grow substantially as the dimension increases. (These proper-ties are central to the unified approach to both BRDFs and SBRDFsin Sects. 5 and 6.) In addition, an RBF interpolant is simply com-puted by solving a linear system of equations, and existence anduniqueness is guaranteed with few restrictions on the sample points.Furthermore, there is a growing body of work investigating conver-gence rates and efficient computational methods [Buhmann 2003].For these reasons, RBFs are an attractive choice for our problem.

Since RBFs provide a general scattered-data interpolation tech-nique, the only assumption we make by using them is that the spa-tial reflectance function is ‘smooth’ in some sense (see, e.g., [Buh-mann 2003] and references therein for details.) Radial basis func-tions have been shown to perform well in a variety of cases, includ-ing problems involving ‘track data’ (data in which some dimensionsare sampled more densely than others [Carlson and Foley 1992])similar to the image-based reflectance data considered here.

Although it is not the focus of this paper, in Sect. 5 we showthat in addition to representing SBRDFs, radial basis functions pro-vide an alternative and potentially useful representation for homo-geneous BRDFs as well—one that is well suited to sparse, image-

3The orthographic/directional case is considered here only for illustra-tive purposes; our method for reflectance sharing handles arbitrary camerasand point sources.

4Jaroszkiewicz and McCool [2003] handle this by approximating thehigh-dimensional SBRDF by a product of 2D functions, each of which istriangulated independently.

3

Online Submission ID: papers 0056

SpatialVariation

IrregularSamples

ManyDimensions

Zernike polynomials 7 7 7Matusik bases 7 7 7

Spherical harmonics 7 7 7Wavelets X 7 X

Polynomial regression X X 7Multivariate splines X X 7

RBFs X X X

Table 1: Comparison of reflectance representations and interpolation methods. Rep-resentations such as Zernike polynomials, spherical harmonics and the data-drivenbases of Matusik et al. are defined on the double-sphere or double-hemisphere andare not easily extended to the spatially-varying case. Although wavelets are definedin R

d and could be used for spatially-varying reflectance, they require uniformly dis-tributed input samples. Of the representations suitable for scattered data interpolation(local polynomial regression, splines, and radial basis functions (RBFs)), only RBFsare computationally efficient in many dimensions.

based datasets. Unlike most existing BRDF representations (e.g.,spherical harmonics, Zernike polynomials, wavelets and the data-driven basis of Matusik et al. [2003b]), RBFs do not require a localpreprocessing step to resample the input data at regular intervals.Table 1 provides a summary of advantages of RBF interpolation.

4.1 Radial Basis Functions

To briefly review RBF interpolation (see, e.g., [Powell 1992; Buh-mann 2003]), consider a general function g(~x), ~x ∈ R

d from whichwe have N samples gi at sample points ~xi. This function is ap-proximated as a sum of a low-order polynomial and a set of scaled,radially symmetric basis functions centered at the sample points;

g(~x) ≈ g(~x) = p(~x)+N

∑i=1

λiψ(‖~x−~xi‖), (2)

where p(~x) is a polynomial of order at most n, ψ : R+ →R is a con-

tinuous function, and ‖·‖ is the Euclidean norm. The sample points~xi are referred to as centers, and the RBF interpolant g satisfies theinterpolation conditions g(~xi) = g(~xi).

Given a choice of n, an RBF ψ , and a basis for the polynomialsof order n or less, the coefficients of the interpolant are determinedas the solution of the linear system

[

Ψ PPT 0

][

~λ~c

]

=

[

~g0

]

, (3)

where Ψi j = ψ(‖~xi −~x j‖),~λi = λi,~gi = gi, Pi j = p j(~xi) where p jare the polynomial basis functions, and ~ci = ci are the coefficientsin this basis of the polynomial term in g. This system is invert-ible (and the RBF interpolant is uniquely determined) in arbitrarydimensions for many choices of ψ , with only mild conditions on nand the locations of the data points [Duchon 1977; Micchelli 1986].For example, in two dimensions the familiar ‘thin-plate spline’ cor-responds to the RBF ψ(r) = r2 log r, and the corresponding systemis invertible whenever n ≥ 1 and the sample points are not collinear.

In practical cases the samples gi are affected by noise, and itis desirable to allow the interpolant to deviate from the data points,balancing the smoothness of the interpolant with its fidelity to thedata. This is accomplished by replacing Ψ in Eq. (3) with Ψ−ρNI,where I is the identity matrix and ρ is a stiffness parameter. Furtherdetails are provided by Wahba [1990].

So far we have dealt exclusively with radially symmetric basisfunctions. In many cases we benefit from using radially asymmet-ric basis functions which are stretched in certain directions. Radi-ally asymmetric RBFs are used by Dinh et al. [2001] to representcorners and edges of 3D surfaces. Here we use them to manage theirregularity in our sampling pattern. (Recall from Fig. 2 that the

(u,v) dimensions are sampled almost continuously while we haveonly as many samples of w as we have images.) Following Dinh etal. [2001], an asymmetric radial function is created by scaling theEuclidean distance in Eq. (2) so that the basis functions become

ψ(‖M(~x−~xi)‖), (4)

where M ∈ Rd×d . In our case we choose M = diag(1,1,mw). For

mw < 1, the basis functions are elongated in the w dimension, whichis appropriate since our sampling rate is much lower in that dimen-sion. The appropriate value of this parameter depends on the angu-lar density of the input images, and empirically we have found thattypical values for mw are between 0.1 and 0.5.

As a final note, when the number of samples is large (i.e., N >10,000), solving Eq. (3) requires care and can be difficult (or im-possible) using direct methods. This limitation has been addressedquite recently, and iterative fitting methods [Beatson and Powell1995], and fast multipole methods (FMMs) for efficient evalua-tion [Beatson and Newsam 1992] have been developed for manychoices of ψ in many dimensions. In some cases, solutions forsystems with over half a million centers have been reported [Carret al. 2001]. The next sections include investigations of the num-ber of RBF centers required to accurately represent image-basedreflectance data, and we find this number to be sufficiently small toallow the use of direct methods.

5 Homogeneous SurfacesIn this section, we apply RBF interpolation to homogeneous sur-faces, where we seek to estimate a global BRDF that is notspatially-varying. While it is not the focus of the paper, this spe-cial case provides a convenient platform for performance evalua-tion. Additionally, the dimension-independence of RBF interpola-tion means that the methods developed here for the homogeneouscase can be applied to the spatially-varying case (Sect. 6) as well.

As discussed in Sect. 3, in the case of homogeneous BRDF data,reflectance is a function of three dimensions, (u,v,w). In R

3, agood choice for ψ is the linear (or biharmonic) RBF, ψ(r) = r, withn = 1, since in this case, the interpolant from Eq. (3) exists for non-coplanar data, is unique, minimizes a generalization of the thin-plate energy, and is therefore the smoothest in some sense [Duchon1977; Carr et al. 2001]. The BRDF is expressed as

f (~θ) = c1 + c2u+ c3v+ c4w+N

∑i=1

λi‖~θ −~θi‖, (5)

where ~θi = (ui,vi,wi) represents a BRDF sample point from theinput images, and~λ and ~c are found by solving Eq. (3).

As a practical consideration, since each pixel represents a sam-ple point ~θi, even with modest image resolution, using all availablesamples as RBF centers is computationally prohibitive. Much ofthis data is redundant, however, and an accurate BRDF representa-tion can be achieved using only a small fraction of these centers.A sufficient subset of centers could be chosen using knowledge oftypical reflectance phenomena. (To represent sharp specular peaks,for example, RBF centers are generally required near θh = 0.) Al-ternatively, Carr et al. [2001] present a simple and effective methodfor choosing this subset without assuming prior knowledge. Theyuse a greedy algorithm to select RBF centers, and a slightly modi-fied version of the same algorithm is applied here. The procedurebegins by selecting a small subset of the sample points ~θi and fit-ting an RBF interpolant to these. Next, this interpolant is evaluatedat all sample points and used to compute the radiance residuals,εi = ( fi − f (~θi))cos θi, where θi is the angle between the surfacenormal at the sample point and the illumination direction. Finally,points where εi is large are appended as additional RBF centers, andthe process is repeated until the desired fitting accuracy is achieved.

4

Online Submission ID: papers 0056

0 25 50 750

0.05

0.1

0.15

0.2

0.25

0.3

Rel

ativ

e R

MS

radi

ance

err

or

100 200 300 400 500 600 700 800 900 1000

# Centers/lobes

RBF

Lafortune (computed)

Lafortune (projected)

Figure 3: The accuracy of the RBF representation as the number of centers is in-creased using a greedy algorithm. The input is 10 images of a sphere synthesized usingthe metallic-blue BRDF measured by Matusik et al. This is compared to the accuracyof the isotropic Lafortune representation as the number of lobes is increased. Dueto the difficulty in fitting a multi-lobe Lafortune representation, a single generalizedcosine lobe was fit to the data, and additional lobes were added in succession. Aftereach lobe was added, the parameters of only that lobe (and the diffuse term) were op-timized, followed by a second optimization over all lobes. (The lobes were randomlyinitialized to Cx = Cy = −1±0.25,Cz = 1±0.25,n = 20±5 (forward scattering) andCx = Cy = Cz = 1± 0.25,n = 20± 5 (backward scattering) in alternation.) Fitting a300-lobe Lafortune representation in this way required approximately 1 week of pro-cessing time (hence the projection), whereas fitting a 1000-center RBF required lessthan one hour on the same machine.

It should be noted that an algorithmic choice of center locationscould increase the efficiency of the resulting representation, sincecenter locations would not necessarily need to be stored for eachmaterial. This would require assumptions about the function beingapproximated, however, and here we choose to emphasize general-ity over efficiency by using the greedy algorithm.

5.1 Evaluation

To evaluate the BRDF representation in Eq. (5), we perform an ex-periment in which we compare it to both parametric BRDF modelsand to a non-linear basis (the isotropic Lafortune model [1997].)The models are fit to synthetic images of a sphere, and their accu-racy is measured by their ability to predict the appearance of thesphere under novel conditions. (Note that many other representa-tions, such as wavelets and the Matusik bases are excluded fromthis comparison because they require dense, uniform samples.)

The input images simulate data from image-based BRDF mea-surement systems such as that of Matusik et al. [2003a]. They areorthographic, directional-illumination images with a resolution of100×100, and are generated such that θd is uniformly distributedin [0, π

2 ]. The accuracy of the recovered models is measured bythe relative RMS radiance error over 21 images—also uniformlyspaced in θd—that are not used as input. (The error is computedover all pixels corresponding to surface points.) For these simu-lations, we use both specular and diffuse reflectance, one drawnfrom measured data (the metallic-blue BRDF, courtesy of Matusiket al. [2003b]), and the other generated using the physics-basedOren-Nayar model [1994].

Figure 3 shows the accuracy of increasingly complex RBF andLafortune representations fit to ten input images. The complexityof the RBF representation is measured by the number of centersselected by the greedy algorithm, and that of the Lafortune model ismeasured by the number of generalized cosine lobes. It is importantto note that the size of each representation is different for equivalentcomplexities; an N-lobe isotropic Lafortune model requires 3N +1parameters, while an N-center RBF interpolant requires 4N +4.

Since the basis functions of the Lafortune model are specificallydesigned for representing BRDFs (and are therefore embedded withknowledge of general reflectance behavior), they provide a reason-ably good fit with a small number of lobes. For example, a 6-lobeLafortune model (19 parameters) yields the same RMS error as a300-center RBF model (1204 parameters.) In addition to being

2 4 6 8 10 120

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

# Images

Rel

ativ

e R

MS

radi

ance

err

or

RBFWardLafortune

ACTUAL RBF LAFORTUNE WARD

Figure 4: Top: Error in the estimated BRDF for an increasing number of imagesof a metallic-blue sphere. As the number of images increases, the RBF representation(with 1000 centers) converges to the true BRDF. For comparison, the isotropic Wardmodel and the 6-lobe Lafortune representation are fit to the same data, and these arefound to be too restrictive to provide an accurate fit. (As shown in Fig. 3, the accuracyof the Lafortune representation does not change significantly with more than 6 lobes.)Bottom: Synthesized images using the three BRDF representations estimated from 12input images. The angle between the source and view directions is 140. Only the RBFrepresentation accurately captures the Fresnel effects at this moderate grazing angle.

compact, the Lafortune model has the advantage of being more suit-able for direct rendering [McAllister et al. 2002]. But the accuracyof this representation is fundamentally limited; the lack of flexi-bility and the difficulty of the required non-linear fitting processprevent the Lafortune model from converging to the true BRDF.

In contrast, RBFs provide a general linear basis, and given a suf-ficient number of centers, they can represent any smooth reflectancefunction with arbitrary accuracy. In this example, the RBF rep-resentation converges with less than 1000 centers, suggesting thatonly a small fraction of the available centers are required to sum-marize the reflectance information in the ten input images.

Similar conclusions can be drawn from a second experiment inwhich we investigate the accuracy of these (and other) representa-tions with a fixed level of complexity but an increasing number ofinput images. The results are shown in Figs. 4 and 5 for predom-inantly specular and diffuse reflectance. Since RMS error is oftennot an accurate perceptual metric, these figures include syntheticspheres rendered with the recovered models. Again, the parametricmodels and the non-linear generalized cosine basis are too restric-tive to accurately fit the data, whereas the RBF representation con-verges to the actual BRDF as the number of images increases. Thebottom of Fig. 4 shows that only the RBF representation is flexibleenough to capture the Fresnel effects at grazing angles.

6 Inhomogeneous SurfacesAs shown in the previous section, RBFs can provide a useful repre-sentation for homogeneous BRDFs. The primary motivation for de-veloping this representation, however, is to model spatially-varyingreflectance. This section discusses the spatially-varying case, whichis the main contribution of the paper. We show how the RBF repre-sentation enables an exchange between spatial and angular resolu-tion (a process we refer to as reflectance sharing), and we demon-strate that it can drastically reduce the number of images requiredto estimate the spatial reflectance. We begin by assuming that the5D SBRDF varies smoothly in the spatial dimensions, and in thenext section, we show how this can be generalized to handle rapidspatial variation in terms of a multiplicative albedo or texture.

In the homogeneous case, the BRDF is a function of three dimen-sions, and the linear RBF ψ(r) = r yields a unique interpolant thatminimizes a generalization of the thin-plate energy. Although opti-

5

Online Submission ID: papers 0056

2 4 6 8 10 120

0.04

0.08

0.12

0.16

0.2

# Images

Rel

ativ

e R

MS

radi

ance

err

orRBFLambertianLafortune

ACTUAL RBF LAFORTUNE LAMBERTIAN

Figure 5: Top: Plot comparing the error in the estimated BRDF for an increasingnumber of input images of a sphere synthesized with diffuse Oren-Nayar reflectance. Asin the previous figure, the RBF representation (with 1000 centers) converges to the trueBRDF as the number of images increases. The 6-lobe Lafortune representation and theLambertian BRDF model are too restrictive to accurately represent the data. Bottom:Synthesized images comparing the three BRDF representations estimated from 12 inputimages. The angle between the source and view directions is 10.

mality cannot be proved, this RBF has shown to be useful in higherdimensions as well, since it provides a unique interpolant in any di-mension for any n [Powell 1992]. In the spatially-varying case, theSBRDF is a function of five dimensions, and we let~q = (x,y,u,v,w)be a point in its domain. Using the linear RBF with n = 1, theSBRDF is given by

f (~q) = p(~q)+N

∑i=1

λi‖~q−~qi‖, (6)

where p(~q) = c1 + c2x + c3y+ c4u+ c5v+ c6w.We can use any parameterization of the surface ~s, and there has

been significant recent work on determining good parameteriza-tions for general surfaces [Lee et al. 1998; Gu et al. 2002]. The idealsurface parameterization is one that preserves distances, meaningthat ‖~x1 −~x2‖ is equivalent to the geodesic distance between ~s(~x1)and ~s(~x2). For simplicity, here we treat the surface as the graph ofa function, so that~s(x,y) = (x,y,s(x,y)), (x,y) ⊂ [0,1]× [0,1].

The procedure for recovering the parameters in Eq. (6) is almostexactly the same as in the homogeneous case. The coefficientsof f are found by solving Eq. (3) using a subset of the SBRDFsamples from the input images, and this subset is chosen using agreedy algorithm. Radially asymmetric basis functions are realizedusing M = diag(mx,mx,1,1,mw), where mx controls the exchangebetween spatial and angular reflectance information. When mx 1,the basis functions are elongated in the spatial dimensions, and therecovered reflectance function approaches a single BRDF (i.e., ahomogeneous representation) with rapid angular variation. Whenmx 1, we recover a near-Lambertian representation in which theBRDF at each point approaches a constant function of ~θ . Appropri-ate values of mx depend on the choice of surface parameterization,and we found typical values to be between 0.2 and 0.4 for the ex-amples in this paper.

6.1 Evaluation

The SBRDF representation of Eq. (6) can be evaluated using experi-ments similar to those for the homogeneous case. Here, spatial vari-ation is simulated using images of a hemisphere with a Torrance-Sparrow BRDF with a linearly varying roughness parameter. Fiveimages of the hemisphere are shown in Fig. 6, and they demonstratehow the highlight sharpens from left to right across the surface.

0 200 400 600 800 1000 1200 1400 1600 1800 2000

0.05

0.1

0.15

0.2

# Centers

Rel

ativ

e R

MS

radi

ance

err

or

Figure 6: Top: Accuracy of the SBRDF recovered by reflectance sharing using theRBF representation in Eq. (6) as the number of centers is increased using a greedyalgorithm. The input is 10 synthetic images of a hemisphere (five of which are shown)with linearly varying roughness.

The graph in Fig. 6 shows the accuracy of the recovered SBRDFas a function of the number of RBF centers when it is fit to tenimages of the hemisphere under ten uniformly distributed illumi-nation directions. The error is computed over 40 images that werenot used as input. Fewer than 2000 centers are needed to accu-rately represent the spatial reflectance information available in theinput images. This is a reasonably compact representation, requir-ing roughly 12,000 parameters. (For comparison, an SBRDF rep-resentation for a 10,000-vertex surface consisting of two uniqueLafortune lobes at each vertex is roughly five times as large.)

Figure 7 contrasts reflectance sharing with conventional methodsthat interpolate only in the angular dimensions, estimating a sepa-rate BRDF at each point.5 (This ‘no sharing’ technique is used byMatusik et al. [2002], and is similar in spirit to Wood et al. [2000],who also estimate a unique reflectance function at each point.) Forboth the reflectance sharing and ‘no sharing’ cases, the SBRDFis estimated from images with fixed viewpoint and uniformly dis-tributed illumination directions such as those in Fig. 6, and the re-sulting reflectance models are used to predict the appearance of thesurface under novel lighting. The top frame of Fig. 7 shows theactual appearance of the hemisphere under five novel conditions,and the lower frames show the reflectance sharing and ‘no sharing’results obtained from increasing numbers of input images.

In this example, reflectance sharing reduces the number of re-quired input images by more than an order of magnitude. Fiveimages are required to accurately estimate the SBRDF using theRBF representation, whereas at least 150 are needed if one doesnot exploit spatial coherence. Figure 7 also shows how differentlythe two methods degrade with sparse input. The reflectance sharingmethod provides a smooth SBRDF whose accuracy gradually de-creases away from the convex hull of input samples. (For example,the sharp specularity on the right side of the surface is not accu-rately recovered when only two input images are used.) In contrast,when interpolating only in the angular dimensions, a small num-ber of images provides only a small number of reflectance samplesat each point; and as a result, severe aliasing or ‘ghosting’ occurswhen the surface is illuminated by high-frequency environmentslike the directional illumination shown here.

Finally, even when the input images are captured from a sin-

5In this discussion, angular interpolation in the BRDF domain is as-sumed to require known geometry. This is different from lighting interpola-tion (e.g., [Debevec et al. 2000]) that does not. For the particular example inFig. 7, however, the ‘no sharing’ result can be obtained without geometry,since it is a special case of fixed viewpoint.

6

Online Submission ID: papers 0056

ACTUAL

REFLECTANCE SHARING

2

5

NO SHARING

5

50

150

Figure 7: Estimating the spatially-varying reflectance function from a sparse setof images. Top frame: five images of a hemisphere under illumination conditions notused as input. Middle frame: appearance predicted by reflectance sharing with twoand five input images. (The input is shown in Fig. 6; the two left-most images are usedfor two-image case.) Bottom frame: appearance predicted by interpolating only in theangular dimensions with 5, 50 and 150 input images. At least 150 images are requiredto obtain a result comparable to the five-image reflectance sharing result.

gle viewpoint, our method recovers a full SBRDF, and as shownin Fig. 8, view-dependent effects can be accurately predicted. Thisis made possible by spatial sharing (since each surface point is ob-served from a unique view in its local coordinate frame) and by reci-procity (since we effectively have observations in which the viewand light directions are exchanged.)

7 Generalized Spatial VariationThis section considers generalizations of the radial basis functionSBRDF model by softening the requirement for spatial smoothness,and applies the model to image-based rendering of a human face.

Rapid spatial variation can be handled using a multiplicativealbedo or texture by writing the SBRDF as

f (~x,~θ ) = a(~x)d(~x,~θ ),

where a(~x) is an albedo map for the surface and d(~x,~θ ) is a smoothfunction of five dimensions. As an example, consider the humanface in Fig. 9(a). The function a(~x) accounts for rapid spatial varia-tion due to pigment changes, while d(~x,~θ ) models the smooth spa-tial variation that occurs as we transition from a region where skinhangs loosely (e.g., the cheek) to where it is taut (e.g., the nose.)

In some cases, it is advantageous to express the SBRDF as a lin-ear combination of 5D functions. For example, Sato et al. [1997]and many others use the dichromatic model of reflectance [Shafer1985] in which the BRDF is written as the sum of an RGB dif-fuse component and a scalar specular component that multiplies thesource color. We employ the dichromatic model for the example in

ACT

UAL

REFLECTAN

CE

SH

ARIN

G(5

)

Figure 8: Actual and predicted appearance of the hemisphere under fixed illumina-tion and changing view. Given five input images from a single view (bottom Fig. 6), thereflectance sharing method recovers a full SBRDF, including view-dependent effects.

Fig. 10, and compute the emitted radiance using

Ik(~x,~θ ) = sk

(

ak(~x)dk(~x,~θ )+g(~x,~θ ))

cosθi, k = R,G,B, (7)

where s = skk=RGB is an RGB unit vector that describes the colorof the light source. In Eq. (7), a single function g is used to modelthe specular reflectance component, while each color channel ofthe diffuse component is modeled separately. This is significantlymore general than the usual assumption of a Lambertian diffusecomponent, and it can account for changes in diffuse color as afunction of ~θ , such as the desaturation of the diffuse component ofskin at large grazing angles witnessed by Debevec et al. [2000].

Finally, although not used in our examples, more general spa-tial variation can be modeled by dividing the surface into a finitenumber of regions, where each region has spatial reflectance as de-scribed above. (See, e.g., [Lensch et al. 2001; Jaroszkiewicz andMcCool 2003].) Given a set of disjoint regions R j, j = 1 . . .M thattile the spatial domain, the SBRDF is expressed as

f (~x,~θ) =M

∑j=1

δ j(~x) f j(~x,~θ), (8)

where δ j evaluates to 1 if ~x ∈ R j and 0 otherwise, and f j is theSBRDF in region R j.

7.1 Data Acquisition and SBRDF Recovery

For real surfaces, we require geometry and a set of images takenfrom known viewpoint and directional illumination. In addition,in order to estimate the separate diffuse and specular reflectancecomponents in Eq. (7), the input images must be similarly decom-posed. While it can be accomplished in many ways (e.g., [Sato andIkeutchi 1994; Nayar et al. 1997]), we perform the specular/diffuseseparation by placing linear polarizers on both the camera and lightsource and exploiting the fact that the specular component pre-serves the linear polarization of the incident radiance. Two expo-sures are captured for each view/lighting configuration, one withthe polarizers aligned (to observe the sum of specular and diffusecomponents), and one with the source polarizer rotated by 90 (toobserve the diffuse component only.) The specular component isgiven by the difference between these two images. Geometry isrecovered using a variant of photometric stereo, since it providesthe precise surface normals required for reflectometry. Examplesof geometry and specular and diffuse images are shown in Fig. 9.

The effects of shadows and shading are computed using theknown geometry. Shadowed pixels are discarded and shading ef-fects are removed by dividing by cosθi. The RGB albedo a(~x) inEq. (7) is estimated as the median of the diffuse samples at eachsurface point, and normalized diffuse reflectance samples are com-puted by dividing by a(~x). The resulting normalized diffuse sam-ples are used to estimate the three functions dk(~x,~θ ) in Eq. (7) usingthe RBF techniques described in Sect. 4.1. The samples from thespecular images are similarly used to compute g.

7

Online Submission ID: papers 0056

(a) (b) (c)Figure 9: (a,b) Specular and diffuse images for one camera/source configuration.(c) Geometry of the face used for SBRDF recovery and rendering.

7.2 Rendering

Images under arbitrary view and illumination are synthesized usingthis representation as follows. The SBRDF coordinates~q at any sur-face point are determined by the spatial coordinates ~x, the surfacenormal, and the view and lighting directions in the local coordinateframe. The radiance emitted from that point toward the camera isgiven by Eq. (7). Because Eq. (7) involves sums over a large num-ber of RBF centers (up to 5000) for each pixel, image synthesis canbe slow. We have explored methods for accelerating this processusing programmable graphics hardware and precomputation.

Hardware Rendering Using the GPU. Equation (7) is well suitedto implementation in graphics hardware because the same calcula-tions are done at each pixel. A vertex program computes each~q andthese are interpolated as texture coordinates for each pixel. A frag-ment program performs the computation in Eq. (6), which is sim-ply a sequence of distance calculations. Implementing the sum inEq. (7) is straightforward, since we simply modulate by the albedomap and source color. Currently, we use one rendering pass foreach RBF center, accumulating their contributions. On a GeForceFX 5900, rendering a 512×512 image with 2000 centers (and 2000rendering passes) is reasonably efficient, taking approximately 30s.We believe further optimizations are possible, such as consideringthe contributions of multiple RBF centers in each pass.

Real-Time Rendering with Precomputed Images. To enablereal-time rendering, and to allow complex illumination, we can useour synthesized images as input to methods based on precomputedimages. As a demonstration, we implemented the double-PCA im-age relighting technique of Nayar et al. [2004]. This algorithm isconceptually similar to clustered PCA methods such as [Chen et al.2002; Sloan et al. 2003], and allows real-time relighting of specu-lar objects with complex illumination and shadows (obtained in theinput images using shadow mapping in graphics hardware.)

We emphasize that despite these gains in efficiency, the RBF rep-resentation does not compete with parametric representations forrendering purposes. Instead, it should be viewed as a useful inter-mediate representation between acquisition and rendering.

7.3 Results

As a demonstration, the representation of Eq. (7) was used to modela human face, since this surface exhibits complex diffuse texture inaddition to smooth spatial variation in its specular component.

The spatially-varying reflectance function recovered from fourcamera/source configurations is shown in Figs. 10–12. For these re-sults, two polarized exposures were captured in each configuration,and the viewpoint remained fixed throughout. For simplicity, thesubject’s eyes remained closed. (Accurately representing the spa-tial discontinuity at the boundary of the eyeball would require thesurface to be tiled as in Eq. (8).) The average angular separation ofthe light directions is 21, spanning a large area of frontal illumina-tion. (See Fig. 10.) This angular sampling rate is considerably lessdense than in previous work; approximately 150 source directionswould be required to cover the sphere at this rate compared to over2000 source directions used by Debevec et al. [2000].

In the recovered SBRDF, 2000 centers were used for each diffuse

10

20

ACTUAL REFLECTANCE SHARING

Figure 10: Comparison of actual and synthetic images for a novel illuminationdirection. The image on the right was rendered using the reflectance representationin Eq. (7) fit to four input images. Left inset shows a polar plot of the input (+) andoutput () lighting directions, with concentric circles representing angular distancesof 10 and 20 from the viewing direction.

color channel, and 5000 centers were used for the specular compo-nent. (Figure 12 shows scatter plots of the specular component asthe number of RBF centers is increased.) Each diffuse channel re-quires the storage of 2006 coefficients—the weights for 2000 cen-ters and six polynomial coefficients—and 2000 sample points ~qi,each with five components.6 Similarly, the specular component re-quires 5006 coefficients and 5000 centers. The total size is 66,024single precision floating-point numbers, or 258kB. This is a verycompact representation of both view and lighting effects.

Figure 10 shows real and synthetic images of the surface undernovel lighting conditions (the relative RMS difference is 9.5%), andshows how a smooth SBRDF is recovered despite the sparsity of theinput images. In particular, the ghosting effects observed in the ‘nosharing’ results of Fig. 7 are avoided. Figure 11 shows the spatialvariation in the specular component of the recovered SBRDF. Thegraph in the right of this figure is a (θh,φd) scatter plot of the spec-ular SBRDF on the tip of the nose (in transparent blue) and on thecheek (in red). (Large values of θh are outside the convex hull ofinput samples and are not displayed.) The recovered specular lobeon the nose is substantially larger than that on the cheek.

Small differences between the actual and predicted appearance inFig. 10 could be caused by global effects such as interreflection andsubsurface scattering if these effects are not completely absorbedby the representation. A more perceptible difference can be seenat the boundary of the lips, where the discontinuity in the specularcomponent is smoothed over due to the assumption of smooth spa-tial variation. In addition, the specular lobe on the nose in the rightof Fig. 10 is more broad than that in the actual image. Small move-ments of the subject during acquisition, source non-uniformity, anderrors in the estimated light source positions cause misalignmentof the images (both relative to the geometry and to each other) andadd noise to the reflectance samples. Accuracy could be improved,for example, using a high speed acquisition system such as that ofDebevec et al. [2000].

Figure 13 shows synthetic images with a novel viewpoint, againdemonstrating that a full SBRDF is recovered despite the fact thatonly one viewpoint is used as input. Additional examples of view-point variation can be seen on the accompanying video.

7.4 A special case: One Input Image

Due to its dimension-independence, the RBF representation can beadapted very easily to the extreme case when only one input im-age is available. This is an important special case that has not beenwell-addressed by either parametric or non-parametric methods. In

6This could be reduced, for example, by using the same centers locationsfor all three color channels.

8

Online Submission ID: papers 0056

Figure 11: Spatial variation in the estimated specular reflectance function. Left:synthesized specular component used to generate the image in the right of Fig. 10.Right: magnitude of the recovered specular SBRDF at the two surface points indicatedat left. Plots are of the SBRDF as a function of (θh,φd ) for θd = 5. The specular lobeon the nose is substantially more peaked than that on the cheek. (For comparison, theinset shows a Torrance-Sparrow lobe in the same coordinate system.)

3500 4500 5000

Figure 12: Estimated SBRDF as the number of RBF centers is increased using thegreedy algorithm. The 5000-center plots are the same as those on the right of Fig. 11.

one (orthographic, directional illumination) image all reflectancesamples lie on a hyperplane of constant w, reducing the dimensionof the SBRDF by one. Thus, we can use a simplified SBRDF rep-resentation, computing the surface radiance according to

Ik(~q) = sk

(

ak(~x)+N

∑i=1

λi‖~q−~qi‖

)

cosθi, k = R,G,B, (9)

where ~q = (x,y,u,v).In this case, the diffuse component is modeled as Lambertian,

and the albedo a(~x) is estimated directly from the reflectance sam-ples in the diffuse component of the input image (after shading andshadows are removed.) The specular component is estimated fromthe specular reflectance samples using the same fitting procedure asthe multi-image case. Figure 1 shows an example of a 2000-centerSBRDF under natural lighting from environment maps. (Thesewere rendered in real-time using precomputation as discussed inSect. 7.2, and the accompanying video demonstrates real-time light-ing manipulation.) Since a single image is used as input, only a 2Dsubset of the angular variation is recovered, and Fresnel effects areignored.7 Also, by using a complete environment map, we neces-sarily extrapolate the reflectance function beyond the convex hullof input samples, where it is known to be less accurate. Despitethese limitations, the method obtains reasonable results, and wedo not know of any existing technique, either parametric or non-parametric, that provides comparable output.

7As done by Debevec et al. [2000], this representation could be enhancedto approximate Fresnel effects by using a data-driven microfacet model withan assumed index of refraction.

Figure 13: Synthesized images for two novel viewpoints. Even though the input im-ages are captured from a single viewpoint, a complete SBRDF is recovered, includingview-dependent effects.

8 Conclusions and Future WorkThis paper presents a method for exploiting spatial coherence to re-cover a non-parametric, spatially-varying reflectance function froma sparse set of images of known geometry. Reflectance estimationis framed as a scattered-data interpolation problem in a joint spa-tial/angular domain, an approach that allows the exchange of spa-tial resolution for an increase in angular resolution of the reflectancefunction. The method combines some of the benefits of paramet-ric methods and dense, non-parametric methods that do not exploitspatial coherence. For surfaces with (generalized) smooth spatialvariation, spatial reflectance can be approximated from a very smallnumber of images, and at the same time, the estimate converges tothe true appearance as the number of images increases.

This paper also presents a flexible representation of reflectancebased on radial basis functions (RBFs), and shows how this repre-sentation can be adapted to handle: 1) homogeneous BRDF data,2) smooth spatially-varying reflectance from multiple images, 3)spatial variation with texture, and 4) a single input image.

The most immediate practical issue for future work involvescomputational efficiency. We have demonstrated that the RBF-based representation is a useful intermediate representation ofspatially-varying reflectance, since it can be used in combinationwith current rendering techniques based on precomputed informa-tion. To improve this, it may be possible to develop real-time ren-dering techniques directly from the RBF-based representation. Forexample, fast multipole methods can be used to reduce the evalu-ation of Eq. (2) from O(N2) to O(N logN) [Beatson and Newsam1992]. This may be a viable alternative to using factored forms ofBRDFs [McCool et al. 2001; Jaroszkiewicz and McCool 2003] andmay provide a practical approach for real-time rendering of surfaceswith spatially-varying, non-parametric reflectance.

In addition, the method could be improved to more easily handlegeneral spatial variation, including discontinuities in the shape ofthe specular lobe. At present, we assume these discontinuities tobe given as input, but they could be automatically detected throughclustering and segmentation (perhaps using diffuse color as a cue.)

The increasing use of measured reflectance data in computergraphics requires efficient methods to acquire and represent suchdata. In this context, appropriate parameterizations, representationsand signal-processing techniques are likely to be crucial. This pa-per presents a step in this direction by providing a method for usingsparse, image-based datasets to accurately recover reflectance.

References

BEATSON, R., AND NEWSAM, G. 1992. Fast evaluation of radial basisfunctions: I. Comput. Math. Appl. 24, 7–19.

9

Online Submission ID: papers 0056

BEATSON, R.K. GOODSELL, G., AND POWELL, M. 1995. On multigridtechniques for thin plate spline interpolation in two dimensions. Lecturesin Applied Mathematics 32, 77–97.

BUHMANN, M. 2003. Radial basis functions. Cambridge University Press.

CABRAL, B., OLANO, M., AND NEMEC, P. 1999. Reflection space imagebased rendering. In Proceedings of ACM SIGGRAPH, 165–170.

CARLSON, R., AND FOLEY, T. 1992. Interpolation of track data withradial basis methods. Computers and Mathematics with Applications 24,12, 27–34.

CARR, J., BEATSON, R., CHERRIE, J., MITCHELL, T., FRIGHT, W., MC-CALLUM, B., AND EVANS, T. 2001. Reconstruction and representationof 3D objects with radial basis functions. In Proceedings of ACM SIG-GRAPH, 67–76.

CHEN, W., BOUGUET, J., CHU, M., AND GRZESZCZUK, R. 2002. Lightfield mapping: Efficient representation and hardware rendering of sur-face light fields. ACM Transactions on Graphics (Proc. ACM SIG-GRAPH) 21, 3, 447–456.

DEBEVEC, P., HAWKINS, T., TCHOU, C., DUIKER, H., SAROKIN, W.,AND SAGAR, M. 2000. Acquiring the reflectance field of a human face.In Proceedings of ACM SIGGRAPH, 145–156.

DINH, H., TURK, G., AND SLABAUGH, G. 2001. Reconstructing surfacesusing anisotropic basis functions. In Proc. Int. Conf. on Computer Vision,606–613.

DUCHON, J. 1977. Splines minimizing rotation-invariant semi-norms inSobelev spaces. In Constructive theory of functions of several variables,W. Schempp and K. Zeller, Eds. Springer-Verlag, 85–100.

GEORGHIADES, A. 2003. Recovering 3-D shape and reflectance froma small number of photographs. In Rendering Techniques 2003 (Proc.Eurographics Symposium on Rendering), 230–240.

GORTLER, S., GRZESZCZUK, R., SZELISKI, R., AND COHEN, M. 1996.The lumigraph. In Proceedings of ACM SIGGRAPH, 43–54.

GU, X., GORTLER, S., AND HOPPE, H. 2002. Geometry images. ACMTransactions on Graphics (Proc. ACM SIGGRAPH) 21, 3, 355–361.

HERTZMANN, A., AND SEITZ, S. 2003. Shape and material by example:A photometric stereo approach. In Proc. IEEE Conf. Computer Visionand Pattern Recognition.

JAROSZKIEWICZ, R., AND MCCOOL, M. 2003. Fast extraction of BRDFsand material maps from images. In Proc. Graphics Interface, 1–10.

KARNER, K., MAYER, H., AND GERVAUTZ, M. 1996. An image basedmeasurement system for anisotropic reflectance. Computer Graphics Fo-rum 15, 3, 119–128.

KAUTZ, J., AND MCCOOL, M. 1999. Interactive rendering with arbitraryBRDFs using separable approximations. In Rendering Techniques ’99(Proc. Eurographics Rendering Workshop), 247–260.

LAFORTUNE, E., FOO, S., TORRANCE, K., AND GREENBERG, D. 1997.Non-linear approximation of reflectance functions. In Proceedings ofACM SIGGRAPH, 117–126.

LAWRENCE, J., RUSINKIEWICZ, S., AND RAMAMOORTHI, R. 2004. Effi-cient BRDF importance sampling using a factored representation. ACMTransactions on Graphics (Proc. ACM SIGGRAPH) 23, 3, 494–503.

LEE, A., SWELDENS, W., SCHRODER, P., COWSAR, L., AND DOBKIN,D. 1998. MAPS: Multiresolution adaptive parameterization of surfaces.In Proceedings of ACM SIGGRAPH, 95–104.

LENSCH, H., KAUTZ, J., GOESELE, M., HEIDRICH, W., AND SEIDEL,H.-P. 2001. Image-based reconstruction of spatially varying materi-als. In Rendering Techniques 2001 (Proc. Eurographics Rendering Work-shop).

LEVOY, M., AND HANRAHAN, P. 1996. Light field rendering. In Proceed-ings of ACM SIGGRAPH, 31–42.

LU, R., KOENDERINK, J., AND KAPPERS, A. 1998. Optical properties(bidirectional reflection distribution functions) of velvet. Applied Optics37, 25, 5974–5984.

MARSCHNER, S. R., WESTIN, S. H., LAFORTUNE, E. P. F., TORRANCE,K. E., AND GREENBERG, D. P. 1999. Image-based BRDF measure-ment including human skin. In Rendering Techniques ’99 (Proc. Euro-graphics Rendering Workshop), 139–152.

MATUSIK, W., PFISTER, H., BRAND, M., AND MCMILLAN, L. 2002.Image-based 3D photography using opacity hulls. ACM Transactions onGraphics (Proc. ACM SIGGRAPH) 21, 3, 427–437.

MATUSIK, W., PFISTER, H., BRAND, M., AND MCMILLAN, L. 2003.A data-driven reflectance model. ACM Transactions on Graphics(Proc. ACM SIGGRAPH) 22, 3, 759–769.

MATUSIK, W., PFISTER, H., BRAND, M., AND MCMILLAN, L. 2003.Efficient isotropic BRDF measurement. In Rendering Techniques 2003(Proc. Eurographics Symposium on Rendering), 241–248.

MCALLISTER, D. K., LASTRA, A., AND HEIDRICH, W. 2002. Efficientrendering of spatial bi-directional reflectance distribution functions. InGraphics Hardware 2002 (Proc. Eurographics/ACM SIGGRAPH Hard-ware Workshop), 79–88.

MCALLISTER, D. K. 2002. A Generalized Surface Appearance Represen-tation for Computer Graphics. PhD thesis, University of North Carolinaat Chapel Hill.

MCCOOL, M., ANG, J., AND AHMAD, A. 2001. Homomorphic factoriza-tion of BRDFs for high-performance rendering. In Proceedings of ACMSIGGRAPH, 171–178.

MICCHELLI, C. A. 1986. Interpolation of scattered data: distance matricesand conditionally positive definite functions. Constructive Approxima-tion 1, 11–22.

NAYAR, S., FANG, X., AND BOULT, T. 1997. Separation of reflectioncomponents using color and polarization. Int. Journal of Computer Vi-sion 21, 3, 163–186.

NAYAR, S., BELHUMEUR, P., AND BOULT, T. 2004. Lighting sensitivedisplay. ACM Transactions on Graphics 23, 4, 963–979.

OREN, M., AND NAYAR, S. 1994. Generalization of lambert’s reflectancemodel. In Proceedings of ACM SIGGRAPH, 239–246.

POWELL, M. 1992. The theory of radial basis function approximation in1990. In Advances in Numerical Analysis, Vol. II, W. Light, Ed. OxfordScience Publications, 105–210.

RAMAMOORTHI, R., AND HANRAHAN, P. 2002. Frequency space envi-ronment map rendering. ACM Transactions on Graphics (Proc. ACMSIGGRAPH) 21, 3, 517–526.

RUSINKIEWICZ, S. 1998. A new change of variables for efficient BRDFrepresentation. In Rendering Techniques ’98 (Proc. Eurographics Ren-dering Workshop), 11–22.

SATO, Y., AND IKEUTCHI, K. 1994. Temporal-color space analysis ofreflection. J. Optical Society of America A 11, 11, 2990–3002.

SATO, Y., WHEELER, M. D., AND IKEUCHI, K. 1997. Object shape andreflectance modeling from observation. In Proceedings of ACM SIG-GRAPH, 379–387.

SHAFER, S. 1985. Using color to separate reflection components. COLORresearch and applications 10, 4, 210–218.

SLOAN, P., HALL, J., HART, J., AND SNYDER, J. 2003. Clustered prin-cipal components for precomputed radiance transfer. ACM Transactionson Graphics (Proc. ACM SIGGRAPH) 22, 3, 382–391.

TERZOPOULOS, D. 1983. Multilevel computational processes for visualsurface reconstruction. Computer Vision, Graphics and Image Process-ing 24, 52–96.

WAHBA, G. 1990. Spline models for observational data. No. 59 in CBMS-NSF regional conference series in applied mathematics. SIAM.

WOOD, D., AZUMA, D., ALDINGER, K., CURLESS, B., DUCHAMP, T.,SALESIN, D., AND STUETZLE, W. 2000. Surface light fields for 3Dphotography. In Proceedings of ACM SIGGRAPH, 287–296.

YU, Y., DEBEVEC, P., MALIK, J., AND HAWKINS, T. 1999. Inverseglobal illumination: recovering reflectance models of real scenes fromphotographs. In Proceedings of ACM SIGGRAPH, 215–224.

ZIENKIEWICZ, O. 1979. The Finite Element Method, 3rd ed. McGraw-Hill.

10