inferring reflectance functions from wavelet noise pieter peers philip dutré pieter peers philip...

Inferring Reflectance FunctionsInferring Reflectance Functionsfrom Wavelet Noisefrom Wavelet Noise

Pieter Peers

Philip Dutré

Pieter Peers

Philip Dutré

June 30th 2005

Department of Computer Science

Image-based Relighting / Environment MattingImage-based Relighting / Environment Matting

Scene(fixed viewpoint)


…


Novel Incident Illumination

+


… …


Novel Incident Illumination Compute Relit Image

+ =


… …


Novel Incident Illumination Compute Relit Image

+ =

ReflectanceFunction

Examples of Reflectance FunctionsExamples of Reflectance Functions

Diffuse BallSpecular Ball

Examples of Reflectance FunctionsExamples of Reflectance Functions


Reflectance Function Reflectance Function

Reflectance Functions (frequency domain)Reflectance Functions (frequency domain)


Reflectance Function (frequency domain) Reflectance Function (frequency domain)

Reflectance Functions (wavelet domain)Reflectance Functions (wavelet domain)


Reflectance Function (wavelet domain) Reflectance Function (wavelet domain)

Relight a PixelRelight a Pixel

Novel Incident IlluminationSpecular Ball

Relit pixel value?

Reflectance Function (wavelet space)



Reflectance Function (wavelet space) Incident Illumination (wavelet space)




( )




( )

Onlynon-zero

coefficients

Directly Observing Reflectance FunctionsDirectly Observing Reflectance Functions

Controlled Incident IlluminationPhotograph of Specular Ball

Emit(e.g. from CRT)



ReflectanceFunction

(unknown)

Observed pixel

Controlled Incident Illumination (wavelet space)



Unknown Reflectance Function (wavelet space)

( )





( )

Onlynon-zero

coefficients

Observed coefficient

Unknown Reflectance Function (wavelet space)

Number of ObservationsNumber of Observations

Specular Ball


#Photographs=

#Illumination pixels

Incident Illumination

Number of Observations ProblemNumber of Observations Problem

Specular Ball



1000x

1000

1000x

1000

#Photographs=

#Illumination pixels

Wavelet Noise IlluminationWavelet Noise Illumination

Wavelet Noise

•Normal distribution of wavelet coefficients

•Mean : 0.0

•Standard deviation : 1.0

•Rescale Wavelet Noise Pattern to fit into [0..1] range

Wavelet Noise Pattern

Wavelet Noise Pattern (wavelet space)

Advantages

•Arbitrary number of different patterns possible

•Any reflectance function gives a non-zero response

•Constant average luminance

Estimating Wavelet Coefficients Estimating Wavelet Coefficients

(Unknown)Reflectance Function

Wavelet Noise

Assume: positions of are knownQuestion: what are the magnitudes?

( ) =Observed

Pixel Value


( ) =

Leave out zero coefficients(of the reflectance function)

Wavelet Noise (linearized)

Reflectance Function(linearized)

Observed Pixel Value


= …

Multiple observations matrix-vector multiplication

…

Wavelet NoiseReflectance

Function

Observed PixelValues

# em

itted

pat

tern

s # observations


=

Finding magnitudes : Linear Least Squares problem

… …


Function



=

Estimation error when onlya part is approximated?

… …


Function


Partial EstimationPartial Estimation

+… … …= = …


Function

ObservedPixel Values


According to a normal distribution

+… … …= = …


Function



According to a normal distribution

+… … …= = …


Function


Normal Normal distributiondistribution


+… …= = …


Function


Finding the best approximation for : Linear Least Squares problem

NoIse

Inferring Reflectance FunctionsInferring Reflectance Functions

Reflectance Function(2D wavelet space)

Priority Queueof Candidates

OverviewOverview

Record photographs

Emit

Wavelet Noise

Predetermined number of photographs

OverviewOverview

Record photographs

Infer Reflectance Functions

Reflectance Function

Progressive Algorithm

For each pixel

OverviewOverview

Record photographs

Infer Reflectance Functions

Compute Relit Image

Relight


ResultsResults

64 Haar Wavelet Coefficients256 Photographs

Reference Photograph

ResultsResults

128 Haar Wavelet Coefficients512 Photographs

Reference Photograph

Results: High Frequency IlluminationResults: High Frequency Illumination

Conclusion & Future WorkConclusion & Future Work

Inferring Reflectance Functions from Wavelet Noise– No restriction on material properties– Stochastic illumination patterns– Trade-off quality versus acquisition time

Future Work– Noise filtering– Higher-order wavelets

inferring reflectance functions from wavelet noise pieter peers philip dutré pieter peers philip...

Documents