1

Adaptive Wavelet Rendering

Author: Ryan Overbeck

Ravi Ramamoorthi

Presenter: Guillaume de Choulot

Craig Donner

2

The Problem (combined effects)

∫ Pixel

Area

∫ Area

Light

∫ Camera

Aperture

Pixel = … 6D

3

General combinations of effects

4D 5D 6D

Antialiasing

Depth of field Antialiasing

Depth of field

Motion Blur

Antialiasing

Depth of field

Area Lighting

Antialiasing

Depth of field

Envir. Lighting

Antialiasing

Depth of field

Area Lighting

1 Bounce GI

6D 8D

4

Monte Carlo Problem (1/1)

Noisy for low sample counts (smooth regions)

32 Samples Per Pixel

5

Monte Carlo Problem (2/2)

Requires hundreds to thousands of samples

512 Samples Per Pixel

6

Our Solution (important sampling)

Adaptive Wavelet Rendering

32 Samples Per Pixel

(average)

7

Features of Adaptive Wavelet Rendering

Low Sample Counts Converges from smooth

8

Features of Adaptive Wavelet Rendering

Low Sample Counts Converges from smooth

Near Reference:

32 samples per pixel

Average of 32 Samples Per Pixel

9

Features of Adaptive Wavelet Rendering

Low Sample Counts Converges from smooth

Near Reference:

32 samples per pixel

Smooth Preview Quality:

8 samples per pixel

Average of 8 Samples Per Pixel

10

Features of Adaptive Wavelet Rendering

Low Sample Counts

Efficient

Less samples gives Faster render times

11

Features of Adaptive Wavelet Rendering

Low Sample Counts

Efficient

Less samples gives Faster render times

Monte Carlo (512 spp)

>6 Hours

> 6 Hours Monte Carlo

512 Samples Per Pixel

12

Features of Adaptive Wavelet Rendering

Low Sample Counts

Efficient

Less samples gives Faster render times

Monte Carlo (512 spp)

Our Method (32 spp)

>6 Hours 34 minutes

34 Minutes Adaptive Wavelet Rendering

32 Samples Per Pixel (average)

13

Features of Adaptive Wavelet Rendering

Low Sample Counts

Efficient

General

Insensitive to problem dimensionality

General combinations of effects

∫ Pixel

Area

∫ Area

Light

∫ Camera

Aperture

… 8D ∫ Hemi-

sphere

32 Samples per pixel (average)

15 minutes

14

15

The Insight (Important sampling)

16

Variance is often local

Variance

Low

High

17

Send more samples to high variance

Sample Count

Low

High

Our method

32 Samples Per Pixel

(average)

18

Two forms of variance

Edges

Smooth Variance

19

Variance from image space: edges

Variance Low

High

20

Focused samples to image edges

Samples Low

High

Final Result

21

Variance from other dims: smooth

Variance Low

High

Difficult for Monte Carlo

22

Smooth is easier in multi-scale

23

Smooth is easier for wavelets

calculate pixel

wavelet synthesis

24

Coarse Sampling of Smooth Regions

Samples Low

High

Final Result

25

Algorithm Outline

Start: 4 Samples per Pixel

Adaptive Sampling

Reconstruction

26

Related Work

Adaptive sampling

Bolin & Meyer 1998, Whitted 1980, Mitchell 1987, Veach and Guibas 1997, Walter et al. 2006 (Multidim Lightcuts)

Multi-scale

Keller 2001 (Hierarchical MC), Heinrich and Sindambiwe 1999, Guo 1998, Bala et al. 2003, Walter et al. 2005 (Lightcuts), Perona and Malik 1990

Wavelet sampling and reconstruction

Our method

works well for both edges and smooth regions

27

Background: Wavelets

28

Wavelets made of 2 functions

Wavelet Scale

Low Pass

(smooth)

Band Pass

(edges)

29

Wavelet Hierarchy (1D)

Wavelet Scale

Low Pass

(smooth) Band Pass

(edges)

Wavelet Hierarchy (1D)

k = 1

k = 2

k = 3

Wavelet Scale

1D Tensor Product 2D Basis

=

=

=

=

scale

scale

scale

scale

wavelet

wavelet

wavelet wavelet

scale-scale

scale-wavelet

wavelet-scale

wavelet-wavelet

k = 3

32

Wavelet used

k = 4

JPEG

2000

compression

33

Wavelets Basis (& DWT)

Wavelets (Multi-scale basis) VS Pixel Basis

Multi scale: coefficient/wavelet expressed in

different scale

Discret Wavelet Transform (DWT):

1) Pixels Wavelets coefficient (Analysis)

2) Wavelets Pixels (Synthesis)

34

Wavelet Hierarchy

scale-scale scale-wavelet wavelet-scale wavelet-wavelet

Smooth regions Edges

k=3

k=2

k=1

k=4 k=5

35

III) Algorithm Outline

0) Start: 4 Samples per Pixel (skipped)

1) Adaptive Sampling

2) Reconstruction

36

1) Adaptive Sampling

Insert all scale coefficients into a priority queue

While more samples:

Send samples to highest priority coefficient

Update priority queue

The problem:

How to compute priority for each scale coefficient?

37

scale-scale

38

scale-scale

39

Wavelet synthesis is smoothing

scale-scale

40

Coarse scale captures smoothness

scale-scale

Smooth Region

41

Edges are in the fine scale

scale-scale

Edges

Smooth Region

Smoothed out

42

Adaptive Sampling

Goals:

In smooth regions, more samples to coarse coefficients

Near edges, more samples to fine coefficients

Solution:

Compute priority based on variance and smoothness

43

Start with Scale Coefficients’ Variance

Scale Variance

low high

44

Smooth variance grows fine to coarse

Scale Variance

low high

Higher

Smooth Region

45

Edge variance stays the same

Scale Variance

low high

Same

Edge Region

46

Squared wavelet magnitudes

Scale Variance (Wavelet Magnitude)

low high

2

47

Edge wavelets grow fine to coarse

Scale Variance (Wavelet Magnitude)

low high

2

Higher (ok)

Edge Region

48

Smooth wavelets stay the same

Scale Variance (Wavelet Magnitude)

low high

2

Same

(problem)

Smooth Region

49

Priority equals the difference

Scale Variance (Wavelet Magnitude)

- =

Priority

low high

2

50

Edges: Higher priority at fine scales

Scale Variance

- =

Priority

low high

Higher

Edge Region (Wavelet Magnitude) 2

51

Smooth: Higher priority at coarse scales

Scale Variance

- =

Priority

low high

Higher

Smooth Region (Wavelet Magnitude) 2

52

After adaptive sampling

Smooth Region in Coarse Scale

Accurate

53

After adaptive sampling

Smooth Region in Coarse Scale

Edges in Fine Scale

Accurate

Accurate

54

After adaptive sampling Smooth Region in Coarse Scale

Edges in Fine Scale

Smooth Region in Fine Scale

Accurate

Accurate

Noisy

55

Reconstruction: smooth away fine scale noise

Smooth Region in Coarse Scale

Edges in Fine Scale

Smooth Region in Fine Scale

Accurate

Accurate

Noisy

56

Wavelets capture edges

scale-scale scale-wavelet wavelet-scale wavelet-wavelet

Edges

57

Wavelets capture edges and Noise

scale-scale scale-wavelet wavelet-scale wavelet-wavelet

Edges and Noise

58

Algorithm Outline

0) Start: 4 Samples per Pixel

1) Adaptive Sampling

-> 2) Reconstruction

59

Wavelet Reconstruction

Remove noise by suppressing wavelet magnitudes

How? Choose smoothest image which fits

samples

60

Monte Carlo: statistics

High

Low

Mean

+Standard Deviation (estimate of Error)

-Standard Deviation

61

Monte Carlo: statistics

High

Low

Mean

+Standard Deviation

-Standard Deviation

Monte Carlo Estimate Valid

Range

62

For pixel:

High

Low

Mean

+Standard Deviation

-Standard Deviation

Pixel

Luminance

63

For pixel: luminance

High

Low

Mean

+Standard Deviation

-Standard Deviation

Pixel

Luminance

64

For pixel: luminance

High

Low

Mean

+Standard Deviation

-Standard Deviation

Pixel

Brightness

65

For wavelet:

High

Low

Mean

+Standard Deviation

-Standard Deviation

Wavelet Coefficient

66

For wavelet: Smoothness

High

Low

Mean

+Standard Deviation

-Standard Deviation

Wavelet Coefficient

Smoothness

67

Take the smoothest value

High

Low

Mean

+Standard Deviation

-Standard Deviation

Wavelet Coefficient

Smoothness Our Estimate

68

Reconstruction computation?!

69

After Sampling

70

After Reconstruction

71

Results (1/2)

72

Results (2/2)

73

Limitations (1/3)

Wavelet artifacts when not enough samples

Ringing around edges

Our Method

Monte Carlo

Ringing

Noise

32 samples per pixel (avg.)

74

Limitations (2/3)

Wavelet artifacts when not enough samples

Ringing around edges

Overly smoothing

Our Method

Monte Carlo

Too Smooth

Noise

32 samples per pixel (avg.)

75

Limitations (3/3)

Wavelet artifacts when not enough samples

Ringing around edges

Overly smoothing

Potential solutions

Variance reduction (path splitting, QMC, etc.)

Reduce smoothing during reconstruction

Use depth and normals to improve statistics

Use more samples

76

Conclusion/Summary

Sample and reconstruct in wavelet basis

Features

Low Sample Counts

Efficient

General

Best for smooth image features