depth-of-field rendering by pyramidal image processing

Depth-of-Field Renderingby Pyramidal Image Processing

Martin Kraus (TU München) and

Magnus Strengert (Universität Stuttgart)

E U R O G R A P H I C S07

01 Introduction

02 Related work

03 Proposed method

04 Experiments

05 Future work

Outline of this Talk00

What Is Depth of Field?• The depth in front and beyond the focus

plane where objects appear to be in focus

• A property of all real optical systems– only virtual pin-hole cameras have infinite

depth of field

• Used for important photographic and cinematographic techniques

I N T R O D U C T I O N01

photo by Jon Sullivan (http://pdphoto.org/PictureDetail.php?mat=pdef&pg=8202)

photo by "che" (http://commons.wikimedia.org/wiki/Image:Dandelion_clock_detail.jpg)

Classification of Techniques:• Splatting [Potmesil & Chakravarty 1982]

• Stochastic sampling [Cook et al. 1984] – e.g., in distributed ray tracing or in REYES– state of the art in offline rendering

• Pre-filtering [Rokita 1993]– lots of artifacts [Demers 2004]– state of the art in real-time rendering

• Blurring of sub-images [Barsky 2004]

R E L A T E D W O R K02

High Quality in Real Time?• Real-time performance requires:

– independence of scene complexity• excludes stochastic sampling

– thus: image post-processing of a pin-hole color image and depth map

• independence of image synthesis for free

– convolution filtering is too expensive• even with FFTs on GPUs

– thus: blurring using pyramid algorithms

R E L A T E D W O R K02

High Quality in Real Time?• High image quality requires:

– smooth blurring without interpolation artifacts• pyramid blurring [Kraus & Strengert 2007]

– no incorrect color bleeding• excludes pre-filtering• separate blurring of sub-images

– disocclusion of semi-transparent pixels• inpainting of colors and depths with pyramid

algorithm [Strengert et al. 2006]

R E L A T E D W O R K02

Overview

P R O P O S E D M E T H O D03decomposition into sub-images

blending of sub-images blurring

resulting image

pin-hole image & depth map

1

2

5

3

4

matting disocclusion

Decomposition• decompose into sub-images and cull

foreground pixels according to depth map

P R O P O S E D M E T H O D03 1

Disocclusion• for each sub-image: disocclude culled

foreground (using pyramidal inpainting)

P R O P O S E D M E T H O D03 2

Matting• for each sub-image: compute alpha-

matting according to each pixel's depth

P R O P O S E D M E T H O D03 3

Blurring• for each sub-image: blur color and alpha

(using pyramidal blurring)

P R O P O S E D M E T H O D03 4

Blending• back-to-front blending of all sub-images

P R O P O S E D M E T H O D03 5

result computed in 70.4 ms

(12 sub-images, hardware:

NVIDIA GeForce 7900 GTX)

Main Features• independent of scene complexity

– and independent of image synthesis

• interactive perfomance on GPUs– real-time for small circles of confusion

• high image quality– avoids artifacts of pre-filtering techniques– let's do some experiments …

P R O P O S E D M E T H O D03

Our Method vs. pbrt• pbrt uses

stochasticsampling

• Which iswhich?

E X P E R I M E N T S04

Our Method vs. pbrt

E X P E R I M E N T S04pbrt

our method

Our Method vs. pbrt• Which is

which?

E X P E R I M E N T S04

Our Method vs. pbrt

E X P E R I M E N T S04pbrt

our method

Can We Break Our Method?• Yes, with a

very largelens radius.

E X P E R I M E N T S04pbrt

our method too opaquebleeding gray

Can We Break Our Method?

E X P E R I M E N T S04• Video with very large lens radii: eg07.mov

– also available at: http://wwwcg.in.tum.de/Research/Publications/DepthOfField

Are We there yet?• No, but we avoid typical rendering artifacts

of real-time techniques

• Specialized variants for better performance and image quality

• Alternative blur filters

• High-potential application:– gaze-directed focus

F U T U R E W O R K05

• The photos appear courtesy of "che" and

Jon Sullivan.

• The dragon model appears courtesy of the Stanford University Scanning Repository.

• The pbrt scene appears courtesy of Gregory Humphreys and Matt Pharr.

A C K N O W L E D G M E N T S06

Thank you!

And have a safe trip home!

Questions?

E U R O G R A P H I C S07

E U R O G R A P H I C S07