A Perceptual Heuristicfor Shadow Computationin Photo-Realistic Images

Wednesday, 2 August 2006

Peter Vangorp Olivier Dumont Toon Lenaerts Philip Dutré

Katholieke Universiteit Leuven

{peter.vangorp,olivier.dumont,toon.lenaerts,philip.dutre}@cs.kuleuven.be

IntroductionIntroduction

• 3 types of realistic rendering

IntroductionIntroduction

• 3 types of realistic rendering

– radiometric accuracy“render everything a photometer can detect”

IntroductionIntroduction

• 3 types of realistic rendering

– radiometric accuracy“render everything a photometer can detect”

– physiological perception“render only what the eye can see”

IntroductionIntroduction

• 3 types of realistic rendering

– radiometric accuracy“render everything a photometer can detect”

– physiological perception“render only what the eye can see”

– psychological perception“render only what the brain can see”

Radiometric accuracyRadiometric accuracy

• measure shapes, light sources, materials, ...

photometer

Cornell box

[Meyer et al. 1986]

Physiological perceptionPhysiological perception

• use low-level limitations of human visual system

threshold vs intensity, contrast sensitivity, ...

reference threshold map visibly indistinguishableadaptive rendering[Ramasubramanian et al. 1999]

Physiological perceptionPhysiological perception

• use low-level limitations of human visual system

threshold vs intensity, contrast sensitivity, ...

reference threshold map visibly indistinguishableadaptive rendering[Ramasubramanian et al. 1999]

Psychological perceptionPsychological perception

• use higher-level heuristics“Is a shadow necessary for the

realism of a scene?” [Thompson et al. 1998]

Psychological perceptionPsychological perception

• use higher-level heuristics“Is a shadow necessary for the

realism of a scene?”

“Do we need highlights to convey material properties?”

[Thompson et al. 1998]

[Fleming et al. 2004]

Psychological perceptionPsychological perception

• use higher-level heuristics“Is a shadow necessary for the

realism of a scene?”

“Do we need highlights to convey material properties?”

“How detailed should the geometry be?”

[Thompson et al. 1998]

[Luebke 2001]

[Fleming et al. 2004]

Motivation & GoalsMotivation & Goals

• Motivation

– shadows are important for perception of realism

• Goal

– detect perceptually important shadows in the scene

– render important shadows accurately

– approximate unimportant shadows

[Kersten et al. 1997]

MethodologyMethodology

1. Psycho-physical experiments

2. Derive a heuristic predicting shadow importance

3. Design a perceptually driven algorithm

4. Experimental validation

MethodologyMethodology

1. Psycho-physical experiments

2. Derive a heuristic predicting shadow importance

3. Design a perceptually driven algorithm

4. Experimental validation

MethodologyMethodology

1. Psycho-physical experiments

2. Derive a heuristic predicting shadow importance

3. Design a perceptually driven algorithm

4. Experimental validation

MethodologyMethodology

1. Psycho-physical experiments

2. Derive a heuristic predicting shadow importance

3. Design a perceptually driven algorithm

4. Experimental validation

1. Perceptual experiments1. Perceptual experiments

• Test setup: 162 images, varying sphere over 9 radii and 9 heights

correct shadow:

no shadow:(avg. illumination)

1. Perceptual experiments1. Perceptual experiments

• “Does the lighting in this image look realistic?”

– single stimulus– 5000+ decisions– avg. 2 sec / decision

2. Heuristic2. Heuristic

large differencein realism, e.g.

small differencein realism, e.g.

sphere radius sp

here

heig

ht

diff

eren

ce in

rea

lism

3. A perceptually driven algorithm3. A perceptually driven algorithm

• Ray tracing

3. A perceptually driven algorithm3. A perceptually driven algorithm

• Ray tracing– shoot viewing ray

3. A perceptually driven algorithm3. A perceptually driven algorithm

• Ray tracing– shoot viewing ray– evaluate heuristic in hit point to be shaded

in function of distance and solid angle

solid angledistance

3. A perceptually driven algorithm3. A perceptually driven algorithm

• Preprocessing step

– shadow photon map

[Jensen and Christensen 1995]

3. A perceptually driven algorithm3. A perceptually driven algorithm

• Preprocessing step

– shadow photon map

– shadow photons augmented with heuristic

[Jensen and Christensen 1995]

shadow photon map

3. A perceptually driven algorithm3. A perceptually driven algorithm

For eachviewing ray

Gather nearestshadow photons

Calculate averageperceptual value

avg < threshold

approximate:photon map

render accurately:shadow rays

yes no

3. A perceptually driven algorithm3. A perceptually driven algorithm

For eachviewing ray

Gather nearestshadow photons

Calculate averageperceptual value

avg < threshold

approximate:photon map

render accurately:shadow rays

yes no

3. A perceptually driven algorithm3. A perceptually driven algorithm

For eachviewing ray

Gather nearestshadow photons

Calculate averageperceptual value

avg < threshold

approximate:photon map

render accurately:shadow rays

yes no

For eachviewing ray

Gather nearestshadow photons

Calculate averageperceptual value

avg < threshold

approximate:photon map

render accurately:shadow rays

yes no

user-definedthreshold

3. A perceptually driven algorithm3. A perceptually driven algorithm

3. A perceptually driven algorithm3. A perceptually driven algorithm

For eachviewing ray

Gather nearestshadow photons

Calculate averageperceptual value

avg < threshold

approximate:photon map

render accurately:shadow rays

yes no

user-definedthreshold

3. A perceptually driven algorithm3. A perceptually driven algorithm

For eachviewing ray

Gather nearestshadow photons

Calculate averageperceptual value

avg < threshold

approximate:photon map

render accurately:shadow rays

yes no

user-definedthreshold

3. A perceptually driven algorithm3. A perceptually driven algorithm

4. Validation4. Validation

• Similar perceptual experiment

– “Do the lighting and the shadows look realistic?”

4. Validation4. Validation

• Similar perceptual experiment

– “Do the lighting and the shadows look realistic?”

• Stimuli: 6 scenes

– threshold 25%, 50%, 75%

– reference rendering (threshold 0%)

4. Validation4. Validation

• Similar perceptual experiment

– “Do the lighting and the shadows look realistic?”

• Stimuli: 6 scenes

– threshold 25%, 50%, 75%

– reference rendering (threshold 0%)

• 15 subjects, almost 6000 decisions, avg. 5 seconds

4. Validation4. Validation

• Up to 50% of the pixels can be approximated, without loss of perceptual realism

Threshold 25%Approx px 13%

Threshold 50%Approx px 24%

Threshold 75%Approx px 48%

ResultsResults

referencethreshold 90%

ResultsResults

referencethreshold 90%

ResultsResults

referencethreshold 90%

ResultsResults

referencethreshold 90%

reference

ResultsResults

threshold 80%

ResultsResults

referencethreshold 80%

ResultsResults

referencethreshold 80%

ResultsResults

referencethreshold 80%

reference

ResultsResults

threshold 90%

ResultsResults

referencethreshold 90%

ResultsResults

referencethreshold 90%

Conclusions & Future WorkConclusions & Future Work

• Intuitions confirmed by statistical data

• Rendering algorithm driven by perceptual information

Conclusions & Future WorkConclusions & Future Work

• Intuitions confirmed by statistical data

• Rendering algorithm driven by perceptual information

• Extend methodology to other phenomena

• Different questions than “Does this look realistic?”

• Better ways to incorporate perceptual information

– currently no significant acceleration yet

Questions?Questions?