Download - Global Illumination Shadow Layers
![Page 1: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/1.jpg)
Global Illumination Shadow Layers
François Desrichard, David Vanderhaeghe, Mathias Paulin
IRIT, Université de Toulouse, CNRS, INPT, UPS, UT1, UT2J, France
July 12, 2019
![Page 2: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/2.jpg)
Cinematic Lighting Design
Arbitrary Output Variables (AOVs) allow editing without re-rendering
We provide shadow layers for compositing with little overhead
2
Diffuse, specular, normal, depth...
Animated geometry Textures, materials Light rig
Scene Composite Render
![Page 3: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/3.jpg)
Shadows
Strong visual cue, subject to artistic expression
Tolerant perception of shadow appearance (Hecher et al., 2014, Sattler et al., 2005)
3 Nightmare by Alla Chernova Horizon-Based Ambient Occlusion (HBAO) in Destiny 2
![Page 4: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/4.jpg)
Previous Work
4
Dragging with mouse (Pellacini et al., 2002)
On-surface deformation (Ritschel et al., 2010)
Rotation, pattern inlay (Obert et al., 2010) Shape simplification (DeCoro et al., 2007)
![Page 5: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/5.jpg)
● Light overestimation ● Empirical matte ● No indirect shadows
Comparison with Available Renderers
5
Main layer = original image Arnold / pbrt / Cycles, no shadow Ours, no shadow
Ours, shadow layer Arnold, shadow matte
![Page 6: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/6.jpg)
Comparison with Current Renderers
6
Main layer
Arnold, shadow matte
Ours, shadow layer
Compositing with the shadow layer
![Page 7: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/7.jpg)
Contributions
● Definition of shadow layer under global illumination
● Characterization of the shadow layer in the path space
● Path tracer rendering all layers in a single pass
7
![Page 8: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/8.jpg)
Defining Shadow
Interaction between three components
The user defines caster and catcher by their surface
How to account for indirect shadows? 8
Light
Caster
Catcher
![Page 9: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/9.jpg)
Occlusion can be direct or indirect
Screen Space Definition
9
Black body render B Invisible render T Main layer I
![Page 10: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/10.jpg)
As the difference between the two altered renders: S = T - B
Screen Space Definition
10
- =
Invisible render T Black body render B Shadow layer S
![Page 11: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/11.jpg)
Adding the shadow layer removes shadows
Screen Space Definition
11
= +
Shadow layer with matte No shadow Main layer
![Page 12: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/12.jpg)
Discussion on this Definition
● Agnostic to the rendering algorithm ● Takes into account indirect shadows ● Has a physical meaning: lost radiance
● Self-shadowing cannot be recovered ● No control on light or catcher ● Two additional renders per object
12
![Page 13: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/13.jpg)
Path integral formulation for the main layer (Veach, 1997)
For the shadow layer of caster C
Translation to the Path Space
13
= { all geometric light paths in the scene }
= { light paths encountering caster C }
= measurement contribution function considering C invisible
![Page 14: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/14.jpg)
● Input: N objects as casters Ci
● Output: the main layer and N shadow layers in a single pass
● The algorithm measures with , but also each with
○ Measuring with ⇔ scattering on all casters
○ Measuring with ⇔ skipping caster Ci
Integration in a Path Tracer
14
i i
i } p = 1 / 2
![Page 15: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/15.jpg)
1. Intersect green
Integration in a Path Tracer
15
3. Contribute to Sgreen
4. Cannot skip blue
2. Skip green now
? Sblue
Sgreen
![Page 16: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/16.jpg)
1. Contribute to Sgreen
Integration in a Path Tracer
16
3. Scatter on blue now
4. Contribute to I
2. Intersect blue
?
![Page 17: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/17.jpg)
Improved Artistic Control
Per-light separation
17
Main layer Separate light sources; shadow ratio I / (I + S)
![Page 18: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/18.jpg)
Improved Artistic Control
Direct and indirect separation
18
Main layer Direct and indirect shadow ratio
![Page 19: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/19.jpg)
Improved Artistic Control
Custom catchers and self-shadowing toggle
19
Main layer With and without self-shadows
![Page 20: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/20.jpg)
Editing Examples
Balancing shadow strength
20
![Page 21: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/21.jpg)
Editing Examples
21
+
Hibiscus / Pandanus shadow layer
Locally color graded image
Main layer
![Page 22: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/22.jpg)
Editing Examples
Shape transformation using the shadow ratio I / (I + S)
22
Main layer I (I + S) ⨯ the transformed shadow ratio
![Page 23: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/23.jpg)
Performance Overhead
● Additional intersection tests bring the most overhead
● Managing samples and filtering incurs a per-caster cost
● The sampling budget is now shared among all layers
For 1 to 5 casters, we measure a 1.1 to 1.3 overhead factor
23
![Page 24: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/24.jpg)
Limitations
When a shadow is created by multiple casters, it cannot be assigned to only one of them. A possible solution: consider them as one.
With N interacting casters, combinatorial explosion (2N shadow layers) 24
Main layer Boxes as separate casters Both boxes as one caster
![Page 25: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/25.jpg)
Limitations
Using the notion of surface to define an object is limiting: how to handle participating media?
25 Main layer No scattering No absorption
![Page 26: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/26.jpg)
Conclusion
● Definition of shadow layer under global illumination ○ Editable at the post-processing stage
● Characterization of the shadow layer in the path space ○ Amenable to Monte-Carlo integration algorithms
● Path tracer rendering all layers in a single pass
○ With an overhead in time and render convergence
26
![Page 27: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/27.jpg)
Thank you for your attention
https://www.irit.fr/STORM/site/shadow-layers
https://github.com/frcsdes/shadow-layers
27
![Page 28: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/28.jpg)
References
(DeCoro et al., 2007) DECORO C., COLE F., FINKELSTEIN A., RUSINKIEWICZ S.: Stylized shadows. In Proc. 5th International Symposium on Non-photorealistic Animation and Rendering (New York, NY, USA, 2007), NPAR ’07, ACM, pp. 77–83. doi:10.1145/1274871.1274884.
(Hecher et al., 2014) HECHER M., BERNHARD M., MATTAUSCH O., SCHERZER D., WIMMER M.: A comparative perceptual study of soft-shadow algorithms. ACM Trans. Appl. Percept. 11, 2 (July 2014). doi:10.1145/2620029.
(Obert et al., 2010) OBERT J., PELLACINI F., PATTANAIK S.: Visibility editing for all-frequency shadow design. Computer Graphics Forum 29, 4 (2010), 1441–1449. doi:10.1111/j.1467-8659.2010.01741.x.
(Pellacini et al., 2002) PELLACINI F., TOLE P., GREENBERG D. P.: A user interface for interactive cinematic shadow design. ACM TOG 21, 3 (July 2002), 563–566. doi:10.1145/566654.566617.
(Ritschel et al., 2010) RITSCHEL T., THORMÄHLEN T., DACHSBACHER C., KAUTZ J., SEIDEL H.-P.: Interactive on-surface signal deformation. ACM TOG 29, 4 (July 2010), 36:1–36:8. doi:10.1145/1778765.1778773.
(Sattler et al., 2005) SATTLER M., SARLETTE R., MÜCKEN T., KLEIN R.: Exploitation of human shadow perception for fast shadow rendering. In Proc. 2nd symposium on Applied perception in graphics and visualization (2005), APGV05, Association for Computing Machinery, p. 131–134. doi:10.1145/1080402.1080426.
(Veach, 1997) VEACH E.: Robust Monte Carlo Methods for Light Transport Simulation. PhD thesis, Stanford University, Stanford, CA, USA, 1997. AAI9837162.
28
![Page 29: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/29.jpg)
Usually exported using a light path expression: .* <[RT].’tallbox’> .*
-
Scattered Layer
29
=
Scattered layer Black body render B Main layer I
![Page 30: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/30.jpg)
Scattered Layer
30
![Page 31: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/31.jpg)
Participating Media
31 Shadow layer test Shadow removal test
![Page 32: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/32.jpg)
Integration in a Path Tracer
32
1. Propagation
The first time a caster C is encountered, the path now belongs to
● Measure with ⇒ skip the surface and contribute to S ○ C is assigned to the path and always skipped ○ No other caster may be skipped
● Measure with ⇒ scatter normally and contribute to I ○ C will never be skipped ○ Other casters can still be assigned
Both outcomes have probability p = 1 / 2 and introduce a factor 1 / p
![Page 33: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/33.jpg)
2. Direct lighting
If the path was assigned a caster the shadow ray can skip it, and contributes radiance to the corresponding shadow layer
Otherwise: ● An unoccluded shadow ray contributes to I ● An occluded shadow ray can skip a caster that was never
encountered and contribute to its shadow layer
Overall, less zero radiance paths
Integration in a Path Tracer
33
![Page 34: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/34.jpg)
Future work: locally adapt p to the probability of an encounter?
Skip Probability
34
p = 0 p = 1
Global illumination Direct shadows
Direct illumination Global shadows
p = 1 / 2
Our choice
![Page 35: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/35.jpg)
Scene N Equal sampling Equal time Equal variance
SPP Time SSIM ZRP Time SPP μ-Var Var Time μ-SPP
Teaser
0 2048 15’ 27” 0.903 26% 15’ 2048 0.003 0.01 18’ 24” 2321
1 2048 17’ 27” 0.901 17% 15’ 1824 0.004 0.01 20’ 08” 2318
2 2048 18’ 07” 0.896 12% 15’ 1728 0.004 0.01 21’ 46” 2360
3 2048 18’ 28” 0.896 10% 15’ 1696 0.004 0.01 22’ 14” 2371
Dragon 0 4096 27’ 53” 0.927 91% 30’ 9856 0.153 0.1 34’ 58” 8122
1 4096 33’ 33” 0.889 90% 30’ 8256 0.153 0.1 41’ 44” 8139
Island
0 1024 34’ 30” 0.992 87% 30’ 960 0.007 0.01 39’ 36” 1277
1 1024 42’ 35” 0.992 81% 30’ 768 0.007 0.01 51’ 11” 1289
3 1024 46’ 05” 0.992 80% 30’ 704 0.008 0.01 54’ 16” 1310
5 1024 47’ 20” 0.992 79% 30’ 704 0.008 0.01 55’ 56” 1317
Flowers 0 256 03’ 19” 0.901 15% 10’ 832 0.271 0.5 4’ 01” 291
1 256 03’ 28” 0.899 12% 10’ 784 0.271 0.5 4’ 15” 291
Performance Table
35
![Page 36: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/36.jpg)
Performance Table
36
Teaser Dragon
Island Flowers
![Page 37: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/37.jpg)
Implementation Guidelines
Bidirectional Path Tracing 1. Propagation
Altered as path tracing for camera and light sub-paths 1. Integration
The connection step can ignore occlusion Only form full paths with coherent propagation history: for instance, a sub-path that scattered on C cannot be connected to one that skipped C
37
![Page 38: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/38.jpg)
Implementation Guidelines
Photon Mapping 1. Propagation
Altered as path tracing for photons and gathering rays 1. Integration
Gathering must remain coherent with propagation history: for instance, photons that scattered on C cannot be gathered by rays that skipped C
38
![Page 39: Global Illumination Shadow Layers](https://reader031.vdocuments.site/reader031/viewer/2022012915/61c56f24db554b3f956d8841/html5/thumbnails/39.jpg)
Implementation Guidelines
Metropolis Light Transport The mutation set does not change Propagation and integration are addressed during mutation acceptance 1. Propagation
A mutated path is allowed to cross the surface of at most one caster C … 1. Integration
… If the resulting history is coherent: it should not scatter on C elsewhere
39