classical methods in offline rendering

Classical Methods in Offline RenderingJerry Cao

Content Team

OverView

� Basics in Computer Graphics

� Whitted Ray Tracing

� Path Tracing

� Light Tracing

� Instant Radiosity

� Bidirectional Path Tracing

Ray Traced Images

Ray Traced Images

Ray Traced Images

Why Ray Tracing?

� Two well known methods for rendering.� Rasterization

� Ray Tracing

� Why Ray Tracing?� Unbiased Methods Available

� Similar to how reality works

� Much higher quality

Why Ray Tracing?

� Soft Shadow

� Color Bleeding

� Reflection & Refraction

� Caustics

� Depth of Field

� Motion Blur

� Subsurface Scattering

� …

How to do Ray Tracing

The very Basics

� Solid Angle� An object's solid angle in steradians is equal to the area of the segment of a unit

sphere, centered at the angle's vertex, that the object covers.

The very Basics

� Flux� Energy passing through a specific area per unit time

� Irradiance� Flux per unit area

� Radiance� Flux density per unit area, per solid angle

� Bidirectional Reflection Density Function (BRDF)� Gives a formalism for describing the reflection from a surface

� 𝑓𝑟 𝑝, 𝑤𝑖, 𝑤𝑜 =𝑑𝐿𝑜 𝑝,𝑤𝑜𝑑𝐸 𝑝,𝑤𝑖

Rendering Equation

� Or Light Transport Equation (LTE)

�𝐿𝑜 𝑝,𝑤𝑜 = 𝐿𝑒 𝑝,𝑤𝑜 + ∫ 𝐿𝑖 𝑝, 𝑤𝑖 𝑓 𝑝,𝑤𝑖, 𝑤𝑜 cos𝜃𝑖𝑑𝑤𝑖

Monte Carlo Integration

� To Evaluate an Integral� 𝐼 = ∫Ω𝑓(𝑥)dx

� We can use the following estimation

� 𝐹𝑁 =1𝑁Σ𝑖=1𝑁

𝑓 𝑥𝑖𝑝 𝑥𝑖

� Proof

�

𝐸 𝐹𝑁 = 𝐸1𝑁Σ𝑖=1𝑁

𝑓 𝑥𝑖𝑝 𝑥𝑖

= 1𝑁Σ𝑖=1𝑁 ∫Ω

𝑓 𝑥𝑖𝑝 𝑥𝑖

𝑝 𝑥𝑖 𝑑 𝑥

= ∫Ω 𝑓 𝑥 𝑑𝑥= 𝐼

Why Monte Carlo

� Estimation goes the same way regardless of how many dimensions there are.

� Only need to be able to evaluate the function at arbitrary point.

� Cons:

� Low Convergence Rate: 1√𝑁

Importance Sampling

� Sample where it matters most

� An example:

� 𝑓 𝑥 = { 0.01 𝑥 ∈ 0, 0.011.01 𝑥 ∈ [0.01, 1)

� Perfect pdf:

� p 𝑥 = { 0.01 𝑥 ∈ 0, 0.011.01 𝑥 ∈ [0.01, 1)

� A terrible pdf:

� p 𝑥 = { 99.01 𝑥 ∈ 0, 0.010.01 𝑥 ∈ [0.01, 1)

Multiple Importance Sampling

�𝐹𝑚𝑖𝑠 = Σ𝑖=1𝑛1𝑛𝑖Σ𝑗=1𝑛𝑖 𝑤𝑖 𝑋𝑖,𝑗

𝑓 𝑋𝑖,𝑗𝑝𝑖 𝑋𝑖,𝑗

�𝑊𝑖 should satisfy the following condition

�Σ𝑖=1𝑛 𝑤𝑖 𝑥 = 1 𝑖𝑓 𝑓 𝑥 ≠ 0�𝑤𝑖 𝑥 = 0 𝑖𝑓 𝑝𝑖 𝑥 = 0

Multiple Importance Sampling

� Common weight factors:� Balance Heuristic

�𝑤𝑖 𝑥 =𝑛𝑖𝑝𝑖 𝑥Σ𝑘𝑛𝑘𝑝𝑘 𝑥

� Power Heuristic

�𝑤𝑖 𝑥 =𝑛𝑖𝑞𝑖

𝑛

Σ𝑘𝑛𝑘𝑞𝑘𝑛

Multiple Importance Sampling

� MIS in direct light sampling

Multiple Importance Sampling

� MIS in Bidirectional Path Tracing

How to Sample a pdf

� Rejection Method� Inefficient

� Inversion method

� Compute the CDF 𝑃 𝑥 = ∫0𝑥 𝑝 𝑡 𝑑𝑡

� Compute the inverse of CDF 𝑃−1 𝑥

� Obtain a uniformly distributed random number, µ

� Compute 𝑋 = 𝑃−1(𝜇)

Whitted Ray Tracing

� Only dirac delta BRDF and light sources are considered.� Pure Reflective & Refractive

� Point light & directional light

My GPU Solution

� CUDA accelerated

� Some data:� 87w+ triangles in the dragon

� 640 x 480 resolution

� 7-8 fps on a GTX 260

Light Transport Equation

�𝐿𝑜 𝑝, 𝑤𝑜 = 𝐿𝑒 𝑝, 𝑤𝑜 + ∫ 𝐿𝑖 𝑝, 𝑤𝑖 𝑓 𝑝, 𝑤𝑖, 𝑤𝑜 cos𝜃𝑖𝑑𝑤𝑖

Relation between dA and dw

�𝑑𝑤 = 𝑐𝑜𝑠𝜃𝑟2𝑑𝐴

�𝑃𝐴 =𝑐𝑜𝑠𝜃𝑟2𝑃𝑤

Deeper dive into LTE

•𝐿 𝑝0,𝑤𝑜 = 𝐿𝑒 𝑝0, 𝑤𝑜 + ∫ 𝐿 𝑝0, 𝑤 𝑓 𝑝0,𝑤, 𝑤𝑜 cos𝜃0→1𝑑𝑤

= 𝐿𝑒 𝑝1 → 𝑝0 + ∫ 𝐿 𝑝2 → 𝑝1 𝑓 𝑝2 → 𝑝1 → 𝑝0𝑉(𝑝1↔𝑝2)𝑐𝑜𝑠𝜃1→2𝑐𝑜𝑠𝜃2→1

𝑟2𝑑𝐴2

= 𝐿𝑒 𝑝1 → 𝑝0 + ∫ 𝐿 𝑝2 → 𝑝1 𝑓 𝑝2 → 𝑝1 → 𝑝0 𝐺 𝑝1 ↔ 𝑝2 𝑑𝐴2

• 𝐺 𝑝1 ↔ 𝑝2 =𝑉(𝑝1↔𝑝2)𝑐𝑜𝑠𝜃1→2𝑐𝑜𝑠𝜃2→1𝑟2

Deeper dive into LTE

� 𝐿 𝑝1 → 𝑝0 = 𝐿𝑒 𝑝1 → 𝑝0 + ∫ 𝐿 𝑝2 → 𝑝1 𝑓 𝑝2 → 𝑝1 → 𝑝0 𝐺 𝑝1 ↔ 𝑝2 𝑑𝐴2

� In the same way we can expend the red one:

� 𝐿 𝑝2 → 𝑝1 = 𝐿𝑒 𝑝2 → 𝑝1 + ∫ 𝐿 𝑝3 → 𝑝2 𝑓 𝑝3 → 𝑝2 → 𝑝1 𝐺 𝑝2 ↔ 𝑝3 𝑑𝐴3

� Dropping the brown part, we have the following equation for direct illumination:

� 𝐿 𝑝1 → 𝑝0 = 𝐿𝑒 𝑝1 → 𝑝0 + ∫ 𝐿𝑒 𝑝2 → 𝑝1 𝑓 𝑝2 → 𝑝1 → 𝑝0 𝐺 𝑝1 ↔ 𝑝2 𝑑𝐴2

Direct(Local) Illumination vs Global Illumination

Deeper dive into LTE

�

𝐿 𝑝0,𝑤𝑜 = 𝐿𝑒 𝑝1 → 𝑝0+ ∫ 𝐿𝑒 𝑝2 → 𝑝1 𝑓 𝑝2 → 𝑝1 → 𝑝0 𝐺 𝑝1 ↔ 𝑝2 𝑑𝐴2+ 𝐿𝑒 𝑝3 → 𝑝2 𝑓 𝑝3 → 𝑝2 → 𝑝1 𝐺 𝑝2 ↔ 𝑝3 𝑓 𝑝2 → 𝑝1 → 𝑝0 𝐺 𝑝1 ↔ 𝑝2 𝑑𝐴3𝑑𝐴2

…+∫…∫𝐿𝑒 𝑝𝑛 → 𝑝𝑛−1 𝑘=2𝑘=𝑛 𝑓 𝑝𝑘 → 𝑝𝑘−1 → 𝑝𝑘−2 𝐺 𝑝𝑘 ↔ 𝑝𝑘−1 𝑑𝐴𝑛 …𝑑𝐴2

� We define the following term:

� 𝑇𝑛(𝑝1 → 𝑝0)=∫…∫𝐿𝑒 𝑝𝑛 → 𝑝𝑛−1 𝑘=2𝑘=𝑛 𝑓 𝑝𝑘 → 𝑝𝑘−1 → 𝑝𝑘−2 𝐺 𝑝𝑘 ↔ 𝑝𝑘−1 𝑑𝐴𝑛 …𝑑𝐴2

Deeper dive into LTE

� 𝐿 𝑝1 → 𝑝0 = 𝑖=1𝑖=∞𝑇𝑖(𝑝1 → 𝑝0)

� It turns out to be a very simple equation, what it says is relatively straightforward, radiance from P1 to P0 is the combination of:� Radiance comes from light directly

� Direct Illumination

� Light contribution from multiple bounces

Russian Roulette

� For each Ti after several bounces, we start Russian roulette:

� 𝑇𝑖′ = {𝑇𝑖𝑝𝑥 ∈ 0, 𝑝

0 𝑥 ∈ [𝑝, 1)

� The average of 𝑇𝑖′ is exactly the same with 𝑇𝑖, which makes the above estimation unbiased.

� One step further:

� 𝐿 𝑝1 → 𝑝0 = T1 + T2 + T3 + T4 +1𝑝(𝑇5 +

1𝑝(𝑇6 + ⋯))

Evaluate the integral of LTE

� We’ll focus on path with specific number of vertices.� 𝑇𝑛(𝑝1 → 𝑝0)=∫…∫𝐿𝑒 𝑝𝑛 → 𝑝𝑛−1 𝑘=2𝑘=𝑛 𝑓 𝑝𝑘 → 𝑝𝑘−1 → 𝑝𝑘−2 𝐺 𝑝𝑘 ↔ 𝑝𝑘−1 𝑑𝐴𝑛 …𝑑𝐴2

� With Monte Carlo method, we only need to evaluate the following equation:

� 𝑇𝑛 𝑝1 → 𝑝0 =1𝑁 𝑖=1𝑁

𝐿𝑒 𝑝𝑖,𝑛→𝑝𝑖,𝑛−1 𝑘=2𝑘=𝑛 𝑓 𝑝𝑖,𝑘→𝑝𝑖,𝑘−1→𝑝𝑖,𝑘−2 𝐺 𝑝𝑖,𝑘↔𝑝𝑖,𝑘−1

𝑃𝐴 𝑝𝑖,0 𝑃𝐴 𝑝𝑖,1 𝑃𝐴 𝑝𝑖,2 …𝑃𝐴(𝑝𝑖,𝑛)

� The PDFs are different in different methods.

Path Tracing

� Tracing rays from camera

� Whitted ray tracing stops if brdf is not a delta one, while path tracing doesn’t

PDF of sampling a path of n+1 vertices

� We’ll assume 𝑃 𝑝0 and 𝑃 𝑝1 are both 1 for simplicity. That said depth of field is not taken into account so far.

� Since we generate new vertex by sampling a new ray from current bxdf, we only have the PDF w.r.t solid instead of area.� Sampling pdf w.r.t area is too much inefficient !

� 𝑃𝐴 𝑝𝑘 =𝑐𝑜𝑠𝜃𝑘→𝑘−1

𝑟2𝑃𝑤 𝑝𝑘−1 → 𝑝𝑘 =

𝐺 𝑝𝑘↔𝑝𝑘−1 𝑃𝑤 𝑝𝑘−1→𝑝𝑘𝑐𝑜𝑠𝜃𝑘−1→𝑘

� Dropping this term in, we have the following equation:

�𝑇𝑛 𝑝1 → 𝑝0 =

1𝑁 𝑖=1𝑁

𝐿𝑒 𝑝𝑖,𝑛→𝑝𝑖,𝑛−1 𝑘=2𝑘=𝑛 𝑓 𝑝𝑖,𝑘→𝑝𝑖,𝑘−1→𝑝𝑖,𝑘−2 𝐺 𝑝𝑖,𝑘↔𝑝𝑖,𝑘−1

𝑃𝐴 𝑝𝑖,0 𝑃𝐴 𝑝𝑖,1 𝑃𝐴 𝑝𝑖,2 …𝑃𝐴 𝑝𝑖,𝑛

= 1𝑁 𝑖=1𝑁 𝐿𝑒 𝑝𝑖,𝑛 → 𝑝𝑖,𝑛−1 𝑘=2𝑘=𝑛

𝑓 𝑝𝑖,𝑘→𝑝𝑖,𝑘−1→𝑝𝑖,𝑘−2 𝑐𝑜𝑠𝜃𝑖,𝑘−1→𝑘𝑝𝑤(𝑝𝑖,𝑘−1→𝑝𝑖,𝑘)

Tricks

� 𝑇𝑛 𝑝1 → 𝑝0 can reuse the existing path of 𝑇𝑛−1 𝑝1 → 𝑝0 , then only one vertex is needed to be sampled to evaluate each T.

�𝑓 𝑝𝑖,𝑘→𝑝𝑖,𝑘−1→𝑝𝑖,𝑘−2 𝑐𝑜𝑠𝜃𝑖,𝑘−1→𝑘

𝑝𝑤(𝑝𝑖,𝑘−1→𝑝𝑖,𝑘)can be computed incrementally.

� Use Multiple Importance Sampling to sample both of light source and bsdf, instead of trying to hit light source by chance, or:� It won’t work for delta lights.

� Terrible convergence rate if light source is small

Path Tracing Conclusion

� Unbiased, easy to implement, Robust� Convergence rate may be low for certain

scenes, No Caustics

Instant Radiosity

� Distribute virtual point light in the first stage.

� Use those virtual point light to evaluate the radiance

PDF of a path in Instant Radiosity

� Again, we assume 𝑃 𝑝0 and 𝑃 𝑝1 are both 1.

� 𝑃𝐴 𝑝𝑘 =𝑐𝑜𝑠𝜃𝑘−1→𝑘

𝑟2𝑃𝑤 𝑝𝑘 → 𝑝𝑘−1 =

𝐺 𝑝𝑘↔𝑝𝑘−1 𝑃𝑤 𝑝𝑘→𝑝𝑘−1𝑐𝑜𝑠𝜃𝑘→𝑘−1

� The above equation only works for vertices from 2 to n-1, we already have the pdf w.r.t area of sampling vertex on light sources, since that’s exactly what we do.

� Again, dropping it in, we have:

�

𝑇𝑛 𝑝1 → 𝑝0 =1𝑁 𝑖=1𝑁

𝐿𝑒 𝑝𝑖,𝑛→𝑝𝑖,𝑛−1 𝑘=2𝑘=𝑛 𝑓 𝑝𝑖,𝑘→𝑝𝑖,𝑘−1→𝑝𝑖,𝑘−2 𝐺 𝑝𝑖,𝑘↔𝑝𝑖,𝑘−1

𝑃𝐴 𝑝𝑖,0 𝑃𝐴 𝑝𝑖,1 𝑃𝐴 𝑝𝑖,2 …𝑃𝐴(𝑝𝑖,𝑛)

= 1𝑁 𝑖=1𝑁 (

𝐿𝑒 𝑝𝑖,𝑛→𝑝𝑖,𝑛−1 𝑐𝑜𝑠𝜃𝑖,𝑛→𝑛−1𝑃𝑤 𝑝𝑖,𝑛→𝑝𝑖,𝑛−1 𝑃𝐴(𝑝𝑖,𝑛)

𝑘=4𝑘=𝑛𝑓 𝑝𝑖,𝑘→𝑝𝑖,𝑘−1→𝑝𝑖,𝑘−2 𝑐𝑜𝑠𝜃𝑖,𝑘−1→𝑘−2

𝑝𝑤(𝑝𝑖,𝑘−1→𝑝𝑖,𝑘−2)

𝑓 𝑝𝑖,2 → 𝑝𝑖,1 → 𝑝𝑖,0 𝐺 𝑝𝑖,2 ↔ 𝑝𝑖,3 𝑓 𝑝𝑖,3 → 𝑝𝑖,2 → 𝑝𝑖,1 )

LTE Evaluation

�

𝑇𝑛 𝑝1 → 𝑝0 =1𝑁 𝑖=1𝑁

𝐿𝑒 𝑝𝑖,𝑛→𝑝𝑖,𝑛−1 𝑘=2𝑘=𝑛 𝑓 𝑝𝑖,𝑘→𝑝𝑖,𝑘−1→𝑝𝑖,𝑘−2 𝐺 𝑝𝑖,𝑘↔𝑝𝑖,𝑘−1

𝑃𝐴 𝑝𝑖,0 𝑃𝐴 𝑝𝑖,1 𝑃𝐴 𝑝𝑖,2 …𝑃𝐴(𝑝𝑖,𝑛)

= 1𝑁 𝑖=1𝑁 (

𝐿𝑒 𝑝𝑖,𝑛→𝑝𝑖,𝑛−1 𝑐𝑜𝑠𝜃𝑖,𝑛→𝑛−1𝑃𝑤 𝑝𝑖,𝑛→𝑝𝑖,𝑛−1 𝑃𝐴(𝑝𝑖,𝑛)

𝑘=4𝑘=𝑛𝑓 𝑝𝑖,𝑘→𝑝𝑖,𝑘−1→𝑝𝑖,𝑘−2 𝑐𝑜𝑠𝜃𝑖,𝑘−1→𝑘−2

𝑝𝑤(𝑝𝑖,𝑘−1→𝑝𝑖,𝑘−2)

𝑓 𝑝𝑖,2 → 𝑝𝑖,1 → 𝑝𝑖,0 𝐺 𝑝𝑖,2 ↔ 𝑝𝑖,3 𝑓 𝑝𝑖,3 → 𝑝𝑖,2 → 𝑝𝑖,1 )

� The green part will be evaluated incrementally in the VPL distribution stage.

� The red part is done during per pixel radiance evaluation.

Quite different way of converging

Special Case Handling

� See the hotspot at corners, it is caused by the inverse distance in G term.

� Clamp the inverse distance to avoid it.� 𝐺 = min 𝐺, 𝐺𝑐𝑙𝑎𝑚𝑝 + max(𝐺 − 𝐺𝑐𝑙𝑎𝑚𝑝, 0)

� Evaluate the red part by using other methods, like path tracing.

Instant Radiosity Conclusion

� Unbiased, easy to implement

� Relatively slow convergence rate, almost no specular surface reflection

� Not very practical unless with much enhancement

Light Tracing

� Trace rays from light source

� Inefficient most of the time

The reverse of Path Tracing

� W is importance function.

�𝑊𝑜 𝑝,𝑤𝑜 = 𝑊𝑒 𝑝,𝑤𝑜 + ∫𝑊𝑖 𝑝,𝑤𝑖 𝑓 𝑝,𝑤𝑖, 𝑤𝑜 cos𝜃𝑖𝑑𝑤𝑖

More than a Pixel

� What is stored in a pixel, radiance?

�𝐼𝑗 = ∫𝐴0 ∫𝑊𝑒 𝑝0, 𝜔 𝐿𝑖 𝑝1,−𝜔 𝑐𝑜𝑠𝜃 𝑑𝑝0𝑑𝜔

= ∫∫𝑊𝑒 𝑝0 → 𝑝1 𝐿𝑖 𝑝1 → 𝑝0 𝐺 𝑝0 ↔ 𝑝1 𝑑𝑝0𝑑𝑝1

� Drop LTE in it, we have the equation for the radiance contribution of a specific length:

� 𝐼𝑗 = ∫…∫𝑊𝑒 𝑝0 → 𝑝1 𝐺 𝑝0 ↔ 𝑝1 𝐿𝑒 𝑝𝑛 → 𝑝𝑛−1 𝑘=2𝑘=𝑛 𝐺 𝑝𝑖,𝑘 ↔ 𝑝𝑖,𝑘−1* 𝑓 𝑝𝑖,𝑘 → 𝑝𝑖,𝑘−1 → 𝑝𝑖,𝑘−2 𝑑𝑝0𝑑𝑝1 …𝑑𝑝𝑛

PDF of sampling a path of n+1 vertices

� 𝑃𝐴 𝑝𝑘 =𝑐𝑜𝑠𝜃𝑘→𝑘+1

𝑟2𝑃𝑤 𝑝𝑘+1 → 𝑝𝑘 =

𝐺 𝑝𝑘↔𝑝𝑘+1 𝑃𝑤 𝑝𝑘+1→𝑝𝑘𝑐𝑜𝑠𝜃𝑘+1→𝑘

�

𝑇𝑛 𝑝1 → 𝑝0 =1𝑁 𝑖=1𝑁

𝑊𝑒 𝑝𝑖,0→𝑝𝑖,1 𝐺 𝑝𝑖,0↔𝑝𝑖,1 𝐿𝑒 𝑝𝑖,𝑛→𝑝𝑖,𝑛−1 𝑘=2𝑘=𝑛 𝑓 𝑝𝑖,𝑘→𝑝𝑖,𝑘−1→𝑝𝑖,𝑘−2 𝐺 𝑝𝑖,𝑘↔𝑝𝑖,𝑘−1

𝑃𝐴 𝑝𝑖,0 𝑃𝐴 𝑝𝑖,1 𝑃𝐴 𝑝𝑖,2 …𝑃𝐴 𝑝𝑖,𝑛

= 1𝑁 𝑖=1𝑁

𝐿𝑒 𝑝𝑖,𝑛→𝑝𝑖,𝑛−1 𝑐𝑜𝑠𝜃𝑖,𝑛→𝑛−1𝑃𝐴 𝑝𝑖,𝑛 𝑃𝑤 𝑝𝑖,𝑛→𝑝𝑖,𝑛−1 𝑃𝐴(𝑝𝑖,0)

𝐺 𝑝𝑖,0 ↔ 𝑝𝑖,1 𝑊𝑒 𝑝𝑖,0 → 𝑝𝑖,1

𝑓(𝑝𝑖,0 → 𝑝𝑖,1 → 𝑝𝑖,2) 𝑘=3𝑘=𝑛𝑓 𝑝𝑖,𝑘→𝑝𝑖,𝑘−1→𝑝𝑖,𝑘−2 𝑐𝑜𝑠𝜃𝑖,𝑘−1→𝑘−2

𝑝𝑤(𝑝𝑖,𝑘−1→𝑝𝑖,𝑘−2)

Light Tracing vs Path Tracing

Light Tracing Conclusion

� Easy to implement, good caustics

� Low convergence rate, especially for larger scene. Almost no specular surfaces.

Light Tracing Conclusion

� Good at rendering caustics for small scene

� Terrible convergence rate for outdoor scene

� Specular or highly glossy surfaces need special treatment

Bidirectional Path Tracing

� Shooting rays from both sides.

Different Cases in BDPT

All Cases in BDPT

� Path Tracing Case, no vertex sampled on light source.

� Light Tracing Case, only one vertex sampled on aperture.

� Direct Illumination Case, only one vertex sampled on light source

� Common Case, sub-path from each side contains at least two vertices

A Naïve BDPT Implementation

� Works terrible, delivers no value at all.� Almost wrong value for specular

surface

� Caustics are a little bit dimmer

� A lot of fireflies around corners

With MIS in BDPT

� Same amount of time

� Much better result

� Quite Robust

� Shows everything!!

Resources for further detail

� Reference Implementation:� https://github.com/JerryCao1985/SORT

� My Tech Blog:� https://agraphicsguy.wordpress.com/

Q&A