realistic reflections and refractions on graphics hardware with hybrid rendering and layered...
Post on 21-Dec-2015
224 views
TRANSCRIPT
Realistic Reflections and Refractionson Graphics Hardware with Hybrid Rendering
and Layered Environment Maps
Realistic Reflections and Refractionson Graphics Hardware with Hybrid Rendering
and Layered Environment Maps
Ziyad S. Hakura
John M. Snyder
Ziyad S. Hakura
John M. Snyder
Goal: Real-time rendering of Parameterized Image Spacesincluding Photorealistic, Reflective/Refractive ObjectsGoal: Real-time rendering of Parameterized Image Spacesincluding Photorealistic, Reflective/Refractive Objects
Viewpoint PositionViewpoint Position
ObjectMotionObjectMotion
Parameterized Image SpacesParameterized Image Spaces
• Space can be 1D, 2D, 3D or more
• Content author specifies parameters
• Space can be 1D, 2D, 3D or more
• Content author specifies parameters
Light motionLight motion
Viewpoint positionViewpoint position
Object motionObject motion
Cockpit LightingCockpit Lighting
Day lightDay light
Night skyNight sky
viewpoint x time of dayviewpoint x time of day
Interactive Toy Story
Interactive Toy Story
• Limited viewpoint motion Head motion parallax puts the user “in the scene”
• Character parameters e.g. happiness/sadness Rose98, Gleicher98, Popović99
• Limited viewpoint motion Head motion parallax puts the user “in the scene”
• Character parameters e.g. happiness/sadness Rose98, Gleicher98, Popović99
viewpoint x time x emotional stateviewpoint x time x emotional state
Overall ModelOverall Model
Fit EnvironmentalImpostors to Match
Hybrid Rendering
Render Ray-TracedImages Offline
PR
EP
RO
CE
SS
RU
N-T
IME
Overall ModelOverall Model
Fit EnvironmentalImpostors to Match
Hybrid Rendering
Render Ray-TracedImages Offline
PR
EP
RO
CE
SS
RU
N-T
IME
Overall ModelOverall Model
Fit EnvironmentalImpostors to Match
Hybrid Rendering
Render Ray-TracedImages Offline
PR
EP
RO
CE
SS
RU
N-T
IME
Overall ModelOverall Model
Fit EnvironmentalImpostors to Match
Hybrid Rendering
Render Ray-TracedImages Offline
PR
EP
RO
CE
SS
RU
N-T
IME
Ray Tracing vs. Hybrid RenderingRay Tracing vs. Hybrid Rendering
Ray TracingRay Tracing Hybrid RenderingHybrid Rendering
local lensobject
environmental impostor
local lensobject
environmental impostor
Related WorkRelated Work
• IBR Gortler96, Levoy96, Miller98, Wood00
• Reflections Diefenbach96, Ofek98, Cabral99, Lischinski98, Bastos99
• Refractions Heidrich99, Zongker99, Chuang00
• Ray Tracing Kajiya86, Pharr97
• IBR Gortler96, Levoy96, Miller98, Wood00
• Reflections Diefenbach96, Ofek98, Cabral99, Lischinski98, Bastos99
• Refractions Heidrich99, Zongker99, Chuang00
• Ray Tracing Kajiya86, Pharr97
Our Related WorkOur Related Work
• Parameterized Texture Maps Hakura00 “pasted on” look away from pre-rendered views
• Parameterized Environment Maps Hakura01 view samples must be close together no refractions
• Parameterized Texture Maps Hakura00 “pasted on” look away from pre-rendered views
• Parameterized Environment Maps Hakura01 view samples must be close together no refractions
OutlineOutline
• Greedy Ray Path Shading Model
• Fitting Environment Maps
• Hybrid Rendering Runtime
• Results and Conclusions
• Greedy Ray Path Shading Model
• Fitting Environment Maps
• Hybrid Rendering Runtime
• Results and Conclusions
Greedy Ray Path Shading ModelGreedy Ray Path Shading Model
NN
Reflective Path
Reflective Path
RefractivePath
RefractivePath
Trace two ray paths until rays exit object.Trace two ray paths until rays exit object.
Refractive Lens ObjectRefractive Lens Object
Greedy Ray Path Shading ModelGreedy Ray Path Shading Model
NN
Reflective Path
Reflective Path
RefractivePath
RefractivePath
Trace two ray paths until rays exit object.Propagate child ray of greatest Fresnel coefficient.Trace two ray paths until rays exit object.Propagate child ray of greatest Fresnel coefficient.
Refractive Lens ObjectRefractive Lens Object
Comparison of Shading ModelsComparison of Shading Models
Full binary ray treeFull binary ray tree Two-term greedyray path
Two-term greedyray path
OutlineOutline
• Greedy Ray Path Shading Model
• Fitting Environment Maps
• Hybrid Rendering Runtime
• Results and Conclusions
• Greedy Ray Path Shading Model
• Fitting Environment Maps
• Hybrid Rendering Runtime
• Results and Conclusions
Impostor Fitting AlgorithmImpostor Fitting Algorithm
For each point in parameter space
For each local lens object
• generate outgoing rays in ray tracer
• record intersections with environment
• cluster intersections into layers
• fit textured impostor to each layer
For each point in parameter space
For each local lens object
• generate outgoing rays in ray tracer
• record intersections with environment
• cluster intersections into layers
• fit textured impostor to each layer
Layered Environment Maps (EMs)Layered Environment Maps (EMs)
local lensobject L1
L2
L3environmental layers
EM Geometry TypesEM Geometry Types
• Sphere at infinity• Sphere• Box• Ellipsoid• Cylinder• Quadrilateral
• Sphere at infinity• Sphere• Box• Ellipsoid• Cylinder• Quadrilateral
Linear Hardware ModelLinear Hardware Model
A x = bA x = bUnknown
Texture PixelsUnknown
Texture PixelsRay-Traced
ImageRay-Traced
ImageHW Filter
CoefficientsHW Filter
Coefficients
HardwareRender
HardwareRender
ScreenScreenTextureTexture
Inferred EMsInferred EMs
L1
cup
L1
cup
L2
cols
L2
cols
L3
walls
L3
walls
ReflectionTerm
ReflectionTerm
RefractionTerm
RefractionTerm Inferred Environment MapsInferred Environment Maps
OutlineOutline
• Greedy Ray Path Shading Model
• Fitting Environment Maps
• Hybrid Rendering Runtime
• Results and Conclusions
• Greedy Ray Path Shading Model
• Fitting Environment Maps
• Hybrid Rendering Runtime
• Results and Conclusions
Adaptive TessellationAdaptive Tessellation• Two criteria:
ray path “topology” outgoing ray distance
• Consider both terms of shading model
• Two criteria:
ray path “topology” outgoing ray distance
• Consider both terms of shading model
Texture Coordinate GenerationTexture Coordinate Generation
OutgoingRay
EM TextureMapping
EM Texture
(u,v)
EM Geometry Layers
Overlaying/Blending PassesOverlaying/Blending Passes
LR = L1
(uR1,vR1) over L2 (uR2,vR2) over L3
(uR3,vR3)
LT = L1 (uT1,vT1) over L2
(uT2,vR2) over L3 (uT3,vT3)
LR = L1
(uR1,vR1) over L2 (uR2,vR2) over L3
(uR3,vR3)
LT = L1 (uT1,vT1) over L2
(uT2,vR2) over L3 (uT3,vT3)
reflection termreflection term
refraction termrefraction term
LR = L1
(uR1,vR1) over L2 (uR2,vR2) over L3
(uR3,vR3)
LT = L1 (uT1,vT1) over L2
(uT2,vR2) over L3 (uT3,vT3)
Single viewpoint (i): Li = LRi FRGR + LT
i FTGT
Blended viewpoint: L = Li + Li+1 (1-)
Requires 4n passes, where n is number of layers
LR = L1
(uR1,vR1) over L2 (uR2,vR2) over L3
(uR3,vR3)
LT = L1 (uT1,vT1) over L2
(uT2,vR2) over L3 (uT3,vT3)
Single viewpoint (i): Li = LRi FRGR + LT
i FTGT
Blended viewpoint: L = Li + Li+1 (1-)
Requires 4n passes, where n is number of layers
reflection termreflection term
refraction termrefraction term
Overlaying/Blending PassesOverlaying/Blending Passes
OutlineOutline
• Greedy Ray Path Shading Model
• Fitting Environment Maps
• Hybrid Rendering Runtime
• Results and Conclusions
• Greedy Ray Path Shading Model
• Fitting Environment Maps
• Hybrid Rendering Runtime
• Results and Conclusions
• 1D viewspace circling teapot
• 8º separation between view samples
• simultaneous solution over 5 viewpoints
• 3 layers for teapot, 2 for cup
• 1D viewspace circling teapot
• 8º separation between view samples
• simultaneous solution over 5 viewpoints
• 3 layers for teapot, 2 for cup
Experimental SetupExperimental Setup
Ray-Traced vs. HybridRay-Traced vs. Hybrid
Ray-Traced480 sec/frameRay-Traced
480 sec/frameHybrid Rendered
19 sec/frameHybrid Rendered
19 sec/frame
Benefit of Hybrid Renderingover Ray-Tracing
Benefit of Hybrid Renderingover Ray-Tracing
• Lower cost Adaptive ray-tracing algorithm
• Lower cost and higher predictability Greedy two-term shading model Environment substitution with layered shells
• Lower cost Adaptive ray-tracing algorithm
• Lower cost and higher predictability Greedy two-term shading model Environment substitution with layered shells
Future WorkFuture Work
• Compression with hybrid rendering
• Handling glossy objects
• Matching full binary tree renderings
• More efficient pre-rendering Multi-dimensional Ray-Tracing
• Accelerating ray traced animations
• Making it faster (local ray tracing hardware?)
• Compression with hybrid rendering
• Handling glossy objects
• Matching full binary tree renderings
• More efficient pre-rendering Multi-dimensional Ray-Tracing
• Accelerating ray traced animations
• Making it faster (local ray tracing hardware?)
Overall ModelOverall Model
Fit EnvironmentalImpostors to Match
Hybrid Rendering
Render Ray-TracedImages Offline
PR
EP
RO
CE
SS
RU
N-T
IME
Greedy Ray Path Shading ModelGreedy Ray Path Shading Model
==
resultresult
reflection termreflection termrefraction termrefraction term
** **++
T T FT GT FT GT RR FR GR FR GR
Surface Light Fields [Miller98,Wood00] Surface Light Fields [Miller98,Wood00]
Surface Light Field Surface Light Field
Dense sampling over surface points of
low-resolution lumispheres
Dense sampling over surface points of
low-resolution lumispheres
PEMPEM
Sparse sampling over viewpoints of
high-resolution EMs
Sparse sampling over viewpoints of
high-resolution EMs
Why Fit Impostors?Why Fit Impostors?
local lens object
outgoing ray
environmentalobject
impostor
intersection
impostor center of projection
local lens object
outgoing ray
environmentalobject
impostor
intersection
impostor center of projection
Why Fit Impostors?Why Fit Impostors?
• geometric error of impostor• geometric error of impostor
local lens object
outgoing ray
environmentalobject
impostor
intersection
impostor center of projection
local lens object
outgoing ray
environmentalobject
impostor
intersection
impostor center of projection
Fitting accounts for: Fitting accounts for:
Why Fit Impostors?Why Fit Impostors?
• geometric error of impostor• view-dependent shading in environment
• geometric error of impostor• view-dependent shading in environment
local lens object
outgoing ray
environmentalobject
impostor
intersection
impostor center of projection
local lens object
outgoing ray
environmentalobject
impostor
intersection
impostor center of projection
Fitting accounts for: Fitting accounts for:
Why Fit Impostors?Why Fit Impostors?
• geometric error of impostor• view-dependent shading in environment
• geometric error of impostor• view-dependent shading in environment
local lens object
outgoing ray
environmentalobject
impostor
intersection
impostor center of projection
local lens object
outgoing ray
environmentalobject
impostor
intersection
impostor center of projection
Fitting accounts for: Fitting accounts for:
• simultaneous inference over multiple views nearby views of lens object
“direct” views (without lens object)
solve confidence-weighted least squares
• simultaneous inference over multiple views nearby views of lens object
“direct” views (without lens object)
solve confidence-weighted least squares
• ray propagation• ray propagation
incomingincoming outgoingoutgoing
Preventing Disocclusion ArtifactsPreventing Disocclusion Artifacts