Visibility Culling III:Visibility Culling III:Image-Space OcclusionImage-Space Occlusion

David LuebkeDavid Luebke

Computer Science Computer Science DepartmentDepartment

University of VirginiaUniversity of Virginia

<luebke@cs.virginia.edu><luebke@cs.virginia.edu>

D2

Review: Review: General Occlusion General Occlusion

CullingCulling• When cells and portals don’t When cells and portals don’t

work…work…– Trees in a forestTrees in a forest– A crowded train stationA crowded train station

• Need general Need general occlusion cullingocclusion culling algorithms:algorithms:– Aggregate occlusion Aggregate occlusion – Dynamic scenesDynamic scenes– Non-polygonal scenesNon-polygonal scenes

D3

Review: Review: Hierarchical Z-BufferHierarchical Z-Buffer

• A A Z-pyramidZ-pyramid represents depth represents depth buffer in hierarchical fashionbuffer in hierarchical fashion– Depth at each pixel at a given level Depth at each pixel at a given level

represents the max of the depth of represents the max of the depth of the four corresponding pixels at the four corresponding pixels at next levelnext level

– Rasterize triangles into buffer Rasterize triangles into buffer hierarchically with early terminationhierarchically with early termination

D4

Review:Review:Hierarchical Z-BufferHierarchical Z-Buffer

• Organize scene into an Organize scene into an octreeoctree (a kind (a kind of spatial hierarchy)of spatial hierarchy)– Before rendering the polygons Before rendering the polygons

associated with a node in the octree, associated with a node in the octree, test if the node’s bbox is visibletest if the node’s bbox is visible

– Do an Do an occlusion queryocclusion query to “test render” to “test render” the faces of the bounding box against the faces of the bounding box against the depth-bufferthe depth-buffer

– If query fails, bbox would not be visible If query fails, bbox would not be visible and we can skip the node’s geometryand we can skip the node’s geometry

D5

Review: Review: Hierarchical Z-BufferHierarchical Z-Buffer

• HZB algorithm is most effective when HZB algorithm is most effective when Z-pyramid is “mostly complete”Z-pyramid is “mostly complete”– Start by rendering nodes visible last frame Start by rendering nodes visible last frame

directly into full-res Z-bufferdirectly into full-res Z-buffer– Build Z-pyramid from full-res Z-bufferBuild Z-pyramid from full-res Z-buffer– Traverse octree front-to-back, using Traverse octree front-to-back, using

Z-query to test nodes and render if Z-query to test nodes and render if neededneeded

• Note: also test nodes already rendered to see Note: also test nodes already rendered to see if they have become invisibleif they have become invisible

D6

Review: Review: Hierarchical Z-BufferHierarchical Z-Buffer

• HZB algorithm exploits:HZB algorithm exploits:– Image-space coherenceImage-space coherence: :

• Polygons occluded in one pixel are Polygons occluded in one pixel are probably occluded in nearby pixelsprobably occluded in nearby pixels

– Object-space coherenceObject-space coherence::• Polygons near an occluded polygon are Polygons near an occluded polygon are

probably also occludedprobably also occluded

– Temporal coherenceTemporal coherence::• Polygons visible recently are probably Polygons visible recently are probably

still visiblestill visible

D7

Hierarchical Hierarchical Occlusion MapsOcclusion Maps

• A more hardware-friendly general A more hardware-friendly general occlusion culling algorithmocclusion culling algorithm

• Two major differences from HZB:Two major differences from HZB:– Separates Separates occludersoccluders from from occludeesoccludees– Decouples occlusion test into an Decouples occlusion test into an

depth testdepth test and a and a overlap testoverlap test

D8

HierarchicalHierarchicalOcclusion MapsOcclusion Maps

• Occluders versus occludees:Occluders versus occludees:

Blue parts: Blue parts: occludersoccluders Red parts: Red parts: occludeesoccludees

D9

Hierarchical Hierarchical Occlusion MapsOcclusion Maps

• Depth versus overlap:Depth versus overlap:

ViewPoint

ZX

Y

Depth + Overlap = Depth + Overlap = OcclusionOcclusion Depth + Overlap = Depth + Overlap = OcclusionOcclusion

D10

Hierarchical Hierarchical Occlusion MapsOcclusion Maps

• Representation of projection for Representation of projection for overlap test: overlap test: occlusion mapocclusion map– Corresponds to a screen subdivisionCorresponds to a screen subdivision– Records average opacity per partitionRecords average opacity per partition

• Generate by rendering occludersGenerate by rendering occluders– Record pixel opacities (i.e., coverage)Record pixel opacities (i.e., coverage)

D11

Rendered Image Occlusion Map

Occlusion MapsOcclusion Maps

D12

Occlusion Map Occlusion Map PyramidPyramid

• Analyzing cumulative projection:Analyzing cumulative projection:– A A hierarchical occlusion maphierarchical occlusion map (HOM) (HOM)– Generate by recursive averaging Generate by recursive averaging

(once per frame)(once per frame)– Records average opacities for blocks Records average opacities for blocks

of multiple pixels, representing of multiple pixels, representing occlusion at multiple resolutionsocclusion at multiple resolutions

– Construction can be accelerated by Construction can be accelerated by texture hardwaretexture hardware

D13

Occlusion Map Occlusion Map PyramidPyramid

64 x 64 32 x 32 16 x 16

D14

Occlusion Map Occlusion Map PyramidPyramid

D15

Overlap TestsOverlap Tests

• Query: is projection of occludee inside Query: is projection of occludee inside cumulative projection of occluders?cumulative projection of occluders?– Cumulative projection: occlusion pyramidCumulative projection: occlusion pyramid– Occludee projection: expensive in generalOccludee projection: expensive in general

• Overestimate occludee with 3-D bounding boxOverestimate occludee with 3-D bounding box• Overestimate projection of 3-D bounding box Overestimate projection of 3-D bounding box

with 2-D bounding rectangle in screen-space with 2-D bounding rectangle in screen-space

D16

Overlap TestsOverlap Tests

• Hierarchical structure enables Hierarchical structure enables some optimizations:some optimizations:– Predictive rejectionPredictive rejection

• Terminate test when it must fail laterTerminate test when it must fail later

– Conservative rejectionConservative rejection• The transparency thresholdThe transparency threshold

– Aggressive Approximate CullingAggressive Approximate Culling• Ignore objects barely visible through Ignore objects barely visible through

holesholes• The opacity thresholdThe opacity threshold

D17

Aggressive Aggressive Approximate CullingApproximate Culling

0 1 2 3 4

D18

Hierarchical Hierarchical Occlusion MapsOcclusion Maps

• Not discussed here:Not discussed here:– Depth testDepth test

• Depth estimation bufferDepth estimation buffer• Modified Z-bufferModified Z-buffer

– Selecting occludersSelecting occluders

• For more details, see Hansong For more details, see Hansong Zhang’s dissertation at UNCZhang’s dissertation at UNC

D19

HOM: DiscussionHOM: Discussion

• Provides a robust, general, Provides a robust, general, hardware-friendly occlusion hardware-friendly occlusion culling algorithmculling algorithm– Supports dynamic scenesSupports dynamic scenes– Supports non-polygonal geometrySupports non-polygonal geometry– Few hardware assumptionsFew hardware assumptions

D20

HOM: DiscussionHOM: Discussion

• Not clear how relevant the actual Not clear how relevant the actual algorithm is on modern hardwarealgorithm is on modern hardware

• Most interesting/important points:Most interesting/important points:– Separation of occluders and occludeesSeparation of occluders and occludees– Factoring occlusion into overlap + depthFactoring occlusion into overlap + depth– Aggressive approximate cullingAggressive approximate culling

• To think about:To think about:– How could we simplify the occluders to How could we simplify the occluders to

accelerate occlusion testing?accelerate occlusion testing?

D21

Current Current Hardware SupportHardware Support

• Recent hardware supports Recent hardware supports Z-query operationZ-query operation– Allows systems to exploit:Allows systems to exploit:

• Object-space coherence (bounding boxes) Object-space coherence (bounding boxes) • Temporal coherence (last-rendered list)Temporal coherence (last-rendered list)

– Examples in OpenGL:Examples in OpenGL:• HP_OCCLUSION_QUERYHP_OCCLUSION_QUERY• NV_OCCLUSION_QUERYNV_OCCLUSION_QUERY• Go to NVIDIA occlusion presentation…Go to NVIDIA occlusion presentation…

– An aside: applies to cell-portal culling!An aside: applies to cell-portal culling!

D22

Visibility Culling: Visibility Culling: DiscussionDiscussion

• When is visibility culling When is visibility culling worthwhile?worthwhile?– When scene has high When scene has high depth depth

complexitycomplexity• Examples: architectural walkthroughs, Examples: architectural walkthroughs,

complex CAD assemblies, dense forest complex CAD assemblies, dense forest • Non-examples: terrain (usually), single Non-examples: terrain (usually), single

highly-tessellated object (e.g., bunny, highly-tessellated object (e.g., bunny, a radiositized room)a radiositized room)

D23

Visibility Culling:Visibility Culling:DiscussionDiscussion

• How does visibility culling compare to:How does visibility culling compare to:– Level-of-detail: Level-of-detail:

• Reduces geometry processingReduces geometry processing• Helps transform-bound appsHelps transform-bound apps

– Visibility culling: Visibility culling: • Reduces geometry Reduces geometry andand pixel processing pixel processing • Helps transform- and fill rate-bound appsHelps transform- and fill rate-bound apps

– Texture / Image representations: Texture / Image representations: • Reduces geometry Reduces geometry andand pixel processing pixel processing • Incurs texture/image processing costsIncurs texture/image processing costs

D24

Visibility Culling: Visibility Culling: DiscussionDiscussion

• How does visibility culling How does visibility culling interact with level of detail?interact with level of detail? – Fairly seamless integration; Fairly seamless integration;

generally a wingenerally a win– One issue: visibility of simplified One issue: visibility of simplified

model may differ from original model may differ from original model; requires some caremodel; requires some care

– LODs can speed up occluder LODs can speed up occluder selection and renderingselection and rendering

D25

Visibility Culling: Visibility Culling: DiscussionDiscussion

• How does visibility culling interact How does visibility culling interact with texture and image-based with texture and image-based representations?representations? – Texture/image reps generally replace Texture/image reps generally replace

far-field geometryfar-field geometry• Involves an implicit occlusion culling stepInvolves an implicit occlusion culling step• Reduces scene depth complexity, decreasing Reduces scene depth complexity, decreasing

the utility of visibility cullingthe utility of visibility culling• If near-field geometry still includes complex If near-field geometry still includes complex

heavily-occlusive assemblies, still a winheavily-occlusive assemblies, still a win

D26

Visibility Culling:Visibility Culling:DiscussionDiscussion

• How much culling effort is appropriate?How much culling effort is appropriate?– Cells and portals: relatively cheap, with large Cells and portals: relatively cheap, with large

potential speedupspotential speedups– Hierarchical occlusion maps: relatively costly, Hierarchical occlusion maps: relatively costly,

carefully weigh potential gainscarefully weigh potential gains– Multiple processors allow much more Multiple processors allow much more

aggressive culling calculationaggressive culling calculation• Pipelining culling calculations, Performer-style, Pipelining culling calculations, Performer-style,

allows allows cull time = render timecull time = render time

• Tradeoff: one frame increased latencyTradeoff: one frame increased latency

D27

SummarySummary

• The basic, very powerful idea:The basic, very powerful idea:– Rapidly compute a Rapidly compute a potentially visible potentially visible

setset– Let hardware handle the restLet hardware handle the rest

• For many scenes, visibility culling is For many scenes, visibility culling is a simple way to get huge speedupsa simple way to get huge speedups– View-frustum culling always a View-frustum culling always a mustmust – For scenes with high depth complexity, For scenes with high depth complexity,

occlusion culling can be a occlusion culling can be a bigbig win win

D28

SummarySummary

• Architectural models: visibility is Architectural models: visibility is practically a solved problempractically a solved problem– Cells and portals work wellCells and portals work well

• Cull-box portal culling: simple, fastCull-box portal culling: simple, fast• Line-stabbing: elegant, powerfulLine-stabbing: elegant, powerful

D29

SummarySummary

• Occlusion culling of general Occlusion culling of general models: still a largely open models: still a largely open problemproblem– Important issues: Important issues:

• Dynamic scenesDynamic scenes• Aggregate occlusion effects Aggregate occlusion effects

– Image-based approaches seem Image-based approaches seem most promising at this timemost promising at this time

D30

SummarySummary

• General occlusion culling algorithms:General occlusion culling algorithms:– Hierarchical Z-buffer: Hierarchical Z-buffer:

• A simple, truly elegant algorithmA simple, truly elegant algorithm• But doesn’t seem amenable to hardwareBut doesn’t seem amenable to hardware

– Hierarchical occlusion maps: Hierarchical occlusion maps: • Separates occluders from occludees, overlap from Separates occluders from occludees, overlap from

depthdepth• Lends itself well to aggressive cullingLends itself well to aggressive culling• Fairly high overhead, only worthwhile with high Fairly high overhead, only worthwhile with high

depth complexitydepth complexity

– Recent hardware supports (asynchronous) Recent hardware supports (asynchronous) occlusion queryocclusion query