remote rendering of massively textured 3d scenes through...
TRANSCRIPT
SIAMES team VIIP2003
Remote Rendering of Massively Textured 3D Scenes Through
Progressive Texture Maps
Jean-Eudes MarvieINSA of Rennes
Pr Kadi BouatouchUniversity of Rennes
9/9/2003 IRISA / INRIA - INSA / University of Rennes 2
SIAMES team VIIP2003
Objective
• Real time navigation• Client-server architecture• Applied to complex 3D scenes
ServerDatabase 1Database 2Database 3
Client 1
WalkthroughModel 1
Client 2
WalkthroughModel 2
Client 3
WalkthroughModel 2
TCP/IP
9/9/2003 IRISA / INRIA - INSA / University of Rennes 3
SIAMES team VIIP2003
Constraints
• Complex 3D scenes :• Millions of polygons, objects,• Textures coordinates, materials, normal vectors,• Large set of texture maps
• Real-time navigation : (25fps)• Limited by graphics hardware performances,• By network bandwidth,• By the amount of data to process
9/9/2003 IRISA / INRIA - INSA / University of Rennes 4
SIAMES team VIIP2003
Used solutions
• Global optimizations• Spatial subdivision of the scenes (set of cells),• Visibility preprocessing between the cells,• Partial visibility graphs
• Local optimizations• Progressive geometry• Progressive texture maps• Automatic selection and adaptation
9/9/2003 IRISA / INRIA - INSA / University of Rennes 5
SIAMES team VIIP2003
Overview
I - Progressive Texture Maps (PTM)• Texture maps are specific images
II - Average Coverage Hint (ACH)• A pre-computed selection metric for mipmap levels
IV - Results and discussion• A virtual museum with high resolution texture maps
III - Automatic adaptation• Using PTM and ACH to perform real time remote navigation
9/9/2003 IRISA / INRIA - INSA / University of Rennes 6
SIAMES team VIIP2003
I - Progressive Texture Maps
PTMA compact file format to encode
texture mipmaps
9/9/2003 IRISA / INRIA - INSA / University of Rennes 7
SIAMES team VIIP2003
I - Progressive Texture Maps
• JPEG+ Hierarchical representation
⇒ Possibility to encode texture mipmaps
+ Lossless compression⇒ Average compression equal 2:1
– Data redundancy between levels⇒ Original size increased by 30%
9/9/2003 IRISA / INRIA - INSA / University of Rennes 8
SIAMES team VIIP2003
I - Progressive Texture Maps
• Computing the mipmap levels
∑≤≤
≤≤++
− =
1010
2.,2.1
, 41
ml
nmjli
nji II
=
25.025.025.025.0
FUsing low pass filter :
Color component for lower level :
00,0I
10,0I 1
0,1I1
1,0I 11,1I
20,0I 2
1,0I20,1I 2
1,1I
22,0I 2
3,0I2
2,1I 23,1I
20,2I 2
1,2I2
0,3I 21,3I
22,2I 2
3,2I2
2,3I 23,3I
F
F
0
1
2
(1)
9/9/2003 IRISA / INRIA - INSA / University of Rennes 9
SIAMES team VIIP2003
I - Progressive Texture Maps
• Mipmap level reconstruction
00,0I
Level 0
Server side Client side
⇒ Mipmap levels memory size = original texture map size !!
10,0I 1
0,1I1
1,0I 11,1I
Full level 1
nji
nji
nji
nji
nji IIIII 12.,2.2.,12.2.,2.
1,12.,12. .4 ++−
++ −−−=(2)
(2)Network transmission and
10,0I 1
0,1I1
1,0I
Partial level 1
10,0I 1
0,1I1
1,0I
Partial level 1
11,1I
9/9/2003 IRISA / INRIA - INSA / University of Rennes 10
SIAMES team VIIP2003
I - Progressive Texture Maps
• Integer solution
⇒= ∑≤≤
≤≤++
−
1010
2.,2.1
, 41
ml
nmjli
nji II(1)
∈
43
,21
,41
,0remainder
With { }3,2,1,01, ∈−njiε , stored using only 2 bits.
⇒ Equation (2) becomes :
nji
nji
nji
nji
nji
nji IIIII 12.,2.2.,12.2.,2.
1,
1,12.,12. .4 ++
−−++ −−−+= ε(3)
9/9/2003 IRISA / INRIA - INSA / University of Rennes 11
SIAMES team VIIP2003
I - Progressive Texture Maps
• PTM+ Hierarchical representation
⇒ Possibility to encode texture mipmaps
+ Losseless compression⇒ Average compression equal 2:1
+ No data redundancy between levels+ Original size increased only by 6%
9/9/2003 IRISA / INRIA - INSA / University of Rennes 12
SIAMES team VIIP2003
II - Average Coverage Hints
ACHA metric to select the mipmap levels to
download
9/9/2003 IRISA / INRIA - INSA / University of Rennes 13
SIAMES team VIIP2003
II - Average Coverage Hints
[Teler et al.] • Bounding box projection area
+ Medium computation cost– Occluded objects are treated as visible
[Dumont et al.] • Frame buffer analysis (two pass)
+ Precise, occluded objects are rejected– Buffer analysis is very costly
9/9/2003 IRISA / INRIA - INSA / University of Rennes 14
SIAMES team VIIP2003
II - Average Coverage Hints
• Off-line : per-cell ACH computationcell
cell
cell
OpenGLcamera
Rendered frame example
9/9/2003 IRISA / INRIA - INSA / University of Rennes 15
SIAMES team VIIP2003
II - Average Coverage Hints
• Off-line : per-cell ACH computation• Height camera positions per cell,• Six directions for each position.
1. Pixel count for each color (cell),2. Normalize values using total pixel number,3. Get a percentage of coverage for each
cell,4. Store the result in the database.
9/9/2003 IRISA / INRIA - INSA / University of Rennes 16
SIAMES team VIIP2003
II - Average Coverage Hints
• On-line : ACH renormalization
Current cell
0.5
0.25
0.25
0.0
0.66 0.33
ACH of visible cells are renormalized
9/9/2003 IRISA / INRIA - INSA / University of Rennes 17
SIAMES team VIIP2003
III - Automatic adaptation
• According to– Network bandwidth – Client machine power– User defined parameters
• Using PTMs and ACHs to provide– Highest visual quality – Maximum interactivity
9/9/2003 IRISA / INRIA - INSA / University of Rennes 18
SIAMES team VIIP2003
III - Automatic adaptation
• User defined parameters :– Amount of graphics hardware memory
• Available for all the texture maps– When viewpoint is moving :– When viewpoint is static more than n seconds :
– Time limit to download a mipmap level• When viewpoint is moving :• When viewpoint is static more than n seconds : sta
t∆dynt∆
M mem
sta
M mem
dyn
9/9/2003 IRISA / INRIA - INSA / University of Rennes 19
SIAMES team VIIP2003
III - Automatic adaptation
• Memory budget allocation
0.66 0.33
MMmem×= 66.0
0
Share among objects
MMMrestmem
0133.0 +×=
Visible cells sorted according to their ACH value
30M
M rest
0
restrest
30M
30M
rest
Share among objects
20M
20M
rest rest M rest
1
9/9/2003 IRISA / INRIA - INSA / University of Rennes 20
SIAMES team VIIP2003
III - Automatic adaptation• Memory budget usage
L = highest mipmap level into graphics hardwareLh = highest mipmap level already downloadedLmax = highest mipmap level availableM = memory budget allocated to the textureS(L) = memory size of mipmap pyramid for level L
S(L+1) ≤ ML+1 ≤ Lmax
L+1 > Lh
yesAdd L+1
to hardware
S(L) > Mno
Not downloadingS(L+1) ≤ ML+1 ≠ Lmax
L+1 > Lh
no
Remove Lfrom hardware
yes
Download L+1from server if download
time is ≤ allowed time
yes
9/9/2003 IRISA / INRIA - INSA / University of Rennes 21
SIAMES team VIIP2003
IV - Results
• Video– Model with 90MB of texture maps,– Pentium IV 1.7GHz,– Nvidia Quadro 2 Pro 64MB– User parameters :
secdyn
t5=∆
MBM mem
sta12=
MBM mem
dyn4=
secdyn
t20=∆
secn 5=
9/9/2003 IRISA / INRIA - INSA / University of Rennes 22
SIAMES team VIIP2003
9/9/2003 IRISA / INRIA - INSA / University of Rennes 23
SIAMES team VIIP2003
Conclusion
• PTMs dedicated to texture mipmaps– Better loss less compression than JPEG– Low CPU cost for the decompression
• ACHs dedicated to conservative visibility– Preprocessed, low cost storage– Low CPU cost on the client side
• Automatic adaptation– Network and Client machine
9/9/2003 IRISA / INRIA - INSA / University of Rennes 24
SIAMES team VIIP2003
Questions ?