aditya mavlankar, pierpaolo baccichet, d avid varodayan and bernd girod
DESCRIPTION
Optimal Slice Size for Streaming Regions of High-Resolution Video with Virtual Pan/Tilt/Zoom Functionality. Aditya Mavlankar, Pierpaolo Baccichet, D avid Varodayan and Bernd Girod Information Systems Laboratory Stanford University. TexPoint fonts used in EMF. - PowerPoint PPT PresentationTRANSCRIPT
Optimal Slice Size for Streaming Regions of Optimal Slice Size for Streaming Regions of High-Resolution VideoHigh-Resolution Video with with
Virtual Pan/Tilt/Zoom FunctionalityVirtual Pan/Tilt/Zoom Functionality
Optimal Slice Size for Streaming Regions of Optimal Slice Size for Streaming Regions of High-Resolution VideoHigh-Resolution Video with with
Virtual Pan/Tilt/Zoom FunctionalityVirtual Pan/Tilt/Zoom Functionality
Aditya Mavlankar, Pierpaolo Baccichet, Aditya Mavlankar, Pierpaolo Baccichet, DDavid Varodayan and Bernd Girodavid Varodayan and Bernd Girod
Information Systems LaboratoryInformation Systems Laboratory
Stanford UniversityStanford University
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 2
Outline
High-resolution video streaming with IROI Proposed coding scheme for IROI video streaming Analysis of optimal slice size selection Experimental results
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 3
High-Resolution Video Streaming with IROI
Related work– Interactive image browsing with JPEG-2000 [Taubman et al. 2003]– Interactive streaming of lightfields [Ramanathan et al. 2004]– Interactive streaming of panoramic videos [Heymann et al. 2005]– ...
Sources of high-resolution videos– High-resolution digital imaging sensors (CMOS technology)– High-resolution videos stitched from multiple cameras
Application scenarios– Surveillance– Instructional videos– Snow cams in ski resorts– Interactive TV with virtual pan/tilt/zoom– ...
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 4
Demo
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 5
H.264/AVC Based Coding Scheme
-
-
↑
↑
...
g
2(N ¡ 1)g...
...
Hierarchical B pictures
P slices
ROI
ROI
Overview video
Resolution layer 1
Resolution layer N
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 6
Tradeoff due to Slice Size
Small slice size Entire scene takes more bits to encode
– Slice headers– Lack of context continuation across slices for context adaptive coding– Cannot exploit inter-pixel correlation across slices
Less pixel overhead: Can adapt to ROI due to fine granularity of slice grid
=)
Pixel Overhead
ROI
sw
sh
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 7
Tradeoff Observed for Pedestrian Area, layer 2
160x160 128x128 64x64 32x32
Slice size in pixels [ ]sw £ sh
1
1.5
2
2.5N
umbe
r of
pix
els
tran
smitt
ed p
er r
ende
red
pixe
l
0.2
0.3
0.4
0.5
Bit
per
pixe
l for
cod
ing
give
n la
yer
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 8
Tradeoff Observed for Pedestrian Area, layer 2
160x160 128x128 64x64 32x32
Slice size in pixels [ ]sw £ sh
0.4
0.45
0.5
0.55
0.6
Bits
tran
smitt
ed p
er r
ende
red
pixe
l
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 9
Tradeoff Observed for Pedestrian Area, layer 3
160x160 128x128 64x64 32x32
Slice size in pixels [ ]sw £ sh
1
1.5
2
2.5N
umbe
r of
pix
els
tran
smitt
ed p
er r
ende
red
pixe
l
0.1
0.2
0.3
0.4
Bit
per
pixe
l for
cod
ing
give
n la
yer
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 10
Tradeoff Observed for Pedestrian Area, layer 3
160x160 128x128 64x64 32x32
Slice size in pixels [ ]sw £ sh
0.28
0.3
0.32
0.34
0.36
0.38
0.4
Bits
tran
smitt
ed p
er r
ende
red
pixe
l
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 11
# pixels transmitted (random variable)
Pixel Overhead Analysis in 1-D
¡ 1 0 1 2
segment index
¢¢¢ ¢¢¢
Imagine an infinitely long line of pixels. In this example,s = 4; d = 3SOISOISOISOI
PP = 2£ sPP = 1£ s
E f P g= d+ s ¡ 1
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 12
Pixel Overhead Analysis in 2-D
ROI
E f P g = (dw + sw ¡ 1)(dh + sh ¡ 1)
Expected number of pixels transmitted
E f Pwg = (dw + sw ¡ 1)
E f Phg = (dh + sh ¡ 1)
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 13
Optimization Criterion and Constraints
Practical constraints narrow down the search:
slice dimensions have to be multiples of macroblock width
many values can be ruled out since they are likely to be suboptimal
constraints due to display dimensions, e.g., restrictions on translation of ROI
(sw;sh) = arg min(sw ;sh )
´(sw;sh) £E f P g
dw £ dh
= arg min(sw ;sh )
´(sw;sh) £(dw + sw ¡ 1)
dw
(dh + sh ¡ 1)dh
Bit per pixel for codinggiven layer
Number of pixelstransmitted per rendered pixel
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 14
Model Vs Experimental Results (Pedestrian Area, layer 2)
160x160 128x128 64x64 32x32
Slice size in pixels [ ]sw £ sh
1
1.5
2
2.5N
umbe
r of
pix
els
tran
smitt
ed p
er r
ende
red
pixe
l
0.2
0.3
0.4
0.5
Bit
per
pixe
l for
cod
ing
give
n la
yer
Model
Experiments
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 15
Model Vs Experimental Results (Pedestrian Area, layer 2)
160x160 128x128 64x64 32x32
Slice size in pixels [ ]sw £ sh
0.4
0.45
0.5
0.55
0.6
Bits
tran
smitt
ed p
er r
ende
red
pixe
l
Model
Experiments
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 16
Model Vs Experimental Results (Pedestrian Area, layer 3)
160x160 128x128 64x64 32x32
Slice size in pixels [ ]sw £ sh
1
1.5
2
2.5N
umbe
r of
pix
els
tran
smitt
ed p
er r
ende
red
pixe
l
0.1
0.2
0.3
0.4
Bit
per
pixe
l for
cod
ing
give
n la
yer
Model
Experiments
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 17
Model Vs Experimental Results (Pedestrian Area, layer 3)
160x160 128x128 64x64 32x32
Slice size in pixels [ ]sw £ sh
0.28
0.3
0.32
0.34
0.36
0.38
0.4
Bits
tran
smitt
ed p
er r
ende
red
pixe
l
Model
Experiments
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 18
Summary
Coding scheme provides random access to – arbitrary resolutions – arbitrary spatial regions within every resolution
Slice size is optimized given – the video signal – the QP – the ROI display area dimensions
Other coding parameters could be further optimized, for example, joint selection of the QP for the base layer and the enhancement layers
The EndThe End
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 20
Backup Slides Follow Hereafter
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 21
Parts of the Client’s Display
dw
dh
bw
bh
Overview display area
ROI display area
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 22
Region-of-Interest Trajectory
oh;N
ow;N
ow;1
oh;1¢¢¢Original video is available in resolutionsN
(ow;i = 2¡ (N ¡ i )ow oh;i = 2¡ (N ¡ i )oh) i = 1:: :Nby for
ow = ow;N oh = oh;Nand , i.e., highest resolution
dw
ROIdhROI
ROI
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 23
Pixel Overhead
Pixel Overhead Analysis in 1-D
¡ 1 0 1 2
segment index
¢¢¢ ¢¢¢
Imagine an infinitely long line of pixels. In this example,s = 4; d = 3SOISOISOISOI
Theorem: Given that ,
increases monotonically with
is independent of
d;s 2 N E f £ gs
d
E f £ g = s ¡ 1
£ = 5££ = 1£
Streaming High-Resolution Video with IROI Functionality Sept. 06, 2007 24
Pixel Overhead Analysis in 2-D
dw
ROI
E f £ hg = (sh ¡ 1)
E f £ wg = (sw ¡ 1)
dh
E f £ g = (sw ¡ 1)(sh ¡ 1) + dh(sw ¡ 1) + dw(sh ¡ 1)
E f P g = (dw + sw ¡ 1)(dh + sh ¡ 1)
Expected value of pixel overhead in 2-D
Expected number of pixels to be transmitted