adaptive content-aware scaling for improved video streaming
DESCRIPTION
Adaptive Content-Aware Scaling for Improved Video Streaming.. Avanish Tripathi Advisor: Mark Claypool Reader: Bob Kinicki. Outline. Introduction Motivation Related Work Methodology Experiments Results Conclusions and Future Work. Motivation. - PowerPoint PPT PresentationTRANSCRIPT
Adaptive Content-Aware Scaling for Improved Video Streaming.
Avanish Tripathi Advisor: Mark ClaypoolReader: Bob Kinicki
Outline
Introduction Motivation Related Work
MethodologyExperimentsResultsConclusions and Future Work
Motivation
Internet disseminates enormous amounts of informationTCP is the de facto standard… but TCP is not ideal for multimedia and…77% of all Web traffic is Multimedia, of this about 33% is streaming content.
[Chandra, Ellis ’99]
Multimedia Flows…
…tend to use UDP with no congestion controlOther network protocols are being developed: TFRC: smooth reduction in rates as against abrupt
drops in TCP [Floyd et. al. ’00] RAP: Architecture for delivery of layered encoded
streams. [Rejaie et. al. ’99] MPEG-TFRCP: Mapping MPEG to TFRC Protocol
[Miyabayashi et. al. ’00]
Idea-rate based with smooth increase and decrease
Multimedia issues
Generally very high bandwidth requirements Random packet drop by routers during congestion is detrimental to perceptual quality due to interdependencies between packetsNeed application level solution…
…Media Scaling
Need media scaling: Application level data-rate reductionScaling types:
Temporal Quality Spatial
“Content of the stream should influence the choice of scaling mechanism”To the best of our knowledge this idea has not yet been employed
Related Work
Quality Scaling: Receiver-driven Layered Multicast [McCanne ’96]Temporal Scaling:
Player for adaptive MPEG Streaming [Walpole et. al. ‘97] Better Behaved Better Performing MM networking
[Chung, Claypool ‘00]
Content based forwarding for differentiated networks: use priorities based on MPEG characteristics [Shin et. al. ’00]Filtering System: used for media scaling of MPEG streams. [Yeadon ’96]
Outline
Introduction Motivation Related Work
MethodologyExperimentsResultsConclusions and Future Work
Methodology: Content-Aware Scaling
Develop and verify motion measurement mechanismDefine temporal and quality scaling levelsEvaluate the potential impact of content-aware scalingBuild system to do content-aware scaling adaptivelyEvaluate the practical impact of the full system
MPEG Overview
Three kinds of pictures I- Intra encoding P- Predictive encoding B- Bi-directional predictive encoding
Subdivided into Macroblocks Intra, predictive, interpolated macroblocks
Motion vectors are used for motion compensation
Motion Measurement
Higher percentage of interpolated macroblocks means low motionLower percentage of interpolated macroblocks means high motionConducted a pilot study to verify our hypothesis
Divide frame into 16 sub-blocks Count the number of blocks that have motion Correlate that with the percentage of
Interpolated macroblocks.
Pilot Study Result: Motion Measurement
Motion ComputationKeep latency low so that the system is sufficiently reactive
Methodology: Content-Aware Scaling
Develop and verify motion measurement mechanismDefine temporal and quality scaling levelsEvaluate the potential impact of content-aware scalingBuild system to do content-aware scaling adaptivelyEvaluate the practical impact of the full system
Filtering
We extend the system developed at Lancaster university
Frame dropping filter (Temporal Scaling) Requantization filter (Quality Scaling)
User Study Details
22 graduate and undergraduate students in the departmentPlatform: 3 Pentium III machines with 128MB RAM
running Linux Clips were on local hard drives
Four ~10 second clips (2 high motion, 2 low motion)Users rated the clips with numbers from 0 -100
User Study DetailsFive versions of each clip:
Perfect, Temporal Level 1, Temporal Level 2, Quality Level 1, Quality Level 2
Methodology: Content-Aware Scaling
Develop and verify motion measurement mechanismDefine temporal and quality scaling levelsEvaluate the potential impact of content-aware scalingBuild system to do content-aware scaling adaptivelyEvaluate the practical impact of the full system
Results Four men sitting at a bar
Low Motion ( 70 % interpolated macroblocks)
ResultsA girl walks across a room while talking on the phone
Low Motion (57% interpolated Macroblocks)
ResultsRodeo scene where a man on horseback tries to rope a bull
High Motion (27% interpolated macroblocks)
Results Car commerical
High Motion (20% interpolated macroblocks)
Methodology: Content-Aware Scaling
Develop and verify motion measurement mechanismDefine temporal and quality scaling levelsEvaluate the potential impact of content-aware scalingBuild system to do content-aware scaling adaptivelyEvaluate the practical impact of the full system
Full System Architecture
MPEG Server
Input
MotionMeasurement
High
Low
Temporal Filter
Quality Filter
Internet Feedback Generator Client
System Functionality
Server is capable of quantifying motion as the movie playsThe filtering system has five scale levels for finer granularity The system is adaptive and scales movies in real-time depending on the loss pattern as received from the feedback module
User Study
Four clips (2 or more scene) ~30 seconds Four versions of each
Perfect Quality Temporal scaling Quality scaling Adaptive scaling
Bandwidth distribution functions: how often the rate changes
Every 3 seconds Every 200ms
Fit the scale values(1 through 5) on a normal curve [Floyd ‘00]
Future Work
Accurately determine the threshold below which temporal scaling is unacceptableMore accurate bandwidth distribution functionHybrid scaling methods (Quality + Temporal)Audio Scaling
Conclusions
Application level solution to the problem of congestion due to unresponsive video streamsDeveloped a mechanism to quantify the amount of change in a video streamShown that content aware scaling can improve user perceived quality by as much as 50%Developed a system to do adaptive content-aware scaling and are in the process of determining it impact on user perceived quality