nus.soc.cs5248 ooi wei tsang systems support for continuous media ooi wei tsang
TRANSCRIPT
NUS.SOC.CS5248Ooi Wei Tsang
Systems Support for Continuous MediaOOI WEI TSANG
NUS.SOC.CS5248OOI Wei Tsang
Scope of this classSystems Support:
Application Middleware Operating System
Continuous Media: Video, audio, animation
NUS.SOC.CS5248OOI Wei Tsang
Local Media Applications Watch Movie Listen to Music Video Editing :
(Much Less Interesting …)
NUS.SOC.CS5248OOI Wei Tsang
Network Media Applications
Live Webcast Pre-recorded Webcast Video Conferencing Video on Demand
NUS.SOC.CS5248OOI Wei Tsang
Categories
InteractiveNon-
Interactive
LiveVideo
ConferencingNUSLive,NUSCast
Pre-recorded
Lecture/Video on Demand
NUS.SOC.CS5248OOI Wei Tsang
Streaming Media RequiresReliable networksLow latencyBounded latencyPlenty of bandwidth
Not in today’s Internet.
NUS.SOC.CS5248OOI Wei Tsang
Overview
Network
Encoder
Sender
Middlebox
Receiver
Decoder
NUS.SOC.CS5248OOI Wei Tsang
We will NOTDesign new codec
Study media processing CS6212/CS5240
Attempt to improve the Internet EE5910
NUS.SOC.CS5248Ooi Wei Tsang
Summary of TopicsCovered
NUS.SOC.CS5248OOI Wei Tsang
Sample of Topics Covered
Review of Media CompressionNetwork Communication Model
Network
Encoder
Sender
Middlebox
Receiver
Decoder
NUS.SOC.CS5248OOI Wei Tsang
Sample of Topics Covered
AdaptationVideo on Demand
Network
Encoder
Sender
Middlebox
Receiver
Decoder
NUS.SOC.CS5248OOI Wei Tsang
Sample of Topics Covered
Buffering Audio/Video Synchronization Packet Loss Recovery
Network
Encoder
Sender
Middlebox
Receiver
Decoder
NUS.SOC.CS5248OOI Wei Tsang
Sample of Topics Covered
CachingApplication-Level Multicast
Network
Encoder
Sender
Middlebox
Receiver
Decoder
NUS.SOC.CS5248OOI Wei Tsang
Lecture PhilosophyCover the breadth, rather than
depth
Classic papers, rather than recent papers
NUS.SOC.CS5248Ooi Wei Tsang
Class Organization
NUS.SOC.CS5248OOI Wei Tsang
GoalsLearn about existing techniques
and systemsLearn about current research
effortsLearn about how to do research
NUS.SOC.CS5248OOI Wei Tsang
PhilosophyResearch orientedStudents are expected to be
mature and independentWhat you learned is more
important than grade
NUS.SOC.CS5248OOI Wei Tsang
Please don’t ask.. “Is this equation important?” “Is this equation examinable?” “Do I have to memorize this
header format?” “Post answer to
quiz/survey/review”
NUS.SOC.CS5248OOI Wei Tsang
Please do ask .. “What is the effect of changing k in
the equation?”
“Why did the designer added this bit in the header?”
NUS.SOC.CS5248OOI Wei Tsang
Academic HonestyNo copying among studentsNo copying from published work
ZERO TOLERENCE to Plagiarism
NUS.SOC.CS5248OOI Wei Tsang
Discussion?Strongly encouraged ..but
must acknowledge all contributions write up solutions independently
NUS.SOC.CS5248OOI Wei Tsang
Gilligan’s Island RuleDiscuss with friends
Throw notes away
Go watch “Gilligan’s Island” (or “Phua Chu Kang”, or “Super Sunday”)
Write down your solution
NUS.SOC.CS5248Ooi Wei Tsang
Continuous Assessments
NUS.SOC.CS5248OOI Wei Tsang
Continuous AssessmentsProject (50%)Paper Survey (20%)Paper Review (15%)One of
Programming Assignment Random Quizzes Class Participation Mid-term
NUS.SOC.CS5248OOI Wei Tsang
Research ProjectDefine a problemPropose a solution Implement the solutionEvaluate the solution
NUS.SOC.CS5248OOI Wei Tsang
Types of ProjectNew problem + solutionExisting problem + new solutionExisting problem + existing
solutions + comparisons + new insights
See website for more..
NUS.SOC.CS5248OOI Wei Tsang
VariationsCombine with your own research
subjected to your advisor’s approval
Combine with another class project subjected to the other instructor’s
approval must be more significant
NUS.SOC.CS5248OOI Wei Tsang
Research ProjectDeliverables:
10-12 page paper poster presentation demo (if applicable)
In a group of 1 to n students
NUS.SOC.CS5248OOI Wei Tsang
Survey Paper In-depth study of a topic4-10 papers expected
Suggested list of topics available
(can be the same topic as your project!)
NUS.SOC.CS5248OOI Wei Tsang
Survey Paper summarize recent resultsorganize, generalize, categorize add understanding
NUS.SOC.CS5248OOI Wei Tsang
Survey PaperDeliverables:
10-12 page survey paper
Individual work
NUS.SOC.CS5248OOI Wei Tsang
Paper ReviewsRead, summarize, and criticize 10
papers
One is assigned almost weekly
Deliverables: One page report per paper
NUS.SOC.CS5248OOI Wei Tsang
Expected WorkloadRead 3-4 papers per weekSome hours of coding per week
NUS.SOC.CS5248OOI Wei Tsang
Office HoursSaturday11 am – 12 noonSOC1 04-20
or email for appointmentor just pop in
NUS.SOC.CS5248OOI Wei Tsang
Web SitesMain Website
http://www.comp.nus.edu.sg/~cs5248
IVLE Website
NUS.SOC.CS5248Ooi Wei Tsang
Questions
NUS.SOC.CS5248Ooi Wei Tsang
Review of Media Compression
NUS.SOC.CS5248OOI Wei Tsang
You are Here
Network
Encoder
Sender
Middlebox
Receiver
Decoder
NUS.SOC.CS5248OOI Wei Tsang
Why compress? “Bandwidth Not Enough” “Disk Space Not Enough”
Size of Uncompressed DVD Movie =
NUS.SOC.CS5248OOI Wei Tsang
Why compress? “Bandwidth Not Enough” “Disk Space Not Enough”
Size of Uncompressed DVD Movie = 720 x 576 x 3 x 25 x 60 x 120 =208.6GB
NUS.SOC.CS5248Ooi Wei Tsang
JPEG Compression
Original Image (1153KB)
1:1
Original Image (1153KB)
3.5:1
Original Image (1153KB)
17:1
Original Image (1153KB)
27:1
Original Image (1153KB)
72:1
Original Image (1153KB)
192:1
NUS.SOC.CS5248OOI Wei Tsang
Compression Ratio
Quality Size Ratio
Raw TIFF 1153KB 1:1
Zipped TIFF 982KB 1.2:1
Q=100 331KB 3.5:1
Q=70 67KB 17:1
Q=40 43KB 27:1
Q=10 16KB 72:1
Q=1 6KB 192:1
NUS.SOC.CS5248OOI Wei Tsang
Magic of JPEGThrow away information we cannot
see Color information “Low-frequency signals”
Rearrange data for good compression
Use standard compression
NUS.SOC.CS5248OOI Wei Tsang
Throw away color information
Y
V U
NUS.SOC.CS5248OOI Wei Tsang
Color Sub-sampling
4:4:4
4:2:2
4:2:0
4:1:1
Original Image (1153KB)
Original Image (1153KB)
NUS.SOC.CS5248OOI Wei Tsang
Discrete Cosine TransformDCT demo applet and source code
is available on the web.
NUS.SOC.CS5248OOI Wei Tsang
Quantization
242 65
-54 -10
23 5
-4 -2
13 6
2 1
3 5
-1 -2
8 8
8 8
8 8
8 16
8 8
8 16
16 32
32 64
30 8
-6 -1
2 0
0 0
1 0
0 0
0 0
0 0
/ =
NUS.SOC.CS5248OOI Wei Tsang
Differential Coding
30 8
6 -1
2 0
0 0
1 0
0 0
0 0
0 0
25 3
2 1
1 0
0 0
4 0
0 0
1 0
0 0
27 3
2 1
1 0
0 0
4 0
0 0
1 0
0 0
30 8
6 -1
2 0
0 0
1 0
0 0
0 0
0 0
-5 3
2 1
1 0
0 0
4 0
0 0
1 0
0 0
2 3
2 1
1 0
0 0
4 0
0 0
1 0
0 0
NUS.SOC.CS5248OOI Wei Tsang
Zig-zag ordering
27 3
2 1
1 0
0 0
4 0
0 0
1 0
0 0
27, 3, 2, 4, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0
NUS.SOC.CS5248OOI Wei Tsang
Run-Length Encoding
27 3
2 1
1 0
0 0
4 0
0 0
1 0
0 0
27, 3, 2, 4, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0
(27, 1) (3, 1) (2, 1), (4, 1), (1, 2), (0, 5), (1, 1), (0, 4)
NUS.SOC.CS5248OOI Wei Tsang
Idea: Motion JPEGCompress every frame in a video
as JPEG
DVD-quality video = 208.6GB Reduction ratio = 27:1Final size = 7.7GB
NUS.SOC.CS5248Ooi Wei Tsang
Video Compression
NUS.SOC.CS5248OOI Wei Tsang
Temporal Redundancy
NUS.SOC.CS5248OOI Wei Tsang
Motion Estimation
NUS.SOC.CS5248OOI Wei Tsang
Bi-directional Prediction
NUS.SOC.CS5248OOI Wei Tsang
H.261
I-Frame
P-Frame
NUS.SOC.CS5248OOI Wei Tsang
MPEG-1
B-Frame
NUS.SOC.CS5248OOI Wei Tsang
MPEG Frame PatternExample display sequence:
IBBPBBP …
Example encoding sequence: IPBBPBB
NUS.SOC.CS5248OOI Wei Tsang
Compression Ratio
Frame Type Typical Ratio
I 10:1
P 20:1
B 50:1
NUS.SOC.CS5248OOI Wei Tsang
Sequence
sequence header:• width• height• frame rate• bit rate• :
NUS.SOC.CS5248OOI Wei Tsang
GOP: Group of Picture
gop header:• time• :
NUS.SOC.CS5248OOI Wei Tsang
Picture
pic header:• number• type (I,P,B)• :
NUS.SOC.CS5248OOI Wei Tsang
Picture
NUS.SOC.CS5248OOI Wei Tsang
Slice
NUS.SOC.CS5248OOI Wei Tsang
Macroblock
NUS.SOC.CS5248OOI Wei Tsang
Block
Y Y
Y Y
U
V
1 Macroblock =
NUS.SOC.CS5248OOI Wei Tsang
MPEG in daily life
MPEG Standards
Bit-rate Usage
MPEG-1 1.5Mbps VCD
MPEG-2 10-40 MbpsDVD, SVCD,
HDTV
MPEG-4 ScalableQuickTime,
DivX;)
NUS.SOC.CS5248OOI Wei Tsang
Impact on Systems DesignHow to package data into packets?How to deal with packet loss?How to deal with bursty traffic?How to predict decoding time? : :