nus.soc.cs5248 ooi wei tsang systems support for continuous media ooi wei tsang

NUS.SOC.CS5248Ooi Wei Tsang

Systems Support for Continuous MediaOOI WEI TSANG

NUS.SOC.CS5248OOI Wei Tsang

Scope of this classSystems Support:

Application Middleware Operating System

Continuous Media: Video, audio, animation

NUS.SOC.CS5248OOI Wei Tsang

Local Media Applications Watch Movie Listen to Music Video Editing :

(Much Less Interesting …)

NUS.SOC.CS5248OOI Wei Tsang

Network Media Applications

Live Webcast Pre-recorded Webcast Video Conferencing Video on Demand

NUS.SOC.CS5248OOI Wei Tsang

Categories

InteractiveNon-

Interactive

LiveVideo

ConferencingNUSLive,NUSCast

Pre-recorded

Lecture/Video on Demand

NUS.SOC.CS5248OOI Wei Tsang

Streaming Media RequiresReliable networksLow latencyBounded latencyPlenty of bandwidth

Not in today’s Internet.

NUS.SOC.CS5248OOI Wei Tsang

Overview

Network

Encoder

Sender

Middlebox

Receiver

Decoder

NUS.SOC.CS5248OOI Wei Tsang

We will NOTDesign new codec

Study media processing CS6212/CS5240

Attempt to improve the Internet EE5910

NUS.SOC.CS5248Ooi Wei Tsang

Summary of TopicsCovered

NUS.SOC.CS5248OOI Wei Tsang

Sample of Topics Covered

Review of Media CompressionNetwork Communication Model

Network

Encoder

Sender

Middlebox

Receiver

Decoder

NUS.SOC.CS5248OOI Wei Tsang

Sample of Topics Covered

AdaptationVideo on Demand

Network

Encoder

Sender

Middlebox

Receiver

Decoder

NUS.SOC.CS5248OOI Wei Tsang

Sample of Topics Covered

Buffering Audio/Video Synchronization Packet Loss Recovery

Network

Encoder

Sender

Middlebox

Receiver

Decoder

NUS.SOC.CS5248OOI Wei Tsang

Sample of Topics Covered

CachingApplication-Level Multicast

Network

Encoder

Sender

Middlebox

Receiver

Decoder

NUS.SOC.CS5248OOI Wei Tsang

Lecture PhilosophyCover the breadth, rather than

depth

Classic papers, rather than recent papers

NUS.SOC.CS5248Ooi Wei Tsang

Class Organization

NUS.SOC.CS5248OOI Wei Tsang

GoalsLearn about existing techniques

and systemsLearn about current research

effortsLearn about how to do research

NUS.SOC.CS5248OOI Wei Tsang

PhilosophyResearch orientedStudents are expected to be

mature and independentWhat you learned is more

important than grade

NUS.SOC.CS5248OOI Wei Tsang

Please don’t ask.. “Is this equation important?” “Is this equation examinable?” “Do I have to memorize this

header format?” “Post answer to

quiz/survey/review”

NUS.SOC.CS5248OOI Wei Tsang

Please do ask .. “What is the effect of changing k in

the equation?”

“Why did the designer added this bit in the header?”

NUS.SOC.CS5248OOI Wei Tsang

Academic HonestyNo copying among studentsNo copying from published work

ZERO TOLERENCE to Plagiarism

NUS.SOC.CS5248OOI Wei Tsang

Discussion?Strongly encouraged ..but

must acknowledge all contributions write up solutions independently

NUS.SOC.CS5248OOI Wei Tsang

Gilligan’s Island RuleDiscuss with friends

Throw notes away

Go watch “Gilligan’s Island” (or “Phua Chu Kang”, or “Super Sunday”)

Write down your solution

NUS.SOC.CS5248Ooi Wei Tsang

Continuous Assessments

NUS.SOC.CS5248OOI Wei Tsang

Continuous AssessmentsProject (50%)Paper Survey (20%)Paper Review (15%)One of

Programming Assignment Random Quizzes Class Participation Mid-term

NUS.SOC.CS5248OOI Wei Tsang

Research ProjectDefine a problemPropose a solution Implement the solutionEvaluate the solution

NUS.SOC.CS5248OOI Wei Tsang

Types of ProjectNew problem + solutionExisting problem + new solutionExisting problem + existing

solutions + comparisons + new insights

See website for more..

NUS.SOC.CS5248OOI Wei Tsang

VariationsCombine with your own research

subjected to your advisor’s approval

Combine with another class project subjected to the other instructor’s

approval must be more significant

NUS.SOC.CS5248OOI Wei Tsang

Research ProjectDeliverables:

10-12 page paper poster presentation demo (if applicable)

In a group of 1 to n students

NUS.SOC.CS5248OOI Wei Tsang

Survey Paper In-depth study of a topic4-10 papers expected

Suggested list of topics available

(can be the same topic as your project!)

NUS.SOC.CS5248OOI Wei Tsang

Survey Paper summarize recent resultsorganize, generalize, categorize add understanding

NUS.SOC.CS5248OOI Wei Tsang

Survey PaperDeliverables:

10-12 page survey paper

Individual work

NUS.SOC.CS5248OOI Wei Tsang

Paper ReviewsRead, summarize, and criticize 10

papers

One is assigned almost weekly

Deliverables: One page report per paper

NUS.SOC.CS5248OOI Wei Tsang

Expected WorkloadRead 3-4 papers per weekSome hours of coding per week

NUS.SOC.CS5248OOI Wei Tsang

Office HoursSaturday11 am – 12 noonSOC1 04-20

or email for appointmentor just pop in

NUS.SOC.CS5248OOI Wei Tsang

Web SitesMain Website

http://www.comp.nus.edu.sg/~cs5248

IVLE Website

NUS.SOC.CS5248Ooi Wei Tsang

Questions

NUS.SOC.CS5248Ooi Wei Tsang

Review of Media Compression

NUS.SOC.CS5248OOI Wei Tsang

You are Here

Network

Encoder

Sender

Middlebox

Receiver

Decoder

NUS.SOC.CS5248OOI Wei Tsang

Why compress? “Bandwidth Not Enough” “Disk Space Not Enough”

Size of Uncompressed DVD Movie =

NUS.SOC.CS5248OOI Wei Tsang

Why compress? “Bandwidth Not Enough” “Disk Space Not Enough”

Size of Uncompressed DVD Movie = 720 x 576 x 3 x 25 x 60 x 120 =208.6GB

NUS.SOC.CS5248Ooi Wei Tsang

JPEG Compression

Original Image (1153KB)

1:1

Original Image (1153KB)

3.5:1

Original Image (1153KB)

17:1

Original Image (1153KB)

27:1

Original Image (1153KB)

72:1

Original Image (1153KB)

192:1

NUS.SOC.CS5248OOI Wei Tsang

Compression Ratio

Quality Size Ratio

Raw TIFF 1153KB 1:1

Zipped TIFF 982KB 1.2:1

Q=100 331KB 3.5:1

Q=70 67KB 17:1

Q=40 43KB 27:1

Q=10 16KB 72:1

Q=1 6KB 192:1

NUS.SOC.CS5248OOI Wei Tsang

Magic of JPEGThrow away information we cannot

see Color information “Low-frequency signals”

Rearrange data for good compression

Use standard compression

NUS.SOC.CS5248OOI Wei Tsang

Throw away color information

Y

V U

NUS.SOC.CS5248OOI Wei Tsang

Color Sub-sampling

4:4:4

4:2:2

4:2:0

4:1:1

Original Image (1153KB)

Original Image (1153KB)

NUS.SOC.CS5248OOI Wei Tsang

Discrete Cosine TransformDCT demo applet and source code

is available on the web.

NUS.SOC.CS5248OOI Wei Tsang

Quantization

242 65

-54 -10

23 5

-4 -2

13 6

2 1

3 5

-1 -2

8 8

8 8

8 8

8 16

8 8

8 16

16 32

32 64

30 8

-6 -1

2 0

0 0

1 0

0 0

0 0

0 0

/ =

NUS.SOC.CS5248OOI Wei Tsang

Differential Coding

30 8

6 -1

2 0

0 0

1 0

0 0

0 0

0 0

25 3

2 1

1 0

0 0

4 0

0 0

1 0

0 0

27 3

2 1

1 0

0 0

4 0

0 0

1 0

0 0

30 8

6 -1

2 0

0 0

1 0

0 0

0 0

0 0

-5 3

2 1

1 0

0 0

4 0

0 0

1 0

0 0

2 3

2 1

1 0

0 0

4 0

0 0

1 0

0 0

NUS.SOC.CS5248OOI Wei Tsang

Zig-zag ordering

27 3

2 1

1 0

0 0

4 0

0 0

1 0

0 0

27, 3, 2, 4, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0

NUS.SOC.CS5248OOI Wei Tsang

Run-Length Encoding

27 3

2 1

1 0

0 0

4 0

0 0

1 0

0 0

27, 3, 2, 4, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0

(27, 1) (3, 1) (2, 1), (4, 1), (1, 2), (0, 5), (1, 1), (0, 4)

NUS.SOC.CS5248OOI Wei Tsang

Idea: Motion JPEGCompress every frame in a video

as JPEG

DVD-quality video = 208.6GB Reduction ratio = 27:1Final size = 7.7GB

NUS.SOC.CS5248Ooi Wei Tsang

Video Compression

NUS.SOC.CS5248OOI Wei Tsang

Temporal Redundancy

NUS.SOC.CS5248OOI Wei Tsang

Motion Estimation

NUS.SOC.CS5248OOI Wei Tsang

Bi-directional Prediction

NUS.SOC.CS5248OOI Wei Tsang

H.261

I-Frame

P-Frame

NUS.SOC.CS5248OOI Wei Tsang

MPEG-1

B-Frame

NUS.SOC.CS5248OOI Wei Tsang

MPEG Frame PatternExample display sequence:

IBBPBBP …

Example encoding sequence: IPBBPBB

NUS.SOC.CS5248OOI Wei Tsang

Compression Ratio

Frame Type Typical Ratio

I 10:1

P 20:1

B 50:1

NUS.SOC.CS5248OOI Wei Tsang

Sequence

sequence header:• width• height• frame rate• bit rate• :

NUS.SOC.CS5248OOI Wei Tsang

GOP: Group of Picture

gop header:• time• :

NUS.SOC.CS5248OOI Wei Tsang

Picture

pic header:• number• type (I,P,B)• :

NUS.SOC.CS5248OOI Wei Tsang

Picture

NUS.SOC.CS5248OOI Wei Tsang

Slice

NUS.SOC.CS5248OOI Wei Tsang

Macroblock

NUS.SOC.CS5248OOI Wei Tsang

Block

Y Y

Y Y

U

V

1 Macroblock =

NUS.SOC.CS5248OOI Wei Tsang

MPEG in daily life

MPEG Standards

Bit-rate Usage

MPEG-1 1.5Mbps VCD

MPEG-2 10-40 MbpsDVD, SVCD,

HDTV

MPEG-4 ScalableQuickTime,

DivX;)

NUS.SOC.CS5248OOI Wei Tsang

Impact on Systems DesignHow to package data into packets?How to deal with packet loss?How to deal with bursty traffic?How to predict decoding time? : :