CS 584 / CMPE 584

Multimedia CommunicationsSpring 2006-07

Shahab Baqai

LUMS

2

Administrative

Website– http://suraj.lums.edu.pk/~cs584s06

Readingso Multimedia Communications Applications, Networks, Protocols &

Standards, Fred Halsall, 2001, Pearson Education LtdISBN 81-7808-532-1

o Multimedia Communication Systems, techniques, standards and networks, K.R. Roa, Zoran S. Bojkovic and Dragorard A. Milanvanovic, Pearson Education Inc. 2002

o Selected papers

3

Grading Instruments

Quizzes & HWs: ~10%

Project: ~30%– Proposals 5%– Peer Project Reviews 6%– Interim Progress Report 8%– Final Report/ Demo 16%

Midterm: ~25%

Final Exam: ~35%

4

Introduction

A brief history of (electronic) image communicationInvention of photography and cinema Invention of televisionIntroduction of television broadcasting

Recent advances in digital image communicationKey role of compressionWhat will be covered in this course? Organization

5

Perspective Projection

Censored

6

Perspective Projection

7

Photography and Cinema

8

Nipkow Disk I

9

Nipkow Disk II

10

Image Transmission by Line Scanning

11

Cathode Ray Tube (Braun)

12

History of Electronic Image Communication I

13

History of Electronic Image Communication II

14

Recent Developments: 1990s

15

Motivating Image Compression

16

Motivating Video Compression

17

Distributed Multimedia Systems

Applications:– non-interactive: net radio and TV, video-on-demand, e-learning, ...– interactive: voice &video conference, interactive TV, tele-medicine,

multi-user games, live music, ...

18

Characteristics of multimedia applications

Large quantities of continuous dataTimely and smooth delivery is critical– deadlines– throughput and response time guarantees

Interactive MM applications require low round-trip delaysNeed to co-exist with other applications– must not hog resources

Reconfiguration is a common occurrence– varying resource requirements

Resources required:– Processor cycles in workstations – and servers– Network bandwidth (+ latency)– Dedicated memory– Disk bandwidth (for stored media)

At the right timeand in the right quantities

19

Application requirements

Network phone and audio conferencing– relatively low bandwidth (~ 64 Kbits/sec), but delay times must be short ( <

250 ms round-trip)

Video on demand services– High bandwidth (~ 10 Mbits/s), critical deadlines, latency not critical

Simple video conference– Many high-bandwidth streams to each node (~1.5 Mbits/s each), high

bandwidth, low latency ( < 100 ms round-trip), synchronised states.

Music rehearsal and performance facility– high bandwidth (~1.4 Mbits/s), very low latency (< 100 ms round trip), highly

synchronised media (sound and video < 50 ms).

20

Networking: Historical PerspectiveLate 1960’s - Early 1970’s– Basic Concepts (Packet Switching etc.)– Resource Sharing --- ARPANET

Mid 1970’s - Mid 1980’s– LANs– Connectivity

Mid 1980’s - Mid 1990’s– Internetworking– Global Connectivity

Mid 1990’s ---– Meeting the needs of Applications

MultimediaIntegrated Services

– Taking advantages of advances in technology

21

Traditional Applications

Resource sharing

Remote Login

Electronic mail

File transfer

and more recently

World-Wide-Web

22

New Applications

News

Collaboration

Distance Learning

Tele Medicine

Telephony

Video Conferencing

Etc.

23

Categories of Applications

Communication among people

News and Entertainment

Education and Training

Business Applications

Medical Applications

24

Communication Among People

Voice Communication (VoIP, IP Telephony)– ubiquity of the Internet– alternative to Telcos– integration with other applications– new functionality

conferencing (made easier)storage (record, play-back, index, edit, integrate…)

25

Communication Among People

Video Conferencing– A picture is worth a thousand word

facial expressions, gestures, reactions…– Same advantages as with voice communication– Insertion of video clips– Fly-on-the-wall– Quality

26

Communication Among People

Collaboration– shared white board

more frequent meetings

27

News and Entertainment

News in all its forms (paper, audio, video, web, combination; live and stored)– selectivity (on-line, by profile…)– accessibility without frontiers– urgent notification– linkage among various sources– archival

28

News and Entertainment

Movies and TV programming– Movie-on-demand (pay-per-view)

large selectionfull VCR functionality

– Live broadcasts (sports, weddings, …)– Wider audience

Interactive Games

29

Education and Training

Distance Learning– distance independence

Asynchronous Learning– time-independence

Flexible curriculum

Flexible pace

Monitoring

30

Education and Training

Two concrete examples:– Stanford University

Stanford-on-line– Harvard Business School

on-line case studies

The Virtual classroom– the real-experience– many other benefits

31

Education and Training

Desktop training– criticality– efficiency– productivity– convenience

Example– Professionals (lawyers, medical doctors)

32

Business Applications

Information kiosks

Corporate communication

Factory floor reference

Banking

Home Shopping

E-Commerce

Publishing

etc.

33

Medical Applications

Medical Imaging

Tele-surgery!

Health education

34

Multimedia Applications Characteristics

Applications involving many types of media

Data/Text

Audio

Video

Images

Graphics

35

Data Applications Requirements

Bursty sources

Relatively low average data rate per source

Full end-to-end reliability is required

No latency requirements

Mostly point-to-point

Traffic pattern is bursty

All applications exhibit similar behavior and have

similar requirements– no service differentiation requirement

36

Voice Communication

Voice traffic is Stream-Oriented– continuous flow of data– duration of a call is on the order of minute to an hour

Relatively low data rate per stream (2 to 64 Kbps)

Some data loss may be tolerated (1 to 2 %)– clipped segments below 50 ms cause degradation in the form of background

noise– larger segments cause intelligibility to be affected

Strict end-to-end latency requirement– below 150 ms for interactive voice communication

Very low degree of burstiness (silence suppression)

37

Video Based Applications

Video traffic is stream-oriented

Wide range of data rates– 10’s of Kb/sec to 10’s of Mb/sec– data rate depends on content and quality requirement

Latency requirements depending on application:– interactive communication: 100 ms– one-way broadcast: 1 sec– Video-on-Demand: 1 sec

Burstiness depends on a number of factors– content and quality requirement– compression scheme

38

Shared White Board

Relatively low data rate

full reliability requirement

low latency requirement

39

Multimedia Applications Requirements

Digital Video Data Rates

Low quality or talking heads (video conferencing)– 64 kb/s to 784 kb/s

Business quality (training, video mail)– 1 Mb/s to 2 Mb/s

Broadcasting quality (NTSC, PAL)– 4 Mb/s to 8 Mb/s

High-Definition TV– 20 Mb/s

Studio quality– 10 Mb/s to 45 Mb/s

40

High Bandwidth Requirement

41

Multicasting Requirement

Many multimedia applications involve multiple participants

Size of multicast depends on applications– Videoconferencing (3-4 participants, many-to-many)– group meeting (10’s of participants, one-to-many)– video broadcasting (100’s of participants, one-to-many)

Two models– fixed (closed) predefined set of participants– open set of participants

42

Integrated services Requirement

Coexistence of different media within same application

Coexistence of different applications within the same network

Must deal with:– high and low data rates– bursty and stream traffic– real-time and non-real-time traffic– point-to-point and multi-point modes of communications

43

Networking Requirements

Network Infrastructure– network technologies– network protocols:

routing

Multicasting,

resource reservations

Higher Layer Protocols– end-to-end data transport protocols– session layer protocols

Media Servers

44

Multimedia Application Requirements

1. Bandwidth Requirement– High bandwidth– Guaranteed bandwidth

2. Latency Requirement– Guaranteed maximum end-to-end latency– maximum jitter

3. Multicasting Requirement

4. Integrated Services Requirement

45

Data Traffic MM TrafficData Rate Low High

Traffic Pattern Brusty Stream OrientedHighly Brusty

Reliability Req. No Loss Some Loss

Latency Req. None May be Small

Communication Mode

Point-to-Point Multipoint

Temporal Relationships

None Synchronized Transmission

Type of Service Single Traffic Type Multiple Types

Applications Characteristics

46

Networking Requirements

Network Infrastructure– network technologies– network protocols:

RoutingMulticasting,resource reservations

Higher Layer Protocols– end-to-end data transport protocols– session layer protocols

Media Servers

47

System support issues and requirements

Scheduling and resource allocation in most current OS’s divides the resources equally amongst all comers (processes)– no limit on load– ∴ can’t guarantee throughput or response time

MM and other time-critical applications require resource allocation and scheduling to meet deadlines– Quality of Service (QoS) management

Admission control: controls demandQoS negotiation: enables applications to negotiate admission and

reconfigurationsResource management: guarantees availability of resources for

admitted applications– real-time processor and other resource scheduling

48

Typical infrastructure components for multimedia applications

Microphones

Camera

Screen

Window system

CodecD

BMixer

PC/workstation PC/workstation

C Videostore

Networkconnections

K

L

M

CodecA G

CodecH

Windowsystem

Video file system

: multimedia stream

White boxes represent media processing components, many of which are implemented in software, including:

codec: coding/decoding filtermixer: sound-mixing component

Component Bandwidth Latency Loss rate Resources required

Camera Out: 10 frames/sec, raw video640x480x16 bits

Zero

A Codec In:Out:

10 frames/sec, raw videoMPEG-1 stream

Interactive Low 10 ms CPU each 100 ms;10 Mbytes RAM

B Mixer In:Out:

2 44 kbps audio1 44 kbps audio

Interactive Very low 1 ms CPU each 100 ms;1 Mbytes RAM

H Windowsystem

In:Out:

various50 frame/sec framebuffer

Interactive Low 5 ms CPU each 100 ms;5 Mbytes RAM

K Networkconnection

In/Out: MPEG-1 stream, approx.1.5 Mbps

Interactive Low 1.5 Mbps, low-lossstream protocol

L Networkconnection

In/Out: Audio 44 kbps Interactive Very low 44 kbps, very low-lossstream protocol

This application involves multiple concurrent processes in the PCs

Other applications may also be running concurrently on the same computers

They all share processing and network resources