october 4, 2000 © 2000, columbia university kundan singh internet real-time lab. columbia...

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Kundan SinghInternet Real-Time Lab.

Columbia University

• What it means ?• What can we do ?• How can we do ?• What we have done ?• Q/A

Multimedia Communication Multimedia Communication ApplicationsApplications

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Multimedia + CommunicationMultimedia + Communication

• Audio (voice + music)• Video • Data

• Internet (IP)• Telephone (PSTN)

Interactive Vs non-interactive

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

ApplicationsApplications

Yahoo audio chatYahoo audio chat

Telephony over InternetTelephony over Internet

DialpadDialpad

Email/phoneEmail/phone

Unified messagingUnified messaging

Internet radio/TVInternet radio/TV

Video conferencingVideo conferencing

Instant messaging and presenceInstant messaging and presence

Multi-user gamesMulti-user games

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

ApproachesApproaches

Telco (ITU-T)Telco (ITU-T)IETFIETF

Collaborative Collaborative WorkWork

GamesGames

Web based servicesWeb based services

…add multimedia to my PSTN, ISDN, ATM, … use it for internet…

…signaling and media are different …

…develop and use mbone tools: vic, vat, rat,…

…add new services (email, voice mail, fax) accessible from web … …add any

network support to my multi-user games…

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Signaling ProtocolsSignaling Protocols

Telco (ITU-T)Telco (ITU-T) IETFIETF

H.320 (ISDN) H.320 (ISDN) H.324 (POTS) H.324 (POTS) H.323 (LAN Internet)H.323 (LAN Internet)(H.245, H.225, H.450.x,(H.245, H.225, H.450.x,H.332, …)H.332, …)

RTP/RTCP (Media transport)RTP/RTCP (Media transport)HTTP (Web)HTTP (Web)RTSP (Streaming)RTSP (Streaming)SIP (Session initiation)SIP (Session initiation)……

RMI different approach

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

ScenariosScenarios

• Client/server – RTSP media player and RTSP serverClient/server – RTSP media player and RTSP server• Distributed infrastructure – VoIPDistributed infrastructure – VoIP• Multiparty work - conferencingMultiparty work - conferencing• Messaging - emailMessaging - email• . . .. . .

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Another approachAnother approach

Focus on multimedia communication components instead of specific applications

Audio input

Audio output

Video input Display

Storage

Different communication protocols (SIP, RTSP, H.323,…)

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

What you need ?What you need ?

• Ease of development and experimentation

- Examples - … write a software which calls me on my phone wheneverI get an important mail …… call a number and leave a message. The message isconverted to text and sent as email …

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

What you need ?What you need ?

• Framework for development• Properties: scalable, extensible, interoperable, easy to use

APIAPI

Streaming application,Multimedia conferences,

Gateways, …

RTP/RTCP, SIP, RTSP,DNS, H.323, …

Applications

Libraries

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

CINEMACINEMAColumbia InterNet Extensible Multimedia Architecture

• SIP, RTSP (and HTTP)

• Additional external libraries: RTP, H.323, …

• API: C++ and Java

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

SIP/SDP Parser

Authentication

User registrationD

ynam

ic s

essi

on c

hang

e

SIP/SDP parser

AuthenticationBasic and Digest

User registrationCGI/CPL upload

Dynamic sessionchange

ComponentsComponents

to be added ...to be added ...

• Call transfer• Three party call• Instant messaging and presence

Easy to use !

Easy to use !

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Related workRelated work

• API • JTAPI• Java Servlet• JAIN SIP API

• MASH

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

CINEMACINEMA

sipd sip323 sipconf sipum sipgwrtspd

CINEMA

libNT

Win32 stub

libcine

Utilities parsing

libsip

Basic SIP library

libsip++

SIP UA library

libmixer

RTP audio mixer

libdict

Hash table

libdb++

mySQL intf

RTSP mediaserver

SIP proxy server

SIP/H.323gateway

SIP conferencing

SIP/RTSP unified messaging

SIP/MGCPgateway

LDAPBerkeley DBxml4j

OpenH323

PGPPWLibResparse

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Example: audio conferencingExample: audio conferencing

• Types• Centralized• Distributed• Multicast• Local media mixing

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Example: audio conferencingExample: audio conferencing

• Types• Centralized• Distributed• Multicast• Local media mixing

A

B C

D

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Example: audio conferencingExample: audio conferencing

• Types• Centralized• Distributed• Multicast• Local media mixing

Use multicast for media traffic.

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Example: audio conferencingExample: audio conferencing

• Types• Centralized• Distributed• Multicast• Local media mixing

Mix audio

A

B C

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Sipconf - 1Sipconf - 1

• Features• SIP/SDP and RTP/RTCP

• Audio mixing – sender does not get his own audio

• Play-out delay algorithm

• Web based setup

• G.711 A and Mu law, G.721, DVI ADPCM

• Multiple simultaneous conferences

SIP based audio conference server SIP based audio conference server

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Sipconf - 2Sipconf - 2Comparison with other types Comparison with other types

• Multicast – deployment• Simple endpoints – no local mixing• Overall bandwidth• Add all and subtract one – O(N); N=participants• Extensible – codecs, video• Heterogeneous participants• Silence suppression - scalability

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

What else ?What else ?

SIP based Audio/Video conferencing bridgeSIP based Audio/Video conferencing bridge

Web based SIP clientWeb based SIP clientSIP/PSTN gatewaySIP/PSTN gateway

SIP/RTSP based voice and video mailSIP/RTSP based voice and video mail

SIP for instant messaging and presenceSIP for instant messaging and presence

SIP based email/phoneSIP based email/phone

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Missing piecesMissing pieces

• QoS• Billing• Security• Telephone interface• . . .

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

SummarySummary

• What it means ? -multimedia + communication

• What can we do ? -chat, radio/TV, games, telephony, …

• How can we do ? - SIP, RTSP, RTP, H.323, …

• What we have done ? - sipc, sipd, rtspd, libsip++, sip323, sipum, sipconf, …

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Additional materialAdditional material

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

e*phone

H.323H.323SIPSIP

PSTNPSTNsipgw

PBX

SIP/RTP based embedded ethernetSIP/RTP based embedded ethernetphone device (user-agent)phone device (user-agent)Desktop based SIP user-agentDesktop based SIP user-agentAudio, video, white-board, chat, address bookAudio, video, white-board, chat, address bookSIP server (proxy, redirect, registrar)SIP server (proxy, redirect, registrar)Authentication, sip-cgi, Ldap, unix and windowsAuthentication, sip-cgi, Ldap, unix and windows

SIP/H.323 signaling gatewaySIP/H.323 signaling gatewaybuilt-in gatekeeper, fast-start as well as non-built-in gatekeeper, fast-start as well as non-fast-start, multiple calls, transparent media,fast-start, multiple calls, transparent media,unix and windowsunix and windows

SIP audio conference serverSIP audio conference serveraudio mixing, play-out delay, multiple conferences,audio mixing, play-out delay, multiple conferences,web setupweb setup

SIP/RTSP based unified messagingSIP/RTSP based unified messagingvoice messaging, answering machine, voice messaging, answering machine, web based setup, email and web integrationweb based setup, email and web integration

Missing component …Missing component …

SIP/PSTN gatewaySIP/PSTN gateway

sipc

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Related work - ParlayRelated work - Parlay

• Technology independent interface for telecom• Services developed outside core network• Access network function in secure way• APIs: IT applications Communication world

• Service interface: Network capabilities and informationCall management, messaging, user interaction, billing, policy

management…• Framework interface: external interface in secure and manageable

way• Service registration, subscription, discovery• Authentication, authorization• Integrity management and other management support

• Call model not suited for Internet telephony

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Related work - JTAPIRelated work - JTAPI

• (Java) Telephony API – connection, call, numbers, …• Not suited for Internet Telephony

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Related work – SIP-CGI, CPLRelated work – SIP-CGI, CPL

• Control of server behavior • SIP-CGI similar to HTTP CGI• Call processing language is XML based

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Related work – SIP ServletRelated work – SIP Servlet

• Similar to HTTP (Java) Servlet• Advantages: stateful, low overhead, typed API, other Java based

services like directory service, database, JMF…• Example: location manager, redirection or proxy server, …• More close to SIP-CGI

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Related work – JAINRelated work – JAIN

• Java API for Integrated Networks PSTN, wireless, Internet.• Very extensive framework• Protocol API spec: ISUP, MGCP, SIP, …• Application API spec: JCC/JCAT Call model vs transaction model,

JSPAParlay, SCE/SLEE service creation and logic execution environment, …

• JAIN SIP API:Stack provider (JainSipProvider) and Application (JainSipListener)

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Related work – MASHRelated work – MASH

• Multimedia architecture for communication and collaboration over scalable and heterogeneous networks using IP multicast

• Technologies• SRM• Active objects• Scalable consensus based bandwidth allocation• Media archival and playback• Coordination and control• Transmission over heterogeneous networks

• Mash shell OTcl multimedia components• Multimedia collaboration tools vic, vat, …

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Related workRelated work

• CINEMA concept is close to MASH – SIP and RTSP based• API Provides different levels of abstraction

• Low level – like JAIN SIP API and SIP Servlet API• High level – Call model for user agents (and servers)

• Can inter-work with SIP-CGI, CPL

October 4, 2000 © 2000, Columbia University© 2000, Columbia University

Thank you

http://www.cs.columbia.edu/~kns10/talks/irt3.ppt