Creating Resource-Efficient V2oIP Applications for Low-MHz Mobile Processors

Fred WydlerVP VoIP ProductsSPIRIT DSP

3

About SPIRIT

10 years in global embedded voice software business

200+ first-class customers, including largest OEMs and software vendors

140 people, 70 engineers, 15 PhDs 100+ millions embedded (HW) voice

channels powered Multiple international “Product of the

year” awards:

Selected SPIRIT Customers:

…and 200+ more

Acknowledged Communications Expert

4

Agenda

VoIP on mobile: new rules of the game

To be IMS ready is the goal

Video communication – must-have functionality

Mobile VoIP. Quality wins. Wideband coming

Where the troubles come from

5

Agenda

VoIP on mobile: new rules of the game

To be IMS ready is the goal

Video communication – must-have functionality

Mobile VoIP. Quality wins. Wideband coming

Where the troubles come from

6

WiFi phones

Game Consoles

Portable players

Mobile phones

VoIP

Set-top boxesWireless residential gateways

Mobile VoIP: Start of a New Era

7

Demands of Today and Tomorrow

Mobile form factor

Time-to-market

High quality echo free voice

Basic layer – traditional requirements

Real-time video calls

Multimedia accelerators

IMS ComplianceRequirements grow higher and deeper

8

Agenda

VoIP on mobile: new rules of the game

To be IMS ready is the goal

Video communication – must-have functionality

Mobile VoIP. Quality wins. Wideband coming

Where the troubles come from

9

Each Service over Each Access Network

ü Voiceü Instant messagingü Video callsü Content sharingü Video sharingü Interactive videoü 3D multimediaü Mp3 playbackü Gamingü Broadcastingü PPT, POC

Cellular (GSM, W-CDMA, CDMA200)

Data channels (GPRS, EDGE, EV-DO, HSPDA)

Wireless IP(WiFi, WiMAX)

IMS idea goes to the real life fast

10

VoIP Evolution

Are you 100% sure of your Voice Engine supplier?M

arke

t Dem

ands

Time

Voice Engine

Voice Engine

Voice Engine

Voice Engine

Video Engine

Video Engine

Video Engine

Application Enabler

Application Enabler

Defined Quality Level

Voice Engine

Video Engine

Application Enabler

Defined Quality Level

Full IMS Client

11

Agenda

VoIP on mobile: new rules of the game

To be IMS ready is the goal

Video communication – must-have functionality

Mobile VoIP. Quality wins. Wideband coming

Where the troubles come from

12

Video Means See not Guess

Video performance in terms of resource limitations of mobile platforms

– Regular RISC processor inside offers up to 15fps on a PDA display– Adding HW accelerator significantly improves frame rate

Video packet loss - Video PLC

Packets’ delivery conditions do not allow smooth playback– Automatic adaptation to the channel bandwidth (ARS)– Intelligent CPU load control/playback rate control– Audio/Video synchronization

voice, audio, video

13

Hardware Accelerators Welcome

Standard integration (OpenMAX™)

Custom integration TeamSpirit® Mobile Voice + Video Engine

Application Layer

Call Manager (SIP, H.323, H.324M)

Media Transport (RTP/RTCP/SRTP)

Thin Layer

Hardware Accelerator

H.263+

H.264MPEG.4

AMR

G.723.1 AEC

Voice&Video Framework

Video RTP Packetizers

CPU Load Control

VoIP Speech Codecs

Audio/Video Synchronization

NetJet™(Jitter Buffer, PLC,

“playout rate”/”clock drift” adaptation)

RealDuplex™ (Speech Enhancement)

DTMF

Integrating hardware accelerators into a complete solution

Already deployed by:

14

Agenda

VoIP on mobile: new rules of the game

To be IMS ready is the goal

Video communication – must-have functionality

Mobile VoIP. Quality wins. Wideband coming

Where the troubles come from

15

Quality as Competitive Advantage

Mobile VoIP is a commodity (lots of providers offer VoIP services)

Zero switching costs while changing VoIP provider (caused by poor user experience: voice breaks, delays, echo)

To tie up the customer VoIP providers need to pursue voice quality

Customer satisfaction is the goal

3.9MOS 4.32MOSAverage real conversation voice quality delivered by typical VoIP service providers nowadays

Voice quality delivered by TeamSpirit® for VoIP calls over WiFi

vs.

16

Wideband Voice Engine

Wideband AEC

Wideband speech codec

Wideband adapted algorithms

Customer’s demand – wideband communication

17

Agenda

VoIP on mobile: new rules of the game

To be IMS ready is the goal

Video communication – must-have functionality

Mobile VoIP. Quality wins. Wideband coming

Where the troubles come from

18

Where the Troubles Come From

Network challenges

Voice processing challenges

Device challenges

Integration challenges

19

Why Engine, not Codec?

Smooth operation in IP networks originally not suited to real-time voice transmission

– Network jitter elimination– Packet loss concealment (standard speech codecs)– Delay reduction– Packets re-ordering

Robustness to WiFi-specific network impairments– Applying new WiFi standard revisions– Advanced algorithms handling packet bulk scenario

Network challenges

20

Seamless Network Access

Handover– Apply VoIP to different access networks (WiFi, 3G)

– Network evolution to the all-IP network and mobility

– Service provider has to be involved

– Active and automatic handover

WiFi roaming

Network challenges

21

PSTN Experience to Achieve with VoIP

Voice processing challenges

Quality issues – Voice packet loss robustness– Efficient echo cancellation– Round-trip delay minimization– Clock drift adaptation

Regular telephony services – Call mute – Call hold/resume – N-way conferencing – Call transfer

Advanced telephony services – 4x dialing – Voice mail – Caller ID

22

Mobile Form Factor to Survive with

Device challenges

Hardware acoustic differences – Earpiece mode support– Microphone and speaker AGC– Filtering– Intelligent non-linear processing

Battery life – Minimal MHz requirements– The least number of threads in a system and the lowest memory

footprint– Effective usage of DTX mode to minimize Tx and Rx power

consumption for the WiFi interface– Highly-optimized speech and video codecs– Careful optimization to specific processor architecture

23

VoIP Encapsulation

VoIP is a commodity – not everyone is an expert in VoIP– Integration is the key

–Minimum number of threads–CPU load control–Time for debugging

– Cross-platform code (ARM9/9E/11, TI DaVinci, TI C64x, Renesas, XScale, TI OMAP)

– Support for Windows/Linux OS families– HW accelerators support

High-level API – application enabler required– Signalling (H.324M, H.323, SIP, Skype, XMPP, any proprietary

protocol)– IMS application compliance

Integration challenges

24

Integration is the Key

vs.

− Single components work good, but the complete solution does not

− Single components are optimized, but the complete solution is not

− Integration is difficult to manage

I/OG.7xx

H.26xNetwork

I/O

G.*

H.*

I/O

G.*

H.*

What’s wrong with components?

25

Complete Solution

Application Layer

Call Manager (SIP, H.323, H.324)

TeamSpirit® Mobile Voice+Video

Media Transport (RTP/RTCP/SRTP)

Speech Codec Library(G.7xx, AMR-NB)

NetJet™(Jitter Buffer, Packet Loss Concealment, “Clock Drift”

Adaptation)

DTMF

RealDuplex™(AEC, VAD, AGC, CNG, NS,

Mode Detection)

OS/Hardware Dependent IO (Sound/Video IO, Device Detector)

Voice Engine

Video Codec Library(H.26x, MPEG.4)

RTP Packetizers

Playback Rate Control

Video Packet Loss Concealment

CPU Load Control

Video Engine

Audio/Video Synch.

N-wayConferencing

Mixer

26

Real Life Requirements

Interoperability with IMS infrastructure vendors

Firewall overcoming

Testing with major operators in USA, Asia, Europe

Beyond standard specifications

27

www.SpiritDSP.com

Thank you!

Voice and Video Enable your Application!Request TeamSpirit® Mobile evaluation SDK

with free SIP Voice&Video phone application in it running on starting from 200 MHz WinMobile devices

request@spiritdsp.com