Aug 25, 2005page1 Aug 25, 2005

Integration of Advanced Video/Speech Codecs into AccessGridNational Center for High Performance Computing

Speaker: Barz Hsu

Email: barz@nchc.org.tw

Aug 25, 2005page2

Outline

• Overview

• NCHC-VIC:– Introduction to MPEG4/H.264 codec– Codec Optimization

• NCHC-RAT:– Introduction to Speex codec– Echo Cancellation and Noise Control

• Future Work

Aug 25, 2005page3

Overview

• AccessGrid is scalable collaborative environment.• AccessGrid is complicated and not easy to use.• Major complexity comes from the two core media

– vic (video) and rat (speech)

• Most of video/speech codecs inside AccessGrid are out-of-date.

• Integrate state-of-art MPEG4/H.264 codecs for video coding, and Speex codec for speech coding.

• High quality and low latency

Aug 25, 2005page4

MPEG4 Codec

• Related Open Source Project– FFmpeg, xvid, and sklmp4

• FFmpeg/libavcodec– High performance on CPU usage– Support various codec

• MPEG4, MPEG2, MPEG1• H.263++, H.263, H.261

– Provide error resilience tools• slices, data partition

– Cross Platform• Linux, Windows, and Mac OS

Aug 25, 2005page5

H.264 Codec

• NCHC-VIC based on X264 project of VideoLAN.• X264 is widely used by other OpenSource projects,

ex., FFMpeg, Mplayer, VideoLan.• X264 is the only free software among current H.264

codecs.• The project is still under aggressive development.

– http://developers.videolan.org/x264.html

Aug 25, 2005page6

H.264 Codec (cont’d)

• Adopt new techniques to effectively reduce bandwidth requirement and provide better visual quality– Smaller block division– Multiple prediction modes– Multiple reference frames– Finer motion estimation (Quarter-Pel)– Deblocking filter

• More computation intensive (5 to 10 times of MPEG4 ASP)

Visual Comparison between H.264 and MPEG4 (static video with 128kbps)Visual Comparison between H.264 and MPEG4 (static video with 128kbps)

Aug 25, 2005page7

Codec Optimization

• Multithread Encoding– Take advantage of symmetric multiple processor

architecture

• Pipeline the latency of transmission and encoding

• SIMD Instruction Sets– Optimized by MMX, SSE/SSE2/SSE3

Aug 25, 2005page8

H.264 Optimization

• Due to the need of huge computation for H.264 encoding, we close some advanced features to meet the requirement of the real-time video conference– Using CAVLC instead of CABAC– Larger block division– Simpler block mode prediction

Aug 25, 2005page9

FFMpeg MPEG4 vs x264 (RD Curve)

x264: cavlc, only 16x16 block

Aug 25, 2005page10

FFMpeg MPEG4 vs x264 (Speed)

x264: cavlc, only 16x16 blockCPU: P4 2.8GHzRAM: 512MB

Aug 25, 2005page11

Visual Comparison between H.264 and MPEG4 (static video with 128kbps)

H.264 MPEG4

Aug 25, 2005page12

Visual Comparison between H.264 and MPEG4 (dynamic video with 128kbps)

H.264 MPEG4

Aug 25, 2005page13

Full Screen Snapshot

Aug 25, 2005page14

Experimental Video

Aug 25, 2005page15

NCHC-VIC for Win32 Port

• Based on Linux Version• Update IPv6 code• Compiled by Visual Studio 6• Some parts are compiled by gcc and then link.

Aug 25, 2005page16

NCHC VIC for Win32 Screenshot

Aug 25, 2005page17

Speex Codec for Speech

• NCHC-RAT adopts Speex open source project• Paten-free audio compression format• Narrowband (8 kHz), wideband (16 kHz), and ultra-

wideband (32 kHz) • Subband coding to reduce computation overhead• Packet loss concealment• Intensity stereo encoding• Project website,

– http://www.speex.org/

Aug 25, 2005page18

Echo Cancellation and Noise Control

• Adaptive filter to model acoustic echo path– Multidelay Block Frequency Domain Adaptive Filter

• Double Talk Detector– Multidelay Double-Talk Detector Combined with the MDF Adaptive

Filter

• Noise Control– Mexico-hat low pass filter

Aug 25, 2005page19

Echo Model

Aug 25, 2005page20

NLMS Algorithm

• Normalize Least Mean Square Algorithm

Aug 25, 2005page21

Future Work

• Session Description Protocol• Session Announcement Protocol • Advanced User Interface• Zero-Configure Cluster Decoding

Aug 25, 2005page22

Thanks for

your attendance