a3c mmgc13-slideshare

Activity-AwareAdaptive Compression

This work is supported by NSF CNS10-12194, CNS09-64081KN

A Morphing-Based Frame Synthesis Application in 3D Tele-Immersion

A3C

Shannon Chen, Penye Xia, and Klara NahrstedtMONET group, UIUC

3D Human Reconstruction

2

Observer

Producer

Compression via Morphing

3

Morphing-based Frame Synthesis

4

Producer Site

Receiver Site

Morphing-based Frame Synthesis

5

Morphing

Morphing

6

Auto Morphing

• Matching skeleton joints (14 pairs, fast)• Matching graphical features (~20 pairs, slow)

7

Auto Morphing

• Matching skeleton joints (14 pairs, fast)• Matching graphical features (~20 pairs, slow)

Motion vs. Compression Ratio

6

High MotionMore differences

between adjacent frames

Low MotionFewer differences

between adjacent frames

Motion vs. Compression Ratio

6

High MotionMore differences

between adjacent frames

Low MotionFewer differences

between adjacent frames

MorphMorph

Morph Morph

Motion vs. Compression Ratio

6

High MotionMore differences

between adjacent frames

Low MotionFewer differences

between adjacent frames

MorphMorph

Morph Morph

Action Recognition

7

Smartphone (motion sensors)

+

ML Classification(SVM)

Lecture Storytelling

Exercise Gaming

Activity-Aware Adaptive Compression

8

Sensor Data

User Activity

Component 1

Component 2

User

Mobile phone

Motion sensor

SVM

Posture features

Motion features

- Storytelling- Speech- Exercise learning- Gaming

Subjective experiment

SFPS MOS

SFPS=x

- Temporal resolution- Spatial resolution

- Acceleration statistics- Power spectrum

QoS Demand

FPS Deduct

Original Video

(FPS=y)

Compressed Video

(FPS=y-x)

Compressed Video

(FPS=y-x)

MBFS

Uncompressed Video (FPS=y)

Receiver side

Producer side

Activity-Aware Adaptive Compression

8

Sensor Data

User Activity

Component 1

Component 2

User

Mobile phone

Motion sensor

SVM

Posture features

Motion features

- Storytelling- Speech- Exercise learning- Gaming

Subjective experiment

SFPS MOS

SFPS=x

- Temporal resolution- Spatial resolution

- Acceleration statistics- Power spectrum

QoS Demand

FPS Deduct

Original Video

(FPS=y)

Compressed Video

(FPS=y-x)

Compressed Video

(FPS=y-x)

MBFS

Uncompressed Video (FPS=y)

Receiver side

Producer side

User Activity Classifier

Synthesized Frame

Determination

Morphing Compressor

Evaluation

• Objective evaluationCompression ratio (versus zlib)

• Subjective evaluationGame play experience(Questionnaire with Likert Scale scores)

9

Results

• Objective evaluationcompression ratio (versus zlib)

25% more bandwidth saving than zlib• Subjective evaluation

Effect on game play experience No significant difference (F<0.01, p=1)

10

Thank You