a3c mmgc13-slideshare
TRANSCRIPT
Activity-AwareAdaptive Compression
This work is supported by NSF CNS10-12194, CNS09-64081KN
A Morphing-Based Frame Synthesis Application in 3D Tele-Immersion
A3C
Shannon Chen, Penye Xia, and Klara NahrstedtMONET group, UIUC
3D Human Reconstruction
2
Observer
Producer
Compression via Morphing
3
Morphing-based Frame Synthesis
4
Producer Site
Receiver Site
Morphing-based Frame Synthesis
5
Morphing
Morphing
6
Auto Morphing
• Matching skeleton joints (14 pairs, fast)• Matching graphical features (~20 pairs, slow)
7
Auto Morphing
• Matching skeleton joints (14 pairs, fast)• Matching graphical features (~20 pairs, slow)
Motion vs. Compression Ratio
6
High MotionMore differences
between adjacent frames
Low MotionFewer differences
between adjacent frames
Motion vs. Compression Ratio
6
High MotionMore differences
between adjacent frames
Low MotionFewer differences
between adjacent frames
MorphMorph
Morph Morph
Motion vs. Compression Ratio
6
High MotionMore differences
between adjacent frames
Low MotionFewer differences
between adjacent frames
MorphMorph
Morph Morph
Action Recognition
7
Smartphone (motion sensors)
+
ML Classification(SVM)
Lecture Storytelling
Exercise Gaming
Activity-Aware Adaptive Compression
8
Sensor Data
User Activity
Component 1
Component 2
User
Mobile phone
Motion sensor
SVM
Posture features
Motion features
- Storytelling- Speech- Exercise learning- Gaming
Subjective experiment
SFPS MOS
SFPS=x
- Temporal resolution- Spatial resolution
- Acceleration statistics- Power spectrum
QoS Demand
FPS Deduct
Original Video
(FPS=y)
Compressed Video
(FPS=y-x)
Compressed Video
(FPS=y-x)
MBFS
Uncompressed Video (FPS=y)
Receiver side
Producer side
Activity-Aware Adaptive Compression
8
Sensor Data
User Activity
Component 1
Component 2
User
Mobile phone
Motion sensor
SVM
Posture features
Motion features
- Storytelling- Speech- Exercise learning- Gaming
Subjective experiment
SFPS MOS
SFPS=x
- Temporal resolution- Spatial resolution
- Acceleration statistics- Power spectrum
QoS Demand
FPS Deduct
Original Video
(FPS=y)
Compressed Video
(FPS=y-x)
Compressed Video
(FPS=y-x)
MBFS
Uncompressed Video (FPS=y)
Receiver side
Producer side
User Activity Classifier
Synthesized Frame
Determination
Morphing Compressor
Evaluation
• Objective evaluationCompression ratio (versus zlib)
• Subjective evaluationGame play experience(Questionnaire with Likert Scale scores)
9
Results
• Objective evaluationcompression ratio (versus zlib)
25% more bandwidth saving than zlib• Subjective evaluation
Effect on game play experience No significant difference (F<0.01, p=1)
10
Thank You