miss chan pik wah, pat miss ngai cheuk han, edith
DESCRIPTION
6 th IT Excellence Awards (Post Secondary) 2004/06/09. An Automatic Video Information Processing and Conversion System for Multimedia Messaging Service and Mobile Internet. Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith Video over InternEt and Wireless Technologies Lab., CUHK. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/1.jpg)
An Automatic Video Information Processing and Conversion System for Multimedia Messaging Service and Mobile Internet
Miss Chan Pik Wah, PatMiss Ngai Cheuk Han, Edith
Video over InternEt and Wireless Technologies Lab., CUHK
6th IT Excellence Awards (Post Secondary)
2004/06/09
![Page 2: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/2.jpg)
VIEWLab, CSE, CUHK
Video over InternEt and Wireless Technologies Laboratory
VIEWLab is a research and application development laboratory Computer Science and Engineering, CUHK Prof. Michael R. Lyu
Focus on Video Information Processing Technologies Mobile and Wireless Application
Development
![Page 3: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/3.jpg)
VIEWLab, CSE, CUHK
Presentation Outline
MotivationFunctionality & DesignDemonstrationInnovationProspect for future commercializationQ&A
![Page 4: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/4.jpg)
VIEWLab, CSE, CUHK
Motivation
Needs of mobile video content is increasingVideo information processing technologies developedA prototype system is created to automate the video authoring processEnabling faster and cheaper mobile video entertainment
![Page 5: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/5.jpg)
VIEWLab, CSE, CUHK
Functionality & Design
An automatic video information processing and conversion system for MMS and Mobile InternetTargets Fully automation Mechanical processing with pre-defined
knowledge Various video content type Short processing time Configurable for different needs
![Page 6: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/6.jpg)
VIEWLab, CSE, CUHK
Overview of System
Advertisement Removal
Program Identification
Scene Analysis
MMS(Pictures +
Sound) /Mobile Video
Video Streaming / Progressive Download
MMS Component
Creation
Mobile InternetVideo Clip Conversion
TV Broadcast
![Page 7: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/7.jpg)
VIEWLab, CSE, CUHK
Video Information Processing
These are the technologies to extract information from video using visual and audio channelsBy extracting/identifying the information, “effective” condensation/summary techniques can be applied through information fusion or recognition stepsWe define a type of information a “modality”
Transcript/Speech On-screen characters Face Visual features
![Page 8: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/8.jpg)
VIEWLab, CSE, CUHK
Every Bit Counts
MMS size is limited to within hundred KB In order to delivery “useful” information to
viewer Condensation/summary techniques are required
to locate/prioritize the most important bits
Bandwidth cost is relatively higher than other channels the “value” of the content transmitted is curial
to successfully MMS business Utilize the “value” of the bandwidth use, only
transmitting the important bits
![Page 9: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/9.jpg)
VIEWLab, CSE, CUHK
Main Features of Video2MMS
Advertisement removal TV program identification Video condensation/summaryMMS creation and adaptation for different mobile handsets
![Page 10: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/10.jpg)
VIEWLab, CSE, CUHK
Advertisement Removal
Based on the logo detection techniques developed, TV program and non TV program segments can be separated automatically
Program
TV
Broadcast
Non Program Non Program
![Page 11: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/11.jpg)
VIEWLab, CSE, CUHK
TV Program Identification
Based on the visual features and audio characteristics, the desire TV program can be identified
![Page 12: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/12.jpg)
VIEWLab, CSE, CUHK
Video Condensation / Summary
Condense the long video into short slide show presentation Segment the video into different scene
Two pass minimum difference color histogram Choose the most representative shots
Video editing style Longest scene
Video Summary
![Page 13: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/13.jpg)
VIEWLab, CSE, CUHK
MMS Creation and Adaptation
Different handsets have different MMS size and screen size Nokia 7650, 3650 - 100KB, 176 x 136 SonyEricsson T68i, T300 – 30KB, 101 x 80 Panasonic GD88 – 45KB, 176 x 136
Convert the pictures to correct screen size Utilize the MMS size for most information (number of pictures and duration)Configurable for new handsets
![Page 14: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/14.jpg)
VIEWLab, CSE, CUHK
MMS Matrix
MMS Size = audio size + (picture size * n)Audio size = length of audio * 0.8k, (4.75 bit-
rate)Picture size = f(screen size, number of colors)
Handset MMS Size
Duration
Screen Size
Number of Color
AverageSize
Number of Pictures
Nokia 3650 100KB 60sec 174 x 136
128 10K 4 ~ 5
SonyEricsson T68i
30KB 30sec 101 x 80
32 3K 3 ~ 4
Panasonic GD88
45KB 45sec 176 x 136
64 5K 4 ~ 5
![Page 15: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/15.jpg)
VIEWLab, CSE, CUHK
Mobile Internet Video Clips Conversion
Control third party modules to cut and convert video into short video clipsDifferent handsets have different MMS size and screen size 3GPP format RealVideo format Microsoft Windows Media Format
Clips size configurable for different usage 100KB for MMS delivery >100KB for video streaming
![Page 16: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/16.jpg)
VIEWLab, CSE, CUHK
Demonstration Video (1)
(click to play)
ATV News Headline
AMR + Animated Gif
Nokia 3650
100KB
20 seconds
174 x 136
128 colors
Animated Gif (6 pic.)
AMR Audio
![Page 17: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/17.jpg)
VIEWLab, CSE, CUHK
Demonstration Video (2)
(click to play)
ATV News Headline
3GP for Mobile Internet
Nokia 3650
195KB
20 seconds
174 x 136
256 colors
![Page 18: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/18.jpg)
VIEWLab, CSE, CUHK
Demonstration Video (3)
(click to play)
ATV Weather Report
AMR + Animated Gif
Nokia 3650
100KB
60 seconds
174 x 136
128 colors
Animated Gif (5 pic.)
AMR Audio
![Page 19: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/19.jpg)
VIEWLab, CSE, CUHK
Demonstration Video (4)
(click to play)
ATV Hourly News
AMR + Animated Gif
Nokia 3650
100KB
40 seconds
174 x 136
128 colors
Animated Gif (6 pic.)
AMR Audio
![Page 20: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/20.jpg)
VIEWLab, CSE, CUHK
Demonstration Video (5)
(click to play)
Horse Racing
3GP in MMS
Nokia 3650
95KB
20 seconds
174 x 136
![Page 21: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/21.jpg)
VIEWLab, CSE, CUHK
Demonstration Video (6)
(click to play)
Horse Racing
RealVideo in MMS
Nokia 3650
95KB
20 seconds
120 x 96
![Page 22: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/22.jpg)
VIEWLab, CSE, CUHK
Demonstration Video (7)
(click to play)
Soccer Goal Shot
RealVideo in MMS
Nokia 3650
92KB
5 seconds
120 x 96
![Page 23: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/23.jpg)
VIEWLab, CSE, CUHK
System Demonstration
ModulesEditor
Main Panel
Task Scheduler
TaskProfile
![Page 24: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/24.jpg)
VIEWLab, CSE, CUHK
Innovation
We believed this is the first application applied Video Information Processing
technologies on automating the content
preparation processes for mobile handset in Hong Kong
![Page 25: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/25.jpg)
VIEWLab, CSE, CUHK
Prospect for future commercialization
Base on this prototype, our laboratory technology commercialization team developed a production system called Video2MMS™Video2MMS™ was licensed to HKCSL in spring 2003 to prepare daily news and weather video clips and MMSPress release http://www.hkcsl.com/show.jsp?
pid=4&cnid1=407&type1=1&rhtID1=3&lid=2
![Page 26: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/26.jpg)
VIEWLab, CSE, CUHK
Tested Handsets
MMS Nokia 7560, 3650 SonyEricssion T68i, T300 Panasonic GD88
VideoClips (Download) MS PocketPC, XDA
![Page 27: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/27.jpg)
VIEWLab, CSE, CUHK
Recommended System Requirements
Intel P4 1.0G Hz 512MB RAM Microsoft Windows 2000 or above DirectX 8.1 TV Capture Card (MPEG-1) Optional third party software
Real Encoder Nokia 3GP Encoder Sony Ericsson AMR Convertor Microsoft Windows Media Encoder
![Page 28: Miss Chan Pik Wah, Pat Miss Ngai Cheuk Han, Edith](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813b47550346895da42a68/html5/thumbnails/28.jpg)
VIEWLab, CSE, CUHK
Q&A