keyframe-based video summarization designer
Post on 27-Jan-2017
434 Views
Preview:
TRANSCRIPT
KEYFRAME-BASED VIDEO SUMMARIZATION DESIGNER
Carlos Ramos Caballero
Advisors: Horst Eidenberger and Xavier Giró I Nieto
Introduction
Motivation
Designer Master: keyframe-based video summarization interface Object Maps: system for automatic video summarization
5
Graphical User Interface (Designer Master)
Computer Vision Engine (Object Maps)
Introduction
Goals of the thesis Improving the keyframe extraction module Assessing the improvement
8
State of the art
Shot segmentation
10
Hierarchical decomposition and representation of video content [1]
[1] http://www.scholarpedia.org/article/Video_Content_Structuring
State of the art
Shot segmentation example
11
Shot boundary detection example [2].
[2] Martos, M. “Content-based Video Summarization to Object Maps”, Vienna University of Technology, Austria (2013).
State of the art
Shot segmentation techniques
Pixel-to-pixel methods
• Global pixel-to-pixel • Cumulative pixel-to-pixel
Histogram-based methods • Simple histogram • Maximum histogram • Weighted histogram
Hausdorff method
12
Methodology : Implemented solution
Uniform sampling
𝑓𝑝𝑠𝑖: frame rate of the input video.
𝐿𝑖: total number of frames of the input video.
𝑁0: total number of frames we want to keep (𝑁0=100).
15
Methodology : Implemented solution
Difference computation
Where 𝐼(𝑡,𝑖,𝑗) represents the intensity value at frame t in pixel(𝑖,𝑗).
X and Y are the width and height of the video frames, respectively.
17
Methodology : Implemented solution
Normalization
Where 𝑑 ̂ is the normalized value, 256 is the number of grey levels, X and Y are the width and height of the video frames, respectively.
18
Methodology : Implemented solution
Decision making
The threshold value used in our application is 𝜏 = 0.1 (as defined in [2]).
19
[2] Martos, M. “Content-based Video Summarization to Object Maps”, Vienna University of Technology, Austria (2013).
Results assessment
TEST 1: Testing the applications + ‘in situ’ survey 11 participants Test data: The intouchables trailer
22
Results assessment
EVALUATION Quality of the generated summaries Representativeness of the generated summaries Mean Opinion Score
• 1. Unacceptable • 2. Poor • 3. Good • 4. Very good • 5. Excellent
25
Results assessment
Representativeness of the summaries
“Which summary let you better recognize the video content?”
28
Results assessment
Ease-of-use of the application
“Do you think the application is intuitive and easy to use?”
30
Conclusions
Accomplishment of the initial goals Improving the keyframe extraction module by integrating both
projects. Assessing the improvement.
34
Conclusions
Accomplishment of the initial goals Improving the keyframe extraction module by integrating both
projects. Assessing the improvement.
Our work has slightly improved Designer Master Users can create better video summaries and easily due the better
quality of the extracted keyframes.
35
Conclusions
Accomplishment of the initial goals Improving the keyframe extraction module by integrating both
projects. Assessing the improvement.
Our work has slightly improved Designer Master Users can create better video summaries and easily due the better
quality of the extracted keyframes.
It is hoped to develop this work into a product for the Austrian Broadcasting station ORF
36
Conclusions
Accomplishment of the initial goals Improving the keyframe extraction module by integrating both
projects. Assessing the improvement.
Our work has slightly improved Designer Master Users can create better video summaries and easily due the better
quality of the extracted keyframes.
It is hoped to develop this work into a product for the Austrian Broadcasting station ORF
37
top related