player action recognition in broadcast tennis video with applications to semantic analysis of sport...
TRANSCRIPT
Player Action Recognition in Broadcast Tennis Video with Applications
to Semantic Analysis of Sport Game
Guangyu Zhu, Changsheng Xu Qingming Huang, Wen Gao
Liyuan Xing
Outline
• Introduction
• Framework Overview
• Player Action Recognition
• Video Analysis
• Experimental Results
Introduction
• Semantic gap– between user semantics and low-level
feature– Object in sports video can consider as
an effective mid-level representation
• Action Recognition– Far-view– Foreside-swing backside-Swing
Introduction
• Multimodal Framework– Action recognition method based on
motion analysis– High-level analysis
• Video Indexing• Highlight ranking• Tactic analysis
Framework Overview
• Sports video database
• Low-level analysis
• Middle-level analysis
• Fusion scheme
• High-level analysis
Low-level Analysis
• Dominant color-based algorithm in [16] was used to identify all the in-play shots
Player Action Recognition
• Related Work– Shah[8], Gavrila[9] recognition with close-up
views– Motion representation
• Motion history/energy image [12]• Spatial arrangement of moving points [13]• Several Constraints
– Efroes[11]• Motion descriptor in a spatio-temporal volume• NNC similarity measure
– Miyamori[14][15]• Base on silhouette transition• Appearance feature is not preserved across videos
Player Tracking and Stabilization
• Player Tracking– Initial position: detection algo. in [16]– SVR particle filter [24]
• Player region centroid
Local Motion Representation
• S-OFHs– slice based optical flow histogram
• The prob. of bin(u)
• The prob. of bin(u) in slice
Local Motion Representation
• Two slice of the figure is used• Horizontal and vertical optical field is used
Action Classification
• Using SVM• The concatenation of four S-OFHs is fed
as feature vector• Audio keywords
– Silence, hitting ball, applause
Video Analysis
• Fusion of mid-level features
• Action Based Tennis Video Indexing
• Highlights Ranking and Browsing
• Tactics Analysis and Statistics
Highlights Ranking
• Affective Features(4 for this paper)• Features on action
– Swing Switching Rate
Highlights Ranking
• Features on trajectory– Speed of Player (SOP)– Maximum Covered Court
• The rectangle shaped with left most, rightmost, topmost, and bottommost points
– Direction Switching Rate
Highlights Ranking
• The feature vector comprised of four affective features is fed into the ranking model
• Support vector regression• User defined threshold
Future Work
• More effective slice partition• Involve more semantic action
– Ex. Overhead-swing
• Action recognition apply to more applications such as 3-D scene reconstruction
• Include the ranking accuracy by combining audio features