semantic scenes detection and classification in sports videos soo-chang pei ( 貝蘇章 ) and fan...
TRANSCRIPT
![Page 1: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/1.jpg)
Semantic Scenes Detection and Classification
in Sports Videos
Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 )
Conference on Computer Vision, Graphics and Image Processing (CVGIP 2003)
指導老師:吳宗憲 教授 組員 p76974050 王志宏 p76974238 趙郁婷 p76974555 蔡佩珊
1
![Page 2: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/2.jpg)
Outline
× Introduction× Method
■ Scene change detection■ Semantic scenes detection in baseball
game videos
× Experimental Result× Discussion
2
![Page 3: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/3.jpg)
Introduction
× Motivation
3
![Page 4: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/4.jpg)
Introduction (Cont.)
4
Infield
Outfield
Player
Pitching
? Pitching scene
![Page 5: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/5.jpg)
Introduction (Cont.)
× Index and retrieval the baseball video× Semantic scene detection methods
■ Combining with domain-specific knowledge■ Index keyframes by low-level features
5
Skeleton outline of our semantic scenes detection
![Page 6: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/6.jpg)
Introduction (Cont.)
× Video content analysis methods may be classified into the following three categories:■ Syntactic structurization of video■ Video classification■ Extraction of semantics
× Don’t use video object tracking in object level verification in order to reduce the complexity
× Use object-location to verify which view type the keyframe belongs to
6
![Page 7: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/7.jpg)
Method
× Scene change detection
× Semantic scenes detection in baseball game
videos
7
![Page 8: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/8.jpg)
Scene change detection
× Execute the action of scene change detection and key frame extraction
■ IBM VideoAnnEx Annotation Tool
•annotate video sequences with MPEG-7 metadata
8
![Page 9: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/9.jpg)
Semantic scenes detection
× Analysis and structuring in baseball■ A play usually starts with a pitching scene■ After the play starts, if after a scene change the
camera is shooting the field, then the current play should continue; otherwise, the current play ends when switched to a player scene
9The model of baseball broadcasting videos
the play continuousthe play is end
![Page 10: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/10.jpg)
Semantic scenes detection (Cont.)
× Block diagram
10
![Page 11: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/11.jpg)
Semantic scenes detection (Cont.)
× Field color percentageField color percentage■ Detected field color distribution and percentage
■ Three situations• Medium(20%~45%) : pitching scene• Large (>=45%) : outfield scene or an infield scene• Small (<20%) : close-up scene or others
11
grass color range : 0.19<H<0.46 、 0.2<S<0.7 、 V>100soil color range : 0.06<H<0.15 、 0.25<S<0.8 、 V>100
grass color range :34.2<H<82.8 、 51<S<178.5 、 V>50soil color range : 0<H<22 、 63.75<S<204 、 V>100
![Page 12: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/12.jpg)
Semantic scenes detection (Cont.)
× Block diagram --- Field color percentage
12
20~45%>45%
<20%
![Page 13: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/13.jpg)
Semantic scenes detection (Cont.)
× Pitching scene verificationPitching scene verification■ First build a binary image by assigning field color to
1-pixel and non-field color to 0-pixel
13
(a) original pitching scene (b) the result of binarizing(c) histogram of horizontal projection
T = 0.15
Horizontal projection histogram
![Page 14: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/14.jpg)
Semantic scenes detection (Cont.)
× Pitching scene verificationPitching scene verification
14
(a) original pitching scene (b) the result of binarizing(d) histogram of vertical projection
Vertical projection histogram
m = 15M = 100
![Page 15: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/15.jpg)
× Block diagram --- Pitching scene verification
15
Semantic scenes detection (Cont.)
![Page 16: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/16.jpg)
Semantic scenes detection (Cont.)(Cont.)
× Close-up scene detectionClose-up scene detection■ A close-up scene always target on one’s
face • Face detection is a key point
■ Skin color can be segmented out of an image
• Hues are between 0 and 50 degrees and saturation between 0.23 and 0.68
16
![Page 17: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/17.jpg)
Semantic scenes detection (Cont.)
× Close-up scene detectionClose-up scene detection■ Two step
• Step 1 : Label skin color to 1-pixel, or label to 0-pixel
• Step 2 : Find the largest region in our defined “red-block”
17
depicts the skin color distribution
depicts the region considered as face region
![Page 18: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/18.jpg)
Semantic scenes detection Semantic scenes detection (Cont.)(Cont.)
× Block diagram --- Close-up scene detection
18
![Page 19: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/19.jpg)
Semantic scenes detection (Cont.)(Cont.)
× Player scene detectionPlayer scene detection■ Player scene : Lead role is a figure but
background is composed of lots of field components
19
Can be taken as a scene between infield or outfieldand close-up scene
![Page 20: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/20.jpg)
Semantic scenes detection (Cont.)(Cont.)
× Player scene detectionPlayer scene detection■ Field color percentage is large and there are
some big concaves in the vertical projection diagram
20
K
![Page 21: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/21.jpg)
Semantic scenes detection (Cont.)
× Block diagram --- Player scene detection
21
![Page 22: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/22.jpg)
Semantic scenes detection (Cont.)(Cont.)
× Infield and outfield scene detectionInfield and outfield scene detection■ Calculate the ratio of grass to soil
Infield scene Outfield scene 22
GrassRatio
Soil
>5≦ 5
![Page 23: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/23.jpg)
Experimental Result
× Data : MPEG-1× Frame size : 360*240× Frame rate : 30Hz
23
![Page 24: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/24.jpg)
Experimental Result (Cont.)
× Result
24
Original Binarizing
Infield_scene
![Page 25: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/25.jpg)
Experimental Result (Cont.)
× Result
25
Original Binarizing
Outfield_scene
![Page 26: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/26.jpg)
Discussion
× pitching 畫面會因為土和草的比率而偵測有誤× 當投手站於右方時,可能造成錯誤
26
![Page 27: Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697bfc91a28abf838ca8c9a/html5/thumbnails/27.jpg)
27