entropy features trained support vector machine based logo

International Journal of Graphics and Multimedia (IJGM), ISSN 0976 – 6448(Print),

ISSN 0976 – 6456(Online) Volume 4, Issue 1, January - April 2013, © IAEME

20

ENTROPY FEATURES TRAINED SUPPORT VECTOR MACHINE

BASED LOGO DETECTION METHOD FOR REPLAY DETECTION

AND EXTRACTION FROM SPORTS VIDEOS

Vilas Naik1, Raghavendra Havin

2

1 Department of CSE, Basaveshwar Engineering College, Bagalkot, India

2 Department of CSE, Basaveshwar Engineering College, Bagalkot, India

ABSTRACT

In many sports, the majority of highlights are confined to relatively short durations of

intense action. In some sense these segments capture the essence of a game and summarize

the moments of important action. Automatic detection of these highlights could provide an

important browsing mechanism in a video library of sports games. These replays often cor-

respond to highlights in a game and can be used as indices of a sports video. The proposed

mechanism employs support vector machine (SVM) for detection of logos that are flashed at

beginning and end of every reply action. The algorithm is composed of logos detection and

replay segment extraction. First SVM is trained with features of all possible logos normally

used in various sports videos. Then the SVM is used for detection of logos that sandwich

replay segment further that segment is automatically extracted to produce replay clip. The

SVM classifier is trained with histogram features of logos is utilized. Experiments conducted

on IPL and Soccer videos demonstrate the effectiveness of this method. Moreover, algorithm

can be easily applied to other sports videos where replay is sandwiched by pair of logos.

Keywords: Logo-based detection, Support vector machine (SVM learning (ML), Sports

replay detection, Video summarization.

1. INTRODUCTION

In a sports video library, one may find thousands of hours of recordings, representing

many hundreds of individual games. Fortunately, in any given game, not every minute com-

mands the same interest. A majority of the excitement is contained in a small fraction of the

coverage of the game. These highlights are the essence of the game and present a succinct

INTERNATIONAL JOURNAL OF GRAPHICS AND

MULTIMEDIA (IJGM)

ISSN 0976 - 6448 (Print) ISSN 0976 -6456 (Online) Volume 4, Issue 1, January - April 2013, pp. 20-30 © IAEME: www.iaeme.com/ijgm.asp

Journal Impact Factor (2013): 4.1089 (Calculated by GISI) www.jifactor.com

IJGM

© I A E M E



21

summary of the significant actions and events. For many viewers of sports video these high-

lights are all they are really interested in seeing. Detection of these highlights is therefore of

interest in itself, but might also serve as an important indexing mechanism for searching for

more substantial sections of material from large archives. With the rapid increase of media

data, it is in urgent need of an efficient and effective method for information management and

retrieval. Semantics based event detection and indexing has been attracted much attention

recently for interesting events are highly useful for video browsing, indexing and highlights

generation. Replay is an important video editing way in broadcasting programs. Commonly,

an important or interesting segment will be played with a slow-motion pattern in order to let

audiences explore and enjoy the detail. Replay is a significant indication for highlights and

often taken as a key factor in event detection. The sports video summarization is widely in-

vestigated since sports holds a large amounts of audiences worldwide. As an important com-

piled clue, the replay shows the details of an important video segment with a slower speed.

Thus it is widely employed in sports video analysis especially for event detection, highlight

creation and summarization etc.

Replays are related to the sports, which are shown more than once when any impor-

tant event in sports takes place. To get more information from replay, generally it is shown

with slow motion. Hence it is also called slow motion replay (SMR). Replay is which is ne-

cessary for highlights generation. Highlight is a shorter version of sports video that is also

called as sports video summarization. Replay is important for all spectators and referees to

judge and take final decision in most of the sports. When an important event happen like goal

in football or hockey cricket. When referee is in doubtful condition, or strong appeal is made

by players, putting all such events in an original temporal order we can have highlights of any

sports video. Replay contains all-important events. Replays are used for highlight making.

Replays can be used as a summary of game for searching it in digital video library.

Replay is an important video editing way in broadcasting programs. Commonly, an

important or interesting segment will be played with a slow-motion pattern in order to let au-

diences explore and enjoy the detail. Replay is a significant indication for highlights and of-

ten taken as a key factor in event detection.

In recent years, researchers have reported many approaches of replay detection in lite-

ratures. In this paper, an automatic and effective logo based replay detection method is pro-

posed. It is well known that in most cases there exists a special transition at both start and end

of a replay, in which a highlighted logo comes in and out gradually. This transition is called

as “logo-transition”. Logos often keep the same during the whole program. Our motivation is

to use the logos to assist replay segments detection. Firstly some logo-transitions are de-

tected and further extract logo-samples from them at the beginning of a program. After that,

the logo-template is extracted from these samples. Further, this template is employed to

detect the other logos. After all logos are obtained, the video can be partitioned into segments

with taking logos as boundaries. Then, shot and motion features are extracted in each seg-

ment. Finally, based on these cues, an SVM classifier is used to replay segment identification.

2. RELATED WORK

Slow motion replays in broadcast sports video are important and reliable clues for

highlights and key events, since they are usually chosen by human experts. Therefore, detec-

tion of replays could facilitate semantic-based high level video processing tasks such as high-

light generation [1], event detection [2], video summarization [3], etc.



22

Some slow motion replay detection methods have been proposed in the literature.

They can be broadly classified into three categories. The first category of method directly

analyzes the inherent attributes of replay video segments and tries to use those attributes to

differentiate the slow motion replays from the normal plays. Before high speed cameras are

widely used, slow motion replays were usually produced by repeating frames recorded from

standard cameras. A few detection methods are based on this assumption. A replay detection

method based on the macroblock type, motion vector and bit rate information is presentd in

[4]. The detection is performed in the MPEG compressed domain to find the locations of re-

peating frames. The algorithm in [1] calculates the image differences between adjacent

frames and search fluctuations in the differences to identify replays. These methods usually

get low performance on replays that captured by high speed cameras and they can not provide

accurate boundaries of replays.

The second category of method employs statistical techniques to perform shot classi-

fication for the purpose of detecting replay shots. The author of [5] presented a method based

on the difference of motions between replays and normal plays. A support vector machine

(SVM) was trained with features of color ratio, shot length, mean color value and motion re-

lated feature to detect replay shots. The proposal in. [6] applied scene transition structure

analysis on the shot classification results. Replays were extracted based on the generated shot

label sequences by using some predefined rules. A HMM to classify replay and non-replay

shots is employed in [7]. The performance of this category is usually not satisfied and results

are still not precise enough.

The third category of method tries to find the differences between replays and normal

plays in terms of specific production behaviors, such as special video effects or logo patterns.

Replays are then determined according to the existence of such behaviors. The work pre-

sented in [8] captures a digital video effect (DVE) l interactively for given videos. It is de-

scribed by color and motion of the gradually changing boundary between adjacent shots. A

replays are located by two DVEs detected with the model. Based on the observation that

more and more replays that are sandwiched in between two certain logo transition sequences,

some researchers try to use logo transition locations to identify replays. The mean-square dif-

ferences of intensity and color histogram as frame similarity measure to detect logo template

and to search logos in videos is discussed in[9]. A method in [10] employed the spatiotem-

poral mode seeking on selected logo sequences to capture the dominant color mode. This

color mode is then used to perform EMD-based similarity matching for logo searching pur-

pose. Tong et al. [11] used frame-to-frame difference with MSD to select candidate logo se-

quence. They calculated the average image of those candidate logo frames near to the center

to obtain logo template. Template matching was performed based on color and shape fea-

tures. Huang et al. [12] first extracted gradual transitions and employed motion features to

learn the logo template. A color representation was then computed from the logo template

and was used to search logos. Dang et al. [13] semiautomatically extracted logo template se-

quences and detected replays based on template sequence matching. Han et al. [14] learned

gradual transition patterns based on motion vector reliability classification from a database by

SVM. Replay detection was performed by fusing the pattern matching results and slow mo-

tion detection results based on motion vector information. The third category of method can

usually obtain high accuracy on replay boundaries and the performance is better than above

two categories. However, these methods often make assumptions on the logo transition se-

quences such as logo locations, moving patterns and speed, types of digital effects, etc. And



23

nearly all of them assume that the two logo transition sequences before and after the replay

are identical.

The Proposed solution is a methodology of determining the replay detection in a

sports video using support vector machine for classifying frames as Logo frame and non logo

frame. In most of sports video a logo swaps at beginning and end of the replay, the frames in-

between the start logo frame to its corresponding end logo frame of a videos construct a rep-

lay event. Applying same procedure for total video it results into collection of replays of

sports video. These replays can be used to form a summary of video.

3. PROPOSED ALGORITHM FOR REPLAY DETECTION

The proposed method employs a Support Vector Machine based classifier for detect-

ing the logos that will swap across the screen in broadcasted video at start and end of the rep-

lay action.

The proposed algorithm consist of mainly TWO stages

i. Train the SVM classifier with logo patterns and non logo patterns for classifying

the frames in the video as logo frames and non logo frames.

ii. Detecting logos at start and end of the replay action and extracting the frames be-

tween them.

The flow charts for two phases of proposed algorithm are presented in Fig. 1.

Fig. 1 Flow diagram of proposed Algorithm, (a) Flow diagram for Training SVM Clas-

sifier (b) Flow diagram for Replay Detection and Extraction.

Train the SVM Classifier with Entropy

Features of Logo frame and Non logo

frame

Start

Extract Logo frames from the Input

Video and take some Non logo frames

Extract the Entropy Features of a

frame

END

Read the Input Video

Extract the Entropy Features of each

frames

Extraction of frames between two

logos

Test the frame on trained SVM Clas-

sifier

Detection of Two logos at beginning

and end of each replay event

Start

END



24

In most of the sports video typical logo-transition lasts 0.5-0.8 seconds or 5-24 frames. In

logo-transition, there is an image frame that contains the clearest and the most complete logo.

Usually, the logo is highlighted and located at the middle part of a frame. With these prior

knowledge, the logo-samples from video can be detected and extracted.

3.1 Logo-transition Detection A typical logo-transition lasts 0.5-0.8 seconds or 15-24 frames. The frame-to-frame

difference is measured by intensity mean square difference (MSD). The difference sequence

after median filtering is a plateau-like shaped pattern (Fig. 3). Therefore, the logo-transition

detection can be performed through plateau-like shape detection over the frame-to-frame dif-

ference sequence. The proposed algorithm is

i. Compute frame-to-frame difference with MSD.

ii. Check the frame difference. Proceed to step 3 When the difference exceeds a

threshold, otherwise go back to step 1) for next frame.

iii. Count the number of consecutive frame-differences that all exceed the frame-

difference-threshold until encounter several consecutive frame-differences that

drop below this threshold. If the counter exceeds a certain threshold, a wipe

transition can be determined. Otherwise go back to step 1) for next frame.

Fig. 2 A logo-transition (8 frames are displayed)

Fig. 3 Frame difference sequences.

Original(left) and after median filtering(right)

3.2 Logo-sample Determination In a logo-transition, there is an image frame that contains the clearest and the most

complete logo. Usually, the logo is highlighted and located at the middle part of a frame.

With these prior knowledge, the logo-samples can be extracted.



25

3.3 Feature Extraction The color, texture, shape and motion estimation have been commonly used features in

the literature, It is observed that color and motion features play a dominant role in the extrac-

tion of characteristics from the videos and hence the color information is adopted in this

work. Normally color distribution is estimated in the form of histogram, the proposed work

estimates the information of color content at different color intensity planes of images in

terms of entropy values. The entropy computed for a color plane in an image gives the aver-

age information conveyed by an image. In the pr oposed work the entropy of three layers of

RGB are computed and feature vector is written as in equation (1)

(1)

Where

: Feature vector of frame from video sequence

: Entropy of Red plane

: Entropy of green plane

: Entropy of blue plane

of an RGB frame from video sequence.

The feature vectors as above are constructed for Logo templates extracted from vari-

ous videos and some Non logo frames and are used for training SVM.

3.4 Support Vector Machine (SVM) The support vector machine (SVM) algorithm seeks to maximize the margin around a

hyperplane that separates a positive class from a negative class [20]. Given a training dataset

with n samples (x1, y1); (x2,y2)……….(xn, yn), where xi is a feature vector in a v-

dimensional feature space and with labels yi � {-1,1}belonging to either of two linearly se-

parable classes C1 and C2. Geometrically, the SVM modeling algorithm finds an optimal

hyper plane with the maximal margin to separate two classes, which requires to solve the op-

timization problem, as shown in equations (2) and (3).

where, αi is the weight assigned to the training sample xi. If αi > 0, xi is called a sup-

port vector. C is a regulation parameter used to balance the training accuracy and the model

complexity so that a superior generalization capability can be achieved. K is a kernel func-

tion, which is used to measure the similarity between two samples. Different choices of ker-

nel functions have been proposed and extensively used in the past and the most popular are

the gaussian radial basis function (RBF), polynomial of a given degree, and multi layer per-

ceptron. These kernels are in general used, independently of the problem, for both discrete

and continuous data.

(2)

(3)



26

3.5 SVM classifier based Replay detection Algorithm

The proposed algorithm model uses priorly retrieved logos of sports video for the

training, then its features i.e its entropy values are extracted and trained to SVM classifier.

SVM is used as a classifier due to its clear classification method. Some randomly selected

frames without logo are also trained to SVM classifier as Non Logo templates.It classify two

Class vectors by a linear hyper plane , by which classifier can easily classify Logo and Non

Logo frames.

Fig. 4 Flow chart of replay detection Algorithm

READ Video file (.avi,.mpeg) frame wise

LOGO_COUNT =0

Start

Extract all the frames between Current

LOGO detected and previous detection

write them to file which gives a replay

event.

Is LOGO

frame detected

��

Extract Entropy features of current frame

Test the Frame with trained SVM for

LOGO or NON LOGO frame

For LOGO frame LOGO_COUNT =1 for

NON LOGO LOGO_COUNT =2

End



27

Algorithm:

Step 1: Extract the Logo frames from the input videos as explained in above sections.

Step 2: Extract the entropy features of a frames as given in section 3.3.

Step 3: Store these entropy features into vectors as feature patterns.

Step 4: Train the SVM Classifier with the feature patterns.

Step 5: Include non logo frames features in SVM training set.

Step 5: The pair of logos can be detected as described in flowchart of Fig. 4.

Step 6: Replay is Detected between a pair of logos.

Flow Diagram of Testing Phase to Identify an Logo Image in a Sports Video and Replay

Detection Phase is shown in Fig. 4

4. EXPERIMENTAL RESULTS AND DISCUSSIONS

In the proposed work experimentation is conducted on data set of ten videos compris-

ing of soccer and IPL videos. The dataset of all videos logo frames are extracted from all ten

videos and are shown in next page in Fig.5, the features of which are used to train support

vector machine. The SVM trained with these features is employed for detection of pairs of

logos at the beginning and end of each replay action in soccer and cricket videos. The sum-

mary of experimentation is presented in Table 1.

.



28

Fig. 5 Logo frames trained to SVM classifier The Table 1 summarizes the performance of proposed algorithm for replay detection

over ten sample sports video out of which 3 are soccer video and 7 are cricket video. The proposed

mechanism for replay detects the two logos that sandwich the replay action. The logos are detected

by SVM classifier by testing every frame of the video. The video segment between two consecu-

tive logos is picked as replay. The average values of performance parameters recall and precision

up to 94% and 97% for the algorithm indicates that the proposed mechanism is suitable for ex-

traction of replay actions in sports video and construct summary of sports videos.



29

TABLE1 Results of proposed algorithm for 10 test video samples

5. CONCLUSION

The proposed algorithm for SVM based method for replay detection and extraction

from sports video is designed and experimented with sufficient number of soccer and cricket

clips. The algorithm is implemented in Matlab 2010b and executed on Pentium® P6200 pro-

cessor with 2GB RAM memory. The replay detection and extraction in soccer and cricket

games are tested well with a sufficient number of video clips. The Proposed algorithms iden-

tify a replay by finding a pair of logos and hence extract the replay by extracting all frames

in-between the pair of logos. The proposed method identify a pair of logos by matching a

entropy features of trained logos with the each frames histogram values of input videos. The

proposed work uses support vector machine to classify between true logo and non logos. The

replays are used as highlights of a game, and by which we formed a summary of sports video

by these highlights.

REFERENCES

[1] H. Pan, P. van Beek, and M.I. Sezan, “Detection of slowmotion replay segments in sports

video for highlights generation,”in Proc. of ICASSP'01, vol. 3, pp. 1649–1652, 2001.

[2] X. F. Tong, H. Q. Lu, and Q. S. Liu, “A three-layer event detection framework and its

application in soccer video,” in Proc.of ICME'04, vol. 3, pp. 1551–1554, 2004.

[3] A. Ekin, A. M. Tekalp, and R. Mehrotra, “Automatic soccer video analysis and summari-

zation,” IEEE Transactions on Image Processing, vol. 12, no. 7, pp. 796–807, 2003.

[4] V. Kobla, D. DeMenthon, and D. Doermann, “Detection of slow-motion replays for iden-

tifying sports videos,” in Proc. of MMSP'99, pp.135–140, 1999.

[5] L. Wang, X. Liu, S. Lin, G. Y. Xu, and H.-Y. Shum, “Generic slow-motion replay detec-

tion in sports video,” in Proc. of ICIP'04,vol.3, pp. 1585–1588, 2004.

[6] J. J. Wang, E. Chng, and C. S. Xu, “Soccer replay detection using scene transition struc-

ture analysis,” in Proc. of ICASSP'05,vol.2, pp. 433–436, 2005.

[7] Y. Yang, S. X. Lin, Y. D. Zhang, and S. Tang, “A statistical framework for replay detec-

tion in soccer video,” in Proc. Of ISCAS'08, pp. 3538–3541, 2008.

Video File

Name

Actual Num-

ber of Replays

Total Num-

ber of replay

correctly

detected

replays

Undetected

replays

False

detected

replays

Recall Precision

Video1.avi 10 9 1 1 90.00% 90.00%

Video2.avi 60 56 4 2 93.33% 96.55%

Video3.avi 20 20 0 0 100.00% 100.00%

Video4.avi 34 31 3 1 91.17% 96.85%

Video5.avi 8 7 1 0 87.50% 100.00%

Video6.avi 15 14 1 1 93.33% 93.33%

Video7.avi 19 19 0 0 100.00% 100.00%

Video8.avi 10 10 0 1 100.00% 90.90%

Video9.avi 25 23 2 0 92.00% 100.00%

Video10.avi 12 12 0 0 100.00% 100.00%



30

[8] N. Babaguchi, Y. Kawai, Y. Yasugi, and T. Kitahashi, “Linking live and replay scenes in

broadcasted sports video,” in Proc. of MULTIMEDIA'00, pp. 205–208, 2000.

[9] H. Pan, B. Li, and M. I. Sezan, “Automatic detection of replay segments in broadcast

sports programs by detection of logos in scene transitions,” in Proc. of ICASSP'02, pp. 3385–

3388, 2002.

[10] L.-Y. Duan, M. Xu, Q. Tian, and C.-S. Xu, “Mean shift based video segment representa-

tion and applications to replay detection,” in Proc. of ICASSP'04, pp. 709–712, 2004.

[11] X. F. Tong, H. Q. Lu, Q. S. Liu, and H. L. Jin, “Replay detection in broadcasting sports

video,” in Proc. of ICIG'04, pp.337–340, 2004.

[12] Q. Huang, J. M. Hu, W. Hu, T. Wang, H. L. Bai, and Y. M. Zhang, “A reliable logo and

replay detector for sports video,” in Proc. of ICME'07, pp. 1695–1698, 2007.

[13] Z. H. Dang, J. Du, Q. M. Huang, and S. Q. Jiang, “Replay detection based on semi-

automatic logo template sequence extraction in sports video,” in Proc. of ICIG'07, pp. 839–

844, 2007.

[14] B. Han, Y. Yan, Z. H. Chen, C. Liu, and W. G. Wu, “A general framework for automatic

on-line replay detection in sports video,” In Proc. of MM'09, pp. 501–504, 2009.

[15] Reeja S R and Dr. N. P Kavya, “Motion Detection for Video Denoising – The State of

Art and the Challenges”, International Journal of Computer Engineering & Technology

(IJCET), Volume 3, Issue 2, 2012, pp. 518 - 525, ISSN Print: 0976 – 6367, ISSN Online:

0976 – 6375.