web viewwith the rapid development of modern electronic ... basis of similarity measurement. ... and...

I. INTRODUCTION

With the rapid development of modern electronic equipment, the amount of video data increase tremendously in recent years. For the video browsing, editing and sharing, video similarity search (VSS) which can provide content based video retrieval service in large storage device has been proposed as important research area.

Video similarity in this paper means visual similarity which is one of basis of content-based video retrieval. Besides, the visual similarity does not contain strict restriction on video objects and scenes because they are related to users’ interests. Video similarity search includes two important aspects: one is similarity measure and another is efficient search technique. The feature representation is the basis of similarity measurement.

Video similarity in this project means visual similarity which is one of basis of content-based video retrieval. Besides, the visual similarity does not contain strict restriction on video objects and scenes because they are related to users’ interests.

Literature Study:

ViSig: method was introduced based on randomized summarization which randomly selects a number of seed frames and assigns a small collection of closest frames to each seed.

Gao et al. proposed a video summarization tool to remove redundant video content with the hierarchical agglomerative cluster in the shot level.

Shen et al. employed bipartite graph to map relationship between the query and database video. The irrelevant subsequences to the query were pruned by a filter-and-refine search strategy.

Besides, several other feature extraction algorithms such as distinctive local descriptors, color fingerprint, DCT and visual rhythm spectrum etc were also proposed. For similarity measurement, many approaches have also been proposed such as neighborhood similarity considering local sample and label distribution, distance measure based on Kullback-Leibler divergence and high-dimensional statistical measure according to probability density function etc. Though these techniques are quite valuable for the further studies on image or video feature extraction and similarity measure, they are usually too complicated to accomplish feature extraction in large video database.

An Efficient Video Search Algorithm Project Report 1

Methodology:

Video similarity search includes two important aspects: one is similarity measure and another is efficient search technique. The feature representation is the basis of similarity measurement. In the past decade, the effective feature representation of image and video has been extensively studied.

Efficient indexing and search is another important aspect of video similarity search. Recently, there were numerous efficient strategies designed to improve similarity search in database, such as max nearest distance, local constant embedding and optimal indexing etc. The indexing methods based on locality sensitive hashing (LSH) have been widely used for approximate neighbor search problem in high-dimensional space. However, it has been shown in that when the dimensionality exceeds about ten, the indexing data structures based on space partitioning are slower than the brute-force, linear-scan approach. The existing LSH techniques, usually containing numerous of hash tables, are inefficient for scalable distributed computing and fast query in large database. Although these techniques provide valid solutions for high-dimensional search in large database, the search methods with complex index structure are not efficient enough for fast search in large database.

In this paper, an efficient VSS approach is proposed. The feature extraction of image and video was achieved by image characteristic code (ICC) based on the statistics of spatial temporal distribution. The similarity of video was measured according to the computation of the number of similar video components. Besides, a fast search approach for scalable computing was presented based on clustering index table (CIT). By virtue of proposed feature computation and index clustering, the VSS can be implemented very efficiently with satisfying recall and precision rate. The experimental results indicate this approach performs well in large database.


II. VIDEO FEATURE COMPUTATION AND SIMILARITY

A. Feature Computation

Video feature computation is generally based on image feature extraction. Several low-level features such as color, texture, edge are usually adopted for image fingerprint. It has been shown that YCbCr histogram is an effective video signature because of its valuable information for video matching. Besides, YCbCr coding is widely used in consumer electronic equipment such as TV, DVR and DVD etc. Since the volume of video database grows very fast, video feature extraction should not be too complicate to compute for the computational efficiency. In proposed approach, the mean of YCbCr was employed for image feature computation, where M and N are the width and height of image, respectively. Yij, Cbij, Crij stand for the value of Y, Cb and Cr components of each pixel in the image. Since there is no strict restriction on video object and scene, the feature computation based on the spatial statistics of image can obtain the major information including luminance and color which is the basis of visual similarity. For video similarity search and noise resistance, the mean statistics were four digits rounding off integers. Image characteristic code (ICC) c is a joint feature representation made up of three statistical integers of every pixel components: Y, Cb and Cr. In this way, high dimensional feature was transformed into compact characteristic code and video similarity search can be implemented as text search.

B. Similarity Measure

According to the characteristics of spatial-temporal distribution (STD) of image frame feature in video, video similarity is related to image frame feature, shots type, length and their temporal variation. The ICC in video makes up a characteristic time series of video according to STD of image frame. Therefore, the statistics of STD based on the ICC containing the general


information of similar shots and their temporal distribution can be adopted for video similarity measure. Since the type and length of shots are usually different, the video similarity is computed based on the statistics of video components. The video component (VC) is the set of image frames with the same ICC, which can be identified as a statistical feature cluster based on STD. According to the statistics of video component, the video similarity measure can be defined as following: giving two video clips whose video component set is Ci, Cj (2) respectively, nk is the number of common video component clusters in their intersection set. The video similarity D is the sum of common component clusters. In this way, the video similarity is measured by fine computation of video component based on the statistics of STD.


III. FAST SIMILARITY SEARCH

A. Clustering Index Table

By virtue of the compact image feature representation ICC, the VSS can be implemented in high efficiency. For the efficiency improvement and scalable computing, Clustering Index Table (CIT) search method is proposed in this paper. The indexes of tables are ICC for video feature clustering. The video components are arranged into the same video clustering index table according to the definition mentioned before. The VSS can be carried out by index clustering and redundant dissimilar video search is no longer executed, so the search efficiency can be greatly improved. Compared with the existing search techniques whose computational complexity are O(n) or O(logn), the CIT search approach can perform with higher efficiency, which is implemented on the basis of hashing technique. There are four types of table in the proposed search strategy. The first is video CIT (VCIT) containing the number of video components. The second is database CIT (DCIT) recording video ID and the number of video component of each video. The third is search table (ST) including the corresponding database CIT of each video clip. The forth is video similarity table (VST) recording similar video ID and video similarity measure. Every video clip in database has its video CIT, search table and video similarity table. There is database CIT for the video similarity measure and video similarity search.

B. Similarity Search based on CIT

The proposed video similarity search system is on the basis of four types of table mentioned above, which can improve the similarity search efficiency and meet the requirement of scalable computing. In video database, every video clip has its video CIT recording the result of video component clustering. Since the index of video CIT and database CIT is the same ICC, the search table can keep recording the indexes of its corresponding database CIT with the computation of video component clustering. For the video similarity search, the similar video clips can be easily found in corresponding database CIT according to the records in search table. In database CIT, the similarity measure can be computed by the statistics of video components. In fact, the database CIT can be identified as the intersection set of video components shared by several video clips. After the entire search in corresponding database CIT, video similarity table including video ID and similarity measure can be generated with the computation of video similarity. Since every video has its video similarity table, the search engine only needs to show the content of video similarity table according to query video, which is also in the video database. If there are new video clips to be added in the database, the video similarity search


system will accomplish two tasks: one is video processing and another is database update. The video processing implements the video feature computation and ICC generation. A series of video CIT based on video component clustering can be created simultaneously. Meanwhile, the search table containing the index of database CIT will be generated according to the video CIT. The video ID and the number of video components will be added at the end of corresponding database CIT. Then, video similarity can be calculated according to the number of video components and the video similarity tables can be updated by searching in corresponding database CIT. The tables of related video in database will also update. All these operation can be implemented simultaneously for computational efficiency and video similarity search system can meet the requirement of scalable computing.

The framework of this project VSS system is shown in Fig.1.

In this system, feature computation of every video clip in video database DB is implemented by ICC based on the statistics of STD. Then, a series of video CIT can be generated by index clustering according to video components. ST is created according to video CIT and corresponding database CIT will update. VST is generated by searching corresponding database CIT according to ST to provide the list of similar video clips and their similarity ranking. For the video similarity search, the content of VST will be shown directly by the search engine to give the similarity search result of the query video clip. In this way, the efficiency of VSS can be greatly improved.


IV. Implementation Details

Tools Used:

MATLAB

MS-Access

Ffmpeg


V. EXPERIMENTAL RESULTS

A. Database Setup

The database in the experiments was made up of a few hundred video clips downloaded from internet. The total length of video data is approximately one hundred and sixty hrs, including various contents such as news, sports, movie, cartoon, teleplay and natural scene etc. The experiments were performed on Intel processor 1.5 GHz, 2G Memory.

The database used is MS-ACCESS. There four tables as mentioned in the above.

Cluster Mechanism: Cluster Indexing.

Due to this cluster indexing it is possible to group similar frames together so as there is no need to store same frame information more than once.


Feature Extraction:

Extracted Key frames from the input query video.

Results:

Query By Example:

Given input search Query:


Retrieved Videos are:


Search Query2:

Retrieved videos are


VI. CONCLUSION & FUTURE WORK

In this project, an efficient video similarity search approach is proposed. A new video feature representation ICC is computed based on the statistics of video spatial-temporal distribution. The video similarity is measured by computing the number of similar video components. To meet the scalable computing requirement, an efficient search method based on clustering index table is proposed and several types of tables are employed for the improvement of search efficiency. The experimental results show this method is efficient and effective and it can be deployed in various storage devices for video similarity search.

This project can be further improved by adding several other local and global descriptors.


VII. REFERENCES

[1] An Efficient Video Similarity Search Algorithm. Zheng Cao, Ming Zhu. IEEE Transactions on Consumer Electronics, Vol. 56, No. 2, May 2010.

[2] http://www.mathworks.com/help/toolbox/images/f12-12267.html

[3] http://www.physicsforums.com/showthread.php?t=24029

[4] http://www.mathworks.com/products/viprocessing/

[5]http://www.mathworks.com/company/events/webinars/index.html?id=&language=en&b y=application

[6]http://www.mathworks.com/company/events/webinars/wbnr43666.html?id=43666&p1=723907038&p2=72390756

[7] Cheung S, Zakhor “A Efficient video similarity measurement with video signature”. IEEE Trans. on Circuits and Systems for Video Technology, vol.13 pp.59-74 2003

[8] Yue Gao Weibo Wang, Junhai Yong. “A video summarization tool using two-level redundancy detection for personal video recorders”, IEEE Trans. Consumer Electronics. vol.54 pp.521-526 2008

[9] Heng Tao Shen, Jie Shao, Zi Huang, Xiaofang Zhou “Effective and Efficient Query Processing for Video Subsequence Identification” IEEE Trans. on Knowledge and Data Engineering, vol.21 pp. 321-334 2009

[10] Yan Ke Rahul Sukthankar Larry Huston “An efficient parts-based nearduplicate and sub-image retrieval system” Proc. of the 12th annual ACM international conference on Multimedia pp.869-876 2004

[11] Yang X, Tian Qi, Chang E-C “A color fingerprint of video shot for content identification” Proc. of the 12th annual ACM int’l conf. on Multimedia pp.276-279 2004

[12] Xuefeng Pan, Jintao L, Yongdong Zhang,, Sheng Tang, Lejun Y, “Format-Independent Motion Content Description based on Spatiotemporal Visual Sensitivity” IEEE Trans. Consumer Electronics. vol.53 pp.769-774 2007

[13] Kwang-deok Seo, Seong Park, Soon-heung Jung, “Wipe scene-change detector based on visual rhythm spectrum” IEEE Trans. Consumer Electronics. vol.55 pp.831-838 2009

[14] Meng Wang, Xian-Sheng Hua, Jinhui Tang, Richang Hong. “Beyond Distance Measurement: Constructing Neighborhood Similarity for VideoAnnotation”


https://docs.google.com/viewer?a=v&pid=explorer&chrome=true&srcid=0Bze1Nf5_LsQiYjJjMDYzZmEtMmRkMS00MGYyLTlmMmQtYjFjYzdjZjI3YTUx&hl=en&pli=1

web viewwith the rapid development of modern electronic ... basis of similarity measurement. ... and...

Documents