[ieee 2012 international conference on pattern recognition, informatics and medical engineering...

Proceedings of the International Conference on Pattern Recognition, Informatics and Medical Engineering , March 21-23, 2012

978-1-4673-1039-0/12/$31.00 ©2012 IEEE

Medical Image Retrieval System using GGRE Framework

J. Yogapriya Department of Computer Science and Engineering

Paavai Engineering College Namakkal, Tamilnadu,India.

E-Mail id: [email protected]

Dr. Ila Vennila Department of Electrical and Electronics Engineering

PSG College of Technology Coimbatore, Tamilnadu, India.

Abstract— This paper seeks to focus on Medical Image Retrieval based on Feature extraction, Classification and Similarity Measurements which will aid for computer assisted diagnosis. The selected features are Shape(Generic Fourier Descriptor (GFD)and Texture(Gabor Filter(GF)) that are extracted and classified as positive and negative features using a classification technique called Relevance Vector Machine (RVM) that provides a natural way to classify multiple features of images. The similarity model is used to measure the relevance between the query image and the target images based on Euclidean Distance(ED). This type of Medical Image Retrieval System framework is called GGRE. The retrieval algorithm performances are evaluated in terms of precision and recall. The results show that the multiple feature classifier system yields good retrieval performance than the retrieval systems based on the individual features.

Keywords: CBMIR, Shape, Texture, Classification, Similarity Measurements.

I. INTRODUCTION The medical imaging systems are used to deliver the needed images to physicians at right time to diagnose and treat diseases. However, access or make use of particular image is not possible unless it is organized. Thus the need for systems that can provide efficient retrieval of images of particular interest from different modalities of medical images in the database is becoming very high. The examples of these modalities are the following: Ultrasound (US), Magnetic Resonance (MR), Positron Emission Tomography (PET), Computed Tomography (CT), Endoscopy (ENDO), Mammograms (MG), Digital Radiography (DR), Computed Radiography (CR), etc., The modalities of images are visual characteristic of images that can be used to assist the retrieval process and to detect the anatomical and functional information about different body parts for the purpose of diagnosis, medical research, and education. The radiology department of hospitals is equipped with Picture Archiving and Communications Systems (PACS). The universal format for PACS [1] image storage, retrieval and transfer is DICOM (Digital Imaging and Communications in Medicine) format. The imaging modalities, body parts, orientations are available as textual information in the DICOM header. The PACS systems have limitations because the search for images is carried out based on the textual attributes of image headers

(such as patient id name and other technical parameters describing body parts, orientations etc.,). There have also been reported errors in the accuracy of DICOM headings. On the other hand the medical department often does not enter appropriate or sufficient data into the systems. So it requires the need to build automatic indexing by visual features of images such as texture, shape, color, etc., to provide sufficient information. This type of system is called Content based Medical Image Retrieval System (CBMIR). CBMIR systems are currently being integrated with PACS for increasing the overall search capabilities and tools available to radiologists.

The main thing in visual features based image retrieval system is the gap between visual feature representations and semantic concepts of Images. Medical images are usually subject to high variation and composed of different minor structures. For efficient similar image retrieval and integration, the medical image should be processed systematically to extract a representing feature vector for each member image. So there is a need for developing a framework to combine visual features using classification techniques for effective image retrieval. Texture and Shape features are essential for medical image retrieval system. The main objective of this work is to retrieve the images from medical image databases with high accuracy by performing feature extraction and classification process. As a result of a medical image retrieval system, the physician can gain more confidence in his/her decision for diagnosis. The organization of the paper is as follows. The existing methods survey is presented in section II.The proposed method and its related works are described in section III. Experimental results and performance evaluation are specified in section IV and finally conclusion is explained in section V.

II. LITERATURE SURVEY The reason for providing appropriate tools for easily

managing medical images databases are: 1. When the Physicians or Radiologists manage several departments in the hospital, they are very often to search for an image for immediate diagnosis and effective treatments. 2. The medical students and teaching institutions are willing to get the required images effectively for further analysis of their research. For these reasons there was a lot of work in the last

International Conference on Pattern Recognition, Informatics and Medical Engineering (PRIME-2012)

247

years for the construction of CBMIR systems [2] and represented in Table I.

TABLE I. CBMIR SYSTEMS

CBMIR Images used Visual Features ASSERT (Automatic Search and Selection Engine with Retrieval Tools)[3]

High-Resolution Computed Tomography (HRCT) of lung

Texture, Shape, Edges, and Gray-scale Properties

CasImage[4] A variety of images from CT, MRI, and radiographs, to color photos

Global and Regional Color and Texture features

IRMA (Image Retrieval in Medical Applications)[5]

Various imaging modalities

Global and Local Shape and Texture Features

NHANES II (The Second National Health And Nutrition Examination Survey)[6]

cervical and lumbar spine X-ray image

Shape Features

Image Map[7] Multiple Images of organs

Individual Regions and Spatial Relationships between Regions of Shape and Texture Features

MIMS Medical Image Management System [8]

X ray ,CTs of the Head Images

Text and Shape Features

Plaque CBIR system[9]

Plaque Images

Shape and Texture Features

Medical image categorization, registration, feature extraction, Classification indexing and retrieval is performed over the entire image database in the above mentioned Image Retrieval Systems. Each method has its own advantages and disadvantages in their retrieval performance.

Different authors has used different image features set and image classification methodology for their medical image retrieval applications.

A. Image Visual Features: Feature extraction is the base for CBMIR. Within the

visual feature scope, the features can be further classified as general features and domain specific features. The former include color, texture, and shape features while the latter is application-dependent and may include, for example, human faces and finger prints.

General visual features such as Shape and Texture are most widely used in CBMIR [17][20]

B. Texture Features: Texture[11] refers to visual patterns with properties

of Homogeneity and consists of basic primitives (texels or micro patterns) whose spatial distribution in the image creates the appearance of a texture. There are two basic classes of texture descriptors, namely, statistical model-based and transform-based. The former one explores the grey-level spatial dependence of textures such as Gray Level Co-occurrence Matrix and then extracts some statistical features as texture representation. The latter approach is based on spatial frequency and Transform Domain Features such as Gabor Filter Features and Wavelet Features.

C. Shape Features: Shape[12] is an important feature for medical image

retrieval. There are two types of approaches used in shape representation. One is the contour based shape method and the other is the region based method. Contour shape techniques only exploit shape boundary information and Region based methods consider all the pixels within a shape region. The contour based shape method can be represented by moment invariants, Generic Fourier descriptors, chain code, eccentricity, Shape signature etc., The region based shape method can be represented as Zernike Moments, Grid Method, Shape Matrix, Convex Hull etc.,

D. Image Classification: The feature vectors computed in different points of a

shape and texture image [21] are not identical. Training the classification systems with these features could increase the accuracy rate. Image classification is performed in which extracted features are given as input to the image classification tool in order to classify the images. Then the features are grouped into two parts positive and negative.

In supervised learning, given a set of examples of input vectors {Xn}N

n=1 along with corresponding targets {tn}N

n=1,the latter of which might be real values (in regression) or class labels (classification). From this ‘training’ set, learn a model of the dependency of the targets on the inputs with the objective of making accurate predictions of it. In real-world data, the presence of noise (in regression) and class overlap (in classification) implies that the principal modeling challenge is to avoid ‘over-fitting’ of the training set. The Support Vector Machine (SVM) of Vapnik [13][14] [18][19]has become widely established as one of the well-known approach for Image Classification. It expresses predictions in terms of a linear combination of kernel functions centered on a subset of the training data, known as support vectors.

Even though it’s widespread success, the SVM suffers from some important limitations [16] such as, 1. An SVM Classifier is unstable on a small-sized training set. 2. SVM’s optimal hyper plane may be biased due to certain condition. 3. Over fitting happens because the number of feature dimensions is higher than the size of the training set.


248

4. It makes point predictions rather than generating predictive distributions.

To overcome the difficulties of Support Vector Machines (SVMs), the proposed framework uses Relevance Vector Machines (RVMs) as a classical learning model.

III. PROPOSED WORK The Fig 1 represents the proposed CBMIR system

consists of three major steps called GGRE framework Step I:

� Medical Image as given as Input to the system. � For a given query image, extract Shape Features

using Generic Fourier Descriptor(GFD).Each GFD features reflect about radial frequencies and angular frequencies.

� Extract Texture Features using Gabor Filter (GF) by calculating Mean and Standard Deviation of query image.

Step II: Next image classification is performed in which

extracted features are given as input to the image classification tool in order to classify the images then the features are grouped into two parts (positive, negative). The image classification process is applied using Relevance Vector Machine (RVM). Step III:

Finally searching and retrieval process is performed using well known Similarity Measurement as Euclidean Distance(ED).

Figure 1.Proposed Image Retrieval System

3.1 Feature Extraction 3.1.1. Generic Fourier Descriptor: The extraction of Generic Fourier Descriptor[10] is performed in spectral domain by applying Two Dimensional Fourier Transform (FT) on polar raster sampled shape image. The extracted features from FT are not rotation invariant so a modified polar FT is proposed by taking the polar image in

polar space as a normal Two Dimensional rectangular image in Cartesian space.

(a) (b)

Figure 2. (a) Polar space of image; (b) Cartesian space of polar image

For a given image f(x,y) the polar FT is defined in Equation 1

��

��

��

� � ��

Ti

RrjrfPF

rii

i

22exp),(),( (1)

Where � � Ryyxxr cc �� 2122 )()(0

�

� ��

Tii

�� 2 Ti ��0

� �cc yx , is the center of mass of the shape.

TR �� 0,0 R and T are the radial and angular resolutions. The shape determination of � and � are achieved by few lower frequencies but the acquired polar coefficients are translation invariant. By using the following normalization Equation (2), Rotation and Scaling invariance are achieved.

��

!"

�)0,0(),(,...

)0,0()0,(,...

)0,0(),0(,...

)0,0()1,0(,)0,0(

PFnmPF

PFmPF

PFnPF

PFPF

areaPFGFD

(2) Where, area is the area of the bounding circle in which shape exists. m is the number of maximum radial frequencies selected. N is the number of maximum angular frequencies selected.

In the experiments 4 radial frequencies and 8 angular frequencies are selected (32 GFD features) for efficient shape representation. 3.1.2. Gabor Filter:

Gabor filters[11] use sets of filters to analyze the image spectrum. They are represented as a set of banks, in which each filter is tuned to a specific orientation and scale. Using Gabor Filter, the texture feature vectors are found by calculating the mean and standard deviation for all images & then stored in a database. Given an image I (x, y), its Gabor wavelet transform is shown in Equation (3)

# �� 111111 ),(*),(),( dydxyyxxgyxIyxG mnmn (3)

Where, * indicates the complex conjugate. g (x, y) is a two-dimensional Gabor function and its Fourier

Transform is given in Equation (4)


249

��

�

��

��

��

��

��

�� jWxyxyxg

yxyx

�$$$�$

221exp

21),( 2

2

2

2 (4)

Gabor functions form a complete but non-orthogonal basis set. Expanding a signal using this basis provides a localized frequency description. A class of self-similar filter dictionary can be obtained by appropriate dilations and a rotation of g (x, y) through the generating function is given in Equation (5):

� � ),(, '' yxgayxg mmn

�� (5) a >1, m, n=integer

)sincos(' �� yxax m �� And

),cossin(' �� yxay m ��

Where Kn /�� and K is the total number of orientations. 3.1.2.1 Redundancy reduction:

Let Ul, Uh denotes the lower and upper center frequencies of interest. Let K be the number of orientations and S is the number of scales. This results in the computation of following filter parameters σx and σy is given in Equation (6).

,)/( 11�� S

lh UUa ,2ln2)1(

)1(��

�a

Ua hx$

� �,2ln22ln22ln2

2tan

21

2

222 �

��

��

��

�

��

��

��

��

��

h

x

h

xhy UU

Uk

$$�$ (6)

Where, a is a scaling factor and m=0, 1… S-1. It is assumed that the local texture regions are spatially homogeneous, and so the mean mn% and deviation mn$ of the magnitude of the coefficients are used to represent the retrieval purposes:

##� ,)( dxdyxyGmnmn% (7)

� �## �� dxdyyxG mnmnmn2),( %$ (8)

A feature vector is now constructed using µmn and σmn as feature components. In the experiments it uses two scales S = 4 and four orientations K =6, resulting in a feature vector is given in Equation (9)

� �242401010000 ....., $%$%$%�f (9) 3.2 Classification 3.2.1 Relevance Vector Machine:

Tipping [15] has formulated the Relevance Vector Machine (RVM) as a probabilistic model whose functional form is equivalent to the SVM. It is proved to be faster than SVM since it yields an optimum solution with few training samples. These are called relevance vectors.

It achieves comparable recognition accuracy to the SVM, yet provides a full predictive distribution, and also requires substantially fewer kernel functions represented in Table II.

TABLE II. TYPES OF KERNEL FUNCTIONS

Kernel Function

Inner Product Kernel K (X, Xi), I=1,2…N

Polynomial ...2,1,)1( �� PXX Pi

T Radial-basis $$ ),||

21exp( 22

iXX ��

is decided by the user Sigmoid ),tanh( 10 && �XX T

&0 and &1are decided by the user Apply RVM classifier in order to reduce

dimensionality of feature set and grouped as correctly classified feature set vectors by using small training samples of features. RVM use hyper planes in order to separate the two parts of the image classes such as positive, negative. Extend the relevance vector approach to the case of classification- i.e., where it is desired to predict the posterior probability of class membership given the input x. Generalize the linear model by applying the logistic sigmoid function

)1/(1)( yey ��$ to y (x) and writing the likelihood as

' ( ' (� �)�

��N

nt

nxytxywtP nnn

1

1)(1)()|( $$ (10)

A set of training samples Ti is given as input to the RVM. Let training samples are t1…..tn, where, each sample belongs to a class labeled such as yi Є (+1(Positive),-1(Negative)), then, the hyper plane decision function can be written as

��

�n

inin ttKwxy

1),()( (11)

Where ),( nttK is a kernel function, ti, i=1, 2…N are the training samples and {wi} are the model weights. The kernel function is used to form expansion basis functions for RVM, and in theory, is not limited by the Mercer conditions as in the case of Support Vector Machine (SVM) or Kernel Fisher Discriminate (KFD).

The Fig 3 represents positive and negative feature vectors for specific classes.

. Figure 3.Separation between Positive and Negative Class

features. 3.3 Similarity Measurements 3.3.1. Euclidean Distance

Euclidean distance is used for similarity comparison between query image and classified image in the database by using the following equation


250

��

��N

iDBQED iCFiFS

1

2])[][( (12)

Where ][iFQ the ith query image is feature and

][iCFDB is the corresponding feature in the classified feature vector database. Here N refers to the number of images in the database. The Similarity between Images are represented in Fig 4.

Figure 4.Similarity between Images

IV. EXPERIMENTAL RESULTS AND PERFORMANCE EVALUATION

This medical retrieval system is implemented using Java with image database of 1000 images containing a variety of images. These images include various parts of the body like lung, liver, kidney, brain etc. These images are gray level images and structurally similar images. Fig 5 shows some image examples from the database.

Figure 5.Sample Images

4.1. Comparisons on retrieval performance Precision and Recall [21] are the most widely used

retrieval performance measurement in literature. Precision measures the retrieval accuracy while recall measures the ability of retrieving relevant images from the database. Precision and recall are inversely related, i.e., precision normally degenerates as recall increases. The Precision (P) and Recall (R) are then defined as

snrp

image retriveved ofNumber imagesrelavant ofNumber

1��

Databasein images retriveved ofnumber Total

imagesrelavant ofNumber 2��

nrR

The Fig 6 and Fig 7 represent the sample query image and output of retrieved images of liver.

Figure 6 Sample query image “liver.jpg”

Figure 7.Output of retrieved images of “liver.jpg”

4.2. Analysis of performance Classification Among 1000 images the number of images taken for

testing is 162 and the number of images taken for training is 838.For each images, extracted Shape and texture features are given as input to classification phase .The total number of feature classes are 52. The Table III represents brain, liver, lung and kidney Images for testing, training and number of classes.

TABLE III. TESTING,TRAINING AND NO. OF CLASSES.

Image Type Image Features Brain Liver Lung Kidney

Images for Testing

42 32 39 49

Images for Training

214 154 182 288

Total no. of Images

256 186 221 337

Number of classes

13 11 12 16

Table IV represents retrieval results of individual features and combined features precision and recall. The Fig 8 and Fig 8 represent Precision and Recall Graph.

TABLE IV. MEDICAL IMAGE RETRIEVAL RESULTS.

Retrieval Mode Percentage of Precision for Top Relevant Images

Percentage of Recall for Top Relevant Images

Shape Feature Image Retrieval

70 66

Texture Feature Image Retrieval

75 72

Combined Features Image Retrieval

84 80

RVM Classified Feature Image Retrieval

94 92

The Fig 10 represents the performance analysis of the RVM and SVM classifier for the detection of specific classes from the grouped features. This graph plots the detection rate such as positive feature images versus the negative features per image wide ranging over the continuum of the decision threshold. The trained classifiers were evaluated using all the features in the test subset. RVM approximately achieved an accuracy of 90% also this accuracy level is similar to that


251

obtained from the SVM. But the RVM classifier could reduce the detection time from nearly 300 s to about 40 s per image.

Figure 8: Precision Graph.

Figure 9: Recall Graph.

Figure 10.Comparison between RVM and SVM classification

methodologies. The experimental results show the RVM technique

can reduce the computational complexity of the SVM based on considering its detection accuracy. This makes RVM more feasible for CBMIR System.

V. CONCLUSION In keyword feature based image retrieval systems,

semantics of images are accurately specified but vast amount of labor required in manual image annotations. In visual feature based image retrieval systems, images would be indexed by their own visual content, such as color, shape, texture. The main thing in visual features based image retrieval system is the gap between visual feature representations and semantic concepts of Images. For efficient similar image retrieval and integration, the medical image should be processed systematically to extract a representing feature vector for each member image.

This proposed System uses multiple feature vectors includes texture, shape features and Image Classification by Relevance Vector Machine and used Euclidean distance as the similarity measure between combined features and query image features. In our experiment we used 1000 medical image covers liver, lung, kidney, brain images. The database images are divided into 10 categories, each category containing 100 images. Results reflect that our content based medical image retrieval (CBMIR) using multiple features with classification outperforms with respect to retrieval accuracy and precision as compared with single features.

REFERENCES [1] Md. Mahmudur Rahmana, Bipin C. Desai a, Prabir Bhattacharya

b”Medical image retrieval with probabilistic multi-class support vector machine classifiers and adaptive similarity fusion” Elsevier

Computerized Medical Imaging and Graphics 32 (2008) Page No. 95–108,2008.

[2] Muller H, Michoux N, Bandon D, Geissbuhler A. “A review of content based image retrieval systems in medical applications clinical benefits and future directions.” International Journal of Medical Information vol 73 Page No:1–23. 2004.

[3] Chi-Ren Shyu, Carla E.Brodley, Avinash C.Kak, Akio Kosaka, “ASSERT: A Physician-in-the-Loop Content-Based Retrieval System for HRCT Image Databases”, Computer Vision and Image

Understanding,Vol.75,Nos.1/2,July/August,pp.111-132, 1999. [4] Thies C, Guld MO, Fischer B, Lehmann TM, “Content based queries on

the CasImage database with in the IRMA framework”, Lec Notes in Comp Sci x3491:781-92 2004.

[5] Thomas M.Lehmann, Mark O. Gulda, Christian Thiesa, Bartosz Plodowskia, Danial Keysersb, Basian Ottc and Henning Schubertc, “IRMA Content-based image retrieval in medical applications”, Proc.SPIE Conf. on Storage and Retrival for image and video Databases 2004;73(1):P.1-23.

[6] Antani S, Long LR, Thoma GR, “Content-based Image retrieval for large biomedical image archives. In: Proc 11th World Cong Medical Informatics: 2004. P.829-33.

[7] Petrakis, Euripidies G.M and C.Faloutsos, “ImageMap: An Image indexing Method Based on Spatial Similarity”, IEEE Transaction on Knowledge and Data Engineering., vol:14(issue 5), P.979-987, 2002.

[8] Chbeir, R., Y.Amghar, and .Flory,“MIMS: A Prototype for Medical Image Retrieval”, Proctor the 6th Conference on Content-Based Multimedia Information Access- Recherched’Informations, P.846-861, 2002

[9] Amores J, Radeva P., “Medical image retrieval based on plaque appearance and image registration” Plaque Imaging: Pixel to Molecular Level, J.S. Suri (Eds.) IOS Press, and 2005.Assist’ee par Ordinateur. Paris, France, 2000.

[10] D.S.Zhang and G.Lu, “Shape-based image retrieval using Generic Fourier Descriptor”, Signal processing: Image Communication 17,825-848, 2002.

[11] B.S.Manjunat and W.Y.Ma,”Texture Features for Browsing and Retrieval of Image Data”,IEEE transction on Patteren Analysis and Machine Intelligence,Vol.18,No.8,August 1996 Pages:837-842.

[12] Chenyang Xu, Jerry L Prince, “Snakes,Shapes, and Gradient Vector Flow”, IEEE Transactions on Image Processing, Vol-7, No 3,Page no: 359-369, March 1998.

[13] Feng Jing; Mingjing Li; Hong-Jiang Zhang; Bo Zhang” A unified framework for image retrieval using keyword and visual features”, IEEE Transactions on image Processing,Volume 14, Issue 7, July 2005 Page no.: 979 – 989

[14] T.P.Minka and R.W.Picard, ”Interactive learning using a society of models,” Pattern Recognition, Vol.30.No.4, page.565-581, April 1997.

[15] Michael E. Tipping, ‘The Relevance Vector Machine’, Microsoft research, Cambridge, UK. P.211–244, 2000

[16] Xu Xiang-min; Mao Yun-feng; Xiang Jia-ni; Zhou Feng-le ”Classification Performance Comparison between RVM and SVM” IEEE International Workshop on anti-counterfeiting, Security, Identification, 2007 Volume , Issue , 16-18 April 2007 Page:208 – 211.

[17] Thomas Deselaers Æ Daniel Keysers Æ Hermann Ney“Features for image retrieval: an experimental Comparison” Springer Science+Business Media, LLC 2007

[18] P. Rajendran, M.Madheswaran “Hybrid Medical Image Classification Using Association Rule Mining with Decision Tree Algorithm ”JOURNAL OF COMPUTING, VOLUME 2, ISSUE 1, JANUARY 2010, ISSN 2151-9617

[19] Md. Mahmudur Rahmana, Bipin C. Desai a, Prabir Bhattacharya b”Medical image retrieval with probabilistic multi-class support vector machine classifiers and adaptive similarity fusion” Elsevier Computerized Medical Imaging and Graphics 32 (2008) 95–108

[20] Greenspan H, Pinhas T. Adi. Medical Image Categorization and Retrieval for PACS Using the GMM-KL Framework. IEEE Transactions on Information Technology in Biomedicine. 2006

[21] A. Grace Selvarani and S.Annadurai, “Content Based Medical Image Retrieval System using Shape and Texture Features”, ICGST-BIME Journal, Vol 8, Issue 1, December 2008.

[ieee 2012 international conference on pattern recognition, informatics and medical engineering...

Documents