medical image retrieval algorithms and techniques … · medical image retrieval algorithms and...

5
International Journal of Modern Computer Science (IJMCS) ISSN: 2320-7868 (Online) Volume 4, Issue 5, October, 2016 RES Publication © 2012 Page | 132 http://ijmcs.info Medical Image Retrieval Algorithms And Techniques Based On CBIR : A Review Mr. Suhas.S Department of CSE NIE Mysore, India E-mail: [email protected] Dr. C.R.Venugopal Department of ECE SJCE Mysore, India E-mail: [email protected] Abstract: Digital images are being produced and used in large quantities and volumes in the medical field for diagnostics and therapy. Due to the large sizes of the images as well as the image databases, CBMIR (Content based Medical Image Retrieval) arose. It has, therefore, led to an increased demand for efficient medical image data retrieval and management. In current medical image databases, images are mainly indexed and retrieved by alphanumerical keywords, classified by human experts. However, purely text-based retrieval methods are unable to sufficiently describe the rich visual properties or features inside the images content, and therefore pose significant limitations on medical image data retrieval. The ability to search by medical image content is becoming increasingly important, especially with the current trend toward evidence-based practice of medicine. This need is mainly due to the large amount of visual data produced and the unused information that these data contain, which could be used for diagnostics, teaching and research. The systems described in the literature and published propositions for image retrieval in medicine are critically reviewed and sorted by medical departments, image categories and technologies used. This article gives an overview of available technologies in the field of content based access to medical images. Keywords: Content based image retrieval, Content based Medical Image Retrieval I. INTRODUCTION The recent escalating use of digital images in diverse application areas such as medicine, education, remote sensing and entertainment has led to enormous image archives and repositories, and therefore requires for effective image data management and retrieval. Every day large volumes of different types of medical images such as dental, endoscopy, skull, MRI, ultrasound, radiology are produced in various hospitals as well as in various medical centres. Medical images play a significant role in detecting anatomical and functional information of the human body for medical research, diagnosis and education. Medical images are generally complex in nature and require extensive image processing techniques for computer aided diagnosis. Due to this reason, in most of the cases physicians or radiologists examine images in conventional ways based on their individual experiences and knowledge. In the medical domain, the ultimate goal of image retrieval is to provide diagnostic support to physicians or radiologists by displaying relevant past cases, along with proven pathologies as ground truth [2]. However, medical image retrieval can also be useful as a training tool for medical students and residents in education, follow-up studies for detecting the growth of tumors, and for research purposes. The ultimate goal of medical image retrieval system, whether CBIR or text based is to deliver the similar images compared to a query image in a most effective and efficient way. The goals of medical information systems have often been defined to deliver the needed information at the right time, the right place to the right persons in order to improve the quality and efficiency of care processes. Such a goal will most likely need more than a query by patient name, series ID or study ID for images. Early work on image retrieval was still based on the textual annotation of images, i.e., images were first manually annotated by keywords or descriptive text and organized by topical or semantic hierarchies in traditional DBMS to facilitate easy access based on standard Boolean queries. However, such purely text-based method poses significant limitations in image retrieval [3]. Manual annotation is subjective, time-consuming and prohibitively expensive, and the sheer content volume of very large image databases is simply beyond the manual indexing capability of human experts. Furthermore, many visual features in images, such as irregular shapes and jumbled textures, are extremely difficult to describe in text. Such text based approach also limits the scope of searching to that predetermined by the author of the system and leaves no means for using the data beyond that scope. A. Content based Image Retrieval Content-based image retrieval (CBIR) methods support full retrieval by visual content / properties of images, i.e., retrieving image data at a perceptual level with objective and quantitative measurements of the visual content and integration of image processing, pattern recognition and computer vision.

Upload: dangxuyen

Post on 09-Apr-2018

215 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: Medical Image Retrieval Algorithms And Techniques … · Medical Image Retrieval Algorithms And Techniques ... textual annotation of images, ... This method deals with the “dimensionality

International Journal of Modern Computer Science (IJMCS) ISSN: 2320-7868 (Online) Volume 4, Issue 5, October, 2016

RES Publication © 2012 Page | 132 http://ijmcs.info

Medical Image Retrieval Algorithms And Techniques

Based On CBIR : A Review

Mr. Suhas.S Department of CSE

NIE

Mysore, India

E-mail: [email protected]

Dr. C.R.Venugopal Department of ECE

SJCE

Mysore, India

E-mail: [email protected]

Abstract: Digital images are being produced and used in large quantities and volumes in the medical field for diagnostics and therapy.

Due to the large sizes of the images as well as the image databases, CBMIR (Content based Medical Image Retrieval) arose. It has, therefore, led

to an increased demand for efficient medical image data retrieval and management. In current medical image databases, images are mainly

indexed and retrieved by alphanumerical keywords, classified by human experts. However, purely text-based retrieval methods are unable to

sufficiently describe the rich visual properties or features inside the images content, and therefore pose significant limitations on medical image

data retrieval. The ability to search by medical image content is becoming increasingly important, especially with the current trend toward

evidence-based practice of medicine. This need is mainly due to the large amount of visual data produced and the unused information that these

data contain, which could be used for diagnostics, teaching and research. The systems described in the literature and published propositions for

image retrieval in medicine are critically reviewed and sorted by medical departments, image categories and technologies used. This article

gives an overview of available technologies in the field of content based access to medical images.

Keywords: Content based image retrieval, Content based Medical Image Retrieval

I. INTRODUCTION

The recent escalating use of digital images in diverse

application areas such as medicine, education, remote sensing

and entertainment has led to enormous image archives and

repositories, and therefore requires for effective image data

management and retrieval. Every day large volumes of

different types of medical images such as dental, endoscopy,

skull, MRI, ultrasound, radiology are produced in various

hospitals as well as in various medical centres. Medical images

play a significant role in detecting anatomical and functional

information of the human body for medical research, diagnosis

and education. Medical images are generally complex in nature

and require extensive image processing techniques for

computer aided diagnosis. Due to this reason, in most of the

cases physicians or radiologists examine images in

conventional ways based on their individual experiences and

knowledge. In the medical domain, the ultimate goal of image

retrieval is to provide diagnostic support to physicians or

radiologists by displaying relevant past cases, along with

proven pathologies as ground truth [2]. However, medical

image retrieval can also be useful as a training tool for medical

students and residents in education, follow-up studies for

detecting the growth of tumors, and for research purposes.

The ultimate goal of medical image retrieval system, whether

CBIR or text based is to deliver the similar images compared to

a query image in a most effective and efficient way. The goals

of medical information systems have often been defined to

deliver the needed information at the right time, the right place

to the right persons in order to improve the quality and

efficiency of care processes. Such a goal will most likely need

more than a query by patient name, series ID or study ID for

images. Early work on image retrieval was still based on the

textual annotation of images, i.e., images were first manually

annotated by keywords or descriptive text and organized by

topical or semantic hierarchies in traditional DBMS to facilitate

easy access based on standard Boolean queries. However, such

purely text-based method poses significant limitations in image

retrieval [3]. Manual annotation is subjective, time-consuming

and prohibitively expensive, and the sheer content volume of

very large image databases is simply beyond the manual

indexing capability of human experts. Furthermore, many

visual features in images, such as irregular shapes and jumbled

textures, are extremely difficult to describe in text. Such text

based approach also limits the scope of searching to that

predetermined by the author of the system and leaves no means

for using the data beyond that scope.

A. Content based Image Retrieval

Content-based image retrieval (CBIR) methods support full

retrieval by visual content / properties of images, i.e., retrieving

image data at a perceptual level with objective and quantitative

measurements of the visual content and integration of image

processing, pattern recognition and computer vision.

Page 2: Medical Image Retrieval Algorithms And Techniques … · Medical Image Retrieval Algorithms And Techniques ... textual annotation of images, ... This method deals with the “dimensionality

International Journal of Modern Computer Science (IJMCS) ISSN: 2320-7868 (Online) Volume 4, Issue 5, October, 2016

RES Publication © 2012 Page | 133 http://ijmcs.info

Figure1. General architecture of the CBIR system

A block diagram of a general CBIR system, shown in figure

1, comprises of modules for feature extraction, feature

indexing, similarity, measurement, query feature extraction and

query interface [1]. In the feature extraction subsystem, image

processing and pattern analysis techniques are used to extract

numeric descriptors of various visual features. Each feature

may have several representations and generally a weight is

assigned to each of these features and their descriptors or

representations. Color is the most widely used feature in any

general domain CBIR system. To retrieve images, the user

submits a query example image to the system and the example

image is then converted into an internal feature vector via a

feature extraction process. The similarity or closeness between

the feature vector of the user‟s query image and those which

are present as feature meta data from the image database

already present is calculated and ranked based on similarity. In

the comparison stage retrieval is performed by applying an

indexing scheme which can be used to support fast retrieval

and to make the system scalable to large image databases. If the

retrieval results are not fully satisfactory, the user can give

some positive or negative feedback to the system and the

modified query can be resubmitted via the interactive relevance

feedback. Such feedback-retrieval cycle can be repeated until

the user is satisfied with the retrieval results.

1. CBIR in Medical Domain (CBMIR)

In the medical domain, the majority of acquired medical

images are currently stored with a limited text-based

description of their content. As image databases expand, it is

becoming increasingly apparent that these simple text-based

descriptions are inadequate for the proper search and retrieval

of medical images. As a consequence, valuable diagnostic and

prognostic information contained in such databases remains

unusable, and it leads to an increased demand for efficient

retrieval techniques which can tap the expertise contained in

these databases. Modern standards such as Digital Imaging and

Communication in Medicine (DICOM) and Picture Archival

and Communication Systems (PACS) make it relatively easy to

store and transport these images and increase interoperability

[4]. Usually the information related to the images is stored in

DICOM image headers. But DICOM headers have proven to

contain a fairly high rate of errors, for example for the field

anatomical region, error rates of 16% has been reported. This

can hinder the correct retrieval of all wanted images.

Hence CBIR has been shown to be a viable alternative to text

based image retrieval, with the abilities to search for an image

depending on metrics for comparing image with visual/ spatial

properties that can match human judgments of similarity. It is

therefore very natural to apply such CBIR to medical domain

(CBMIR) by retrieving medical images according to their

domain-specific image features, providing an important

alternative and complement to the traditional text-based

retrieval. The potential benefits emanating from the CBMIR

range from clinical decision support to medical education and

research. Clinical knowledge has shown that visual

characteristics of medical images have strong effect on

diagnosis. Therefore, diagnosis by comparing past and current

medical images associated with pathologic conditions has

become one of the primary approaches in case-based reasoning

or evidence-based medicine, while the clinical decision-making

process can be beneficial to find other images of the same

modality, the same anatomic region of the same disease[6]. The

CBMIR could aid on such diagnosis in the following way: by

observing an abnormality in a diagnostic image, the physician

can query a database of known cases to retrieve images (and

associated textual information) that contain regions with

features similar to what is observed in the image of interest.

With the knowledge of disease entities that match features of

the selected region, the physician can be more confident of the

diagnosis, and maybe to expand the differential diagnosis to

include pathological entities not previously considered. Here

the CBMIR provides relevant supporting evidence from prior

known cases, offering the physician with training-set examples

that are close to his/her decision boundary, along with the

correct class labels (proven pathology) for these examples. As

such, a less experienced practitioner can benefit from this

expertise in that the retrieved images, if visually similar,

provide the role of an expert consultant. The CBMIR could be

used to present cases that are not only similar in diagnosis, but

also similar in appearance and in cases with visual similarity

but different diagnoses.

II. METHODS FOR IMPLEMENTING CBIR

Content-based image retrieval hinges on the ability of the

algorithms to extract pertinent image features and organize

them in a way that represents the image content. Additionally,

the algorithms should be able to quantify the similarity between

the query visual and the database candidate for the image

content as perceived by the viewer. Thus, there is a systemic

component to CBIR and a more challenging semantic

component.

A. Shape Based Method

Page 3: Medical Image Retrieval Algorithms And Techniques … · Medical Image Retrieval Algorithms And Techniques ... textual annotation of images, ... This method deals with the “dimensionality

International Journal of Modern Computer Science (IJMCS) ISSN: 2320-7868 (Online) Volume 4, Issue 5, October, 2016

RES Publication © 2012 Page | 134 http://ijmcs.info

For shape based image retrieval, the image feature extracted

is usually an N dimensional feature vector which can be

regarded as a point in a N dimensional space. Once images are

indexed into the database using the extracted feature vectors,

the retrieval of images is essentially the determination of

similarity between the query image and the target images in

database, which is essentially the determination of distance

between the feature vectors representing the images. The

desirable distance measure should reflect human perception.

Various similarity measures have been exploited in image

retrieval. In one of the implementation by Muller .H they have

used Euclidean distance for similarity measurement.

B. Texture Based Method

Texture measures have an even larger variety than color

measures. Some common measures for capturing the texture of

images are wavelets and Gabor filters[5]. The texture measures

try to capture the characteristics of images or image parts with

respect to changes in certain directions and scale of changes.

This is most useful for regions or images with homogeneous

texture.

C. Continuous Feature Selection Method

This method deals with the “dimensionality curse” and the

semantic gap problem. The method applies statistical

association rule mining to relate low-level features with high-

level specialist‟s knowledge about the image, in order to reduce

the semantic gap existing between the image representation and

interpretation. These rules are employed to weigh the features

according to their relevance. The dimensionality reduction is

performed by discarding the irrelevant features (the ones whose

weight are null). The obtained weights are used to calculate the

similarity between the images during the content-based

searching[8]. Experiments performed show that the proposed

method improves the query precision up to 38%. Moreover, the

method is efficient, since the complexity of the query

processing decreases along the dimensionality reduction of the

feature vector.

D. Retrieval based on Color

Since the majority of medical images are intensity-only-

images carrying less information than color images, the color-

based retrieval would only be applicable to medical images

based on light photography, where color is an inherent feature

and any deviations or changes in the color of a particular

sample from normal sample can have significant medical

implications. Such medical images that could benefit from the

color-based CBMIR include histological images,

dermatoscopic images, endoscopic images, and tongue images,

in which the well-known color moments and color histogram

approaches are often adopted. These images possess uniquw

color signatures from which the RGB(Red, Green, Blue)

components and the HSV(Hue, Saturation, Value) color space

are analysed and local features are extracted from the image.

E. Retrieval by 3D Volumetric Features

The majority of medical images capture human anatomy that

is in nature a 3D structure. The 3D medical images can be used

for non-destructive inspection of the body and its component

regions in vivo and in vitro, supporting execution and

monitoring of interventions, and providing quantitative

measurements to determine if an abnormality is present by

comparison with normal controls in diagnosis and treatment

planning[5]. However, the feature extraction and retrieval of

3D medical images in most of the existing CBMIR systems so

far are still performed based on 2D slices – simply following

the way of the conversional image retrieval in 2D domain, and

departing from their originally obtained 3D form. It is believed

that the more accurate CBMIR with more discriminating power

can be achieved if we can take full advantage of the

information available in 3D spatial domain by performing

retrieval of these medical images based on their 3D volumetric

features.

A CBMIR approach based on 3DiMSP(ideal midsagittal

plane) features for retrieving 3D CT neuroimages of

haemorrhage (blood), bland infarct (stroke) and normal brains

was proposed. Giving the following basic observation of

neuroradiologic imaging: normal human brains exhibit an

approximate bilateral symmetry which is often absent in

pathological brains, a symmetry detector was firstly

constructed to automatically extract an iMSP – a virtual

geometric plane about which the 3D anatomical structure

captured in the given brain image presents maximum bilateral

symmetry. The basic idea is to find where the MSP is supposed

to be for a given 3D brain image, especially for pathological

brains where the anatomic MSP is often distorted(shifted or

bent) due to large lesions. Automatically locating and retrieving

possible lesions (such as bleeds, stroke, tumours) can therefore,

be conducted by detecting asymmetrical regions with respect to

the extracted iMSP[7]. After the iMSP is aligned in the middle

of each 3D volumetric image, a stack of cross-sectional 2D

slices was temporarily derived as basic image units, from

which three types of asymmetry features were then calculated

to quantify and capture the statistical distribution difference of

various brain asymmetries: global statistical properties;

measures of asymmetry of halved and quartered brains; and

local asymmetrical region-based properties. These features

were calculated from the original image unit with its iMSP

aligned, the difference image of the original image and its

mirror reflection with respect to iMSP, the thresholded

difference image, and the original image masked by the

thresholded binary image. For each image unit, a feature vector

with 48 image features was constructed, including the means,

Page 4: Medical Image Retrieval Algorithms And Techniques … · Medical Image Retrieval Algorithms And Techniques ... textual annotation of images, ... This method deals with the “dimensionality

International Journal of Modern Computer Science (IJMCS) ISSN: 2320-7868 (Online) Volume 4, Issue 5, October, 2016

RES Publication © 2012 Page | 135 http://ijmcs.info

the variances, and the X and Y gradients of the intensity images

at different regions and under various Gaussian smoothing,

scaling and thresholding.

F. Retrieval by Semantic Pathology Interpretation

Sometimes medical images derived from a specific organ are

similar visually and normally differ only in small details which

could be disregarded by untrained eyes but such domain

specific subtle differences maybe of pathological significance .

One such domain is medical radiology, for which the clinically

useful information consists of gray level variations in highly

localized regions of the image. For example, for HRCT images

of the lung, the number of pathology bearing pixels as a

fraction of all the image pixels is so small that it is not possible

to use global signatures for image characterization. Moreover,

the unclear boundaries between the pathology bearing pixels

and the surrounding healthy part are difficult to discern and

such pathology bearing regions (PBRs) are unlikely to be

automatically extracted. Since medical images frequently give

rise to ambiguity in interpretation and in diagnosis, current

retrieval techniques using the primitive image characteristics

such as color, texture, and shape, are likely to be insufficient

for some domain-specific CBMIR systems. One approach to

remedy such limitation is to associate high-level semantic

information with low-level visual image data, albeit such

approach is still controversial in terms of subjectiveness.

G. By Physiological Functional Features

Physiological function can be estimated at the molecular level

by observing the behaviour of a small quantity of an

administered substance „tagged‟ with radioactive atoms.

Images are formed by the external detection of gamma rays

emitted from the patient when the radioactive atoms decay.

Glucose metabolism, oxygen utilisation, and blood flow in the

brain and heart can be measured with compounds labelled with

carbon (11

C), fluorine (18

F), nitrogen (13

N), and oxygen (15

O),

which are the major elemental constituents of the body[9].

Existing CBMIR approaches may not be optimal when applied

to functional images due to their unique characteristics with

regard to the inherent knowledge of the disease state as it

affects the physiological and biochemical processes before the

morphological change of the body.

III. CONCLUSION

This article presents highlights and reviews of recent

research in medical image retrieval and registration domain,

main technologies, major trends and point out some promising

research directions and combined system architecture for

automatic and efficient diagnosis in a hospital or clinical

environment. The need for the CBIR in medical domain

(CBMIR) and its related challenges were discussed, and

followed by a detailed review on the current major CBMIR

techniques in four different categories: retrieval based on

physical visual features (color and texture); retrieval based on

geometric spatial features (shape, and spatial relationships);

retrieval by combination of semantic and visual features

(semantic pathology interpretation); and retrieval based on

physiological functional features. In summary, we investigate

two important technologies from a combined perspective of

automatic and computer assisted clinical decision-making and

point out some observation about how they can be integrated in

a larger system that complements each other[4]. The need for a

real and practical image based diagnostic system is growing

rapidly as more and more healthcare professionals utilize image

guided diagnosis in their daily activities and research. Although

it is a complex task to build an enterprise class system that is

reliable and robust, benefits it would bring to the health

community is unimaginable. The difficulty faced by CBIR

methods in making in-roads into medical applications can be

attributed to a combination of several factors. Some of the

leading causes can be categorized according to the “gaps”

model presented above. An idealized system can be designed to

overcome all the above gaps, but still fall short of being

accepted into the medical community for lack of (i) useful and

clear querying capability; (ii) meaningful and easily

understandable responses; and (iii) provision to adapt to user

feedback. The opposite is also true to some extent. A

technically mediocre, but promising, system may obtain

valuable end user feedback, and by technical improvement may

increase user acceptance with the application of usability

design principles. Other than item (iii), which still needs

significant research effort, the usability gap can only be bridged

by keeping the end user in mind from early system

development, as well as by conducting well designed usability

studies with targeted users. In general, a high involvement of

the user community in system design and development can

significantly improve adoption and acceptance.

REFERENCES

[1] Prachi.Bhend, Prof.A.N.Cheeran, “Content Based Image

Retrieval in Medical Imaging”, International Journal of

Computational Engineering Research, Vol 03,Issue 8, August

2013

[2] Mrs. Mallika, Mr. S. Anantha Krishnan, “An Efficient Content Based

Image Retrieval Using Hierarchical Clustering Approach”,

International Journal Of Research In Computer Applications

And Robotics, Vol.2, Issue.9, Pg.: 119-126, September 2014

[3] Abdol Hamid Pilevar, “CBMIR: Content-based Image Retrieval

Algorithm for Medical Image Databases”, Journal of Medical

Signals and Sensors, J Med Signals Sens. 2011 Jan-Apr; 1(1):

12–18.

[4] Md. Mahmudur Rahman, Tongyuan Wang, Bipin C. Desai,

“Medical Image Retrieval and Registration: Towards Computer

Page 5: Medical Image Retrieval Algorithms And Techniques … · Medical Image Retrieval Algorithms And Techniques ... textual annotation of images, ... This method deals with the “dimensionality

International Journal of Modern Computer Science (IJMCS) ISSN: 2320-7868 (Online) Volume 4, Issue 5, October, 2016

RES Publication © 2012 Page | 136 http://ijmcs.info

Assisted Diagnostic Approach”, IEEE IDEAS Workshop on

Medical Information Systems: The Digital Hospital, September

2004.

[5] H Müller N Michoux D Banden A Geissbuhler "A Review of

Content-based Image Retrieval Applications Clinical Benefits

And Future Directions" International Journal of Medical

Informatics 73 pp. 1-23 2004.

[6] T. M. Lehmann M. O. Gould C. Thies B. Fischer M. Keysers D.

Kohnen H. Schubert B. B. Wein "Content-Based Image

Retrieval In Medical Applications For Picture Archiving And

Communication Systems" Medical Imaging of SPIE Proceedings

Vol. 5033 San Diego California USA 2003.

[7] Kanupriya, Amanpreet Kaur, “Medical Image Retrieval Using

Content Based Image Retrieval System”, International Journal of

Engineering Research & Technology , Vol. 3 Issue 1, January –

2014.

[8] Ritendra Datta, Jia Li, and James Z. Wang, “Content-Based

Image Retrieval - Approaches and Trends of the New Age”,

MIR‟05,November 11-12, Singapore, 2005.

[9] Ashnil Kumar, Jinman Kim, Weidong Cai, Michael Fulham,

Dagan Feng, “Content-Based Medical Image Retrieval: A

Survey of Applications to Multidimensional and Multimodality

Data”, Society for Imaging Informatics in Medicine, 2013.