medical image retrieval algorithms and techniques … · medical image retrieval algorithms and...
TRANSCRIPT
International Journal of Modern Computer Science (IJMCS) ISSN: 2320-7868 (Online) Volume 4, Issue 5, October, 2016
RES Publication © 2012 Page | 132 http://ijmcs.info
Medical Image Retrieval Algorithms And Techniques
Based On CBIR : A Review
Mr. Suhas.S Department of CSE
NIE
Mysore, India
E-mail: [email protected]
Dr. C.R.Venugopal Department of ECE
SJCE
Mysore, India
E-mail: [email protected]
Abstract: Digital images are being produced and used in large quantities and volumes in the medical field for diagnostics and therapy.
Due to the large sizes of the images as well as the image databases, CBMIR (Content based Medical Image Retrieval) arose. It has, therefore, led
to an increased demand for efficient medical image data retrieval and management. In current medical image databases, images are mainly
indexed and retrieved by alphanumerical keywords, classified by human experts. However, purely text-based retrieval methods are unable to
sufficiently describe the rich visual properties or features inside the images content, and therefore pose significant limitations on medical image
data retrieval. The ability to search by medical image content is becoming increasingly important, especially with the current trend toward
evidence-based practice of medicine. This need is mainly due to the large amount of visual data produced and the unused information that these
data contain, which could be used for diagnostics, teaching and research. The systems described in the literature and published propositions for
image retrieval in medicine are critically reviewed and sorted by medical departments, image categories and technologies used. This article
gives an overview of available technologies in the field of content based access to medical images.
Keywords: Content based image retrieval, Content based Medical Image Retrieval
I. INTRODUCTION
The recent escalating use of digital images in diverse
application areas such as medicine, education, remote sensing
and entertainment has led to enormous image archives and
repositories, and therefore requires for effective image data
management and retrieval. Every day large volumes of
different types of medical images such as dental, endoscopy,
skull, MRI, ultrasound, radiology are produced in various
hospitals as well as in various medical centres. Medical images
play a significant role in detecting anatomical and functional
information of the human body for medical research, diagnosis
and education. Medical images are generally complex in nature
and require extensive image processing techniques for
computer aided diagnosis. Due to this reason, in most of the
cases physicians or radiologists examine images in
conventional ways based on their individual experiences and
knowledge. In the medical domain, the ultimate goal of image
retrieval is to provide diagnostic support to physicians or
radiologists by displaying relevant past cases, along with
proven pathologies as ground truth [2]. However, medical
image retrieval can also be useful as a training tool for medical
students and residents in education, follow-up studies for
detecting the growth of tumors, and for research purposes.
The ultimate goal of medical image retrieval system, whether
CBIR or text based is to deliver the similar images compared to
a query image in a most effective and efficient way. The goals
of medical information systems have often been defined to
deliver the needed information at the right time, the right place
to the right persons in order to improve the quality and
efficiency of care processes. Such a goal will most likely need
more than a query by patient name, series ID or study ID for
images. Early work on image retrieval was still based on the
textual annotation of images, i.e., images were first manually
annotated by keywords or descriptive text and organized by
topical or semantic hierarchies in traditional DBMS to facilitate
easy access based on standard Boolean queries. However, such
purely text-based method poses significant limitations in image
retrieval [3]. Manual annotation is subjective, time-consuming
and prohibitively expensive, and the sheer content volume of
very large image databases is simply beyond the manual
indexing capability of human experts. Furthermore, many
visual features in images, such as irregular shapes and jumbled
textures, are extremely difficult to describe in text. Such text
based approach also limits the scope of searching to that
predetermined by the author of the system and leaves no means
for using the data beyond that scope.
A. Content based Image Retrieval
Content-based image retrieval (CBIR) methods support full
retrieval by visual content / properties of images, i.e., retrieving
image data at a perceptual level with objective and quantitative
measurements of the visual content and integration of image
processing, pattern recognition and computer vision.
International Journal of Modern Computer Science (IJMCS) ISSN: 2320-7868 (Online) Volume 4, Issue 5, October, 2016
RES Publication © 2012 Page | 133 http://ijmcs.info
Figure1. General architecture of the CBIR system
A block diagram of a general CBIR system, shown in figure
1, comprises of modules for feature extraction, feature
indexing, similarity, measurement, query feature extraction and
query interface [1]. In the feature extraction subsystem, image
processing and pattern analysis techniques are used to extract
numeric descriptors of various visual features. Each feature
may have several representations and generally a weight is
assigned to each of these features and their descriptors or
representations. Color is the most widely used feature in any
general domain CBIR system. To retrieve images, the user
submits a query example image to the system and the example
image is then converted into an internal feature vector via a
feature extraction process. The similarity or closeness between
the feature vector of the user‟s query image and those which
are present as feature meta data from the image database
already present is calculated and ranked based on similarity. In
the comparison stage retrieval is performed by applying an
indexing scheme which can be used to support fast retrieval
and to make the system scalable to large image databases. If the
retrieval results are not fully satisfactory, the user can give
some positive or negative feedback to the system and the
modified query can be resubmitted via the interactive relevance
feedback. Such feedback-retrieval cycle can be repeated until
the user is satisfied with the retrieval results.
1. CBIR in Medical Domain (CBMIR)
In the medical domain, the majority of acquired medical
images are currently stored with a limited text-based
description of their content. As image databases expand, it is
becoming increasingly apparent that these simple text-based
descriptions are inadequate for the proper search and retrieval
of medical images. As a consequence, valuable diagnostic and
prognostic information contained in such databases remains
unusable, and it leads to an increased demand for efficient
retrieval techniques which can tap the expertise contained in
these databases. Modern standards such as Digital Imaging and
Communication in Medicine (DICOM) and Picture Archival
and Communication Systems (PACS) make it relatively easy to
store and transport these images and increase interoperability
[4]. Usually the information related to the images is stored in
DICOM image headers. But DICOM headers have proven to
contain a fairly high rate of errors, for example for the field
anatomical region, error rates of 16% has been reported. This
can hinder the correct retrieval of all wanted images.
Hence CBIR has been shown to be a viable alternative to text
based image retrieval, with the abilities to search for an image
depending on metrics for comparing image with visual/ spatial
properties that can match human judgments of similarity. It is
therefore very natural to apply such CBIR to medical domain
(CBMIR) by retrieving medical images according to their
domain-specific image features, providing an important
alternative and complement to the traditional text-based
retrieval. The potential benefits emanating from the CBMIR
range from clinical decision support to medical education and
research. Clinical knowledge has shown that visual
characteristics of medical images have strong effect on
diagnosis. Therefore, diagnosis by comparing past and current
medical images associated with pathologic conditions has
become one of the primary approaches in case-based reasoning
or evidence-based medicine, while the clinical decision-making
process can be beneficial to find other images of the same
modality, the same anatomic region of the same disease[6]. The
CBMIR could aid on such diagnosis in the following way: by
observing an abnormality in a diagnostic image, the physician
can query a database of known cases to retrieve images (and
associated textual information) that contain regions with
features similar to what is observed in the image of interest.
With the knowledge of disease entities that match features of
the selected region, the physician can be more confident of the
diagnosis, and maybe to expand the differential diagnosis to
include pathological entities not previously considered. Here
the CBMIR provides relevant supporting evidence from prior
known cases, offering the physician with training-set examples
that are close to his/her decision boundary, along with the
correct class labels (proven pathology) for these examples. As
such, a less experienced practitioner can benefit from this
expertise in that the retrieved images, if visually similar,
provide the role of an expert consultant. The CBMIR could be
used to present cases that are not only similar in diagnosis, but
also similar in appearance and in cases with visual similarity
but different diagnoses.
II. METHODS FOR IMPLEMENTING CBIR
Content-based image retrieval hinges on the ability of the
algorithms to extract pertinent image features and organize
them in a way that represents the image content. Additionally,
the algorithms should be able to quantify the similarity between
the query visual and the database candidate for the image
content as perceived by the viewer. Thus, there is a systemic
component to CBIR and a more challenging semantic
component.
A. Shape Based Method
International Journal of Modern Computer Science (IJMCS) ISSN: 2320-7868 (Online) Volume 4, Issue 5, October, 2016
RES Publication © 2012 Page | 134 http://ijmcs.info
For shape based image retrieval, the image feature extracted
is usually an N dimensional feature vector which can be
regarded as a point in a N dimensional space. Once images are
indexed into the database using the extracted feature vectors,
the retrieval of images is essentially the determination of
similarity between the query image and the target images in
database, which is essentially the determination of distance
between the feature vectors representing the images. The
desirable distance measure should reflect human perception.
Various similarity measures have been exploited in image
retrieval. In one of the implementation by Muller .H they have
used Euclidean distance for similarity measurement.
B. Texture Based Method
Texture measures have an even larger variety than color
measures. Some common measures for capturing the texture of
images are wavelets and Gabor filters[5]. The texture measures
try to capture the characteristics of images or image parts with
respect to changes in certain directions and scale of changes.
This is most useful for regions or images with homogeneous
texture.
C. Continuous Feature Selection Method
This method deals with the “dimensionality curse” and the
semantic gap problem. The method applies statistical
association rule mining to relate low-level features with high-
level specialist‟s knowledge about the image, in order to reduce
the semantic gap existing between the image representation and
interpretation. These rules are employed to weigh the features
according to their relevance. The dimensionality reduction is
performed by discarding the irrelevant features (the ones whose
weight are null). The obtained weights are used to calculate the
similarity between the images during the content-based
searching[8]. Experiments performed show that the proposed
method improves the query precision up to 38%. Moreover, the
method is efficient, since the complexity of the query
processing decreases along the dimensionality reduction of the
feature vector.
D. Retrieval based on Color
Since the majority of medical images are intensity-only-
images carrying less information than color images, the color-
based retrieval would only be applicable to medical images
based on light photography, where color is an inherent feature
and any deviations or changes in the color of a particular
sample from normal sample can have significant medical
implications. Such medical images that could benefit from the
color-based CBMIR include histological images,
dermatoscopic images, endoscopic images, and tongue images,
in which the well-known color moments and color histogram
approaches are often adopted. These images possess uniquw
color signatures from which the RGB(Red, Green, Blue)
components and the HSV(Hue, Saturation, Value) color space
are analysed and local features are extracted from the image.
E. Retrieval by 3D Volumetric Features
The majority of medical images capture human anatomy that
is in nature a 3D structure. The 3D medical images can be used
for non-destructive inspection of the body and its component
regions in vivo and in vitro, supporting execution and
monitoring of interventions, and providing quantitative
measurements to determine if an abnormality is present by
comparison with normal controls in diagnosis and treatment
planning[5]. However, the feature extraction and retrieval of
3D medical images in most of the existing CBMIR systems so
far are still performed based on 2D slices – simply following
the way of the conversional image retrieval in 2D domain, and
departing from their originally obtained 3D form. It is believed
that the more accurate CBMIR with more discriminating power
can be achieved if we can take full advantage of the
information available in 3D spatial domain by performing
retrieval of these medical images based on their 3D volumetric
features.
A CBMIR approach based on 3DiMSP(ideal midsagittal
plane) features for retrieving 3D CT neuroimages of
haemorrhage (blood), bland infarct (stroke) and normal brains
was proposed. Giving the following basic observation of
neuroradiologic imaging: normal human brains exhibit an
approximate bilateral symmetry which is often absent in
pathological brains, a symmetry detector was firstly
constructed to automatically extract an iMSP – a virtual
geometric plane about which the 3D anatomical structure
captured in the given brain image presents maximum bilateral
symmetry. The basic idea is to find where the MSP is supposed
to be for a given 3D brain image, especially for pathological
brains where the anatomic MSP is often distorted(shifted or
bent) due to large lesions. Automatically locating and retrieving
possible lesions (such as bleeds, stroke, tumours) can therefore,
be conducted by detecting asymmetrical regions with respect to
the extracted iMSP[7]. After the iMSP is aligned in the middle
of each 3D volumetric image, a stack of cross-sectional 2D
slices was temporarily derived as basic image units, from
which three types of asymmetry features were then calculated
to quantify and capture the statistical distribution difference of
various brain asymmetries: global statistical properties;
measures of asymmetry of halved and quartered brains; and
local asymmetrical region-based properties. These features
were calculated from the original image unit with its iMSP
aligned, the difference image of the original image and its
mirror reflection with respect to iMSP, the thresholded
difference image, and the original image masked by the
thresholded binary image. For each image unit, a feature vector
with 48 image features was constructed, including the means,
International Journal of Modern Computer Science (IJMCS) ISSN: 2320-7868 (Online) Volume 4, Issue 5, October, 2016
RES Publication © 2012 Page | 135 http://ijmcs.info
the variances, and the X and Y gradients of the intensity images
at different regions and under various Gaussian smoothing,
scaling and thresholding.
F. Retrieval by Semantic Pathology Interpretation
Sometimes medical images derived from a specific organ are
similar visually and normally differ only in small details which
could be disregarded by untrained eyes but such domain
specific subtle differences maybe of pathological significance .
One such domain is medical radiology, for which the clinically
useful information consists of gray level variations in highly
localized regions of the image. For example, for HRCT images
of the lung, the number of pathology bearing pixels as a
fraction of all the image pixels is so small that it is not possible
to use global signatures for image characterization. Moreover,
the unclear boundaries between the pathology bearing pixels
and the surrounding healthy part are difficult to discern and
such pathology bearing regions (PBRs) are unlikely to be
automatically extracted. Since medical images frequently give
rise to ambiguity in interpretation and in diagnosis, current
retrieval techniques using the primitive image characteristics
such as color, texture, and shape, are likely to be insufficient
for some domain-specific CBMIR systems. One approach to
remedy such limitation is to associate high-level semantic
information with low-level visual image data, albeit such
approach is still controversial in terms of subjectiveness.
G. By Physiological Functional Features
Physiological function can be estimated at the molecular level
by observing the behaviour of a small quantity of an
administered substance „tagged‟ with radioactive atoms.
Images are formed by the external detection of gamma rays
emitted from the patient when the radioactive atoms decay.
Glucose metabolism, oxygen utilisation, and blood flow in the
brain and heart can be measured with compounds labelled with
carbon (11
C), fluorine (18
F), nitrogen (13
N), and oxygen (15
O),
which are the major elemental constituents of the body[9].
Existing CBMIR approaches may not be optimal when applied
to functional images due to their unique characteristics with
regard to the inherent knowledge of the disease state as it
affects the physiological and biochemical processes before the
morphological change of the body.
III. CONCLUSION
This article presents highlights and reviews of recent
research in medical image retrieval and registration domain,
main technologies, major trends and point out some promising
research directions and combined system architecture for
automatic and efficient diagnosis in a hospital or clinical
environment. The need for the CBIR in medical domain
(CBMIR) and its related challenges were discussed, and
followed by a detailed review on the current major CBMIR
techniques in four different categories: retrieval based on
physical visual features (color and texture); retrieval based on
geometric spatial features (shape, and spatial relationships);
retrieval by combination of semantic and visual features
(semantic pathology interpretation); and retrieval based on
physiological functional features. In summary, we investigate
two important technologies from a combined perspective of
automatic and computer assisted clinical decision-making and
point out some observation about how they can be integrated in
a larger system that complements each other[4]. The need for a
real and practical image based diagnostic system is growing
rapidly as more and more healthcare professionals utilize image
guided diagnosis in their daily activities and research. Although
it is a complex task to build an enterprise class system that is
reliable and robust, benefits it would bring to the health
community is unimaginable. The difficulty faced by CBIR
methods in making in-roads into medical applications can be
attributed to a combination of several factors. Some of the
leading causes can be categorized according to the “gaps”
model presented above. An idealized system can be designed to
overcome all the above gaps, but still fall short of being
accepted into the medical community for lack of (i) useful and
clear querying capability; (ii) meaningful and easily
understandable responses; and (iii) provision to adapt to user
feedback. The opposite is also true to some extent. A
technically mediocre, but promising, system may obtain
valuable end user feedback, and by technical improvement may
increase user acceptance with the application of usability
design principles. Other than item (iii), which still needs
significant research effort, the usability gap can only be bridged
by keeping the end user in mind from early system
development, as well as by conducting well designed usability
studies with targeted users. In general, a high involvement of
the user community in system design and development can
significantly improve adoption and acceptance.
REFERENCES
[1] Prachi.Bhend, Prof.A.N.Cheeran, “Content Based Image
Retrieval in Medical Imaging”, International Journal of
Computational Engineering Research, Vol 03,Issue 8, August
2013
[2] Mrs. Mallika, Mr. S. Anantha Krishnan, “An Efficient Content Based
Image Retrieval Using Hierarchical Clustering Approach”,
International Journal Of Research In Computer Applications
And Robotics, Vol.2, Issue.9, Pg.: 119-126, September 2014
[3] Abdol Hamid Pilevar, “CBMIR: Content-based Image Retrieval
Algorithm for Medical Image Databases”, Journal of Medical
Signals and Sensors, J Med Signals Sens. 2011 Jan-Apr; 1(1):
12–18.
[4] Md. Mahmudur Rahman, Tongyuan Wang, Bipin C. Desai,
“Medical Image Retrieval and Registration: Towards Computer
International Journal of Modern Computer Science (IJMCS) ISSN: 2320-7868 (Online) Volume 4, Issue 5, October, 2016
RES Publication © 2012 Page | 136 http://ijmcs.info
Assisted Diagnostic Approach”, IEEE IDEAS Workshop on
Medical Information Systems: The Digital Hospital, September
2004.
[5] H Müller N Michoux D Banden A Geissbuhler "A Review of
Content-based Image Retrieval Applications Clinical Benefits
And Future Directions" International Journal of Medical
Informatics 73 pp. 1-23 2004.
[6] T. M. Lehmann M. O. Gould C. Thies B. Fischer M. Keysers D.
Kohnen H. Schubert B. B. Wein "Content-Based Image
Retrieval In Medical Applications For Picture Archiving And
Communication Systems" Medical Imaging of SPIE Proceedings
Vol. 5033 San Diego California USA 2003.
[7] Kanupriya, Amanpreet Kaur, “Medical Image Retrieval Using
Content Based Image Retrieval System”, International Journal of
Engineering Research & Technology , Vol. 3 Issue 1, January –
2014.
[8] Ritendra Datta, Jia Li, and James Z. Wang, “Content-Based
Image Retrieval - Approaches and Trends of the New Age”,
MIR‟05,November 11-12, Singapore, 2005.
[9] Ashnil Kumar, Jinman Kim, Weidong Cai, Michael Fulham,
Dagan Feng, “Content-Based Medical Image Retrieval: A
Survey of Applications to Multidimensional and Multimodality
Data”, Society for Imaging Informatics in Medicine, 2013.