review - daniraco.free.frdaniraco.free.fr/pubs/journals/nakai2008mrms.pdf · research and clinical...

141

*Corresponding author, Phone: ＋81-562-44-5651, extension5633, Fax: ＋81-562-46-7827, E-mail: toshi＠nils.go.jp a http://sig.biostr.washington.edu/projects/fm/AboutFM.html

141

Magn Reson Med Sci, Vol. 7, No. 3, pp. 141–155, 2008

REVIEW

Ontology for fMRI as a Biomedical Informatics Method

Toshiharu NAKAI1,*, Epifanio BAGARINAO2, Yoshio TANAKA2, Kayako MATSUO1,and Daniel RACOCEANU3,4

1Functional Brain Imaging Lab, Department of Gerontechnology, National Center forGeriatrics and Gerontology

36–3 Gengo, Morioka-cho, Ohbu, Aichi 474–8522, Japan2Grid Technology Research Center, National Institute of Advanced Industrial Science

and Technology, Tsukuba, Ibaraki, Japan3IPAL-Image Perception, Access and Language, French National Center for Scientiˆc

Research (CNRS, NUS, I2R/A*STAR, UJF), Singapore4Faculty of Sciences, University of Besançon, Besançon, France

(Received April 17, 2008; Accepted July 2, 2008)

Ontological engineering is one of the most challenging topics in biomedical informaticsbecause of its key role in integrating the heterogeneous database used by biomedical infor-mation services. Ontology can translate concepts and their real-world relationships into ex-pressions that can be processed by computer programs or web services, providing a uniquetaxonomic frame to describe a pathway for extracting, processing, storing, and retrievinginformation. In developing clinical functional neuroimaging, which requires the integra-tion of heterogeneous information derived from multimodal measurement of the brain,these features will be indispensable. Neuroimaging ontology is remarkable in that it re-quires detailed description of the hypothesis, the paradigm employed, and a scheme fordata generation. Neuroimaging modalities, such as functional magnetic resonance imaging(fMRI), magnetoencephalography (MEG), electroencephalography (EEG), and near in-frared spectroscopy (NIRS), share similar application purposes, imaging protocol, analyz-ing methods, and data structure; semantic gaps that remain among the modalities will bebridged as ontology develops. High-performance, global resource information database(GRID) computing and the applications organized as service-oriented computing (SOC)will support the heavy processing to integrate the heterogeneous neuroimaging system. Wehave been developing such a distributed intelligent neuroimaging system for real-timefMRI analysis, called BAXGRID, and a neuroimaging database. The fMRI ontology ofthis system will be integrated with established medical ontologies, such as the Uniˆed Medi-cal Language System (UMLS).

Keywords: BAXGRID, functional magnetic resonance imaging (fMRI), GRID, neuro-imaging, ontology

Introduction

Since the discovery of the blood-oxygen-level de-pendent (BOLD) phenomenon,1 functional mag-netic resonance imaging (fMRI) has contributedenormously to cognitive neuroscience, but stand-ardization of the activation map and precision ofindividual analysis remain as technical issues forclinical applications of fMRI. We review the appli-

cation of ontology in biomedicine and initiate astudy of ontology for neuroimaging; we focus espe-cially on fMRI because it is indispensable for com-paring multi-center data as well as for integratingneuroimaging data to improve the reliability offunctional brain maps. In medicine, well-knownontologies include the Uniˆed Medical LanguageSystem (UMLS), Systematized Nomenclature ofMedicine (SNOMED), and Foundational Model ofAnatomy (FMAa). Besides these systematic frame-

142

b http://www.ebi.ac.uk/ontology-lookup/c http://protege.stanford.edu/

142 T. Nakai et al.

Magnetic Resonance in Medical Sciences

works that cover the entire medical ˆeld, ontologycan be applied for speciˆc biomedical research pur-poses, such as verifying the radiologic-pathologiccorrelation of brain tumor cases for multicenterstudy.2 Although this ontological approach wasuseful in evaluating the relationship between thesensitivity/speciˆcity of pathology and the phrase-ology employed for neuroimaging classiˆcation, af-ter dataset assembly, post-test probability estima-tion remained. This retrospective approach is closerto indexing text contents than controlling lexiconsfor coding schemes to frame the features of imagesand pathology. Neuroimaging ontology will sys-tematically organize the knowledge and techniquesfor cognitive task design, experimental procedure,neuroanatomy, brain signals, and behavioral dataso that imaging sessions can be precisely repro-duced and knowledge eŠectively shared for bothresearch and clinical purposes. In particular, ontol-ogy will enhance multimodal neuroimaging, whichre‰ects neuronal activation via diŠerent physicalprinciples. We herein summarize the concept of on-tology and its development in bio- and medical in-formatics and discuss the application of ontologyfor neuroimaging, especially for the fMRI data-base, and the direction for future development.

Ontology as Technology for Formal and Logi-cal Expressions

Ontology originated as a philosophical conceptin Greece that explored the essence of things. Incomputer science, its meaning is diŠerent.3 We willnot explore philosophical considerations, and un-like computer science, we do not believe that ontol-ogy is a mechanism for building queries by using acommon ontological form mapped to each under-lying resource. Rather, we consider ontologies asexplicit formal speciˆcations of the terms in thearea of interest (domain) and the relations amongthem;4 ontology is a technology that represents for-mal and logical expressions of the concepts relatedto the domain. It aims to describe the standard hi-erarchical structure of the classes (concepts), sub-classes, properties of each class (slots), i.e. its vari-ous features and attributes, and restrictions on slots(facets). A knowledge base consists of a group ofindividual instances of classes. Thus, ontologyrepresents a model of the reality of the world, andthe concepts in ontology must re‰ect this reality(Fig. 1). This process of iterative design will likelycontinue throughout the life cycle of the ontology(Fig. 2). Examples of published ontologies can bechecked using the ontology look-up service (OLSb).

Ontologies are developed to share a common un-

derstanding of the structure of information amongpeople or software agents, enable reuse of domainknowledge, make domain assumptions explicit,separate domain from operational knowledge, andstructure and analyze domain knowledge.

The most remarkable example of ontology issemantic web design to enhance visibility of knowl-edge on the web.5 Using several languages, such asExtensible Markup Language (XML), XML sche-ma, Resource Description Framework (RDF), RDFschema (RDFS), and Web Ontology Language(OWL), the semantic web enables the descriptionof information that is understandable by comput-ers so that they can perform more of the tediouswork involved in ˆnding, sharing, and combininginformation supplied by the web. These languagesform a stack in the architectural layer of a semanticweb: RDF oŠers a simple graph reference model;RDFS, a simple vocabulary and axioms for object-oriented modeling; and OWL, an additionalknowledge base oriented toward ontology con-structs and axioms. These ontological languagesare themselves meta-ontology, instances of whichare semantic web ontology.

A single bit of knowledge consists of 3 elements,subject, property (predicate), and value (object).This RDF triple is stored in the RDF database (tri-ple stored) as instances of ontology in the semanticweb according to the deˆnition of the ontologicallanguage. The query languages, such as RDQL(RDF Data Query Language), have been imple-mented in a number of RDF systems for extract-ing information from RDF graphs. The RDF graphis a directed labeled graph to describe RDF datamodels; it consists of a set of nodes connected byarcs that form a node-arc-node pattern.6 Prot áeg áec,the popular ontology editor, is also an RDFS editorand provides interfaces to create/edit ontologiesand store them in RDF/XML or OWL format.Inclusion of these ontology resources enables aservice to be semantic.

Bio-ontology

Ontologies support recent rapid and gross ad-vances in biosciences and biotechnologies. Mostconcepts in biomedicine cannot be expressed as for-mulas, but the entities can be described in naturallanguage. To merge heterogeneous knowledgecomponents in biomedicine and enable computa-tional query of the concepts of interest, the onto-logical ``conceptualization of reality'' is essential in

143

Fig. 1. Ontology uses ontological languages to translate the concepts and instances and theirrelationships in the real world into knowledge base resources. This ˆgure shows the scheme in func-tional magnetic resonance imaging (fMRI). In the real world, subjects perform tasks based on ahypothesis. Paradigm generators give the subjects stimuli designed to invoke hypothetical neuronalactivities of interest. Neuronal ˆrings are detected as a hemodynamic response to the bloodoxygenation level-dependent (BOLD) phenomenon, and ‰uctuation of signal intensity of echo-pla-nar imaging (EPI) is detected by statistical analysis. Ontology translates this series of biologicalphenomenon in the real world into systematic descriptions for knowledge-based informationresources. This knowledge base enables not only semantic indexing of the database contents or fur-ther data mining but also standardization of paradigm generation and imaging protocols. CBIR,content-based image retrieval

d http://www.geneontology.org/ e http://obofoundry.org/

143Ontology for fMRI

Vol. 7 No. 3, 2008

deciding indexing performance and characterizingdomain.

Protein Ontology (PO)7 and Gene Ontology(GO) are 2 remarkable bio-ontological ˆelds. Pro-tein Ontology consists of various domains, such asa protein kinase resource, peptidase database, andtranscription factor database,7 that assist researchand diagnosis. Gene Ontologyd has been developedcollaboratively to support the biologically mean-ingful annotation of genes and their products.8 The4 central domains of GO are molecular function,biological process, cellular component, and se-quence features. The main part of the ontology ˆlerefers to these domains, and the annotation ˆles aretaxon-speciˆc ˆlters for database projects dedicat-ed to species. These main parts of GO are correlat-ed to external databases by mapping ˆles, which at-tempt to regress the concepts of the external data-

bases, although translation is not necessarily com-plete. To explore a number of Gene Ontology data,search tools have been provided to answer ques-tions of biological interest by using natural lan-guage.9 Expansion of Gene Ontology to include thedesign of a measurement protocol was designedinto the ontology for Microarray Gene ExpressionData (MGED),10 which provides semantics to de-scribe the treatment of the sample and microarraychip technology according to the concepts speci-ˆed. Neuroimaging ontologies will employ this ap-proach.

Open Biomedical Ontology (OBO)e is a wide-spectrum collaborative project of science-based on-tology developers, including PO and GO. Its goal isto create a suite of orthogonal interoperable refer-ence ontologies in the biomedical domain. OBO isopen to any developers and has provided ˆlters to

144

Fig. 2. The ``real world'' is inspection, neurological examination, or diagno-sis in clinics, functional magnetic resonance imaging (fMRI) or other neuro-imaging, and treatment. Neuroimaging ontology handles the acquisition of in-formation from the real world, knowledge building in the database, query ac-cess for decision-making by users, and connection to other knowledge units.The results of medical evaluation and treatment are re‰ected in the real world.As＝assessment of information in the real world; IF＝(user) interface

f DICOM–Digital Imaging and COmmunication in Medicine,http://medical.nema.org/

g PACS–Picture Archiving and Communication Systems

144 T. Nakai et al.


inter-map the ontologies and attempt to establish acommon design philosophy and implementation inthe biomedical domain.

UMLS Usage in the Current Medical Informat-ic System and Ontology for Medical Images

In modern hospitals, digital images are producedin huge quantities and used primarily for immedi-ate diagnosis and therapy. In medical informatics,despite the introduction of Digital Imaging andCommunications in Medicine (DICOM)f, medicalimage format standardization, and picture archiv-ing and communication system (PACS)g medicalinformation storage and management systems,much eŠort is needed to use these standards e‹-ciently and eŠectively for diagnosis assistance. Inthe same way that PACS expands the possibilitiesof conventional hard-copy medical image storageby providing capabilities for oŠ-site viewing andreporting and simultaneous access to informationby practitioners at various physical locations,Content-based Medical Image Retrieval (CBMIR)11

opens the way to the next generation of medicalprocedures. For instance, CBMIR systems couldprovide advanced diagnosis assistance and set up

semantic links between the related medical infor-mation to improve health care. Furthermore, data-mining could be used for research purposes, med-ical query expansion, and evidence-based medicine(EBM) and image-based reasoning (IBR) applica-tions generated by similarity-based image retrieval.In addition, as medical domain knowledge becomesmore complex, the decision support systems in radi-ology and computer-aided diagnostics for radiolog-ical practice need more powerful data and meta-data management and retrieval.

The role of ontologies will be increasingly moreimportant in all these medical image- and medicalmultimedia-based management, retrieval, and rea-soning systems. Common components of ontolo-gies (individuals, classes, attributes, relations, res-trictions, rules, assertions, events, and others) con-stitute an interesting support to formalize equiva-lent medical knowledge models (diagnosis rules,radiologic clues, decision trees, contextual graphs,and others). Because of the critical responsibility ofmedical doctors and the sensitivity to false-negativeresponses in medicine, more eŠorts should be madein medical computer-aided systems to validate exist-ing knowledge models (truth maintenance systems,etc.).12 Nevertheless, inspiration from semantic webapproaches can ensure a coherent approach, guid-ing the medical informatics community to the nextgeneration of the Medical Multimedia Semanticwebh. Among existing medical ontologies, one of

145

Table 1. Well-known uniêd medical language system (UMLS) metathesaurus sources

CPT Current Procedural Terminology http://www.ama-assn.orgICD–9-CM International Classiˆcation of

Diseases, Ninth Revision, ClinicalModiˆcation

U.S. Department of Health and Human ServicesCenters for Medicare & Medicaid ServicesBaltimore, MD

LOINC Logical Observation IdentiêrNames and Codes

The Regenstrief InstituteIndianapolis, IN

MeSH Medical Subject Headings National Library of Medicine, Bethesda, MDhttp://www.nlm.nih.gov/pubs/factsheets/mesh.html

NLM-MED National Library of MedicineMedline Data

http://www.nlm.nih.gov/

RxNorm RxNorm Vocabulary http://www.nlm.nih.gov/research/umls/rxnorm/index.htmlSNOMED SNOMED Clinical Terms College of American Pathologists, Chicago, IL

http://www.snomed.org

h MMedWeb project (A*STAR, NUS, IPAL, Singapore)–http://ipal.i2r.a-star.edu.sg/Projects/index.html.

i US National Library of Medecine–http://www.nlm.nih.gov/

j About 150 vocabulary sources that contribute strings orrelationships to the 2007AC UMLS Metathesaurus are listedat: http://www.nlm.nih.gov/research/umls/metab4.html

k MetamorphoSys–an UMLS installation wizard and customi-zation tool included in each UMLS release

l Neuronames Brain Hierarchy, Seattle (WA): University ofWashington,PrimateInformationCenter.http://rprcsgi.rprc.washington.edu/neuronames/

m National Center for Biomedical Ontology–http://www.bioontology.org/resources.html


Vol. 7 No. 3, 2008

the most complete and most used is the NationalLibrary of Medicine's (NLM)i UMLS. Its purposeis to facilitate the development of computer sys-tems that behave as if they `ùnderstand'' themeaning of the language of biomedicine andhealth. The UMLS Metathesaurusj is a large, mul-tipurpose, and multilingual vocabulary databasethat contains information about biomedical andhealth-related concepts, their various names, andthe relationships among them. All concepts in theMetathesaurus are assigned to at least one semantictype from the semantic network, providing consis-tent categorization of all UMLS concepts.

The UMLS Metathesaurus knowledge sourceuses several tools (programs) to ˆlter the UMLSconcepts and relationships needed for a particularêld or application; MetamorphoSysk creates use-ful UMLS Metathesaurus subsets by selecting ap-propriate sources (Table 1) and applying ˆlters andoptions to reˆne selected source content in cus-tomized subsets.

UMLS utilizes the structured medical Metathe-saurus, which allows homogeneous fusion betweenUMLS-compliant concepts from diŠerent medicalmedia (images, reports, and others)13,14 as well asautomatic query expansion and rule extraction.The Metathesaurus is updated many times a year.

Some of the source vocabularies of the UMLSMetathesaurus, such as the Neuronames Brain

Hierarchy (NEU)l, can be used as a basis for neu-roimaging ontology. The ontology obtained afterMetamorphoSys ˆlters these sources should becompleted and validated with the help of special-ized ontology validation on-line servicesm assistedby neurospecialists.

Medical image analysis and conceptualization isan important use of medical ontology for medicalimage management. Each image (Fig. 3) or regionof interest (ROI) from the image is associated witha semantic label that corresponds to a combinationof UMLS concepts and visual percepts (visual voca-bulary).15 At least 3 types of UMLS concepts (seeImage Retrieval in Medical Applications (IRMA)code)16 can be deˆned17 that could be associated toone image or region: modality concepts belongingto the UMLS semantic type ``Diagnostic Proce-dure''; anatomy concepts belonging to UMLSsemantic types ``Body Part, Organ, or Organ Com-ponent'' or ``Body Location or Region''; and pa-thology concepts belonging to the UMLS semantictypes `Àcquired Abnormality'' or ``Disease orSyndrome.''

A structured learning framework based on Sup-port Vector Machines (SVM) is often used17 tofacilitate modular design and extract medicalsemantics from images. Complementary indexingapproaches are developed within this statisticallearning framework: global indexing to access im-age modality; local indexing to access semantic lo-cal features; anatomy concept; and pathology con-cepts. Because the global approach seems to be

146

Fig. 3. a) Medical image classiˆcation according to modality concept: example of use of low-levelfeature (3 moment herpes simplex virus [HSV], gray-level histogram, HSV histogram, color statisti-cal parameters, texture-Gabor ˆlters, thumbnails, etc.) as input for support vector machine (SVM;one-versus-all approach) to identify high-level semantic information associated with the modalityconcept in the neuroimage content. b) Modality tree showing the hierarchy between modality con-cepts retained for the classiˆcation. Link with uniêd medical language system (UMLS) conceptunique identiêr (CUI). c) Example of medical image classiˆcation based on SVM.

146 T. Nakai et al.


e‹cient but the training is time consuming andtraining-set sensitive, the local approach seems tobe e‹cient for classiˆcation but less e‹cient formedical image retrieval, at least in its regularGlobal Resource Information Database (GRID)form. The solution may pass through patches(visual vocabularies) assigned to particular ROIs inmedical images. The size and position of the patch-es seem to be the main concern for such pathology,modality, and anatomy-related approach usingadaptive patches.

In most classic approaches, each classiêr istrained in the `òne-versus-all'' (OVA) mode (theconcept of interest versus everything else); we referto this semantic labeling framework as supervisedOVA. There has been an eŠort to solve a problemin greater generality by resorting to unsupervisedlearning,18 particularly by latent semantic analy-sis.19,20 An interesting initiative (applied to naturalimages)21 proposes combining the advantages ofOVA and unsupervised formulation through a

reformulation of the supervised formulation. Thisconsists of deˆning a multiclass classiˆcation prob-lem, in which each semantic concept of interest de-ˆnes an image class. This Supervised MulticlassLabeling (SML) formulation retains the classiˆca-tion and retrieval optimality of supervised OVA aswell as its ability to avoid restrictive independenceassumptions.

Although early retrieval architectures were basedon the query-by-example paradigm, which formu-lates image retrieval as a search for the best data-base match to a user-provided query image, it wasquickly realized that the design of fully functionalretrieval systems would require support for seman-tic queries, i.e., use of ontologies,22 which openedthe way to content-context-based EBM, able to givemore pertinence and reliability to a classical EBMapproach and based essentially on group patientstatistics.

147

n http://www.creatis.insa-lyon.fr/MEDIGRID/o Press release: http://www–03.ibm.com/press/us/en/pressrelease/21553.wss

p http://www.medgrid.org/

q Participating institutes include the National Institute of Ad-vanced Industrial Science and Technology (AIST), the Na-tional Center for Geriatrics and Gerontology (NCGG), andOsaka University (OU) all in Japan, the Ateneo de ManilaUniversity (ADMU) in the Philippines, and the National Tai-wan University (NTU) in Taiwan.

Fig. 4. The graphical user interface (GUI) of BAX-SQL, showing several datasets from 2 remote dataservers. Users interact directly with the GUI andmanipulate remote datasets using commands accessi-ble from the menu items provided. Access to remotedata servers is transparent to the user as if datasetsare located locally.


Vol. 7 No. 3, 2008

Functional MR Imaging Database System forGlobal Access

Establishing a shared ontology for fMRI oŠersseveral advantages: users can transparently retrievedata from across diŠerent fMRI database systems;interoperability of the diŠerent applications forfMRI analysis is ensured, which is useful for com-puter-mediated meta-analysis of datasets; andtogether with GRID technology,23 a shared ontolo-gy can facilitate global access to distributed fMRIdatabase systems, which until now have remainedindependent from each other.

GRID technology can be used to facilitate con-trolled sharing and management of a large numberof distributed datasets, such as digital medical im-ages. Replication of medical images will be un-necessary when they can be shared over wide net-works. It can also optimize the use of storageresources by pooling heterogeneous storage be-tween distributed sites and enabling existing medi-cal applications, such as PACS, to treat distributedimages as if they are local.

Several initiatives have explored the use of GRIDtechnology to enable database systems for secureglobal access. One, the MammoGrid project,24,25

used the GRID as its information infrastructure todevelop a Europe-wide database of mammograms.Another Europe-based project, the MediGridn,aimed to explore the use of GRID technology toprocess medical image databases available in hospi-tals today. The Globus Medicus project extendedthe Globus Toolkit's capability to provide seamlessGRID integration of the DICOM standard proto-col used in most healthcare and medical researchinstitutes. Some players in the information technol-ogy industry have also started oŠering an enter-prise GRID-based business solution to medical da-ta storage and management. Early last year, IBMstarted oŠering its new GRID Medical Archive So-lution (GMAS), providing hospitals with a multi-tier, -application, and -site enterprise storage ar-chiveo.

Similar projects have been undertaken in the ˆeldof neuroscience. One, the Medical GRID (Med-Grid) projectp, aimed to study and demonstrate theuse of GRID technology in analyzing and manag-ing functional MR imaging datasets. The MedGridtestbed was formed in 2004 and involved research-ers from 5 institutions in 3 countriesq. To enable

global access to shared data sources availablewithin the testbed, a GRID-based fMRI data man-agement and analysis tool, called BAXSQL,26 wasdeveloped (Fig. 4).

BAXSQL facilitates the federation of fMRIdatasets from the 3 data servers available in theMedGrid testbed. Its features include multi-data-base querying, dataset selection, and downloadcapabilities to the local machine. Moreover, it hasseveral built-in functions for common functionalMR imaging analysis routines, such as realignment,smoothing, and standard general linear model(GLM)-based statistical analysis. The routines areimplemented such that the testbed's analysis serversare used to perform the computations.

Security is important for systems allowing globalaccess. BAXSQL implements a 2-level securitymechanism using standard-based GRID technolo-gy. On the GRID level, each user needs a standardX.509 security certiˆcate issued by the virtual or-ganization's (VO) certiˆcate authority. This certiˆ-

148

r http://ndg.sfn.org/s http://www.fmridc.org/t http://www.nitrc.org/

148 T. Nakai et al.


cate can be employed to authenticate the user whenresources within the GRID are utilized. In addition,data server owners need to grant access privilegesbefore GRID users can access stored fMRIdatasets. This is controlled by the backend data-base's access control mechanism. The implicationis that even if the user gains access to the data serv-er using his X.509 certiˆcate, he still may be unableto manipulate the stored datasets without properprivileges being granted by the data server's owner.

Overall, BAXSQL can be used to build fMRIdatabase systems accessible globally over the publicInternet and with appropriate security features. Itcan facilitate the sharing of fMRI datasets evenacross national borders. To attain maximum com-patibility, BAXSQL uses only a limited number ofmetadata to describe fMRI datasets, but as moreresearch groups share their respective datasets, astandard for fMRI data sharing is necessary. Thus,a shared ontology for fMRI is very important. Fu-ture releases of BAXSQL will support a commonontology for fMRI data management and analysis.

Ontology for fMRI

In this section, we conceptualize fMRI ontologyfor clinical neuroimaging as a tutorial of ontologybuilding. Bodenreider and colleagues noted 7points to ensure successful ontology development:community involvement, clear goals, limited scope,simple structure, continuous evolution, active cura-tion, and early use.3 From this perspective, thescientiˆc neuroimaging ontology is intended to es-tablish consistent annotation among the cognitiveprocess, functional map overlaid to neuroanatomy,measurement methods, and analysis principles sothat any examiner can reproduce brain mapping.Its scope is to explore the principles and structuresof the neuronal system, clarifying the mechanismsof cognition and approaching the theory of mindby merging major neuroimaging modalities, suchas fMRI, magnetoencephalography (MEG), elec-troencephalography (EEG), and near infrared spec-troscopy (NIRS). The goal of clinical neuroimagingontology, in contrast, is to establish an intelligentassistance system for early diagnosis of cognitiveimpairment, preoperative mapping, investigationsof pathophysiological status, and monitoring oftreatment. Its scope will be more focused on thequery of similar brain images and activation mapsfor individual assessment. Although scientiˆc andclinical neuroimaging will share the majority ofresources, the annotation of concepts and instanceswill be slightly diŠerent, which can be explained bythe contrast of ``research ‰ow'' and ``clinical ‰ow''

(Fig. 5).The Neuroscience Database Gateway (NSD) lists

several fMRI database projectsr. The fMRI DataCenter (fMRI-DC)s, a representative project to sup-port the fMRI research community, supplies com-mon datasets for meta-analyses and future develop-ment of new methods. The data center (DC) tool isdedicated to register the original datasets submittedby researchers, and it serves browsing and dis-playing functions, all of which are written in Javaprogramming language. They also developed anontological template for data management basedon the Prot áeg áe 2000 platform. Table 2 lists themain classes. Overall, the ontology of fMRI DC isdedicated to support neuroscience research com-munities so that registered researchers can interactand exchange knowledge. The Source for Neuro-imaging Tools and Resources (NITRCt), a projectinitiated in 2007, is a consortium supported by theNational Institutes of Health (NIH) to identifyfMRI tools and resources for the neuroimagingcommunity, provide information about these toolsand access to them in a common format, facilitatecommunity interaction to make them more usableby a broader research community, and facilitateneuroimaging science and neuroinformatics col-laboration and education. The project includes on-tology development.

Determining the domain and scope of the ontologyThe database access and management style will

be diŠerent for clinical neuroimaging databaseresources. Only trained and qualiˆed experts cancontribute contents to the database according topredeˆned guidelines. Research-based paradigmsare not necessarily applicable to the clinical index.Clinical users are mostly interested in retrievingdata as reference activation maps to compare withindividual neuroimaging results; they are not neces-sarily supposed to expand the database. The do-main of ontology we model is to cover fMRI dataindexing and retrieval to assist in clinical neuropsy-chological diagnosis. The basic question that ontol-ogy should answer is how an individual activationmap matches the standard data for each paradigmor condition. For clinical neuroimaging, the ques-tions are: 1) What tasks should be chosen for thesubject based on the symptoms and ˆndings in neu-ropsychological examinations? 2) How was thevariance of the patient's activation from that ofnormal subjects? 3) What are the history, clinical

149

Table 2. Major classes in the ontology of functionalmagnetic resonance imaging (fMRI) data center (DC)

Analysis (overview of analysis)Analysis transform (data processing procedures)Date (days since epoch)Event (description of the examination)Experiment description (scan protocols, task design)Experimental data (personal, behavioral, and rundata)Machine (types of magnetic resonance [MR] systemand radiofrequency [RF] coils)Measurement (display measurement)Miscellaneous (analysis context, bulk data, keywords,data format, type of experiment)Person (researcher, subject)Text (format for the texts)Time of day (oŠset time)Timestamp (scale of timestamps)Unit of measure (time, length, magnetic êld, etc.)URI (uri value)System-class (basic deˆnition of ontology structure)

u http://protege.stanford.edu/publications/ontology_development/ontology101.html

Fig. 5. Comparison of ``clinical'' and ``scientiˆc'' neuroimaging. Imaging and analysis tech-niques are the same, but purpose and information of interests are diŠerent. This should berepresented in the ontology of each type of functional magnetic resonance imaging (fMRI) usage.The science mode requires preliminary studies to develop or revise the paradigms, the protocol forfMRI experiments, and related recording methods.


Vol. 7 No. 3, 2008

status, pathology, behavior, brain functional dataobtained by other modalities, and genetic informa-tion of subjects who have similar activation mapsto that of the patient? 4) How can the patient's cog-nitive status or dysfunction be characterized andclassiêd? Was cognitive function recovered bytreatment? Was there any secondary or plasticchange of the neuronal system? 5) For preoperativeevaluation, what is the anatomical relationship be-tween the lesion and eloquent areas (language, mo-

tor, or other important centrals)? Can the neuro-logical symptoms be explained by the lesion?

From the viewpoints of the domain and scope offunctional neuroimaging ontology, the followingdetails should be clariêd in the ontology descrip-tion: 1) complete task designs, instructions, and in-strumentations, including behavioral or physiologi-cal data acquisition; 2) standard imaging param-eters for anatomical and functional images; and 3)standard analysis protocols and parameters togenerate an activation map. Including these points,ontology for clinical functional neuroimaging canshare concepts of scientiˆc neuroimaging ontology;therefore, we discussed clinical neuroimaging on-tology by modifying the ontology proposed byfMRI DC, which has already deˆned many of theimportant terms for neuroimaging. The followingsteps tracked the procedure of `Òntology Develop-ment 101'' from the developer groups of Prot áeg áe2000u.

Deˆning the classes and the class hierarchyAfter the essential terms for classes are chosen,

they should be organized into a taxonomic (sub-class-superclass) hierarchy. We combined the top-down and bottom-up approach. Table 3 shows thelist of major classes and initial superclass-subclassdesign. The hierarchy for clinical functional neuro-imaging is organized by the computational struc-ture similarity of concepts. This point is quite relat-

150

Table 3. Initial proposal of functional magnetic resonance imaging (fMRI) ontology

Feature descriptionStudy description

title, category, target diagnosis, synopsis, (references)Examination description

subject, study, session list, simultaneous recordingsSession description

task, imaging parameters, dataTask description

title, category, target cognition, synopsis, task control/resources, (instructions, references)Fusion description

modality, mathematical methodData format deˆnition

DataMagnetic resonance (MR) imaging

functional, anatomical, diŠusionOther functional modality

Electroencephalography (EEG), magnetoencephalography (MEG), near infrared spectroscopy (NIRS),electromyogram (EMG), etc.

Behavioral datareaction time, accuracy, motion performance

Mapactivation, fusion, template

Task resourcespicture, sound, characters, text

Parameter descriptionTask control

block/event related (ER), stimulus onset asynchrony (SOA), duration, jittering, epoch orderImaging

pulse sequence, repetition time (TR), echo time (TE), slice thickness, matrix, ˆeld of vision (FOV)Data preprocessing

motion correction, normalization, smoothingStatistical analysis

statistical method, threshold, post-hoc, covarianceUnit deˆnition

Relational infoID deˆnition

study, examination, session, task, data, subject, personal recordKeywords

anatomy, cognition, physiology, pathology, imaging, data processingIndividual record

Clinical record

Top classes and their subclasses of functional magnetic resonance imaging (fMRI) ontology for clinical applicationbased on medical global resource information database (GRID). The major classes are based on the similarity of thedata type and structure. The subclasses re‰ect the procedure of neuroimaging or the role of the information.

150 T. Nakai et al.


ed to the development of the actual application forimage analysis, data storage, and retrieval to inte-grate multimodal measurements. Among the majorclasses, feature description explains the universalconcepts for the database, experiments, and dataprocessing, whereas parameter description deˆnesthe variable part of data acquisition and analysis.``Relational info'' supports handling of the in-stances and knowledge for user interface andsemantic indexing. The current version is based on

an MR-centered design, but the sub-classes can beexpanded for each neuroimaging modality.

Deˆning the properties of classes and slotsNext, the internal structure of each concept, i.e.,

the property of the class, which comprises slots andfacets of the slot, should be described. Slots charac-terize the class from diŠerent viewpoints, andfacets explain a slot's actual values. Most remain-ing terms other than the classes in the term list of

151

Fig. 6. The class editor view of Prot áeg áe 2000 showing the class hierarchy and the slot template ofclinical functional magnetic resonance imaging (fMRI) ontology. The facets of each slot are listedas column titles of the slot table; their properties are shown in each row.


Vol. 7 No. 3, 2008

the ontology are the properties. Here, 1) all sub-classes of a class inherit the slot of that class, and 2)a slot should be attached to the most general classthat can have that property. Accordingly, the class``Exam_description'' can be organized as in Fig. 6.A class's internal structure can be brie‰y summa-rized as ``the SLOT of the CLASS is the FACET.''For example, ``the study_id (slot) of the study_description (class) is an integer (facet),'' or ``thestudy_category (slot) of the study_description(class) is the class motor_behavior (facet).''

Deˆning the facets of the slots and describing theirallowed values

The properties of a slot are facets. A slot canhave facets to describe value type, cardinality, al-lowed values, classes to be called, and further fea-tures of the values the slot may take. The commontypes of value are a string; a numeric type, such asan integer and ‰oat; Boolean (true/false); anenumerated type listing the possible values; and theinstance type. An instance-type slot indicates a setof instances listed in other classes. Slot cardinalitydeˆnes how many values a slot can have. Figure 7shows an example of a facet set of a slot.

Practically, we frequently encounter the ques-tion, ``Will it be a new class or a property value?''If a distinction is important in the domain and wethink that objects with diŠerent values are recog-nized as diŠerent kinds of objects, then we should

create a new class for distinction. In the example,the diŠerence of ``gradient-strength'' is representedas the maximum extent of phase or frequency en-coding, i.e., the diŠerence of k-space trajectory andthe slice proˆle. ``Gradient-strength'' is related toother imaging parameters, such as the number ofslices in a volume, slice thickness, or echo time(TE). It is a ˆxed property of the hardware relatedto quality issues; however, it is not the primary fac-tor to answer the competency questions mentioned;therefore, ``gradient-strength'' is assigned to a slotin this ontology.

Filling in the values of slots for instancesThe base of the class hierarchy consists of in-

stances. Instances are actual data in the knowledgebase. Creating an individual instance of a class is tochoose a class, creating an individual instance andˆlling the slot values. We encountered a similarquestion as that previous–should it be a class or aninstance? The answer depends on the potential ap-plications of the ontology. If the concept is themost speciˆc representation in the hierarchy of theknowledge base, then it will be an instance. Forexample, ``quadrature birdcage head coil,'' a sub-class of the class ``RF coil,'' represents a type ofelectric circuit for head imaging and can includeseveral products from each vendor; therefore,``quadrature birdcage head coil'' is a class, and theproducts speciˆed by the product name or number

152

Fig. 7. The slot editor view of Prot áeg áe 2000. This example shows the properties of ``gradient-strength'' from fMRI-DC ontology with the unit ``mT/m.'' The domain of this slot is ``MRI Scan-ner'' class. `Àllowed classes'' indicates constraint on the values of a type-instance slot. The valueof the slot ``gradient-strength'' can only be an instance of the class ``Measurement'' or any of itschildren.

v 'BMI' is also used for biomedical imaging or brain machineinterface.

w http://community.healthgrid.org/

x http://www.nbirn.net/y ONtology and COntext related MEdical image DistributedIntelligent Access, http://www.onco-media.com/

152 T. Nakai et al.


are instances. However, if usage of the ontologydoes not require the products to be speciêd,``quadrature birdcage head coil'' may be an in-stance of the class ``RF coils.'' When an instance ofthe class ``quadrature birdcage head coil'' has newproducts and is a modiˆcation of current products,then it will be a new instance, but it cannot be asub-instance of the current product because onlyclasses can be arranged in a hierarchy.

Biomedical Informatics and Future Direction

Medical and bioinformatics have been independ-ent of each other, having diŠerent backgrounds,research and development topics, and application;however, recent advances in biomedical engineer-ing urged merging of the 2 êlds into biomedical in-formatics (BMIv). In Europe, BIOINFOMEDstudy of this fusion was initiated in 2001, and manyprojects have been derived from this idea.27 As in-frastructures to support those projects, GRID com-puting has been employed as HealthGRIDw at thelevels of both data-sharing and human collabora-tion.28,29 In the United States, NIH launched the

Biomedical Informatics Research Network (BIRN)in 2001x. BIRN is more oriented toward medical in-formatics, the major sub-domains of which aremorphometry, fMRI, and animal imaging. Ontolo-gy takes part in organizing the resources ofhealthGRIDs,30 such as biomedical databases, com-puting power, medical experience, medical devices,and management of the projects. In Asia, Med-GRID, a project to apply GRID computing to neu-roimaging, was initiated in 2004, as describedabove. In 2006, the ONCO-Mediay project wasstarted as the collaboration of 5 Asian countriesand Europe. This project attempts to develop anovel GRID-distributed, contextual and semantic-based, intelligent information access frameworkfor medical images and to explore new access appli-cations for medical images in diagnosis assistance,teaching, and research using semantic, visual, andcontext-sensitive medical information with GRIDcomputing. Ontology development is a primary ac-tivity in this project because the ontologic approacheŠectively integrates heterogeneous database sys-tems, which are common in medicine.27,31

Service-oriented computing (SOC)32 is the new


Vol. 7 No. 3, 2008

technical trend in software development for this in-tegration; SOC is intended to compose applicationsby discovering and invoking network-availableservices to accomplish various tasks in an environ-ment of mass distribution. The representative SOC-based technologies, such as the Simple Object Ac-cess Protocol (SOAP), Web Services ChoreographyDescription Language (WS-CDL), and BusinessProcess Execution Language for Web Services(BPEL4WS), are Web services based on openstandards and are employed to assemble applica-tion components into a loosely coupled network ofservices that can enable dynamic processes for thetask. XML-based languages, such as BPEL4WS,are used to `òrchestrate'' the whole system, i.e., todescribe how services interact at the message level,including the business logic and execution order ofinteractions. WS-CDL describes the ``choreogra-phy'' of individual processes of public message ex-changes, interaction rules, and agreements amongmultiple business processes.

Thus, GRID computing achieves high-perfor-mance fMRI for clinical applications and organizesthe neuroimaging database as part of biomedicalinformatics, and ontology for fMRI plays a role inintegrating them into e-health. Ontology is for ap-plication development as well as for indexing neu-roimaging data.

Acknowledgements

This research was supported by a Grant-in-Aidfor Scientiˆc Research (KAKENHI) # 18300179,from the Ministry of Education, Culture, Sports,Science, and Technology, Japan and by Frenchprogram ICT-Asia, supported by the French Minis-try of Foreign AŠairs and the French NationalResearch Center (CNRS), #AFD : 2006-GOE/CDE/AJ-no 376.

Appendix

List of AbbreviationsBIRN Biomedical Informatics Research Net-

workBOLD Blood Oxygen Level DependencyBPEL4WS Business Process Execution Language

for Web ServicesCBIR Content Based Image RetrievalCBMIR Content-Based Medical Image Retriev-

alCUI Concept Unique IdentiêrDC Data CenterDICOM Digital Imaging and Communications

in Medicine

EBM Evidence-Based MedicineEEG ElectroencephalographyEMG ElectroMyoGramEPI Echo Planar ImagingFMA Foundational Model of Anatomy on-

tologyfMRI functional Magnetic Resonance Imag-

ingGLM General Linear ModelGMAS GRID Medical Archive SolutionGO Gene OntologyGRID Global Resource Information Data-

baseGUI Graphic User InterfaceIBR Image-Based ReasoningIRMA Image Retrieval in Medical Applica-

tionsMedGrid Medical GRIDMEG MagnetoEncephaloGraphyNEU Neuronames Brain HierarchyNIRS Near InfraRed SpectroscopyNSD Neuroscience Database GatewayNTRC Neuroimaging Tools and ResourCesOBO Open Biomedical OntologyOLS Ontology Look-up ServiceOVA One-Versus-AllOWL Web Ontology LanguagePACS Picture Archiving and Communica-

tion SystemPO Protein OntologyRDF Resource Description FrameworkRDFS ResourceDescriptionFrameworkSche-

maRDQL Resource Description framework data

Query LanguageROI Region Of InterestSML Supervised Multiclass LabelingSNOMED Systematized Nomenclature of Medi-

cineSOAP Simple Object Access ProtocolSOC Service-Oriented ComputingSVM Support Vector MachineUMLS Uniêd Medical Language SystemWS-CDL Web Services Choreography Descrip-

tion LanguageXML Extensible Markup Language

References

1. Ogawa S, Lee TM, Kay AR, Tank DW. Brainmagnetic resonance imaging with contrast depend-ent on blood oxygenation. Proc Natl Acad SciUSA, 1990; 87:9868–9872.

2. Juli àa-Sap áe M, Acosta D, Maj áos C, et al. Compari-son between neuroimaging classiˆcations and

154154 T. Nakai et al.


histopathological diagnoses using an internationalmulticenter brain tumor magnetic resonance imag-ing database. J Neurosurg 2006; 105:6–14.

3. Bodenreider O, Stevens R. Bio-ontologies: currenttrends and future directions. Brief Bioinform 2006;7:256–274.

4. Gruber TR. A translation approach to portable on-tology speciˆcation. Knowledge Acquisition 1993;5:199–220.

5. Ding L, Kolari P, Ding Z, Avancha S. Using ontol-ogies in the semantic web: a survey, In: SharmanR, Kishore R, Ramesh R, eds. Ontologies–A hand-book of principles, concepts and applications in in-formation systems. New York: Springer, 2007;79–113.

6. Shelley P. Practical RDF. Sebastopol, CA: O'Reil-ly & Associates, 2003.

7. Wolstencroft K, McEntire R, Stevens R, Taber-nero L, Brass A. Constructing ontology-drivenprotein family databases. Bioinformatics 2005; 21:1685–1692.

8. Gene Ontology Consortium. The Gene Ontologyproject in 2008. Nucleic Acids Res 2008; 36(data-base issue):D440–444.

9. Shoop E, Casaes P, Onsongo G, et al. Data explo-ration tools for the Gene Ontology database.Bioinformatics 2004; 20:3442–3454.

10. Whetzel PL, Parkinson H, Causton HC, et al. TheMGED Ontology: a resource for semantics-baseddescription of microarray experiments. Bioinfor-matics 2006; 22:866–873.

11. M äuller H, Michoux N, Bandon D, Geissbuhler A.A review of content-based image retrieval systemsin medical applications–clinical beneˆts and futuredirections. Int J Med Inform 2004; 73:1–23.

12. Tutac AE, Racoceanu D, Putti T, Xiong W, LeowW-K, Cretu V. Knowledge-guided semantic index-ing of breast cancer histopathology images, bio-medical engineering and informatics: new develop-ment and the future, In: Yonghong Peng, YufengZhang, eds. Proc of the First International Confer-ence on BioMedical Engineering and Informatics,IEEE Computer Society, Sanya, Hainan, China,2008; 107–112.

13. Racoceanu D, Lacoste C, Teodorescu R, Vuilleme-not N. A semantic fusion approach between medi-cal images and reports using UMLS, In: Ng HT, etal. eds. Information Retrieval Technology, LectureNotes in Computer Science 4182; Third Asia Infor-mation Retrieval Symposium, Singapore. SpringerBerlin/Heidelberg, 2006; 460–475.

14. Teodorescu R, Racoceanu D, Leow W-K, Cretu V.Prospective study for semantic inter-media fusionin content-based medical image retrieval. MedicalImaging Technology 2008; 26:1–11.

15. Lim J-H, Chevallet J-P. Vismed: a visual vocab-ulary approach for medical image indexing andretrieval, In: Information Retrieval Technology,Lecture Notes in Computer Science; Second Asia

Information Retrieval Symposium, Jeju Island,Korea. Springer Berlin/Heidelberg, 2005; 84–96.

16. Lehmann TM, Schubert H, Keysers D, Kohnen M,Wein BB. The IRMA code for unique classiˆcationof medical images, In: Proceedings SPIE, 2003;109–117.

17. Lacoste C, Chevallet J-P, Lim J-H, et al. Inter-media concept-based medical image indexing andretrieval with UMLS at IPAL, In: Evaluation ofMultilingual and Multi-modal Information Re-trieval, Lecture Notes in Computer Science 4730,7th Workshop of the Cross-Language EvaluationForum, Alicante, Spain. Springer Berlin/Heidel-berg, 2006; 694–701.

18. Barnard K, Forsyth D. Learning the semantics ofwords and pictures. The Eighth IEEE Internation-al Conference on Computer Vision, 2001; Vancou-ver, Canada; 408–415.

19. Hofmann T. Unsupervised learning by probabilis-tic latent semantic analysis. Machine Learning2001; 42:177–196.

20. Deerwester S, Dumais ST, Furnas GW, LandauerTK, Harshman R. Indexing by latent semantic anal-ysis. J Am Soc Info Sci 1990; 41:391–407.

21. Carneiro G, Chan AB, Moreno PJ, VasconcelosN. Supervised learning of semantic classes for im-age annotation and retrieval. IEEE Trans PAMI2007; 29:394–410.

22. Picard R. Digital libraries: meeting place for high-level and low-level vision, In: Li SZ, Mital DP,Teoh EK, Wang H, eds. Recent Developments inComputer Vision, Lecture Notes in ComputerScience 1035; Asian Conference on ComputerVision, Singapore. Springer, 1995; 3–12.

23. Foster I, Kesselman C. The Grid: Blueprint for aFuture Computing Infrastructure. Morgan Kauf-mann Publishers, Inc., San Francisco; 1999.

24. Warren R, Thompson D, del Frate C, et al. A com-parison of some anthropometric parametersbetween an Italian and a UK population: ``proofof principle'' of a European project usingMammoGrid. Clin Radiol 2007; 62:1052–1060.

25. Estrella F, Hauer T, McClatchey R, Odeh M,Rogulin D, Solomonides T. Experiences of en-gineering GRID-based medical software. Int J MedInform 2007; 76:621–632.

26. Bagarinao E, Tanaka Y. A functional MRI datamanagement tool for the medical grid using Ninf-G; 6th IEEE International Conference on Com-puter and Information Technology, Seoul, Korea.IEEE Computer Society, 2006.

27. Maojo V, Tsiknakis M. Biomedical informaticsand healthGRIDs: a European perspective. IEEEEng Med Biol Mag 2007; 26:34–41.

28. Olive M, Rahmouni H, Solomonides T. FromHealthGrid to SHARE: a selective review ofprojects. Stud Health Technol Inform 2007; 126:306–313.

29. Breton V, Blanquer I, Hernandez V, Legr áe Y,


Vol. 7 No. 3, 2008

Solomonid áes T. Proposing a roadmap forHealthGrids. Stud Health Technol Inform 2006;120:319–329.

30. Smirnov A, Pashkin M, Chilov N, Levashova T.Ontology-based knowledge repository support forhealthgrids. Stud Health Technol Inform 2005;112:47–56.

31. Camarasu S, Benoit-Cattin H, Montagnat J,

Racoceanu D. Content-Based Medical Image In-dexing and Retrieval on Grids. First InternationalSymposium on ICT for Health, Ateneo de ManilaUniversity, Manila, Philippines, Philippine J InfoTech, 2008; 1:46.

32. Papazoglou MP, Traverso P, Dustdar S, LeymannF. Service-oriented computing: state of the art andresearch challenges. Computer 2007; 40:38–45.

review - daniraco.free.frdaniraco.free.fr/pubs/journals/nakai2008mrms.pdf · research and clinical...

Documents