explaining semantic search results of medical images in medico

Explaining Semantic Search Results of Medical Images in MEDICO

Björn Forcher, Manuel Möller, Michael Sintek, and Thomas Roth-Berghofer

Mittwoch, 15. Juli 2009

Reality checkMittwoch, 15. Juli 2009

„Trust me. I know what I am doing!“


Goal of Medico Project

Development of

• intelligent

• robust and

• scalable

semantic search engine for medical images


5

RadSem

• Tool to support medical doctors (esp. radiologists) in annotating and searching for medical images (and text)

• Part of the MEDICO project (funded by BMWi in the research programme THESEUS)

• Developed together with medical experts (who have to use the tool to annotate real images)


Intended Users of RadSem

• Medical doctors

• Medical IT professionals

• Patients and citizens

• Policy makers


7

MEDICO System Architecture


8

MEDICO Ontology Hierarchy


9

Foundational Model of Anatomy FMA

• developed and maintained by Structural Informatics Group at University of Washington

• contains more than 70.000 anatomical entities (classes)

• more than 1.5 million relations between the entities

• most comprehensive human ontology


http://sig.biostr.washington.edu/index.html




http://www.washington.edu/

http://www.washington.edu/

10

ICD-10 in OWL

• Problem: No disease terminology available in OWL

• Established standard: International Classification of Diseases (WHO), but only available in semi-structured formats

• Approach: Crawler for online version of ICD-10 generates light-weight OWL ontology


Example annotation

• FMA

• ICD 10


Example annotation

Region of Interest

• FMA

• ICD 10


Basic explanation scenario

Explainer

Originator

User Interface



Explainer

Originator

User Interface

Problem solving

knowledge



Explainer

Originator

User Interface

Problem solving

knowledge

Explanation

knowledge



Explainer

Originator

User Interface



Explainer

Originator

User Interface

Semantic Search



Explainer

Originator

User Interface

Semantic Search

• Query expansion with ontology concepts

• Count path length from search to found concept


Motivations for explanations in RadSem• Test whether the Search Engine works

correctly

• Test whether the ontologies are correctly modelled

• Learn about the medical domain

• Justify results in order to increase trust



correctly




Medical IT professionals



correctly





Patients and citizens


Motivations for explanations in RadSem

• Help users to improve their search

• Activate passive knowledge

• Users learn how to use the engine concerning ontologies






Medical doctors


What are explanations?


What are explanations?

Explanations are answers to questions.


When are questions being asked?


When are questions being asked?

Whenever expectations are not met.


Explanation goals

• Transparency

• Justification

• Relevance

• Conceptualisation

• Learning

Sørmo, F., Cassens, J., Aamodt, A.: Explanation in Case-Based Reasoning – Perspectives and Goals, 2005.

How did the system reach an answer?

Why is the answer a good answer?

Why is the question relevant?

What is the meaning of a concept?

Teach the user about the given domain.


When are explanations good explanations?

• Short and easy to overlook

• Innovative

• Relevant

• Convincing

• Different perspectives and follow-up questions

• Natural

W. R. Swartout and J. D. Moore. Explanation in second generation expert systems. In J. David, J. Krivine, and R. Simmons, editors, Second Generation Expert Systems, pages 543–585, Berlin, 1993. Springer Verlag.


Kinds of explanations

• Action explanations and justifications:„How do search concepts relate to found concepts?“

• Concept explanations


Action explanations: “Why was this seat post selected?” – “For the given price, only one other seat post was available. But this was too short.

Action explanations

• Action explanations explain the activities of the respective system (originator).

• In RadSem: Reconstructive explanations based on search and found concepts.


Why-explanations

• Why-explanations provide causes or justifications for facts or events.

• Examples:• Justification: “Why does the universe expand?” – “Because we

can observe a red shift of the light emitted by other galaxies.”

• Cause: “Because the whole matter was concentrated at one point of the universe and because the whole matter moves away from each other


Concept Explanations

• The goal of concept explanations is to build links between unknown and known concepts.

• Variations:• Definition: “What is a bicycle?” – “A bicycle is a land vehicle

with two wheels in line. Bicycles are a form of human powered vehicle.”

• Functional mapping: “What is a bicycle?” – “A bicycle serves as a means of transport.”

• Prototypical usage of individual things or actions: “What is a bicycle?” – “The thing, that man over there just crashed with.”

• …



Explainer

Originator

User Interface

Semantic Search



Explainer

Originator

User Interface

Semantic Search

• Dijkstra algorithm estimates semantic search


Example search


Exploration interface


„Bridge concepts“


FMA problem

• Same concept, different labels


Label problems of FMA


User experiment wrt explanations in RadSem• Test whether the Search Engine works

correctly







User experiment wrt explanations in RadSem• Test whether the Search Engine works

correctly






→ Results supported our motivations for providing explanations.


• Selection of proper labels wrt different user groups

• Search for alternative paths

• Exploration of paths

• Tailoring of paths

• Dictionary for lexical concepts

• Links to Wikipedia

Future Work


Take home messages


• RadSem is a complex annotation and search tool.

Take home messages



• Goals and kinds of explanations are a useful tool in designing a software system in an explanation-aware manner.

Take home messages




• Basic explanation scenario helps identify communication partners

Take home messages

Explainer

Originator

User





• Exploration interface with concept explanations support domain understanding.

Take home messages

Explainer

Originator

User





• Exploration interface with concept explanations support domain understanding.

• Justification interface provides action explanations, which counteract encapsulation and information hiding.

Take home messages

Explainer

Originator

User


Thomas Roth-BerghoferSenior researcher, [email protected] German Research Centre for Artificial Intelligence DFKI GmbH

Thank you!

Explaining Semantic Search Results of Medical Images in MEDICO


mailto:[email protected]

mailto:[email protected]

explaining semantic search results of medical images in medico

Technology