Connecting User Interfaces and User Impairmentsfor Semantically Optimized Information Flow

in Hospital Information Systems

Shuaib Karim1,2, A Min Tjoa1

1Institute of Software Technology and Interactive SystemsVienna University of Technology (http://www.ifs.tuwien.ac.at/)

2Computer Science DepartmentQuaid-i-Azam University, Islamabad (http://www.qau.edu.pk/)

September 05 / 2007September 05 / 2007

Technisches Universität Wien

2

Outline

● Need to connect diverse information domains● Connecting Ontology approach● Accessibility Framework● Test Case (Connecting Ontology between User´s

Impairments and User Interface Characteristics)● Results● Concluding Remarks

3

HIS (A mix of heterogeneous subsystems)

Finance

Food Section

Labs.

Pharmacy

Operation Theatres

Administration

Clinical Support

Medical Records

Registration

Best Possible Patient Care

4

Sample Questions Requiring Connections

Disease ↔ Medicine Disease ↔ DietDisease ↔ Clinical ProcedureDisease ↔ Operative ProcedureProcedure ↔ Resource RequirementResource Requirement ↔ CostUser Interfaces ↔ User Profile

Info. Semantics of Domain1 ↔ Info. Semantics of Domain2

Encoded in software or in data → based upon some rules → ?

Connection of heterogeneous domains => Rules

5

Significance of Connections (Ex. Food & Wine)

Suggests different foods and wine combinations

→ Domain expert´s knowledge encoded as OWL DL, such as;

• taste• appetite or stimulation to eat• digestion• availability

6

Ex. Food & Wine

Another Approach (“Connecting Ontology´´) Ontology of Food – Taxonomy with concepts such as;

• the calories with respect to the quantity• the tendency to get digested on ist own (Quick, Medium

duration, Long duration)• …

Ontology of Wine - Taxonomy with concepts such as;• the appetizing effects (Nil, Low, Medium, High)• the digestive effects (Nil, Low, Medium, High)• …

Above knowledge normally available from Food and Wine producers

8

Characteristic Features of Connecting Ontology OC b / w two ontologies O1 and O2

● O1 , O2 not related with same domains of discourse

● O1 , O2 developed using their own CODeP

● Similarity b / w CODeP indicate inter-connection

● Creation of new knowledge

9

Benefits of Connecting Ontology

● Useful for top-down evolution of ontologies / applications

● Incompatibilities b / w the two ontologies are solved at the ontological level

● Helpful in code automation● Possibility to determine the Cause - Effect

relationship b / w the two ontologies● Reliance on domain experts reduced due to

encoding of domain knowledgeThe key is “How to capture and represent the domain knowledge ??´´

10

Connecting Ontology’s Domain Knowledge

Available as;

Structured documents

Unstructured documents

Tacit knowledge with domain experts

11

Possible Line of Action

● Text processing of knowledge about domains of CO and the two participating ontologies

● Refinement of the above by aligning participating ontologies with global standard ontologies

● Exploiting valid queries on the two ontologies● Exploiting queries result set and data mining● Using Semantic Web Rules

12

Semantic Web – Architecture

13

Generic Accessibility Pattern (CODeP)

18

Visualization and Accessibility Plug-ins

Triple store

Ontology

Storage

Data Preparation for UI

UI Mapping Transformation

Data Transformation

Grouping Aggregation Rendering

Transformation for Image

Formatted Data

User InterfaceQuery

XMLTreeRDF

Graph

Filtering

Visualization Plug-in

Accessibility Plug-in

Info-VizBridge Service

AccessibilityService

19

Accessibility Plug-in

● A scalable framework using SemWeb Technology

Info-VizBridge Service

Accessibility Framework

AccessibilityService

ConnectingOntology

DeviceProfiles

DomainOntology

User Profile(Impairments)

map

s maps

maps

ConnectingOntology

map

s

maps

Uses

Use

s

RepresentationOntology

20

Test case

Connecting Ontology for

User’s Impairments & UI Characteristics

23

Human Disease

TreatmentCause

Chemotherapy

Drug Therapy

Surgery

Psychotherapy

Physiotherapy

Type

EnvironmentalGenetic

Symptom

Extension of ontology (Ref: “Hadzic M. and Chang E., Ontology-based Support for Human Disease Study, HICSS’05”)

Type

Treatment

24

Imapirment-User interface Connecting Ontology - Sample user scenarios

● Avoiding the confusing colors on an interface for a user with particular type of color blindness

● Font adjustments according to user‘s visual acuity

● Information presentation on the better part of the screen for a user suffering from Hemianopsia (absence of vision in half of visual field)

25

Impairment-User Interface Ontology (Ref: “Shuaib Karim and A Min Tjoa, Towards the Use of Ontologies for Improving User Interaction for People with Special Needs. ICCHP 2006: 77-84”)

26

Impairment ontology - competency questions

For a given impairment name;● What is / are the related body parts ?

● What is the impaired side (right, left,...) ?

● What is its severity (on a predefined scale) ?

● What are the perception cues which are affected, and up to what degree (on a predefined scale) ?

● What is the effect of one impairment on another w.r.t. affected perception ?

27

Impairments Ontology

28

Some derived concepts (1 / 2)

● LeftSidedImpairment

LeftnhasPositioyPartrelatedBodmpairmentLeftSidedIImpairmentmpairmentLeftSidedI

.

● RightSidedImpairment

RightnhasPositioyPartrelatedBodImpairmentRightSidedImpairmentImpairmentRightSided

.

● BothSidedImpairment

RightnhasPositioLeftnhasPositioyPartrelatedBodmpairmentBothSidedI

ImpairmentmpairmentBothSidedI

.

.

29

Derived concepts (2 / 2)

● AnySidedImpairment

RightnhasPositioLeftnhasPositioyPartrelatedBodpairmentAnySidedIm

ImpairmentpairmentAnySidedIm

.

.

● AnySidedImpairment

ImpairmentRightSided ImpairmentLeftSidedpairmentAnySidedIm

30

User interface Ontology - competency questions

● Find the part-whole relationship of UI components

● Find the attributes of a component and their values (according to predefined usability scale for a normal user in normal conditions)

● For a given attribute, find the related UI components

31

User Interface Ontology

32

Some derived concepts (1 / 2)

● GoodUsabilityComponent

GoodtyhasUsabilitUiComponenntityComponeGoodUsabiltUiComponenntityComponeGoodUsabil

.

● FairUsabilityComponent

FairtyhasUsabilitUiComponenntityComponeFairUsabiltUiComponenntityComponeFairUsabil

.

● FairUserControlComponent

FairluserControtUiComponennentntrolCompoFairUserCotUiComponennentntrolCompoFairUserCo

.

33

Rules to Connect Impairments and UI

Low perception implies suggesting high usability components (VisualAcuityLow uiLegibilityGood)

(?x rdf:type imp:VisualAcuity) (?x imp:perceptionMeasure imp:Low) (?y rdf:type ui:UiComponent) (?y ui:hasLegibility ui:Good) (?x eg:suggests ?y).

34

Rules to Connect Impairments and UI

High perception implies suggesting fair usability components (VisualAcuityHigh uiLegibilityFair)

(?x rdf:type imp:VisualAcuity) (?x imp:perceptionMeasure imp:High) (?y rdf:type ui:UiComponent) (?y ui:hasLegibility ui:Fair)

(?x eg:suggests ?y).

35

Rules to Connect Impairments and UI

High rheumatism implies suggesting easily operatable components (RheumatismHigh uiUserControlGood)

(?x rdf:type imp:Rheumatism) (?x imp:impairmentMeasure imp:High) (?y rdf:type ui:UiComponent) (?y ui:userControl ui:Good) (?x eg:suggests ?y).

36

Generated RDF triples

<rdf:RDF xmlns:co="http://www.ifs.tuwien.ac.at/ontologies/co#" ...> <rdf:Description rdf:about="http://www.ifs.tuwien.ac.at/ontologies/imp#VisualAcuity_High"> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#TextSize_09"/> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#TextSize_26"/> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#TextStyle_Italic"/> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#TextStyle_BoldItalic"/> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#TextSize_08"/> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#TextSize_24"/> </rdf:Description> <rdf:Description rdf:about="http://www.ifs.tuwien.ac.at/ontologies/imp#Rheumatism_High"> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#LabelledButton"/> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#ComboBox"/> </rdf:Description> <rdf:Description rdf:about="http://www.ifs.tuwien.ac.at/ontologies/imp#VisualAcuity_Low"> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#TextSize_18"/> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#TextSize_10"/> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#TextFont_TimesNewRoman"/> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#TextStyle_Bold"/> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#TextSize_22"/> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#TextSize_11"/> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#TextSize_20"/> </rdf:Description><rdf:Description rdf:about="http://www.ifs.tuwien.ac.at/ontologies/imp#ColorBlindness_RG"> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#Color_Grey"/> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#Color_Orange"/> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#Color_Cyan"/> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#Color_Blue"/> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#Color_Yellow"/> </rdf:Description> <rdf:Description rdf:about="http://www.ifs.tuwien.ac.at/ontologies/imp#ColorBlindness_YB"> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#Color_Grey"/> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#Color_Orange"/> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#Color_Cyan"/> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#Color_Green"/> <co:suggests rdf:resource="http://www.ifs.tuwien.ac.at/ontologies/ui#Color_Red"/> </rdf:Description></rdf:RDF>

37

38

39

40

Consequences of Impairments-user Interface Connecting Ontology

• Helpful in automatically adapting UI for the user• Helpful in deducing the best match of UI characteristics

for a user with multiple impairments• Possibility to use the ontology for diversity• Historical data for studying the cause-effect relationship

b/w the impairments and the computer interfaces• Useful for rehabilitation purposes• Possibility to extract impairment related semantics from

user´s information stored in SemanticLIFE repository, and modify the impairments ontology accordingly

41

Concluding Remarks

● Integrating heterogeneous domains is always challenging

● The Connecting Ontology concept using the ontological rule-based approach paves the way for a generic solution

● Another abstraction level for persistence of rules

● The effort could be useful in HIS and other IMS

● Ontology for Impairments and Usability introduced

42

Future directions

● User testing● Instantiating the Impairments and UI ontologies● Integrating capability measuring tools (MMSE)● Ontology Elevation / Lifting● Integration for ontology of visualization techniques

43

Thanks!

http://www.ifs.tuwien.ac.at/~skarim

SemanticLIFE Project http://storm.ifs.tuwien.ac.at/

ASEA-UNINET(Asean European University Network)

http://www.uibk.ac.at/asea-uninet/

HEC (Higher Education Commission of Pakistan)

http://www.hec.gov.pk/

Technisches Universität Wien