a model-driven development methodology for 3d user interface for information systems in a principle...
TRANSCRIPT
Unuse
d
Sectio
n
Space
1
A Model-Driven Approach for Developing 3D User Interfaces
of Information Systems in a Principle–Based WayJuan Manuel González Calleros
PhD Public Defense
February 4, 2010 - LLN
1
February 4, 2010 - LLN PHD Public Defense
Virtual Reality
Augmented Reality
2
February 4, 2010 - LLN PHD Public Defense
Windows Vista
3
February 4, 2010 - LLN PHD Public Defense
Locate your sit in a stadium
4
Outline
1. Introduction2. State of the Art3. Methodology
1. Ontology2. The Method3. Towards a UIDL for 3DUI4. Software for Supporting the Methodology
4. Validation5. Conclusion
February 4, 2010 - LLN PHD Public Defense 5
What is a 3D User Interfaces?
February 4, 2010 - LLN PHD Public Defense
Augmented Reality Virtual Reality
3D User Interface
3D desktop environments
6
Why 3D User Interfaces?
Increase user satisfaction.
Improve cognitive perception
Indices sense of (tele)presence
3DUIs are not automatically superior or inferior to
2DUIs.
February 4, 2010 - LLN PHD Public Defense 7
Why NOT 3D User Interfaces?
Decrease user performance
Increase manipulation complexity for an average user
Are not appropriate for any task
Are hard to evaluate for their usability
February 4, 2010 - LLN PHD Public Defense 8
Focus of the dissertation
Information systems (data, process, resources)
Administrative tasks (routine, repetition)
Interaction styles (form filling, multi-windowing, direct manipulation, iconic interaction, graphic interaction, multimedia interaction, and 3DUIs)
February 4, 2010 - LLN PHD Public Defense 9
Thesis Statement
In this thesis we argue that developing 3DUIs for Information Systems is an activity that would benefit from the application of a methodology which is typically composed of:
1. a set of models gathered in an ontology, 2. a method manipulating the involved models
based on guidelines, 3. a language that express models in the
method.Define a model-driven approach for structuring
the development life cycle of a Three-Dimensional User Interface of an Information System in a principle based way.
February 4, 2010 - LLN PHD Public Defense 10
Comparative analysis
February 4, 2010 - LLN PHD Public Defense
• 3DUI development methodologies
• Language• Transformation
Engines
• 3D Toolkits
• Models
• Language
• Methods
• Software tools
11
Requirements
Methodological (7)
Ontological (3)
Language (1)
Software tool support (1)
Methodological support for the development life-cycle of 3DUI for Information Systems
February 4, 2010 - LLN PHD Public Defense 12
Outline
1. Introduction2. State of the Art3. Methodology
1. Ontology2. The Method3. Towards a UIDL for 3DUI4. Software for Supporting the Methodology
4. Validation5. Conclusion
February 4, 2010 - LLN PHD Public Defense 13
February 4, 2010 - LLN PHD Public Defense
• 3D UI as an specialization of 2D
• Benefits:
• Relying on existing 2D UI by specializing
• Factoring out 3D vs 2D UIs
VS.
14
February 4, 2010 - LLN PHD Public Defense
3D Rendering of a 2D GUI
VUITOOLKIT [Moli08]
15
February 4, 2010 - LLN PHD Public Defense
• Shortcomings:• Adding another modality breaks the factoring
out• Hard to provide multiple representations of
3D objects• No full separation of concerns
16
February 4, 2010 - LLN PHD Public Defense
• Shortcomings:• Adding another modality breaks the factoring
out• Hard to provide multiple representations of
3D objects• No full separation of concerns
17
February 4, 2010 - LLN PHD Public Defense
Support for different
representations
Support for basic Haptic Interaction
3DUIs
18
Outline
1. Introduction2. State of the Art3. Methodology
1. Ontology2. The Method3. Towards a UIDL for 3DUI4. Software for Supporting the Methodology
4. Validation5. Conclusion
February 4, 2010 - LLN PHD Public Defense 19
Method Outline
February 4, 2010 - LLN PHD Public Defense
Task and Domain Model
Model to Model
Abstract UI Model
Model to Model
Concrete UI Model
Code Generatio
n
Final UI
Control
Task and Domain Model
Physical Control Software Control
Physical interaction object 2D 3D
20
February 4, 2010 - LLN PHD Public Defense
Task and Domain Model
Model to Model
Abstract UI Model
Model to Model
Concrete UI Model
Code Generatio
n
Final UI
• A structured catalog of transformation rules that form a body of design knowledge that can be reused in any 3D method
21
February 4, 2010 - LLN PHD Public Defense
Code Generatio
n
Final UI
Task and Domain Model
Based onGuidelines
Model to model
Abstract UI Model
Based onGuidelines
Model to model
Concrete UI Model
Usability Advisor
AutomaticEvaluation
Refined Concrete UI
Model
Code Generatio
n
3D User Interface
A set of Principles were added to the method
•Guidelines•Task patterns•Canonical list of task types
22
February 4, 2010 - LLN PHD Public Defense
Canonical list of task types
23
Task and Domain Model
Based onGuidelines
Model to model
Abstract UI Model
Based onGuidelines
Model to Model
Concrete UI Model
Usability Advisor
AutomaticEvaluation
Refined Concrete UI
Model
Code Generatio
n
3D User Interface
February 4, 2010 - LLN PHD Public Defense
Facet Selection
24
Task and Domain Model
Based onGuidelines
Model to model
Abstract UI Model
Based onGuidelines
Model to Model
Concrete UI Model
Usability Advisor
AutomaticEvaluation
Refined Concrete UI
Model
Code Generatio
n
3D User Interface
February 4, 2010 - LLN PHD Public Defense
Select Element
Input
Slider
AIO Selection
25
Task and Domain Model
Based onGuidelines
Model to model
Abstract UI Model
Based onGuidelines
Model to Model
Concrete UI Model
Usability Advisor
AutomaticEvaluation
Refined Concrete UI
Model
Code Generatio
n
3D User Interface
February 4, 2010 - LLN PHD Public Defense
Graphical representation selection
26
Task and Domain Model
Based onGuidelines
Model to model
Abstract UI Model
Based onGuidelines
Model to Model
Concrete UI Model
Usability Advisor
AutomaticEvaluation
Refined Concrete UI
Model
Code Generatio
n
3D User Interface
February 4, 2010 - LLN PHD Public Defense
Automatic guidelines evaluation
27
Task and Domain Model
Based onGuidelines
Model to model
Abstract UI Model
Based onGuidelines
Model to Model
Concrete UI Model
Usability Advisor
AutomaticEvaluation
Refined Concrete UI
Model
Code Generatio
n
3D User Interface
February 4, 2010 - LLN PHD Public Defense 28
Outline
1. Introduction2. State of the Art3. Methodology
1. Ontology2. Method3. Towards a UIDL for 3DUI4. Software for Supporting the Methodology
4. Validation5. Conclusion
February 4, 2010 - LLN PHD Public Defense 29
Language Engineering Approach
Semantics – Meta Models, UML Class diagrams
SyntaxAbstract – XML SchemaConcrete – XML
Stylistics – Different graphical representations of the concepts
February 4, 2010 - LLN PHD Public Defense 30
UsiXMLStructured accordingly to the Model Driven
paradigmUsiXML relies on a transformational
approach UsiXML allows the modification of the
developments stepsUsiXML allows reusing parts of previously
specified UsiXML is openFollows a Language Engineering Approach
February 4, 2010 - LLN PHD Public Defense 31
Outline
1. Introduction2. State of the Art3. Methodology
1. Ontology2. Method 3. Towards a UIDL for 3DUI4. Software for Supporting the Methodology
4. Validation5. Conclusion
February 4, 2010 - LLN PHD Public Defense 32
February 4, 2010 - LLN PHD Public Defense 33
Task and Domain Model
Based onGuidelines
Model to model
Abstract UI Model
Based onGuidelines
Model to Model
Concrete UI Model
Usability Advisor
AutomaticEvaluation
Refined Concrete UI
Model
Code Generatio
n
3D User Interface
IdealXML
February 4, 2010 - LLN PHD Public Defense 34
Task and Domain Model
Based onGuidelines
Model to model
Abstract UI Model
Based onGuidelines
Model to Model
Concrete UI Model
Usability Advisor
AutomaticEvaluation
Refined Concrete UI
Model
Code Generatio
n
3D User Interface
YATE
February 4, 2010 - LLN PHD Public Defense 35
Task and Domain Model
Based onGuidelines
Model to model
Abstract UI Model
Based onGuidelines
Model to Model
Concrete UI Model
Usability Advisor
AutomaticEvaluation
Refined Concrete UI
Model
Code Generatio
n
3D User Interface
TransformiXML
February 4, 2010 - LLN PHD Public Defense 36
Task and Domain Model
Based onGuidelines
Model to model
Abstract UI Model
Based onGuidelines
Model to Model
Concrete UI Model
Usability Advisor
AutomaticEvaluation
Refined Concrete UI
Model
Code Generatio
n
3D User Interface
UsabilityAdvisor
February 4, 2010 - LLN PHD Public Defense 37
Task and Domain Model
Based onGuidelines
Model to model
Abstract UI Model
Based onGuidelines
Model to Model
Concrete UI Model
Usability Advisor
AutomaticEvaluation
Refined Concrete UI
Model
Code Generatio
n
3D User Interface
February 4, 2010 - LLN PHD Public Defense 38
February 4, 2010 - LLN PHD Public Defense 39
Outline
1. Introduction2. State of the Art 3. Methodology
1. Ontology2. Method 3. Towards a UIDL for 3DUI 4. Software for Supporting the Methodology
4. Validation5. Conclusion
February 4, 2010 - LLN PHD Public Defense 40
February 4, 2010 - LLN PHD Public Defense
Case Study
Complexity
Man/Month
LOC Context
Polling System
+ 6 6, 000VRML
Information Systems
Student TrainerSystem
+ 6 3, 000X3D
Information Systems
Haptic Browser
++ 4 (collaboration)
0 Information SystemsVisual Impair users
AHMI +++ 48 2, 000X3D10, 000 +Open GL
Information SystemsAeronautics
41
February 4, 2010 - LLN PHD Public Defense
• Polling System. Devoted to the development of an opinion polling system.
42
February 4, 2010 - LLN PHD Public Defense
• Hapget = 3D widget + haptic feedback
43
February 4, 2010 - LLN PHD Public Defense
• Haptic Browser- Google web site. Rendering of web site in the haptic web browser.
44
February 4, 2010 - LLN PHD Public Defense
• The AFMS is a text-only system that calculates the exact route of the aircraft along a given list of waypoints considering possible altitude, time or speed constraints.
• After a trajectory is generated and successfully negotiated with ATC the AFMS can generate input commands for the autopilot and thus control the aircraft in space and in time.
45
February 4, 2010 - LLN PHD Public Defense
• The AFMS application is controlled via the Advanced Human Machine Interface (AHMI), the main application of the target system to be investigated in the HUMAN project
46
February 4, 2010 - LLN PHD Public Defense
• Problems:• Discontinuity with
the rest of the environment
• Difficult to modify due to its Complexity
• Usability guidelines• Widgets have a 3D
nature in the context of a cockpit
47
February 4, 2010 - LLN PHD Public Defense
• Our goal is to provide a 3D Advanced Human Machine Interface.• New set of CUI models• Consolidation of
existing models• New application
domain• …
48
External evaluation
0,00
1,00
2,00
3,00
4,00
5,00
6,00
7,00
SYSUSE INFOQUAL INTERQUAL OVERALL
Val
ue
Metric
CSUQ Perceived Methodology
High
Low
Mean
February 4, 2010 - LLN PHD Public Defense 49
External evaluation
February 4, 2010 - LLN PHD Public Defense 50
• Final rendering – Feedback from the haptic browser• Difficult recognition of some hapgets.• Lack of user preference adaptation• Limited exploitation of the haptic characteristics• Long sentences to provide a value• Size of hapgets• Speech recognition problems• Sound become annoying• Slow loading
Outline
1. Introduction2. State of the Art 3. Methodology
1. Ontology2. Method 3. Towards a UIDL for 3DUI 4. Software for Supporting the Methodology
4. Validation 5. Conclusion
February 4, 2010 - LLN PHD Public Defense 51
We introduced a 3DUI Development Methodology articulated on three axes:
Models
Method
Language
February 4, 2010 - LLN PHD Public Defense 52
Models (More than 200 attributes, 90 classes, 100 relationships)
Task: Canonical list of task types and guidelines for task
modeling.
Domain Model: Improving existing Domain Model that will be
considered for the NexOF-RA the European standardization
process
AUI: Improving existing AUI Model that will be considered for
the NexOF-RA The European standardization process
CUI: Enriching CUI Model by adding haptic support,
separation of concerns 2D vs 3D by specialization. 3D support.
Hapgets (set of 3D widgets enhanced with haptic feedback)
February 4, 2010 - LLN PHD Public Defense 53
February 4, 2010 - LLN PHD Public Defense
DomainModel
DomainSubModel
+subModelId
DomainElement
*
DomainItem
+itemId+itemType
DomainCollection
+collectionId+collectionType
Constraint
+condition
0..*
0..*
RelevanceExistence
DomainRelationship
FunctionalDependency
+semanticFunction
ValueConstraint
ReplicationConstraintCardinalityConstraint
Domain Model
AUI Model
Integrates 3DUI in the context of MBUI development Incubator group
Integrates new concepts to their ontology based on UsiXML
http://www.w3.org/2005/Incubator/model-based-ui/wiki/Main_Page
http://forge.morfeo-project.org/wiki_en/index.php/Interactive_Application_Models
54
Methodological aspects adheres to the MDE paradigm. Models and transformations are explicitly defined and used (Around 85 transformations rules) Step 1: Task and concepts (T&C) definition
Task and Concepts Consolidation Selection of task types, items and user categories Guidelines of best-practices for structuring task models
Step 2: From T & C to Abstract UI (AUI) Model Identification of abstract UI structure Selection of AIC Spatio-Temporal arrangement of abstract interaction objects Definition of abstract dialog control Derivation of AUI to domain mappings
Step 3: From AUI to CUI Model Reification of AC into CC Selection of CICs Selection of the graphical representation of the CIC
Step 4: From CUI to FUI Model
February 4, 2010 - LLN PHD Public Defense 55
All aspects are stored in UsiXML files that can be exchanged, shared, and communicated between stakeholders (designers, developers, and end users).
Advantages of this language were discussed Modifiability Complexity Rigorous. Reasoning. Processable.
Adopt a Language Engineering Approach Semantics – Meta Models Syntax
Abstract – XML Schema Concrete – XML
Stylistics – Different graphical representations of the concepts Our contribution is independent of UsiXML
February 4, 2010 - LLN PHD Public Defense 56
Five Software modules are used to support the methodology. Conceptual definition of the tools. Supervising of their developmentTesting with our case studies the performance
of the toolsFinal rendering (48 men/month, more than
10, 000 LOC)
February 4, 2010 - LLN PHD Public Defense 57
February 4, 2010 - LLN PHD Public Defense 58
Benefits from our methodology
February 4, 2010 - LLN PHD Public Defense
MDA Expected Benefits
Reducing the gap between requirements and implementation
Developer coordination. Dividing the system into several parts and defining interfaces between them
Well-structured systems. This facilitates implementation itself as well as maintenance.
Planning on adequate level of abstraction
Improved communication by visual models
Validation: (Semi-)Formal modeling languages enable automatic validation of the design
Documentation: Models can be used as documentation when maintaining the system.
Platform-independence. At least serve as starting point when implementing the system for a different platform
Enhanced productivity.
Expert knowledge can be put into the code generator
Reduction of errors
Knowledge about creation of modeling languages
Frameworks and tools
Maintenance of modeling language and transformations
Reuse of meta-models and transformations
59
Future WorkExternal evaluation should be targeted to three:
designers, developers and users:User testing with control experiments in labsDesigner and developers
Extend the set of 3DUIs components. Integrated development Environment Explore Other dimensions of the Cameleon
frameworkExplore other context of use
Learning ObjectsAugmented reality
February 4, 2010 - LLN PHD Public Defense 60
Acknowledgements
February 4, 2010 - LLN PHD Public Defense
2004 20092007 20082005 2006
•UsiXML Consortium
•The Information Systems Research Unit
•The ALBAN program
•The CONACYT program
•The SIMILAR project
•The Human project
•The NexOF project
•The Itea 2 UsiXML project
•The Promep project
_____________PhD_____________________DEA_______
2010
61
Q&A
February 4, 2010 - LLN PHD Public Defense 62