![Page 1: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/1.jpg)
George VellisGeorge Vellisaa
Dimitrios KotsalisDimitrios Kotsalisaa
Demosthenes AkoumianakisDemosthenes Akoumianakisaa
Jean VanderdoncktJean Vanderdoncktbb
aaDepartment of Applied Informatics & Multimedia, Department of Applied Informatics & Multimedia, Technological Education Institution of Crete, GreeceTechnological Education Institution of Crete, GreecebbUniversité catholique de Louvain,Université catholique de Louvain,Louvain Interaction Laboratory (www.lilab.eu)Louvain Interaction Laboratory (www.lilab.eu)Louvain School of Management, BelgiumLouvain School of Management, Belgium
![Page 2: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/2.jpg)
Introduction
• User interface challenges for software engineers– Programming intensive– Broad range of interaction components and widgets– Platform specificity (GUI objects, Web widgets, etc) – Lack of interoperability due to platform specificity
• (Some) Model-based engineering claims – UI engineering = Model specification– Focus on design rather than implementation– Linking (rather than direct calls) to platform-specific
libraries
![Page 3: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/3.jpg)
Introduction
• User interface challenges for software engineers– UI development time takes a large portion of time
Software plans and requirements
Validation
Validation
Product design
Verification
Detailed design
Verification
CodeUnit test
IntegrationProduct verification
Operations and maintenance
Revalidation
Implementation
System test
System feasibility
UI Design occupied 45% of the totalamount of time devoted to each activity
50%
37%
Mean = 44%
![Page 4: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/4.jpg)
Retrospective assessment
• Claims proven useful– Model specifications can provide an engineering
basis for UI engineering– Run-time linking alleviates the complexity of
programming
• Claims proven possible– Multi-platform execution– Relatively straightforward Uis (e.g., form-based UI)
![Page 5: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/5.jpg)
Retrospective assessment (Cont.)
• Several initiatives – One of them is UsiXML which is used in this work
• Reference Architectures– CAMELEON Reference framework
• Recommended by W3C Working Group on Model-based User Interface Design
• UsiXML is compliant with this framework
• Plethora of methods with some focus on collaboration– FlowiXML, AMENITES, CIAM, TOUCHE
www.usixml.euwww.usixml.org
![Page 6: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/6.jpg)
Retrospective assessment (cont.)
• Challenges still pending attention – non-standard and customized widgets?– synchronous and collaborative UIs?– new affordances such as social awareness in
distributed collaborative settings?– radically different interaction vocabularies?– design and run-time support tools for all the above?
![Page 7: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/7.jpg)
• Theory development– Build abstraction to capture interaction components as
assemblages of capabilities (or affordances)– Substantiate such capabilities (or affordances) to polymporphic
instantiation schemes– Specify polymorphic properties in a model-based fashion
• Engineering framework– Extend UsiXML as needed to support the new specification– Provide dedicated design tools and run-time components– Demonstrate the concept’s validity through pilot scenarios
Research design
![Page 8: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/8.jpg)
• Consider various generations of ‘buttons’– GUI buttons (in the 90s)
• The (primitive) assemblage of button = {two-state device to exercise control over functionality}
Why abstraction?
![Page 9: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/9.jpg)
• A continuum of abstractions (Cameleon Ref. Framework)
Why abstraction?
Final UserInterface (FUI)
Concrete UserInterface (CUI)
Abstract UserInterface (AUI)
Task &Concepts
Rendering
Code
Platform-independentCIO
Modality-independentAIO type
TaskClasses
Download
<input type=submit value=“Download" name=btnG>
HTML pushbutton
Graphical 2D push button
Software control AIO
Control AIO
Method triggered: download fileObject: computer file
OSF/MotifXmButton
Windowspush button
Download Down
Load
VRML97/X3Dbutton
Softwarekey
Functionkey
Graphical 3D push button Physical push buttonPlatform-independentCIO type
Physical control AIOModality-dependentAIO
Final UserInterface (FUI)
Concrete UserInterface (CUI)
Abstract UserInterface (AUI)
Task &Concepts
Rendering
Code
Platform-independentCIO
Modality-independentAIO type
TaskClasses
DownloadDownload
<input type=submit value=“Download" name=btnG>
HTML pushbutton
Graphical 2D push button
Software control AIO
Control AIO
Method triggered: download fileObject: computer file
OSF/MotifXmButton
Windowspush button
DownloadDownload Down
LoadDown
Load
VRML97/X3Dbutton
Softwarekey
Functionkey
Graphical 3D push button Physical push buttonPlatform-independentCIO type
Physical control AIOModality-dependentAIO
![Page 10: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/10.jpg)
• Scented button (cf. Willett et al., 2007)
• The (augmented) assemblage of ‘scented’ button ={ Two-state device to exercise control }
+{ Collectivity (i.e., who else is in the same state of mind)}
‘Adding’ social scent
![Page 11: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/11.jpg)
• ‘Like’ buttons (in the Web 2.0)
• The (new) assemblage of the ‘like’ button ={Two-state devices to exercise control}
+{Expression of opinion (i.e. I like this)}
‘Adding’ expression of intention
![Page 12: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/12.jpg)
• ‘Like’ buttons with social scent
• The (combined) assemblage ={Two-state device to exercise control}
+{Expression of opinion (i.e. I like this)}
+ {Collectivity (i.e., who else is in the same state of mind)}
Putting them all together
![Page 13: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/13.jpg)
• Why limit our creativity to conventional widgets?– elastic buttons and their interactive behaviour
Generalizing the concept
Note the embedded affordances
![Page 14: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/14.jpg)
• What happens when an elastic button is to be instantiated across devices and platforms
Adding to complexity
![Page 15: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/15.jpg)
• How can we …– model digital assemblages in interaction components?– instantiate them as the need arises and depending on
context / platform capabilities– integrate them into specifications as first class elements /
properties– compile specs that are general enough
Motivating question
![Page 16: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/16.jpg)
• A scheme that relies on implementation agnostic (i.e., abstract) specifications of UIs– i.e. an abstract button can be instantiated as
• a conventional GUI button when implemented with the ‘primitive’ assemblage
• a scented button when implemented with the ‘augmented’ assemblage
• a ‘like’ button when implemented with the ‘combined’ assemblage
• At run-time and once user and usage context parameters are discovered, the implementation agnostic spec is translated to context-specific interaction vocabularies using dedicated tools
(One) Answer: Polymorphic instantiation
![Page 17: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/17.jpg)
• Introduce new widgets and their properties as first-class design elements
Polymorphic Widget SpecificationsBasic concept
WSL LibrariesResources
Widget Archive
![Page 18: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/18.jpg)
• Specify complex capabilities & affordances in a model-based fashion– Devise required properties– Define model elements– Create specification– Mapping models
The role of MBUI engineering
![Page 19: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/19.jpg)
The role of MBUI engineering the case of a socially aware button
![Page 20: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/20.jpg)
• Extend UsiXML as needed to support the new specification– New set of models
• Widget Resource Model• Behavior model
– Additional models for collaborative work• Squad model• Consistency model• Abstraction model• Session model
A new set of models
![Page 21: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/21.jpg)
A new design tool
![Page 22: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/22.jpg)
A new run-time environment
Design Environment
Platform Server
Abstraction Classes
Collaboration Plugin
Social Proxy
Polling
Social Proxy
Social Proxy
Abstraction Classes
Abstraction Classes Abstraction
Classes
Collaboration Plugin
Social Proxy
Polling
Social Proxy
Social Proxy
Abstraction Classes
Abstraction Classes
Runtime Environment
Server Side Framework (SSF)
UI-1
Axis2-Framework
Web-Services Layer
Session Manager
Widget Archives Repository
Notification Queue
UI-Model Repository2.join
3.getUIModel
4.getArchives
5.buildUI6.attachStates
1.initPS
trackChanges
applyChangesdistributeChanges
Client-1 Client-N
UI-N
P.S
de
plo
yU
I UI Design Tool
Widget Spec Tool
![Page 23: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/23.jpg)
UIDL’2011 (simple) desktop scenario
![Page 24: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/24.jpg)
MDSD’12 multiplatform scenario (in the paper)
Player-1:Desktop/PC Player-2: Android/mobile Viewer-1: Desktop/PC
![Page 25: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/25.jpg)
• Collaborative assembly of vacation packages• User roles
– Business partner (desktop)– Administrator (desktop or mobile– Platforms (desktop/swing, mobile/Android)– Context (synchronous and collaborative)
• Complex and custom widgets (i.e., elastic buttons)• Synchronous collaboration
Post MDSD’12 scenario
![Page 26: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/26.jpg)
The desktop case administrator’s view of vacation package / swing
![Page 27: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/27.jpg)
The desktop casebusiness partners’ view of vacation package / swing
![Page 28: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/28.jpg)
The mobile caseadministrator’s view of vacation package - Android
Due to space constraints certain functions become automatically adapted to vocal interaction
![Page 29: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/29.jpg)
The designer’s role?Designing for certain affordances
3
2
456
1
![Page 30: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/30.jpg)
The run-time instantiation the specified affordances enabled
3 4 5 6
2
2
1
![Page 31: Model-based engineering of multi-platform, synchronous & collaborative UIs](https://reader033.vdocuments.site/reader033/viewer/2022052822/554f43d3b4c90572088b54c5/html5/thumbnails/31.jpg)
• Current and on-going work– Advanced toolkits (i.e., visualization)– Run-time adaptivity and UI plasticity in distributed and
ubiquitous settings– ‘Social’ games
• Acknowledgements – The work is supported by ARCHIMEDES III– Thanks to Université catholique de Louvain (UCL) for
supporting the first two authors’ PhD dissertations• Due to be submitted in 2013
Future work & acknowledgements