25 April 2000

S

E

E

S

C

O

A STWW - Programma

Implementation of a component-based UI

Deliverable 4.2

S

E

E

S

C

O

A

Contents

Objectives

UI Guidelines for Embedded Systems

Description of a User Interface

User Interface Components

UI Components – implementation

Conclusions

S

E

E

S

C

O

A

Objectives Deliverable 4.2

Human-computer interaction in the context of embedded systems Design guidelines UI description

Component-based technologies for User Interfaces

Practical work, design and implementation SEESCOA component system case

S

E

E

S

C

O

A

UI Guidelines for Embedded Systems

General guidelines Keep it simple! Assume there will be no conventional I/O devices … (Deliverable 4.2, p5)

Case study: implementation of UI controls for Palm “Unconventional“ input method: stylus with a touch-screen Limited resources Small screen In Case study: remote control to operate camera surveillance

system

S

E

E

S

C

O

A

UI Guidelines for Embedded Systems

UI controls for Palm computer (example) Joystick Control for Palm, testing the stylus

Combining User Interface elements: save screen space and add up to functionality

Uses direct manipulationPinpoint in cross is attached to a virtual spring: gives a kind of push and pull feeling with the stylus

e.g.:

S

E

E

S

C

O

A

UI Guidelines for Embedded Systems (2)

What do we need? Look-and-feel should be consistent for family of products Look-and-feel should be delegated to experts (e.g. graphical,

non technical) Rapid prototyping is extremely important Possibility of embedding constraints in User Interface

description (e.g. screen update) Describing a User Interface, independent from the platform

S

E

E

S

C

O

A

Description of a User Interface

A possible solution: using XML... XML is a meta language Already used for describing HCI (WML, XUL, SpeechML,

VoxML,...) Can describe UI device independent (e.g. UIML) Cross-platform To work with XML, the designer does not need a technical

background e.g: Harmonia UIML, Glade,... User Interface becomes reusable by its XML description XML can describe alternative kinds of human-system

S

E

E

S

C

O

A

Description of a User Interface (2)

... with Java XML-parsers for Java (plenty!)

IBM Alphaworks Xerces, XP, JAXP Not small enough - write our own Used in SEESCOA

XML integration in Java Component Model "Long time" serialization e.g. IBM Bean Markup Language

UI becomes "downloadable" A User Interface can be downloaded as an XML description,

and reused on a remote machine Extra information can be contained in the XML description

of the User Interface Demo

S

E

E

S

C

O

A

Description of a User Interface (3)

Implementation Java User Interface to XML Using Java reflection mechanism User Interface converted into XML and transported (file, socket

stream,...) Problems to solve:

XML parsing is too slow and consumes too much memoryAWT is not as consistent as Swing, requires ad hoc solutionsRedundancyLayout Management, preferably Java independentClass definitions of UI are required for deserialization

S

E

E

S

C

O

A

Description of a User Interface (4)

Implementation (continued) Future work

A more generic approachLayout ManagementEvent HandlingRedundancy checkingFaster and smaller XML-parser Integration with SEESCOA Component SystemJava Independent, XML to ... generation

S

E

E

S

C

O

A

User Interface Components

ActiveX ActiveX Controls COM/DCOM Microsoft Windows CE

JavaBeans Original purpose: creation of User Interface building blocks Available in “standard” Java and PersonalJava Easy Java component model Fits in choices made for implementation

S

E

E

S

C

O

A

UI Components - implementation

1 main component

1 sub-component per UI (e.g. a local, a remote, …)

UIMain (the UI component)

UI1

UI2

UI3

S

E

E

S

C

O

A

UI Components – implementation (2)

Realization of sub-components In JavaBeans

Represent GUI partsHandle all the GUI-eventsAWT-componentsSend events to sub-component

Out JavaBeans interaction with other componentsChecking password

Sub-component

JavaBeanJavaBean

JavaBeanJavaBean

S

E

E

S

C

O

A

UI Components - implementation (3)

Example working of UI component

MainPanel LocalUI MainUI

Sub-component

Event Admin Admin

AcceptedSetAdmin()

History

Event History RequestInfo

SetHistory()

GenerateInfo

Info

S

E

E

S

C

O

A

Conclusions

Further insight in UI development for Embedded Systems is gained concerning Consistency Design SEESCOA component system

Usage of XML for describing User Interfaces for Embedded systems There are many possibilities Continuing to explore possibilities

S

E

E

S

C

O

A

Conclusions (2)

Exploration possibilities of existing technologies concerning HCI for embedded systems (e.g. Palm UI)

Case of a User Interface was developed using SEESCOA component system

our experience gives us a foundation for a more general approach.