towards a task-oriented framework for smart spaces chuong c. vo, torab torabi, & seng w. loke...

Towards a Task-Oriented Framework for Smart Spaces

Chuong C. Vo, Torab Torabi , & Seng W. LokeDepartment of Computer Science & Computer Engineering

La Trobe University, Australia

1

Speaker: Seng W. Loke

C.C. Vo, T. Torabi, and S. W. Loke - La Trobe University

Outlines• Introduction

– Smart spaces– Usability problems with smart spaces– Task-oriented scenarios

• Approach: a task-oriented framework– Concepts & components

• Current implementation• Related work• Conclusion & future work

2C.C. Vo, T. Torabi, and S. W. Loke - La Trobe University

Introduction

• The first era of computer (past, many-one)– Mainframe: central, heavy, and expensive

• The second (now, one-many)– Personal computers: personal, light, inexpensive

• The third (now & future, many-many)– Post-PC: pervasive (everywhere, every time),

computers blend into environments .– Whole environments are computers.


Smart space

• What is a smart space?– Seamlessly integrating computational elements

into the fabric of everyday life…” [Weiser 1991],– Everyday objects and environments are aware of

their surroundings & peers and behave smartly.

• The aims:– Support our activities, complement our skills, add

to our pleasure, convenience, accomplishments [Norman 2007].


Usability problems with smart spaces

• Complexity of use: Variety of devices, UIs, remote controllers– Requires too many buttons and menus on UIs, exceeds capacity of UIs

for users to operate them intuitively [Rich 2009].– Users need to understand how to map devices' functions to their tasks

& sub-tasks.5

Slideshowcontroller

TV controller

A future smart space

Audio controller

Blind controller

TV

Video-conferenceDisplay Temperature

controller

Projector

DVD player

Printer

ComputerLightcontroller

Recorder

Smart phone

Door

Coffeemaker


Usability problems (cont.)

• Invisibility & Overload of features– Technologies blend into environments.– Frequently adding/removing devices and services

to/from the spaces.– One device tens of features– Different combinations of devices

Hundreds of features

6MediaCup [Beigl et al. 2000]


Usability problems (cont.)

• Inconsistency of user interfaces– Brand identification, product differentiation [Rich

2009; Oliveira 2008]

• Inconsistency of task executions– Same tasks but different operations/procedures

when being executed in different smart spaces.

7

How to abate these usability problems?

Our approach is based on task-oriented computing.


Task-oriented scenarios• Pervasive city’s scenario

– “It’s 7p.m., it’s raining, and you’re walking in the centre of Melbourne. You consult your phone and it suggests ‘Dinner?’, ‘Taxi?’, ‘Bus?’.Selecting ‘Dinner?’ will present restaurants you’re apt to like and even dishes that you may want…”

• Pervasive university campus’s scenario– “You’re driving approaching La Trobe Uni. Campus,

the LCD on your car suggests ‘Campus map?’, ‘Find a place?’, ‘Parking spot?’. Selecting ‘Parking spot’ will guide you to find a parking spot.”

8

Task-oriented scenarios (cont.)

• Pervasive personal office’s scenario– “You enter your office. The lighting, heating, and

cooling levels are automatically adjusted based on you electronic profile. The coffeemaker works to give you a cup of hot white coffee.”

9

Our long term aim is to realize these scenarios!


Task-Oriented Computing

• Our approach is based on task-oriented computing [Wang et al. 2000]:– A task is a user’s goal or objective [Loke 2009].– Users interact with/think of the computing in

terms of tasks instead of applications/devices’ functionality.

– Users focus on the tasks at hand rather than on the means for achieving those tasks [Masuoka2003].

– Application function is modeled as tasks and sub-tasks.


Approach: a task-oriented framework

Problem Proposed solutionComplexity of use Task-based user interfacesInvisibility & overload of features

Context-aware task recommendation

Inconsistency of UIs & task executions

Abstraction of task models


Overview of the task-oriented framework

12

Developer/Designer

Task modelTask modelTask models

Task Repository

Context-Aware Task Recommender

Task Execution Engine

Context Information Manager

User(s)

(1) P

ublis

hes

(2a) Advertises

task models

(2b)

Pro

vide

s

cont

ext

(3) Recomm

ends

tasks

(4a)

selec

ts

task

s

(4b) Provides

context

Devices

Services

(5) invoke


Task description language

• Syntax and semantics for describing tasks.• Some requirements:

– Abstraction– Interpretability– Support for inter-task dependencies– Support for proactive task guidance:

• What should I do next? How do I do <task>? When should I do <task>? Why did you do <task>? Did <task> succeed? Why did the task failed? How to fix the problem?...

C.C. Vo, T. Torabi, and S. W. Loke - La Trobe University13

Task description repository

• Manage and advertise task model descriptions• Challenges:

– Effective mechanisms for searching and matching task models

– indexing techniques and task description queries


Context-aware task recommendation• A combination of the following methods:

– Location based– Pointing gesture based– Collaborative filtering using situation similarity

15

Location-based task recommendation

16

Location = La Trobe University Campus

Find a pathFind a placeFind a parking spot…

Tasks



17



Tasks

Enroll a subjectFind a room…

Tasks

Location = Building PS1



18



Tasks


Tasks


Location = Personal office PS1-219

Make coffeeDim lightsWatch TV…

Tasks



19



Tasks


Tasks


Location = Personal office PS1-219


Tasks Location = TV’s zone


Tasks


Pointing-based task recommendation

20

Location = the Agora

Find a placeMeet friendCoffee…

Tasks

Theatre Coffee shop

…



21



Tasks

Theatre Coffee shop

…

Pointing at = the Theatre

What is on?Special offer?Ticket booking…

Tasks



22



Tasks

Theatre Coffee shop

…

Pointing at = the Coffee shop

What is on?Special offer?Ticket booking…

Tasks

CoffeeFoodSpecial offer?…

Tasks



23

Location = Personal office

TV

Air-conditioner


Tasks



24

Location = Personal office


Tasks

Pointing at = the Air-conditioner

TV

Air-conditioner

Temperature upTemperature downSet fan speed …

Tasks


Situation-based collaborative filtering

• Situation-based collaborative filtering– Assumption: “People tend to accomplish similar

tasks in similar situations”.– The tasks previously accomplished by similar

users in similar situations are recommended.

• Steps:

25

Find similar users

Find similar situations

Rate tasks done by the similar users in the similar situations

The details are our previous work Vo et al. 2009).

Task execution engine

• Load and execute descriptions of the tasks selected by the user.

• Provide instructions to guide the user through the task accomplishment.

• Inform the user of the progress of task completion and failures.

• Manage sessions of task executions.


Current implementation

• Technologies for estimating locations



28

• Location-based task recommendation

Location = University campus Location = Personal office

Television


29

• Indoor pointing: uses Cricket system• Outdoor pointing: uses iPhones with compass & GPS built-in

Air-conditioner

Cricket Listener

Cricket Listener

Cricket Beacon

pointing at TV

pointing at Air-conditioner

tasking

tasking



30

• Executing tasks

Power line

Task Execution EngineW

irele

ssly

exe

cute

task

s

We’ve implemented some simple tasks

using X10 technology

Kettle

Light


Future work

• Design a comprehensive task description language• Develop a graphical editor for authoring task descriptions• Extend the task execution engine• Develop mechanisms for effectively publishing and

retrieving task models:• Indexing, matching, searching, composing, recognizing task

models

• Address conflicts of task executions in multi-user environments….


Related work• Situation-aware application recommendation [Cheng et al. 2008]

– They recommend applications <> We recommend tasks (multi-apps)– They use pure situation similarity <> We use task based similarity

• Homebird system [Rantapuska et al. 2008]– It recommends tasks based on features of devices discovered– However, because this approach does not consider user situation, it can recommend feasible

tasks which may be not relevant.• InterPlay [Messer et al. 2006]

– For device integration and task orchestration in a networked home.– It asks user to express their intended tasks and assumes that the users have knowledge about

feasible tasks.– In contrast, our approach can recommend relevant, feasible tasks without these

requirements.• Context-dependent task discovery [Ni et al. 2006]

– Discovering active tasks by matching current context with required context of tasks.– This can discover feasible tasks but potentially irrelevant tasks.

• Task retrieval [Fukazawa et al. 2005]– Ask user to specify target names (e.g., cafe shop, theatre) for retrieving tasks which are

associated with these names.– Our system has integrated this knowledge into place/devices-related task repository.


Conclusion

• Defined usability problems with smart spaces: Complex of use, Invisibility & overload of features, and Inconsistency of UIs & tasks.

• Presented the task-oriented framework with components: Task description language, task repository, context-aware task recommender, task execution engine…

• Given an early implementation & applications.• Outlined the research agenda.


ReferencesM. Weiser, “The computer for the 21st century,” Sci. American, 3(265), pp. 94–104, 1991.C. Vo, T. Torabi, and S. Loke. Context-aware task recommendation. In ICPCA-09, Taiwan, 2009.Z. Wang & D. Garlan, “Task-driven computing,” School of Computer Science, Carnegie Mellon University, Tech.

Rep., 2000.D. Garlan et al. “Project Aura: toward distraction-free pervasive computing,” Pervasive Computing, IEEE, 1(2), pp.

22–31, 2002.R. Masuoka et al. “Task computing – the semantic web meets pervasive computing,” The SemanticWeb, pp. 866–

881, 2003.D. Magnusson & B. Ekehammar, “Similar situations–similar behaviors? a study of the intraindividual congruence

between situation perception and situation reactions,” J. of Research in Personality, 12, pp. 41–48, 1978.A. K. Dey, “Understanding and using context,” Per. and Ubi. Computing, 5(1), pp. 4–7, 2001.D. Cheng et al. “Mobile situation-aware task recommendation application,” in The 2nd Int. Conf. on Next

Generation Mobile App., Services, and Tech., 2008.A.Messer et al. “InterPlay: A middleware for seamless device integration and task orchestration in a networked

home,” in PERCOM’06. 2006, pp. 296–307.H. Ni et al. “Context-dependent task computing in pervasive environment,” Ubi. Comp. Sys., pp. 119–128, 2006.Y. Fukazawa et al. “A framework for task retrieval in task-oriented service navigation system,” in OTM Workshops

2005, pp. 876–885.O. Rantapuska and M. Lahteenmaki, “Homebird–task-based user experience for home networks and smart

spaces,” in PERMID 2008, 2008.

34

Questions?

Thank you!


towards a task-oriented framework for smart spaces chuong c. vo, torab torabi, & seng w. loke...

Documents

trobe university slide

taskoriented computing

proactive task guidance

objective loke

tasks subtasks

torab torabi

sub tasks

different smart spaces