1

Ubiquitous Computing Lecture 14

Dr Mark WrightKey points:

– Making the computer part of the environment– Mobile devices– Implicit input– Ambient output– Continuous interaction– Issues for design and evaluation

2

Introduction• Human-computer interaction addresses the relationship between humans

and computers, doing tasks in environments

• The focus should be on supporting human activities• Ideally, the computer would just be part of the background environment in

which we do the task• This concept has been called invisible, pervasive or ubiquitous

computing (Weiser, 1991)

Human

Task

Computer

Environment

3

Dr. Mark Weiser (1952-1999), Chief Technologist at the Xerox Palo Alto Research Center

4

Introduction

• Computation beyond the workstation or desktop computer (fixed location, screen, keyboard & pointing device).

• Includes:• Handheld, portable and

wearable devices• Mobile phones, PDAs, digital

cameras, sat-navs etc.

Introduction

5http://www.pranavmistry.com/projects/sixthsense/

SixthSense: A Wearable, Gestural Interface to Augment Our World (MIT Media Lab, 2009)

6

Introduction• Computation beyond the workstation or

desktop computer (fixed location, screen, keyboard & pointing device).

• Includes:– Handheld, portable and wearable devices

• Mobile phones, PDAs, digital cameras, active badges etc.

– Very different scales or styles of output• Very small and very large displays,

distributed, 3-dimensional • Richer sounds, device actuation, ambient

cues • Augmented vs. virtual reality

9

IntroductionComputation beyond the workstation or desktop computer (fixed location, screen, keyboard & pointing device). – Handheld, portable and wearable devices

• Mobile phones, PDAs, digital cameras, sat-navs etc.– Very different scales or styles of output

• Very small and very large displays, distributed, 3-dimensional • Richer sounds, device actuation, ambient cues • Virtual or augmented reality

– Novel forms of input • Pen-based, touch based, proximity sensing• Voice operated• Tilt or motion sensing• Implicit input – location, time, context

– Embedded computers in other technologies • Cars, washing machines, etc.• Instrumented rooms, buildings, environments

10

Introduction• Often these factors interact:

Mobile device Smaller interface Alternative I/O

methodsReduced power and

functionality Specialised for certain functions Task specific

interfaces

Improved usability!

11

Guidelines for Small Devices• Should distinguish mobile (usable while moving) from portable (movable

but need to be stationary to used successfully)

• Use(fulness) immediately apparent• Structure interface to task• Short cuts and flexibility• Minimise memory load• Use consistent screen templates• Provide a Back function on every screen• Selection is better than writing

12

The Evolving iPod

13

Small devices and the web• Increasingly common for mobile devices to access the internet• Important constraints:

– Smaller screens show less, so memory load increases– Have awkward and error prone input methods– Very wide variety of device specifications (screen size, resolution, layout,

interaction options) compared to standard desktop/laptop– More connectivity and bandwidth issues– People are unlikely to try to do all the tasks offered by the full site using

their mobile device • Ideally should always make alternative ‘mobile’ site available, and make it

easy to switch between this and the full site• General principle: being able to do anything tends to make everything take

longer to do.

14

Norman’s “The Invisible Computer”• “Personal” computer is “massive, impersonal, abrupt and rude”, used

mostly to do social things and not for computing• A device that does everything will be convenient but will probably perform

worse than specialised devices: – Swiss army knife vs. kitchen knife, corkscrew, screwdriver, scissors

• Also get incremental addition of features, amount of information stored (disk space), amount available (WWW), so can no longer make everything visible or discoverable

• Norman suggests technology should move towards “information appliances”, i.e. many items serving specific needs:– ‘Home financial centre’ in right location with right connections– Displays providing weather, news, sports (compare to clock) – ‘Foreview’ mirror in cars for traffic, parking spots

15

Norman’s “The Invisible Computer”

“Devices that are easy to use, not only because they will be inherently simpler, but because they fit the task so well that to learn the task is to learn the appliance.”

• Simplicity and visibility of function paramount

• Will require infrastructure to allow seamless information transfer between devices

16

The iPhone

• Opposite direction: towards multipurpose device• Sensors: proximity, ambient light, accelerometer (orientation) and

touch screen – several advanced touch features• Loss of physical buttons removes ‘tactile landscape’

17

Natural interaction• Much concern in HCI about finding more ‘natural’ forms of input• `Context aware’ computing suggests computers should be able to recognise

implicit inputs: – E.g. walking into a space should be sufficient to announce your presence

and identity– More generally, tracking the user’s location to supply them with relevant

information (satnav)– Alternative forms of identification also have many obvious applications (with

varying requirements for reliability)– Time is another implicit cue: could exploit to detect interest, deviation from

routine, even identity• General idea is that input is a side-effect of doing the activity• Automated capture as a background technology

18

‘Invisible’ interaction• Potential to ‘eliminate’ conscious interaction

– E.g. remove ticket sales by tracking where people went and charging their accounts

– Verichip implant– Identify patients in emergency situtation– Other uses…?

• Replaces interaction programming problems with ethical problems– Those without resources to be

connected to system are disenfranchised

– Removes option of anonymous interaction

19

Natural interactionTangible input devices – manipulate ‘ordinary’ physical object

Tangible interfaces

20Sifteo Cubes (https://www.sifteo.com/)

22

Continuous interaction• Obtaining natural interaction may involve collapsing the usual input/ output distinction• `Continuous interaction’ emphasises the closed loop rather than stages of action

• On more extended time scales, includes thinking of HCI in terms of extended, ongoing activities rather than tasks that have a clear beginning and end– Should support interruptions, concurrency etc.

Goals

EvaluationSequence of actions

The world The world

Ongoing activity

23

Design processes for continuous interaction• Importance of ethnography in assessing user

requirements (don’t trust post-hoc rationalisation) what iswanted

analysis

design

implementand deploy

prototype

• Augment task analysis with: - Instantaneous information

requirements: will it be part of task, users memory, computers memory, or in environment?

- Trigger analysis: does next step occur immediately, after fixed or variable delay, or in response to external event?

24

Evaluation of ubiquitous computing• Some problems for evaluation

– Hard to measure relevant variables– Inappropriate to use methodologies that interfere with the normal

process (e.g. co-operative evaluation, lab experiment)– Need long term analysis– May be fuzzy usability criteria

n Standards and guidelines are developing for mobile devices, but still lagging for more immersive technologies

Spimes

• Bruce Sterling• Tracking objects in space and time

22

Internet of Things - IOT

23

Connected sensors and objects

New Meaning and Memory Practices - TOTEM

Smart home

21

Smart Cities• Environmental Sensors• Ambient Displays

25

Augmented Reality

7

Mixed Reality

27

Dourish: Ubiqitous Computing in the present

• Weiser’s original vision was of a ‘near future’ of seamless integration

• Dourish argues the future has arrived and it is messy and fragmented

28

25

References

Weiser, M. (1991) The computer for the 21st century. Scientific American, 265/3: 94-104

Norman, D. (1998) The Invisible Computer, MIT Press

The Computer Reaches Out: The Historical Continuity of Interface DesignJohnathan Grudin, CHI 90

Dourish P., Bell G., Divining a Digital Future: Mess and Mythology in Ubiquitous Computing, MIT Press, ISBN 978-0-01555-4

See also:

Dix et. al. sections 18.3, 18.4, and chapter 20