Ken Fishkin, Nov. 2000 1

Invisible Interfaces

Ken Fishkin, Anuj Gujar, Beverly Harrison, Roy Want

Ken Fishkin, Nov. 2000 2

Trend

Exponential growth in computer speed; Moore’s Law, but

Limited growth of bandwidth and naturalness of our physical interaction with the computer

Ken Fishkin, Nov. 2000 3

Emerging UI Paradigm

Time50’s 60’s 70’s 80’s 90’s 98

InteractionBandwidth

switches& knobs

commandline

GUI

AR, VRPhicons

“InvisibleInterfaces”

Ken Fishkin, Nov. 2000 4

Towards “Invisible Interfaces”

Goal: to seamlessly blend the affordances and strengths of physically manipulatable objects with virtual environments, devices and artifacts

Physical Virtual

Invisible Interfaces

Ken Fishkin, Nov. 2000 5

Our Invisible Interface Efforts

Manipulative UI PDAs pressure sensors, tilt sensors

Proximal UI mobile computing RF sensors wireless networking

Ken Fishkin, Nov. 2000 6

Manipulative UI Approach

interaction through transparently mimicking familiar physical manipulations

via augmenting handheld devices

invisible interfaces

+

Ken Fishkin, Nov. 2000 7

Video...

3 tasks navigation through an ordered list navigation within a document annotation optimization (handedness

detection)

Ken Fishkin, Nov. 2000 8

Key Design Considerations

Match properties e.g., book - unit pages, sequential

ordering preserved but thickness or extent is lost

Match manipulations e.g., book - flicking corners consistent but

moving by “chunks” is notMatch feedback

visual, auditory, kinesthetic (tactile)

Ken Fishkin, Nov. 2000 9

Navigation - Rolodex

Physical Rolodexmanipulation

Virtual Rolodexmanipulation

Ken Fishkin, Nov. 2000 10

Navigation - Rolodex

Side view - Palm PilotPressure sensors

Custom circuitry on back

Behind the scenes...

Ken Fishkin, Nov. 2000 11

Navigation Within a Book

Page turning via strokes

Physical page turning

Ken Fishkin, Nov. 2000 12

Navigation Within a Book

Moving relative to beginning/ending of document

Moving by “chunks” relativeto beginning/ending of book

Ken Fishkin, Nov. 2000 13

Navigation Within a Book

Behind the scenes... degree of spatial pressure location

Ken Fishkin, Nov. 2000 14

Next Iteration

Ken Fishkin, Nov. 2000 15

Annotation Optimization

Physical annotationVirtual annotation (handedness)

Ken Fishkin, Nov. 2000 16

Annotation Optimization

Behind the scenes...

Pressure pad

Ken Fishkin, Nov. 2000 17

Annotation Optimization

The result...

Ken Fishkin, Nov. 2000 18

Lessons Learned

No prior model that physical interaction has an effect (unexpected, learning needed)

Fidelty of matching is crucial (“interface fusion”), based on real world expectations

Inadvertent action is problematicCareful kinesthetics design needed (range

of motion, precision, JNDs)

Ken Fishkin, Nov. 2000 19

Lessons Learned

User comments: “intuitive”, “cool”, “pretty obvious in terms of what’s going on”

Users explored the range and space of manipulations (as in novel GUIs)

Passive interactions perceived as “magical”

Enhanced interaction experience “beyond the laptop”

Ken Fishkin, Nov. 2000 20

Related Work

Physical input devices: “phicons” [Ishii et al], “bricks” [Fitzmaurice]

Unique input devices: “doll’s head” [Hinkley et al]

Augmented devices [Rekimoto] [Small & Ishii]

Ubicomp, PARC tabs [Weiser & Want PARC]