Direct Manipulation: Ideas, Benefits, & Limitations

數位內容學院遊戲開發研究班第一期

3D圖學

沈育德 Edward ShenMay 28, 2005

Papers

• “Direct Manipulation for Comprehensible, Predictable, and Controllable User Interfaces”, Ben Shneiderman, IUI 1997

• “The History and Future of Direct Manipulation”, David Fronhlich, HP Lab, 1993

• “Direct Manipulation vs Software Agents”, ACM Interactions, 1997

My Motivation

• With our behavior authoring system, I would like to introduce to you this 20-year-old design methodology, which has greatly influenced today’s software interfaces.

• And with this opportunity, I wish to bring you some ideas about different HCI design philosophies

Outline

• Preface - Behavior Authoring v.s. Synthetic Animals• Direct Manipulation: Definition, Benefits,

Implementations• Direct Manipulation v.s. Software Agents• Direct Manipulation: From a historic point of view• Conclusion - Toward a newer philosophy

Preface

• Behavior Authoring

Our Behavior Authoring System

More Directness

Programming with codes/graphsDirect Manipulaiton

But Indeed, the Difficulties Are:

• The more power users are endowed, the more complex the manipulative interface would become

• Variables, conditions, iterations are hard to achieve

• What if the system is smarter???– Preferences, Habits, History, etc

Synthetic Characters

• By Bruce Blumberg, MIT, since 1995

Blumberg, B. et al: Integrated learning for interactive synthetic characters. SIGGRAPH 2002, pp. 417–426.

Which is the better interface design?

• Is it more direct to specify details from a programming point of view?

• Or to interact with the characters in an interaction/conversational pattern?

• Which is the better way for users?

Direct Manipulation

Origin

• Ben Shneiderman, “Direct Manipulation, A Step Beyond Programming Languages”, 1983

• “One of the most significant development of the 1980’s in HCI … as a theoretical concept and design practice” (Frohlich93)

Motivation

• To create environments where – users comprehend the display– users feel in control– the system is predictable– both users and the systems are willing to be

responsible for their actions

• “user-friendly” -> time spent, error rate/distribution, retention over time, etc

Definition

• Direct Manipulation is characterized by :1. Continuous representation of the objects

and actions of interests

2. Physical actions or presses of labeled buttons instead of complex syntax;

3. Rapid incremental reversible operations whose effect on the object of interest is immediately visible.

Physically Manipulating the Graphical Representations

• Benefits:– Ease of learning– Ease of use– Retention of learning– Reduction and ease of error correction– Reduction of anxiety and greater system

comprehension

Examples

• Video games

• Spreadsheets

• Other office systems

Why Exciting

• “Programming Language” sort: database, statistics, editing, systems, etc

Tasks with tedious command or programming

languages

Lively, enjoyable interactive systems

that reduce learning time,

speed performance and increase satisfaction

FilmFinder (1994)

FilmFinder (cont’d)

FilmFinder (cont’d)

Visible Human Explorer (1996)

The Direction Towards DM

• Information Visualization

• Enabling users to navigate through 4000 or more icons (2~3 orders of magnitude of current use)

• The visual presentation gives users enormous bandwidth and feeling of being control and responsible for decisions they make

Debate (1997):Direct Manipulation

v.s. Software Agents

Pattie Maes

• MIT Media Lab (software agents -> ambient intelligence)

• PostDoc in MIT AI Lab

• Software agents, machine learning, collaborative filtering

Software Agents

• Difference from conventional software– Personalized: habits, preference, interests– Proactive: take the initiatives– More Long-lived– Adaptive

Why Agents?

• Personalization is needed as programs goes complex and users become naïve– Computers have become a window onto a

vast dynamic network impossible to visualize– Users are different too

• To delegate tasks and information to people/assistants

• Suggestion (web browsing), reminder (email), matchmaking (interests, marketing)

Kasbah

Complementary, Not Alternative?

• Pattie: benefiting untrained users – Suggestions, not decisions (e.g. movies)– Direct manipulation interface still in need

• Ben: full control, achievement, responsibility– Human-to-human interaction is NOT a good

model for the design of user interfaces – Adaptive systems may be annoying,

irresponsible, unsuitable for financial, military usages

Direct Manipulation: From a historic point of view

by David Frohlich, 1993

David Frohlich

• Senior Researcher @ HP Lab

• “In fact, the entire debate about the relative advantages and disadvantages of language versus action based interfaces turns on an attempt to explicate the conditions under which each is most direct”

Historical thinking on direct manipulation (I)

• Hutchins, Hollan, Norman (1986)– Directness:

• Distance between user’s goals and actions• Engagement of feeling oneself in full control over sth

– Lost of the power of abstraction • Unseen objects• Retrieval would be much easier with conversation• Agents: “The user should be able to have a conversation abo

ut the world with the agent, and both the user and the agent should be able to manipulate the shared world”

Historical thinking on direct manipulation (II)

• Laurel (1990)– Difficult to do with first-person mode:

• Retrieving, sorting, organizing, programming, scheduling

– Needing computers to do proactively:• Information filtering, reminding, help, tutoring

Historical thinking on direct manipulation (III)

• Claassen et al (1990)– Conversation mode

• Bad for maintaining a mental model, spatial structures, referring to entities

– Manipulation mode• Bad for functional, causal properties, abstractions,

referring to invisible objects

Major Tech. Development since 1983

• Virtual world systems

• Virtual partner systems

• Mixed mode systems

Virtual World Systems

• Traced back to Sutherland’s Sketchpad, 1963

• VR: DataGlove, Head-Mount Display used in military, entertainment, medical area

• Mark Weiser: Expecting people to transfer their

everydayactivities into the

artificial environment

Computers should invisibly enhance

the real world environment that people are familiar

with operating

Virtual Partner Systems

• Knowledge based or expert systems (e.g. MYCIN)

• Natural language information retrieval systems (e.g. INTELLECT)

• Handwritten input devices/systems• Computer mediated communication such as

email, videoconferencing

Mixed Mode Systems

• The separation between the conversational and manipulative components is less clear.

• Linguistic commands on menus are invoked in combination with manipulative metaphors (e.g. cut-n-paste) in desktop office systems

Mixed Mode Systems

Summary: Constraints of Direct Manipulation

• Interaction constraints:– Informing and reminding– Responding to interrogation– Helping and advising– Delegation and problem solving

Summary: Constraints of Direct Manipulation

• Task constraints:– Referring to parts of the previous interaction– Scheduling actions to take place in the future– Identifying unseen objects– Identifying groups of objects– Performing repetitive actions – Doing more than one thing at a time– Specifying actions very precisely

Today’s Technologies

Tangible User Interface

Commonsense Computing

Conclusion

• Pure direct manipulation will eventually impossible to empower novices over complex tasks

• I personally agree with the mixed mode, reserve the control to users while provide proper suggestions