Designing Future Environments

David Kirsh

Dept of Cognitive Science

UCSD

Question

• How can we design environments that are:

– Cognitively congenial• Cognitively more efficient• Less stressful• Reduced cognitive overload

– More fulfilling – provide a better experience• More aesthetic• More fun• Let us be more creative

– ?? what else??

Agenda

• Background

– Technology

– Changing conception of agent-environment coupling

• Cognitive Principles of Interactivity

• A Science of Design?

• Coordination at Starbucks

Background of Inquiry

Background of Inquiry: Technology

– Walls are data walls

– Internet everywhere

– Wireless everything

– Near field haptics

– Easy telepresence

– Effective digitization of paper– Sensors make it easy to cross

from physical to digital

– Rooms are context aware

Context aware Ubiquitous computing Peripheral Robotic

Architecture

Gehry’s Disney Auditorium: Los Angeles

Agent-Environment Coupling

Classical Model of Activity

D CA

Project Structure (Meaning, Interpret environment )

AGENT ENVIRONMENT

DI’ve got to make

DCA

Formalizing problem solving

Move D onto A

Move C onto AMove A onto D

Move C onto DMove A onto C

Move D onto C

What are we abstracting from?

Activity Space

Activity Space

Activity Space

Activity Space

Activity SpacePure Structure of Task

task environment (state space) is abstraction

Classical formulation

• Humans adapt to structured environments – Develop efficient routines

• Problem is to describe the environment of activity– Classical approach environment is

collection of task environments

– Formally each task environment is a connected graph of choice points

A single task environment

Interim Summary

• Agent lives in many task environments• Switches tasks as necessary• Projects lots of structure

Physical space andStuff occupying it

Theorist postulates as

many task environments

as tasks agent performs

in an environment

So how wrong is this view?

Throw it out completely - almost

• Natural tasks are ill defined

– We do lots of extra-task actions

• Action set not well defined

– Our goals are more flexible

– Other considerations

• Multi tasking causes interference and must be managed

• Consequence function is not well defined or hard to predict

• No well defined set of choice points

• Metric of closeness to goal not usually well defined

X

Two Types of Environment

E state changes autonomously

Griddle

E state changes onlywhen agent intervenes

Tower of Hanoi

Dynamical Environment

Autonomous changes

•Gravity

•Heat

•Syrup

•Air Temp

manageJob is to control a process

regulate

Continuous action, states

Some Very Dynamical Problems

• Coordination problems

World acts back continuously

Part of a system

Some state space problems

Forced Choice

Most of life has a little of both

Starbucks

Real environments are hectic

• Given:– We do lots of things in them– We change tasks a lot– We negotiate our tasks with others– We ‘negotiate’ our tasks with ourselves– Plenty of interruption– We get distracted

Kitchen Environment

Familiar collaborative environment

What extra is going on in these environments?

• Representation rich• Dialogue• Gesture• Situated thinking

– Pointing to representation and talking about what something means in that context

• Negotiating what the goal is and when you have made done well enough

• Managing space• Managing attention• Coordinating your activity so you know what to do next

and what you have already done

How do we put them together?

• Better science of interactivity

• Develop principles of design

Cognitive Principles of Interactivity

Externalization

We externalize to increase our power

Externalization

Externalization

• Externalizing let’s us interact using principles of visual processing that are different than internal visualization and thinking

• We are more data driven – more coupled – than we think

• Externalizing can break cognitive set

• We learn norms of reasoning,tricks of manipulation

Internal necker cube does not oscillate

Examples of Externalization

• Verbalize our thoughts

• Sketch

• Gesture

• Write on paper

• Point

• Set our ingredients before cooking

External Representations that help us think

Conceptual Mathematics. Visual Proofs.

Prove: the sum of the odd numbers,

1 + 3 + 5 + … + 2n – 1 = n2

Baigrie, Brian S., Ed. Picturing Knowledge. Buffalo: University of Toronto Press, 1996.

Conceptual. Mathematics. Visual Proofs.

Baigrie, Brian S., Ed. Picturing Knowledge. Buffalo: University of Toronto Press, 1996.

We coordinate our actions - inside and outside to increase our power

36

Scrabble shows Dynamic Couplingof Projecting & Creating Structure

ecesrrruutt

ee cs rrr uu tt

ece s rrr uu tt crust strut e

restructure

Project Structure = Mentally Represent

Create structure

Mentally reorder

Physically reorder

37

Interaction at high speed

• Manipulate physical objects to save mental manipulation

• Tetris examples:

– physical rotation saves mental computation & is faster

• Piece recognition

• Placement decision

– physical translation improves certainty

• Both interactive strategies involvemillisecond coordination

Work done with Paul Maglio

We develop systems that encode information

39

Short Order Cook: Hamburgers

• Activity is coordinated by reference to state

• Environment is prepared to make state explicitAlex Kirlik’s example

Preparing by demarcating regions

• reduce clutter, reduce combinatorics of problem, track state better

Encoding Assembly Order

• Converts combinatoric nightmare to ‘simple’ hill climbing

Preparing the workspace

• Topological constraints are more natural

Why Re-arrange Cards?

As dealt

A Science of Design?

Interface

Interface - redesigned

Why is one better than another?

Why??

Why is one better than another?

• Cleaner, more white space• Better visual layout• Modularizes activity – helps to plan, review, compare

Design Principle One

• What is semantically related is visually related

• What goes together semantically should of together visually

• What is semantically associated should be visually associated

Discovering Organizing principle

• People try to impose order– We need to figure out what order they will project

so that their behavior becomes predictable

• Pattern discovery 1 4 9 16 25 …

• Gestalting

• _ r _ a _ b _ e -- fragment completion

Enough for now

Design Principle Two

• When you operate on a to effect A – Use intuitive correspondences so that

• Actions on a have a natural interpretation in A• Design so that there are intuitive correspondences

• When there are multiple widgets that affect multiple targets – Use intuitive correspondences so that

• Widgets in action domain have a natural interpretation in target domain

• Design so that there are intuitive correspondences

Collection of symptoms in patient

Diagnosis = organizing principle

Understand Organizing Principles

Some people think of this as a mental model that allows one to predicthow they will interpret other symptoms, or predict what they will expect

Discovering Organizing principle

• People try to impose order–

• 1 4 9 16 25 …

•

• p r e a m b l e -- fragment completion

Cow

Cow with outline

Predictability

• Once we know how someone organizes a set of elements we can predict their behavior better

Design Principle Three

• Facilitate construction of patterns or mental model or organizing principle so that we can increase the probability that people will make the right correspondence between a and A

Inconsistent organizing principles

Design Principle Four

• If subjects have more than one organizing principle ensure that they are consistent otherwise you cannot predict whether a means A or whether a means B

Design Principle Five

• Recognition is better than recall

• Give users visual choice rather than conceptual choice

Recognition vs. Recall

Better still

Design Principle Six

• Find effective ways of coordinating individuals

Putting it together in an environment• When is one environment better than another?

• Relative to a set of tasks

Question

How does redesigning an environment reshape routines?

•Artifacts

•Technology

•Cue structure •Spatial layout

How do we know they are better?

• Performance measures– Faster– Fewer Errors– Agents can do more complex things with them– Fewer serious errors – less variance

Speed Accuracy

Probabilityoferror

Time

1

0

Better

Speed Accuracy of Routine

Ri is better than Rj if it can be performed (dominates)

• More quickly without increase in expected error

• More error-free without decrease in speed

Complexity of Routines

Probabilityoferror

Time

1

0

Complexity of Routines

AcceptableError

AcceptableTime

Acceptable

C2

C1

C3

C4

Ri is better than Rj if

• Ri tasks are more complex

• can be performed in acceptable time and error rate

Variance of Routines

Variance

Time

1

0

Variance of Routines

AcceptableVariance

AcceptableTime

Acceptable

V2

V1

V3

V4

• Reduce the variance in output

• For each error rate in the speed accuracy curve the output will be

more standardized

• Narrowing the distribution of error size not the number of errors

Seriousness of error

Distribution of Errors

0

Recovery of Routines

Design Challenge:

• redesign the environment to lower recovery time

• redesign to facilitate vigilance and error detection

Learnability

100%

Degree of Mastery

Incentive

Late adopter

Average adopter

Early Adopter

$$$

0

More usable

Cost to Learn a New Technology

Starbucks

Five major Steps in espresso cafés

1. interact with client to specify order

3. take cashmake change offer receipt

2. communicate order

4. prepare the order

5. announce completion of orderqueue for client to collect

Espresso facts

• Called espresso because made for a specific customer and served immediately.

• A double espresso is – 1.5 - 2 ounce liquid extract – prepared from 14-17 grams of (medium) ground coffee – purified water of 88-95°C has been forced through– at 9-10 atmospheres of pressure – for a brew time of 22-28 seconds. – Crema should make up 10-30% of the beverage

• Cappuccino – A shot of espresso topped with equal parts of steamed and foamed milk

(wet cappuccino) – a shot of espresso topped with all foamed milk (dry cappuccino).

• Frothed milk should be 150°• Steamed milk should be 150° to 170°

Design Challenge

• Increase robustness of process– Reduce error

– Reduce variance of error

– Eliminate disastrous errors

• Process more drinks per hour – Routines and tech support

higher throughput

• Increase quality of service– Better interaction with customer

• Increase drink complexity• Routines are easier to master

• Error is always lurking– Noisy– Distractions– Surprises

• Interruptions, intrusions

• Multi-tasking, Task

Switching

• Multiple tasks in same

physical space

• High staff turnover

Costs to Minimize Problem Areas

Starbucks Revolutionary Technology

• Changes cognitive efficiency of whole system

• Minimizes costs in most areas

Technology of coordination

Form on cup

Why is it so remarkable?• Reduces errors

– Losing the order – Confusing one order with another

• Robust to interruption

– If barista forgets order just look it up

– Supports recoverability – increase state

• Tolerates breakdown

– If barista burned another picks it up off the floor

• Supports multi-tasking

– Locks info to object so more modular– Move along as in production like process

• Order complexity can go up

– Cup allows linear process

• Read, execute, read execute …

• Lowers cognitive demands

The End

Epistemic: Hard to reach states

TC WA E R S

Task: Call out all the words you can think of that can be made with some or all of these letters.

Hard to get more than 20 words in 5 min

Scrabble

Epistemic Action: Self-Cueing

TC

W

A

E R

S

Re-arrangement is allowed.

Close Spatial Coupling

• Humans are closely coupled in space and time to their environment

• This can be exploited:– re-arrange environmental resources to

• simplify judgment • computation

Preparing a Hand for Gin Rummy

• organization encodes current strategy

Player 1 Player 2

Segment E in a more congenial way

Count the dots.

which dot is the starting dot?

Adapt the world to our perceptual system

First and last

now stand out

Actions that compensate for Attention limitations

How many dots?

Requires coordination of rhythmic inner counting with hand movement

Re-arrange Distractors

reduce descriptive complexityreduce visual complexity