the growth of cognitive modeling in human computer interaction since goms judith reitman olson and...
TRANSCRIPT
The Growth Of Cognitive Modeling in Human Computer Interaction
Since GOMS
Judith Reitman Olson and Gary M. Olson
The University of Michigan
Presenters: Tosin Aiyelokun and Norman Makoto Su
Outline Cognitive Modeling Introduction to GOMS GOMS Extensions Modeling Specific Components GOMS Limitations Summary
Outline Cognitive Modeling Introduction to GOMS GOMS Extensions Modeling Specific Components GOMS Limitations Summary
Cognitive Modeling: Definition A theory that produces a computational model
of how people perform tasks and solve problems by using psychological principles and empirical studies.
Cognitive Modeling: Research Methods
EmpiricalEmpirical MethodsMethods
ProgrammingProgrammingTechniquesTechniques
Formal AnalysisFormal Analysis
Philosophy,
Logic,
Linguistics,
Mathematics,
Computer
Science
Experimental Psychology, Neuroscience
Artificial Intelligence
Cognitive Modeling: Role Limits the design space Answers specific design decisions Estimates total task time Estimates training time Identifies complex, error-prone stages of the
design A means of testing current psychological
theories
Cognitive Modeling: Human Information Processor (HIP)
ReceptorsReceptors
(perception)(perception)
EffectorsEffectors
(motor actions)(motor actions)
ProcessorProcessor
MemoryMemory
External World
HIP
The Human Processor Model
Perceptual Processor-sensory input (audio & visual)-code info symbolically -output into audio and visual image storage (WM buffer)
Cognitive Processor-input from sensory buffers-access LTM to determine response -output response into WM
Motor Processor-input response from WM-carry out response
Cognitive Modeling: Applications GOMS
Today’s presentation Soar
Integrated architecture for knowledge-based problem solving, learning and interacting with external environments.
ACT-R Atomic Components of Thoughts - Rational
Outline Cognitive Modeling Introduction to GOMS GOMS Extensions Modeling Specific Components GOMS Limitations Summary
GOMS: Overview Formal representation of routine cognitive
skill. A description of knowledge required by an
expert user to perform a specific task. Provides a description of what the user must
learn.
GOMS: Classification Provides a predictive, descriptive and
prescriptive model Predictive
Predicts the time it will take user to perform the tasks under analysis
Descriptive Represents the way a user performs tasks on a system
Prescriptive Guides the development of training programs and help
systems
GOMS: Definition GOMS models user’s behavior in terms of:
Goals What the user wants to do.
Operators Specific steps a user is able to take and assigned a specific execution
time.
Methods Well-learned sequences of subgoals and operators that can accompli
sh a goal.
Selection Rules Guidelines for deciding between multiple methods.
GOMS: A Family of Models Keystroke-Level Model (KLM) Card, Moran, and Newell (CMN-GOMS) Natural GOMS Language (NGOMSL) Cognitive-Perceptual-Motor GOMS
(CPM-GOMS)
GOMS: Keystroke-Level Model (KLM) Simplest GOMS technique
The basis for all other GOMS techniques Predicts execution time
Requires analyst-supplied methods Assumes that routine cognitive skills can be decompose
d into a serial sequence of basic cognitive operations and motor activities, which are: K: A keystroke (280 msec) M: A single mental operator (1350 msec) P: Pointing to a target on a small display (1100 msec) H: Moving hands from the keyboard to a mouse (400 msec)
Top-level Goal: Edit Manuscript (move “quick brown” to before “fox”)
Subgoal: Highlight text
Operators: Move-mouse Click mouse-button
Type characters (keyboard shortcuts)
Methods: 1. Delete-word-and-retype (retype method) 2. Cut-and-paste-using-keyboard-shortcuts (shortcuts method) 3. Cut-and-paste-using menus (menus
method)
Selection Rules: If the text to be moved is one or two characters long, use retype method
Else, if remember shortcuts, use shortcuts method
Else, use the menus
method
KLM Example
Description Operator Duration (sec)
Mentally Prepare M 1.35
Move cursor to “quick” P 1.10
Double-click mouse button K 0.40
Move cursor to “brown” P 1.10
Shift-click mouse button K 0.40
Mentally Prepare M 1.35
Move cursor to Edit Menu P 1.10
Click mouse button K 0.20
Move cursor to Cut menu item P 1.10
Click mouse button K 0.20
Mentally Prepare M 1.35
Move cursor to before “fox” P 1.10
Click mouse button K 0.20
Mentally Prepare M 1.35
Move cursor to Edit menu P 1.10
Click mouse button K 0.20
Move cursor to Paste menu item P 1.10
Click mouse button K 0.20
TOTAL PREDICTED TIMETOTAL PREDICTED TIME 14.9014.90
Method Used
Cut-and-paste-using-menus
1
2
3
4
5
M=1.35P=1.10K=0.20
GOMS: Card, Moran, and Newell (CMN-GOMS)
Subgoal invocations and method selection are predicted by the model given the task situation
In program form – analysis is general and executable
Predicts operator sequence and execution time Based directly on the Model Human Processor
Description Duration (sec)
GOAL: MOVE-TEXT
…….GOAL: CUT-TEXT
……………GOAL: HIGHLIGHT-TEXT
………………..MOVE-CURSOR-TO-BEGINNING 1.10
………………..CLICK-MOUSE-BUTTON 0.20
………………..MOVE-CURSOR-TO-END 1.10
………………..SHIFT-CLICK-MOUSE-BUTTON 0.48
………………..VERIFY-HIGHLIGHT 1.35
……….........GOAL: ISSUE-CUT-COMMAND
…………………MOVE-CURSOR-TO-EDIT-MENU
…………………PRESS-MOUSE-BUTTON 0.10
…………………MOVE-MOUSE-TO-CUT-ITEM 1.10
…………………VERIFY-HIGHLIGHT 1.35
…………………RELEASE-MOUSE-BUTTON 0.10
………GOAL: PASTE-TEXT
…………….GOAL: POSITION-CURSOR-AT-INSERTION-POINT
…………………MOVE-CURSOR-TO-INSERTION-POINT 1.10
…………………CLICK-MOUSE-BUTTON 0.20
…………………VERIFY-POSITION 1.35
…………….GOAL: ISSUE-PASTE-COMMAND
…………………MOVE-CURSOR-TO-EDIT-MENU
…………………PRESS-MOUSE-BUTTON 0.10
…………………MOVE-MOUSE-TO-PASTE-ITEM
…………………VERIFY-HIGHLIGHT 1.35
…………………RELEASE-MOUSE-BUTTON 0.10
TOTAL PREDICTED TIMETOTAL PREDICTED TIME 14.3814.38
CMN-GOMSCMN-GOMS
Outline Cognitive Modeling Introduction to GOMS GOMS Extensions Modeling Specific Components GOMS Limitations Summary
Extending GOMS: Grammars Explicitly represent knowledge a user needs to
translate from goals to actions. Task-Action-Grammar (TAG) by Payne and
Green Model of content knowledge rather than a full
system to generate user performance estimation. However, we can measure by number of rules.
Extending GOMS: Grammars TAG Example for EMACS:
Task[Direction, Unit] Symbol[Direction] + Letter[unit] Symbol[forward] “cntl” Symbol[backward] “meta” Letter[word] “W” Letter[character] “C”
Task: Move one word forward. Task[forward, word]
Symbol[forward] + Letter[word] “cntl” + “W” “cntl-W”
Extending GOMS: Production Systems Like grammars but models a goal stack and
working memory. Tedious to write but can be fed into a program
to automatically check for completeness and accuracy.
Can predict errors and learning time behavior.
Extending GOMS: Production Systems Production to see if a closing
JOIN statement is needed:
Rule 1: (StartUp.SeeifJoinNeededIF ((GOAL SeeIfJoinNeeded)
(NOT(NOTE SeeingIfJoinNeeded TRUE))THEN ((Add NOTE SeeingIfJoinNeeded TRUE)
(Add STEP CountTables)))
Rule 2: (CountTables((DoTask Count NumberOfTables *NumberOfTables)(Add NOTE NumberOfTables *NumberOfTables)(Delete STEP CountTables)(Add Step AddJoinNote)))
Insert intoWorking Memory
Delete from Working Memory
Extending GOMS: Learning How to estimate time to learn? One solution: Soar (UMICH)
From the FAQ: “Soar has also been used for modeling learning in many of these tasks; however, learning adds significant complexity to the structuring of the task…”
Extending GOMS:Natural GOMS Language (NGOMSL) Structured natural language notation Based directly on the Cognitive Complexity T
heory (Kieras and Polson) Allows GOMS to model working memory (WM) a
nd setup subgoals Unlike CMN-GOMS, provides quantitative predicti
ons about time to learn each new piece of a task.
Description
Method for goal: Move text
Step 1. Accomplish goal: Cut text
Step 2. Accomplish goal: Paste text
Step 3: Return with goal accomplished
Method for goal: Cut text
Step 1. Accomplish goal: Highlight text
Step 2. Retain that the command is CUT, and accomplish goal: Issue a command
Step 3: Return with goal accomplished
Method for goal: Paste text
Step 1. Accomplish goal: Position goal at insertion point
Step 2. Retain that the command is PASTE, and accomplish goal: Issue a command
Step 3: Return with goal accomplished
Selection rule set for goal: Highlight text
If text-is word, then accomplish the goal: Highlight word
If text-is arbitrary, then accomplish goal: Highlight arbitrary text
Return with goal accomplished
Method for goal: Highlight word
Step 1. Determine position of middle of word
Step 2. Move cursor to middle of word
Step 3. Double-click mouse button
Step 4. Verify that correct text is selected
Step 5. Return with goal accomplished
Description Duration (sec)
Method for goal: Highlight arbitrary text
Step 1. Determine position of beginning of text
1.20
Step 2: Move cursor to beginning of text 1.10
Step 3: Click mouse button 0.20
Step 4: Determine position of end of text. 0.00
Step 5. Move cursor to end of text 1.10
Step 6. Shift-click mouse button 0.48
Step 7. Verify that correct text is highlighted 1.20
Step 8: Return with goal accomplished
Method for goal: Position cursor at insertion text
Step 1. Determine position of insertion point 1.20
Step 2. Move cursor to insertion point 1.10
Step 3. Click mouse button 0.20
Step 4. Verify that correct point is flashing 1.20
Step 5. Return with goal accomplished
Method for goal: Issue a command
Step 1. Recall command name and retrieve from LTM the menu name for it, and retain the menu name
Step 2. Recall the menu name, and move cursor to it on Menu bar
1.10
Step 3: Press mouse button down 0.10
Step 4: Recall command name, and move cursor to it
1.10
Step 4: Recall command name, and verify that it is selected
1.20
Step 5: Release mouse button 0.10
Step 6: Forget menu name, forget command name and return with goal accomplished
TOTAL PREDICTED TIMETOTAL PREDICTED TIME 16.3816.38
Extending GOMS: Parallel Processes Cognitive processes are not always sequential
Clerks imprinting checks often realize an error two checks past
When typing, you often realize an error while typing the next sentence or letters
Extending GOMS: Cognitive-Perceptual-Motor (CPM-GOMS) Predicts a substantially shorter execution time
than the other models. Allocates less time for “prepare for action” type o
perations. Allow parallel processes.
Requires analyst-supplied methods. Uses Critical Path Analysis to investigate
parallel processes
Extending GOMS: Cognitive-Perceptual-Motor (CPM-GOMS) Collect-call example1, operator hits a “collect-call”
key and says “Thank you” to customer:
You can save time by repositioning the key for faster access in the sequential example, but not in the parallel example.
1Courtesy of Newman, Lemming’s TAO (Toll & Assistance Operator) study
Extending GOMS: Cognitive-Perceptual-Motor (CPM-GOMS) Critical Path: a connected sequence that represents the
greatest total time and therefore determines the overall time for a task.
Critical Path1 below is 400 + 280 + 2000 + 280 = 2.96 seconds
1Courtesy of Newman, Lemming’s TAO (Toll & Assistance Operator) study
GOMS Family: SummaryKLM CMN NGOMSL CPM
Architectural BasisSimple Cognitive Architecture
Model Human Processor
Cognitive Complexity Theory
Model Human Processor, assume expertise in use
Goal Hierarchy Implicit Explicit Implicit Implicit
Models Learning/Transfer No No Yes No
Models Parallel Processes No No No Yes
Assigned Mental TimeYes, use operator M
No YesYes, but very short for expert users
Notation UsedPrimitive Operators
Programming Language
Natural Language Schedule/PERT chart
Outline Cognitive Modeling Introduction to GOMS GOMS Extensions Modeling Specific Components GOMS Limitations Summary
Goals
Intention Evaluationexpectation
Execution
Mental Activity
Physical Activity
Perception
Interpretation
7 steps1 of user activities involved
in computer-based tasks
Action Specification
1Norman, D. (1986)
Action Specification
Goals
Intention Evaluation
MEMORY:Retrieve a unit from long term memory
expectation
COGNITION:Execute a mental step
Choose among methods
Mental Activity
Memory & Cognition: Memory Retrieval GOMS provides
modeling of Memory Retrieval Time to retrieve next
unit of information Moving information
from long-term memory into working memory
1
2
≈ 1350 msec
@MAX(D2…D12)
Memory & Cognition: Execution of a Mental Step GOMS allows explicit representation of mental
steps of a task (the “Cognitive Processor”):
Retrieval of goal
Find the max value in a column
Select a method to achieve the goal
Retrieval of motor movements necessary
to execute the command Execution of each of the chosen commands
Use the “MAX” formula
Type the formula
≈ 70 msec
Memory & Cognition: Method Decision Hick’s Law
T = k log2(n+1), k ~ 150 msecn = # of choices
Time to make a decision is roughly proportional to the log of the number of choices
However, determining T is problematic Spreadsheet task where parameters in a formula are can b
e indicated via numerous methods Hick’s law predicts 200 msec Real time is 2 seconds → Order of magnitude difference!
Memory & Cognition: Method Decision Choice is a complex task that requires many
cognitive steps Steps differ task to task
Action Specification
Goals
Intention Evaluation
MEMORY:Retrieve a unit from long term memory
expectation
COGNITION:Execute a mental step
Choose among methods
Execution
Mental Activity
Physical ActivityMOTOR MOVEMENTS:KeystrokePoint Move hands
Motor: Key Input Parameters of keyboard input based on
Skill of the typist Best Typist (120 wpm): 80 msec Worst Typist: 1200 msec
Predictability & continuity of the text to be typed Typing random letters: 500 msec
Motor: Mouse Movement Fitts’s Law is a robu
st predictor of mouse movement
Sometimes distance metric is not clear-cut Nested menus
Motor: Applying Fitts’s Law Fitts’s law recommends
Larger target sizes Smaller distances to targets Usage of corners and edges (they have “infinit
e” height and width) Macintosh menus are faster than Windows/Unix style
menus because they lie on the screen edge
Motor: Applying Fitts’s Law
Target size grows as distance from cursor’s position increases
Borders for shorter selectiontime
Fittsized Menus
Motor: Fisheye Model Provide local context
against a global context Focuses on screen space
versus user’s attention 3 properties
Focal point Distance from focus, D Level of detail, LOD
Degree of Interest Function to determine
whether to display an item or not and its size
Motor: Fisheye Menu Good for browsing task
s Allows one to present e
ntire menu without having to use hierarchies or scrolling
Longer learning curve http://www.cs.umd.edu
/hcil/fisheyemenu/fisheyemenu-demo.shtml
Motor: Hand Movements Switching between keyboard and mouse
≈ 360 msec Differences in times due to distance from hom
e position on keyboard and the size of the targets Joystick ≈ 260 msec Arrow keys ≈ 210 msec
Action Specification
Goals
Intention Evaluation
MEMORY:Retrieve a unit from long term memory
expectation
COGNITION:Execute a mental step
Choose among methods
Execution
Mental Activity
Physical ActivityMOTOR MOVEMENTS:KeystrokePoint Move hands
Perception
Interpretation PERCEPTION:PerceiveSaccade
Perception Recognition or perception
Measure the time to respond to stimuli Responding to lights Recognizing words
Saccade: fast movement of eye, head, etc. Measure the time to move and take in information
in each jump Eye jerking around, scanning or moving to the next
location
Perception An example: spreadsheet perception
Looking for cell addresses and retrieving data
230 msec130 msec, store row label
230 msec130 msec, store col label
230 msec1350 msec, retrieve row & col label
Total: 2300 msec
Summary of Cognitive ParametersRetrieve from memory 1200 msec
Execute a mental step 70 msec
Choose among methods 1250 msec
Enter a keystroke 230 msec
Point with a mouse 1500 msec
Move hands to mouse 360 msec
Perceive 100 msec
Make a saccade 230 msec
Outline Cognitive Modeling Introduction to GOMS GOMS Extensions Modeling Specific Components GOMS Limitations Summary
GOMS Limitations: User Skill Level Nonskilled or casual users
Current GOMS is best applied towards skilled users.
Transfer and training from one system to another. Example: Transfer GUI operational skills betwee
n MacOS, Windows and KDE.
GOMS Limitations: Errors Even skilled performers make mistakes Errors are probably caused by an overload of
Working Memory (WM) or the goal stack. The higher the WM load, the more errors.
How to model users adapting to errors arising from interface design? Users write down critical cell in a spreadsheet so that it can
be scanned quickly the next time.
Lotus IFPS
WM Load 19 14
Error (%) 14 6
GOMS Limitations: Parallel Processes Must be careful of over simplifying assumptio
ns: “A character on the same hand cannot be initia
ted with a cognitive operator until the motor processor execution of the previous character is complete” – John (1998) Psych literature doesn’t support the above!
Critical path method requires fine grained characterizations of task dependencies and parameters.
GOMS Limitations: Cognitive Processes Cognitive processes are unclear
Gestalt principles exist, but many more factors exist which influence user’s interpretation of screen
GOMS Limitations: Other Problems Usability Functionality Fatigue Mental Workload Individual differences
Some Work Has Addressed This
Topic
Straightforward Extension Seems
Possible
Cognitive Science Does Not Inform Us
Requires Another Kind of
Modeling
Nonskilled users − X − −
Learning X X − −
Errors X X − −
Cognitive Processes
X X X −
Parallel Processes
X X X −
Mental Workload − X − −
Functionality − − − X
Fatigue − − X −
Individual Differences
X − − −
Acceptance − − − X
Fit to organizational
life
− − − X
Outline Cognitive Modeling Introduction to GOMS GOMS Extensions GOMS Limitations Modeling Specific Components Summary
Summary In HCI, GOMS is, by far, the most detail oriented modeling
method for user activities. Great for getting quantitative and qualitative metrics. Easy to explain results. Once constructed, easy to modify in later design iterations.
Readily available tools are scarce, those that do exist have a high learning curve (e.g. CPM-GOMS)
Not as easy as heuristic analysis, walkthroughs or guidelines. Only works for goal-directed tasks. However, GOMS has been highly successful in applications
where human interaction performance is of utmost importance.
Summary Simulations of airplanes and helicopters in simulated
theatres of war (STOWs) with SOARS Sun’s webpage, CAD, word processors, mobile phone
input methods, etc. Project Ernestine: Adding new, “improved” workstati
ons for Telephone Operators CPM-GOMS revealed that the new workstations w
ould have cost an additional $2 million a year to operate!
Summary GOMS model is a predictive, descriptive and
prescriptive model1. Predicts the time it will take user to perform a
task
2. Describes the way a user performs tasks on a system
3. Prescribes ways to develop of training programs and help systems