björn hartmann loren yu, abel allison, yeonsoo yang scott r. klemmer
DESCRIPTION
Design as Exploration Creating Interface Alternatives through Parallel Authoring and Runtime Tuning. Björn Hartmann Loren Yu, Abel Allison, Yeonsoo Yang Scott R. Klemmer. Exploration of alternatives is central to design. MAP THE DESIGN SPACE. Prototypes for the Microsoft mouse - PowerPoint PPT PresentationTRANSCRIPT
stanford hci group oct 21, 2008
Björn HartmannLoren Yu, Abel Allison, Yeonsoo
YangScott R. Klemmer
Design as ExplorationCreating Interface Alternatives through Parallel Authoring and Runtime Tuning
2
Exploration of alternatives is central to design.
3
Prototypes for the Microsoft mouseFrom Moggridge, Designing Interactions, Ch2
MAP THEDESIGN SPACE
4From Design Secrets: Products 2
CommunicatE
5
Tohidi et al, CHI 2006
TEST
6
How can tools support the creation of user interface alternatives?
Color picker #1
Color picker #2 Color picker #3
7
“[D]esigners frequently wanted to have multiple designs side-by-side [...] However […] there is no built-in way in today’s implementation tools to have two versions of a behavior operating side-by-side.”
Myers et al., VL/HCC 2008
Adobe Flash Arduino / Processing
Adobe Dreamweaver Xcode (iPhone)
Interaction Designers Write Code
8
9
Programming requires working with two representations.
Editor: author behavior
… …
………
Runtime: observe behavior
10
Outline Requirements for Alternatives of
Programmed Interactions Juxtapose for Desktop Interactions:
System & Evaluation Alternatives off the Desktop Related & Future Work
11
3 Requirements for Programmed Interactions
Manage parameter variations Manage code alternatives Access variations & alternatives at
runtime
(based on 3 interviews & code inspection)
12
Manage parameter variations Manage code alternatives Access variations & alternatives at
runtime
Processing code brought in by an interviewee
3 Requirements for Programmed Interactions
13
Manage parameter variations Manage code alternatives Access variations & alternatives at
runtime
3 Requirements for Programmed Interactions
14
Juxtapose: Source alternatives…
15
Juxtapose: Source alternatives…
16
… are executed in parallel,
17
and tuned through an generated UI.
18
Parallel Editing
Juxtapose editor forActionScript 2.Generates Flash files.
19
Parallel Editing
20
Parallel Editing
Linked Editing: Toomim 2004
21
Parallel Editing
Linked Editing: Toomim 2004
22
Example: Rethinking the Progress Bar Harrison et al., UIST 2007
Progress profilesProgress bar test application
23
24
Parallel Input: Event Echoing
25
Implementing Parallel Editing: The Cursor Correspondence Problem
Straightforward solution:Longest common subsequence algorithmdiff(A,B,…,N) Run during idle times
Doc. A Doc. B Doc. C
26
Limits of Parallel Editing & Execution
Level of granularity for editing:Extension to multi-file projects is not trivial.
Can the user make sense of behaviors running in parallel?
27
Limits of Event Echoing
Yes No
Delete this file?
Ok
Delete this file?
Yes No
28
Parameter Tuning
29
Example: Tuning Phosphor Baudisch, UIST 2006
30
Example: Tuning Phosphor Baudisch, UIST 2006
31
Generating Tuning Interfaces
Juxtapose User Library
User’s Application://…
Number A = 10;Boolean B = true;
User ProcessJuxtapose Process
Inspect
Send variable info
X B
A
32
Generating Tuning Interfaces
Juxtapose User Library
User’s Application://…
Number A = 10;Boolean B = true;
User ProcessJuxtapose Process
Update
Send tuning message
X B
A
33
What parameter ranges should be chosen?0.0-1.0?
0-100?0-255?Number.minValue –
Number.maxValue?
Any nontrivial program property is undecidable.
34
Take a guess, then give control to the user.
At authoring time: source annotations
35
Limits of Tuning Which variables matter?
Juxtapose extracts only globals
36
Limits of Tuning Which variables matter?
Juxtapose extracts only globals Are variables read again?
Juxtapose uses callbacks
01 var tunable = 5; //@RANGE 0..100 03 var callback= function(varName,oldVal,newVal){ 04 redraw(); 05 return newVal; 06 } 07 this.watch(′tunable′,callback);
Understanding User Strategies & Measuring Benefits Questions:
How does Juxtapose fit into interaction prototyping practice?
Can we quantify the benefits? Method:
Lab evaluation, N=18, students 75-90 minute sessions with design
tasks & survey
37
38
Mapping TaskGiven ActionScript code that loads a
multilayered map, develop navigation options for a handheld GPS prototype for pedestrians and bicyclists.
39
Alternatives for MapNavigation & Rendering
40
Participants’ Strategies Linked vs. unlinked editing with
multiple alternatives applied by all participants.
Alternatives used both for different scenarios, as well as “scratch space”
Snapshots important to save state
41
Suggested improvements Automate code restructuring for
callbacks Introduce direct manipulation for
parameters in user’s application (when possible)
42
Tree Matching TaskSearch in 4D parameter space. Given recursive drawing code for this:
Produce these:
43
44
0
10
20
30
40
50
60
70
2.60
64.26Mean Parameter Changes
Tested per Minute
ControlJuxtapose
Chan
ges/
min
ute
45
Tree 1 Tree 2 Tree 3 Tree 40
50100150200250300350
Tree Matching Task:Mean Completion Times by Tree
ControlJuxtapose
seco
nds
p<0.01 p~0.01not significant
not significant
46
Outline Requirements for Alternatives of
Programmed Interactions Juxtapose for Desktop Interactions:
System & Evaluation Alternatives off the Desktop Related & Future Work
47
Juxtapose Mobile
Alternative 1
Alternative 2
Alternative3
48
49
JuxtaposeRuntime on PC
Juxtapose Mobile
Juxtapose WrapperUser’s FlashApplication
Juxtapose WrapperUser’s FlashApplication
TCP/IPvar1
Alt. 1 Alt. 2
var2
50
Juxtapose for Microcontrollers
Arduino:8bit Atmel AVR RISC chip
programmed in C with gcc
51
How might one implement variable tuning in a language without reflection?
52
Parse code and build symbol table
Var. Name Type Pointer“dly” int &dly“blink” boolean &blink
int dly = 25;boolean blink=false;void setup() {...}//…
53
Emit table into code & compile
void initVarTable(void) { varTable[1].varName = PSTR("dly"); varTable[1].varType = VAR_TYPE_INT; varTable[1].varPtr = &dly;
varTable[2].varName = PSTR("blink"); varTable[2].varType = VAR_TYPE_BOOLEAN; varTable[2].varPtr = &blink;}
// plus a bunch of communication code...
Auto-generatedtable
54
At Runtime
Serial message:
tune “dly” value 100
*(varTable[“dly”].varPtr) = 100;X
dly blink
55
So What’s Different?Target Platform How are
alternatives executed? Why?
How do users interact with alternatives?
In parallel Sequentially or in parallel
In parallel, because commodity hardware is (relatively) cheap
Sequentially(parallelism is possible, but less appealing)
Sequentially, because custom hardware is expensive to build
Sequentially
56
What’s Important?A structured approach to alternatives eliminates cruft and enables more exploration.
57
What’s Important?A structured approach to alternatives eliminates cruft and enables more exploration.Tool support requires connecting authoring and execution environments.3 Techniques:
Linked editing to manage code alternatives
Execute set of programs side-by-side Auto-generate tuning interface
58
Related Work
Design Galleries, Marks, SIGGRAPH 97Spreadsheets for Images, Levoy, SIGGRAPH 94Spreadsheets for Visualization, Chi, IEEE CGA 98Parameter Spectrums & Parallel Pies, Terry, UIST 02 & CHI 04Subjunctive Interfaces, Lunzer, TOCHI 08TEAM STORM, Hailpern, CHI 07
(only first authors listed)
Partials, Terry, PhD Thesis 05Amulet Inspector, Myers, IEEE ToSE 97JPie, Goldman,Sci. of Comp. Prog. 98ChucK, Wang, NIME 04 Adobe Image Foundation Tookit, 08
Alternatives in Other DomainsAugmented Development Tools
59
Related Work
Language-level support
Partials, Terry, PhD Thesis 05Amulet Inspector, Myers, IEEE ToSE 97JPie, Goldman,Sci. of Comp. Prog. 98ChucK, Wang, NIME 04 Adobe Image Foundation Tookit, 08
Augmented Development Tools
60
Related Work
Alternatives are generated automatically from formal specification
Design Galleries, Marks, SIGGRAPH 97Spreadsheets for Images, Levoy, SIGGRAPH 94Spreadsheets for Visualization, Chi, IEEE CGA 98Parameter Spectrums & Parallel Pies, Terry, UIST 02 & CHI 04Subjunctive Interfaces, Lunzer, TOCHI 08TEAM STORM, Hailpern, CHI 07
Alternatives in Other Domains
61
Related Work
Exploit spatial syntax
Design Galleries, Marks, SIGGRAPH 97Spreadsheets for Images, Levoy, SIGGRAPH 94Spreadsheets for Visualization, Chi, IEEE CGA 98Parameter Spectrums & Parallel Pies, Terry, UIST 02 & CHI 04Subjunctive Interfaces, Lunzer, TOCHI 08TEAM STORM, Hailpern, CHI 07
Alternatives in Other Domains
62
Current Work: Revisions for Interaction Design
Revisions & commentsfor text in Microsoft Word
Revisions & comments for interactionsin d.tools
stanford hci group oct 21, 2008
http://hci.stanford.edu
Björn HartmannLoren Yu, Abel
Allison, Yeonsoo Yang
Scott R. Klemmer
Supported by:NSF grant IIS- 0745320Equipment from Intel & NokiaSAP Stanford Graduate Fellowship
64
At runtime: editable min/max values
Take a guess, then give control to the user.
65
Implementing Parallel Editing: Incremental Structure Tracking
Doc. A
User inserts
textEmpty blockcreated
Blockssub-divided
User deletes
text
Doc. B
Doc. A Doc. B
Doc. A Doc. B
Doc. A Doc. B
INSERT DELETE
66
Future Work
Graphics + Code?
67
68
FRAME DEMONSTRATE UNDERSTAND
69
FRAME DEMONSTRATE UNDERSTAND
70
FRAME DEMONSTRATE UNDERSTAND
71
Tangible Control
72
Switching alternatives
Load
Alternative2.hex
Alternative 2
Bootloader
73
0
50
100
150
200
250
300257.81
161.81
Tree Matching Task:Mean Completion Times
ControlJuxtapose
seco
nds
p<0.001 (paired two-tailed Student’s t)
Buxton, Sketching User Experiences
75
How can interaction design toolssupport creation and management of user interface alternatives?
76
JuxtaposeRuntime on PC
Juxtapose Mobile
Juxtapose WrapperUser’s FlashApplication
Open socket, Send variable info
Juxtapose WrapperUser’s FlashApplication
TCP/IP
Open socket, send variable info
var1
Alt. 1 Alt. 2
var2
77
Juxtapose Mobile
Juxtapose WrapperUser’s FlashApplication
Tuning messagesFor Alternative 2
Juxtapose WrapperUser’s FlashApplication
Tuning messagesFor Alternative 1
TCP/IPJuxtaposeRuntime on PC
var1
Alt. 1 Alt. 2
var2
78
Adding Tuning to the Arduino IDE
79
Tuning without ReflectionParse source to extract global variable declarations
Emit variable table + wrapper code into source
Compile modified source
80
Related WorkDesign Galleries, Marks, SIGGRPAH 97Parameter Spectrums & Parallel Pies, Terry, UIST 02 & CHI 04Subjunctive Interfaces, Lunzer, TOCHI 08Spreadsheets for Images, Levoy, SIGGRAPH 94Spreadsheets for Visualization, Chi, IEEE CGA 98TEAM STORM, Hailpern, CHI 07Partials, Terry, PhD Thesis 05Amulet Inspector, Myers, IEEE ToSE 97Jpie, Goldman,Sci. of Comp. Prog. 98ChucK, Wang, NIME 04 Adobe Image Foundation Tookit, 08(only first authors listed)
Mixed initiative
Redrawn fromBuxton, Sketching User Experiences
Generate Options, Select. Repeat.
82
Limits of Parallel Editing Cognitive challenge:
Changes to alternatives are not visible during editing
Level of granularity for editing:Extension to multi-file projects not trivial
83
Beyond the DesktopMOBILE PHONES MICRO
CONTROLLERS