novel visualization and interaction for small to jumbo displays mary czerwinski microsoft research...
TRANSCRIPT
Novel Visualization and Interaction for Small to
Jumbo Displays
Novel Visualization and Interaction for Small to
Jumbo Displays
Mary CzerwinskiMicrosoft Research
Mary CzerwinskiMicrosoft Research
Who Contributed: VIBE TeamWho Contributed: VIBE Team
Large Display Surfaces Are HereLarge Display Surfaces Are Here
Workstation in the Workstation in the realreal world world
OverviewOverview
Initial large display researchPrototypes around usability issues observed
Visualization and interaction New user experiences have to scale the wide continuum of displays
Future directions
Initial large display researchPrototypes around usability issues observed
Visualization and interaction New user experiences have to scale the wide continuum of displays
Future directions
Harris Poll Responses (7/02, N=1197)Harris Poll Responses (7/02, N=1197)
Mutiple PCs and Displays
0%
10%
20%
30%
40%
50%
60%
70%
80%
None Multiple monitorsattached to
multiplecomputers.
Laptop anddesktop monitor
connectedtogether.
Dualmon or higher
Config
Pe
rce
nt
Re
sp
on
da
nts
All
Mutiple PCs and Displays
0%
10%
20%
30%
40%
50%
60%
70%
80%
None Multiple monitorsattached to
multiplecomputers.
Laptop anddesktop monitor
connectedtogether.
Dualmon or higher
Config
Pe
rce
nt
Re
sp
on
da
nts
All
no multimonno multimon30%30%
plan to useplan to use multimonmultimon
38%38%
use multimonuse multimon32%32%
Peddie Peddie Research, Research, 2001 (N>6000)2001 (N>6000)
Multimon Usage TrendsMultimon Usage Trends
Why A Larger Display Surface?Why A Larger Display Surface?
Productivity benefits 10-30% (despite sw usability issues)Users prefer more display surface
Prices droppingFootprints getting smaller
Productivity benefits 10-30% (despite sw usability issues)Users prefer more display surface
Prices droppingFootprints getting smaller
Projected LCD Pricing 2002-2005
$437$378 $327 $283
$699$597
$510$436
$1,089
$905
$752$625
$0
$200
$400
$600
$800
$1,000
$1,200
2002 2003 2004 2005$
US
15" -13.5%
17" -14.6%
18" -16.9%
1st Prototype--dSharp Display1st Prototype--dSharp Display
Triple projectionmatrox parhelia card3028 x764 resolution42 in. acrossSlightly curved120 degree FOV
Triple projectionmatrox parhelia card3028 x764 resolution42 in. acrossSlightly curved120 degree FOV
Task Times – SignificantTask Times – Significant
Effects of Display Size on Task Times
0
20
40
60
80
100
120
140
160
DISPLAY
Ave
rage
Task
Tim
e (S
econ
ds)
Small
Large
Effects of Display Size on Task Times
0
20
40
60
80
100
120
140
160
DISPLAY
Ave
rage
Task
Tim
e (S
econ
ds)
Small
Large
User Satisfaction - SignificantUser Satisfaction - Significantthe tasks were easy to perform
0
1
2
3
4
5
Small Large
Display Size
Ave
rage
Rating
(1=D
isag
ree,
5=Agr
ee)
the tasks were easy to perform
0
1
2
3
4
5
Small Large
Display Size
Ave
rage
Rating
(1=D
isag
ree,
5=Agr
ee)
Windows Layout - SignificantWindows Layout - Significant
I was satisfied with the ease of windows layout
012345
Display Size
Ave
rage
Rat
ing
(1=D
isag
ree,
5=Agr
ee)
Small
Large
I was satisfied with the ease of windows layout
012345
Display Size
Ave
rage
Rat
ing
(1=D
isag
ree,
5=Agr
ee)
Small
Large
Tenets--Large Display UXTenets--Large Display UX
User studies show large display surfaces fundamentally change user interaction
Designed tools to better understand/complement how work practice changes
Large display surfaces provide non-linear productivity increases
Additional space has different utilitye.g. focal/peripheral displays provide different cues
User studies show large display surfaces fundamentally change user interaction
Designed tools to better understand/complement how work practice changes
Large display surfaces provide non-linear productivity increases
Additional space has different utilitye.g. focal/peripheral displays provide different cues
But…Usability IssuesBut…Usability Issues
Why click to bring a clearly visible window into focus? caused many errorsWhere is my cursor?Where is my start button?Where is my taskbar?Where are my dialogs?The software doesn’t know where the bezel is…
Why click to bring a clearly visible window into focus? caused many errorsWhere is my cursor?Where is my start button?Where is my taskbar?Where are my dialogs?The software doesn’t know where the bezel is…
VibeloggerVibelogger
1st activity repository for studying windows usage in aggregate
can profile users based on display sizecan be extended to visualize workflow and capture context
single user: capture task contexts to surface pertinent ui or provide reminders
1st activity repository for studying windows usage in aggregate
can profile users based on display sizecan be extended to visualize workflow and capture context
single user: capture task contexts to surface pertinent ui or provide reminders
Multitask VisualizationMultitask Visualization
colored block for each time point and appamount of shading indicates percentage of visibility of the window taskssubtasks
colored block for each time point and appamount of shading indicates percentage of visibility of the window taskssubtasks
Task Switching VisualizationTask Switching Visualization
switching tasks (red to blue)
how are email windows arranged and used?
compare to...
switching tasks (red to blue)
how are email windows arranged and used?
compare to...
Windows and Task Management Issues Emerge
Windows and Task Management Issues EmergeLarger displays = more open windows
Multimon users arrange windows spatially
Taskbar does not scale:
aggregation model not task-based
users can’t operate on groups of related windows
Larger displays = more open windows
Multimon users arrange windows spatially
Taskbar does not scale:
aggregation model not task-based
users can’t operate on groups of related windows
Relationship between # of Monitors and # of Windows Left Open
0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
16.00
18.00
No. of Monitors
Avg
. #
of
Win
do
ws
Lef
t O
pen
Single Monitor
DualMon
TripleMon
Relationship between # of Monitors and # of Windows Left Open
0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
16.00
18.00
No. of Monitors
Avg
. #
of
Win
do
ws
Lef
t O
pen
Single Monitor
DualMon
TripleMon
Changes in Window Access Patterns
Changes in Window Access Patterns
0
10
20
30
40
50
60
70
80
90
100
1 2 3
Number of Monitors
Per
cen
tag
e o
f A
cces
s T
ech
niq
ue
Win
Taskbar
Input: Drag-and-PopInput: Drag-and-Popproblem
large displays create long distance mouse movement
touch & pen input has problems moving between screen units
solution
drag-and-pop brings proxies of targets to the user from across display surfaces
the user can complete drag interactions locally—no need to deal with distances or to cross display borders
problem
large displays create long distance mouse movement
touch & pen input has problems moving between screen units
solution
drag-and-pop brings proxies of targets to the user from across display surfaces
the user can complete drag interactions locally—no need to deal with distances or to cross display borders
Table ClothTable ClothProblem:User wants to access content physically far away
Solution:Pan the desktop to user
Compress content to the right of focus
Grab content you need and snap back
Problem:User wants to access content physically far away
Solution:Pan the desktop to user
Compress content to the right of focus
Grab content you need and snap back
Multitasking SupportMultitasking Support
Projectbar, layoutbar, groupbarScalable fabricTask flasherSecret passwordsTable clothWincuts
Projectbar, layoutbar, groupbarScalable fabricTask flasherSecret passwordsTable clothWincuts
Task Management: GroupbarTask Management: Groupbar
Taskbar for lightweight grouping of windows
Allows for multiple bars, spatial placement of bars
Desktop snapshotting; task snapshots
Taskbar for lightweight grouping of windows
Allows for multiple bars, spatial placement of bars
Desktop snapshotting; task snapshots
Task management: Scalable Fabric
Task management: Scalable Fabric
Configurable central focus + periphery
Easy task switch from periphery to focus
Leverages human spatial memory
Configurable central focus + periphery
Easy task switch from periphery to focus
Leverages human spatial memory
Task FlasherTask Flasher
A more visual alt + tab
Uses 3d scaling and selection animation
Windows stay on the monitor on which they are positioned
A more visual alt + tab
Uses 3d scaling and selection animation
Windows stay on the monitor on which they are positioned
Secret PasswordsSecret Passwords
Problem: many touchscreen systems (eg. SmartBoard, TabletPC) have no keyboard
Use software virtual keyboard
Hard to hide password on a virtual keyboardPrevious research showed users think anything on large display is considered public…they watch (Tan et al, CHI ‘03)
Problem: many touchscreen systems (eg. SmartBoard, TabletPC) have no keyboard
Use software virtual keyboard
Hard to hide password on a virtual keyboardPrevious research showed users think anything on large display is considered public…they watch (Tan et al, CHI ‘03)
Solution: Secret PasswordsSolution: Secret Passwords
Meeting Support: WincutsMeeting Support: WincutsWhat about co-located collaborative work?
People bring different expertise and information on personal devices to meetings and need to share
today’s model is broken—only one person gets to display entire desktop at a time (for better or worse)
or, must share applications and files with otherswhat about private information?
What about UI that should be scaled for the task at hand?
What about co-located collaborative work?People bring different expertise and information on personal devices to meetings and need to share
today’s model is broken—only one person gets to display entire desktop at a time (for better or worse)
or, must share applications and files with otherswhat about private information?
What about UI that should be scaled for the task at hand?
Visualization ResearchVisualization Research
DateLens with Ben Bederson
Facetmap
Team Tracks (SW Vis)
DateLens with Ben Bederson
Facetmap
Team Tracks (SW Vis)
Datelens with Ben BedersonDatelens with Ben BedersonFisheye representation of dates
Compact overviews
User control
Integrated search
Integrated with outlook
Pen-enabled
Fisheye representation of dates
Compact overviews
User control
Integrated search
Integrated with outlook
Pen-enabled
FacetMapFacetMap
FacetMapFacetMap
Scalable Visualization for “all of your stuff”Queries MyLifeBits SQL database interactively
Domain includes web pages, digital images, documents, email, SenseCam physical activity data, VibeLog PC activity dataHeterogeneous data, metadata, annotations, and relationships organized into “facets”Browsing and searching accomplished by iterative selection from among available item attributes (and/or full text search)
Uses Piccolo.Net (on top of GDI+) for graphics & animation
Scalable Visualization for “all of your stuff”Queries MyLifeBits SQL database interactively
Domain includes web pages, digital images, documents, email, SenseCam physical activity data, VibeLog PC activity dataHeterogeneous data, metadata, annotations, and relationships organized into “facets”Browsing and searching accomplished by iterative selection from among available item attributes (and/or full text search)
Uses Piccolo.Net (on top of GDI+) for graphics & animation
ScalabilityScalability
Scales in three ways:Space: Recursive, space-filling algorithm generates useful views for any size screenItems: Large numbers of items aggregated/abstracted into groups and counts at multiple levelsFacets: New item attributes can be added to the visualization dynamically
Fixed minimum node size to guarantee readability
Consistent information densityGreater levels of detail are collapsed until they are feasible to present, given display constraints
Scales in three ways:Space: Recursive, space-filling algorithm generates useful views for any size screenItems: Large numbers of items aggregated/abstracted into groups and counts at multiple levelsFacets: New item attributes can be added to the visualization dynamically
Fixed minimum node size to guarantee readability
Consistent information densityGreater levels of detail are collapsed until they are feasible to present, given display constraints
Medium Size (No Filters)Medium Size (No Filters)
Medium Size (Filters Active)Medium Size (Filters Active)
Small SizeSmall Size
Large Size (Wall Display)Large Size (Wall Display)
FacetMap for Mobile Phones
FacetMap for Mobile Phones
Quick searching through structured data
Scalable UI
Half visual and half text list
Prototyping stage (desktop)
Multiple visualizations (pick the best for a given results set)
Quick searching through structured data
Scalable UI
Half visual and half text list
Prototyping stage (desktop)
Multiple visualizations (pick the best for a given results set)
SW Visualization: Team TracksSW Visualization: Team Tracks
Goal: help developers new to a code base familiarize themselves quickly
Based on logs of teams using the code base
Assumption: most frequently visited areas of the code are more important
We empirically verified this
Show developers most related areas of code based on where they currently are
Also give them previews of that code
Goal: help developers new to a code base familiarize themselves quickly
Based on logs of teams using the code base
Assumption: most frequently visited areas of the code are more important
We empirically verified this
Show developers most related areas of code based on where they currently are
Also give them previews of that code
Screenshot of Team TracksScreenshot of Team Tracks
Evaluation ResultsEvaluation Results
Better task completion rates9 / 9 completed tasks 1 and 2 (same)
3 / 9 completed task 3 (versus 1 / 7)dominated by algorithmic detail
7 / 9 completed task 4 (versus 1 / 7)dominated by finding relevant code fragments
Better quiz scores (t(16)=-2.04, p<.03, one-tailed)
Same importance ratings (r=0.45, p=.02)
Better task completion rates9 / 9 completed tasks 1 and 2 (same)
3 / 9 completed task 3 (versus 1 / 7)dominated by algorithmic detail
7 / 9 completed task 4 (versus 1 / 7)dominated by finding relevant code fragments
Better quiz scores (t(16)=-2.04, p<.03, one-tailed)
Same importance ratings (r=0.45, p=.02)
Future VIBE DirectionsFuture VIBE Directions
Novel interaction and visualization techniques that scale from small to very large displays
Continued evaluation and iteration of designs from a user-centered perspective
Automatic task identification
For more information: http://research.microsoft.com/research/vibe
Novel interaction and visualization techniques that scale from small to very large displays
Continued evaluation and iteration of designs from a user-centered perspective
Automatic task identification
For more information: http://research.microsoft.com/research/vibe
Thank You!
http://research.microsoft.com/research/vibe
Thank You!
http://research.microsoft.com/research/vibe