MicrosoftMicrosoft®® Research Research Faculty Summit 2002Faculty Summit 2002

Building Scalable User Interfaces for Building Scalable User Interfaces for Mobile Devices using .NET TechnologiesMobile Devices using .NET Technologies

Benjamin B. BedersonBenjamin B. BedersonComputer Science DepartmentComputer Science DepartmentHuman-Computer Interaction LabHuman-Computer Interaction LabUniversity of MarylandUniversity of Maryland

Mary CzerwinskiMary CzerwinskiLarge Display User Experience GroupLarge Display User Experience GroupMicrosoft CorporationMicrosoft Corporation

Scaling Information, Scaling Information, Scaling DevicesScaling DevicesWe always want more: information, We always want more: information,

pixels, speed, … pixels, speed, … Whether big or small, the problems are Whether big or small, the problems are

similar:similar: Detect patterns and outliersDetect patterns and outliers Find details without losing global contextFind details without losing global context Concentrate on task (stay “in the flow”)Concentrate on task (stay “in the flow”)

Information Visualization

Scaling UpScaling Up

How to scale up to large information How to scale up to large information sets?sets? Technical problemsTechnical problems Perceptual limitationsPerceptual limitations Design problemsDesign problems

Solution in just three easy steps:Solution in just three easy steps:

External CognitionExternal Cognition

Step 1: Recognize Step 1: Recognize human limitationshuman limitations

Human Visual PerceptionHuman Visual Perception

Step 2: Don’t underestimate the Step 2: Don’t underestimate the human brainhuman brain

InteractionInteraction

Step 3: Add interaction. If a picture is Step 3: Add interaction. If a picture is worth a thousand words, an interactive worth a thousand words, an interactive interface is worth a thousand pictures.interface is worth a thousand pictures.

Scaling UpScaling Up

How do you show more than fits How do you show more than fits on the screen?on the screen? abstractabstract link (web)link (web) scroll (long documents)scroll (long documents) overview+detail (e.g., photoshop)overview+detail (e.g., photoshop) zoomzoom fisheye distortionfisheye distortion

traditional

new paradigms

Menus, menus, menusMenus, menus, menus

• Problem: Selection from a long listProblem: Selection from a long list• Growing importance with data-driven Growing importance with data-driven

applicationsapplications• Traditional approaches:Traditional approaches:

• ArrowBarsArrowBars• ScrollBarsScrollBars• HierarchiesHierarchies

Fisheye MenusFisheye Menus

• Apply fisheye distortion to linear listApply fisheye distortion to linear list• Shows detail in contextShows detail in context• Reduces mouse Reduces mouse presses / tapspresses / taps

• Applet written in JavaApplet written in Java• Pocket PC version written in embedded Pocket PC version written in embedded

C++C++

Demo

Fisheye MenusFisheye Menus

Fisheye Menu EvaluationFisheye Menu Evaluation

Fisheye Menu v. Fisheye Menu v. Microsoft Pocket PC 2000™ Microsoft Pocket PC 2000™

Goal:Goal: Understand scalability of fisheye menusUnderstand scalability of fisheye menus

Used real applications and iconsUsed real applications and icons

MethodsMethods 22 participants22 participants Three independent variablesThree independent variables

Menu type (fisheye vs. regular)Menu type (fisheye vs. regular) Menu length (20, 30, 40, 50)Menu length (20, 30, 40, 50) Position of target within menuPosition of target within menu

(first, second, or last third) (first, second, or last third)

Measured:Measured: TimeTime AccuracyAccuracy

Total of 1230 trialsTotal of 1230 trials

Results —Task TimesResults —Task Times

Tasks were performed faster using Tasks were performed faster using Fisheye Menus, F(1,1206)=29.4, Fisheye Menus, F(1,1206)=29.4, p<0.001p<0.001 25% faster (4.0 vs 5.3 secs)25% faster (4.0 vs 5.3 secs)

Difference more pronounced for longer Difference more pronounced for longer menusmenus

And more pronounced And more pronounced for items near the end for items near the end of the menusof the menus

FISHEYE START

menutype

Dot/Lines show Means

Time vs. Menu Length

20 30 40 50

menuleng

0.00

1000.00

2000.00

3000.00

4000.00

5000.00

6000.00

7000.00

tim

e

Results – Task AccuracyResults – Task Accuracy

But, tasks were completed successfully But, tasks were completed successfully significantly more often using traditional menu significantly more often using traditional menu (97% v. 92%), F(1,1206)=22, p<0.001(97% v. 92%), F(1,1206)=22, p<0.001

Difference more pronounced for items in Difference more pronounced for items in middle of menumiddle of menu

Results – User PreferenceResults – User Preference

StartStart FisheyeFisheye

Prefer for short:Prefer for short: 82% 18% p=.04

Prefer for long:Prefer for long: 14% 86% p<.001

Perceived faster:Perceived faster: 14% 86% p<.001

Perceived easier:Perceived easier: 68% 32%p=.005

DiscussionDiscussion

Cool idea, but not a clear winnerCool idea, but not a clear winner Some older users complained that it Some older users complained that it

made them feel bad about their eyesightmade them feel bad about their eyesight Other users wanted it immediately for Other users wanted it immediately for

all of their menus…all of their menus… Interaction simpler on PDA because of Interaction simpler on PDA because of

Pen inputPen input Bottom line: probably a good idea to Bottom line: probably a good idea to

offer as an optionoffer as an option

Many tools support image management, Many tools support image management, annotation, and search.annotation, and search.

Our goal is to focus on Our goal is to focus on browsingbrowsing avoid scrollbars, menus, and window avoid scrollbars, menus, and window

mgmtmgmt work in a home setting for family usework in a home setting for family use support co-present collaborative usesupport co-present collaborative use support serendipitous photo findingsupport serendipitous photo finding

Photo BrowsingPhoto Browsing

Motivated by my personal use of other Motivated by my personal use of other tools with my 3-year-old daughtertools with my 3-year-old daughter

Important that: Important that: she can direct useshe can direct use follow flow as I control computerfollow flow as I control computer interface feels comfortableinterface feels comfortable eliminate advance organizationeliminate advance organization

Design MotivationDesign Motivation

Design:Design: Zoomable User InterfaceZoomable User Interface

Simple navigation interactionSimple navigation interaction

Quantum Strip Treemap layoutQuantum Strip Treemap layout Shows multiple directories of imagesShows multiple directories of images Can group by metadataCan group by metadata

PhotoMesaPhotoMesa

Variation on treemaps: Space-Variation on treemaps: Space-filling subdivision of an input filling subdivision of an input rectanglerectangle

Guarantees that each Guarantees that each rectangle’s dimensions are an rectangle’s dimensions are an integral multiple of a constantintegral multiple of a constant

Lays out images so that all Lays out images so that all images are the same size and images are the same size and are aligned on a single grid are aligned on a single grid across rectanglesacross rectangles

Quantum TreemapsQuantum Treemaps

Strip TreemapsStrip Treemaps

Squarified treemaps: avg aspect ratio: 1.75Squarified treemaps: avg aspect ratio: 1.75Strip treemaps: avg aspect ratio: 2.6Strip treemaps: avg aspect ratio: 2.6

User study showed that “readability” was 83% User study showed that “readability” was 83% faster for strip than squarified on search task w/ faster for strip than squarified on search task w/ 100 rects (2.5 sec vs 14.8 sec w/ 20 subjs).100 rects (2.5 sec vs 14.8 sec w/ 20 subjs).

Squarified => <= Strip

DemoWritten in embedded C++

Calendar ManagementCalendar Management

How to better:How to better: support support planningplanning and and analysisanalysis tasks tasks move between multiple device typesmove between multiple device types

The answer:The answer: more Information Visualization techniquesmore Information Visualization techniques

DateLensDateLens

To scale up and maintain context:To scale up and maintain context: Uses 2D fisheye distortionUses 2D fisheye distortion Carefully designed interactionCarefully designed interaction

simple or manual control over spacesimple or manual control over space

Integrated search with or without Integrated search with or without text entrytext entry

Written in C# with same code Written in C# with same code running on Tablet PC and desktoprunning on Tablet PC and desktop

Demo

DateLens PerformanceDateLens Performance

Careful codingCareful coding Custom renderer and pickerCustom renderer and picker Space-time trade-offSpace-time trade-off Minimal rendering during transitionsMinimal rendering during transitions Careful evaluation of how to renderCareful evaluation of how to render

DateLens Benchmark StudyDateLens Benchmark Study

DateLens v. DateLens v. Microsoft Pocket PC 2002™ Microsoft Pocket PC 2002™

GoalsGoals 11stst iteration of UI with potential users iteration of UI with potential users to compare its overall usability against an to compare its overall usability against an

existing productexisting product

Used Mary’s calendar, seeded with Used Mary’s calendar, seeded with artificial calendar eventsartificial calendar events

MethodsMethods 11 knowledge workers (5 F) 11 knowledge workers (5 F)

All experienced PC (not PDA) usersAll experienced PC (not PDA) users

11 isomorphic browsing tasks on each 11 isomorphic browsing tasks on each calendarcalendar All conditions counterbalancedAll conditions counterbalanced All tasks had deadline of 2 minutes All tasks had deadline of 2 minutes

Measured:Measured: task timestask times success ratesuccess rate verbal protocolsverbal protocols user satisfaction and preferenceuser satisfaction and preference

Results —Task TimesResults —Task Times

Tasks were performed faster using Tasks were performed faster using DateLens, F(1,8)=3.5, p=.08DateLens, F(1,8)=3.5, p=.08 Avg=49 v. 55.8 sec’s for the Pocket PCAvg=49 v. 55.8 sec’s for the Pocket PC Complex tasks significantly harder, p<.01, Complex tasks significantly harder, p<.01,

but handled reliably better by DateLens but handled reliably better by DateLens (task x calendar interaction), p=.04 (task x calendar interaction), p=.04

Task SuccessTask Success

Tasks were completed successfully Tasks were completed successfully significantly more often using DateLens significantly more often using DateLens (88.2% v. 76.3%), p<.001. (88.2% v. 76.3%), p<.001.

In addition, there was a significant main In addition, there was a significant main effect of task, p<.001. effect of task, p<.001.

For the most difficult task (#11), no For the most difficult task (#11), no participant using the Pocket PC completed participant using the Pocket PC completed the task successfully. the task successfully.

Usability IssuesUsability Issues

Many users disliked the view when Many users disliked the view when more than 6 months were shownmore than 6 months were shown

Concerns about the readability of text, Concerns about the readability of text, needs to be customizableneeds to be customizable

Wanted more control about how weeks Wanted more control about how weeks were viewed (e.g., start with Sunday or were viewed (e.g., start with Sunday or Monday?)Monday?)

Needed better visual indicators of Needed better visual indicators of conflicts for both calendars, e.g., red conflicts for both calendars, e.g., red highlights and/or a “conflicts” filterhighlights and/or a “conflicts” filter

DiscussionDiscussion

DateLens performed well despite its novelty DateLens performed well despite its novelty and its first iteration of user testing and its first iteration of user testing

DateLens Positive Features:DateLens Positive Features: Responsivity to direct user manipulationResponsivity to direct user manipulation Ability to create custom views easilyAbility to create custom views easily Clearer presentation of conflictsClearer presentation of conflicts Integrated search utility Integrated search utility

The PPC calendar was seen to be consistent The PPC calendar was seen to be consistent with other MS calendar products (good)with other MS calendar products (good)

A combination of the 2 would be a great final A combination of the 2 would be a great final productproduct

© 2002 Microsoft Corporation. All rights reserved.© 2002 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.