low-fidelity user testing is 485, professor matt thatcher

Low-Fidelity User Testing

IS 485, Professor Matt Thatcher

2

Agenda Administrivia High-fidelity vs. low-fidelity

prototyping Hand out CD of user testing from past

project

3

What is Fidelity? High-fidelity (Hi-fi)

– prototypes look like their final product » computer-based prototypes (e.g., Palm Pilot prototype)» limited functionality so users can actually interact with

it» more polished and aesthetically pleasing

Low-fidelity (Low-fi)– artists rendition with many details missing

» paper-based prototypes» quick and inexpensive, provide valuable insights, but

don’t really demonstrate functionality» think about a fashion designer or an architect

4

Why Use Low-Fi Prototypes? Traditional methods take too long

– sketches --> hi-fi prototype --> evaluate --> iterate

Can simulate the prototypes– sketches --> evaluate --> iterate– sketches act as prototype– test prototypes in days vs. weeks

Kindergarten implementation skills– allows non-programmers to participate

5

Disadvantages of Hi-Fi Prototypes

Perceptions of the tester/reviewer?– formal representation indicates finished nature– users focus on appearance (color, fonts, etc.)

Time?– encourages precision– specifying details takes more time and results

in fewer design iterations

Creativity?– we tend to fall in love with our software

solutions and our program code

6

Stages of Low-Fi Prototyping Initial rough sketches of UI design ideas

– evaluate and iterate (usually w/i design team)

Storyboards– explicitly conveys the design model– evaluate and iterate (often w/i design team)

Chauffeured prototype– evaluate (with actual users) and iterate

When do you stop?– when you converge on a design model that is stable

enough to commit to software

7

Evaluating Low-Fi Prototypeswith Real Users

Constructing chauffeured low-fis– the materials– faking the interaction (wizard of oz technique)

Conducting a low-fi test with real users– preparing test materials– conducting the test (various roles)– debriefing test users– generating a report

Evaluate the results and iterate based on user feedback

8

The Basic Materials(Chauffeured Prototype)

Large, heavy, white paper (11” x 17”) 5” x 8” index cards Post-it notes of various size Tape, glue stick, correction tape Pens and markers (many colors and

sizes) Overhead transparencies Scissors, X-acto knives, etc. And more…

9

Constructing the Model Set a deadline

– don’t think too long - build it!!– it’s about getting ideas on paper

Draw a window frame on large paper Put different screen regions on cards

– anything that moves, changes, appears/disappears

Ready responses for any user action– e.g., have those pull-down menus already made

Use photocopiers to make many versions

10

Some Examples

Edit Dialog Box

UndoRepeatCutCopyPastePaste SpecialEtc…

File Edit Format Help …..

Edit Submit

NewSaveSave as…Print PreviewPrint…Etc…

File Edit

Record Added

OK

Error -4325

OK

16

Briefing Room Interactive (BRI)

17

Clarissa’s Embroidery Website (continued on next page)

19

A small part of the prototype for the SALT Center (UofA)

20

Low-Fi Computer Screen(pretty cool)

21

“Computer’s” viewTest participant’s view(with a low-fi calculator and scan-o-matic)

And a printer, too!!!!

Scenario materials

22

Constructing the Model With UI Tools / Builders (but do not

code)

23

UI Builder Tools Pen and paper Visio or Smartdraw Software drawing tools

– Microsoft Acess, Paint, PowerPoint, Word, Excel, etc.

If you construct the model with tools– print out the background screens and moving

pieces– the prototype should still be “paper-based” even

if you use software to make it look clean– hand write most of the words

24

Video “Paper Prototyping: A How-To Video”

– Nielsen Norman Group. Get some ideas and take some notes

25

Testing the Model Select your users

– look at user profile for target users– use a questionnaire to get the people you need– don’t use friends or family– get at least 3 users to evaluate the system

Prepare scenarios that are:– typical of the product during actual use– make prototype support these (small, yet broad)– use 3 scenarios of varying difficulty in the evaluation

Practice to avoid “bugs”

26

Conducting a Chauffeured Test Four testers (minimum)

– greeter - puts users at ease & gets data– facilitator - only team member who speaks

» gives instructions to the users» hands the scenarios to be completed to the users» encourages thoughts, opinions (think aloud protocol)

– computer - knows application logic & controls it» always simulates the response, w/o explanation

– observers - take notes & recommendations Typical session is 1 hour

– preparation, the test, debriefing

27

Conducting a Chauffeured Test Greet

– get consent forms filled– reads greeter’s script (or orientation script)

» provides brief background of what test is about, describes what a low-fi prototype is and its importance, assures confidentiality, put’s user at ease (voluntary, quit at any time, testing UI, etc.)

Test– facilitator reads demo script & hands written scenarios to user

» must be clear & detailed– chauffeur the prototypes

» involves user watching while another person “drives” the system– facilitator keeps getting “output” from participant

» Think Aloud Protocol - “what are you thinking right now?”– observe -> no “a-ha”, laugh, gape, etc.– possible to make changes to prototype in “real-time” if you want

28

Conducting a Chauffeured Test Debrief

– fill out post-evaluation questionnaire– discuss prepared list of debriefing topics– ask questions about parts you saw problems– gather impressions– give thanks

29

Video “Paper Prototyping: A How-To Video”

– Nielsen Norman Group. Get some ideas and take some notes

30

Evaluating Results Sort & prioritize observations

– what was important?– were there lots of problems in the same area?– assign a severity rating (0-4 scale) to each

problem identified or uncovered during the testing

Create a written report on findings– gives agenda for meeting on design changes

Make changes & iterate– note that you can make some changes “as you

test”

31

IBM’s Prototype Design Labs (Participatory Design Sessions)

“We start with low fidelity prototypes using a pencil and paper toolkit with paper window templates as well as colored sticky notes. Although our prototyping tools are can create these prototypes as quickly as pencil and paper can, we find that users are more willing to propose changes when the prototypes are low fidelity ones. We then use higher fidelity prototypes typically using IBM VisualAge. Rather than discussing what users would want in team meetings, we simply ask users directly.” (from IBM’s Ease of Use Website)

32

Microsoft Corporation “Prior to specifying the product, teams will perform

contextual inquiry with usability engineers. Early user interface evaluations may take the form of paper prototypes…Much early testing relies heavily on paper prototypes, which are used to iteratively lock in a design or a set of features and characters on the screen.”

(from the article, “Organizing Usability Work to Fit the Full Product Range”)

33

Clothes-Shopper(An Example)

Problem– contemporary approach to shopping is often slow and painful– people often have problems shopping for clothing through

catalogs (on-line or otherwise) because they are unable to visualize how they will look in the clothing

Design solution– design a UI that allow shoppers to quickly and easily visualize

how various combinations of clothing will look on them

Method– use low-fi prototyping to develop a good, general understanding

of the strengths / weaknesses of the initial UI design w/o spending a large amount of time on developing sftwr

34

Three Scenarios to Be Evaluated Moderate scenario

– Joe from Berkeley wants to buy a new shirt and sports coat to be more fashionable in school. He does not have a car today and decides to give the Clothes Shopper a try. Help him navigate through the system, buy the things he wants and leave. Although there is no budget constraint, he wants a white plain shirt and a cool brown jacket.

Moderate - difficult scenario– Sally is a regular user of the Clothes Shopper system. She wants to buy a white

gown for an upcoming prom but only has $150 as her budget. She already has an existing profile and does not wish to reenter the same information. Help Sally buy the gown and get her on a happy date!

Beginner scenario– Joanna is a new user to the Clothes Shopping system. She wants to go on the

tour first and become familiarized with the system before doing any shopping. After the tour she wants to window shop some blue shorts, but doesn't feel like buying anything today. She prefers to try on some shorts casually and leave the mall.

35

Procedure Greet Conduct the test

– facilitator, computer, observers Debrief Evaluate and analyze

– discuss what went well, what went poorly– comment on users’ reactions and comments– compile into master copy and make

suggestions for next iteration

39

Some Raw Notes from Observers (1/3)

Participant #2: Marlon Urias Scenario 1 (Joe from Berkeley):

Moved mouse around at first

"Is Tours clickable?"

Confused at mall entrance

"If you want to buy clothes, do you tour or go shopping?"

Easily finishes login screen

Claims he doesn’t know four of the measurements asked for, one of them "maybe" he knows

Thinks it’s "good" the male measurement screen didn’t block the profile setup screen, but was bottom part of that screen

Would be "helpful" if you knew where to get pants and shirts. Guesses it’s the Navigational Assistant. Clicks on torso.

Spent a lot of time at Navigational Assistant, confused. Pop-up notes should show up as you move around the Navigational Assistant, so you know where to click for clothes

40

After he picks shirts, realizes he needs to choose color, assuming program knows he is choosing color for shirt.

Wanted to see what results you would get so far before you finished choosing your preferences, "Would you click on the red curtain?" Since there are graphics, he hopes/assumes you would see results there. "Wish I could see it" (wanted to see Search Results updated automatically after changing one preference)

Clicked on left page flipper in Search Results window looking for clothes not there yet, before he hit Search button

Finally realized he’s supposed to hit Search button, says the term should change to "Show Product" or "See Product" or "See Progress"

Goes back to change size after seeing Search Results

He hopes preferences are kept track of, it wasn’t clear.

"Does preference change update the Search Results?" To him, Search button doesn’t equal update Search Results

Wishes he could buy the clothes before he sees it on Dressing Room model by clicking on Search Results thumbnails


41

Wants to see a big "Buy" button to buy clothes in Search Results window without opening Dressing Room

Wanted to delete things from Search Results window with right-click

"Can you decide to buy clothes from the thumbnail?"

Wonders what happens if model wears multiple shirts

Thinks Select All Clothes button chooses everything in Search Results

Confused how to make Dressing Room model wear clothes Would rather have right-click menus than "make assumptions"

Doesn't like Search button, "What's the difference between search and narrow down options?" (which is done in preferences)

Says that go and previous buttons are annoying.

Again there is problems with the page tabs, "are the numbers sizes or choices?" Recommended to avoid page tabs because not intuitive.

Had confusion with Select All, "does it buy it?" "Returns it to the model."

After opening the Shopping Cart, has no problem buying the items. "happy, very intuitive shopping cart window."


42

Summary Results (1/2)(or Merged List of Critical Incidents)

User knew how to enter system by clicking on diagram– one user had question about what “Tour” meant

Users got through logon screen pretty easily– comments about how much information the user was able or willing to reveal

(e.g., body weight, exact dimensions of clothes, etc.)

Big confusion at the main menu– did not know what to do to begin shopping (clicked on a lot of areas)– finally noticed “Navigation Assistant” after much searching and

experimentation and learned that they needed to click on it to begin– when selected garment that they wanted, many clicked on preferences

(colors) thinking they would be presented with choices» specifying search parameters was not intuitive and natural in choosing

their garment– began to play with search parameters and figured out how to use it to get

results

43

Summary Results (2/2)

Big confusion at the main menu (continued)– once the user got to the search results, two users were often confused on

how to get the garment of their choice on the model » they just clicked on the garment and expected it to appear on the

model» the design model was to have the user drag the garment to the model

– after getting the garments on the model the users expected to see a “checkout” or “buy” option that would send them to a checkout screen

» the designers intended for the users to “double-click” on the shopping cart to do this function

– after the users figured out how to get to the checkout, they bought the garments and exited the system pretty easily

Final note was that once the user learned how the system worked, they were able to quickly run through the system to perform the other scenarios

44

Discussion and Recommendations(What to Do Next?) (1/3)

Change Begin Tour to Begin Tutorial Login screen was too wordy -- the number of buttons

will be reduced to 3– New User, Use existing profile, Change Current Profile

Profile setup was fine except, for women, bra and chest size were considered redundant (eliminate one)

Navigational Assistant was not intuitive -- maybe add text or have menus automatically pop up instead of requiring a click– also need to add dresses and suit to torso section of assistant

since users asked where these would be– gray out preferences menu initially so user knows to click on

the navigational assistant first

45


Change preferences menu– use words Show Clothes instead of Search (not very intuitive)

Change some things about the Search Results section– user did not know whether to click or drag the thumbnails to

dress the dressing room model. They decided to just require a click to minimize confusion

– change title from Search Results to Clothes Gallery Dressing room changes?

– users liked the red curtain that concealed the model until a search was performed

– liked the Shopping Cart and Trash Can options– were very confused by the Select All Clothes option

» designers just dropped the idea to reduce confusion

46


Dressing room changes?– so users can just select clothing on the model and put it in the

shopping cart or trash can as desired– add a Checkout button so the user can go ahead and buy what

s/he has in his / her shopping cart (many users asked if such an option existed)

A user also requested if it were possible to have a counter showing how many pieces of clothing were shown in the Search Results window (soon to be called the Clothes Gallery)– this function will be added

47

High-Fi Revisions

50

Hints for Low-Fi User Testing(from “Waving Magic Wands”)

Interaction techniques to optimize lo-fi testing– establish a paper world

» e.g., low-fi computer screen, printer, keyboard, etc.

– provide a pointing toy– use input and output interaction– prototype all relevant elements– let test participants design (on the spot)– facilitate fun!

51

Milestones 2 and 3 Milestones 2

– look at initial sketches from Milestone 1, discuss with team, and converge on a design model

– develop low-fi storyboards to support scenarios

Milestone 3– develop a low-fi chauffeured prototype for user testing– identify and recruit 3 evaluators (preferably real users)– prepare and conduct a chauffeured user test– discuss the results (what test users said and did, + or -)– discuss likely changes to design for the next design

iteration

52

Milestone 3 Results

– raw data» what each observer writes down during user testing,

video taping, audio taping, etc.

– merged list of critical incidents (and severity ratings)

» get rid of redundant observations, make clarifications, etc.

– textual summary of results Discussion and recommendations

– specific recommendations to address potential usability problems uncovered during testing

53

Low-Fidelity Prototypes Paper-based prototypes

– focus on high-level aspects of UI– see if the design model matches the user model– not concerned about details (e.g., color, fonts, alignment)

Low-fis help answer certain questions– are the various UI screens organized well?– does the layout of each screen make sense?– is the navigation natural and intuitive?– is the wording/terminology user-centered?– are important tasks missing?– are the metaphors/mappings effective and appropriate?

54

Disadvantages of Low-Fis Paper mock-ups don't go far enough

– not realistic interaction with the interface» how would you show a drag operation?» how will users interact with I/O devices to be used» doesn’t provide realistic response time (how measure

performance)

– has no functionality– evaluators may scoff at low-tech prototypes– little feedback about details (appearance like

colors, fonts, alignments, etc.) of the design– don’t know if design is technically feasible

55

Summary Use prototypes to involve the user Advantages of low-fi prototypes

– cheap and quick to develop and evaluate– requires few technical skills– leads to fast iterations (and more iterations)– users and designers less likely to see design as fixed

56

E-Groceries Case Study

low-fidelity user testing is 485, professor matt thatcher

Documents