Usability testing for library catalogs

October 25, 2001Nicole Hennig, Web Managerlibraries.mit.edulibraries.mit.edu/barton

Thank you

Tracy Gabridge

Librarian for Civil & Environmental Engineering

• led the HTML customization team

Details available ...

http://macfadden.mit.edu:9500/webgroup/usability2001/barton/test1/overview.html

Outline

1. background

2. the tests

3. problems & solutions

4. future directions

1. Background

6 month process

January - June 2001

• old system: GEAC Advance

• new system: ExLibris ALEPH

Web OPAC project teams

• web OPAC team- public service librarians

- circulation staff - processing staff - cataloger - web manager

Web OPAC project teams

• HTML customization teamsame as previous, plus

- systems office staff- programmer

Bibliography

• on handout

• includes background on display and interface design of library catalogs

Background research

• a lot of research on OPAC design available

• but not based on observing usersor usability testing

• library system vendors are not following basic good design principles

Who makes design decisions?

• we have more control now that we can customize HTML screens

• the vendors need to practice good design in building the system

A work in progress

• libraries.mit.edu/barton

• more rounds of testing and improvements are coming later in the spring

Usable design goals

• every page is self-explanatory

• “self-teaching” interfaces

Will it apply?

• some things are specific to ExLibris systems

• many things are general - could apply to any OPAC

General principles

• success summary

http://macfadden.mit.edu:9500/webgroup/usability2001/barton/test2/success.html

2. The tests

The test

• we had already done extensive usability testing while redesigning our web site

Latest thinking has changed

• 1999:Large test, 30 users, timed people- quantitative

• 2001:- More frequent, smaller tests, 5-6people at a time- qualitative

The test

• 1/2 hour long

• 10 questions

• think out loud

The test

• observer takes detailed notes

• train observers to not answer how it was supposed to work until end of the test

• each observer tests 2 people(2 week time frame)

Designing questions

• easy, basic tasks that a first-time user should be able to accomplish

• real-world tasks(give them a real article citation)

Designing the questions

• no need to obsess over perfect, “scientific” questions

• you will learn plenty from watching people use the catalog

The questions

• 1 - 5: known items• 6 - 10: general research

• complete list:http://macfadden.mit.edu:9500/webgroup/usability2001/barton/test1/questions.html

The questions

• test the questions

• get the bugs out

• print out the questionsin large type

Who we learned from

• Washington State University

Janet Chisman, et al.“Usability Testing: A Case Study”College & Research LibrariesNov. 1999

What we learned

• multi-part questions

- if user can’t complete first part, observer does it so they can try second part

What we looked for

• features that were confusing or unclear

• aspects of the system that worked well

The teststest 1 test 2

Who 7 students 3 students3 library staff 4 library staff

4 disabled

Catalogs our old web catalog:Barton (6) 1st draft ofMcGill: MUSE (2) new BartonBoston College: QUEST(2) screens

Dates Jan. 22 - Feb. 1, 2001 May 21 - June 1

Successes 4 of 10 tasks 7 of 10 tasks

3. Problems & solutions

Problem 1

• people usually picked the default choices or the first choices without thinking much about it

(not always the best strategy for their search)

Example

people used first box,ignored second

Solution

Default choice is keyword. This casts a broadnet for those who forget to make a choice.

Problem 2

• Difference between browse& keyword search not clear

Example

?

?

Solution

No need to know difference between keywordand browse search. Combined in one menu.

Problem 3

• it wasn’t clear how to input a search string

(people used initial articles, author’s first name first, thought they had to type the entire title)

Example

• carefully typed complete title, with article:The Journal of the American Chemical Society

Examples far away

Solution

• include examples and instructions of how to input data near the search box and in the search menu

Examples for each type

Example changes when menu changes.

Examples for each type

Example changes when menu changes.

Grouping

Group different title searches, author searches, and subject searches together.

{

Problem 4

• very busy screens with many buttons were overwhelming for people

Example

Solution

• Present choices only where needed

• Group navigation links in ways that make sense

Problem 5

• it was difficult to find clickable URLs for electronic titles

No URL on brief results

Better: URLs showing

Problem 6

• not enough information on brief resultsscreen to choose the most relevant titles

(especially for subject searching)

• [see David Thomas article in bibliography]

keyword: women scientists

includes subject headings

Problem 7

• pop-up windows caused confusion

Solution

• minimize number of pop-up windows

• use only in cases where it’s handyto have the previous screen in thebackground

Problem 8• title you input isn’t at the top

Solution• add marker (wish list)

Problem 9

• holdings info for serials was very confusing to everyone

Example

Do we have Dec 13, 2000 issue?

Example

Example

• people don’t understand open date range

v.1 (1879)-

Not easy to fix

• NISO standard

• way the data is input

• limitations of the system

Example

A better holdings display

A better holdings display

Problem 10

• back buttons or back links didn’t behave as expected (a problem with frames)

Coming soon

• no frames version due from ExLibris soon

• telnet version next year

• minimal javascript

What worked well?

- hyperlinked author names- hyperlinked subject headings

people found and used these very successfully

Self-teaching interfaces

• For difficult searches where youneed to combine fields in a specific way:

- design screenso user doesn’t need to know

- it just does the right thing

Users with disabilities

• 2 blind users: one used “Jaws”, one used “Window Eyes”

• 1 user with dyslexia

• 1 user without use of hands, usedpencil in fist to type, and large trackball

Users with disabilities

• these users had same problems and successes as everyone else

(but the problems were magnified)

• everything took longer

Users with disabilities

• solutions that help everyone help disabled

• solutions that help disabled help everyone

Categories of problems

problems we can fix by:

• changing the HTML• changing the tables in the database• adding custom programming• changing our indexing decisions• changing cataloging practice

problems that only the vendor can fix

Other problems

• many other problems not mentioned here today are described on our web site

4. Future directions

Latest usability research

• most large web sites have hundreds of usability problems

• continuous rounds of testingare necessary to find and fix all problems

• better to begin with “user-centered design”

Latest usability research

User Interface 6 East, Cambridge, MAOct. 2001

Proceedings available

http://www.uie.com

Still to test

Not so basic features, like:

• email/save/print• your bookshelf• advanced searching• complex limiting

• etc.

Cycles of testing

• frequent small tests

• test your solutions

• informal tests with handful of people

Future directions

• share information

• compile guidelines

• influence vendors

Future directions

• are other libraries testing web OPACs?

• contact me to share test results

Nicole Hennighennig@mit.edu