Browse design in digital libraries: Impact on user experience

Xiangmin Zhang, Yuelin Li, Jingjing Liu & Ying ZhangSCILS, Rutgers University{xzhang,lynnlee, jingjing,yinzhang}@scils.rutgers.edu

LIDA, Dubrovnik, May 28, 2007

22

OutlineOutline

BackgroundBackground Purpose of the studyPurpose of the study MethodologyMethodology ResultsResults DiscussionDiscussion

33

BackgroundBackground

IEEE supported researchIEEE supported research Part of a large scale usability Part of a large scale usability

evaluation of Xplore digital library:evaluation of Xplore digital library:• Searching & browsing designsSearching & browsing designs• System performance issuesSystem performance issues• In comparison with ACM and IEEE CSIn comparison with ACM and IEEE CS

Started in March 2005, reports Started in March 2005, reports submitted to IEEE, Incsubmitted to IEEE, Inc

Part of data yet to be analyzedPart of data yet to be analyzed

44

Purpose of the current studyPurpose of the current study

The purpose of the study has two The purpose of the study has two foldsfolds• to find out how the interaction designs to find out how the interaction designs

for browsing would affect the user for browsing would affect the user experience, and experience, and

• to inform the design of better user to inform the design of better user experience in digital libraries experience in digital libraries

55

Research QuestionsResearch Questions

Do different browse design features Do different browse design features (browsing conference proceedings) (browsing conference proceedings) have an effect on users’ browsing have an effect on users’ browsing experience?experience?

What design features would lead to What design features would lead to poor user experience? poor user experience?

66

MethodologyMethodology

Three DLs and their browse featuresThree DLs and their browse features Research designResearch design

• ParticipantsParticipants• Browsing taskBrowsing task• ProcedureProcedure

77

Three Digital LibrariesThree Digital Libraries

Three operational digital libraries:Three operational digital libraries:• ACM Digital Library (ACM)ACM Digital Library (ACM)• IEEE Computer Society Digital Library IEEE Computer Society Digital Library

(IEEE CS)(IEEE CS)• IEEE Xplore Digital Library (Xplore)IEEE Xplore Digital Library (Xplore)• WebsitesWebsites

88

ACMACM

99

IEEE CSIEEE CS

1010

XploreXplore

1111

Major design characteristicsMajor design characteristicsDesign featuresDesign features ACMACM IEEE CSIEEE CS XploreXplore

Browse Browse initiationinitiation

Click “proceedings” link Click “proceedings” link on the home pageon the home page

Click “alphabetic list” to Click “alphabetic list” to access the proceedings access the proceedings from home page. from home page.

Click “proceedings” link on Click “proceedings” link on the home page.the home page.

Browse options Browse options for conference for conference proceedingsproceedings

Search within proceedings Search within proceedings and browse alphabetically, and browse alphabetically, then providing “search then providing “search within (the name of a within (the name of a specific proceeding)”specific proceeding)”

AlphabeticallyAlphabetically

Browse by keyword and Browse by keyword and browse alphabetically, then browse alphabetically, then providing “search this providing “search this conference proceedings” conference proceedings” and “search this and “search this publication”publication”

Navigation for Navigation for browsing browsing proceedingsproceedings

No navigation support for No navigation support for “Browse the ACM “Browse the ACM proceedings;” but proceedings;” but providing a navigation bar providing a navigation bar for “Search the ACM for “Search the ACM proceedings” result page: proceedings” result page: 11     22     33     44     55     66     77     88     99     1010       nextnext””

No navigation facilitiesNo navigation facilities

Providing a navigation bar Providing a navigation bar as: “View: 1-25 | as: “View: 1-25 | 26-5026-50 | | 51-7551-75 | 76-100 | 101-125 | | 76-100 | 101-125 | Next > | Back to top” if Next > | Back to top” if multiple pages returned for multiple pages returned for an initial letter; Providing a an initial letter; Providing a link “Back to top” on every link “Back to top” on every page.page.

1212

Research DesignResearch Design

36 participants from Rutgers students 36 participants from Rutgers students communitycommunity

Included both graduate and Included both graduate and undergraduate studentsundergraduate students

Science, engineering, MLIS studentsScience, engineering, MLIS students

1313

Research Design (2)Research Design (2) Browsing Task:

• Xplore:Please browse the proceedings of ITCC: International conference on Information Technology (2004): Coding and Computing, and locate and save two papers about data streaming.

• ACMPlease browse the proceedings of Annual Symposium on Computational Geometry (SCG) (2004), and locate and save two papers about data streaming.

• IEEE Computer SocietyPlease browse the proceedings of ITCC: International conference on Information Technology (2004): Coding and Computing, locate two papers about data streaming, and save their abstracts

1414

Research Design (3)Research Design (3)

Latin Square design: randomize the order Latin Square design: randomize the order of tasks and systemsof tasks and systems

Every 3 participants followed the same Every 3 participants followed the same orderorder

One student droppedOne student dropped

1515

User Interaction MeasuresUser Interaction Measures

The amount of browsing The amount of browsing timetime

The number of browsing The number of browsing stepssteps

The number of user errors The number of user errors related to browsingrelated to browsing

1616

ResultsResults

User characteristics:User characteristics:• Experienced computer users, and very Experienced computer users, and very

experienced (83%) Web search engine experienced (83%) Web search engine usersusers

• Inexperienced users of the three digital Inexperienced users of the three digital libraries in the study: libraries in the study:

66% never used ACM and Xplore 66% never used ACM and Xplore 80% never used IEEE CS. 80% never used IEEE CS. Strong correlationsStrong correlations

1818

Comparison of interaction measuresComparison of interaction measures

ACMACM IEEE CSIEEE CS XploreXplore FF

Mean browsing Mean browsing time (seconds)time (seconds)

276.86276.86 610.71610.71 345.51345.51 25.931*25.931*

Mean # of Mean # of browsing browsing stepssteps

12.9112.91 37.3137.31 29.4029.40 8.92*8.92*

Total # of Total # of browsing browsing errorserrors

11 66 22

*Significant difference was found at p<.01, tested by ANOVA

1919

User satisfactionUser satisfaction

StatementUser mean ratings

ACM IEEE CS Xplore

Satisfaction with the browse result*

5.34 2.31 4.77

Overall easy to browse* 5.63 3.29 5.37

Overall satisfaction with the browse feature of the system*

5.77 3.57 5.31

Easy to learn to use 5.80 5.20 5.77

* Significant difference at p < .05, ANOVA test

2020

DiscussionDiscussion

Major user problems with IEEE CS design:Major user problems with IEEE CS design:• No search function provided with browseNo search function provided with browse

Does not meet the user’s expectation of using search Does not meet the user’s expectation of using search as short-cuts.as short-cuts.

• Unpleasant visual/presentation designUnpleasant visual/presentation design Major user problems with Xplore design:Major user problems with Xplore design:

• Proceedings not ordered either on full title or Proceedings not ordered either on full title or acronym. acronym.

Suggestions to improving system designsSuggestions to improving system designs

2121

Further studiesFurther studies

More tasksMore tasks Visual/presentation design issuesVisual/presentation design issues Experienced users Experienced users

2222

Thank you!Thank you!

Questions?Questions?