One System, Different ExpectationsThe User at the Center of Discovery

Laura Morse, Harvard UniversityChristine Stohn, Ex Libris

2014 Charleston Conference - November 5 – 8, 2014

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

Introduction - Christine Stohn

User Centric Discovery Platform Implementation – Laura Morse

First Results and Next Steps – Christine Stohn

Discussion

2

Agenda

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

3

User Studies – Purpose and Methodology

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

4

User come with different expectations• A quick overview of a specific topic?

• The newest articles for my research area?

• Material for an undergraduate course assignment?

• Material for a research project?

• Access to a specific book or article?

Different material typesDifferent subjectsInterdisciplinary studies…

5

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

User Studies

Usability studies

Log Analysis

UI designSearch and rankingPersonalizationContent decisionsData handling

Focus user

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

6

Log analysis

Broad-topic query

Narrow-topic query

Author-related query

Known-item query

Tells us for example- What do users search for- How do users search- How do users and system

interact

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

7

Usability studies

Tells us for example- How intuitive is the UI- Do users find all they are looking for- Does the system meet the user’s

expectation

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

8

User Studies or

Usability Studies?

Usability is the ease of use and learnability of a human-made object (Wikipedia) User studies is the

investigation of users (and potential users) for example of a service or system

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

Understanding users

9

• Challenge assumptions and gain new insights

• Who are the (potential) users and what do they need? • Understand differences and similarities• Identify recurring themes• Cluster them into groups with the same

characteristics• Create Personas and identify common themes

• Enhance discovery experience and effectiveness

Personas?

Personas are fictional characters. They represent the goals, assumptions, skills and attitudes of a specific group.

The core questions for our personas:

• Who is the person – e.g. degree, subject• What characterizes this person – what do they like, how do

they work, what annoys them• What can we say about their information literacy skills and the

systems/tools they are using?• What do they use information for

Methodology

• Library reported user scenarios

• Workshops with librarians from different institutions• Harvard University workshop, with participation from

• Harvard University• Brandeis University• CUNY• Boston College• Boston University• NYU

• Oxford University workshop

• User interviews/questionnaires11

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

Methodology

• 42 Scenarios• 24 questionnaires and interviews• 7 provided by Harvard librarians• 11 created in the Harvard workshop

12

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

13

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

User Centric Discovery Platform Implementation

Established1636FacultyAbout 2,100 faculty members and more than 10,000 academic appointments in affiliated teaching hospitalsStudentsHarvard College – About 6,700Graduate and professional students – About 14,500Total – About 21,000

Faculties, Schools, and an InstituteHarvard University is made up of 11 principal academic units – ten faculties and the Radcliffe Institute for Advanced Study. The ten faculties oversee schools and divisions that offer courses and award academic degrees.

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

14

• 70 separate library units• nearly 1,000 fulltime employees• 18.9 million volumes • 42 million digital files • 174,000 serial titles• 10 million photographs• an estimated 400 million manuscript items

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

15

16

Discovery at Harvard

• Selected Aquabrowser in 2008• Implemented Aquabrowser in 2009 with ILS Data• Added in EAD Finding Aids and VRA core”ish” data in 2010• Planned to include federated search in 2008, but . . . this was

before index based search• Piloted Ex Libris Metalib+ (Primo Interface with Primo Central

Index) in 2010• Fast forward to 2012 . Users began expressing interest in

“books and articles” together in surveys related to redesign of website

• Search and Discovery project prioritized with goal of being user centric.

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

Discovery at Harvard

Fall 2013 Harvard Library User Survey• 434 undergraduates, graduate students, faculty, staff,

and other non-Harvard researchers across all disciplines. 297 (68%) ranked single search as important.

“One form, one search, one syntax.” – faculty member survey respondent

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

17

Discovery at Harvard

• Successfully implemented Primo for Fall 2014 (books & articles)

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

18

User Centric Implementation

• Keeping the user at the core of our thinking• Usability testing configurations across a multitude of devices• Resourcing project to quickly respond to user feedback• Defining personas for users • Determining differences related to discipline of study

• Many issues encountered and addressed in implementation were universal to all users and disciplines

• Relevancy concerns were different depending on type of user and discipline

• Partnered with Ex Libris to work on improvements to relevancy

• Continued work on the NISO Open Discovery Initiative

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

19

Now for something completely different - ODI

• ODI Recommended Practice released in June, 2014 • Standing Committee charged:• to promote educational opportunities about adoption of

these recommended practices• to provide support for content providers and discovery

providers during adoption (including championship of self-check conformance lists)

• to provide a forum for ongoing discussion related to all aspects of discovery platforms for all stakeholders (content providers, discovery providers, libraries), and

• to determine timing for next steps for ongoing workMore Information: www.niso.org/workrooms/odi/

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

20

Harvard Relevancy Advisory Group

• David Osterbur, Lead - MedicineTimothy Driscoll - ArchivesAnna Esty - HumanitiesMelinda Kent - LawGloria Korsman - DivinityLaura Morse - Systems OfficeAndras Riedlmayer -Mary Sears - ZoologyMelissa Shaffer - BusinessEmily Singley (staff)

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

21

Relevancy across disciplines

BusinessDentalDesignDivinityEducationFaculty of Arts and SciencesGovernmentLawMedicalPublic HealthRadcliffe

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

22

Process for Information Gathering

• Local Personas• Research Worksheets• User Interviews• Relevancy Workshop

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

23

Personas

• Redesign of library portal in 2012• Consultants solicited input from users in face to face

interviews, surveys, drop in sessions• Defined personas

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

24

Personas

• Reviewed & updated personas for discovery project in 2014

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

25

Research Worksheets

Example Search Scenario Worksheet• System Tested • Submitted By • Persona • Discipline/Research Area • Research Context • Example Search • User expectation • Screen shot of search term as entered in system• Screen shot of search results• Results Meet Expectation • Please describe improvements needed and provide examples

of results you would expect.

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

26

Research Worksheets

Persona - FacultyDiscipline/Research Area – EducationExample Search – reading difficultiesResearch context – Has been researching in this area for several years and knows the standard education journals publishing on this topic User expectation – Hopes to discover very recent articles or articles in journals from a broader range of disciplines (psychology, linguistics, neuroscience, etc.)

Persona - Graduate student/researcherDiscipline/Research Area – Humanities/MusicologyExample Search – Mozart opera manuscriptsResearch Context – This graduate student is interested in analytical studies of primary source material: manuscript musical scores useful for research on compositional practice & performance practiceUser expectation – Possibility 1: Scholarly articles; Possibility 2: links to digitized manuscripts

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

27

Research Worksheets

Persona – UndergradDiscipline/Research Area – Film StudiesResearch context – Need to locate recent articles on Iranian cinema for short research paperExample Search – Iranian cinema User expectation – Hopes to discover very recent articles or articles in journals from a broader range of disciplines (psychology, linguistics, neuroscience, etc.)

Persona - PractitionerDiscipline/Research Area – MedicineExample Search – antimalarials side effects interactionResearch context – Looking for articles on side effects and drug interactions of antimalarialsUser expectation – Looking for new research that might shed light on patient’s unexpected response to a new drug regime

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

28

Relevancy Workshop

August 4th, 2014

11:00 Welcome and IntroductionChristine Stohn, Ex Libris

11:15 Lightning talks on different user scenarios (5 minutes each)

12:00 Round table discussions to identify themes and user scenarios, their differences and similarities. Create clusters of scenarios and their distinct characteristics.

13.30 Different forms of discovery, including searching, browsing, serendipity, exploration.

14:00 Round-table discussions – mapping user scenario clusters to different forms of discovery

15:15 Round table discussions, continued

16:30 Wrap-up and Summary

17:00 Meeting adjourns

System agnostic day to why users seek scholarly material and how they embark on the task of searching for and discovering it.

GOAL: identify themes and personas together with their differences or similarities and diverse needs.

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

29

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

30

User studiesFirst results and next steps

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

The User Studies questionnaire

• Subject and academic level• What literature do you mostly need for your work • Where do you usually get your literature from• What devices do you use and what for• Preferences/needs when looking for information• Preferred characteristics of material• Where do you usually start your search• Familiarity with topic when starting the search• Favourite web sites (give reasons)• What do you find annoying• Specific use case

31

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

Challenges

• Terminology is not consistent

• Qualitative vs. quantitative

• Changes to the questionnaire – lack of consistency

• Providing options leads to suggestive choices, no options to misinterpretation

32

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

• Total: 24

33

Respondents (Interviews/questionnaire)

female 14male 10

Canada 2Germany 5Israel 1UK 2US 14

Faculty/researcher 3Graduate 9PhD student 5Undergraduate 7

Architecture 1Humanities 4Design 1Law 1Economics/Business 4Social Sciences 4Psychology 1Urban planning 1Medicine 2Sciences 3Biotechnology 1Engineering 1

Recurring themes – User groups• Students often need to build up terminology first• Graduates to understand their research area first and then narrow it down• Undergraduates to gain more knowledge for papers/projects

• Undergraduate students take most of their resources from reading lists/set text books – but there are exceptions

• Faculty members and researchers know their core publications• they are usually interested in the newest material• they often follow citations trails to find similar material• they may also follow author trails (known authors and co-authors)

• Graduates and researchers • tend to look for thorough lists• often work on projects where they need an overview as well as very

specific aspects of a topic

34

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

Recurring themes – Subjects

35

• Undergraduates in the humanities work often on papers and essays and need literature for that

• Undergraduates in the sciences work towards exams and are often satisfied with text books, they use other literature for occasional tasks

• Also noteworthy - the same subject is not taught in the same way in every country: Example Medicine

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

Other

36

• Almost all respondents use Laptops, only few use tablets

• Tablets are used mostly for quick tasks and on the go

• Google Scholar is the most mentioned resource for users who search themselves

“Use cases”One of my most recent tasks was to gather more information about a specific therapy (Photopherese) used for Graf-versus-Host-Disease. Most important for me was to gain a general overview of the topics and the current status of research. I started with searching for photosphere in Google but didn’t find anything useful but once I found more material by getting help from peers I really enjoyed to get deep into the subject.

Undergraduate student, Medicine

37

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

In researching Scandinavian migration to the United States in the 19th century and its impact on local history and religious institutions, it was actually necessary to research older publications dating from 1820 to 1940 to find more first hand material. No one book or journal article was comprehensive, so many titles had to be reviewed to determine this impact.

Researchers, History

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

38

“Use cases”

I work mostly towards exams. I use textbooks a lot and online learning programs. I think this will change when I start working on my PhD next year.

Graduate Student, Medicine

Recently I needed to find research papers to prepare for a talk about semiconductor optical amplifiers. The goal was to both present recent developments in the field but also original publications about the topic to explain the broader concept.

Graduate student, Physics

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

Discussion

39

One

Sys

tem

, Diff

eren

t Exp

ecta

tions

: The

Use

r at t

he C

ente

r of D

isco

very

Thank you!

laura_morse@harvard.educhristine.stohn@exlibrisgroup.com

40