Owning The Discovery Experience For Your Patrons

Charleston Conference, 2014

Robert McDonald, Courtney Greene McDonald,Esther Onega & Kate Lawrence

Welcome

Robert McDonald

What We’ll Share

Kate Lawrence, Vice President of User Research, EBSCO Information Services

The Google Influence on Discovery

Esther Onega, Head, Charles L. Brown Science & Engineering Library, The University of Virginia

Discovery Tools, A Case Study: The University of Virginia

Courtney Greene McDonald, Head Discovery & Research Services, Indiana University Libraries

“Everybody, Everything”: Crafting a Systemwide Discovery Strategy

Kate Lawrence, Vice President of User Research, EBSCO Information Services

The Google Influence on Discovery

@bykatelawrence

klawrence@ebsco.com

User Research at EBSCOMore than just usability testing

DataWhat is the story

behind the data?

Secondary ResearchWhat questions have been asked and

answered previously?

Primary Research

Matching the right method to

the research topic

College Students StudyDigging deep to understanding their digital lives

154 3

22 Students:

High School College Graduate

School

Schools included MIT, UNLV, Rice, Georgetown, UMass Amherst, GWU, UCSF, and more

“It’s their oxygen.”Google

The first half of the first page of results is critical.

Page 1 matters most“If it doesn’t exist on Google, does it exist?”

I trust itGoogle is their point of triage, how they discover the world.

“My door to the world.”

Wikipedia is often the first result.

Go to Images, News etc from this page.

Students are more likely to search again than to look at page 2+ of results.

WikipediaBecause college students like to start with an overview

1

2

3

The overview in “layman’s language”

The table of contents – “preview”

The references and external links at the bottom

Skimming and scanning

Skimming and scanning

What gets students interested in researchMotivation

“When it’s about me, it’s just more

interesting and worthwhile.”

Friends

Peers are the most significant

influencer among college students.

Future

Will this help me pay off my debt?

Get a job? Not have to move home with my parents after graduation?

Subject

“I fell in love with computational fluid dynamics. Google wasn’t

good enough any longer.”

RelevanceMentor

“I want my advisor to be proud of me.”

Students “make the turn” towards informavore

Motivation

The Library WebsiteGetting there via Google

“I don’t speak library-ese.”

Core

colle

ction

Abst

ract

s

Subj

ect I

ndex

Onl

ine

Data

base

sAr

chiv

es

Bool

ean

A simplified, elegant library website

When Discovery Means Find

“Everybody, Everything”: Crafting a Systemwide Discovery StrategyCourtney Greene McDonald, Head Discovery & Research Services, Indiana University Libraries

@xocg

crgreene@indiana.edu

One University, Many Front Doors

Systemwide Discovery

IUCATShared online catalogPowered by Blacklight

OnCourseShared learning management systemCurrently SakaiMoving to Canvas in 2016

Indiana University Libraries System

Diverse User Base, Divergent User PathsGoal: Consistency regardless of chosen entry point

Defining opportunities at the campus level

Discovery at Bloomington

Leverage EDS API to pull results into website, a crucial discovery platform

EDS IntegrationResponsivePlanned study of new library website, specifically search results

User InputIncreasing proportion of mobile users – designing for that need

Balancing consistency with choice

Discovery Systemwide

Leverage API to pull results into campus views; EDS Reading List for CMS

EDS IntegrationFlexibilityPlanned study of catalog interface

User InputCampus views: consistency with options; open source community collaboration

Esther Onega, The University of Virginia

Discovery Case Study: The University of Virginia

It all started with an email

Why Discovery?TO: [lb-pubserv] Primooooooo!

On Jun 5, 2013, at 11:26, "Burks, Todd (tcb2e)" <tcb2e@eservices.virginia.edu> wrote:

Does anybody really like Primo? To be perfectly frank, I don’t. I don’t show it to First-Year students. In fact, I actively discourage them from using it. They’re set up for failure if they use it, and I need them to succeed in order to show the value of our services. Primo doesn’t add any value to the sessions I teach. What about you? Todd P.S. I’d like to have the default VIRGO search NOT include the Primo column on the right. It makes my job more difficult to steer new students away from it. If there are users that use it all the time, they should be able to set that preference without us having to do it for them.

TO: [lb-pubserv] Primooooooo!

On Jun 5, 2013, at 11:26, "Burks, Todd (tcb2e)" <tcb2e@eservices.virginia.edu> wrote:

Does anybody really like Primo? To be perfectly frank, I don’t. I don’t show it to First-Year students. In fact, I actively discourage them from using it. They’re set up for failure if they use it, and I need them to succeed in order to show the value of our services. Primo doesn’t add any value to the sessions I teach. What about you? Todd P.S. I’d like to have the default VIRGO search NOT include the Primo column on the right. It makes my job more difficult to steer new students away from it. If there are users that use it all the time, they should be able to set that preference without us having to do it for them.

How we got hereFirst, the meeting

Identified problems

Brainstormoptions

Analysis

StartedTask force

Process for Evaluation• Reinitiated ExLibris (Primo) customer service

• Limited results to peer-reviewed

• Included abstract metadata in search results

• Contacted peer institutions

• Arranged open vendor presentations of top WSDS products & established evaluation criteria:• EBSCO (EDS)• ProQuest (Summon)• OCLC (WorldCat Local)

• Analyzed link performance

• User testing

Evaluation CriteriaHow we decide

7 Essential Elements

FunctionalitySpeed, relevancy

APINecessary for incorporation into our

catalog, VIRGO

MetadataSearchable fields, field consistency,

faceting

Quality ControlResults reflect library holdings

CoverageCompatibility of Library holdings with

those of the discovery service

Customer ServiceMUST be available, helpful, and

responsive

RelevancyAre librarians and users finding results

to be useful ?

Evaluation CriteriaHow we decide

API

• Each service has a web API:• The service is engaged by opening a URL containing the

specifics of a given request• The output from the URL is used to construct the contents

of the requesting page• A separate URL query and response is required for:• Each page of search results• Each individual article details page

• Each service has its own unique URLs for sending requests• Each service has its own unique XML reply format for receiving

results.

Comparing Metadata

Metadata

Ferreiro, Alberto. "Simon Magus, Nicolas Of Antioch, And Muhammad." 

Church History 72.1 (2003): 53-70.

Primo•No abstract•No subject headings•Incomplete author name•Junk fields

There is a duplicate record for this article further down in the results did have full name and abstract, but due to an incorrect title could not connect through the link resolver.

Comparing Metadata

Metadata

Ferreiro, Alberto. "Simon Magus, Nicolas Of Antioch, And Muhammad." 

Church History 72.1 (2003): 53-70.

EBSCO•Abstract•4 subject headings•No junk fields

No duplicate records and the link resolver connected properly.

Comparing Metadata

Metadata

Ferreiro, Alberto. "Simon Magus, Nicolas Of Antioch, And Muhammad." 

Church History 72.1 (2003): 53-70.

Summon•Abstract•2 subject headings•More and better identifiers•No junk fields

No duplicate records and the link resolver connected properly.

Limiting Results to Our Holdings

• Results automatically limited by integration with Serials Solutions holdings.

• Updating Serials Solutions holdings automatically updates Summon results.

Yes Yes YesSummon EBSCO Primo

• Manually export and upload of single Serials Solutions holdings file.

• Updating Serials Solutions holdings requires updating EBSCO.

• Manually export, combine, and upload 27 separate Serials Solutions holdings files.

• Updating Serials Solutions holdings requires updating Primo.

Article Detail Page: Primo

Not part of the subjects data

Includes subtitle and title of magazine feature

Article Detail Page: EBSCO [EDS]

Refine?

Article Detail Page: Summon

LC-style subject headings broken into

separate terms (e.g. “Moose – Portrayals”

becomes “Moose” and “Portrayals”)

Library Staff Lunch & Test• 25 worksheets completed

• 14 votes received• Many cookies consumed• No clear winner, but 1 clear loser (Primo)

User Test Results

EBSCO EDS was favored overall,

but only by 3 searches

Qualitative Results – EBSCO [EDS]

Positives Negatives

• Quantity of articles• Facets• Relevancy

• Specificity• Non-English language

articles• Speed

“Seems to come up with much more specified results of these keywords, possibly quite useful for a researcher already well-into the research process and quite a bit familiar with the materials...but not sure how good it would be for a beginning researcher.

“One of the first things I noticed about the articles in these results was that the titles seemed more interesting and engaging, which made me more inclined to explore the article further. The search results included a good variety policy evaluation, study analysis, and general pieces on medical tort reform.”

Qualitative Results – Primo

Positives Negatives

• Diversity and range of articles• Journal titles, specificity, focus

• High quality• Speed

• Dated materials• Duplicates• Medium quality

“The first page of results covers good ground in terms of major works one would need to be familiar with when conducting research in this field, and displays a good diversity in terms of sources.”

“Using Firefox: Yielded 1,327 article results that were not exactly relevant to the paleo diet. The word 'paleo' [paleo diet] was included in some but the results were more on chemical reactions and such and i was looking more for experiments and peer reviewed journals covering the topic. It was not really relevant to the paleo diet at all.”

Qualitative Results – Summon

Positives Negatives

• Perceived selectivity• International diversity• Cross-disciplinary

• Recency• Non-English language

materials• Questionable faceting

“I liked the Summon results a lot. The first three results were directly relevant, and most on the first page of results were at least somewhat relevant. There were also about 600 results (as opposed to about 1200 in Existing Virgo and Ebsco) so it seems that Summon weeded out some of the not relevant results.”

“First article was from 1998, which is too old to be still relevant. Do not know why this is at the top of the articles list. Quality of journals is slightly suspect, although less so than EBSCO.”

Products, Prices & Contracts

ExLibris (Primo)$ • Would insist on annual contract

• Set-up fees paid 3 years ago

EBSCO [EDS]$ • 6-month free trial included

• 1-year contract• No set-up fee• Comes with full EDS

ProQuest [Summon]$$ • 1-, 2-, 3-year contract options

• Significant set-up fee

Questions

Thank you