February 24, 2015

DISCOVERY, ANALYSIS, AND

SHAPING THE USER EXPERIENCE

Tracy GilmoreCollections Assessment Librarian

Virginia Techtgilmore@vt.edu

Angie ThorpeDigital User Experience Librarian

Indiana University Kokomoatthorpe@iuk.edu

TWO LIBRARIES, TWO STORIES

Indiana University KokomoFTE 2660

Baccalaureate GeneralNursing, Business, Health SciencesNew strategic plan 2015-2020

DISCOVERY AT IU KOKOMO

• Spring beta• Fall launch

2011

• Faculty sessions

• Marketing campaign

2012 • Start Google Analytics tracking

2013

• Analyze Analytics data

2014

TWO LIBRARIES, TWO STORIES

Virginia Tech

FTE 31,000

Doctoral/Research Extensive

Engineering, Business, Agriculture

DISCOVERY AT VIRGINIA TECH

• Summer beta test

• Fall Launch

2011

• Conducted a Mixed-Method Summon Usability Study

2012 • Launched a Summon Usability Survey only study.

2013

• Implemented Summon version 2.0

2014

QUANTITATIVE

USER ENGAGEMENT BY FISCAL YEAR

Abstra

cts

Total F

ul l Text

PDF

Ful l Text

HTM

L Ful l

Text

EBSCO S

mart

Lin

ks

Custom

Lin

k-outs

30.8K

20.4K

12.4K

6.9K

1.4K

5.8K

33.6K

19.7K

12.0K

6.5K

1.2K

10.0K

34.8K

23.6K

14.9K

7.8K

3.2K

11.7K

FY 2011-12 FY 2012-13 FY 2013-14

STATES WITH ZERO SESSIONS

Desktop Tablet Mobi le

98.2%

1.6% 0.2%

DEVICES

41%

24%

21%

14%

IEChromeFirefoxSafari

WHICH BROWSER DO EDS USERS CHOOSE?

11.7%

17.4%

19.4%19.6%

15.9%

8.7%

7.3%

SundayMondayTuesdayWednesdayThursdayFridaySaturday

ON WHICH DAYS OF THE WEEK IS EDS MOST USED?

AT WHAT TIMES OF THE DAY IS EDS MOST USED?

12:0

0 AM

2:00

AM

4:00

AM

6:00

AM

8:00

AM

10:0

0 AM

12:0

0 PM

2:00

PM

4:00

PM

6:00

PM

8:00

PM

10:0

0 PM

0.0%

2.0%

4.0%

6.0%

8.0%

10.0%

% of Total Pageviews

Successful Queries

• Field codes• AU flannery o'connor• DE “active learning”• SO Harvard business review

• Phrase searches• “deforestation effects”• “poverty rates” “United

States”

• Advanced Search• American dream AND home

ownership AND 1930• “cord blood collection” NOT

banking

Not-so-successful Queries

• Broad terms• birds• Japan• Psychology

• Source types• books• journals

• Questions• Do animals have emotions?• Is a vaccine harmful to you?• What are the risk factors for

bipolar disorder?

WHAT ARE USERS ACTUALLY ENTERING IN THE SEARCH BOX?

QUALITATIVE

VIRGINIA TECHUNIVERSITY LIBRARIES

E-preferred policy implemented in 2011

Reduced print collections 2012 - 2013 Reclaimed and revitalized space 2014

Summon introduced in the fall 2011

ASSESSMENT & ANALYSIS

Usability study

Google Analytics

COUNTER statistics

MIXED METHOD STUDY

Task Analysis and Usability StudyTotal of 10 participants

2 Focus GroupsTotal of 8 Interviews

Online Survey470 Participants

ONLINE USER SURVEY FEEDBACK

In 2012 we received 470 respondents to the online survey.

In 2013 we received 789 respondents.

Longitudinal data provided an understanding of the experiences and perceptions of the Summon service over time.

2012 Survey 2013 Survey

PREFERRED RESEARCH TOOL

Addison (library catalog)

Google Scholar Summon

34%

27%

47%

27%29% 28%

8%11%

8%

High Preference Moderate Preference Low Preference

Addison (library catalog)

Google Scholar Summon

29%28%

36%

32%34%

37%

7%

12%

8%

High Preference Moderate Preference Low Preference

SEARCH & RETRIEVAL EFFECTIVENESS

Strongly Agree Agree Disagree Strongly Disagree

32%

45%

14%

9%

23%

46%

21%

10%

2012 2013

FOCUS GROUP FEEDBACK

Provide more instruction on how to use Summon• Filtering searches• Using search terms• Using Summon features

Make changes to the Library home page• Make the Library home page customizable• Make Addison the default on the home page

Directly link to the article• Reduce the number of steps to get to the actual article

STUDENT VOICES

“Summon is great if I know the article I’m looking for.”

I like that I only have to search in one place.”

“I need more skill at searching”

I get so many hits that are not useful.”

I introduce Summon as the ‘Google’ of the library subscription data bases.

It would be nice to have a drop-down menu providing options to search by author, title, keyword, subject, LC number, etc.

Easy and convenient tools are not always effective.

The power of Summon is in faceted searching.

Need ‘true subject headings’.

INSTRUCTION LIBRARIAN VOICES

GOOGLE ANALYTICS

Search library content and more

Printed scholarly books

Perception and imaging: Photography a way of seeing

978-0-387-797113

Agroecology

Nutrition care manual

iPhone

Web of science

Pubmed

Search library content and beyond

Types of search terms frequently used

COUNTER

IMPACT

Information Literacy

• Significant time spent on teaching Summon

• Shift focus from teaching tools to research process

• Power of faceted & Boolean searching

• Emphasize effective search management skills

• Focus on competencies for evaluating the quality & relevance of results

Summon Development

• Improved click through to full text

• Simplified ILL GET VTech page

• Searches available by LCCN

• “Best bets” include links to databases

• Third preview panel added to better assess resource prior to clicking through to full text

VIRGINIA TECH ASSESSMENT IMPACTS

Information Literacy

• Students are finding and using EDS

• Teach students (and faculty) how to use EDS more effectively

• IL Assessment Plan• Learning outcomes

(face-to-face + online)• General Education

Curriculum• ACRL IL Standard Two +

Performance Indicator 2

EDS Development

• Integration with other campus systems

• Improved relevancy of search results, especially for typos

• Text call numbers• Tools to connect to non-

indexed materials• Widgets• Customized placards

• What can we learn from other discovery tools?

IU KOKOMO ASSESSMENT IMPACTS

Source: American Library Association. (2000, January 18). Information Literacy Competency Standards for Higher Education. Retrieved from http://www.ala.org/acrl/standards/informationliteracycompetency

Tracy GilmoreCollections Assessment

LibrarianVirginia Tech

tgilmore@vt.edu

Angie ThorpeDigital User Experience

LibrarianIndiana University Kokomo

atthorpe@iuk.edu

QUESTIONS

Virginia Tech University Relations Photo Library http://www.photolibrary.unirel.vt.edu/pages/

http://pixabay.com/en/pay-numbers-arrangement-circle-530338/http

://pixabay.com/en/feedback-handicraft-creativity-520527/

http://pixabay.com/en/domino-circuit-element-concept-163522

/

IMAGE CREDITS