usability, the user experience & interface design: the role of reference july 30, 2013
TRANSCRIPT
Usability, the User Experience & Interface Design: Usability, the User Experience & Interface Design:
The Role of ReferenceJuly 30, 2013
Evaluating user behaviors and experiences Evaluating user behaviors and experiences
Summon discovery service at Virginia Tech
2
ByTracy Gilmore & Lesley Moyo
3
IntroductionIntroduction
• Continual assessment necessary in providing proactive user support
• At Virginia Tech we endeavor to gather data on user behaviors every spring
• Study of user experiences with discovery services is our current focus
• Summon is our choice of Web scale discovery services
Collecting Data and Observing BehaviorsCollecting Data and Observing Behaviors
• Virginia Tech has adopted a mixed methods approach to studying user research behaviors
1. Surveys
2. Focus Groups & One-on-one interviews
3. Usability task analysis
• This helps triangulate the data and gives us grounds for making strong inferences
4
Unveiling the DataUnveiling the Data
• Using a robust survey tool is very helpful
• We used Qualtrics Research Suite: www.qualtrics.com
• Qualtrics offers strong data analysis features
• We crunched some of the survey data using SPSS to get more detailed analysis
• NVIVO for analysis of interviews and focus group data
6
What we learned about user behavior and preferences What we learned about user behavior and preferences
Survey Results – User PreferencesSurvey Results – User Preferences
Preference for Summon:
• Stronger than Google Scholar
• Stronger than the Library Catalog
7Easy to use Moderately easy to use Neither easy nor difficult Moderately difficult use Difficult to useHigh preference Moderate preference Neither high nor low preference Somewhat low preference Low preference
User Preference - 2012
User Preference - 2013
2012 Survey results - Predictors of satisfaction2012 Survey results - Predictors of satisfaction
8
Chi Square
Degrees of freedom
P-value
My rating of Ease-of-use of Summon
Chi Square
Degrees of freedom
P-value
Overall I am very satisfied with Summon
41.40
16
0.00
I always use the facets in Summon to improve my search strategy
Chi Square
Degrees of freedom
P-value
Overall I am very satisfied with Summon
246.68
16
0.00
I can find articles from top ranked journals in my field
Ease of use & Satisfaction Facets use & Satisfaction
Top ranked journals & Satisfaction
Overall I am very satisfied with Summon
229.90
16
0.00
2013 Survey results - Predictors of satisfaction2013 Survey results - Predictors of satisfaction
9
Chi Square
Degrees of freedom
P-value
My rating of Ease-of-use of Summon
Chi Square
Degrees of freedom
P-value
Overall I am very satisfied with Summon
54.90
16
0.00
I always use the facets in Summon to improve my search strategy
Chi Square
Degrees of freedom
P-value
Overall I am very satisfied with Summon
436.74
16
0.00
I can find articles from top ranked journals in my field
Ease of use & Satisfaction Facets use & Satisfaction
Top ranked journals & Satisfaction
Overall I am very satisfied with Summon
263.08
16
0.00
User voices - Open ended questionsUser voices - Open ended questionsComments 2012
• I like that I only have to search in one place
• On several occasions Summon has found some good papers that Google Scholar didn't
• Summon is great if I know the article I’m looking for
• I need more skill at searching
• I get so many hits that are not useful
Comments 2013
• The faceted searching (I didn’t know it was called that) is absolutely essential
• Ability to get many different types of sources with one search
• I found it to be more helpful when looking for books than articles
• Even with facets, Summon casts too broad a net for my needs
10
User voices - Interviews/Focus groupsUser voices - Interviews/Focus groups
Comments
• Summon is easy to use
• Facets are a great tool
• Too many results!
• Boolean operators please
• Training would be nice
Suggestions
• Provide direct linking to articles
• Provide more instruction on how to use Summon:
• Using facets• Using Subject Terms
11
Task Analysis observations Task Analysis observations
• Good results achieved with advanced search
• Some participants able to effectively use facets
• Effective use of quotations for known items
Challenges
• Page navigation
• Evident lack of familiarity with Summon features
• Large result sets still an issue
• Mixed success rates in task performance
• Get VText (360 link resolver)
12
Strengths
Enter librarian perspectivesEnter librarian perspectives
• In spring 2013 - We included librarian voices
• This provided us with comparative perspectives
13
How do we as librarians interpret user behaviors, given our own biases of what research processes should look like?
Librarian voicesLibrarian voices
14
More students use it as a starting place, so when troubleshooting that's what's involved more often than not
Summon has been especially helpful with interdisciplinary topics even in identifying specific databases to more fully explore
Fast way of gathering extensive information and familiar to the "Google generation”
I am a very happy user of Summon. The interface, the single user box, and the facets are intuitive and easy to use
Changes implemented from findingsChanges implemented from findings
• Better integration in the instruction program especially at the freshman level
• Improvements to the interface - better direct linking to full text
• Increase in the content index - emphasis on eBooks and reference sources
• Better visibility on the Library homepage
15
Going forward - Continuing assessment and evaluationGoing forward - Continuing assessment and evaluation
• Implementation of Summon 2.0 (anticipated fall 2013)
• Analysis of usage statistics from Google Analytics added to Summon in November 2012
• Expansion of Mixed Methods study to academic research libraries utilizing the Summon discovery service (fall 2013)
Contact informationContact informationLesley Moyo
Director for Research & Instructional Services
University Libraries
Virginia Tech 560 Drillfield Drive
Blacksburg, VA 24061
Tel: 540-231-2708
Mobile: 814-777-7565
E-mail: [email protected]
Library Web: www.lib.vt.edu
Tracy Gilmore
Collections Assessment Librarian
University Libraries
Virginia Tech 560 Drillfield Drive
Blacksburg, VA 24061
Tel: 540-231-4956
Mobile: 540-239-4384
E-mail: [email protected]
Library Web: www.lib.vt.edu
17