drilling down: libqual quantitative and qualitative data allison sivak assessment librarian...
TRANSCRIPT
Drilling Down: LibQUAL Quantitative
and Qualitative Data
Allison Sivak
Assessment Librarian
University of Alberta Libraries
October 25, 2007
Assessment Challenges
• Gathering meaningful data• Acquiring methodological skills• Managing assessment data• Organizing assessment as a core activity• Interpreting data within the context of user
behaviors and constraints
Troll Covey, 2002
Major Trends
• Appetite for electronic access constantly increasing• Access and content are the same thing• Expectation that electronic access & searching is easy and
reliable• I need it, now is when I want it, as easy and as quickly as
possible, by myself, from wherever I am• Library space solution a challenge of conflicting
expectations – not yet solved• ILL service a problem with GSs, declining issue with UGs
(specificity vs. satisficing?)• Inconsistencies in customer service noticed and noted
How We’ve Responded
• Strong focus on improving electronic access– Proxy server replaced with seamless remote access– Link resolver– Aggressive e-collection development
• Hours• Study space zoning
– Space planning and renovation• Customer Service Standards• Notifications / request services
– User-initiated online holds, recalls, renewal– Retrieve and send material to library of choice
• Follow-up studies: web experience testing, collections data, e-books survey, faculty /partnership investigation
Ryan, 2006
4.00
5.00
6.00
7.00
8.00
9.00
2002 2003 2004 2005 2006 2007
UGGSFA
Overall Quality of Service Satisfaction, 2002 - 2007
4.00
5.00
6.00
7.00
8.00
9.00
2002 2003 2004 2005 2006 2007
UG GS
FA
Overall Satisfaction with Library Support for Learning, Research, and/or Teaching Needs, 2002 - 2007
4.00
5.00
6.00
7.00
8.00
9.00
2002 2003 2004 2005 2006 2007
UG GS
FA
Overall Satisfaction with Treatment at Library 2002 - 2007
Year 2004 2005 2006 2007
Cases 589 615 452 527
Desired Mean 7.947 7.896 7.984 7.906
Desired Minimum 5 4.727 5 1.591
Desired Maximum 9 9 9 9
Perceived Mean 7.01 7.018 7.124 6.935
Adequacy Gap Mean 0.483 0.545 0.529 0.372
Most Important Services: Information Control
Undergraduate Students Desired Score
Making electronic resources available from my home or office
8.30
Print and/or electronic journal collections I require for my work
8.20
Modern equipment that lets me easily access needed information
8.12
The electronic information resources I need 8.11 Easy-to-use access tools that allow me to find things on my own
8.10
Making information easily accessible for independent use 8.06 A library Web site allowing me to locate information on my own
8.01
Graduate Students Desired
Score Print and/or electronic journal collections I require for my work
8.58
A library Web site allowing me to locate information on my own
8.52
Making electronic resources accessible from my home or office
8.50
The electronic information resources I need 8.51 Making information easily accessible for independent use 8.38 Easy-to-use access tools that allow me to find things on my own
8.36
Modern equipment that lets me easily access needed information
8.15
The printed library materials I need for my work 8.06
Most Important Services: Information Control
Faculty Desired Score
Making electronic resources accessible from my home or office
8.66
Print and/or electronic journal collections I require for my work
8.64
The electronic information resources I need 8.54 A library Web site allowing me to locate information on my own
8.40
Making information easily accessible for independent use 8.37 Easy-to-use access tools that allow me to find things on my own
8.36
Modern equipment that lets me easily access needed information
8.15
Most Important Services: Affect of Service
Undergraduate Students Desired Score
Employees who have the knowledge to answer user questions
8.08
Employees who are consistently courteous 8.08
Graduate Students Desired Score
Adequate hours of service 8.23 Readiness to respond to users’ questions 8.20 Employees who are consistently courteous 8.18 Employees have the knowledge to respond to user questions
8.17
Dependability in handling users’ service problems 8.16 Willingness to help users 8.09
Faculty Desired
Score Dependability in handling users’ service problems 8.11 Employees have the knowledge to respond to user questions
8.04
Employees who are consistently courteous 8.01
Most Important Services: Library as Place
Undergraduate Students Desired Score
A getaway for study, learning, or research 8.03
Graduate Students - No ‘Desired level’ scores over 8.0 in this dimension.
Faculty - No ‘Desired level’ scores over 8.0 in this dimension.
Undergraduate Students 2002 2003 2004 2005 2006 2007 Library space that inspires study and learning n/a -1.30 -1.30 -1.25 -1.45 -1.86 Making electronic resources accessible from my home or office
-1.68 -1.43 -1.23 -1.28 -1.38 -1.33
Making me aware of library resources and services n/a n/a n/a n/a n/a -1.23 The electronic information resources I need n/a -1.14 -1.20 -1.17 -1.20 -1.20 Quiet space for individual activities n/a -1.28 -1.06 -0.87 -1.20 -1.18 Print and / or electronic journal collections I require for my work
n/a -1.32 -1.29 -1.31 -1.16 -1.18
A getaway for study, learning, or research n/a -0.92 -0.94 -0.88 -1.10 -1.17 Easy-to-use access tools that allow me to find things on my own
-1.18 -1.51 -1.11 -1.14 -1.00 -1.06
Teaching me how to access, evaluate, and use information n/a n/a n/a n/a -0.73 -1.02 A library website enabling me to locate information on my own
-1.08 -1.17 -1.04 -1.10 -1.08 -1.02
A comfortable and inviting location -0.65 -0.95 -1.16 -0.74 -1.02 -1.02 Modern equipment that lets me easily access the information I need
-1.19 -1.02 -0.85 -0.97 -0.98 -0.99
Dependability in handling users' service problems -1.11 -0.97 -0.87 -0.97 -1.03 -0.95 Making information easily accessible for independent use -1.08 -1.10 -0.88 -0.93 -0.83 -0.95 Employees who are consistently courteous -0.64 -1.13 -0.86 -0.91 -1.08 -0.84 The printed library materials I need for my work n/a -1.10 -0.98 -1.02 -0.97 -0.82 Employees who deal with users in a caring fashion -1.19 -0.97 -0.81 -0.74 -0.78 -0.74 Community space for group learning and group study n/a -0.97 -0.67 -0.52 -0.50 -0.68 Timely ILL / document delivery -1.03 -1.09 -0.88 -0.87 -0.96 n/a Convenient business hours -1.49 -1.20 -1.14 -1.03 -0.92 n/a Informing me of useful library services n/a n/a -0.82 -1.05 -1.02 n/a
Graduate Students 2002 2003 2004 2005 2006 2007
The electronic information resources I need n/a -1.40 -1.43 -1.39 -1.06 -1.38 Print and / or electronic journal collections I require for my work
n/a -1.72 -1.62 -1.47 -1.14 -1.27
Library space that inspires study and learning n/a -1.05 -1.13 -0.77 -0.87 -1.22
A library website enabling me to locate information on my own
-0.91 -1.20 -1.36 -1.08 -0.94 -1.20
Making electronic resources accessible from my home or office -1.45 -1.17 -1.25 -1.29 -0.95 -1.15
Easy-to-use access tools that allow me to find things on my own
-1.01 -1.47 -1.22 -1.10 -0.89 -1.14
Quiet space for individual activities n/a -0.90 -0.72 -0.73 -0.66 -1.10
Modern equipment that lets me easily access needed information
-0.96 -1.06 -0.92 -0.96 -0.76 -0.84
A getaway for study, learning, or research n/a -0.85 -0.88 -0.83 -0.65 -1.07
Making me aware of library resources and services n/a n/a n/a n/a n/a -1.05
The printed library materials I need for my work n/a -1.32 -1.15 -1.17 -0.72 -1.05
Making information easily accessible for independent use -1.04 -1.15 -1.04 -0.91 -0.71 -1.01
Adequate hours of service n/a n/a n/a n/a n/a -0.95
Ease and timeliness in getting materials from other libraries n/a n/a n/a n/a n/a -0.93
Employees who are consistently courteous -0.64 -0.92 -0.79 -0.81 -0.73 -0.93
Informing me of useful library services n/a n/a -1.03 -1.03 -0.74 n/a
Faculty 2003 2007
Ability to navigate library web pages easily n/a -1.75 Print and / or electronic journal collections I require for my work
-1.78 -1.71
A library website enabling me to locate information on my own
-1.30 -1.59
Easy-to-use access tools that allow me to find things on my own
-1.40 -1.55
The printed library materials I need for my work -1.41 -1.43
Making electronic resources accessible from my home or office -1.52 -1.40
The electronic information resources I need -1.21 -1.43 Ease and timeliness in getting materials from other libraries n/a -1.28 Dependability in handling users’ service problems -0.72 -1.11
Modern equipment that lets me easily access needed information
-1.07 -1.15
Adequate hours of service n/a -1.10
Library space that inspires study and learning -0.38 -1.11
Making information easily accessible for independent use -1.13 n/a
Timely document delivery / interlibrary loan -1.85 n/a
Convenient service hours -1.25 n/a
Convenient access to library collections -1.20 n/a
0
50
100
150
200
250
300
2003 2004 2005 2006 2007
GSUGFaculty
Number of Comments by User Group and Year
0
50
100
150
200
250
300
350
400
2003 2004 2005 2006 2007
Affect of Service
Information Control
Library as Place
Number of Comments by Broad Category and Year
0
100
200
300
400
500
600
Faculty GS UG
Affect of Service
Information Control
Library as Place
Total Comments by User Group and Broad Category
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2003 2004 2005 2006 2007
convenientservicesconnectivity
lack of room
group space
comfort
lighting
maintenance
wayfinding
aesthetics
noise
hours
equipment
Library as Place: Subissues
0%
20%
40%
60%
80%
100%
2003 2004 2005 2006 2007
general praise
general criticism
Library as Place: Proportions of Criticism / Praise
Affect of Service: Subissues
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2003 2004 2005 2006 2007
Affect of Service staff
Affect of Servicereference & instruction
Affect of Service generalpraise
Affect of Service generalcriticism
Affect of Service: Staff Courtesy
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2003 2004 2005 2006 2007
positive
negative
Size Matters
• Affect of Service wasn’t as far from “desired service level” for UAL users in quantitative ratings
• Library as Place not ranked above 8.0 in importance (except for undergraduates)– Length of comments suggest impact when
unsatisfied!• Unevenness of service: “Good but….”• Lengthy descriptions of outstandingly positive or
negative experiences
• Although I get good service most of the time, I have occasionally encountered a library staff member who seemed reluctant to expend the effort in fully assisting me. The worst example of such happened this 2006 Winter term when I was trying to access my [] reserve material at [] Library. This was my first time doing so and I was unaware that there were call numbers for reserve material. I'd misplaced my syllabus (which probably had that info) but I thought I could just obtain some staff assistance in finding the reserve material instead of making another special trip to []. The staff member I first approached at the reserve desk told me that I needed to find the call numbers. When I stood there looking confused, she showed me how to get to the webpage on her computer. Then she indicated that I should learn to find the call numbers on my own at a different computer (which were all full by the way). I would have appreciated it if she had walked me through ALL of the steps...especially considering that this was my first time, she was already at the right webpage to start searching, and there were NO OTHER STUDENTS IN LINE. In fact, I was the ONLY student being helped at that moment for the entire reserve desk and if the search had become longer than expected or if there were other people waiting for assistance, then I would have gladly taken myself elsewhere to continue it. I left the reserve desk feeling angry since I did not feel that circumstances warranted that type of treatment. Fortunately, when I finally returned to the reserve desk, I received better help from a different person the second time around. Not only did I obtain the reserve material, but she also helped me renew a couple of items on my account. I appreciated the fact that she didn't tell me to go somewhere else to do it on my own!
• This is a slightly difficult survey to respond to, because my overall concern is with the variability of service at [] Library. There is a core of highly professional and knowledgeable librarians and technicians who staff Rutherford; these are highly capable professionals who have greatly enhanced my own research, the quality of research at the University of Alberta generally, and who clearly provide highly conscientious stewardship over library collections and services, respectively. I pray that my students get them when they go to the library. But I know very well that this is purely the luck of the draw, and that it is quite possible that they will be helped by someone whose qualifications and abilities are significantly lower than what I am evoking here. Thus I would suggest that the library make it a top priority to make the *point of contact* librarians more universally professional. This really will have a major impact on whether students seek out the help of a librarian in doing research, and will, I believe, have a major impact on the quality of work they are able to do in our libraries. I talk up our libraries whenever I can, as I really believe that they are a big part of what makes the U of Alberta a great place to work or to be student. But seriously. I feel undercut in this advocacy when I hear tales of library staff with very little academic or professional formation helping students or researchers. I have been helped by a fair number of such folks myself, so I know very well that this is more than just student bellyaching.
• I have primarily used the [] library and feel I have to comment on the staff in this library. Any time I have required information or assistance they have been more than willing to help and this is greatly appreciated by myself and others. I know of students, in at least two other faculties, that specifically come to the [] library because the staff is more helpful and they prefer this library on campus. I believe this says much about the staff and we appreciated the friendly, courteous manner in which we are treated by these individuals.
• My biggest concern, however, is the out-of-control situation regarding students. The library's decision to allow food and drink has been disastrous, not only for the long-term safety of the collections, but for those of us who use the library for working purposes. Even the allegedly "silent" areas echo to the sound of munching and crunching, fiddling with food wrapping, and the pervasive smell of chinese food/burgers/fries etc. It is unbelievably distracting at best, and for those of us brought up to worry about insect damage and grease on pages, it's a situation which is genuinely distressing. Add to this the endlessly ringing cellphones, computer games, high-volume iPods, etc. and you have something which resembles a high school cafeteria more than a library. Can we stop the political nonsense about "accessibility" please? There is nothing "inaccessible" about a library with sensible rules. As it stands, the library is become "inaccessible" for those who need it most: students genuinely trying to study and faculty members desperate to make the most of that morning or afternoon of research time. We want our libraries back!
What Else?
SLIS Studies• Given, Lisa M. 2007. Faculty and librarians ﾕ perspectives on
academic space: Setting the stage for undergraduates ﾕ information behaviors. In The Library as Place: History, Community and Culture, edited by John Buschman and Gloria J. Leckie, 177-189. Westport, CT: Libraries Unlimited.
• Given, Lisa M. [In press]. Emotional entanglements on the university campus: The role of affect in undergraduates ﾕ information behaviors. In The Emotional Information Environment: The Emergent Affective Paradigm in Information Behavior Research and Theory [working title], edited by Diane Nahl and Dania Bilal. Medford, NJ: American Society for Information Science & Technology (Information Today, Inc.).
• Sadler, Elizabeth (Bess), and Lisa M. Given. 2007. Affordance theory: A framework for graduate students ﾕ information behaviors. Journal of Documentation 63, no.1:115-141.
What Else?
Internal Assessment Projects– new service model
• Single help / access services desk• Librarian and associate staff work side by side• Research referral• Self-sufficiency in circulation• Self-serve reserve and e-reserve
– Delving deeper into website• Assessment of user guides• Assessment of digitized collections• Creating web usability testing plan• Usability testing of link resolver
– Talking to institutions that score higher on Information Control
