what do users think of us? mining three years of cul libqual data

1

What Do Users Think of Us? Mining Three Years of CUL LibQUAL Data

Liane O’Brien, Linda Miller, Xin Li

May 21, 2008

2

LibQUAL at CUL

CUL administered 4 rounds of the survey In 2001, CUL was a developing partner (pilot phase) Total respondents for 2002-2005 was 1,742

287 faculty 707 grads 748 undergrads

The response rate has trended downward from 20% in 2002 to 15% in 2005

3

Why go back to past LibQUAL data?

It reveals how CUL has done in four areas that matter to users: Human touch Library as place User empowerment Access to materials/resources

These data will help us to: Understand better where CUL has done well or not well over time Identify areas for improvement Spot changes and early signs of new trends

4

Methodology of this analysis

Focused on the core questions Excluded 2001 data due to pilot-nature Selected only the questions asked in all three surveys

(with a few exceptions) Analyzed the “Perceived, Desired, & Minimum” ratings

for each group

5

LibQUAL borrows some assumptions from the business world

Users will choose the most satisfactory/efficient services that fits their personal needs

We can’t change our users’ expectations, but we can change what we offer

Whether we changed enough is reflected in users’ responses

6

LibQUAL ratings variables

Nine point scale: 1 = lowest , 9 = highest

The level the user wants

The lowest service level the user can accept

Where the user perceives the library service is now

7

Understanding the graph

User’s range of tolerance for a particular CUL service

User’s desire

Expectation mid-way or “average”

Minimum level user can accept

Where CUL is in user’s mind

Above average

Below average

8

Users’ average perception ratings

The scores are well above average

CUL seems to be doing better in 2005 than it did in 2003

Print and/or electronic journal collections I require for my work: Graduate students

4.55

5.56

6.57

7.58

8.59

2003 2005

Sco

re (1

-9,

9 h

igh

est)

9

User’s average perceptions ratings in the context of their expectations

Print and/or electronic journal collections I require for my work: Graduate students

4.55

5.56

6.57

7.58

8.59

2003 2005

Sco

re (

1-9,

9 h

igh

est)

… until you put them in the context of their expectations

Expectations have gone up and the tolerance range has narrowed

Both scores are close to the minimum level acceptable

The score is up in 2005, but we lost ground in meeting expectations

10

Human Touch services (Questions 1-9)

Willingness to help users (1) Dependability in handling user’s service problems (2) Employees who deal with users in a caring fashion (3) Employees who have the knowledge to answer user

questions (4) Employees who instill confidence in users (5) Employees who understand the needs of their users (6) Readiness to respond to users’ questions (7) Giving users individual attention (8) Employees who are consistently courteous (9)

11

Library As Place services (Questions 10-11)

Quiet space for individual activities (10) A comfortable and inviting location (11)

12

User Empowerment services (Questions 12-16)

Making electronic resources accessible from my home or office (12)

A library Web site enabling me to locate information on my own (13)

Modern equipment that lets me easily access needed information (14)

Easy-to-use access tools that allow me to find things on my own (15)

Making information easily accessible for independent use (16)

13

Access to Materials/Resources services (Questions 17-20)

Timely document delivery (17) The printed library materials I need for my work (18) Print and/or electronic journal collections I require for my

work (19) The electronic information resources I need (20)

14

How we analyzed the data

Overview data

Data by the four areasHuman TouchLibrary as a PlaceUser EmpowermentAccess

Question-specific data

15

Overall, CUL is doing very well

Most scores are well above the survey scale’s midpoint (5) Most scores increased from 2003

CUL has also faired well against its participating ARL peers

16

Overall, CUL users expect a lot

Although the survey score range is 1-9, the majority of all CUL scores are 6+, with a few in 5+

17

Services most important to user groups(desired service levels above 8 at least 2 times from 2002 to 2005)

Faculty

12, 13, 14, 15, 16, 20

2

9, 17

10, 11

Question #s: 1-9 Human TouchQuestion #s: 10-11 Library as a PlaceQuestion #s: 12-16 User EmpowermentQuestion #s: 17-20 Access

Undergrads

Grads

1, 4, 7, 18, 19

None

None

18

Perceived service levels in bottom half of user groups’ ranges of tolerance

ALL user types

Faculty only Faculty & Grads only

Grads only Grads & Undergrads

only

Undergrads

only

2005 12, 13, 15, 19

18 4, 11, 14, 16

10, 20

2003 12, 15,

19, 20

13, 18 10 16

2002 12 18 4, 13,15, 16

10, 14

Question #s: 1-9 Human TouchQuestion #s: 10-11 Library as a PlaceQuestion #s: 12-16 User EmpowermentQuestion #s: 17-20 Access!

19

Perceived service levels in bottom quarter of user groups’ ranges of tolerance

ALL Faculty only Faculty & Grads only

Grads only Grads & Undergrads

only

Undergrads

only

2005 12, 13, 15, 19, 20

2003 12 15 19 10

2002 13 10 12

Question #s: 1-9 Human TouchQuestion #s: 10-11 Library as a PlaceQuestion #s: 12-16 User EmpowermentQuestion #s: 17-20 Access

!!

20

The Human Touch Area (9 services)

Over all years, only 3 perceived service levels did not fall above user group’s average expectations

Faculty and grads’ desired service levels were higher than undergrads’

Overall, more of faculty’s expectations were met

21

The Human Touch Area (9 services)

In 2005, services each group rated with 1 of their 4 highest desired service levels

Willingness to help users Employees who have the knowledge to answer user questions Readiness to respond to users’ questions Employees who are consistently courteous

In 2005, services each group rated with 1 of their 3 lowest desired service levels

Employees who instill confidence in users Giving users individual attention

Overall, services that met less of user groups’ expectations Dependability in handling users’ service problems Employees who have the knowledge to answer user questions

22

Employees who have the knowledge to answer user questions

Faculty desired service level rose from 2002 to 2005, while those for students fell

More of each groups’ expectations were met in 2005 than in 2002

23

Giving users individual attention

Faculty perceived service levels rose each year Students’ 2005 perceived service levels were down from 2002 In 2005, faculty and undergrads’ perceived service levels were 1.4

survey scale units apart

24

The Library as a Place Area (2 services)

Obvious differences between user groups: faculty desired service levels were lower, and a larger part of their

expectations were met for students, the opposite was true

More of all groups’ expectations were met for “A comfortable and inviting location” than for “A quiet space for individual activities”

25

Quiet space for individual activities

All user groups’ perceived service levels rose each year Undergrad’s desired service level rose from 2002 to 2005 (true for

only 3 of the 20 services covered) Undergrads’ range of tolerance narrowed each year Less of undergrads’ expectations were met each year; more of

faculty and grads’ expectations were met in 2005

26

User Empowerment Area (5 services)

Clearly an area of concern Desired service levels were high for all groups Many perceived service levels fell below groups’ average expectations

All groups’ scores are more similar to each other than in previous areas

27

Making electronic resources accessible from my office or home

All perceived service levels fell bellow groups’ average expectations; however, more of each groups’ expectations were met in 2005 than in 2003

Undergrads’ desired service level increased from 2002 to 2005 All groups’ scores are similar

28

A library website enabling me to locate information on my own

High desired service levels for all groups (all above 8) More of undergrads’ expectations were met than those of faculty

and grads Faculty and undergrads’ desired service levels dropped in 2005

29

Access Area (4 services)

High desired service levels for all groups (esp. faculty and grads) Overall, desired service levels for collections higher than for those

for document delivery / ILL Many perceived service levels for collection-specific services fell

bellow user groups’ average expectations

30

The printed library materials I need for my work

Faculty’s desired service levels higher than those of students in 2003 and 2005

Faculty and grad desired service levels rose each year; undergrads’ desired service levels dropped from 2002 to 2005

Less of each group’s expectations were met in 2005 than in 2002

31

The electronic information resources I need

Desired service levels high for all groups, but rising for grads in 2005, while falling for faculty and undergrads

Faculty and undergrads’ perceived service levels fell higher in their ranges of tolerance in 2005; grads’ perceived service level fell lower

32

Recent Library actions?

The Human Touch Library as a Place Space renovations in

various libraries Open up staff meetings

space for study after hours

Access GetIT Buy instead of borrow Favoring e-resources Scholarly Communications Large Scale Digitization

User Empowerment Web Vision Usability testing Loaner laptops and

equipments VIVO

?

33

Questions to Ponder

SWOT Analysis (Strengths, Weaknesses, Opportunities, and Threats) Where are we consistently doing well? Where are we consistently weak? Are these weak areas the Library’s opportunities to invest in, or, Are these areas we should give up or give over to competitors?

Would you use LibQUAL data to make decisions? Should we do LibQUAL again? If so, how often?

34

Questions?

Thank you very much!

ResearchAndAssessment@cornell.edu