LibQUAL+ Triads: Results from the Pilot Experiences

10th Northumbria International Library Performance Measurement Conference

Royal York Hotel, UK

Monday, July 22nd 2013

www.libqual.org

LibQUAL+™

Martha Kyrillidou

Association of Research Libraries

Bruce Thompson

Texas A&M University Libraries

Overview

• Introduction to LibQUAL+

• Why Triads

• Trustworthiness

• Pilot activities

• Results

• Future exploration

13 Libraries

English LibQUAL+™ Version

4000 Respondents

QUAL

QUAN

QUAL

QUAL

QUAN

QUAL

PURPOSE DATA ANALYSIS PRODUCT/RESULT

Describe library

environment;

build theory of library

service quality from

user perspective

Test LibQUAL+™

instrument

Refine theory

of service quality

Refine LibQUAL+™

instrument

Test LibQUAL+™

instrument

Refine theory

Unstructured interviews

at 8 ARL institutions

Web-delivered survey

Unstructured interviews at

Health Sciences and the

Smithsonian libraries

E-mail to survey

administrators

Web-delivered survey

Focus groups

Content analysis:

(cards & Atlas TI)

Reliability/validity

analyses: Cronbachs

Alpha, factor analysis,

SEM, descriptive statistics

Content analysis

Content analysis

Reliability/validity analyses

including Cronbachs Alpha,

factor analysis, SEM,

descriptive statistics

Content analysis

Vignette

Re-tooling

Iterative

Emergent 2000

2004 …. 315 Libraries English, Dutch, Swedish,

German LibQUAL+™ Versions

160,000 anticipated respondents

LibQUAL+™ Project

Case studies1

Valid LibQUAL+™ protocol

Scalable process

Enhanced understanding of

user-centered views of service

quality in the library

environment2

Cultural perspective3

Refined survey delivery

process and theory of service

quality4

Refined LibQUAL+™

instrument5

Local contextual

understanding of

LibQUAL+™ survey

responses6

Association of Research Libraries

Dimensions of Library Service Quality

Information Control

Library Service Quality

Self-Reliance

Equipment

Timeliness

Ease of Navigation

Convenience

Scope of Content

Affect of Service

Library as Place

Reliability

Assurance

Responsiveness

Empathy

Refuge

Symbol

Utilitarian Space

Why LibQUAL+ Triads

• The key paradox of library service quality

assessment is that we want to focus our

assessment efforts on the service quality

criteria that are most highly desired by

users, and yet the process of measuring

primarily highly-desired behaviors in and

of itself inherently creates challenges to

differentiating user perceptions of what

libraries are most and least successfully

doing.

LibQUAL+ Suite

• LibQUAL+® Long

• LibQUAL+® Lite

• LibQUAL+® Triads

• Bruce Thompson and Martha Kyrillidou. “An Introduction to the LibQUAL+ Triads

Protocol: Using Ipsative Measurement to Assess Highly Desired Outcomes.” Concept

paper presented at the 9th Northumbria International Conference on Performance

Measurement in Libraries and Information Services, York, England, August 22, 2011.

<https://www.libqual.org/documents/LibQual/publications/LQ_triad.pdf>

Survey Composition

• 22 Core Questions

– Affect of Service

– Information Control

– Library as Place

• 5 Optional Questions

• Information Literacy

• General Satisfaction

• Demographics

• Free-text Comments

Box

LibQUAL+ Lite

Sampling

Methodology:

• All questions asked

• Respondents answer a

random sample only

• Reduced Survey Time

• Increase Response Rates

Survey Composition:

• 8 Core Questions

• 1 Local Question

• 1 Information Literacy

• 2 General Satisfaction

• Demographics

• Free-Text Comments Box

Trustworthiness

• LibQUAL+® Long and LibQUAL+® Lite do

incorporate some methods for excluding

untrustworthy data from users.

• The LibQUAL+® Triads protocol offers yet

another way to evaluate the

trustworthiness of a given library's service

quality data, so that service quality

improvement decisions will be based only

on meaningful scores.

Ipsative measurement

• Ipsative measurement collects data such

that responses to a given item constrain

choices on other items. Items of this sort

have forced-choice features, such as

requirements to rank order choices or to

allocate a fixed number of points across a

set of items (Thompson 2008, p. 21).

Example

Please allocate exactly 10 points across the

following three items that reflect how

important these three elements of library

service quality ("0" = not important at all) are

to you: (a) employees who deal with users in

a caring fashion; (b) a library web site

enabling me to locate information on my

own; (c) community space for group learning

and group study.

Advantage

• We prefer measurement methods to honor

the ecology in the real-world where the

measured decisions are made. Forced-

choice measurement has appeal when the

real-world ecology is itself a forced-

choice world.

Protocol description

• The protocol yields data on six

LibQUAL+® core items. The six core items

are presented in all possible combinations

of item triads (i.e., three items at a time).

• On the Triads perception protocol survey

respondents are asked for each of the

triads,

– At which one of the following 3 things is your

library doing BEST? And at which one of the

following 3 things is your library doing

WORST?

ASU Initial Screen

www.libqual.org

Second Screen

www.libqual.org

Third Screen

www.libqual.org

Comparing Score Pairs

• Arizona State

Pilot Libraries

Phase I:

New Mexico State U.,

Towson U., and

U. of Dallas

Phase II:

Arizona State,

Merrimack College,

Pennsylvania State,

Radford University,

U of Central Florida,

U of Manitoba,

West Virginia U, and

York University Libraries [Canada]

Questions

Phase I

A - The electronic information resources I need

B - Employees who deal with users in a caring fashion

C - Library space that inspires study and learning

D - Making electronic resources accessible from my home or office

E - A library Web site enabling me to locate information on my own

F - Print and/or electronic journal collections I require for my work

Phase II

A - The electronic information resources I need

B - Employees who deal with users in a caring fashion

C - Library space that inspires study and learning

D - Employees who have the knowledge to answer user questions

E - A library Web site enabling me to locate information on my own

F - Print and/or electronic journal collections I require for my work

Results

• We increased the observations used to calculate

the intra-individual correlations from 15 pair

comparisons to 3x15 = 45 pair comparisons to

maximize the information in the data

• Finally, persons whose intra-individual reliability

coefficients indicate that their scores are not

trustworthy (perhaps < 0.6 or 0.7) and will be

omitted from the second and final report

delivered to libraries.

Example Table included in the report

Example graph included in the report

www.libqual.org

Pilot Libraries presentations

• LibQUAL+® Triads Pilot at Arizona State University

Lisa Kammerlocher, Presentation, American Library Association Midwinter Meeting,

Seattle,

• LibQUAL+® Triads Pilot -- Fall 2011 at New Mexico State University

Norice Lee, Presentation, American Library Association Midwinter Meeting, Seattle,

Washington, January 28, 2013. Washington, January 28, 2013.

• LibQUAL+® Triads Pilot: Results from the University of Manitoba Libraries

Marie Speare, Presentation, Library Assessment Conference, University of Virginia,

Charlottesville, VA, October 29-31, 2012.

• Protocols and Priorities, Comparing Radford University Users' Priorities using

LibQUAL+® Long and Triads Survey Data: A Preliminary Study

Eric Ackermann, Poster, Library Assessment Conference, University of Virginia,

Charlottesville, VA, October 29-31, 2012.

• York University Libraries Implementation of LibQUAL+® Triads

Dany Savard, Marcia Salmon, and Aaron Lupton, Presentation, Library Assessment

Conference, University of Virginia, Charlottesville, VA, October 29-31, 2012.