measurement and assessment of library programs and services eileen e. hitchingham dean, university...
TRANSCRIPT
Measurement and
Assessment of Library Programs
and Services
Eileen E. Hitchingham Dean, University LibrariesVirginia Tech ([email protected])
The Changing Face of Libraries SOLINET 2002Atlanta, GAMay 2, 2002
Measurement and
Assessment of Library Programs
and Services
Eileen E. Hitchingham Dean, University LibrariesVirginia Tech ([email protected])
The Changing Face of Libraries SOLINET 2002Atlanta, GAMay 2, 2002
Q: How many statisticians does it taketo change a light bulb?
A: One -- plus or minus threeStatistics means never having to say
you're certain.
In God we trust... All others must bring data
Did you hear about the statistician
who had his head in an oven and his
feet in a bucket of ice?
When asked how he felt, he replied,
"On the average I feel just fine."Sources:GARY C. RAMSEYER'S FIRST INTERNET GALLERY OF STATISTICS JOKES, http://www.ilstu.edu/~gcramsey/Gallery.html, Statistics?!, Don’t Make Me Laugh! http://www.keypress.com/fathom/jokes.html
To ShowTo Show Library Visits
(Jul -Nov)
0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
98/99 99/00 00/01
oror To Know?To Know?
Library Library Statistics...Statistics...
0
10,000
20,000
30,000
40,000
97/98 98/99 99/00 2000/01 2001/02
Info/dir Ref
The University Library
Ptolemaic Information SystemPtolemaic Information System
Faculty
Grad
UGrad
Sack, J. Open Systems for Open Minds: Building the Library Without Walls (College & Research Libraries 47(6): 535 -44, 1986
VT Community of Users
Copernican Information SystemCopernican Information System
University Libraries
Other Information
Sources
The Web
Assessment at VT Libraries• This year every position description in the library notes that the staff
member is responsible for collecting and using data to support continuous process improvement
• Assessment goals at the library:
– to better understand the wants, needs, and information seeking behaviors of our user community so that we can provide them with information collections and services that support their research, teaching, learning and outreach goals
– to benchmark our processes and services against other libraries so that we can consider emulating operations that appear likely to contribute to making us more effective or efficient, and
– to continuously improve the processes and services associated with acquiring, organizing, providing access to, preserving, and assisting users in working with, information resources.
Examples of What We Examples of What We CollectCollect
• OPAC searches• Circulations• In-house use of
materials• Library Visits• Reference
interactions
• Storage items used• Interloan data from
ILLiad• Web hits, pages, etc.• Peer group data• Instruction
Evaluations• LibQUAL+TM
0
100,000
200,000
300,000
400,000
500,000
600,000
1997 1998 1999 2000 2001 2002
Author Subject Title Call Number Boolean Keyword Total searches
OPAC searchesOPAC searches
(Nov - Feb, kw and boolean counted as of Nov 99)
CirculationsCirculations
0
60,000
120,000
180,000
1997 1998 1999 2000 2001 2002
(Nov - Feb, initial and renews)
OPAC Searches and CirculationOPAC Searches and Circulation
0
100,000
200,000
300,000
400,000
500,000
600,000
1997 1998 1999 2000 2001 2002
OPAC searches External Circulations
(Nov-Feb)(Nov-Feb)
Circulation Rates at Service LocationsCirculation Rates at Service Locations
0
5,000
10,000
15,000
20,000
25,000
30,000
A&Arch GeoSci Media Ctr Storage Vet Med NVC
97 98 99 2000 2001
(Jul - Feb, initial checkouts and (Jul - Feb, initial checkouts and renews)renews)
Circulation at Main DeskCirculation at Main Desk
0
50,000
100,000
150,000
200,000
250,000
Main Desk
97 98 99 2000 2001
(Jul - Feb, initial checkouts and (Jul - Feb, initial checkouts and renews)renews)
Internal Use of Materials Internal Use of Materials
79,075 77,71083,639
0
10,000
20,000
30,000
40,000
50,000
60,000
70,000
80,000
90,000
98/99 99/2000 2000/01
Internal bks and bnd jrs Current Sci sers Current other sers
Microforms Government Docs Ref not sci
Newspapers Ref sci Maps
Spec Collections
Library Visits and Library Visits and UsesUses
Jul-Nov
0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
800,000
98/99 99/00 00/01
Physical Materials Use Gate
Web Pages Delivered From Web Pages Delivered From Library Web - OctoberLibrary Web - October
0
100,000
200,000
300,000
400,000
500,000
600,000
96 97 98 99 2000 2001
(October data, excludes Digital Library and ILLiad)
Storage: Items Requested and Storage: Items Requested and Items in StorageItems in Storage
330,605
450,654
524,900560,917 568,601 572,263
0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
95/96 96/97 97/98 98/99 99/00 00/01
Items in Storage Items Requested
Storage: Types of Items RequestedStorage: Types of Items Requested
0
6,000
12,000
18,000
24,000
95/96 96/97 97/98 98/99 99/00 00/01
Bks requested Articles requested Total items requested
ILLiad Everywhere EvaluationILLiad Everywhere Evaluation (Fall 2001 Customers)(Fall 2001 Customers)
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Very bad Bad Adequate Good Very Good
Interloan Doc Deliv from VT campus
Illiad: Illiad: Number of Journals Titles Needed to Number of Journals Titles Needed to Deliver a Percentage of 32,664 ArticlesDeliver a Percentage of 32,664 Articles
30 82163
275419
603
831
1,122
1,486
1,930
0
200
400
600
800
1,000
1,200
1,400
1,600
1,800
2,000
10Pct
20Pct
30Pct
40Pct
50Pct
60Pct
70Pct
80Pct
90Pct
100Pct
Jour
nal T
itle
s
Rethinking Operations and Staff Rethinking Operations and Staff Assignments Assignments
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
97/98 98/99 99/00 2000/01 2001/02
Info/dir Ref
First Qtr
Staff Resources by Service AreaStaff Resources by Service Area
0% 5% 10% 15% 20% 25%
TS
Ref
Instruc
Coll Mgmt CL Shelv
Systems
Branches
Dig Lib
Admin
ILL
Circ/Res
Bus Services
Instruction and Outreach DataInstruction and Outreach Data
Projected Total Library Expenditures Projected Total Library Expenditures
$0
$5,000,000
$10,000,000
$15,000,000
$20,000,000
$25,000,000
$30,000,000
1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999
VT Top 21 -30
LibQUAL+TM 2001
• 2002 is third year of LibQUAL+TM
• Participants include ARL, ACRL, medical libraries, community colleges
• Measures user perceptions of library services• 3 questions overall satisfaction, satisfaction way treated,
satisfaction with access to collections• 25 questions address service issues of library as place,
affect, personal control, access to collections• Measures minimum acceptable service, perceived level
of service, desired level of service
General Satisfaction Questions -General Satisfaction Questions -20012001
6
6.5
7
7.5
8
8.5
9
Overall quality of service Satisfied with way treatedin libraries
Satisfied with support forlearning, research,
teaching
VT UGRAD VT GRAD VT FAC All LibQUAL
Gap score Affect: Desired minus PerceivedGap score Affect: Desired minus PerceivedLibQUAL+LibQUAL+TMTM 2001 2001
0
0.5
1
1.5
employeesinstill
confidence inusers
giving usersindividualattention
emplyees dealwith users incaring fashion
employeeswho are
consistentlycourteous
willingness tohelp users
employeeswho have
knowledge toanswer userquestions
dependabilityin handling
users' serviceproblems
employeesunderstandneeds ofusers
ready torespondquestions
VT Ugrad VT Grad VT Fac
Gap score Personal Control: Desired minus Perceived
0
0.5
1
1.5
makingelectronicresoucesaccessible
from home oroffice
library websiteenabling me to
locateinformation on
my own
makinginformation
easilyaccessible forindependent
user
modern eqptthat lets meeasily access
theinformation I
need
convenientaccess tolibrary
collections
easy-to-usetools allow meto find thingson my own
VT Ugrad VT Grad VT Fac
Gap score Library as Place: Desired Gap score Library as Place: Desired minus Perceivedminus Perceived
0
0.5
1
1.5
haven for quietand solitude
contemplativeenvironment
place forreflection and
creativity
comfortable andinviting location
space thatfacilitates quiet
study
VT Ugrad VT Grad VT Fac
0
0.5
1
1.5
2
2.5
3
convenientbusiness hours
timely documentdelivery/ill
comprehensiveprint collections
interdisciplinarylibrary needsaddressed
complete runs of jrtitles
VT Ugrad VT Grad VT Fac
Gap scores Access Collections: Desired minus Perceived
VT LibQUAL+TM 2001
0
0.5
1
1.5
2
2.5
3
convenientbusiness hours
timely documentdelivery/ill
comprehensiveprint collections
interdisciplinarylibrary needsaddressed
complete runs of jrtitles
VT Ugrad All Ugrad VT Grad
All Grad VT Fac All Fac
Gap scores Access Collections: Desired minus Perceived
VT and All LibQUAL+TM 2001
THANKS!THANKS!