roswitha poll münster, germany ten years after: „measuring quality“ revised
TRANSCRIPT
![Page 1: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/1.jpg)
Roswitha PollRoswitha PollMünster, GermanyMünster, Germany
Ten years after:„Measuring Quality“ revised
![Page 2: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/2.jpg)
2/21
MeasuringQuality
1st edition 1996 English, German, French, Russian, Italian only academic libraries 17 indicators
![Page 3: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/3.jpg)
3/21
MeasuringQuality
2nd revised edition 2007 academic and public libraries 40 indicators Authors: Peter te Boekhorst, Roswitha Poll
![Page 4: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/4.jpg)
4/21
MeasuringQuality: Why?
So what are you doing at the moment?Writing a book about performance indicators for libraries
What’s that, performance indicators? Methods for seeing whether the library is good
And how do you know whether you are good? By asking users, by measuring whether we are doing things right
For example? That all the books are always in their right place
Yes, that’s important. And such things show you whether you are good?Well, not quite, but I can compare with other libraries
And if they are worse, you know that you are good?That’s about it
Okay, sounds easyBut why do you need a book for
that?
![Page 5: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/5.jpg)
5/21
Changes in the library world
electronic services predominant the teaching library as new role cost-effectiveness crucial competition of the institutions library as physical place widens to meeting and communication
![Page 6: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/6.jpg)
6/21
Trends in performance measurement
stakeholders‘ views quantitative and qualitative measures standardisation has proceeded - ISO 11620 will comprise e-measures user surveys find broad acceptance impact/outcome measures
![Page 7: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/7.jpg)
7/21
More use of performance measures?
More data available
For electronic services - resources- expenditure- usage
In national statistics- reference transactions- data for library space- attendances at user training and events
But:• Visits still only physical visits• Digitisation missing
![Page 8: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/8.jpg)
8/21
More use of performance measures?
Joint benchmarking projects
BIX –Library IndexBIX –Library IndexGerman public and academic librariesGerman public and academic libraries
CASL CASL (Council of Australian State Libraries)(Council of Australian State Libraries)public librariespublic libraries
Swedish Quality HandbookSwedish Quality Handbookall types of librariesall types of libraries
HELMS HELMS (UK Higher Education(UK Higher Education Library Library Management Statistics)Management Statistics) academic librariesacademic libraries
Netherlands University LibrariesNetherlands University Libraries
![Page 9: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/9.jpg)
9/21
More use of performance measures?
Joint benchmarking projects
Projects: Projects: 55
Indicators: Indicators: 5555
Corresponding to ISO 11620: Corresponding to ISO 11620: 2525
Used in more than one project: Used in more than one project: 2020
Individuality obvious
![Page 10: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/10.jpg)
10/21
More use of performance measures?
Interactive national statistics allow combinations of data (e.g. per capita) Less reports from individual libraries But: service level agreements with performance indicators on websites Performance measures have been integrated
Time to re-shelve: 100% in 6 hours
![Page 11: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/11.jpg)
11/21
The handbook:structure
Balanced Scorecard chapter on impact/outcome bibliography for each indicator examples for possible scores
![Page 12: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/12.jpg)
12/21
The handbook:How to choose indicators
The set of indicators should cover the full range of library services
consider traditional and electronic services if possible „merged“ indicators include indicators for „potentials and development“ consider the view of different stakeholders
![Page 13: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/13.jpg)
13/21
Library quality:Stakeholder views
UsersAccess to information worlwideDelivery of information to the desktopSpeed and accuracy of deliveryGood in-library working conditionsResponsiveness of staffReliability of services
![Page 14: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/14.jpg)
14/21
Library quality:Stakeholder views
Financing authoritiesCost-effectivenessClear planning, effective organisationPositive outcome on usersBenefits for the institution’s goalsEffective cooperation High reputation of the library
![Page 15: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/15.jpg)
15/21
Library quality:Stakeholder views
StaffGood working conditionsClear planning, straight processesSystematic staff developmentHigh reputation of the library
![Page 16: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/16.jpg)
16/21
The handbook:How to choose indicators
Only one indicator for one question?Example: How intensively is the collection used?Indicators:
collection turnoverloans per capitapercentage of stock not used
Each indicator gives a different picture and might serve different purposes of the evaluating library
![Page 17: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/17.jpg)
17/21
The indicatorsA. Resources, infrastructure: What does the library offer?
Library as place forlearning and research
1. User area per capita
2. Seats per capita
3. Opening hours compared to demand
Collections 4. Expenditure on information provision per capita
5. Availability of required titles
6. Percentage of rejected sessions
7. Ratio of requests received to requests sent out in interlibrary lending
8. Immediate availability
Staff 9. Staff per capita
Website 10. Direct access from the homepage
![Page 18: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/18.jpg)
18/21
Opening hours compared to demand
DefinitionThe actual number and time of opening hours compared to opening hours as desired by users Method: SurveyPossible formula: A/B A = the number of present opening hours
B = the number of hours which the users state that they need (present hours + additional hours)
If a library opens 60 hours per week and in the survey users ask for 10 hours more, the score would be 60:70 = 0.86
![Page 19: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/19.jpg)
19/21
Immediate availability
DefinitionThe percentage of immediate loans of total loans Method:Immediate loans = loans minus reservationsTotal loans = loans plus ILL loans received
Question: The probability that a user’s loan request will be fulfilled immediately
![Page 20: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/20.jpg)
20/21
Direct access from the homepage
DefinitionThe availability of the most frequently used resources and services via the homepage of the library’s website, measured by the number of clicks necessary and the comprehensibility of the terms usedMethodCognitive walk-through: A small group of experts simulates user behaviourQuestionWhether the homepage leads directly or very quickly to the most frequently needed information
![Page 21: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/21.jpg)
21/21
Direct access from the homepage
Academic libraries
AddressAddress
Way to the libraryWay to the library
Opening timesOpening times
Online catalogueOnline catalogue
Lending serviceLending service
User cardUser card
User accountUser account
ILL/document deliveryILL/document delivery
ReferenceReference
DatabasesDatabases
E-journalsE-journals
Subject accessSubject access
User trainingUser training
News, eventsNews, events
![Page 22: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/22.jpg)
22/21
Direct access from the homepage
Public libraries
AddressAddress
Way to the libraryWay to the library
Opening timesOpening times
Online catalogueOnline catalogue
Lending serviceLending service
User cardUser card
User accountUser account
Electronic collectionElectronic collection
Link collectionLink collection
ReferenceReference
FeesFees
Services for childrenServices for children
Branch librariesBranch libraries
Community servicesCommunity services
News, eventsNews, events
![Page 23: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/23.jpg)
Service Clicks Points
direct information on the homepage, e.g. times open, address, catalogue search
0 10
unmistakable term on the homepage
1 8
unmistakable term on the homepage
2 6
unmistakable term on the homepage
3 4
unmistakable term on the homepage
3 0
ambiguous term on the homepage 1 2
ambiguous term on the homepage 1 0
Access via the homepage
![Page 24: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/24.jpg)
24/21
The indicatorsB. Use: How are the services accepted?
General 1. Market penetration
2. User satisfaction
3. Library visits per capita
Library as place
4. Seat occupancy rate
Collections 5. Number of content units downloaded per capita
6. Collection use (turnover)
7. Percentage of stock not used
8. Loans per capita
9. Percentage of loans to external users
Information services
10. Attendances at training lessons per capita
11. Reference questions per capita
Cultural activities
12. Attendances at events per capita
![Page 25: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/25.jpg)
25/21
Library visits per capita
DefinitionThe total number of library visits per year, either physical or virtual, by members of the population, divided by the number of persons in the populationMethodPhysical visits: turnstileVirtual visits:
web browser visitsweb browser visits
IP visitsIP visits
homepage visitshomepage visits
![Page 26: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/26.jpg)
26/21
Library visits per capita:A method for counting homepage visits
Counting the deliveries of a Counting the deliveries of a Transparent GIF Transparent GIF
placed on the home page for counting purposesplaced on the home page for counting purposes
The number of deliveries of the pixel is identical The number of deliveries of the pixel is identical
to the number of page deliveries. to the number of page deliveries.
The advantage of this method is that it easy to use. The advantage of this method is that it easy to use.
The disadvantage is that access to other pages The disadvantage is that access to other pages
is not counted.is not counted.
German Benchmarking Project BIX http://www.bix-bibliotheksindex.de/
![Page 27: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/27.jpg)
27/21
The indicatorsC. Efficiency: Are the services offered cost-effectively?
General 1. Cost per user
2. Cost per visit
3. Cost per use
4. Ratio of acquisitions costs to staff costs
Collection costs 5. Cost per database session
6. Cost per download
Processes - speed
7. Acquisition speed
8. Media processing speed
9. Employee productivity in media processing
10. Lending speed
11. Interlibrary loan speed
Processes - reliability
12. Correct answer fill rate
13. Shelving accuracy
![Page 28: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/28.jpg)
28/21
Cost per use
DefinitionThe total operating or recurrent expenditure of the library during the reporting year divided by the number of loans + in-house use + downloads from the electronic collection Methodloans = lending systemin-house use = samplingdownloads = suppliers and library servers
Question The indicator assesses the library costs per case of collection use and therewith the cost-efficiency of library services
![Page 29: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/29.jpg)
29/21
The indicatorsD. Potentials and development: Is the library fit for future?
Electronic services
1.Percentage of acquisitions expenditure spent on the electronic collection
2. Percentage of library staff providing end developing electronic services
Staff development
3. Attendances at training lessons per staff member
Budget 4. Percentage of library means received by special grants or income generation
5. Percentage of institutional means allocated to the library
![Page 30: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/30.jpg)
30/21
Attendances at training lessons per staff member
DefinitionThe number of attendance hours of staff members at formal training lessons during one year divided by the total number of library staffMethod- number of attendants at each training- duration of a trainingA training of 3 hours with 12 attendants would be 36 attendance hours.
Question What priority does the library give to staff training and therewith to the library’s ability to cope with development?
![Page 31: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/31.jpg)
31/21
Staff training: What has been counted?
Hours of training per staff memberDays of training per staff member (hours calculated in days)Percentage of staff members who received training during the yearNumber of training lessons per staff memberTraining hours as percentage of total staff working hoursExpenditure for staff training
![Page 32: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/32.jpg)
32/21
Measuring Quality: What we tried to do
indicators for all
services
new indicators
practical examples for each indicator
„merged“ indicators
find the ultimate set of measures
??
bibliography for each indicator
![Page 33: Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised](https://reader035.vdocuments.site/reader035/viewer/2022081515/56649ea45503460f94ba860d/html5/thumbnails/33.jpg)
It is difficult to catch a black cat in a dark room especially when it is not there
Chinese Proverb