demonstrating the "l" factor, virginia power - cilip ireland/ lai joint conference 2015
TRANSCRIPT
![Page 1: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/1.jpg)
Demonstrating the ‘L’ factor: providing quality impact &
evaluation evidence
Virginia Power,Graduate Tutor/Researcher in Information Management & Science, University of the West of England, Bristol
Some statistics provided by Carol Parker, Barton Peveril College, Hampshire
![Page 2: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/2.jpg)
Drawing on research from the UK, the US and Scandinavia, the LISU study shows a world in which libraries are actively engaged in gathering evidence to demonstrate their value – but challenges remain. Though a good deal of evidence is collected, much of it is evidence of activity rather than evidence of value and impact. Barr, S. (2012) How should academic libraries communicate their own value? Guardian Online http://www.theguardian.com/higher-education-network/blog/2012/aug/20/academic-libraries-value-research-teaching
Evolving Value for Libraries
![Page 3: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/3.jpg)
![Page 4: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/4.jpg)
Your motivation. Identify the key reasons why you want to engage in a proving and/or improving process, your expectations about the process, ‘windows of opportunity’, the available resources, leadership issues and the appropriate tools or methods.
Your vision. Set forth the organisation’s mission, values, and objectives. Identify its major activities.
Stakeholders. Know who is affected by or affects your organisation in all of its activities, whether intentionally or not.
Impact Map. Specify what the organisation does and how it does it; how it intends to achieve its social, environmental/economic mission as well as its financial sustainability; and how it plans to live up to its values.
Measuring the Library
![Page 5: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/5.jpg)
Identifying how well a service fulfils its purpose and adds value to an organisation
Accountability and justification of financial spending
Identifying areas for improvement Monitoring progress against goals Comparing standards with other organisations Monitoring effects of past decisions Monitoring interactions with customers and
suppliers
Purposes of Performance Measurement
![Page 6: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/6.jpg)
Have YOU read and understood the inspection framework for your institution?
![Page 7: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/7.jpg)
‘Distance travelled’ is the description given to the change in ‘soft outcomes’, that is, those achievements that cannot be directly measured in the same way as ‘hard outcomes’ such as qualifications, earnings or employment status. Mostsimply, ‘distance travelled refers to the progress that a beneficiary makes towards employability or harder outcomes’. It is easy to measure an individual’s educational progress, for example, as this can be a number of qualifications or ajump from level 2 to level 3. It is much more difficult to measure progress in social skills or motivation, although these are as important for an individual’s life chances.
Soft Outcomes for Universal Learning (SOUL)
Are YOU able to demonstrate the distance travelled by the students that you work with in Library & Information Services?
![Page 9: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/9.jpg)
The library needs to be able to identify and show progress
Identify both intended and unexpected outcomes.
Prioritise what is most important to measure Choose the ways in which you will see
change happening.
Indicators
![Page 10: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/10.jpg)
What do you consider to be the most effective Key Performance Indicators?
![Page 11: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/11.jpg)
![Page 12: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/12.jpg)
Choose KPIs that align with service and organisational aims/objectives
Review KPI annually How will they be measured/monitored? Choose quality over quantity Adopt a range of methods to measure
performance How much time/money/tools do you have to
measure/monitor PI?
Choosing Performance Indicators
![Page 13: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/13.jpg)
Make a plan. After prioritising the indicators, decide on methods for collecting information relating to the indicators.
Collect data. Collecting information can be done in a number of ways as outlined in your methodology or plan, as long as it can be stored and drawn upon usefully.
Analyse information. Seeing what the data you collect tells you can happen throughout the process, and can also be summed up at the end of a cycle to make some conclusions.
Planning
![Page 14: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/14.jpg)
Objectives
You are trying to understandWhether, how and what impact occursWhat the outcome of this impact is for the people concernedWhat the broader impact of this evidence might be
Before you begin to collect evidence decideWhy you are seeking this evidenceWhat kind of information you will collectWho will collect itWhen it will be collectedHow to use the results What do you want to find out?
Keep it simple and focusedWhat are the learning outcomes of the information skills session?What new knowledge did students learn and how did this knowledge increase their understanding?How did using the library change attitudes towards libraries and learning?
![Page 15: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/15.jpg)
What do you consider to be the most effective methods of user feedback?
![Page 16: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/16.jpg)
I Already Do This….
Review the data you collect alreadyComment booksTeacher's evaluation sheetsLetters from usersFootfall countsUser surveysProject evaluation formsOther...
For each data source, ask the question: does it give evidence of impact?YesNoMaybe?
![Page 17: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/17.jpg)
Discussion at meetings
Comments and suggestions
Feedback forms
Staff voice meetings
One-to-one sessions
Surveys
Boards of study
Annual survey
Liaison role
Course committees
Focus groups
0 2 4 6 8 10 12 14 16 18 20
Q.4 Feedback methods from internal users
Percentage of respondents
Feedback methods
Feedback Methods from Internal Users
![Page 18: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/18.jpg)
Most Effective Method for Gathering User Feedback
Percentage of respondents
Notice boards 9.0
Student union 23.9
Social media 26.9
Student council 26.9
National surveys 29.9
Online feedback mechanisms 35.8
Organisational/institutional surveys 44.8
Formal complaints/feedback procedures 47.8
Focus groups 53.7
User surveys 82.1
Most Effective Methods for Gathering User Feedback
![Page 19: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/19.jpg)
Simple Informal Methods
![Page 20: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/20.jpg)
Library managers need to ensure that they recognise the concept of the internal customer within their organisation. This is because
“errors in the service provided within an organisation will eventually affect the product or service which reaches the external customer” (Slack et al, 2004, p. 723).
Measuring Internal Feedback
![Page 21: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/21.jpg)
![Page 22: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/22.jpg)
TICSI (no date) states that benchmarking encourages a service to “become open to new methods, ideas, processes and practices that improve effectiveness, efficiency and performance in customer service”
Further Education – CoLRIC Higher Education and Research – SCONUL
and National Student Survey
Benchmarking
![Page 23: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/23.jpg)
Balanced scorecard CoLRIC – performance and impact indicators
/ peer accreditation scheme EFQM – European Foundation for QM Inspiring Learning for All SCONUL benchmarking LibQUAL+ Prove and Improve Comparison Chart Return on investment – ROI Calculator
Performance Measurement Tools
![Page 24: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/24.jpg)
Balanced Scorecard
Business ProcessesStreamlining processesNew products/suppliers
Supply chain
Staff skills and expertiseTraining needs
Customer satisfactionMeeting customer needs
Cash flowCosts
Budgets
Balanced scorecard
![Page 25: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/25.jpg)
Return on Investment
Olson (2002, p.19) states “information services are considered valuable when they are perceived to be equivalent to the funds spent on them”.
ROI Calculator
![Page 26: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/26.jpg)
Library Performance Measurement in the UK and IrelandAssociation of Research Libraries, 2009
Library Performance Measurement
![Page 27: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/27.jpg)
Sharing information. Make what you have learnt a part of a dialogue or conversation, offering different stakeholders the opportunity to learn from the process and feed their thoughts and opinions into the next round.
Learning & Action. A proving and/or improving process is a chance to do things better in the future. Make what you have learnt part of a plan for improvement –the necessary actions, changes and the resources needed to enable the desired changes.
Sharing
![Page 28: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/28.jpg)
Informal Communication Methods
![Page 29: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/29.jpg)
Service level agreement
Annual report
Quarterly report
Self assessment review
Business report
0 10 20 30 40 50 60
Q.7 How do you communicate the results of per-formance measurement?
Percentage of respondents
Methods of Com-munication
Communicating Performance Measurement
![Page 30: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/30.jpg)
Performance measurement and impact assessment is time consuming and difficult to do.
Anecdotal evidence suggests that even when mechanisms are put in place to measure performance e.g. feedback and customer satisfaction - it isn't always forthcoming!
Services need to use a variety of methods to get good results.
Library Managers need a range of skills to support this.
![Page 31: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/31.jpg)
What skills, knowledge & understanding do you consider to be the most important for librarians to develop in this field?
![Page 32: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/32.jpg)
Communication Skills Marketing Skills Promotional Tools Systematic collection of evidence Analytical skills Social Informatics Understanding of big data
We need ….
![Page 33: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/33.jpg)
The role of data informing decision-making New approaches to collecting, analysing
and using data Using analytics to develop new services and
improve the user experience The opportunities of library data as big data Role of library analytics uncovering new
opportunities to demonstrate impact and value
The future is bright….
![Page 34: Demonstrating the "L" Factor, Virginia Power - CILIP Ireland/ LAI Joint Conference 2015](https://reader030.vdocuments.site/reader030/viewer/2022032505/55c634c3bb61eb9f668b465f/html5/thumbnails/34.jpg)
Thank You – further questions?
Virginia Power,Graduate Tutor/Researcher in Information Management & Science, University of the West of England, Bristol [email protected]