measuring the impact of information literacy instruction: a starting point for moving forward
DESCRIPTION
Presentation made by Lorna Dodd, User Services Manager, University College Dublin Library, at ANLTC Seminar "Library Impact and Assessment", held on Tuesday, 7th May 2013 at Trinity College Dublin Library.TRANSCRIPT
Measuring the impact of information literacy instruction: A starting point for moving forward
Lorna DoddUser Services Manager, UCD [email protected]:@LornaDodd
Why?
…snapshot of impact to date & best path forward
Looking back
Large University with no single strategy for
the development of Information Literacy
Deliver a heavy load of information skills,
driven by individuals
Sometimes we evaluate but we rarely measure. with no clear indication in meeting learning outcomes
Looking back
Looking ForwardNew Library Strategic Plan 2010-2014 aims to “monitor, measure and evaluate the Library’s teaching and learning strategy and activities.” (UCD Library, 2010)
As a result there has been a realignment of
staffing structure.
We have shifted from module to programme approach
And from a responsive to consultative role
A key goal is to measure our impact on student learning
Focus on process &
outputs
Limitations of traditional
approach in measuring success
Important to use results of any study which measures
the impact of Library Service
Focus on Library use rather
than instruction
Common tools include
surveys, behavioural observation and pre/post tests
Literature Review
Challenges
Data protection rules
Measuring long-term impact
It takes time to measure impact of Library Instruction
Promote new service
Identify what has been successful
Gather feedback from those not engaged
Identify useful methods going forward
Measuring impact retrospectively
Perceived impact is subjective
External factors influence skills development
Opt for the familiar
Survey all module coordinators
Promotion
Survey Results
Yes 43%
No 35%
Don’t know 22%
Has the Library ever delivered instruction in your module?
35%
Lecture
33%
Workshop
25%Tour
7%Other
What kind of instruction did your students have?
Impact of Library Instruction
“Students have always received instruction to
date and so difficult to answer the
comparison questions below”
Should ALL students have Library Instruction?
Yes86% Don’t
Know9%
No 5%
Should ALL students have Library Instruction?
Yes96% Don’t
Know3%
No 2%
Those who had Library Instruction
Should ALL students have Library Instruction?
Yes77% N0
14%
Don’t Know 10%
Those who did NOT have Library Instruction
Should Library Instruction happen in every year?
“Instruction in Year one should be
built on in
subsequent years to
enhance skills by year”
Preferred Services
120
Subject specific
workshops
62
Library Tours by
peer mentors
58
Generic workshops
50
Online tutorials
48
Online instructional
videos
45
Tutorials
First choice for undergraduates
66
Generic workshops
52
Tutorials
50
Online instructional
videos
45
Online tutorials
42
Subject specific
workshops
36
Library tours with peer mentors
Second choice for undergraduates
First choice for taught postgraduates
154
Subject specific
workshops
62
Library Tours by peer mentors
57
Generic workshops
53
Online instructional
videos
51
Online
tutorials
Influence of instruction on preference?
Influence of instruction on preference?
Influence of instruction on preference?
“…create materials that are reusable and for
any student, regardless of level. Just because a student moves up a level
doesn't mean they've acquired all of the
learning they should have”
“…really good online video
materials, available on demand is the way
to go”.
Qualitative Feedback 20% of respondents
Themes
In line with
strategy
Online Options
Current & Future
Services
Comments on service
Comments on survey
“…same 'introductory' talks
by librarian using lecture slots of
various modules for at least three times
(the third time he skipped
as he found it ridiculous)
“Students just skip the
guest lecture.”
“…requires personal contact with a friendly face
early on.
“we really love our librarians; thank you”
“…it makes a noticeable difference in the
quality of assignment students produce. The
Library Tutorial is typically rated as one of the
most important lectures they have in that Stage 1 module.”
Implications for the Future
Impact of Instruction
Definite benefit in Library instruction
Stronger evidence in discovery and use of resources
Active learning tailored to specific subject needs
Tours perceived as ineffective
Preferred delivery of instruction
Asked to identify how
effective instruction was
at improving students’ information literacy skills.
When they request instruction, do they usually consider this?
Questions
More QuestionsDid respondents choose workshop environment over online version due to familiarity?
Then why did those who had no instruction choose workshop?
If delivering workshops on this scale is unsustainable,
What can we do ensure effectiveness?
What work do we need to do in terms of marketing?
Blagden, P. (2005) ‘The LIRG/SCONUL Measuring Impact Initiative: Overview of phase 1 impact projects’, Library & Information Research (LIR) vol. 29 (91)
Dodd, L. & Kendlin, V. (2010) ‘Damned if we do and Damned if we don’t: How to address sustainability in the delivery of information literacy components in UCD’: Librarians' Information Literacy Annual Conference (LILAC), March 29-31, Limerick, Ireland.
Markless, S & Streatfield D. (2006) ‘Gathering and applying evidence of the impact of UK university libraries on student learning and research: A facilitated action research approach’, Journal of Information Management vol. 26 pp. 3-15
Poll, R & Payne, P. (2006) ‘Impact measures for libraries and information services’, Library Hi Tech vol. 24 (4) pp. 547-562
Schilling, K. & Applegate, R. (2012) ‘Best methods for evaluating education impact: a comparison of the efficacy of commonly used measures in library instruction’, Journal of the Medical Library Association, vol. 100 (4) pp. 258-269
Stone, G. et al (2011) ‘Does Library use affect student attainment? A preliminary report on the library impact data project’ Liber Quarterly vo. 21(1) pp. 5-22
UCD Library (2010) ‘UCD Library Strategic Plan 2010-2014’
http://www.ucd.ie/library/news_publicity/showcase_strategy/
Lorna Dodd: ANLTC 2013
References
Off the starting blocks http://www.flickr.com/photos/nickuzma/2570878552/
why can't i breathe whenever i think about you? http://www.flickr.com/photos/breannajosephine/5098047755/
Looking Into the Past - Custom House http://www.flickr.com/photos/coreytempleton/5893927056/
To go to http://www.flickr.com/photos/rubensmj/5399226001/
Literature review http://www.flickr.com/photos/carowallis1/4297460415/
Nissan Qashqai Challenge London http://www.flickr.com/photos/parmiter/2521273890/
Playing chess http://www.flickr.com/photos/bburt33/5930699148/
Surveying for Central Station, 1900 http://www.flickr.com/photos/state-records-nsw/4010230662
Shouting kid http://www.flickr.com/photos/jamessarmiento/4353936770/
Results of elections http://www.flickr.com/photos/ntr23/4650249185/
Four stages of the moon http://www.flickr.com/photos/leehaywood/6607585765/
Meteor crater http://www.flickr.com/photos/homer4k/113472442/
Writing http://www.flickr.com/photos/jjpacres/3293117576/
Used books http://www.flickr.com/photos/babblingdweeb/23816128/
Quality Barnstorm http://commons.wikimedia.org/wiki/File:Quality_Barnstar_v2.0.png
Apples & Oranges - They Don't Compare http://www.flickr.com/photos/thebusybrain/2492945625/
Lorna Dodd: ANLTC 2013
Images
Watermelon http://www.dreamstime.com/stock-photography-watermelon-image22521832
Orange http://www.dreamstime.com/royalty-free-stock-photo-orange-image23480475
Strawberry on white background http://www.dreamstime.com/royalty-free-stock-photo-one-strawberry-white-background-image28694065
One gooseberry fruits isolated http://www.dreamstime.com/stock-photography-one-gooseberry-fruits-isolated-image20607952
Construction Tunnel http://www.flickr.com/photos/joshuarothhaas/2250657031 Online http://www.flickr.com/photos/orkomedix/5398470372/
Conversation http://www.flickr.com/photos/stijnnieuwendijk/3098445189/
Backyard fresh http://www.flickr.com/photos/jim_belford/8544552591/
Abandoned amphitheatre http://www.flickr.com/photos/pixarman/5456444943/
Lego love http://www.flickr.com/photos/evaysucamara/5419530024/
Student working http://www.flickr.com/photos/92399131@N07/8399835572/
Future http://www.flickr.com/photos/maniya/4020026753/
Punch http://www.flickr.com/photos/timcaynes/148110505/
teach me http://www.flickr.com/photos/giovannamatarazzo/182108786
Question proposal http://www.flickr.com/photos/eleaf/2536358399/
Light Bulb http://www.flickr.com/photos/polifemus/561986434/
walk ahead and cross the sky http://www.flickr.com/photos/ldawei/4920681478/
Lorna Dodd: ANLTC 2013
Images