is there a statistically significant relationship between library resource access and student...
TRANSCRIPT
The Open University Library Data Project
Is there a statistically significant relationship between library resource accesses and student attainment at the Open University, a distance learning institution?
Anne Gambles, Project Manager
The Northern Collaboration Conference 2016-09
Open University Facts and Figures1.89 million people worldwide have studied with the OU
2
173,899UK students
>21,000people with
disabilities studied with the OU in
2014/15.
76%of OU students
work full or part-time during
their studies
8353Overseas students
30%of new OU
undergraduates are under 25
21% of OU UK
undergraduates live in the 25% most deprived
areas
3
85%surveyed in the 2016
National Student Survey agree library
resources and services are good
enough for their needs
1,295,001 million
downloads from Open Research Online. The
Open University's open access repository of research publications
51kUnique visitors to
the OU Library website per month
Award winning
Digital and information literacy tools for
students
1stUK HE to provide a 24/7 webchat
serviceResource richness
600+ databases & collections,
134k+ electronic journals and over
200k ebooks
Open University Library Facts and Figures
Outline
Project
Background
Library Data
Strategy
Library Data
Project overview
Library Data
Progress and findings
4
5
Early stepsExploring the potential of library data 2010-
• RISE Jisc-funded project in 2010 investigating the potential of using library activity data from EZproxy to create an amazon-like recommender service for a discovery system. Part of the Jisc Activity Data programme
• Working with Jisc Library Analytics and Metrics Project (LAMP) and the Jisc Learning Analytics library community panel
• Keeping up-to-date with work on projects in other HEIs exploring the relationship between library use and student success
6
BackgroundExploring the potential of library data 2013-2014
Needham, Gill; Parker, Jo; Nurse, Richard; Scantlebury, Non and Dick, Sam (2013).
Can an excellent distance learning library service support student retention and how can we find out?
Open Learning, 28(2) pp. 135–140. http://dx.doi.org/10.1080/02680513.2013.847364
Article in Open Learning 2013 and
Poster at Design4Learning conference 2014
7
BackgroundExploring the potential of library data 2014-2015
• Working with institutional Learning Analytics project• Providing library data to module teams• Menu of potential ‘data-driven’ interventions include library
activities
Why
University of Huddersfield 2013‘There is a statistically significant relationship between library resource use and level of degree result’
University of Minnesota 2014‘Students who utilize academic libraries within their first year have
higher cumulative grade point averages and retention, on average, than their peers who did not use library services.’
Library Analytics
“Using data about student engagement with library services and content to help institutions and students understand and improve library services to learners.”
Other UK HEIs report a correlation between student success /
retention and library use – is this the case at the OU?
8
9
Strategy
Project overview
Strategy Evidence impact Integration Skills Infrastructure Business as usual
Data on library use by students not
integrated into institutional data
spaces
No strong evidence base of relationship between
library use and retention/achievement at
OU
Library Services lack data analysis
skills
Challenges
Focus areas
10
11
How can library data benefit students, faculties and the University as a whole?
Analysing OU Library
data alongside OU data
will help to:
•Better understand student success factors; it may also help to improve the accuracy of student success predictions•Provide evidence and understand the impact of students’ engagement with independent study•Inform improvements to library services and trigger interventions for disengaged students
12
Intended outcomes
We can demonstrate that library engagement enhances student achievement and retention
Library engagement data is available for use across the OU and routinely being used where key decisions are made about learning
Library Staff are confident with using data to show how library use contributes to student success and retention
Faculty can be confident that library engagement improves student achievement and retention
Faculty can access real-time library data to identify student engagement
Faculty can be confident that library staff can use data to support retention & achievement
Contribute to improved student success: increased
retention and completion
Progress and findings
13
Skills (library staff)Readiness assessment
14
Strongly Disagree NeutralDisagree Agree Strongly
Agree
Strongly Disagree NeutralDisagree Agree Strongly
Agree
Strongly Disagree NeutralDisagree Agree Strongly
Agree
Strongly Disagree NeutralDisagree Agree Strongly
Agree
Strongly Disagree NeutralDisagree Agree Strongly
Agree
I am confident that I know what analytics data the library has and I
know how to access it
I am confident that I know what analytics data the university has
and I know how to access it
I am confident in my ability to use simple analytics tools
I am confident in my ability to analyse data
I am confident in my ability to use data to support decision making
Strongly Disagree NeutralDisagree Agree Strongly
AgreeOVERALL
High
Feb 2016 SDH
Confidence levelLow
Overall confidence in finding and using data is low.Library staff will need more help to find and use library and university data.Once data tools are in place then they will need help to build skills in analysing and using data.
Online library resource accessesData on online library resource accesses is taken from EZproxy and OpenAthens systems
15
Online library resource accesses
16
• Starting point URL only
• Data files include students who are studying more than one module
• Data files include data from students who didn’t complete their module
Level 1 Level 2 Level 3 PG0
5
10
15
20
25
30
35
40
7.23
13.22
23.25
33.44
Online library resource access by level (all faculties, 2014 presentations)
Mean per student
AVER
AGE
NU
MBE
R O
F AC
CESS
ES O
F LI
BRAR
Y RE
SOU
RCES
PER
STU
DEN
T (M
EAN
)
17
Online library accesses and student achievement
Deferral
Fail
Pass
Distinction
0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0 20.0 22.0
n = 40
n = 2070
n = 6434
n = 1029
Mean number of library resources used per student - 3 level 1 modules (Arts, Law, Technology)
2014-15 presentations
Number of library resource accesses per student (mean)Library Data project 2015/16
n=11,501n=11,406
Pilot study:
18
Pilot StudyLibrary use and Age
i Under 25 ii 26-35 iii 36-45 iv 46-55 v 56 and over0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
n = 3382
n = 3805
n = 2000
n = 1324
n = 990
Age groups - average use of online library resources - 3 level 1 modules (Arts, Law, Technology) 2014-15
AV
ER
AG
E N
UM
BE
R O
F L
IBR
AR
Y R
ES
OU
RC
ES
A
CC
ES
SE
D
19
Pilot StudyLibrary use and gender
Female Male0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
n = 6157
n = 5344
Average use of online library resources by gender - 3 level 1 modules (Arts, Law,
Technology) 2014-15
AV
ER
AG
E N
UM
BE
R O
F L
IBR
AR
Y R
ES
OU
RC
ES
A
CC
ES
SE
D
20
Pilot Study Library use and level of educational qualification on entry
1 No Formal Qual-ifications
2 Less than A Levels
3 A Levels or equivalent
4 HE Qualification 5 PG Qualification 6 Not known0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
16.00
n = 291 n = 3888 n = 4839
n = 2085
n = 370
n = 28
Average use of online library resources by previous education - 3 level 1 modules (Arts, Law, Technology) 2014-15
AV
ER
AG
E N
UM
BE
R O
F L
IBR
AR
Y R
ES
OU
RC
ES
AC
CE
SS
ED
Library Data Project 2016
n=11,501
21
Online library resource accesses and student achievementWider study:
Fail
Pass
Distinction
0.00 5.00 10.00 15.00 20.00 25.00 30.00
n = 46300
n = 176268
n = 29041
Mean library resource accesses per student - 2014 and 2015 presentations (all modules)
Mean resource accesses per student
n = 315,460
22
Online library resource accesses and student achievementWider study:
23
Online library resource accesses and student achievementWider study:
The Wollongong methodology
24
University of Wollongong Library
Cox, B. L. and Jantti, M. (2012) Capturing business intelligence required for targeted marketing, demonstrating value, and driving process improvement, Library & Information Science Research. 34, PP308-316 doi:10.1016/j.lisr.2012.06.002
• Looked at whether there is a relationship between library resource accesses and student success
• Aimed to demonstrate Library impact• The approach and methodology was tested with OU data with some
small adjustments.• UoW used time accessing resources (from the full ezproxy logfiles),
OU has used count of accesses ezproxy and openathens for 2014 students
• UoW used Weighted Average Marks – OU used Overall Continuous Assessment Score (OCAS)
Replicating the Wollongong study
25
Replicating the Wollongong study
26
The average continuous assessment score for students who did not access OU Library online resources is 64. The average continuous assessment score for students that accessed one resource during the year was 68.
There is a non-linear correlation between average resource accesses and average student Score (R squared = 0.54)
Replicating the Wollongong study
27
Findings so far
●The percentage of students accessing library resources and the number of library resources they access on average increases the higher the level of study
●There is quite a wide variance between different faculties, levels and modules in access to library resources
●There seems to be a similar relationship between library access and student success at the Open University, despite students’ mainly online library access
●Early indications are that the relationship is statistically significant
Key indications from our early studies are:
28
29
What’s next?
Questions?Project Blog: www.open.ac.uk/blogs/LibraryData/
30