Working with campus partners to integrate the library into students’ academic experience Jan Fransen ([email protected]) Service Lead for Researcher & Discovery Systems University of Minnesota Libraries Justifying the Library NISO Virtual Conference April 20, 2016
Working with campus partners to integrate the library into students’ academic experience
Jan Fransen ([email protected])Service Lead for Researcher & Discovery SystemsUniversity of Minnesota Libraries
Justifying the LibraryNISO Virtual ConferenceApril 20, 2016
Presenter
Presentation Notes
At the end of this presentation, you’ll find a link to our publications for this project thus far where you can read all about our results and our statistical analysis methods. And we’ll mention those results and methods today. But mostly we’ll focus on actions: The actions we took to get to this point, and the actions our analysis has prompted.
Undergraduate28,638
Professional3,761
Non-Degree3,365
Graduate12,046
Faculty3,709
Staff (all types)19,444
70,963 Twin Cities Students, Faculty, and Staff
Presenter
Presentation Notes
I work at a Research One institution, with large numbers of undergraduates, graduate students, and faculty members, in addition to professional and non-degree-seeking students and University staff. We are constantly balancing our responsibility to serve this broad swath of people with the fact that each one comes to the Libraries individual stories and information needs. We’ve come around to the fact that we need to know who, in general terms, is and who is not using the different services of the Libraries, and we need to know through more than anecdotes and observations about the comparative few who make their way to a reference desk.
Layers of Data
Libraries Data (13 Access Points)Circulation, Digital, Instruction, Reference, and Workstation
Office of Institutional Research Demographics DataCollege, Level, Major, Gender, Ethnicity, Age
Office of Institutional Research Performance DataGPA, Retention
Presenter
Presentation Notes
Making sense of library use is a story of “layers of data.” The base layer is the data that the library can collect and sometimes already collects. In fact, one argument you can make is that libraries should make use of the data they already collect! This includes circ data, digital resource usage, reference and instruction sessions, and workstation logins. The second layer represents the partnerships and collaborations that libraries need to develop to make sense of their own data. In our case, we created a strong and mutually beneficial relationship with the Office of Institutional Research. OIR has the access to the PeopleSoft tables needed to make sense of our data and let us know, in the aggregate, who our users are. The 3rd layer details how OIR took our data and created positive correlations between library use and student success measures. But it all starts with gathering library data …
Circulation Data• Aleph (now Alma)• ILL (both Loans and Articles)
Presenter
Presentation Notes
Loans Both new check-outs and renewals Gathered by extracting data from Aleph transaction records Internet ID and date of transaction About 45% = Renewal data ILL Requests Gathered by extracting data from ILLiad ILLiad ID and date of transaction Not all IDs were U of M Internet IDs
Digital Data• Databases• E-Journals• E-Books• Website/Other• Search
Presenter
Presentation Notes
Anytime someone logged into our digital resources with a U of M Internet ID Database logins E-Journal logins E-Book logins Website logins Catalog searches Due to IP based authentication, we did not track on campus usage of databases, e-journals, and e-books Estimate - Missing 10-20% of our traffic This is only initial point of access, not actual usage
Instruction Data• Workshops• Course Integrated• Online Intro to Library
Research tutorial
Presenter
Presentation Notes
Workshop registrations Captured by Drupal-based registration module Registration does not mean attendance Intro to Libraries I workshop Intro to Libraries II workshop Course-integrated librarian instruction Everyone registered for the course/section All students may not have been present
Reference Data• Online Reference• Peer Research Consultants
Presenter
Presentation Notes
Online reference transactions Captured from QuestionPoint data Some of the more difficult data to capture We did not capture ref desk traffic or research consultations Peer Research consulting data One-on-one assistance to develop research strategies U of M student consultants
Workstation Data• Logins
Presenter
Presentation Notes
U of M library workstation logins Captured by Cybrarian application used to authenticate library users Does not include complete data from SMART Learning Commons Reveals a flaw with regard to capturing “library as place” Difficult to gather Internet IDs if students don’t give them to us
Institutional Research Partnership
Presenter
Presentation Notes
By ourselves, we can basically count “how many” and not a lot more than that. For further analysis, we turned to the Office of Institutional Research. OIR collects and analyzes data to provide information for institutional planning, policy formation, and decision-making. We can’t stress enough how important this partnership has been. For us, again, OIR has access to the university PeopleSoft systems that include the information we need to find out who are users are and how library use affects their success. Key library data numbers (Fall 2011): 1,548,209 total transactions in 5 categories 61,195 unique Internet IDs
Outcomes• Academic achievement• Graduation• Development
Presenter
Presentation Notes
Our primary contact in OIR is Krista Soria. Krista’s background includes years as an academic adviser, teaching as an adjunct at the University and several other institutions, and a PhD in Higher Ed. Krista’s knowledge of how the use of student data in academia is evolving has been priceless. One concept she introduced us to was Astin’s I-E-O Model, for Inputs – Environment – Outcomes. In short, the idea is that we get students with certain pre-existing characteristics, they experience an environment over which we have at least some control, and there are outcomes of their experience that we can observe or measure. There are a number of environmental factors that have previously been correlated to student outcomes, and basically we were setting out to see whether library use was one of them.
2.784
2.628
2.8542.926
3.016
3.258
2.755
2.932
3.0793.141
3.224
3.503
2.5
2.7
2.9
3.1
3.3
3.5
3.7
Below 19 19-22 23-24 25-27 28-30 31-36ACT Score
Undergrad Term GPA – Fall 2011
Did not use library
Used library at least once
Presenter
Presentation Notes
This was our first inkling that there really was something to look for. We aggregated undergraduates into buckets by ACT score. Except for the very lowest scoring students, where the difference was negligible, students at every pre-college level as indicated by their ACT score, did better if they used the library than if they did not.
Ordinary Least Squares Regression
Demographics
Gender
Race/Ethnicity
Pell Grant Recipient
Veteran Status
First Generation college student
Prior Academics
ACT/SAT Score
AP credits
College Environment
Freshman Seminar
Access to Success program
Residence Hall
Holding all of these factors constant…
http://xkcd.com/539/
Presenter
Presentation Notes
Next Krista considered all those factors that have been shown to correlate to student outcomes and did some REAL statistical analysis. In addition to prior academics, she controlled for demographics and college environment factors that are known to correlate with student success factors.
Taking all that into account, we found
.23Increase in GPA associated with using the library one or more times
.07 Increase in GPA associated with each new type of library use
1.54Times more likely to re-enroll for Spring semester forstudents who used the library at least once
1.1Times more likely to re-enroll for Spring semester for each one-unit increase in type of library use
7.58Times more likely to re-enroll for Spring semester for students who participated in Intro to Library Research 2
1.03Times more likely to re-enroll for Spring semester for each one-unit increase in database use
Fall 2011 NHS
Presenter
Presentation Notes
Using statistical methods, Krista found….(the things you see on the slide)
Controlling for• Demographics • College experience • Classmate interactions• Library research skills• Critical thinking skills
Using the library at least once is significantly and positively associated with students’ scholarship
Using the library at least once is significantly and positively associated with students’ academic engagement
Presenter
Presentation Notes
In another study, controlling for all of those other factors and a few others, Krista was able to determine that using the library at least once was significantly and positively associated with both academic engagement and scholarship as measured through the Student Experience in a Research University (SERU) survey our campus does almost every year. Those are just two of several studies we’ve done since we started collecting data and doing analysis since 2011
• Our first work was with the first year students of Fall 2011
• Four years later…is 2015!
• So those who stayed on track all the way to graduation in four years would have graduated in May 2015
Cohort Love
• Using the library at least one time in the first year of enrollment significantly increased the odds that students would graduate in four years OR remain enrolled after four years
• First year students who used electronicresources and books had significantly improved odds of graduation in four years
Results
Presenter
Presentation Notes
We hope to publish these longitudinal cohort results soon.
Presenter
Presentation Notes
Now let’s talk about how we take all those interesting results and actually do something with them. First, I have to say that the studies we’ve done so far and the way we’ve engaged with the data as a library have enriched our interactions with students and our connections on campus in numerous ways that are real but not easily measured, at least right now. We are actively looking for ways, though, to use what we’re collecting to reach the students who need us most in the ways we find to be most effective.
I’m going to misuse Astin’s model for just a moment here to describe what I mean. Let’s say we want to measure a different outcome: Did they use the library or not?
Digital Circulation
InstructionReference
Library Interventions
Student Actions
Presenter
Presentation Notes
Considering the data we currently collect, let’s concentrate on two categories: Library interventions like instruction and reference services, and actions students take that we’re going to define as the Libraries’ success measures. In other words, what do we do that correlates to students using the Libraries’ resources?
Presenter
Presentation Notes
Our data clearly show that mediations like our Intro to Library Research workshop led to an increase in students’ use of databases during that semester. This chart shows that if the students don’t receive any of the instruction types that we collect, there was a 50/50 chance that they’d use the library. But 90 percent of the students who had instruction used the library. We’ve done some preliminary work that SEEMS to indicate that they continue to use the Libraries in higher numbers going forward, but there’s a lot more work to do there. But why wait? We figured reaching out to those students who do not take Intro to Library Resource--a one hour workshop or online tutorial--would be a good thing to do. That’s our most scalable intervention.
The behavior we most want to increase is use of databases (correlates consistently to GPA and Retention)
We can increase use through mediation, and our Intro to Library Research workshop and online learning object is the most successful and scalable way to do that
Reach out to students who aren’t currently taking Intro to Library Research
Presenter
Presentation Notes
We know that database use in particular correlates consistently with GPA and retention, so we really want to increase that. We can only do that with some kind of mediation: We need to create some opportunity to tell them about the resources we have and why they would use them. By far the most successful and scalable way to do that is through instruction we offer called Intro to Library Research, or ITLR. ITLR is normally paired with the University’s WRIT 1301 course, but plenty of students find ways to NOT take that. So what we need to do is find ways to reach those students.
Working with Academic Advisers and First Year Experience staff in the Colleges
Presenter
Presentation Notes
How do we do that? Well, for starters, we find more allies.
CLA 1002: A First Year cohort
Presenter
Presentation Notes
This is one effort that’s slated to start in Fall 2016. Our College of Liberal Arts has a First Year cohort experience, and in the coming year students will be informed about the ITLR class they may not have found out about because they didn’t have to take our first year writing course.
Presenter
Presentation Notes
Another effort is already underway, and we hope to see it grow over time. A couple of years ago at the First Year conference on campus, we had a lunch conversation with some academic advisers. They told us about the tool they use, the homegrown APLUS system, to keep track of the students they advise and get a quick look at what’s going on with the student when they come in for an appointment. We told them about our findings and gave them some ideas of services and tools they could tell students about. Adding library information and contacts to APLUS seemed like the next natural step. Advisers can now see whether the student has completed the online tutorial Intro to Library Research.
Referrals
Presenter
Presentation Notes
The Libraries are now an APLUS referral point for the academic advisers. They can refer a student to the Libraries through APLUS, the Libraries will determine who is best suited to address the need, and an individual in the Libraries will follow up with the student. Best of all, the APLUS system will allow the librarians to add interactions with students so the adviser can see that the loop has been closed and learn more about how library services can help students.
• Begin using APLUS for Libraries interactions with students
• More information enables us to provide the most appropriate help to the student
• Closes the loop: Advisers can see what happens when they refer
• Allows us to refer to other campus partners
• Other ways to use the data
• Get to know the user community in aggregate (by college, department, level…)
• Find out who’s NOT using our resources (but should be) and figure out best ways to reach them
• Go beyond undergrads
Possible next steps
Library Data & Student SuccessA project of the University of Minnesota Libraries, in collaboration with the University of Minnesota Office of Institutional Research
Krista Soria (OIR)
Shane Nackerud (Libraries)
Jan Fransen (Libraries)
Kate Peterson (Libraries)
Kristen Mastel (Libraries)
More information about the project on our blog at z.umn.edu/LDSS
Find a list of our publications at z.umn.edu/LDSSPubs
