assessing information literacy from the ground up
DESCRIPTION
How to start an information literacy program at your academic libraryTRANSCRIPT
ASSESSING INFORMATION LITERACY FROM THE GROUND UP – CREATING A CULTURE OF ASSESSMENT
Kristy Motz, Randall Schroeder & Mari Kermit-CanfieldFerris State University
FLITE ASSESSMENT TEAM
WHAT’S OUT THERE?
Starting Up – Graduate Level Assessments (CMS)
• Developed tools two years ago in Blackboard Vista product
• Ran assessments through several semesters
• Summer 2009: Refined questions and mapped them to Information Literacy Competency Standards for Higher Education
MMBA Pre- and Post-Instruction Assessments
• Mixed delivery or blended delivery class
• Two Saturdays on-site - remaining course online
• Class runs eight weeks = ½ semester
• Assessments run in class first and last day
• 8 questions pre-instruction (first day of class)
• 5 questions post-instruction (end of class)
ONLINE ASSESSMENT IN CMS
PRE-INSTRUCTION COURSE ASSESSMENT
POST-INSTRUCTION ASSESSMENT MMBA
MMBA CMS Issues• Surveys or ungraded quiz tools from
CMS don’t provide results.
• Instructor must allow point credit for results to remain in Gradebook.
• Students with first-day user ID/password problems can’t reach the assessment.
• Questions must be tweaked to fit CMS question software format.
FALL 2008 MMBA - FAGERMAN
SUMMER 2009 MMBA - HAMEL
ESPN 505 Special Education
• 19 students took Pre-Instruction Assessment
• 15 students took Post-Instruction Assessment
• Instructor added 2 point bonus for participation
• Instructor used CMS with on-campus course
ESPN 505 PRE AND POST INSTRUCTION ASSESSMENT
NEXT STEPS: REAL-TIME UNDERGRADUATE ASSESSMENTS
Vogel Library FLITE
• FLITE’s new department head: Feb 2008
• Assessment Team’s new focus on Information Literacy Instrument
• Announcement of new James Madison Information Literacy Testing
HOW TO BEGIN?VOGEL LIBRARY – WARTBURG COLLEGE
Vogel Library
• Seven years of pre-instruction and post-instruction information literacy testing
• Based on Information Literacy Competency Standards for Higher Education
• Designed question by question to highlight targeted standards
Vogel Library
• Developed by
• Librarians Randall Schroeder, Jill Gremmels, and Karen Lehman
• Dr. Judith Griffith, English Faculty Chair
• Led to Faculty Buy-In
Vogel Adaptations to FLITE• FSU Students
• FSU Focus and Mission
• Information Literacy a new Concept
• Implementation and Testing
• Faculty Buy-in: 3 volunteers worked with us
CHALLENGES
•Conflicts with Scantrons and Testing Center
•Need for definitive learning outcomes
•Testing on concepts not always covered in classes
WHAT ARE WE TESTING?16. A student who is deciding what topic to
research for an assigned paper approaches the reference librarian and says, “This summer was too hot. I wonder if it is because of global warming?”What might be the thesis statement of this student’s paper?
1. It was too hot this summer. 2. Global warming is causing the mean
temperature of the earth to rise, which could have disastrous effects on the ecosystems of the planet.
3. Did global warming cause abnormal heat around the planet?
4. We should buy hybrid vehicles because of global warming.
5. This paper will be about the effects of global warming.
SUCCESSES - SURVEY QUESTIONS Rank the activities on which you spent the most (4) to the least (1) time when you are on the computer.
Searching for specific information related to a school or personal task.
Communicating with individuals through Facebook/My Space, IM, or via e-mail.
Downloading movies, music or graphics.
Surfing the Internet for enjoyment or playing
games.
WHAT DO WE KNOW ABOUT OUR STUDENTS?
WHERE DO YOU GET YOUR INFORMATION?
WHICH OF THESE HAVE YOU DONE WITHIN THE LAST YEAR?
HOW MANY PAPERS DID YOU WRITE?
Schroeder Words of Wisdom: Hard Lessons Learned
Give yourself permission to fail
Assess to learn things – Don’t just assess to assess
Be sure to have Faculty Buy-In
Starting Again
Involve new staff
Improve librarian buy-In
Imagine: What will happen when developers leave?
What are the Information Literacy Competency Standards?
http://ala.org/ala/mgrps/divs/acrl/standards/standards.pdf
FLITE INFORMATION LITERACY ASSESSMENT TEAM CREATES LEARNING OUTCOMES
In-depth study of the 5 competency standardsFocus on performance indicators
Curriculum mapping - Group discussion and consensus 7 performance indicators1-2 indicators from each standard
FLITE INFORMATION LITERACY ASSESSMENT TEAM CREATES LEARNING OUTCOMES
Group work: Individual questions written for each learning outcome (performance indicator) - 35 questions for assessments
Discussion and consensus: Team selected 34 questions for Question Bank
12 questions are survey questions asked for every assessment
22 questions can be tailored for class
Information Literacy – Round Two Fall 2009Question Bank built for use with
different classes
Individualized assessments built by individual library instructors from Question Bank
Assessments delivered electronically if possible
SURVEY MONKEY
Started in 1999, SurveyMonkey is an online survey tool that enables people of all experience levels to create their own surveys quickly and easily.
FLITE Library’s Assessment Team picked Survey Monkey because:
• Seven years of pre-instruction and post-instruction information literacy testing
• Based on Information Literacy Competency Standards for Higher Education
• Designed question by question to highlight targeted standards
FLITE LIBRARY’S ASSESSMENT TEAM PICKED SURVEY MONKEY BECAUSE…
History Team members have previous experience
with the program
Easy of use User needs no statistical background User needs no knowledge of statistical
software
Malleable Accessible from web; in class or at home Easily printable for a paper version
Success – Yes, You Can Too
We went beyond the “What if’s” and created a project.
We gave ourselves permission to fail.
We didn’t have to be perfect – we just had to start.
We created a Culture of Assessment: we refined what we did and moved on.
Next StepsPost-instruction assessment – Dec 2009
Refine questions based on data input
Expand to other faculty and departments
Adapt procedures for Distance Learning
Write new assessments for upper-level classes
Accomplished Major Goal
• Closed the loop
• Curriculum mapping: using assessment data to revise and improve instruction curriculum
• Keep assessing – TracDat
Culture of Assessment
Helps us inform faculty where we want them to go with us
Allows us to create customized instructions across the curriculum
Showcases FLITE – the Library is one of the early reporters of assessment data on campus
Just Start!
• Get Faculty Buy-in
• Give Yourself Permission to Fail
• Take the Plunge and Begin
Questions?
Kristen Motz [email protected]
Randall Schroeder [email protected]
Mari Kermit-Canfield [email protected]