measure for measure: using collaborative assessment to build stronger information literacy skills
TRANSCRIPT
- 1. Measure for Measure Using collaborative assessment to build stronger information literacy skills PRESENTED BY: Susan Gardner Archambault
- 2. Private Jesuit and Marymount University in Los Angeles, CA 6,064 Undergraduates 2,189 Graduates Library Open 24/5 New Core Curriculum in Fall of 2013 LMU
- 3. Info. Literacy Learning Outcomes
- 4. Info lit flag Core Structure
- 5. Rhetorical Arts Course Criteria Assign at least 10% of the final course grade on the basis of information literacy, with a librarian-led workshop and one or more course-integrated assignments INFORMATION LITERACY OUTCOMES: 1. Conceptualize an effective research strategy and then collect, interpret, evaluate and cite evidence in written and oral communication 2. Distinguish between types of information resources and how these resources meet the needs of different levels of scholarship and different academic disciplines
- 6. Spring 2014 1272 first-year students had a required library visit Across 72 sections of Rhetorical Arts
- 7. 3 Info Lit Course Assignments in Common Syllabus
- 8. Library Visit Agenda 1. Intro to LibGuide 2. Why use the library? 3. Evaluating sources (RADAR game) 4. Finding sources (demo)
- 9. Library Support Material
- 10. Rhetorical Arts 1000 LibGuide
- 11. RADAR Game Students were paired up and given a source to evaluate Using the resources in the course LibGuide the student pairs answered questions about the source they were given Discussion with the librarian after completing the RADAR Game
- 12. Gamification Incorporating Friendly Competition (points) Motivational Feedback (stars) Measure Progress (showing star count) Reward effort (getting the answer right on the first try earns more points)
- 13. Research Strategies Tutorial (Recommended Homework)
- 14. Research Questions How effective was the face-to-face instruction? How effective were the course-integrated assignments? How effective was the library support material?
- 15. Methodology
- 16. Direct Measures Overall Averages: Student scores across 100 sampled annotated bibliographies. Scored with a calibrated rubric by a group of volunteer R.A. instructors
- 17. Indirect Measures Survey: Stratified random sample of 300 students (48% response rate) All instructors teaching a section (57% response rate)
- 18. Comments Survey comments were analyzed using NVivo and coded
- 19. Major Findings & Applications
- 20. High-Scoring Areas: o citing sources correctly o identifying the main purpose of a source Low-Scoring Areas: o identifying intended audience o authority of the author o accuracy of the evidence o bias Annotated Bibliography
- 21. 3.54 4 0 1 2 3 4 5 Helpfulness/Evaluating Evidence Students Instructors Annotated Bibliography
- 22. Comment No. of Times Not helpful for writing/research process 7 Too many steps/make simpler 3 Annotated Bibliography/Comments
- 23. Comment No. of Times Fun/Engaging 21 Helpful 15 Competition 8 Interactive 7 Liked partner work 6 Not fun/engaging 6 Too easy 6 Not useful 5 RADAR Game/Student Comments
- 24. Students who listed a major in the College of Science & Engineering liked the RADAR game more than the average student Statistically Significant
- 25. Annotated Bibliography Revised
- 26. Added Hints Added More Discussion RADAR Game Revised
- 27. Pre-Search Worksheet
- 28. 56%22% 22% Instructors (Mean = 3.44 / 5) Agree Disagree Neutral 51%31% 18% Students (Mean = 3.17 / 5) Agree Disagree Neutral Helpfulness in Developing a Topic
- 29. Comment No. of Times Too easy/not complex enough 17 Did not integrate with or apply to other projects/assignments 5 Add working bibliography of sources 4 Layout needs more space 4 Add reflective questions re: terminology & inclusion/exclusion choices 3 Add citation tracking element 1 Have instructor approve the topic 1 Allow space to change positions 1 ID stakeholder and their position 1 Pre-Search Worksheet Student/Faculty Comments
- 30. Research Diary
- 31. 3.29 3.71 3.33 3.5 2.77 2.58 0 1 2 3 4 5 Reflecting on search process Helped collect info Grading rubric/significant aspects of learning Grading rubric/clear criteria Student Instructor Research Diary
- 32. Comment No. of Times Busy work/not helpful 24 Add section on relevancy of sources/source reflection/how to use/more source interaction 6 Timing/better integration with other assignments (earlier in semester) 2 Research Diary Student/Faculty Comments
- 33. Pre-Search Worksheet Revised
- 34. Continued
- 35. Library Visits/Library Instruction
- 36. 75% 17% 8% Instructors (Mean = 3.92 / 5) Valuable Somewhat Not Very Valuable 46% 30% 24% Students (Mean = 3.22 / 5) Valuable Somewhat Not Very Valuable How Valuable Are Library Visits?
- 37. 4.3 4.32 4 3.88 3.57 3.57 3.55 3.67 3.8 3.8 3.41 3.58 3.5 3.53 0 1 2 3 4 5 Scholarly vs. Popular Getting Help from Librarian Finding Info Finding a Topic Evaluating Appropriateness/ Info Citing Sources Using Evidence/Make Arguments Student Instructor How Helpful Are Library Visits?
- 38. Comment No. of Times LibGuide 30 Services Offered by Librarians 29 Research Databases 24 Research Strategies 16 Evaluating Sources 13 What Did Students Find Most Helpful?
- 39. Comment No. of Times Fine no suggestions for improvement 50 Research databases/keyword searching 16 Finding scholarly sources 13 Repetition of FYS or other course 10 Too easy 10 Better relevance or integration with real assignment 8 Citing 8 Navigating library web site or LibGuide 8 What Needs Improvement? Student/Faculty Comments
- 40. Revised: Finding Sources Activity (Team-based Active Learning)
- 41. LibGuide Student Ranking
- 42. Comment No. of Times Couldnt find/unaware of 14 Too complicated/confusing/too many links 9 LibGuide: Student/Instructor Comments
- 43. Research Strategies Tutorial
- 44. 29% 50% 21% Instructors Who Required Tutorial Yes No Not Sure 55% 45% Students Who Completed Tutorial Yes No Research Strategies Tutorial: Usage
- 45. Student Ranking of Tutorial 3.71 out of 5
- 46. Revised: Tutorial as Mandatory Homework Assignment
- 47. Encourage better integration of library visit and info lit assignments by assigning the same research topic throughout multiple assignments Additional Improvement
- 48. Thank You: William H. Hannon Library Research Incentive Travel Grant Susan Gardner Archambault [email protected] @susanarcham