Making the Case for Evidence-Based PracticeBy Joanne Bates, Janelle McClure, and Andy Spinks

OPENING ARGUMENTS

Ladies and gentlemen of the jury, you are aboutto be presented with facts that will prove, beyondany reasonable doubt, that data can be used toenhance the effectiveness of your library mediaprograms and gain support for them, even in thetoughest budget climates. Evidence-based practiceis the collection, interpretation, and use of data,such as collection statistics or assessment results,that measure the effectiveness of a library mediaprogram. We, the prosecutors, will present variousforms of evidence and show that any librarymedia specialist can use data to make informeddecisions that improve the impact of the librarymedia program, and he or she can also use datato demonstrate the quality of the program andgarner support for it.

ESTABLISHING A MOTIVE

Why would a busy library media specialist wantto bother with collecting, tabulating, analyzing,and publicizing data about his or her program?Evidence-based practice sounds like a lot ofwork, and it clearly requires some premeditation.However, a library media program can reapsubstantial benefits from evidence-based practice,and in the long run, it can even eliminate some ofthe work required of that busy media specialist.

Most importantly, a library media specialist canuse data about his or her own library mediaprogram to guide decision-making and spurimmediate improvements. For example, if alibrary media specialist learns that students visitthe library media center twice as often beforeschool as they do after school, she might chooseto open earlier in the morning and shortenafternoon hours. Similarly, if a library mediaspecialist uses a formative assessment tool todiscover that a particular lesson was not effective,he can re-teach the concepts using a differentinstructional strategy. Dedicating time andresources to the most powerful practices (andeliminating those activities that are not effective),makes programs operate more efficiently andeliminates unnecessary work.

Data that reflect the effectiveness of an individualprogram can also be an incredibly powerful toolfor public relations and advocacy. For example,data showing that after completing a researchproject, 87% of students in an eighth-gradescience class were able to successfully organize

"A library media program can reap substantial benefitsfrom evidence-based practice, and in the long run, itcan even eliminate some of the work required of thatbusy media specialist."

scientific information in an appropriate chart orgraph could be used to persuade other scienceteachers to collaborate on similar projects. Thesame data could support a request for funding toupdate the book collection—or to support a pleato hold "picture day" somewhere other than inthe library media center!

Make no mistake, administrators andpolicymakers give priority to programs that theybelieve work, and this kind of specific data isa powerful tool for conveying the effectivenessof a library media program. If those leadersunderstand the impact of their library mediaprograms, they will do their best to provide themwith adequate funding and staffing. Also, theywill try to avoid saddling their library mediaspecialists with those dreaded "other duties asassigned." Library media specialists can usedata to avoid unnecessary and unrelated work,garner support for their programs, and enjoy thesatisfaction of knowing they make a difference,so they have strong motives for usingevidence-based practice!

EXHIBIT A: INDIRECT EVIDENCE

Indirect evidence is a sort of "circumstantial"evidence; it shows that the library media programin question has done the kinds of things that haveproven to be effective in other programs aroundthe nation. To establish "guilt" we must compareexisting research findings with data about thelibrary media program in question. Here areseveral examples:

Collection Statistics: Research has shownthat achievement is higher when students haveaccess to larger, more up-to-date library mediacollections. If the collection statistics for a librarymedia center show a significant improvement inthe quantity and quality of information resourcesavailable, a case can be made that the program

is making a greater contribution to studentachievement at that school.

Accessibility Data: Research has shown thatachievement is higher in schools where librarymedia center resources are more accessible. Ifevidence shows hours have been extended, itscollection has been supplemented with onlineresearch databases that extend acce.ssibility intothe classroom and the home, or its loan periodsand circulation policies have been adjusted tomaximize the accessibility of its resources, a casecan be made that the program is making a greatercontribution to student achievement atthat school.

Usage Statistics: Research has shown thatachievement is higher in schools where studentsuse their library media center and its resourcesmore frequently. If measures of book circulation,database usage, or foot traffic show that studentsare using a library media center more, a case canbe made that the program is making a greatercontribution to student achievement atthat school.

Collaborative Instruction Data: Research hasshown that achievement is higher in schoolswhere library media specialists collaborate withclassroom teachers to offer information literacyinstruction that is integrated with subject areacontent. If collaborative planning documentationand the library schedule or calendar show thatthe library media specialist has provided moreand better information literacy instruction, thecase can be made that the program is making agreater contribution to the overall level of studciuachievement at that school.

For more information, jurors are encouragedto examine the briefing filed by Lance andLoertscher that otitlines the research findings onthe impact of library media programs on studentachievement. As in the examples above, research

2 4 LIBRARY MEDIA CONNECTION August/September 201U

"Make no mistake, administrators and

policymakers give priority to programs that they

believe work, and this kind of specific data is a

powerful tool for conveying the effectiveness of

a library media program."

findings can be paired with data collected at alocal school to establish "guilt by association."

EXHIBIT B: DIRECT EVIDENCE

Direct evidence can be used to prove that one'slibrary media program is guilty of improvingstudent achievement. These kinds of evidence areincontrovertible; they show that the actions ofthe library media specialist directly led to studentlearning. Fortunately, this type of data can becollected easily as part of the assessment process.

Some examples are very simple, such as using a"ticket out the door" to check students' grasp ofa concepr at the end of a lesson. In this activity,students respond to a brief prompt such as, "Giveone reason that it is important to cite the sourcesof your information in a research paper," or "Listthree ways to determine the quality ot a Website." The students turn in their responses (their

"tickets out the door") as they leave. Afterward, theteacher or library media specialist tallies correctresponses to measure the impact of the lesson.

Other types of direct evidence require moreadvance planning and greater collaboration withthe classroom teacher, such as comparing theresults of pre-tests and post-tests results, or usingproject assessment rubrics to measure students'ability to apply a skill. Several examples ot directevidence will be discussed in the eyewitnesstestimony to follow, and many more are includedin the briefing on collaborative assessment oflibrary media instruction filed by Haradaand Yoshina.

EYEWITNESS TESTIMONY

Now, the prosecution would like to call otir firstwitness, Joanne Bates, to the stand. Ms. Bates,would you describe for the jury how you becameinvolved with evidence-based practice?

My data use began a few years ago as I wastrying to get more teachers to collaborate oninstruction. At that time, most of my librarymedia lessons were planned and deliveredwithout any true teacher collaboration, andI never did any formal assessment of whatstudents learned. In hopes of motivating

Class 5A - Individual Student ResponsesPre-Assessment (9/4/2007) Post-Assessment (5/8/2008)

A IA2

A3

A4

A5

A6

A 7

A 8

A9

A 10

A11

A12

A13

A 14

A15

A 16

A17

A18

A19

A 20

A21

A 22

A 23

A 24

50

50

80

70

70

70

80

80

70

70

50

40

80

80

60

30

70

60

60

30

10

60

70

30

Green highlighting indicates correct responses. Blue highlighting indicates incorrect responses.

Figure 1. The shift from mostly blue to mostly green is a strong visual representation of the progress students made.

August/September 2010 LIBRARY MEDIA CONNECTION 2 J

teachers to collaborate more, I decided tocreate an assessment to see just what thestudents already knew.

Instead of beginning the year with the usualorientation, I gave our fift:h-graders a simplepre-assessment consisting of 10 generalquestions about information literacy. Iadministered the pre-assessment using a studentresponse system, which made it more fun forstudents and made it easiet for me to collectand tabulate the data. I used Promethean'sActiVote system, which allowed me to exportthe responses for each class as a color-codedExcel spreadsheet, with all the correct responseshighlighted in green and the incorrect answershighlighted in red.

I asked the fifth grade teachers if I could talkto them duting their team planning session,and I came armed with a printed copy of eachclass's spreadsheet, exported from the ActiVotesystem. As I handed out the spreadsheets, theywere visibly shocked. After seeing the amountof red on the spreadsheets, the teachers couldnot deny that their students needed moreinstruction in information literacy. They agreedthat we needed to meet on a more regular basisto collaboratively plan their students' librarymedia instruction.

Throughout the year, I worked with the fifthgrade teachers to plan a series of lessons andprojects to develop these information literacyskills. I conducted formative assessmentsand made adjustments as necessary. Finally,I gave a post-assessment that paralleled thepre-assessment from the beginning of theyear. The results revealed that every classmade improvements in their understandingof information literacy skills. Overall, themastery level increased by 43 percent. SinceI used the ActiVote system to conduct thepost-assessment, I was able to generate anothercolor-coded spreadsheet for each class. Seeingthe shift from mostly red to mostly greenconvinced the teachers of the effectiveness ofour collaboration.

That was three years ago. Each year since, I haveexpanded my use of assessment and improvedthe tools I use. I use pie- and post-assessmentswith third and fourth grades, and I have alignedthem more closely to the curriculum standardsfor each grade level. I use formative assessmentsto measure student progress during individualunits, either with the student response system,project-based activities, "tickets out the door,"or other strategies. The formative assessmentsallow me to see if the students grasped the keyconcepts, or if I need to go back and re-teachusing a different approach.

2 6 LIBRARY MEDIA CONNECTION

Because of the initiative I took to begin usingassessment and collecting data, the majotityof my teachers now actively collaborate withme on information literacy instruction. Nowthat I have been collecting assessment datafor a few years, I use it to prove to teachers,administrators, and the school communitythat my library media program has a directimpact on student learning at our school.I can even say that my efforts have had aneffect beyond the school level. Our assistantsuperintendent came in once while I was givinga post-assessment using the student responsesystem. She was so amazed to see a librarymedia specialist using assessment this way thatshe asked me to give a presentation to all of theprincipals in our area!

I don't know what took me so long to start, butI'm proud that I did. Now I couldn't imaginedoing my job without using data.

Thank yoti, Ms. Bates. Now the prosecutionwould like to call Janell McClure to the stand. Ms.McLure, could you describe for the jury some ofthe ways you have used data in your librarymedia program?

I admit, I have personally engaged in a numberof data-gathering activities, including pte/posttests, student petformance observations, andstudent reflections. I used the results of theseformal and informal assessments to improveinstruction and deepen collaboration with myprofessional accomplices.

Two years ago, elementary, middle, and highschool educators in our area met in "verticalteams" to examine ways to improve ourstudents' transition from elementary to middleschool and from middle to high school. Highschool library media specialists reported thatstudents were entering ninth grade withoutseveral crucial information literacy skills,including searching databases, citing sources,and even saving doctiments. Ba.sed on thesediscussions, I collaborated with the eighthgrade language arts teachers at my school tocreate a series of six lessons that would givestudents the information literacy skills theywould need to be successful in ninth grade.The lessons focused on the concerns expressedby our high school colleagues, but they alsoaligned with approximately 25 state curriculumstandards and AASL information literacystandards. We called the project "EighthGrade Academy," because all six lessons weretaught in the final four weeks of the schoolyear, after the administration of Georgia's high-stakes Criterion-Referenced Competency Test(CRCT).

At the beginning of the program, the teachersand I gave a brief pre-test to determine thestudents' current proficiency in our focusareas. After each lesson, students were alsoasked to write a one-sentence "ticket out thedoor" describing what they learned during thatday's activities. After three lessons, we gavea quiz to assess the students' comprehensionof the concepts and their ability to apply

Number of Incorrect Student Responses, Pre-test vs. Post-Test

Cass 8A

Question # \Pre Post

Class 8D Class 8E

311

5

5

8

11

1112126

08

5

4

4

8

1212146

110

8

4

10

5

12

10

12

iwm

14

4

5

3

59

10

11

1 4

14

7

4

11

10

8

9

13

6

01

8

2

3

8

7

15

2

6

013

11

5

14

12

12

10

15

2

03

3

0

4

10

10

13

15

2

69

9

1

16

8

12

15

19

9

23

6

7

10

10

10

10

8

Green highlighting indicates questions on which the class showed had the same or agreater number of incorrect responses.

Figure 2. Students showed no progress on some questions, indicating that ourinstruction had not effectively addressed those topics or that our questions were notvalid measures of their knowledge in those areas. In response to these findings, wemodified our assessment questions to ensure their alignment with the standards andadjusted our instructional methods to improve their effectiveness.

August/September 2010

these new skills. After the final lesson of theAcademy," the students completed a post-testthat mirrored the pre-test given before the firstlesson. I charted the results, and the teacherswere pleased to find that their students' overallmastery of the concepts had improved in amajority of the classes.

We further examined the data to determineliow it might help improve our own[x-rformance as educators. We realized thatour prc- and post-test questions were not aswell aligned with our lessons as we would haveliked. Also, we discovered a consistent lack ofgrowth in student performance in some specificareas, indicating that our instruction had notaddressed those topics as successfully as we hadhoped. Finally, we noticed that in the students'reflections, their perception of their owninformation literacy skills were slightly differentthan what had been observed by our highschool colleagues; the students felt they hadmastered skills, such as saving documents andunderstanding effective search terms, but thehigh school library media specialists reportedotherwise. Based on the insights from this data,we modified the pre-test questions and thelessons for the following year to better addressthe concerns of the high school library mediaspecialists and the needs of the students.

After the second year, our collaborative teammade more improvements to the instructionalgoals, lessons, and the test questions. Wealso began using an online survey tool called

SurveyMonkey Chart

Question 16: When using the databases in the Cobb Virtual Library, it is BEST to search with:

SurveyMonkey {www.surveymonkey.com) toadministet the pretest. The reporting feature ofSurveyMonkey automatically generated chartsthat visually represented the overall responsesfor each question. At a glance, we were ableto determine which skills students had alreadymasteted and which ones needed the mostinstructional attention.

Now that I have several years' worth of datato show that this instructional program iseffective, it has been much easier to convinceteachers to start the lessons before the CRCTis administered. We have also continued toimprove the quality of the pre- and post-testitems, and we ate using better formativeassessments to ensure that students areretaining the skills they learn and can applythem in context.

After seeing the power of data in improving myinstruction and ptomoting my library mediaprogram, I have started to use it more andmore. CRCT test results reveal areas whereour students struggle, and within those areas,I have identified specific topics that librarymedia instruction can support and collaboratedwith teachers to develop instructional unitsthat address these target areas. I'm so addictedto data, that when it is not readily available, Ido whatever is necessary to get it. I have evenused SurveyMonkey to poll students regardingtheir reading habits and preferences, then usedthis data for reading promotion and collectiondevelopment. I can't get enough!

Answer Options !

Whole questions

One or two keywords

Long phrases

Hyperlinks

Response Percent

16.6%

72.3%

5.1%

6.0%

answered question

skipped question

Response Count

55

240

17

20

332

2

/ ^ B ^ M >̂ When usinq/ ^ k | d > A Library,

( ^ \ ^ 1 D

v _ ^ •

databases in the Cobb Virtualit is BEST to search with:

Whole QuestionOne or two keywordsLong phrasesHyperlinks ^ ^ K

Thank you, Ms. McClure. I believe that the juryis now beginning to see how powerful evidence-based practice can be.

CLOSING ARGUMENTS

Ladies and gentlemen of the jury, you can clearlysee that these various types of evidence aresufficient to prove the effectiveness of a school'slibrary media program. Indirect evidence, suchas circulation or usage data, establishes that thelibrary media program and its staff have engagedin practices known to impact student achievement.Furthermore, direct evidence collected frominstructional assessments proves that a librarymedia specialist is guilty of teaching in the firstdegree! (Such an effect on student learning couldnot have occurred unless the act was intentionaland premeditated.) Library media specialists havea clear motive for engaging in such practices,because they provide a mechanism for improvingthe effectiveness of their programs, eliminatingunnecessary work, and gaining support fromteachers and education leaders during a timewhen it is needed more than ever. ••

COURT DOCUMENTSHarada, Violet H., and Joan M. Yoshina. AssessingLearning: Librarians and Teachers as Partners. Westport,CN: Libraries Unlimited, 2005.

Lance, Keith C , and D,ivid V. l.ocrtschcr. PoweringAchievement: School Library Media i^rograrns Make aDifference: The Evidence Mounts (3rd ed.). Salt Lake City,UT: Hi Willow Research, 2003.

Loertschet, David V., with Ross J. Todd. We BoostAchievement.' Evidence-Based Practice for School LibraryMedia Specialists. Salt Lake City, UT: Hi WillowResearch, 2003.

Joanne Bates is a library me-dia specialist at Tritt Elementary

School in Marietta, Georgia.

Janelle McClure is a librarymedia specialist at Palmer Middle

School in Kennesaw, Georgia.

Andy Spinks « the supervisorof library media education for the

Cobb County (Georgia) SchoolDistria.

Figure 3. The charts generated by SurveyMonkey made it easy to see which conceptsstudents had mastered and which needed more attention.

August/September 2010 LIBRARY MEDIA CONNECTION 2 7

Copyright of Library Media Connection is the property of Linworth Publishing, Inc. and its content may not be

copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written

permission. However, users may print, download, or email articles for individual use.