Running Head: EVALUATION PLAN 1

Evaluation Plan: Fountas & Pinnell Benchmark Assessment System

Kimberly A. Sharp

University of West Georgia

EVALUATION PLAN 2

Evaluation Plan: Fountas & Pinnell Benchmark Assessment System

Introduction

Background Information

Ivy Creek Elementary School is located in northeastern Gwinnett County in Buford,

Georgia. Founded in 2003, the school is comprised of 886 students in grades PreK-Fifth Grade.

At a recent faculty meeting, the principal of Ivy Creek, Yvonne Frey, shared statistics about the

students attending the school. Demographically, the student population is 58% Caucasian, 20%

Hispanic, 10% African – American, 8% Asian and 4% Multi-Racial. Approximately 29% of

students receive free or reduced lunch while the majority of students come from families with

average socioeconomic status (Y. Frey, personal communication, May 17, 2010).

Gwinnett County prides itself on becoming a system of world-class schools. As a

member of the largest school district in the state of Georgia, Ivy Creek Elementary has enjoyed

the support of an active Language Arts Department. The school was recently invited to pilot the

Fountas & Pinnell Benchmark Assessment System largely as a result of the efforts of the reading

coach at Ivy Creek Elementary School.

The administrative staff at Ivy Creek Elementary School requested that this program be

evaluated to determine the impact of the assessment system on reading instruction and student

achievement in reading. This evaluation will provide valuable information to the Ivy Creek

Elementary School Stakeholders: Principal, Assistant Principals, Reading Coach, Teachers and

Staff. In addition, the results of the evaluation will be available for review by the county office

staff. This data may have an impact on whether the program is recommended to all elementary

schools in the county. As the media specialist at Ivy Creek Elementary School and one of the

EVALUATION PLAN 3

few teachers not directly involved with administering the Fountas & Pinnell Benchmark

Assessment System, I am conducting an objective evaluation of the program.

Evaluation Program

Fountas and Pinnell have spent more than twenty years establishing a leveling system for

educators to use when evaluating students’ reading ability. Teachers are able to determine

whether a student is meeting the standards set for each grade level by using leveled books and

measuring a student’s reading fluency and comprehension. Levels follow a twenty-six point

scale from the easiest level (A) to the most challenging level (Z.) These alphabetical levels

correspond with reading ability levels ranging from Kindergarten to 8th Grade (Fountas &

Pinnell, 2008.)

The Fountas & Pinnell Benchmark Assessment Program is designed to measure student’s

reading levels with the use of benchmark books. These leveled books are used during a one-on-

one session between the teacher and the student. During these sessions, the student reads the

book aloud and discusses the book while the teacher observes and codes the behavior of the

reader. This system of coding allows the teacher to identify student errors with regards to

fluency and comprehension. The information gained from this personalized assessment session

provides the teacher with valuable insight into the reading ability of the student and the concepts

that need to be addressed during reading instruction (Fountas & Pinnell, 2008.)

Purpose

The purpose of conducting an evaluation of the Fountas & Pinnell Benchmark

Assessment System at Ivy Creek Elementary School is to determine the impact it has on reading

instruction and student achievement. The program is designed so that teachers administering the

EVALUATION PLAN 4

assessment are able to use student errors to guide instruction. As a result, reading instruction is

more focused and allows students to progress more rapidly through the stages of reading.

This assessment system is closely aligned with the district’s views of reading instruction.

Specifically, the program correlates with the use of guided reading and reading workshops as

methods of delivery. In a program study of a reading initiative designed to improve reading

achievement, Al Otaiba and Hosp recommended that assessment programs should align with

district reading programs (2008.) The Language Arts Department feels confident that the

Fountas & Pinnell Benchmark Assessment Program will meet the needs of reading teachers.

Unfortunately, complete student achievement data will be unavailable during the time

constraints for this project. While there will be some indication of student progression in

reading, official midterm and year-end assessments will not be administered until January and

again in May. Therefore, the focus of this project will be on professional learning related to

implementing the assessment system and the impact this program is having on teacher

instruction.

At the completion of this formative evaluation project, the leaders at Ivy Creek

Elementary School will have a true picture of how the Fountas & Pinnell Benchmark Assessment

System is impacting classroom teachers and their reading instruction. Additional professional

learning opportunities will be offered to teachers based on the needs outlined in this evaluation.

This training, if needed, will be provided to teachers prior to the administration of the midterm

and year-end assessments. The data collected during the course of this project will also provide

professional learning instructors with specific feedback on how to further guide teachers in

administering effective reading instruction.

EVALUATION PLAN 5

These steps re-enforce the ideas suggested by Engstrom and Danielson in their review of

a professional development program on Howard Gardner’s multiple intelligence theory. They

found that teachers need ongoing support when implementing a new program. Additionally,

they recommended that action research be conducted throughout program implementation so that

professional development could be focused on the needs of teachers (2006.) It is also critical for

teachers to have sufficient time and resources to understand the intent of new standards and the

structure of new curriculum (Obara & Sloan, 2009.)

Once the needs of teachers have been identified, professional development opportunities

can be designed to address those needs. It will be important to narrow the focus of these sessions

to the most relevant topics while integrating specific content and teaching ideas. It will also be

necessary to provide teachers with networking time and long-term support (Hutchins, Arbaugh,

Abell, Marra & Lee, 2008) to ensure that the implementation of the Fountas & Pinnell

Benchmark Assessment Program is done correctly.

After teachers have administered the mid-term and post-evaluations, the evaluator and the

client plan to review student reading achievement data; CRCT scores, Guided Reading Levels,

and Reading Standards. This hard data should serve to reinforce the qualitative data obtained as

a result of this evaluation project (Kaufman, Guerra, & Platt, 2006.) Although this data will be

obtained beyond the scope of this evaluation, it will be necessary to include student achievement

results in order to complete the full program evaluation. This summative evaluation is scheduled

for completion by the end of the 2010-11 school year.

Evaluation Questions

In order to provide meaningful information to Ivy Creek Elementary School stakeholders

through this evaluation, the following questions must be answered:

EVALUATION PLAN 6

What impact do professional learning sessions have on teachers as they

implement the Fountas & Pinnell Benchmark Assessment System?

How does the Fountas & Pinnell Benchmark Assessment System impact reading

instruction?

How does the Fountas & Pinnell Benchmark Assessment System compare to the

assessment program used last year?

Should the Fountas & Pinnell Benchmark Assessment System be adopted for use

at Ivy Creek Elementary School next year?

MethodsParticipants

All certified staff members who administer the Fountas & Pinnell Benchmark

Assessment System will be asked to participate in this study. This group will include classroom

teachers in Grades 1-5, the reading coach, intervention specialists and gifted reading teachers.

Design and Procedures

A variety of activities will be designed to address all of the objectives outlined in this

evaluation plan. A quantitative research approach will be used in that data will be gathered from

teacher surveys (Appendix A) and interviews (Appendix B). Questions have been designed to

evaluate teachers’ perceptions of their professional learning experience as related to the Fountas

& Pinnell Benchmark Assessment System. Additional questions address the impact the program

has had on the way teachers delivered reading instruction. Comparing the Fountas & Pinnell

Benchmark Assessment System to the reading programs used in years past will be an important

factor in determining whether or not to proceed with the adoption of the Fountas & Pinnell

EVALUATION PLAN 7

model. Qualitative data will be gathered from open-ended and closed-ended questions in both

surveys and in-person interviews.

Instruments

A comprehensive survey has been developed by the evaluator for approval by the client.

Since only one survey will be administered for the purpose of this project, a select group of three

teachers will be asked to review the survey for clarity. All certified staff members responsible

for reading instruction will be asked to complete the survey. Questions will be both open and

closed-ended.

Once the surveys are complete, in-person interviews will be conducted with teachers who

are willing to participate. Ideally, at least one member from each grade level will be interviewed

using the interview guide approach (Appendix B.) This qualitative data collection method will

allow for topics and issues to be specified in advance while allowing for variance in the sequence

and wording of the questions as the interview progresses. This method will keep the interview

conversational which perhaps will elicit more meaningful data (Johnson & Christensen, 2007.)

Finally, informal student assessment data will be used to determine if students appear to

be progressing in the area of reading achievement. Since the formal assessment will not be

administered before the end of this project, the evaluator will examine student work and rely on

teacher observation to determine if students are showing signs of achievement.

Objective Survey Interview Informal Student Assessment

What impact do professional learning sessions have on teachers as they implement the Fountas & Pinnell Benchmark Assessment System?

x x

EVALUATION PLAN 8

How does the Fountas & Pinnell Benchmark Assessment System impact reading instruction?

x x

How does the Fountas & Pinnell Benchmark Assessment System compare to the assessment program used last year?

x x x

Should the Fountas & Pinnell Benchmark Assessment System be adopted for use at Ivy Creek Elementary School next year?

x x x

Data Analysis

Surveys

Each question on the survey will be carefully analyzed to determine patterns of response

by the participants. Those questions that are answered using a Likert scale will be averaged to

determine an average score for the respondents. These responses will be analyzed further based

on age, teaching experience and educational degree to determine if those factors are relevant.

Open-ended responses will be grouped based on similarities.

Interviews

Ideally, at least five teachers from various grade levels will volunteer to participate in the

interview process. The interviews will be recorded and analyzed to determine patterns of

response by the participants. Anecdotal commentary will be included in the evaluation report.

Informal Student Assessment

As a part of each interview, the evaluator will review the class reading level data

provided by the teacher. Although this will be a work in progress, a majority of students in each

class should show growth in reading level as compared to their initial assessment..

EVALUATION PLAN 9

Conclusion

Upon completion of the data analysis, the evaluator will prepare a written draft of the

evaluation report. This draft will be shared with the reading coach to review the findings and

determine any areas that need clarification. As part of this process, the reading coach will also

be asked to complete a language impact questionnaire (Appendix C) as recommended by

Kaufman, Guerra and Platt (2006.) Based on the findings of the language impact questionnaire,

the evaluator will revise the written evaluation report for presentation to the administration at Ivy

Creek Elementary School.

During the executive briefing, the evaluator and client will discuss the findings of the

formative program evaluation. Plans for modifying professional development and current

teaching practices will be discussed as necessary based on the results of the evaluation.

Additionally, a plan for incorporating mid-term and post-assessment achievement data in the

coming months will be developed in order to complete the summative evaluation. At the end of

the school year, all final results of the evaluation program will be shared publicly with the staff.

EVALUATION PLAN 10

References

Al Otaiba, S., Hosp, J., Smartt, S., & Dole, J. (2008). The challenging role of a reading coach, a

cautionary tale. Journal of Educational & Psychological Consultation, 18(2), 124-155.

Retrieved from ERIC database.

Engstrom, M., & Danielson, L. (2006). Teachers' perceptions of an on-site staff development

model. Clearing House: A Journal of Educational Strategies, Issues and Ideas, 79(4),

170-173. Retrieved from ERIC database.

Fountas, I. & Pinnell, G. (2008). Assessment guide: A guide to the benchmark assessment

system. Portsmouth: Heinemann.

Hutchins, K., Arbaugh, F., Abell, S., Marra, R., & Lee, M. (2008). A consumer guide to

professional development. Science Scope, 31(8), 16-19. Retrieved from ERIC database.

Johnson, B., & Christensen, L. (2007). Educational research: Quantitative, qualitative, and

mixed approaches (3rd ed.). Boston: Allyn and Bacon.

Kaufman, R., Guerra, I., Platt, W. (2006). Practical evaluation for educators: finding what

works and what doesn’t. Thousand Oaks: Corwin Press.

Obara, S., & Sloan, M. (2010). Classroom experiences with new curriculum materials during the

implementation of performance standards in mathematics: A case study of teachers

coping with change. International Journal of Science and Mathematics Education, 8(2),

349-372. Retrieved from ERIC database.

Pinnell, G. & Fountas, I. (2007). The continuum of literacy learning: A guide to teaching.

Portsmouth: Heinemann.

Purdue University Online Writing Lab (OWL) (May 5, 2010). APA formatting and style guide.

Retrieved from http://owl.english.purdue.edu/owl/resource/560/01/.

EVALUATION PLAN 11

Appendix A

Survey – Fountas & Pinnell Benchmark Assessment Program

The following survey is designed to evaluate the Fountas & Pinnell Benchmark Assessment Program (Fountas & Pinnell Program.) It should take approximately 5-10 minutes to complete. Your answers will provide valuable information in determining the effectiveness of the program as it relates to reading instruction and professional development. No names will be used at any time when reporting data for this evaluation, so please answer honestly!

Background Information

1. How old are you?a. 20-29b. 30-39c. 40-49d. 50-59e. 60+

2. How long have you been teaching?a. 0-5 yearsb. 6-10 yearsc. 11-15 yearsd. 16-20 yearse. 21 or more years

3. What degree level do you currently hold?a. Bachelorsb. Mastersc. Specialistd. Doctoratee. Other, please specify

4. Have you used the Fountas & Pinnell Benchmark Assessment Program this year?a. Yesb. No

5. Have you previously used the Fountas & Pinnell Benchmark Assessment Program?a. Yesb. No

EVALUATION PLAN 12

Training and Professional Development

Please indicate how you strongly you agree or disagree with each of the following statements:

1 2 3 4 5Strongly Disagree

Strongly Agree

6. The one-hour pre-planning training session was beneficial to me.

1 2 3 4 5 N/A

7. The two 40-minute professional learning sessions during the first two weeks of school were

beneficial to me.

1 2 3 4 5 N/A

8. The one-on-one coaching provided by the reading coach/intervention coach was beneficial to

me.

1 2 3 4 5 N/A

9. The training DVDs were beneficial to me.

1 2 3 4 5 N/A

10. I received adequate training on implementing the Fountas & Pinnell Program.

1 2 3 4 5

11. I prefer receiving training at the beginning of the school year.

1 2 3 4 5

12. I prefer receiving training during the summer.

1 2 3 4 5

13. I prefer receiving training throughout the school year.

1 2 3 4 5

EVALUATION PLAN 13

14. Please add any additional comments about the nature of the Training and Professional

Development as they relate to the Fountas & Pinnell Program:

Implementation and Reading Instruction

Please indicate how you strongly you agree or disagree with each of the following

statements:

1 2 3 4 5Strongly Disagree

Strongly Agree

15. I find the Fountas & Pinnell Program to be user-friendly.

1 2 3 4 5

16. I believe that the Fountas & Pinnell Program accurately measures my students’ reading

level.

1 2 3 4 5

17. I use the Fountas & Pinnell Program to guide my reading instruction.

1 2 3 4 5

18. I have a greater understanding of student errors as a result of working with the Fountas &

Pinnell Program.

1 2 3 4 5

19. I use student errors as identified by the Fountas & Pinnell Program to guide my reading

instruction.

1 2 3 4 5

EVALUATION PLAN 14

20. The word lists assist me in my reading instruction.

1 2 3 4 5

21. The recording forms assist me in my reading instruction.

1 2 3 4 5

22. Having copies of the support documents in the teacher workroom is beneficial to me.

1 2 3 4 5

23. Please add any additional comments about the implementation and reading instruction as

they relate to the Fountas & Pinnell Program:

Fountas & Pinnell Program vs. Other Reading Assessment Programs

Please indicate how you strongly you agree or disagree with each of the following

statements:

1 2 3 4 5Strongly Disagree

Strongly Agree

24. The Fountas & Pinnell Program provides me with more information about my students

reading abilities than previous reading assessment programs.

1 2 3 4 5

25. The Fountas & Pinnell Program is more accurate than previous reading assessment

programs.

1 2 3 4 5

26. I prefer using the Fountas & Pinnell Program over previous reading assessment programs.

1 2 3 4 5

EVALUATION PLAN 15

27. Please add any additional comments about how the Fountas & Pinnell Program compares to

other reading assessment programs you have used:

Recommendation for Adoption

28. I would like to see the Fountas & Pinnell Program adopted for use at Ivy Creek Elementary

School.

1 2 3 4 5

29. I would like to see the Fountas & Pinnell Program adopted for use in Gwinnett County.

1 2 3 4 5

30. Please add any additional comments about the Fountas & Pinnell Program. Is there

anything you would like to see done differently next year?

If you would be willing to participate in a short interview session to further share your opinions

about the Fountas & Pinnell Program, please include your name: _______________________.

Again, all identifying characteristics of individual responses will remain confidential.

Thank you for taking time to complete this survey.

EVALUATION PLAN 16

Appendix B

Interview Guide – Fountas and Pinnell Benchmark Assessment Program

Tell me about your experience with using the Fountas and Pinnell Benchmark

Assessment Program.

What are the strengths of the Fountas and Pinnell Benchmark Assessment Program?

What are the weaknesses of the Fountas and Pinnell Benchmark Assessment Program?

What part of the training benefitted you the most?

Is there anything you feel should be done differently next year?

In what ways has the Fountas and Pinnell Benchmark Assessment Program impacted

your reading instruction?

In what ways has the Fountas and Pinnell Benchmark Assessment Program impacted

student achievement?

What kind of growth have you seen in your students’ reading levels?

How would you compare Fountas and Pinnell Benchmark Assessment Program to

programs you have used in the past?

What is the most valuable part of the Fountas and Pinnell Benchmark Assessment

Program?

What is the least valuable part of the Fountas and Pinnell Benchmark Assessment

Program?

Is there anything else you’d like to share?

EVALUATION PLAN 17

Appendix C

Language Impact Questionnaire

1. Which course of action would the evaluation report best support for 2011-12?

a. Continue the Fountas & Pinnell Benchmark Assessment Program in the same

way as 2010-11

b. Modify the Fountas & Pinnell Benchmark Assessment Program

c. Discontinue the Fountas & Pinnell Benchmark Assessment Program

2. Based on the evaluation report, how would you describe the effectiveness of the Fountas

& Pinnell Benchmark Assessment Program?

a. Completely successful – meets all program goals

b. Partially successful – meets some goals, achievement of remaining goals is yet to

be determined

c. Partially unsuccessful – meets some goals, remaining goals were not achieved and

will not be achieved with the current approach

d. Unsuccessful – none of the program goals have been achieved

3. In comparison to other reading programs, how would you rate the Fountas & Pinnell

Benchmark Assessment Program based on the evaluation report?

a. Clearly superior

b. Somewhat better

c. About the same

d. Less effective

e. Definitely worse

EVALUATION PLAN 18

4. According to the evaluation report, the main goals of the Fountas & Pinnell Benchmark

Assessment Program are…. (list as many as apply):

5. According to the evaluation report, the Fountas & Pinnell Benchmark Assessment

Program’s important strengths or accomplishments are…. (list as many as apply):

6. According to the evaluation report, the Fountas & Pinnell Benchmark Assessment

Program’s important limitations or deficiencies are…. (list as many as apply):

7. According to the evaluation report, the Fountas & Pinnell Benchmark Assessment

Program is perceived positively overall by what percentage of the users?

a. 90% or higher

b. 80%-89%

c. 70%-79%

d. 60%-69%

e. 50%-59%

EVALUATION PLAN 19

f. 30%-49%

g. 0%-29%

8. According to the evaluation report, which statement best describes the likelihood that the

Fountas & Pinnell Benchmark Assessment Program will positively impact Ivy Creek

Elementary School’s performance?

a. Virtually certain – positive impact is already evident

b. Probably – trends so far are positive

c. Possible – program design is sound, but no evidence is available at this time

d. Unlikely – trends so far are negative and the basic program design is flawed

e. Program will not positively impact performance. No impact or negative impact is

already evident.

Questionnaire based on Table 7.1, p. 211-12 (Kaufman, Guerra & Platt, 2006)

EVALUATION PLAN 20

Data Collection Organization Matrix

Questions Related Results

Level (Mega, Micro, Macro)

Required Data,

Measureable Indicators

Data Source

Data Collection Tools &

Procedures

Data Collected

By (Date)

Primary Responsibility

What impact do professional

learning sessions have on teachers as

they implement the

Fountas & Pinnell

Benchmark Assessment

System?

Teachers will be confident

and successful

reading teachers.

MACRO Teachers will average a

score of 4 or higher on the 5 point Likert

scale on professional development

questions.

Teachers will express

confidence in implementing the program.

Teachers will indicate

needs for additional

professional learning with

regards to implementing the program.

Teachers Survey Results

Interviews

10/18/10 Kimberly Sharp

How does the Fountas &

Pinnell Benchmark Assessment

System impact reading

instruction?

The program will

positively impact reading

instruction.

MACRO Teachers will average a

score of 4 or higher on the 5 point Likert

scale on implement-ation and reading

instruction questions.

Teachers will indicate that they analyze errors made

by students in order to develop

appropriate reading

instruction methods.

Teachers Survey Results

Interviews

Informal: Student Reading Levels

10/18/10 Kimberly Sharp

EVALUATION PLAN 21

Teachers will indicate that

students show growth in

their reading achievement

and that reading levels are positively impacted by the use of the

program.How does the

Fountas & Pinnell

Benchmark Assessment

System compare to the

assessment program used

last year?

Teachers will prefer this program to previous

programs.

MACRO Teachers will average a

score of 4 or higher on the 5 point Likert scale on the Fountas & Pinnell vs.

other reading programs questions.

Teachers will express that

the new program is superior to previous

programs.

Teachers Survey Results

Interviews

10/18/10 Kimberly Sharp

Should the Fountas &

Pinnell Benchmark Assessment System be

adopted for use at Ivy Creek Elementary School next

year?

Teachers will recommend

that this program be adopted at Ivy Creek

and in Gwinnett County.

MACRO Teachers will average a

score of 4 or higher on the 5 point Likert scale on the recommend-

ation for adoption

questions..

Teachers Survey Results

Interviews

Informal: Student Reading Levels

10/18/10 Kimberly Sharp

Matrix taken from Figure 4.3, p. 105 (Kaufman, Guerra, & Platt, 2006).