studying fidelity of implementation (foi): how foi influenced scale-up’s theory of action for...
Post on 03-Jan-2016
219 Views
Preview:
TRANSCRIPT
Studying Fidelity of Implementation (FOI):
How FOI influenced SCALE-uP’s Theory of Action for Middle School Science
Curriculum Materials
*SCALE-uP = Scaling up Curriculum for Achievement Learning and Equity Project, a partnership between George Washington University
and Montgomery County Public Schools**
Sharon Lynch, PICo-PIs: Curtis Pyke, Joel Kuipers, Michael Szesze** & Bonnie Hansen-
Grafton**
http://www.gwu.edu/~scale-up/
Prepared for Researchers Without Borders Webinar, May 26, 2010
FOI Research Group
• Carol O’Donnell • Suzanne Merchlinski & MCPS evaluation staff• Bonnie Hansen-Grafton • Joelle Lastica• Vasuki Rethinam• Bill Watson• Rob Ochsendorf• Liz Hatchuel• Annie HansenWith special thanks to MCPS middle school science
teachers who participated in this study
Disclaimer
• Indebted to the Interagency Educational Research Initiative (IERI) administered by the NSF, for research funding for SCALE-uP (7 year research program).
• I am currently working at NSF as a Program Director in EHR/DRL ( return to my position as a professor at GWU in September).
• However, the ideas and opinions discussed here are entirely my own and in no way represent the those of NSF.
SCALE-uP/FOI Webinar:Cautionary Tale w/ Happy Ending • Background of SCALE-uP and initial Theory of Action• Year 0 Pilot Study: Curriculum modifications are
tricky business!• Year 1+: Comparison groups are (incredibly) handy in
developing FOI instruments and understanding the study context
• Year 2 & 3: Comparison group and FOI evidence are crucial for credible evidence of effectiveness
• Year 4: Putting it all together: How the Theory of Action was changed by FOI evidence: FOI as “process” and “structure” constructs for both teacher and student
Background for SCALE-uP and Initial Theory of Action
• In 1990’s, AAAS Project 2061 developed a Curriculum Analysis to identify curriculum materials likely to help students learn a target idea (benchmark/standard).
• Curriculum Analysis relied on experts’ judgment of written curriculum materials.
• Two parts:
--Focused, accurate, coherent content on a standard/benchmark
--Instructional strategies contained in written curriculum materials
Project 2061 Instructional Strategies for curriculum materials
1. Convey sense of purpose 2. Address student ideas and
misconceptions3. Promote engagement with relevant
phenomena4. Developing, using scientific ideas 5. Encourage student thinking6. Encourage assessment of progress7. Creating positive learning environment:
curiosity, all students
AAAS. Project 2061.
Background for SCALE-uP
• Project 2061 Curriculum Analysis had located only 2 acceptable curriculum units in middle school science.
• Units had been field-tested with small numbers of students (no comparison groups).
• Note. More mathematics curriculum materials had acceptable ratings and were field-tested and studied and scaled.
Background for SCALE-uP and Research Questions
If science curriculum materials having Project 2061 attributes were studied in a series of large (N = ~ 2000) quasi-experiments using carefully matched comparison groups:
• Would they be effective?• Would they be equitable? • Would there be a relationship between fidelity
of implementation to a unit and student outcomes?
• Could the materials be scaled-up in this large school district?
• How did they function in classroom (video-ethnography)?
SCALE-uP’s Interventions: 3 Science Units with Coherent
Focused on Target Ideas• State of Michigan’s Chemistry That Applies
(CTA) focuses on conservation of matter. 8th graders, unit ~ 6 weeks long.
• GEMS Lawrence Hall of Science Real Reasons for the Season (Seasons) focuses on the reasons for the Earth’s seasons. 7th graders, unit, ~ 3 weeks.
• ARIES Harvard Smithsonian Motion and Forces (M&F) focuses on portions of Newton’s Laws. 6th graders, unit ~ 6 weeks long.
Curriculum Analysis: Instructional Strategies
Instructional Category Chemistry That
Applies
ARIESMotion & Forces
GEMSSeasons
Macmillan/McGraw Hill
I. Identifying a Sense of Purpose
Conveying Unit Purpose ○ ○ NR ◔ Conveying lesson/activity purpose ◕ ◒ ● ◔ Justifying lesson/activity sequence ◒ ◒ ◒ ○II. Taking Account of Student Ideas
Attending to prerequisite knowledge and skills ◒ ○ ○ ○ Alerting teacher to commonly held ideas ◒ ○ NR ○Assisting teacher in identifying own students’ ideas ◒ ◒ ○ ○Addressing commonly held ideas ◒ ○ ◒ ○
● =Excellent, ◕=Very Good, ◒=Satisfactory, ◔=Fair ○=Poor
Instructional Category Chemistry That Applies
ARIES
Motion & Forces
GEMS
Seasons
Macmillan/
McGraw Hill
III. Engaging Students with Relevant Phenomena
Providing a variety of phenomena ● ● ○ ○ Providing vivid experiences ● ● ◒ ○ IV. Developing and Using Scientific Ideas
Introducing terms meaningfully ● ◒ ◒ ◔ Representing ideas effectively ◒ ◒ ● ○Demonstrating use of knowledge ◕ ○ ◒ ○Providing practice ● ○ ○ ○V. Promoting Student Thinking about Phenomena, Experiences, and Knowledge
Encouraging students to explain their ideas ● ◒ ○ ○Guiding student interpretation and reasoning ● ○ ● ○Encouraging students to think about what they’ve learned ○ ○ ○ ○
● =Excellent, ◕=Very Good, ◒=Satisfactory, ◔=Fair ○=Poor
SCALE-uP’s Outcome Measures
• Curriculum-independent measure for each unit focusing on the unit’s target idea.
• Assessments had good psychometric properties and were developed using a Project 2061 assessment system.
• Multiple choice and constructed response items designed to be maximally accessible to students of varied language skills.
Questions?
Background for SCALE-uP and Initial Theory of Action, c. 2001
• Curriculum units highly rated on Curriculum Analysis could be effective overall because: – each focused coherently on one big idea/standard/
benchmark– each had a carefully planned sequence of activities,
and – each contained identified instructional strategies
leading students to construct understanding of one target idea/benchmark/standard.
Big Question: Would they be equitable?Assumption: “Business as usual” comparison
classrooms would be less focused, rely more on textbooks and worksheets, and provide less time for guided inquiry and lab work.
“Typical” Theory of Action
TeacherCurriculum Materials
Student Outcomes
Fidelity of implementationMowbray, Holter, Teague & Bybee, 2003
• “Fidelity of implementation is the extent to which the delivery of an intervention adheres to the original program theory behind its development; it confirms that the implementation of the independent variable in outcome research occurred as planned…
• …(and involves) the dynamic nature of fidelity criteria, appropriate validation and statistical analysis methods, the inclusion of structure and process criteria in fidelity assessment and the role of program theory in deciding on the balance between adaptation versus exact replication of model programs.”
SCALE-uP Theory of Action c. 2001
TeacherCurriculum Materials
Student Outcomes
Teacher FOIInstructionalStrategies
SCALE-uP Theory of Actionc. 2001
• Teachers would need to locate and enact the instructional strategies embedded in the curriculum unit (identified via the Project 2061 Curriculum Analysis).
• Doing this well would be to “implement with fidelity”.• Hypothesis: The better the implementation of the
unit’s embedded instructional strategies in a classroom, the higher the student outcomes.
• All SCALE-uP needed to do to create a classroom observation instrument that captured teachers’ implementation of the strategies.
• This would become Instructional Strategies Classroom Observation Protocol (more on this later).
• Simple!
Questions?
On to Fidelity of Implementation (FOI)!
Year 0 (Pilot Study) Results: Chemistry That Applies (CTA) with Modifications Encouraged
• CTA’s results showed effect sizes = .52.• Results disaggregated for subgroups of students showed
that no students disadvantaged by CTA. • Teachers were asked to modify CTA for diverse learners
and record modifications.• Virtually none did! This raised questions about whether
to modify CTA in future. Teacher beliefs about the unit were mixed.
• Co-PI Szesze wanted to be sure units were unambiguously effective, or not.
• Decision to “implement with fidelity” in the future studies . • Teachers and researchers drew up fidelity guidelines
together.
Lesson Learned :Modifying Curriculum Unit is Tricky Business!
• If an intervention does not have solid
evidence of efficacy/effectiveness, then modifications muddy the water; CTA and the other units were “unproven” interventions.
• Therefore, their critical components could not be “known”, but assumed.
• In retrospect, this was a good decision for SCALE-uP; capturing modifications while trying to establish if a unit “worked” would not be credible in this school district context.
Iterative Process of Identifying Critical Attributes, Measuring Attributes, and
Looking for Relationships between FOI and Outcomes
Identify CriticalAttributes of Intervention
Outcomes
Mowbray, C., Holter, M. C., Teague, G. B., & Bybee, D. (2003). Fidelity criteria: Development, measurement, and validation. American Journal of Evaluation, 24(3),315-340.
FOI measureReconsider
CriticalAttributes
Questions?
On to Year 1
Replication of CTA
Building the first FOI Instrument
Yr. 1 Replication of CTA and Develop ISCOP/FOI Process
• Given Theory of Action (the more teachers implemented Project 2061 instructional strategies embedded in curriculum units, the higher student outcomes), we began to develop an instrument that could capture fidelity to identified Project 2061 instructional strategies.
• A “generic” instrument for all 3 units studied, but the units had been carefully vetted and had much in common.
• ISCOP (Instructional Strategies Classroom Observation Instrument), the first FOI measure was born, and developed over the next 4 years.
Effect Sizes: CTA (Year 1)
Years 1, 2, 3, 4: Test ISCOP in Treatment and
Comparison Classrooms• Was the ISCOP capturing fidelity to instructional
strategies in a way that discriminated between Treatment and Comparison classrooms?
• ISCOP did not discriminate very well, so it was refined and refined and refined.
• Was ISCOP a “bad” instrument? Or were Treatment and Comparison classrooms similar for Instructional Strategies?
• If Treatment and Comparison classrooms were similar in strategies, should the Theory of Action be revisited?
Lesson Learned: Comparison Classrooms Invaluable as Counterfactuals
• ISCOP data suggested that more complexity than assumed in the Theory of Action.
• Measuring FOI relying solely with ISCOP might not answer FOI research question if comparison classroom data were taken into consideration.
• Generic measures of instructional strategies/FOI process are notoriously hard to develop—valid and reliable?
Questions?
On to Years 2 and 3 with a new unit, Real Reasons for the Seasons
(Seasons)
Year 2 and 3 Results: Comparison group outscores
Seasons group: How to account for this?
• Seasons studied in 7th grade classrooms (N ~ 2000 students and ~ 40 classrooms).
• Year 2 and 3 results: Comparison group has higher outcome than Seasons on curriculum independent measure (ES = -.36, -.18, respectively).
Comparison Classrooms Invaluable as Counterfactuals
• Comparison classroom teachers surveyed and interviewed.
• Data showed that Comparison classrooms were:
-Non-traditional-Variety of curriculum materials used,
including other inquiry units-Focused on the target idea -Equal Duration of Treatment and
Comparison units-Comparison and Treatment teacher
characteristics similar.
Seasons Replication in Year 3: Focus on FOI
• Use ISCOP in Treatment and Comparison classrooms.
• Develop a new Lesson Flow FOI process measure that gauged Teacher, Student-Group or Individual Centeredness of classrooms based on hunch that Student-Group Centeredness was important to student construction of science ideas in groups.
• Videotaped a Seasons and Comparison classroom.
• Interviewed and surveyed Seasons and Comparison Teachers again.
Lesson Flow Classroom Observation Instrument
Lesson Flow Classroom Observation Protocol (LFCOP)
Effect Sizes: Seasons (Year 3)Overall ES = -.18
Year 3 FOI Results• Only 3 items on ISCOP showed significant
differences between Seasons and Comparison classrooms, and 2 of 3 were observed more often in Comparison classrooms.
• Lesson Flow: Seasons classrooms were more teacher-centered than Comparison:
Teacher-CenterednessSeasons = 71 % of timeComparison = 58 % of time
• Video-data backed this up.
• Emerging Conjecture: Students need time in to work and talk in groups to develop their ideas and Seasons allowed less time for this.
SCALE-uP Theory of Action c. 2006
TeacherCurriculum Materials
Student Outcomes
Teacher FOIInstructionalStrategies
Student FOIStudent GroupCenteredness
Changing Theory of Action and Approach to FOI c. 2006
• Focusing FOI entirely on teacher in a guided inquiry unit may be simplistic.
• Student agency implicated in FOI constructs.• But Lesson Flow is controlled by the teacher
who provides students with the time/space to work in groups; students can choose to do the science work, or not.
• Lesson Flow is not Time On Task.
Seasons Unit and FOI:The Research Re-visited
• Seasons was designed to be a supplementary unit.
• Developers seemed not to understand the implications of this when agreement was made to study effectiveness/FOI.
• SCALE-uP researchers did not appreciate the differences in philosophy of Seasons with research design.
• Seasons was a bad match for this study.• Revisions made to Seasons based upon
feedback from MCPS teachers.
Questions?
On to Year 4 and Motion and Forces(M&F)
Note this is the third curriculum unit studied, not to be confused with the first two, and was
implemented in 6th grade classrooms.
M&F Results in Years 2 and 3:Underwhelming
• Year 2 ES = +.10• Year 3 ES = -.06• FOI had not been emphasized, so there was
little information to explain lackluster outcomes.
• However, we learned that students had not been issued M&F student Journals in Years 2 and 3. Was this an important, overlooked FOI issue?
Year 4 Replication of M&F: Focus on FOI
• Replicated quasi-experiment in 10 new schools (N ~ 2000 students).
• Teachers asked to focus on FOI.
• Wonderful teachers who understood the study goal: study the impact of M&F.
FOI in Year 4 with M&F
FOI included:• ISCOP (Process FOI)• Lesson Flow (Process FOI)• Adherence to unit (ACOP), a new instrument
that measured close adherence to M&F’s structure (Structure FOI)
• Teacher Interviews/Surveys• Teacher Logs • Student Journal Entries, # of responses to
Journal (Structure FOI)• Student survey about self-reporting use of
instructional strategies (Process FOI)
Overall Results for M&F in Year 4
• Student Level results using traditional ANOVA, ES = .23
• Classroom Level results using HLM,
ES = .56 (Rethinam, Lynch, & Pyke, 2008)
Effect Sizes for subgroups of students: M&F (Year 4)
ISCOP Strategies: Means and Correlations with Outcomes for M&F
Instructional Criterion Mean (Scale = 0 - 3) Correlation with Outcomes
Treatment Comparison Treatment Comparison
Identifying a Sense of Purpose
Conveying lesson/activity purpose 0.65 0.58 .57** -.28
Justifying lesson/activity purpose .46 .58 .56** -.40
Taking Account of Student Ideas
Assisting teacher in identifying own students’ ideas
2.40 1.38 -.05 -.52**
Engaging Students with Relevant Phenomena
Providing variety of phenomena 1.71 1.50 .44* -.23
Providing vivid experiences 2.75 2.46 .13 .08
Developing and Using Scientific Ideas
Introducing terms meaningfully 2.48 2.38 .60** -.09
Representing ideas effectively 1.46 1.13 .16 -.01
Promoting Student Thinking about Phenomena, Experiences, Knowledge
Encouraging students to explain their ideas
2.13 2.25 .62** -.20
ISCOP• This “generic” instrument for measuring FOI
instructional strategies did not obviously distinguish between M&F and Comparison classrooms.
• But some items were correlated with student outcomes for M&F classrooms.
• O’Donnell dissertation elegantly teased out which instructional strategies seemed to matter for higher student outcomes.
• ISCOP needs more work on validity and reliability--OR SCALE-uP Theory of Change needs refinement—Project 2061 Curriculum Analysis needs refinement?
TeacherIndividual
Group
TeacherGroup
Individual
M&F Comparison
Lesson Flow for Instruction for units on Motion and Force
Lesson Flow
• M&F classrooms provided more time for students to work in groups and individually than Comparison classrooms.
• Students construct meaning in groups, consistent with notion of community of practice and situated cognition.
Adhering to M&F Lesson Components (ACOP)
• M&F teachers adhered to M&F unit > 80% of time.
• ACOP did not predict outcomes because the range was narrow; this was a good thing for this study because high fidelity in this measure of “structure” is credible and strong—good face validity for FOI, and highly reliable—the unit was well-implemented.
• Teachers adhered to the unit for this study, to ascertain impact of M&F.
• Thank you teachers.
Results: Teacher FOI Structure
Adherence Classroom Observations Protocol ACOP Item
n %Not implemented
%Somewhat implemented
%Mostly implemented
%Fully implemented
Teacher provides students with materials
30 -- -- 53.3 46.7
Teacher provides opportunity to write responses
30 -- -- 10.0 90.0
Teacher discuss with students responses to questions
30 13.3 3.3 3.3 80.0
Teacher provides opportunity to set up equipment
30 10.0 -- 33.3 56.7
Teacher provides opportunity to record and/or discuss predictions
8 12.5 -- -- 87.5
Teacher provides opportunity to conduct investigation
30 -- 3.3 6.7 90.0
Teacher provides opportunity to reflect and write
20 15.0 -- -- 85.0
Interpreting results and discussion in writing
30 -- -- 13.3 86.7
Interpreting results and discussion verbally
30 13.3 6.7 20.0 60.0
Teacher Guide Additions
24 41.7 41.7 8.3 8.3
Adhering to sequence of activities
30 -- 3.3 46.7 50.0
Student Journals
• M&F students completed their journal responses ~ 80% of time.
• Rates of journal question completion predicted classroom outcomes.
Unpublished Year 4 HLM Results for M&F: ISCOP, ACOP, Lesson Flow,
Student Journals• HLM analysis found one classroom-level
factor that predicted student outcomes:
--Amount of Student-Group-Centeredness (Lesson Flow)
• and one student level factor
--Student Journal Completion • Thanks to Dr. Jaewa Choi of GWU for this data analysis.
SCALE-uP Theory of Action c. 2010
TeacherCurriculum Materials
Student Outcomes
TeacherInstructionalStrategies
Teacher Adherence to Lesson Structure
Student GroupCenteredInstruction
Student Adherenceto Lessons
ISCOP ACOP
Lesson Flow Student Journal
ProcessFOI
ProcessFOI
StructureFOI
StructureFOI
SCALE-uP Theory of Action c. 2010
TeacherCurriculum Materials
Student Outcomes
TeacherInstructionalStrategies
Teacher Adherence to Lesson Structure
Student GroupCenteredInstruction
Student Adherenceto Lessons
ISCOP ACOP
Lesson Flow Student Journal
SCALE-uP Theory of Action c. 2010
Teacher
Curriculum Materials
Student Outcomes
TeacherInstructionalStrategies
Teacher Adherence to Lesson Structure
Student GroupCenteredInstruction
Student Adherenceto Lessons
ISCOP ACOP
Lesson Flow Student Journal
Students
SCALE-uP Theory of Action c. 2010
Teacher
Curriculum Materials
Student Outcomes
TeacherInstructionalStrategies
Teacher Adherence to Lesson Structure
Student GroupCenteredInstruction
Student Adherenceto Lessons
ISCOP
ACOP
Lesson Flow
Student Journal
Students
Questions?
Summary Findings• Score Card: CTA and M&F seemed to be effective
and equitable in this school system, Seasons did not seem to be effective.
• Theory of Action: Changed from looking at one relatively “generic” measure of teacher process fidelity to multiple measures of FOI adding student FOI and measures of FOI structure, and developing a better Theory of Action.
• Lesson Flow and Student Journal response best predicted student outcomes; ISCOP analysis illuminated most potent instructional strategies for M&F.
FOI Lessons Learned• Studying curriculum adaptations for an
“unproven” curriculum is a slippery slope because critical ingredients are unknown.
• Study of FOI in Comparison groups provides important counterfactual.
• This study shows potential for both teacher and student FOI, as well as process and structural approaches.
• Study of FOI should reveal more about critical components of an intervention if measures of each component shows a positive relationship with student outcomes.
The End
Thanks to Researchers Without Borders for sponsoring this Webinar.
References• American Association for the Advancement of Science (AAAS). (2003). Project 2061 middle grades
science textbooks: A Benchmarks-based evaluation. Retrieved June 1, 2004, from http://www.project2061.org/tools/textbook/mgsci/index.htm.
• Dusenbury, L., Brannigan, R., Falco, M., & Hansen, W. B. (2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research Theory and Practice, 18(2), 237-256.
• Harvard-Smithsonian Center for Astrophysics. (2001). ARIES: Exploring motion and forces: Speed, acceleration, and friction. Watertown, MA: Charlesbridge Publishing
• Kesidou, S., & Roseman, J.E. (2002). How well do middle school science programs measure up? Findings from Project 2061’s curriculum review. Journal of Research in Science Teaching, 39(6), p. 522-549
• Lastica, J.R., & O'Donnell, C.L. (2007, April). Considering the role of fidelity of implementation in science education research: Fidelity as teacher and student adherence to structure. In C. O'Donnell (Chair), Analyzing the relationship between Fidelity of Implementation (FOI) and student outcomes in a quasi-experiment. Symposium conducted at the Annual Meeting of the American Educational Research Association, Chicago, IL.
Lawrence Hall of Science. (2000).GEMS: The real reasons for seasons—Sun-Earth connections. Berkeley: The Regents of the University of California.
• Lynch, S. (2000). Equity and science education reform. Mahwah, NJ: Lawrence Erlbaum and Associates.
• Lynch, S., Kuipers, J.C., Pyke, C., & Szesze, M. (2005). Examining the effects of a highly rated science curriculum unit on diverse students: Results from a planning grant. Journal of Research in Science Teaching, 42(8), 912-946.
• Lynch, S. & O’Donnell, C. (2005, April). The evolving definition, measurement, and conceptualization of fidelity of implementation in scale-up of highly rated science curriculum units in diverse middle schools. In S. Lynch (Chair), The role of fidelity of implementation in quasi-experimental and experimental research designs: Applications in four studies of innovative science curriculum materials and diverse student populations. Symposium conducted at the Annual Meeting of the American Educational Researchers Association, Montreal, Canada.
• Lynch, S., O'Donnell, C., Hatchuel, E., & Rethinam, V. (2007, April). A model predicting student outcomes in middle school science classrooms implementing a highly-rated science curriculum unit. Paper presented at the Annual Meeting of the National Association for Research in Science Teaching, New Orleans, LA.
• Lynch, S., O'Donnell, C., Hatchuel, E., Rethinam, V., Merchlinsky, S., & Watson, W. (2006, April). What’s up with the Comparison group?: How large quasi-experimental study of highly rated science curriculum units came to grips with unexpected results. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA.
• Lynch, S., Taymans, J. Watson, W., Ochsendorf, R., Pyke, C. & Szesze, M. (2007). Effectiveness of a highly-rated science curriculum unit for students with disabilities in general education classrooms. Exceptional Children, 73(2), 202-223.
• Merchlinsky, S. & Hansen-Grafton, B. (2007, April). Considering the role of Fidelity of Implementation (FOI) in science education research: Evaluation and science specialists' role in collecting FOI data in a large school district. In C.L.
• Michigan Department of Education. (1993). Chemistry That Applies. The State of Michigan.
• Mowbray, C., Holter, M. C., Teague, G. B., & Bybee, D. (2003). Fidelity criteria: Development, measurement, and validation. American Journal of Evaluation, 24(3),315-340.
• O’Donnell, C. L. (2007). Fidelity of implementation to instructional strategies as a moderator of curriculum unit effectiveness in a large-scale middle school science experiment. Dissertation Abstracts International, 68(08). (UMI No. AAT 3276564)
• O'Donnell, C.L., Lynch, S., Lastica, J., & Merchlinsky, S. (2007, April). Analyzing the relationship between Fidelity of Implementation (FOI) and student outcomes in a quasi-experiment. Symposium conducted at the Annual Meeting of the American Educational Research Association, Chicago, IL.
• O'Donnell, C.L., Lynch, S., Watson, W., & Rethinam, V. (2007, April). Teacher and student Fidelity of Implementation (FOI) to process: Quality of delivery and student responsiveness and relationships to classroom achievement. In C.L. O'Donnell (Chair), Analyzing the relationship between Fidelity of Implementation (FOI) and student outcomes in a quasi-experiment. Symposium conducted at the Annual Meeting of the American Educational Research Association, Chicago, IL.
• O'Donnell, C. (2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K-12 curriculum intervention research. Review of Educational Research, 78(1), 33-84.
• O'Donnell, C., & Lynch, S. (2008, March). Fidelity of implementation to instructional strategies as a moderator of science curriculum unit effectiveness. Paper presented at the 2008 annual meeting of the American Educational Research Association, New York, NY.
• Rethinam, V., Pyke, C., & Lynch, S. (2008). Using multilevel analyses to study the effectiveness of science curriculum materials. Evaluation and Research in Education, 21(1), 18-42
• Rethinam, V., Pyke, C., & Lynch, S. (in preparation.). Using Multilevel Analyses to Study Individual and Classroom Factors in Science Curriculum Effectiveness.
• Songer, N. B., & Gotwals, A. W. (2005, April). Fidelity of implementation in three sequential curricular units. In S. Lynch (Chair), “Fidelity of implementation” in implementation and scale-up research designs: Applications from four studies of innovative science curriculum materials and diverse populations. Symposium conducted at the meeting of the Annual Meeting of the American Educational Research Association. Montreal, Canada.
• Stern, L. & Ahlgren, A. (2002). Analysis of students’ assessments in middle school curriculum materials: Aiming precisely at benchmarks and standards. Journal of Research in Science Teaching, 39, 889-910.
• Watson, W., Lynch, S., Rethinam, V., & O’Donnell, C. (2006, April). Development of an instrument to measure student responsiveness to implementation of science curriculum materials. Paper given at the annual meeting of the National Association for Research in Science Teaching, April, 2006 , San Francisco.
• Further acknowledgments of important contributions to this work on fidelity of implementation made by Rob Ochsendorf, Aiyita Ruiz-Primo, Doug Clement, Okhee Lee, Bruce Ward, Carolyn Walton, Theron Blakeslee, Andy Anderson, Phyllis Blumenfeld.
top related