professional development for teachers of mathematics: what do we know and how well do we know it?

Iris Weiss Barbara Miller Horizon Research, Inc. Education Development Center

Professional Development for Teachers of Mathematics:

What do we know and how well do we know it?

Goal: To synthesize knowledge generated through the Math and Science Partnerships and integrate it into the broader knowledge base for education reform– Deepening Teacher Content Knowledge– Teachers as Intellectual Leaders– Involvement of STEM faculty– Professional Learning Communities

MSP Knowledge Management and Dissemination Project

A new mathematics district leader, with responsibility for mathematics professional development, comes to you for advice: What should I do to support a solid and effective professional development program across my district?

There are many things you are now thinking about. Your task is to talk at your table with others about what specific pieces of advice you might offer.

I’ll ask you to share out from your table a single idea for which there was reasonable consensus.

A new mathematics district leader, with responsibility for mathematics professional development, comes to you for advice: What should I do to support a solid and effective professional development program across my district?

• Why to engage in professional development? What are the goals?

• What to do in professional development? What are the strategies?

• Who will lead professional development? What is the delivery?

• What are the conditions for professional development? What are the resources?

• Professional development design and implementation is complex, with many decisions to make.

• Decisions are interrelated.

• Our eyes are always bigger than our stomachs – we always want to do more in professional development than we are able to do.

What do we know?

What do we want to know?

1. Is professional development important?

2. What are effective professional development strategies?

3. For which teachers are those strategies effective, and under what conditions?

Professional Development

TeacherKnowledge and Skills

TeachingPractice

StudentOutcomes

Simplified Logic Model for Professional Development

Professional Development

TeacherContent

Knowledge

TeachingPractice

StudentOutcomes

Teacher content knowledge matters…

For teaching practice:

•Selecting content to emphasize•Selecting instructional strategies and sequence•Selecting assessment tasks•Implementing curriculum materials

Professional Development

TeacherContent

Knowledge

TeachingPractice

StudentOutcomes

Teacher content knowledge matters…

For student learning:Students of teachers with stronger content knowledge have higher achievement scores than other students, in particular on measures of conceptual understanding.

So if TCK matters…

What do we know about deepening teacher content knowledge?

Facets of Teacher Content Knowledge

• Disciplinary content knowledge

• Pedagogical content knowledge

• Ways of knowing content

What facets of content to choose in content-focused PD

• Most people seem to agree that all of these facets are important for teaching

• With unlimited time and resources, you would likely address them all

• But we don’t have unlimited time and resources, so choices have to be made

Plan a PD program

• Middle grades mathematics teachers

• Focus on algebraic thinking

• 30 hour PD workshop in the summer

• There will also be an academic-year component, but we’re not concerned with that right now.

Sequencing PD Goals

One line of reasoning:

• Teachers can’t learn to teach what they don’t know. Therefore, it is important to start with mathematics content, and only after teachers themselves have a sufficiently deep understanding of the content, move to considering classroom application.

Sequencing PD Goals

Another line of reasoning:

• Teachers are by their very nature practitioners. Starting with classroom applications, e.g., trying to analyze student work, provides a purpose and context for engaging the teachers in learning mathematics content.

Sequencing PD Goals

• Available research does not indicate which approach is “better” under a particular set of conditions.

MSP KMD Review of Research

• Identified more than 1000 “studies” on PD to deepen teacher mathematics content-related knowledge

MSP KMD Review of Research

• However, approximately 90% of the studies were screened out because:

– They were advocacy or opinion pieces, not research, and/or

– They were studies of pre-service teachers only, and/or

– They did not include a measure (quantitative or qualitative) of teacher content knowledge.

MSP KMD Review of Research

• Applied standards of evidence to 28 studies of mathematics PD, those that were not simply opinion or advocacy pieces and actually measured teacher content knowledge

What research tells us

• Available research points to some elements of effective PD, but provides very little guidance about how to design and implement PD for particular purposes in particular kinds of situations.

Example 1

Opportunities to learn about student mathematics curricula were positively related to…

• reports of classroom practices advocated in the California mathematics frameworks; and

• student performance.

Cohen & Hill, 2000

Example 2

• Longer duration/more contact hours; and• Opportunity to engage in mathematical

analysis, reasoning, and communication

… were positively related to teacher learning of mathematics content knowledge for teaching.

Hill & Ball, 2004

Example 3

• Focus on a specific practice/set of practices• Coherence with other PD• Active learning opportunities• Collective participation of teachers• “Reform types” of PD focused on higher order

instructional/assessment practices

…were positively related to changes in teachers’ instruction, assessment, and/or technology practices.

Desimone, Porter, Garet, Yoon, & Birman, 2002

An Emerging ConsensusEffective PD:

• Focuses on content knowledge and how students learn content

• Involves a substantial number of hours• Sustains focus over time• Models effective practice, including active learning

experiences• Engages teachers in communities of learning• Involves active participation of school leaders

Study in progressA randomized controlled trial is testing the

consensus view with a PD program that includes…

• Substantial number of contact hours over a full-year duration, including summer institutes, academic year seminars and in-school coaching

• Focus on developing teachers’ content and pedagogical content knowledge

• Collective participation of teachers in a school

Preliminary findings

After one year, the study found impact on …

• Teachers’ use of instructional practices to elicit student thinking

But no statistically significant impact on …

• Teacher content knowledge,• Teachers’ use of representations in instruction,• Teachers’ focus on mathematics reasoning in instruction,• Student achievement.

Garet et al., 2010

To explain these results,one might consider whether

• The content was appropriate for these teachers

• The instruments were sensitive to impacts that were occurring

• There was sufficient time for impacts to occur

An alternative explanation

• The current understanding of PD effectiveness is incorrect or underspecified.

Why don’t we know more from the empirical research?

• In applying standards of evidence, we often found vague or incomplete documentation of programs or interventions.

• Consequently, we know something worked, but we don’t know a lot about what “it” was.

Why don’t we know more?

• Studies tended to be more like program evaluations rather than research on particular strategies.

• Consequently, we know the overall experience worked, but we don’t know how much particular interventions contributed to the gains.

Why don’t we know more?

• We often found serious limitations with study research designs, including:– Selection bias in samples and contexts– Lack of comparison groups or criteria– Idiosyncratic instrumentation, without

evidence of validity/reliability/credibility

Why don’t we know more?

• There are too few studies of any one phenomenon to be able to have confidence in the robustness of the findings in any case.

• High quality research is expensive, which may explain why so many in-depth studies in the literature involved fewer than 5 teachers.

Why don’t we know more?• There is a tension between design for change

and design for learning.

• From a system change perspective, if you plan on having two cohorts, it makes sense to start with the schools that are “ready.”

• But doing so makes research on program effectiveness problematic; it will not be possible to disentangle the effects of differences in readiness from the impact of the interventions.

Why don’t we know more?

• Even when individual studies are well-designed and well-implemented, it is difficult to look across them and figure out the extent to which the findings might generalize.

There is a difference between…

• What has worked

• What will work in my context

Filling the gap

• “Sensible propositions” can provide guidance and serve as hypotheses for research

• MSP-KMD project developed a methodology for collecting and vetting practice-based insights.

Comparing Empirical Research and Practice-based Insights

• We were surprised at how little guidance the available research provides and how much guidance expert practice provides, although without the backing that empirical research would provide.

**

Comparing Empirical Research and Practice-based Insights

• Empirical findings tend to be larger grain size; practice-based insights tend to be more contextualized and nuanced.

Research + Practice = Guidance

in designing and implementing

professional development

Focus on teacher leaders as one part of a professional development program

Teacher leaders matter in terms of impacting teachers’ classroom practice, with limited evidence that TL work impacts students.

Teacher leader work takes different forms (e.g., leading pd, modeling lesson, planning)

What do we know from research on teacher leaders?

Little guidance in terms of choosing particular practice, how to implement it, or under what conditions.

What do we know from research on teacher leaders?

Articulate the purpose and nature of TL practice(s)– Each TL shouldn’t figure this out individually– Not the time to “let 1000 flowers bloom” because

impossible to support– Impact is diluted when focus is dispersed– Trade-off between clear, shared articulation and local

response

What do we know from practice-based insights on teacher

leaders?

Make decisions based on available resources and adjust program accordingly– TL practice should fit available time for working with

teachers– TL practice should be calibrated to available support

from school, district, beyond– Trade-off between “playing with the cards you’re

dealt” and “pushing the envelope”

What do we know from practice-based insights on teacher

leaders?

Develop alignment among TL selection, TL preparation and TL practice

What do we know from practice-based insights on teacher

leaders?

1. You want teacher leaders to be leading school-based professional development in department and faculty meetings and district-wide professional development in after-school and summer workshops. Given this, who do you select as TLs?

Task 1

2. Your selection process yields TLs who are enthusiastic, have credibility with their colleagues, have limited prior experience leading professional development, and display a wide range of content knowledge. Given this, how do you think about preparing TLs for the practice of leading professional development?

Task 1

3. There is limited time and even fewer resources to devote to preparing these TLs to lead professional development. Given this, how do you think about the intended TL practice?

Task 1

In your experience with teacher leaders as part of a professional development program, what was the alignment between TL practice, TL selection, and TL preparation?

Where did the alignment break apart (and wishful thinking take its place)?

Construct your own triangle and identify how/whether each is aligned.

Task 2

TL practice

TL preparation TL selection

Develop alignment among TL selection, preparation and practice– Intended TL practice is an important first consideration– Selecting fewer, highly-qualified TLs may be preferable to

selecting larger numbers of less-qualified candidates– Be clear about content knowledge needed for TL practice and

whether it is a realistic selection criteria or preparation expectation

– Trade-off between TL practice, selection and preparation

What do we know from practice-based insights on teacher

leaders?

KMD Resources

• Knowledge reviews on TCK, TL and STEM

• Measures of teacher content knowledge

• Sustainability cases on TCK and TL

www.mspkmd.net

Sample Knowledge Reviews

• Teacher knowledge: Engaging with challenging mathematics/science content

• Selecting teacher leaders

• Involving STEM disciplinary faculty in deepening teacher/teacher leader content knowledge

How you can contribute• Use the best evidence available;• Document your treatment, including types of

teacher content knowledge addressed, in what ways, in what sequence, etc.;

• Describe your participants and your context;• Monitor how well it worked and how you

know; • Share results/lessons learned with the field.

REFERENCES• Cohen, D. K. & Hill, H. C. (2000). Instructional policy and classroom

performance: The mathematics reform in California. Teachers College Record, 102(2), 294–343.

• Desimone, L. M., Porter, A. C., Garet, M. S., Yoon, K. S., & Birman, B. F. (2002). Effects of professional development on teachers’ instruction: Results from a three-year longitudinal study. Educational Evaluation and Policy Analysis, 24(2), 81–112.

• Garet, M. S., Wayne, A. J., Stancavage, F., Taylor, J., Walters, K., Song, M., Brown, S., Hurlburt, S., Zhu, P., Sepanik, S., Doolittle, F., & Warner, E. (2010). Middle school mathematics professional development impact study: Findings after the first year of implementation (NCEE 2010-4009). Alexandria, VA: U.S. Department of Education.

• Hill, H. C. & Ball, D. L. (2004). Learning mathematics for teaching: Results from California's mathematics professional development Institutes. Journal for Research in Mathematics Education, 35(5), 330–351.