the impact of the mmp on student achievement cindy m. walker, phd jacqueline gosz, ms university of...
TRANSCRIPT
The Impact of the MMP on Student Achievement
Cindy M. Walker, PhD
Jacqueline Gosz, MS
University of Wisconsin - Milwaukee
Primary Goal
• Within a school it is ultimately the teacher that is responsible for teaching mathematics that has the greatest capacity to impact student achievement in mathematics.
• How is the MMP influencing what is going on in mathematics classrooms across the district?
Structure of MMP in MPSMath SpecialistsMath Teacher LeadersLearning TeamsLiteracy CoachesMath TeachersStudent Achievment
Concerns
• Much of the effect of the MMP is not under the direct control of the MMP
• The effect of the MMP on student achievement depends on– School Climate– Quality of Learning Team– Quality of MTL– Support of other Teachers
Solutions
• Capture school level variability in variables that are not under the direct control of the MMP
• Compare and contrast schools to determine if under the best conditions there is an increase in student achievement in mathematics
Methodology
• On-line survey made available to all MPS participants in the MMP that may have an effect on increasing student achievement in mathematics including– Learning Team Members– Literacy Coaches– Math Teacher Leaders– Math Teachers
Participation Rate
• A total of 1,107 MPS employees from 129 schools in the district responded to the survey
• 93% of the elementary schools, 90% of the middle schools, and 9% of the high schools were represented in varying degrees.
Participation Rate by Role
School N LC LT MTL MT %MT
Elementary 1,007 38 228 95 645 28.4%
Middle 87 9 19 18 41 22.2%
High 13 0 2 2 9
Instrumentation
• Six primary areas were evaluated1. Professional Development
2. School/District Functioning
3. Learning Team Functioning
4. Classroom Practice
5. Role of the MTL
6. Role of the LC
Professional Development
• How involved were respondents in organized professional development activities geared toward mathematics this year?
Role in the MMP N Mean SD
Literacy Coach 44 16.06 16.08
Learning Team Member 56 10.67 13.10
LT Member & Math Teacher 123 15.15 19.16
Math Teacher Leader 82 54.19 33.21
Math Teacher Only 513 12.36 15.07
Professional Development• How often did respondents participate
in professional learning experiences related to the teaching and learning of mathematics this year?
Role of Respondent N Mean SD
Literacy Coach 44 3.98 2.88
Learning Team Member 56 3.22 2.83
LT Member & Math Teacher 123 4.42 3.08
Math Teacher Leader 82 6.47 2.59
Math Teacher Only 513 3.44 3.10
Professional Development
• How involved were respondents in informal professional development activities geared toward mathematics this year?
Role of Respondent N Mean SD
Literacy Coach 46 54.09 77.13
Learning Team member only 74 38.41 56.32
LT member & Math Teacher 162 85.09 101.58
Math Teacher Leader 121 99.69 95.11
Math Teacher Only 660 70.45 87.26
Professional Development
• How useful were professional development activities, in terms of helping teachers improve their classroom teaching of mathematics?
Role of Respondent N Mean SD
Learning Team Member 162 3.08 .53
Math Teacher Leader 110 3.39 .46
Math Teacher Only 654 2.91 .52
School Functioning
• How strong is the focus on increasing student achievement in mathematics in MPS schools?
There is a Strong Focus on Increasing Student Achievement in Mathematics at My School
Role of Respondent SD D A SA
LC (n = 47) .0% 17.4% 32.6% 50.0%
LT (n = 83) 1.3% 6.3% 38.8% 53.8%
LT_math (n = 170) 4.2% 4.2% 34.3% 57.2%
MTL (n = 126) 3.2% 12.0% 40.0% 44.8%
MT (n = 692) 3.7% 8.2% 42.9% 45.2%
School Functioning• How cohesive are the mathematics
curriculum and pedagogical approaches used in MPS schools? SD D A SA
Teachers have different expectations about what students can learn
7.6% 42.0% 37.9% 12.5%
Teachers often individually choose the content they teach
12.6% 42.1% 36.0% 9.3%
Teachers are encouraged to use similar instructional practices
3.2% 13.8% 58.1% 24.9%
Within each grade, teachers are expected to follow the same curriculum
3.1% 8.4% 51.9% 36.6%
Across grades, teachers are expected to follow the same curriculum
3.9% 19.4% 48.9% 27.9%
School Functioning
• To what extent do respondents participate in school improvement efforts?
Role of Respondent N Mean SD
Literacy Coach 45 115.20 76
Learning Team Member 75 103.96 80
LT Member & Math Teacher 160 130.20 81
Math Teacher Leader 120 133.18 72
Math Teacher Only 659 81.61 70
School Functioning
• How aligned do respondents feel their mathematics program is with standards and assessments?
D A SA
My school’s math program is aligned with MPS learning targets
9.2% 57.5% 28.6%
My school’s math program is aligned with WI state standards
8.9% 58.2% 28.2%
My school’s math program is aligned with the Comprehensive Math Framework
10.5% 59.0% 23.5%
My school’s math program is aligned with state/district assessments (WKCE/Terra Nova)
14.0% 56.6% 22.4%
School Functioning• What information is used to plan
and evaluate school improvement activities?
Terra Nova/WKCE
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Not at all To a smallextent
To a moderateextent
To a greatextent
LC (N=47)
LT (N=83)
LT_ MT (N=170)
MTL (N=126)
MT (N=692)
School Functioning
Student Work Samples
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Not at all To a smallextent
To a moderateextent
To a greatextent
LC (N=47)
LT (N=83)
LT_ MT (N=170)
MTL (N=126)
MT (N=692)
School Functioning
Classroom Observations
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Not at all To a smallextent
To a moderateextent
To a greatextent
LC (N=47)
LT (N=83)
LT_ MT (N=170)
MTL (N=126)
MT (N=692)
School Functioning
Information Learned About Effective Instructional Practices
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Not at all To a smallextent
To a moderateextent
To a greatextent
LC (N=47)
LT (N=83)
LT_ MT (N=170)
MTL (N=126)
MT (N=692)
School Functioning
Information Learned About Student Learning Processes
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Not at all To a smallextent
To a moderateextent
To a greatextent
LC (N=47)
LT (N=83)
LT_ MT (N=170)
MTL (N=126)
MT (N=692)
District Functioning• How much support do the district’s
learning targets provide?
– Regardless of role in the MMP, the majority of respondents felt that the district’s math learning targets
1. Provide clear information about what students are expected to know and be able to do and
2. Drive their school’s math improvement agenda
District Functioning• How much support do the district’s
math teaching specialists provide? – The majority of LC, MTL, and LT
members that do not teach math felt that the district’s math teaching specialists provided ample support
– Nearly 40% of respondents who teach math teachers did not feel that the district’s math teaching specialists provided ample support. Most likely because they have the least amount of contact with them
Learning Team Functioning
• Who sets the agenda for LT meetings?– 76.5% of LT members indicated that
the principal sets the agenda – 61.6% indicated that the LC sets the
agenda– 37.4% reported that the MTL sets the
agenda– 28.9% reported that someone other
than the LC, principal or MTL set the agenda
Learning Team Functioning
• Who provides input into setting the LT agenda?– 86.3% of LT members respondents
reported that the principal provides input
– 77.3% of respondents reported that the LC provides input
– 69.7% of respondents reported that the MTL provides input
– 55.5% reported that others provide input
Learning Team Functioning
• How often is mathematics discussed at LT meetings? – 19.6% of LT members (n = 419)
reported that math was discussed all of the time
– 16.1% reported three-fourths of the time– 23.3% reported half of the time– 37.3% reported one-fourth of the time– 3.6% LT members reported never
Learning Team Functioning
• How often is mathematics a formal agenda item for LT meetings?
Role of Respondent
MTL(n = 123)
Other LT Member(n = 286)
Never 2.4% 2.1%
Some of the time 36.6% 22.7%
Most of the time 36.6% 32.5%
Always 24.4% 42.7%
Learning Team Functioning
• Does the LT regularly meet to plan and problem solve about issues surrounding mathematics?
Role of Respondent
MTL (n = 123)
Other LT Member (n = 292)
Strongly Disagree 6.5% 3.4%
Disagree 32.5% 18.8%
Agree 46.3% 47.3%
Strongly Agree 14.6% 30.5%
Learning Team Functioning
• What are the work priorities of LT across the district?– Much variability across the district– Supporting the implementation of
improvement efforts in math most often reported as high or top priority
– Monitoring the implementation of improvement efforts, developing and implementing staff development programs was least often reported as high or top priority
Learning Team Functioning• How often are specific math issues
discussed at LT meetings?Useful Discussions About How to Help Students Succeed
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Some of the time Most of the time Always
MTL
LT Member
Learning Team Functioning
Able to Accomplish Goals Related to Math
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Some of the time Most of the time Always
MTL
LT Member
Learning Team Functioning
MTL Shares Info Gained from District Meetings
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Some of the time Most of the time Always
MTL
LT Member
Learning Team Functioning
Collaborate to Construct and Implement Action Plan
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Some of the time Most of the time Always
MTL
LT Member
Learning Team Functioning• How much of an asset is the LC in
helping schools focus improvement in math?
LC Viewed as Asset for Increasing Achievment in Math
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Strongly Disagree Disagree Agree Strongly Agree
MTL
LC
LT
LT_MT
Learning Team Functioning• How much of an asset is the MTL in
helping schools focus improvement in math?
MTL Viewed as Asset for Increasing Achievment in Math
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Strongly Disagree Disagree Agree Strongly Agree
MTL
LC
LT
LT_MT
Learning Team Functioning
MTL and LC Collaborated
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Strongly Disagree Disagree Agree Strongly Agree
MTL
LC
LT
LT_MT
Learning Team Functioning• How collaborative is the LT?
– In general, the majority of all LT members reported that they usually felt comfortable expressing their opinions and the atmosphere of LT meetings was typically respectful, open, and friendly.
– To a lesser extent LT members reported a shared responsibility in decision making regarding the implementation of math programs and development in mathematics.
Classroom Practice
• What pedagogical approaches are being utilized in mathematics classrooms in MPS?– The majority of math teachers in the
district reported utilizing all of the pedagogical approaches described which were designed to on at least a weekly basis
– These approaches were designed to contrast didactic versus interactive approaches to mathematics instruction
Classroom Practice
• Compared to other math teachers, a significantly smaller percentage of MTLs reported regularly spending time – Having students solve problems using a
method taught by the teacher – Having students complete textbook
exercises for review
Classroom Practice
• A significantly greater percentage of MTLs reported regularly spending time having students– Analyze the similarities among
representations, solutions, or methods– Justify why a particular solution
strategy in mathematics worked– Work on math problems that had
multiple answers or solution methods
Classroom Practice• A significantly greater percentage of
MTLs and LT members that teach math reported regularly spending time having students– discuss mathematical ideas,
problems, or solutions in small groups– write explanations of math ideas,
solutions, or methods– practice computational skills
Classroom Practice• What content is being taught in
mathematics classrooms in MPS?
MTL(n = 113)
LT_MT(n = 170)
MT(n = 692)
Number concepts and computation
Less than 10 lessons 9.9% 7.2% 12.8%
More than 10 lessons 90.1% 92.8% 87.2%
Geometry Less than 10 lessons 27.5% 40.0% 47.7%
More than 10 lessons 72.5% 60.0% 52.3%
Measurement Less than 10 lessons 37.6% 55.1% 63.0%
More than 10 lessons 62.4% 44.9% 37.0%
Data analysis/statistics Less than 10 lessons 33.0% 41.4% 49.2%
More than 10 lessons 67.0% 58.6% 50.8%
Probability Less than 10 lessons 66.4% 72.6% 79.0%
More than 10 lessons 33.6% 27.4% 21.0%
Mathematical patterns/algebra
Less than 10 lessons 22.5% 36.5% 38.6%
More than 10 lessons 77.5% 63.5% 61.4%
MTL Functioning• How often do MTLs interact with
others to discuss teaching and learning in math?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
LT Other MathTeachers
LC Principal
None of the time One-fourth of the timeHalf of the time Three-fourths of the timeAll of the time
MTL Functioning• How supported do MTLs feel in
their role?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
LT Other Math Teachers LC Principal Math TeachingSpecialist
Strongly DisagreeDisagree Agree Strongly Agree
MTL Functioning• How closely are MTLs working with
Math Teaching Specialists?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Attends LT meetings Facilitates PD at School Dependable Resource
SD D A SA
LC Functioning• How involved is the LC in helping school
improve in mathematics?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Collaborated with MTLWorked with MathTeachers
Modeled MathTeaching
Modeled LiteracyStrategies During
Math
Covered MTL's Class
Never Once this year A few times during the yearOnce per month A few times per month 1-2 days per week or more
Discussion
• It is apparent that quite an impact has been made by the MMP in the majority of elementary and middle schools in MPS this year
• It is also apparent that there is quite a lot of school level variability in the level of impact that has been made in the district.
Discussion
• In some schools within the district, improving student achievement in mathematics appears to be a top priority
• In other schools in the district improving student achievement in mathematics does not appear to be a priority
Discussion
• This level of variability is to be expected with any educational improvement plan that is implemented on such a large scale
• Only by capturing such variability can the impact of the MMP on increasing student achievement in mathematics be evaluated
Discussion• Is there an increase in student
achievement in mathematics at schools in the district that are
1. Utilizing all the resources provided to them though the MMP?
2. Doing everything within their control to collaborate, plan and implement procedures designed to improve the teaching and learning of mathematics?