new mexico’s three-tiered licensure system and its implications for assessing teacher preparation...

New Mexico’s Three-Tiered Licensure System And Its Implications for Assessing Teacher

Preparation Effectiveness

Rosalita Mitchell, University of New MexicoPeter Winograd, New Mexico Office of Education Accountability

Cindy Sims, Moriarty Municipal SchoolsJames Ball, New Mexico Public Education Department

Madeline Feijoo, New Mexico Office of Education AccountabilitySmith Frederick, University of New Mexico

Wanda Trujillo, Office of Education AccountabilityScott Hughes, Office of Education Accountability

Beata Thorstensen, Office of Education Accountability

Presented at the Conference on Quality Teacher Preparation: Assessment of Teacher Preparation Effectiveness.

Alverno College, Milwaukee, Wisconsin, September 30 - October 1, 2005

Sponsored by the National Commission on Teaching and America’s Future and the Carnegie Corporation of New York

2

Presentation Overview

• The Debates About Teacher Preparation

• The Key Question: Are Teacher Preparation Programs Effective?

• New Mexico’s Three-Tiered Licensure System

• A Statewide Study of Teacher Preparation Program Graduates’ Performance in the Classroom

• What Can We Learn From This Study About Assessing The Effectiveness of Teacher Preparation Programs?

• Summary

3

The Debate About Teacher Preparation

• The Critics - Those who can’t do, teach; Those who can’t teach, teach teachers: – Too many weak students enter the teaching profession– Too many graduates are poorly equipped to teach– Programs focus too much on "soft" pedagogical

knowledge at the expense of subject-matter depth– Programs fail to prepare graduates to teach to student

performance standards– Programs do not provide adequate, real-world,

practical experience

4

The Debate About Teacher Preparation

• The Response – Our programs are good and getting better:– NCTAF and other national initiatives– Federal Efforts (NCLB, Title II)– Teacher Preparation Improvements (school partnerships, more

university support, P-20 systems, induction programs, alternative licensure programs)

– Rigorous Accreditation Standards• NCATE - Standard 2: Assessment System and Unit

Evaluation. NCATE (NCATE 2002, p. 10) defines Standard 2 as “The unit has an assessment system that collects and analyzes data on the applicant qualifications, candidate and graduate performance, and unit operations to evaluate and improve the unit and its programs

5

The Key Question:Do preparation programs prepare teachers to

teach all students how to succeed academically and in other important ways?

The Key Answer:Assess program graduates who are working

with real children in actual classrooms

6

New Mexico’s Three-Tiered Licensure System

“The Legislature further finds that the teacher shortage in this country has affected the ability of New Mexico to compete for the best teachers, and that unless the state and school districts find ways to mentor beginning teachers, intervene with teachers while they still show promise, improve the job satisfaction of quality teachers and elevate the teaching profession by shifting to a professional educator licensing and salary system, public schools will be unable to recruit and retain the highest quality teachers in the teaching profession in New Mexico.” HB 212 – The Public School Reforms Act, 2003

“If our future generations are to prosper, then they must have the skills to compete with the best – and brightest – in the world. In education, the classroom is where the rubber meets the road. Our teachers are the focus of everything that happens in education…” Governor Bill Richardson, State of the State Address, January 21, 2003

7

An Overview of Key Elements of NM’s 3-Tiered Licensure System

Level IProvisional Teacher

Level IIIMaster Teacher

Level II Professional Teacher

√ Successful Annual Evaluations at Level I Indicators;√ Mentoring;√ Verification Teacher is Highly Qualified if assigned to a core content area;

√ Successful Annual Evaluations at Level II Indicators;√ Verification Teacher is Highly Qualified if assigned to a core content area;√ Licensure Renewal;

√ Successful Annual Evaluations at Level III Indicators;√ Verification Teacher is Highly Qualified if assigned to a core content area;√ Licensure Renewal;

•Advance to Level II bysubmitting ProfessionalDevelopment Dossier (PDD);

•Minimum 3 Years Level I Teaching Experience

$30K Minimum $40K Minimum

•Advance to Level III by submitting Professional Development Dossier (PDD);•Minimum 3 Years Level II Teaching Experience; •MA or NBPTS

$45K in 06-07; $50K in 07-08

8

What is The Professional Development Dossier (PDD)?

• A performance-based assessment centered on nine NM Teacher Competencies

• A focused, compact collection of documentation compiled by a NM teacher for licensure advancement

• Structured documentation of student achievement and student learning

9

Student Teacher’s content test scores knowledge

Defining Teacher Quality…Finding a Balance

Teacher performance and student learning

NM TeacherEvaluation

System

10

New Mexico’s Nine Teacher Competencies

1. The teacher accurately demonstrates knowledge of the content area and approved curriculum.

2. The teacher appropriately utilizes a variety of teaching methods and resources for each area taught.

3. The teacher communicates with and obtains feedback from students in a manner that enhances student learning and understanding.

4. The teacher comprehends the principles of student growth, development and learning and applies them appropriately.

5. The teacher effectively utilizes student assessment techniques and procedures.

6. The teacher manages the educational setting in a manner that promotes positive student behavior and a safe and healthy environment.

7. The teacher recognizes student diversity and creates an atmosphere conducive to the promotion of positive student involvement and self-concept.

8. The teacher demonstrates a willingness to examine and implement change, as appropriate.

9. The teacher works productively with colleagues, parents and community members.

11

The Three-Tiered Licensure System Organized New Mexico’s Nine Teacher Competencies Into Three Strands

• Strand A – Instruction1. The teacher accurately demonstrates knowledge of the content area and approved curriculum.



• Strand B – Student Learning3. The teacher communicates with and obtains feedback from students in a manner that enhances student

learning and understanding.



7. The teacher recognizes student diversity and creates an atmosphere conducive to the promotion of positive student involvement and self-concept.

• Strand C – Professional Growth8. The teacher demonstrates a willingness to examine and implement change, as appropriate.


12


Provisional Teacher - LEVEL I Indicators

Professional Teacher – LEVEL II Indicators

Master Teacher – LEVEL III Indicators

E. Actively solicits communication from students about their learning.

E. Solicits communication from students about their learning for the purposes of ongoing instructional planning.

E. Engages students in the analysis and evaluation of their learning and adjusts instruction based on student feedback.

The Three-Tiered Licensure System Provided More Depth To Each Of The Nine Competencies

For example:

13

Review of PDD Design

Strand A -- Instruction (Independent Review)

Strand B -- Student Learning (Independent Review)

Strand C -- Professional Growth (Independent Review)

Strand D -- Verification of teaching experience, mentoring, and authenticity

Strand E -- Superintendent’s recommendation for advancement based on annual evaluations

14

A Statewide Study of Teacher Preparation Program Graduates’ Performance in the Classroom: Overview

• 997 Level 1 and 2 teachers submitted a Professional Development Dossier (PDD) as part of New Mexico’s Three-Tiered Licensure System during the January-February, 2005 Submission Period

• Teachers could receive three kinds of scores:

– Did not pass strands A, B, C – (146 teachers or 14.6%)

– Did not receive school administrator approval (16 teachers or 1.6%)

– Pass (835 teachers or 83.8%)

17

Percent of All Level 1 and 2 Teachers Passing By Gender(N=997)

86.7%

74.4%

50%

60%

70%

80%

90%

100%

Female (795) Male (199)

Note: (Number of Teachers In Each Gender Category)

18

Percent of All Level 1 and 2 Teachers Passing By Ethnicity (N=651)

60.0%

66.7%

71.4%75.0% 75.0%

83.5%

88.2%

92.6%

50%

60%

70%

80%

90%

100%

Other (10) Multiracial (6) Asian/Pacific (7)

African American/Black (8) Native American/Indian (24) Hispanic/Latino (212)

White/Anglo (357) Choose Not To Identify (27)

Note: (Number of Teachers In Each Ethnic Category)

19

Percent of All Level 1 and 2 Teachers Passing By Type of License (N=843)

65.5% 66.7%

83.9% 84.2%87.5%

92.5%

50%

60%

70%

80%

90%

100%

Middle School 5-9 (29) Secondary 7-12 (180) Elementary K-8 (440)

Special Education K-12 (114) Early Childhood (40) Specialty Area K-12 (40)

Note: (Number of Teachers In Each Type of License)

20

Evaluating New Mexico Preparation Program Graduates’ Performance in the Classroom

• 701 of the 997 teachers were Level 1 teachers attempting to advance to Level 2 AND had been prepared in New Mexico teacher preparation programs

• These 701 Level 1 teachers prepared in NM programs could receive three kinds of scores:

– Did not pass strands A, B, C - (93 teachers or 13.3%)

– Did not receive school administrator approval (14 teachers or 2.0%)

– Pass (592 teachers or 84.7%)

• Detailed information about which competencies teachers missed were available on 42 of the 93 teachers who did not pass strands A.B, or C

21

Preliminary Analyses For Level 1 Teachers Prepared in New Mexico Programs

• How did the passage rate for Level 1 teachers vary by New Mexico teacher preparation program?

• How did the passage rate for Level 1 teachers vary by ethnicity?

• How did the passage rate for Level 1 teachers vary by district?

• How did the passage rate for Level 1 teachers vary by program type?

• Which of the nine teacher competencies proved most difficult for those Level 1 teachers who did not pass?

• Which of the nine teacher competencies proved most difficult for Level 1 elementary, secondary and special education teachers who did not pass?

22

Percent of Teachers Passing From Level 1 To Level 2 By Preparation Program

(In State N=697; Out of State N = 226))

57.1%

71.7%

79.3%81.1%

83.2% 83.3%86.6%

92.3%

100.0%

86.3%

50%

60%

70%

80%

90%

100%

Northern New Mexico Community College (7) New Mexico Highlands University (60)College of the Southwest (29) Western New Mexico University (37)Eastern New Mexico University (95) College of Santa Fe (54)Unversity of New Mexico (305) New Mexico State University (104)Santa Fe Community College (6) Out of State (226)

Note: (Number of Teachers for Each Preparation Program)

23

Percent of Teachers Passing From Level 1 To Level 2 By Ethnicity (N=461)

55.6%

66.7%

71.4%

75.0%

80.0%

85.6% 86.5%

94.1%

50%

60%

70%

80%

90%

100%

Other (9) Asian/Pacific (3) Native American/Indian (14)

African American/Black (4) Multiracial (5) Hispanic/Latino (180)

White/Anglo (229) Choose not to identify (17)

Note: (Number of Teachers In Each Ethnic Category)

24

Percent of Teachers Passing From Level 1 To Level 2 By Type of Preparation Program (Alternative Licensure or Traditional Licensure)

(N=698)

81.8%

84.8%

70%

75%

80%

85%

90%

95%

100%

Alternative (22) Traditional (676)

Note: (Number of Teachers In Each Type of Program)

25

Percent of Level 1 Teachers Passing By School District

(N=65 of New Mexico's 89 Districts)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

26

Which competencies proved most difficult for those Level 1 teachers who did not pass?

Competency % Missed

1. The teacher accurately demonstrates knowledge of the content area and approved curriculum. 8.3%

2. The teacher appropriately utilizes a variety of teaching methods and resources for each area taught. 9.7%

3. The teacher communicates with and obtains feedback from students in a manner that enhances student learning and understanding. 11.7%

4. The teacher comprehends the principles of student growth, development and learning and applies them appropriately. 19.3%

5. The teacher effectively utilizes student assessment techniques and procedures. 13.1%

6. The teacher manages the educational setting in a manner that promotes positive student behavior and a safe and healthy environment. 8.3%

7. The teacher recognizes student diversity and creates an atmosphere conducive to the promotion of positive student involvement and self concept.

10.3%

8. The teacher demonstrates a willingness to examine and implement change, as appropriate. 11.7%

9. The teacher works productively with colleagues, parents and community members. 7.6%

27

Which Competencies Proved Most Difficult For Elementary, Secondary And Special Education Level 1 Teachers Who Did Not Pass?

CompetencyElementary

(n=21)

Secondary

(n=9)

Special Ed

(n=5)

1. The teacher accurately demonstrates knowledge of the content area and approved curriculum.

8.3% 8.3% 8.3%


5.8% 16.1% 12.0%


14.5% 9.7% 8.%


24.6% 16.1% 16.0%


10.1% 16.1% 16.0%


10.1% 3.2% 8.0%

7. The teacher recognizes student diversity and creates an atmosphere conducive to the promotion of positive student involvement and self concept.

13.0% 9.7% 4.0%

8. The teacher demonstrates a willingness to examine and implement change, as appropriate.

7.2% 16.1% 12.0%


8.7% 3.2% 12.0%

28

What Can We Learn From This Study About Assessing The

Effectiveness of Teacher Preparation Programs?

29

How Might Preparation Programs And Districts Use This Data?

• Consider the pass rates and competency analyses as a valid sources of information.

• Avoid the blame game, the “good enough” syndrome, or it’s “not our responsibility” reflex.

• Think systematically about the skills, knowledge, and dispositions that students bring to their college experiences, including their time in teacher preparation programs.

• Think systemically about the 1-2 years of teacher preparation and the 1-3 years of district mentoring and professional development.

• Examine how the competencies used for preparation programs align with the 9 competencies used to evaluate teachers.

• Examine how the district’s mentoring and professional development programs align with the 9 competencies used to evaluate teachers.

• Involve college and school faculty in strengthening the preparation and support of beginning teachers.

30

For Example: Examine the Alignment Between the Competencies Used In Teacher Preparation Programs And Teacher Evaluation

Teacher Preparation Program Competencies 9 Teacher Evaluation Competencies

A. Professionalism 1. Knowledge of Content and Curriculum

B. Instructional Planning and Implementation 2. Teaching Methods and Resources

C. Classroom Management 3. Student Feedback and Communication

D. Assessment 4. Knowledge of Student Growth and Development

E. Technology 5. Assessment Techniques and Procedures.

F. Diversity 6. Classroom Management and Environment

G. Family and Community 7. Student Diversity

H. Inclusion 8. Implementing Appropriate Change

I. Development of Student 9. Collaboration with Colleagues, Parents, and Community

J. Knowledge of Content

K. Communication .

31

For Example: Conduct Detailed Analyses Who Passed and Failed and Why By Specific Programs and Districts

MAIN CAMPUS PROGRAM DISTRICT A DISTRICT B(who) ECE pass/fail (who & why) pass/fail (who & why)(who) ELEM pass/fail (who & why) pass/fail (who & why)(who) SEC pass/fail (who & why) pass/fail (who & why)(who) SPED pass/fail (who & why) pass/fail (who & why)

BRANCH CAMPUS(who) ECE pass/fail (who & why) pass/fail (who & why)(who) ELEM pass/fail (who & why) pass/fail (who & why)(who) SEC pass/fail (who & why) pass/fail (who & why)(who) SPED pass/fail (who & why) pass/fail (who & why)

PUBLIC SCHOOLEXAMPLE UNIVERSITY

32

For Example: Examine The Issues in The District’s Professional Development Program

• Teachers need opportunities to reflect on the effectiveness of instructional practice in advancing student learning.

• Teachers need opportunities to identify relevant student learning data, understand what the data is showing for individual students as well as the whole class in terms of gaps in the instructional program and strategies needed to address those gaps.

• Competency 4 requires evidence of instructional differentiation to address individual students’ needs. Teachers need professional development to address student learning levels, rates and styles using appropriate strategies and resources.

• Currently, school districts provide significant support for level 1 teachers to successfully complete dossier to advance to a level 2 status. Professional development opportunities should provide the same level of support to level 2 teachers in their pursuit of a level 3 status.

33

What are the implications of this study for assessing

teacher preparation effectiveness?

• We can develop and implement systems for assessing teacher quality based on a balance of the teacher’s background and knowledge AND student growth.

• Assessments of teacher quality can identify the areas that teachers find particularly challenging.

• It is important to remember that beginning teachers spent 1-2 years in their preparation program and 3 years in their school district before submitting their materials to the Professional Development Dossier.

• We must think carefully about how to assess the performance of program graduates over time – their strengths and areas for improvement may change from year to year.

• We must think carefully about what can be attributed to a preparation program and what should be attributed to district’s training and support.

34

Summary

• Teacher preparation programs should prove their effectiveness by gathering data about the performance of their graduates.

• Gathering almost any data is a start because data will show what needs to be done.

• We must think carefully about what can be attributed to a preparation program and what should be attributed to district’s training and support.

• We must work collaboratively: – For the State: Increased understanding of areas for teaching

improvement.– For University Faculty: Increased responsiveness to the needs of

program graduates in the field.– For School Districts: Improved professional development focused

on student learning.

new mexico’s three-tiered licensure system and its implications for assessing teacher preparation...

Documents