teacher assessment magno_2
TRANSCRIPT
-
7/31/2019 Teacher Assessment Magno_2
1/24
The Development of the Peer
Assistance and Review Form (PARF)Carlo Magno
De La Salle University, Manila
-
7/31/2019 Teacher Assessment Magno_2
2/24
Assessing Teacher Performance
Students assess teachers
*A peer (fellow teacher/supervisor) assessing theteacher
External personnel assessing the teacher
-
7/31/2019 Teacher Assessment Magno_2
3/24
Peer Review of Teaching
When a teacher is observed by another teacherfor a specific purpose, it is called in literature aspeer assistance and review (Goldstein, 2004)
peer review (Kerchner & Koppich, 1993),
teacher peer coaching (Bruce & Ross, 2008)
-
7/31/2019 Teacher Assessment Magno_2
4/24
What peer review of teaching? involving teachers in the summative [also
formative] evaluation of other teachers (Goldstein,
2004, p. 397) evaluating ones peers which allow the assessmentof ones teaching by another person who has similarexperience and goals
a structured approach for building a community in
which pairs of teachers of similar experience andcompetence observe each other teach, establishimprovement goals, develop strategies to implementgoals, observe one another during the revisedteaching, and provide specific feedback (Bruce &
Ross, 2008)
-
7/31/2019 Teacher Assessment Magno_2
5/24
Purpose
Hiring of teachers
Clinical supervision
Modeling for new teachers
Promotion
Rehiring
Common uses of Teacher Performance Results
-
7/31/2019 Teacher Assessment Magno_2
6/24
Developmental Approach
Teachers performance from peer reviews shouldbe conceptualized with the aim of helpingteachers attain success in their teaching ratherthan point out mistakes of teachers (Oakland &Hambleton, 2006; Stiggins, 2008).
It is described as a constructive process wherethe peer aims to provide assistance to a lessexperienced teacher in improving theirinstruction and handling of students.
-
7/31/2019 Teacher Assessment Magno_2
7/24
-
7/31/2019 Teacher Assessment Magno_2
8/24
Conceptual Framework
Learner-centered principles
perspectives that allow the teachers ability tofacilitate the learners in their learning, thelearning in the programs, and other processes thatinvolves the learner
Danielsons Components of Professional Practice
identified aspects of the teachers responsibilitiesthat have been documented through empiricalstudies and theoretical research promotingimproved student learning.
-
7/31/2019 Teacher Assessment Magno_2
9/24
Conceptual Framework
Danielsons Components of Professional Practice
Planning and preparation
Classroom environment
Instruction
Professional responsibility
The LC and components of professional practicefits well together in a model (Magno &Sembrano, 2010).
-
7/31/2019 Teacher Assessment Magno_2
10/24
-
7/31/2019 Teacher Assessment Magno_2
11/24
Method Rubric Construction (analytical rubric)
Domain 1: Planning and Preparation1a. Demonstrating knowledge of content and
pedagogy1b. Demonstrating knowledge of students1c. Selecting instructional goals
1d. Demonstrating knowledge of resources1e. Designing coherent instruction1f. Assessing student learning
Domain 2: The Classroom Environment2a. Creating an environment of respect and
rapport2b. Establishing a culture for learning2c. Managing classroom procedures
2d. Managing student behavior2e. Organizing physical space
Domain 3: Instruction3a. Communicating clearly and accurately3b. Using questioning and discussion
techniques3c. Engaging students in learning3d. Providing feedback to students3e. Demonstrating flexibility and
responsiveness
Domain 4: Professional Responsibilities4a. Reflecting on teaching4b. Maintaining accurate records4c. Communicating with families4d. Contributing to the school and district4e. Growing and developing professionally4f. Showing professionalism
-
7/31/2019 Teacher Assessment Magno_2
12/24
-
7/31/2019 Teacher Assessment Magno_2
13/24
Item Review
FGD
Review checklist
External reviewers
-
7/31/2019 Teacher Assessment Magno_2
14/24
Pretesting Initial pretesting
Developed a manual how to use the PRPF and
how to conduct proper observation Orientation for raters and ratees 2 raters are assigned for a ratee (primary and
secondary raters)
183 ratees were completed Final pretesting
175 ratees were completed Same procedure
-
7/31/2019 Teacher Assessment Magno_2
15/24
PARF results (Reliability)Initial pretesting (N=183, 89
items)
Internal consistency for raters:.98 and .97
Planning and Preparation:.94 & .93
Classroom Environment: .93& .92
Instruction: .94 & .92
Professional responsibility:.93 & .91
Final pretesting (N=175, 59items)
Internal consistency for raters:.97 and .96
Planning and Preparation:.94 & .93
Classroom Environment: .88& .88
Instruction: .93 & .90
Professional responsibility:.92 & .76
-
7/31/2019 Teacher Assessment Magno_2
16/24
PARF results (Rater agreement)
Initial pretesting (N=183, 89items)
=.47,p
-
7/31/2019 Teacher Assessment Magno_2
17/24
-
7/31/2019 Teacher Assessment Magno_2
18/24
PARF results (GRM) Initial pretesting (N=183,
89 items) Final pretesting (N=175,
59 items)
Figure 3. Test Information Function of PRPF for
the Primary Raters
Figure 4. Test Information Function of the PRPFof the Secondary Rater
Figure 1. Test Information Function of PRPF for
the Primary Raters
Figure 2. Test Information Function of the PRPFof the Secondary Rater
-
7/31/2019 Teacher Assessment Magno_2
19/24
Item Maps
Figure 6. Item Map of the PRPF for the Primary Rater Figure 7. Item Map of the PRPF for the Secondary Rater
Item Maps
-
7/31/2019 Teacher Assessment Magno_2
20/24
PARF results (Factorial Validity)
Initial pretesting (N=183, 89 items)
A four factors structure (3 parcels each factor)
2=8829.23, df=3734,
PGI=.97,
Bentler-Bonnett Normed Fit Index=.96,
Bentler-Bonnett Non-Normed Fit Index=.96.
-
7/31/2019 Teacher Assessment Magno_2
21/24
PARF results (Construct Validity)
2=169.56,df=98,
standardizedRMS=.02,
PGI=.95,
GFI=.90,
Bentler-BonettNFI=.97,
BentlerCFI=.99,
Bollens
Rho=.96.
-
7/31/2019 Teacher Assessment Magno_2
22/24
Discussion
Consistent results were obtained in thepreliminary and final pilot testing in terms of the
internal consistencies, concordance of raters,convergence of factors, and factor structure.
Theoretical construction of Danielsons
Components of Professional Practice andLearner-centeredness.
Very few items turn out to be bad fit.
Precision of measurement
-
7/31/2019 Teacher Assessment Magno_2
23/24
Problems:
Majority of scores given were very high
Low discriminating power of the items
Recommendations:
High cut off scores
Include a wide array of faculty in the assessment
Strict and objective raters
-
7/31/2019 Teacher Assessment Magno_2
24/24
Contituous Project
Versions following the same framework:
Students rating for teachers (rating scale)
Screening tool for teacher applicants (checklist)
Community Service classes (Ratings scale)
Culinary Laboratory classes (rubric)
Seminar classes (rating scale) Faculty Advisers (rating scale)