beyond the standard course · 2018-11-15 · beyond the standard course evaluation: effective peer...

BEYOND THE STANDARD COURSE EVALUATION:

EFFECTIVE PEER EVALUATION MODELS

9:00 – 9:55am

Please feel free to make comments/ask questions via

pollev.com/carolina anytime during this session.

Moderator: Kelly HoganAssociate Dean of Instructional Innovation, College A&S

WHAT CONSTITUTES EFFECTIVE OR EXCELLENT TEACHING?

WHO WEIGHS IN ON THE EVIDENCE?

credit: Viji Sathy

Students

PeersSelf-Reflection

Students

• Not linked to student learning (i.e. no correlation between learning more and highly rated professors)

• Biased (age, race, gender)

Limited but can provide students a voice

Solution? “Reduce the weight of student ratings in decision making”

B. Utl et al., Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation, 2017

Self-Reflection

A teaching Statements allows a teacher to reflect on: • what has worked/not

worked• growth over time• their goals for course design

and class time• evidence they have that

students are learning• student and peer

evaluations of their teaching

Peers

Peer Review of materials and classroom• Can evaluate alignment

between curricular goals and teaching practices

• Can evaluate climate/inclusiveness

• Can evaluate content expertise and teaching practices

Formative peer review:occurs frequently with consistency “for the sole purpose of fostering professional growth and improved practice.”

Summative peer revew:are for the purpose of reappointment, promotion, or tenure. “Summative evaluations must be based on a clear set of performance standards that are identical to standards used in the ongoing formative process. They must employ a rubric of criterion-referenced assessments.”

Definitions adapted from: nea.org

VS

PEER REVIEW AT SILS

Ron BergquistAssociate Dean for Academic Affairs

The Mission of SILS is to educate the very best information professionals

WHY WE DO PEER REVIEW

The purpose of the peer observation program is to contribute to the continual improvement of the SILS educational environment

by helping those who teach develop their teaching abilities

and by providing observing faculty members with a broader understanding of the SILS curriculum.

SELECTION OF REVIEWERS

For tenure-track faculty, teams of two faculty members will form visitation groups for the evaluation procedure.

At least one member of each team will be knowledgeable in the area being taught and at least one member of each team (preferably both) will be an associate or full professor.

THE PROCESS

Pre-Observation ConferenceProvide Context

ObservationNarrative Notes

Post-Observation Conferencea balanced picture of the instructor's teaching,

specifying areas of particular effectiveness as well as areas that could be improved

Afterwards

PeriodicityTenured – every third year

Non-tenured – three consecutive semestersTerm – every third semesterAdjuncts – every semester

THE CHALLENGE

The Future?

Maybe some changes needed

Peer Review in Exercise and Sport Science

Alain J. AguilarTeaching Associate Professor

Peer Review in Asian StudiesLi-ling Hsiao

Associate Professor, Chinese Language and LiteratureAssociate Dean and Director of First-year Curricula

When?

•When a faculty teaches a new course in the department

•When the first class observation has not gone well

•When the faculty is reviewed for reappointment or promotion

How?• The faculty observed must provide the course

syllabus, lesson plan of the class, and the reading assignments to the peer observer in advance.

• After the class is observed, the observer will meet the observed faculty and brainstorm on how to improve the course.

• The observer will then submit a report to the Chair to be part of the observed faculty’s dossier.

What?A. PREPARATION

1. The instructor had a clear lesson plan. 2. The instructor planned activities to target specific teaching goals.3. The planned activities are appropriate for the students’ level. 4. The plan was geared toward real/authentic language use.5. The instructor’s plan includes specific exercises for students to perform and allots time for each segment of the class.

B. CLASS PRESENTATION AND MANAGEMENT

1. Class started on time.

2. The lesson was presented effectively.

3. The lesson was presented clearly.

4. There were smooth transitions between activities.

5. The time allotted for activities was appropriate.

6. Major points of teaching and activities were sequenced in a logical way.

7. The amount of teacher talk and student talk was appropriate.

8. The type and amount of teacher feedback was adequate.

9. Cultural instruction was integrated into class activities.

10. The activities/exercises chosen to achieve the objectives were effective.

11. The instructor divided his or her attention among students appropriately.

12. Student participation was focused on the task at hand.

13. The instructor used the target-language appropriately.

14. Audio-visual and other supplementary materials were used appropriately.

15. The use of technology (power point, web materials) was appropriate given the material being presented. 16. The instructor displays enthusiasm for teaching the target language.

C. CLASSROOM ATMOSPHERE1. Student participation was active and lively.

2. The class atmosphere was positive, open and accepting. 3. The instructor was sensitive to students' difficulties and abilities. 4. The instructor treated students with respect while not allowing any disruptive behavior.

Record comments on the observation and any suggestions for the instructor here:

Why?

• To assist new faculty develop the teaching skills at the very beginning stage

• To plan strategies for improvements for new faculty• To ease the loads of multiple course observations

required during the semester of reappointment or promotional review

• To catch any problem early on instead of unexpected surprises in the reappointment or promotional review.

BEYOND THE STANDARD COURSE EVALUATION: EFFECTIVE PEER EVALUATION MODELS

Angela Broome, DDS, MSAssociate ProfessorDepartment of Diagnostic Sciences, Division of Oral and Maxillofacial RadiologyUNC School of Dentistry

9:00 – 9:55am

UNC School of DentistryFaculty Development Committee

PEER EVALUATION OF CLASSROOM TEACHING

Wilder RS, Guthmiller JM. Empowerment through mentorship and leadership. J Evidence Based Dent Pract 2014;14S:222-226.

Mentoring Program

PEER EVALUATION OF CLASSROOM TEACHING

Newman LR, Roberts DH, Schwartzstein RM. Peer Observation of Teaching Handbook. Shapiro Institute for Education and Research at Harvard Medical School and Beth Israel Deaconess Medical Center. MedEdPortal ID# 9150; 2012. Available from: www.mededportal.org/

Faculty Peer Evaluation

RESOURCES•UNC Division of Speech and Hearing Sciences

•Peer Review of Classroom Teaching•University of Texas Dental Branch at Houston

•Peer Evaluation Form for Lectures•Shapiro Institute for Education and Research at Harvard Medical School and Beth Israel Deaconess Medical Center

•Peer Observation of Teaching Handbook•Center for Teaching Vanderbilt University

•Peer Review of Teaching•Center for Faculty Excellence University of North Carolina

•Faculty Peer Visits Program

GOAL

Provide a positive, learning experience through meaningful feedback

Identify strengths and weaknesses as means of formative assessment

Develop strategies excellent teaching skills

Discover innovative ways of teaching

CURRENT PROCESS•Once per year by a peer member (three selected by faculty member, one chosen by mentoring chair)

•Initial meeting of faculty member and peer reviewer for discussing the objectives of the presentation, teaching strategies, and the evaluation process.

•During presentation, peer reviewer attends presentation and evaluates

•Within 2 weeks following, faculty member and peer reviewer meet and discuss evaluation

CHALLENGES

Meaningful Feedback Adherence to ProcessReview Calibration

OPPORTUNITIES

Existing Form

OPPORTUNITIES

Updated Form

OPPORTUNITIES

RubricCategory Exceeds Expectations Meets Expectations Does not meet expectationsOrganization:

1

communicates objectives including how material relates to past/future topics

Clearly communicates objectives; relates objectives to course context; relates objectives to expected leraning outcomes; reinforces objectives throughout the session

States the objectives in outline, presentation, slide, and/or verbally

Does not provide objectives in any format or at any time during the session

2

provides clinical relevance to the topic

Explains relevance to other course content, future course, clinic, private practice application; answers the question, "how is this information significant to the student?" and does so throughout the session

provides topic relevance during the session does not desribe relevance to course content or future applications

3

presents material in an organized manner

presents material in a logical, organized framework and refers back to the framework and learning objectives through verbal or presentation prompts throughout the session

presents material in an organized framework but may not provide periodic queues or links to the objectives

does not provide material in a logical or recognizable framework; content lacks any recognizable flow

4

presents material within the context of the coursedemonstrates an undertanding of the course content and how topic relates to previous and future topics references how material may relate to course content does not link topic with previous or future course topics

5

designs session with effective use of instructional media

uses any of the following as appropriately for intended, meaningful learning: flipped classroom; think, pair, share; audience polls, developed intereactive peer project/break out sessions with facilitator monitoring; effectively implemented

attempts use of a method but not effectively implemented; limited effect or limited facilitator monitoring/interaction during session

lacks any clear use of instructional media; misses opportunities to use media when it would be effective for the session

DISCUSSION “RULES”.

Raise your hand. (Please limit your comments to 60 seconds or less, so that many people have an opportunity to join the conversation.)Write on a notecard and signal to the moderator for pick up.Use pollev.com/carolina for anonymous questions/comments.

We invite you to join into the conversation in multiple ways:

QUESTIONSHow is peer review perceived in your department?

Peer review involves a collection of methods for assessing faculty performance. What processes have others used for peer evaluation?

Should peer review processes be considered formative (improvement) or summative (related to promotion and tenure)?

Are the peer reviewers trained? Who are the peer reviewers?

How is self-assessment/reflection incorporated into the peer review process?