TEACHINGPORTFOLIOS- a systematic approach to enchancing the status of teaching

Katarina Winka, Umeå University, Sweden

2017-08-17

UMEÅ UNIVERSITY

PhD in Systematic Biology 2000, Umeå University, Sweden

Educational development unit since 2003 (part time until 2008)

Director 2012 – 2016

Research interests – interactive teachingmethods, PhD supervision, documenting, assessing and rewarding teachingexcellence in higher education

MY BACKGROUND

UMEÅ UNIVERSITY

…by documenting, assessing and rewarding teaching excellence in higher education

CHANGING THE HIGHER EDUCATIONMERITATION SYSTEM

Teachingportfolios

Improvingassessments

Pedagogicalcareermodels

UMEÅ UNIVERSITY

- I ALREADY HAVE A CV, ISN’T THAT ENOUGH?

- THE ONE WITH MOST EXPERIENCE IS THE BEST TEACHER

- MY COURSE EVALUATIONS ARE ALWAYS POSITIVE – SO, I’M A SKILLED TEACHER

- ANY ASSESSMENT OF TEACHING COMPETENCE MUST BE VERY SUBJECTIVE

- CAN TEACHING COMPETENCE REALLY BE CAPTURED IN A DOCUMENT?

- THE TEACHING PHILOSOPHY IS JUST FANCY WORDS

UMEÅ UNIVERSITY

Skelton, 2005; Seldin et al., 2010; Apelgren & Giertz, 1998; Winka & Ryegård, 2013.

•Job application•Promotion•Teaching awards•Salary revisions•Local quality assessments•Professional development plan•Overview of own competence

Teachingportfolios

UMEÅ UNIVERSITY

THE AIM IS TO ILLUSTRATE…

WHAT you have done (CV)HOW you did itWHY you did like you did, and the RESULT of your choices

UMEÅ UNIVERSITY

• Has a purpose with the research• Make informed choices of methods, preparation,

analysis, which results to present, and how…• Document and communicate the results and welcome

peer scrutiny

A professionalresearcher

• Has a purpose with the teaching• ake informed choices of methods, preparation, analysis,

which results to present, and how…• Document and communicate the results and welcome

peer scrutiny

A professionalteacher

Trigwell, 2001.

Use the same approach to teaching!

UMEÅ UNIVERSITY

• Has a purpose with the research• Make informed choices of methods, preparation,

analysis, which results to present, and how…• Document and communicate the results and welcome

peer scrutiny

A professionalresearcher

• Has a purpose with the teaching• Make informed choices of methods, preparation,

analysis, which results to present, and how…• Document and communicate the results and welcome

peer scrutiny

A professionalteacher

Trigwell, 2001.

UMEÅ UNIVERSITY

THE TEACHING PHILOSOPHY - JUST FANCY WORDS?

Erikson, et al. 2015.

”What is actually the truth about one’s own teachingand what is fantasy - dreamed up from a mixture ofexpectations expressed in managerial speeches and the buzzwords floating around faculty rooms?”

UMEÅ UNIVERSITY

THREE TYPES OF INFORMATION IS REQUIRED IN A PORTFOLIO

Teachingexcellence

1. Your own descriptionof your teachingphilosophy, experienceand competence.

2. Others evaluations of yourteaching competence(students, peers, head of unit, award committees)

3. Evidence - concreteexamples and artefactsproduced for teaching and learning

Winka & Ryegård, 2013

UMEÅ UNIVERSITY

ALL PARTS ARE NEEDED FOR A FAIR ASSESSMENT

Teachingexcellence

1. Your own descriptionof your teachingphilosophy, experienceand competence.

2. Others evaluations of yourteaching competence(students, peers, head of unit, award committees)

3. Evidence - concreteexamples and artefactsproduced for teaching and learning

Winka & Ryegård, 2013

UMEÅ UNIVERSITY

CAN TEACHING PORTFOLIOS BE TRUSTED?

When compared to teaching observations.

Pedagogical competence includes more than the teachers actions in the classroom.

Pedagogical competence

Teaching skills

Magin, 1998; Olsson & Roxå 2013.

THE VALIDITY OF DIFFERENT RECRUITMENT METHODS

1,0 Perfect prediction of work performance0,54 Work samples0,51 General mental ability tests 0,51 Structured interviews0,49 Peer ratings0,38 Unstructured interviews0,26 References0,18 Years of work experience0,10 Years of education0,0 Random correlation with work performance

Schmidt & Hunter, 1998.

THE VALIDITY OF DIFFERENT RECRUITMENT METHODS

1,0 Perfect prediction of work performance0,54 Work samples (included in a Teaching portfolio)

0,51 General mental ability tests 0,51 Structured interviews (based on competencies)

0,49 Peer ratings (included in a Teaching portfolio)

0,38 Unstructured interviews0,26 References0,18 Years of work experience0,10 Years of education0,0 Random correlation with work performance

THE PORTFOLIO IS AN ASYNCHRONOUS INTERVIEW§ Competency based interviews evaluates the candidates abilities in relation to

specific areas of competencies and behaviour§ as described in criteria and appointment regulations

§ Focuses on establishing if the candidate has the required skills or has shownthe requested competencies through the STAR technique (Situation, Target, Action, Result):

§ Tell me about a situation when... What?§ What were you trying to achieve… Why?§ What did you do / How did you act... How?§ How did it go / What was the result... Results?

§ Focus on work-related situations not too far back in time.

Arthur, 2005. Recruiting, Interviewing, Selecting & Orienting New Employees.

UMEÅ UNIVERSITY

When compiled in a professional manner (relevant information and evidence), teaching portfolios can be an useful instrument in identifying the candidate that best meetthe criteria.

This requires that the criteria are relevant and that the assessment is professional as well!

Teachingportfolios

UMEÅ UNIVERSITY

PEDAGOGICAL CAREER MODELSPedagogical

careermodels

”a model in which teachers can qualify for academicpositions or be rewarded in their career based

primarily on pedagogical competence”

This is a way to promote higher status in teaching and pedagogical development.

The models drive quality enhancement by • recognizing and rewarding those who already work in a professional manner, and • motivating the young academics to invest in developing knowledge in higher education theory and in

delivering well-prepared and well-executed teaching• signalling that the institute systematically strives to improve the quality of education.

UMEÅ UNIVERSITY

A NATIONAL SURVEY (2016-2017)COMMISSIONED BY GOTHENBURG UNIVERSITY

• Types of models

• Criteria

• Use of teaching portfolios

• Assessment of pedagogical competence

• Trends

• Challenges

Report available at: http://www.pil.gu.se/publicerat/rapportserie

UMEÅ UNIVERSITYMostly temporarypositions or awards, not permanent career steps

UMEÅ UNIVERSITY

RESULTSAlmost half (49 %) of all higher education institutions in Sweden has some kind of pedagogical career model, most (70 % of all) useteaching portfolios.

1 1 2 2 2 3 36 6 7

11

15

2022

2426 27

0

5

10

15

20

25

30

UMEÅ UNIVERSITY

Promotion within the academic system*

Excellent teacher

Excellent teacher

Distinguishedteacher

Reward and title not connected to an academic position

TWO TYPES OF MODELS

Assistant Professor

Docent/ Associateprofessor

Lecturer

Adjuctteacher

* high level of pedagogical competence required for progression to more senior academic positions

UMEÅ UNIVERSITY

CRITERIA IN THE MODELSThe successful candidate candidate needs to meet some or all criteria in these domains:

• broad and extensive teaching experience• development and improvement of teaching practice over time• knowledge of higher education theory and discipline specific pedagogical methods• focus on student learning• demonstrate a critical approach to their teaching practice• support a collegial and collaborative educational environment• demonstrate leadership in teaching and learning

UMEÅ UNIVERSITY

LEVELS OF PEDAGOGICAL COMPETENCE

Scholarship of Teaching COMMUNICATE

Teaching Expertise KNOW

Teaching Excellence DO

Kreber, 2002; Trigwell et al. 2000

UMEÅ UNIVERSITY

0 50 100 150 200 250

Karolinska institutet (Ped docentur)Chalmers tekniska högskola

Högskolan VästLuleå tekniska universitet

Blekinge Tekniska HögskolaHögskolan i Gävle

Södertörns högskolaSveriges lantbruksuniversitet

Högskolan DalarnaKarlstads universitet

Göteborgs universitetHögskolan Kristianstad

Örebro universitetHögskolan i Skövde

Karolinska institutet (Ped akademi)Högskolan i HalmstadMälardalens högskola

Uppsala universitetUmeå universitet

Lunds universitet totalt

NUMBER OF PROMOTED TEACHERS

550+

UMEÅ UNIVERSITY

WHAT DO THESE TEACHERS DO?Department level

o lead dialogues with colleagues and cooperates with students in teachingdevelopment

o planning of education and conducting pedagogical developmento inspire other teachers to compile teaching portfolios

Faculty levelo engaged as expert eller advisor in educational matters, for example as

member in recruitment or assessment committees

University levelo invited to talk or contribute in other ways in higher education conferenceso contribute to higher education training for university teachers

”For the moment nothing at all, but there are ambitions and plans…”

UMEÅ UNIVERSITY

WHAT DO THESE TEACHERS DO?Department level

o lead dialogues with colleagues and cooperates with students in teachingdevelopment

o planning of education and conducting pedagogical developmento inspire other teachers to compile teaching portfolios

Faculty levelo engaged as expert eller advisor in educational matters, for example as

member in recruitment or assessment committees

University levelo invited to talk or contribute in other ways in higher education conferenceso contribute to higher education training for university teachers

”For the moment nothing at all, but there are ambitions and plans…”

UMEÅ UNIVERSITY

ASSESSING TEACHING COMPETENCE

In the pedagogical career models, it is most common to useonly external experts for the assessment of teachingcompetence, or an internal board with invited external experts.

Preferred qualifications of the external experts• Scientific expertise in the same area as the candidate

• Expertise in higher education teaching and learning

• Experience of, or training in, assessment of teaching competence

• Own experience from a pedagogical career model

Improvingthe

assessments

UMEÅ UNIVERSITY

TRENDS (SINCE 2013)

1. Growing interest in pedagogical career models

2. Two types of models: o Promotion within the academic system o Reward and title not connected to an academic position

3. Web portfolios starting to appear

4. Assessment by external experts is becoming the norm

5. The promoted teachers’ are being recruited to strategic groups and specialized tasks

6. Institutes prefer external experts with experience or training in assessment of pedagogical competence

UMEÅ UNIVERSITY

CHALLENGES

• Diversity or standardization

• Teacher mobility

• Are the criteria adequate

• Incentives for the teacher or the institution

• Labelling of levels (excellent)

• Scepticism towards teaching portfolios

UMEÅ UNIVERSITY

___________________________________________________________________________________________

Katarina Winka, Umeå University; Thomas Olsson, Lund University; Fredrik Oldsjö, Stockholm University; Maja Elmgren, Uppsala University

EXPERIENCES FROM A NATIONAL COURSE FOR PROSPECTIVE EXPERTS OF PEDAGOGICAL ASSESSMENTS

Reason for the courseAssessing teaching competence is different from assessing research competence

• Portfolios vs CV• Local criteria vs national (or personal) criteria

UMEÅ UNIVERSITY

AIMS OF THE COURSEIncrease the potential to assess teaching competence locally by offering support and education for prospective assessors nationally.

o Involve more academics as assessorso Increase the quality of the assessmentso Promote a common languageo Create a community of peer reviewo Promote quality enhancement in teaching and learningo But, the course does not make you an authorized assessor

UMEÅ UNIVERSITY

PROGRAMTWO FULL DAYS AND THREE SHORTER E-MEETINGS IN GROUPS WITH MENTORS

•The role of the external expert -theory and practiceMeeting 1

•Focus on previously assessed portfoliosMeeting 2

•Focus on assessments of new (authentic) portfolios

Meeting3 + 4

•Finalize the written statementsMeeting 5

Test assessment

Pair assessment

Group assessment

Individual assessment

UMEÅ UNIVERSITY

EXPERIENCES

• Wide interest, individual teachers -> national level

• The previous participants are often recruited as assessors of pedagogical competence

You increase your own pedagogical awareness by learning to assess others.

The course makes it visible that pedagogical competence is something that can be assessed, how this can be done, and that it should be a learning experience for the assessed.

UMEÅ UNIVERSITY

…by documenting, assessing and rewarding teaching excellence in higher education

CHANGING THE HIGHER EDUCATIONMERITATION SYSTEM

Teachingportfolios

Improvingassessments

Pedagogicalcareermodels

UMEÅ UNIVERSITY

- IF YOU ALREADY HAVE A CV, IT’S A GOOD START

- GOOD TEACHERS HAVE BOTH EXPERIENCE AND COMPETENCE AND ARE ABLE TO COMMUNICATE THIS TO OTHERS

- MAYBE I SHOULD INCLUDE SOME NOT SO POSITIVE COURSE EVALUATIONS…

- AN ASSESSMENT OF TEACHING COMPETENCE BASED ON CRITERIA IS AS SUBJECTIVE/OBJECTIVE AS AN ASSESSMENT OF RESEARCH COMPETENCE

- TEACHING COMPETENCE CAN BE CAPTURED IN A PORTFOLIO

- THE TEACHING PHILOSOPHY IS NOT JUST FANCY WORDS

UMEÅ UNIVERSITY

REFERENCESApelgren, K. & Giertz, B. 2001. Pedagogisk portfölj, och plötsligt var jag meriterad. Rapport nr 27. Enheten för utveckling och utvärdering. Uppsala

universitet.

Arthur, D. 2005. Recruiting, Interviewing, Selecting & Orienting New Employees. Amacom, New York.

Erikson, M. G., Erlandson, P. & Erikson, M. 2015. Academic misconduct in teaching portfolios. International Journal for Academic Development, 20:4, 345-354.

Kreber, C. 2002. Teaching excellence, teaching expertise and the scholarship of teaching. Innovative higher education, 27: 5 – 23.

Magin, D. J. 1998. Rewarding good teaching: A matter of demonstrated proficiency or documented achievement? International Journal for AcademicDevelopment, 3:2, 124-135.

Olsson, T. & Roxå, T.. 2013. Assessing and rewarding excellent academic teachers for the benefit of an organization. European Journal of Higher Education, 3:1, 40-61.

Ryegård, Å. 2013. Inventering av pedagogiska karriärvägar på Sveriges högskolor och universitet. PIL-rapport 2013:04. Göteborgs universitet.

Schmidt, F.L. & Hunter, J.E. 1998. The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262–274.

Seldin, P., Miller, J., Seldin, E. & Clement A. 2010. The Teaching Portfolio: a practical guide to improved performance and promotion/tenure decisions. 4th ed. San Fransisco: Jossey-Bass.

Skelton, A. 2005. Understanding teaching excellence in higher education – towards a critical approach. Routledge. London

Trigwell, K. 2001. Judging University Teaching. The International Journal for Academic Development 6, s. 65–73.

Trigwell, K., Martin, E., Benjamin, J. & Prosser, M. 2000. Scholarship of teaching: a model. Higher Education Research & development 19(2), s. 155–168.

Winka, K. & Ryegård, Å. 2013. Pedagogisk portfölj, för karriär och utveckling. Studentlitteratur, Lund.

Winka, K. 2017. Kartläggning av pedagogiska meriteringsmodeller vid Sveriges högskolor och universitet. PIL rapport 2017:02. Göteborgs universitet.

UMEÅ UNIVERSITY

THANK YOU

Katarina Winka, UPL, Umeå University, Sweden

Katarina.winka@umu.se

UMEÅ UNIVERSITY

- IF YOU ALREADY HAVE A CV, IT’S A GOOD START

- GOOD TEACHERS HAVE BOTH EXPERIENCE AND COMPETENCE AND ARE ABLE TO COMMUNICATE THIS TO OTHERS

- MAYBE I SHOULD INCLUDE SOME NOT SO POSITIVE COURSE EVALUATIONS…

- AN ASSESSMENT OF TEACHING COMPETENCE BASED ON CRITERIA IS AS SUBJECTIVE/OBJECTIVE AS AN ASSESSMENT OF RESEARCH COMPETENCE

- TEACHING COMPETENCE CAN BE CAPTURED IN A PORTFOLIO

- THE TEACHING PHILOSOPHY IS NOT JUST FANCY WORDS