Evaluating the Impact of Continuing Professional Development

National CPD coordinators network meeting1st February 2008

Gillian Brydson

"Not for the first time, Larry asked himself why he had picked a goat for a coach"

Why is evaluating CPD important

‘Over the years, a lot of good things have been done in the name of professional development. So have a lot of rotten things.

What professional developers have not done is provide evidence to document the difference between the good and the rotten.’

leading to….

‘False or exaggerated claims of success are the basis of many school reform strategies – in part because we lack better and more timely evaluations of new practices and programmes’

Consortium for Policy Research in Education (1996)

2 key questions…

What is the quality of the CPD?

Has the money and time invested in a particular CPD activity made a difference to pupils?

Why is evaluating CPD difficult?

Discussions of evaluation issues often are seen as unwelcome and unnecessary intrusions into the important work that needs to be done.

Thomas Guskey

Why is evaluating CPD difficult?

1. What is quality?

‘What is required for success in professional development is a clear and compelling vision of the improvements needed, combined with explicit ideas on the organisational characteristics and attributes necessary for success’

Guskey (1999)

Why is evaluating CPD difficult?

2. What is ‘evidence’?

Did the earth

move for you?

Behaviour

Instruction

Institution

Psychomotor

Affective Cognitive

Content

Organisation

Method

Facilities

Cost

Pupil Teacher Head Specialists Family Community

So many variables…..

What is your general impression of our

evaluation of CPD?

What types of information do you consider particularly important or meaningful in an evaluation?

(Very Quick )Group Discussion

What are the limitations of our evaluations…

Too brief documentation not evaluation

Too shallow need to probe deeper

Too quick expect too much too soon

Not a self-contained exercise Integral to self-evaluation being

undertaken by teacher, school and local authority

Should be explicit and planned from the outset

Begin with the end in mind.

Overcoming the limitations?

Begin with the end in mind…

If you don't know where you are going you will probably end up somewhere else.

The Peter Principle.

It’s hard but….

We understand what CPD is… We know the major models of

professional development… We have clear purposes and goals

for Scottish Education We agree on our outcomes…

Naomi Model

Kirkpatrick Model

1. reaction 2. learning3. behaviour4. results

Guskey Model

1. Participant reaction2. Participant learning3. Organisational support and change4. Use of new knowledge and skills5. Student learning outcomes

Tools to gather information?

Reaction – post course questionnaire

Learning – reflections / portfolio

Organisational Change – school self-

evaluation,focus groups, stakeholder surveys

Use of new knowledge and skills – interviews, observations

Pupil Learning – achievement, participation, attainment

Professional Discussion

Consider these models in providing an evaluation of the impact of a CPD programme. What questions are addressed? What tools could you use? What measure is assessed? How is the information used?

Is an evaluation framework useful?

If so, to what level?

What measures of outcome are already out there that are relevant to the evaluation of CPD?

Recommendations

Paper considered by CPD coordinators

Work taken forward by national group

When you discover that you are riding a dead horse, the best strategy is to dismount. Dakota tribe proverb

Professional Courage

However…in government, education and the corporate world,

more advanced strategies are often employed, such as:

Giving horse and rider a good bollocking Buying a stronger whip. Changing riders. Appointing a committee to study the horse. Arranging to visit other countries to see how other

cultures ride horses. Convening a dead horse productivity

improvement workshop Reclassifying the dead horse as living-impaired. Hiring outside contractors to ride the dead horse. Outsourcing the management of the dead horse.

Harnessing several dead horses together to increase speed.

Doing a productivity study to see if lighter riders would improve the dead horse's performance.

Declaring that as the dead horse does not have to be fed, it is less costly, carries lower overhead and therefore contributes substantially more to the bottom line of the economy than do some other horses.

Rewriting the expected performance requirements for all horses.

Providing additional funding and/or training to increase dead horse's performance… and the highly effective...

Promoting the dead horse to a supervisory position.

Thank you