What works for Evaluators of Management Education ?

Elaine Clark

Action Learning for Senior Leaders in the NHS in London

Commissioned by NHS London, SHA

Aims:

•Provide support through

individual/organizational transition.

•Facilitate building of personal

resourcefulness though creation of a

supportive network.

•Enhance participant’s passion for improving

services for patients through learning and

reflection.

•Enhance resilience.

Challenges in evaluationIdentifying the meaning…

illuminating impact

What does this mean?

Love conquers all...!

Or…?

Finding surprises…

Evaluative challenges

Accountability…

• Commissioner?• Participants?• Research community?• Self?

Need for complementarity - relationships

Between…•p AND q•Quantitative (positivistic) and qualitative/interpretative•Commissioners and participants

Searching for reconciliation

Practical implications• Causality – mutual, not linear– Causality may not be temporally precedent, but

simultaneous or antecedent– Emphasis on relationships, the networks– Need for multiple perspectives - multi-stakeholder

interviews, with as many different perspectives as possible• Need to ‘leave space’ in design for highly exploratory

stage. Uncover unexpected/unintended impact.• Complementarity of research methodology –

comparative data against a ‘control group’; interviews and participative inquiry.

• Participation, regular and critical: Stakeholders kept informed as evaluation proceeds, encouraged to challenge, dispute, add to findings.

Brings to the fore, tensions, emotions and politics.

• Focus upon relationships; Internal and External

• Reflexive/questioning approach as an evaluator: • Are we doing better things as well as doing things better (Burgoyne,

2010)• this requires action as a consequence

Some findings…

• Importance of relationships: internal and external

• Importance of P, as well as Q• Growing awareness of participants of politics

and emotions within the health care environment of London

• Enhanced resilience• Awareness of importance of, and courage to

ask, questions

Some findings…“Given me the ability to gain a broader view of organisations outside my normal sphere and given me confidence through

a greater knowledge of non exec roles within the organisation.”

“Programme has provided opportunity to better understand current issues affecting NHS providers from an operational

perspective.”

“There was a lot about organisations coming together , with widely different systems, cultures and legacies and starting

to understand those different perspectives.”

“Have gained greater awareness of what other people are doing. Also, opportunity for networking and

(gaining) general awareness of how other people are impacted on by the system and how they operate within

it helps you to reflect on your own practice and your own organization. A practical example is that I am

reviewing some of our board papers on the back of it.”

“What the programme did was to help me to understand some of the sensitivities around NHS London”

What was carried forward?• NHS London became Clinical commissioning groups• Importance of knowledge of, and ability to build,

relationships, networks• Enhanced self-belief• Enhanced ability to question and therefore to learn –

generative learning?• Knowledge about the system – the P, but also the

politics and sensitivities• With a system change like this, is the learning more

likely to be carried forward or less?

A suggestion…

• From sociological paradigms to …

Action learning as a ‘sociology of freedom’?

References• Burgoyne, J. G. (2009) Issues in action learning: A critical

realist interpretation Action Learning: Research and Practice 6, 2: 12

• Burgoyne, J. G. (2010) Evaluating action learning: a critical realist complex network theory approach Action Learning: Research and Practice 7,3: 239-251

• Burrell, G and Morgan, G (1979) Sociological Paradigms and Organizational Analysis, London and Exeter: NH. Heinemann

• Parlett, M; Hamilton, D (1972) Evaluation as illumination: a new approach to the study of innovatory programs Occasional paper: Edinburgh University Available at ed.gov, accessed May 2nd 2014