Formative evaluationfor Faculties of medicine

About CIDMEF

experience

Arkhangelsk, 2012 1

Jacques Roland

Cidmef : International Conference of the deans of French speaking

Faculties of medicine

Arkhangelsk, 2012 2

CIDMEF spread

Arkhangelsk, 2012 3

38421 3

3

42 countries120 faculties of medicine

2

CIDMEFEvaluation Council

First draft : beginning of the 90’ in Canada (Quebec) : Jean Mathieu, Pierre Potvin, Tewfik Nawar.

First experiments : 1993Tunis, Louvain (UCL), Beyrouth

Finalization of a politic of evaluation : 1995Adoption of basic norms : 2007 BrusselFounding of a quality label CIDMEF : 2010 Lille

Arkhangelsk, 2012 4

Exchanges with other organisations

AERES : French official agency for evaluationLCME : liaison committee in medical education

(Canada, USA) (normes)WFME : world federation for medical education

(normes)FAIMER : foundation for the advancement of

international medical education and researchGlobal Health Initiative (USA)

Arkhangelsk, 2012 5

Principles of the Cidmef evaluation

Volontary action, never imposedImportance of self-evaluationFormative process only (without any

sanction)No purpose of comparison with

others faculties or of standardizationArkhangelsk, 2012 6

Three main questions for the Faculty

Are the missions and the objectives well defined ?

Does it have the necessary means to reach these objectives?

Can it bring the proof that the objectives have been achieved ?

Arkhangelsk, 2012 7

Importance of the objectives

There is no good wind for the seaman who don’t know where he sails (Sénèque)

If you don’t know where you’re going, any road will do (Alice in wonderland, Lewis Carroll)

Arkhangelsk, 2012 8

Institutional objectivesSet up by the « society », Department of

Education, of Health, University…Three fields :

Education, Research, Provision of servicesMake Reference to values :

Universal values : respect of the Human Rights and of Life

Basic values of health systems (quality, equity, efficiency)

Social responsability of medical Faculties Arkhangelsk, 2012 9

Derived and specific objectives

- Precise- Operational- Ready to be checked

Arkhangelsk, 2012 10

Cidmef basic norms• Universal, • Minimums must be shared by all the medical

schools• Can be used like quality criteria for a Cidmef

labelling

Arkhangelsk, 2012 11

Evaluation steps1. Local decision and request to

the CIDMEF2. Preliminary visit3. Self-evaluation4. Visit of the external

commission 5. Report6. Follow-upArkhangelsk, 2012 12

1. Local decision

crucial !Evaluation is going to provoke a

great deal of effortsTo make the decision, a strong

support from all is required and in particular :

• From a large majority of the teachers

• From the local and national Authorities

Arkhfrom theangelsk, 2012 13

2. Preliminary visit(1)By a limited external group (2

people)Meeting with :

authoritiesteachersemployeesstudents

Arkhangelsk, 2012 14

2.Preliminary visit (2) : objectives

InformationReassuringMotivationPlanning

Specific status of the students (private meetings)Arkhangelsk, 2012 15

3. Self-evaluation (1)

Fields Governance Programs Continued medical

formation Research Teachers Students Administration Financial and

material resourcesArkhangelsk, 2012 16

In each field:• Description of the staff• Description of the means

In each field self-analysis• Opinion about strengths

and weaknesses• Internal propositions for

corrections, reforms, advices and plans

3. Self-evaluation (2) : the curriculum

Description of contents, duration, management of teaching,

Balance between Care Medicine, Prevention, Public Health

Balance between theory and practice, importance of autonomy

Check the processes of internal evaluation for teaching and programs

rkhangelsk, 2012 17

3. Self-evaluation (3) : professionalism

Connection with society, medical profession,other health professions,

Approaches to ethical and deontologic aspects during curriculum

Reality of formations to communication, decision, adaptation of behaviour

rkhangelsk, 2012 18

3. Self-evaluation (4) : strategy

Direct objectives

Obtain clarification about mission and objectives

Writing the report of self-evaluation for sending to the external commission

rkhangelsk, 2012 19

Meta-objectives

• Generate a collective thought about shared values, missions and objectives

• Mobilize the staff for a commun policy

• Favorise connection in and between the departments

• Develop or create a spirit of evaluation

3. Self-evaluation (5) : Students’ specific case

The report of students is sent directly to the experts of the external commission (no transmission to the local authorities)

Arkhangelsk, 2012 20

4. External commission (1) : composition

Five membersthree foreigners, two from the country

All members are teachers of medecine, with collective responsabilites in their University

Designation of a president and a secretary within the commission

Arkhangelsk, 2012 21

4. External commission (2) : casting

President : leader, allocate tasks to each member, responsible for synthesis of the report

Secretary : connections between the commission and the Faculty, research of documentation

Each member is responsible for a part of the evaluation and then of the final report.

Arkhangelsk, 2012 22

4. External commission (3) : material conditions

Members of the external commission are not paid, the journey is paid by the Cidmef, the stay charges by the evaluated Faculty

The evaluated Faculty is responsible for the organization

Arkhangelsk, 2012 23

4. External evaluation (4) : Progress

Dean and President of University are met individually

Meetings with all the officials, persons in charge for teaching, research, administration, students, professional associations..

Stay during 3 to 5 days according to the importance of the Faculty.

Arkhangelsk, 2012 24

4. External evaluation (5) : investigation

The report of self-evaluation guides the investigation : choice of the persons to be questioned, clarifications to be obtained, further fields to be explored

Comparison between the opinions expressed on the same subjects by different actors.

Arkhangelsk, 2012 25

4. External evaluation(6) :Preliminary report

At the end of the stay, the external commission presents to the Dean and to the President of University a short report.

This report describes the main conclusions and the remaining interrogations. Authorities’ answers will be used for the final report.

Arkhangelsk, 2012 26

5. Definitive report

Each member of the external commission writes his part of the report and sends it to the president of the external commission.

The last step is to harmonize the different chapters, then to highlight strengths and weaknesses in a specific chapter before giving recommandations and advices.

The definitive report is sent only to the Dean and to the President of University. The spread of the document is done under their own responsability.

Arkhangelsk, 2012 27

5. Follow-up

A new visit is organized within the two following years to check the effects of the evaluation

Arkhangelsk, 2012 28

Global assessment

38 Faculties evaluated

Encountered difficulties: Impossibility to finalize (2) Opposition of Authorities (2) Difficulties to put the advices into practice New Dean, new President…

But in 34 cases, a regarded tool for imposing change and moving forwardArkhangelsk, 2012 29

Conclusion

We have the choice…

Arkhangelsk, 2012 30