transforming the health workforce in support of universal health coverage

Transforming the Health Workforce in Support of Universal Health Coverage:

A global toolkit for evaluating health workforce education

Report of the 1st Meeting4–5 December 2013

Report of the 1st Meeting4–5 December 2013

Transforming the Health Workforce in Support of Universal Health Coverage:

A global toolkit for evaluating health workforce education

© World Health Organization 2014

All rights reserved. Publications of the World Health Organization are available on the WHO web site (www.who.int) or can be purchased from WHO Press, World Health Organization, 20 Avenue Appia, 1211 Geneva 27, Switzerland (tel.: +41 22 791 3264; fax: +41 22 791 4857; e-mail: [email protected]).

Requests for permission to reproduce or translate WHO publications –whether for sale or for non-commercial distribution– should be addressed to WHO Press through the WHO web site (www.who.int/about/licensing/copyright_form/en/index.html).

The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of the World Health Organization concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted lines on maps represent approximate border lines for which there may not yet be full agreement.

The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by the World Health Organization in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters.

All reasonable precautions have been taken by the World Health Organization to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall the World Health Organization be liable for damages arising from its use.

Layout L’IV Com Sàrl, Villars-sous-Yens, Switzerland.

Printed in Switzerland.

I A global toolkit for evaluating health workforce education

Contents

Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . II

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Day 1: proceedings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Opening session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Addressing the agenda . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Summary of day 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Day 2: proceedings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Summary of day 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17Next steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Annexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20Annex 1: Agenda . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Annex 2: List of participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Annex 3: WHA resolution 66.23 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Annex 4: Assessment tools for health workforce education (draft) . . . . . . . . . . . . . . . . . . 28

Annex 5: Terms of reference for the technical working group on health workforce education assessment tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

Annex 6: Technical working group subgroups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Annex 7: Asia Pacific Network on Health Professional Education Reform (ANHER) . . . . . . 34

Annex 8: Presentations by members of the Technical Working Group on Health Workforce Education Assessment Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

II Transforming the Health Workforce in Support of Universal Health Coverage:

Executive Summary

This two day meeting focused on the implications for the implementation of Resolution WHA.66.23 Transforming health workforce education in support of universal health coverage, and on finding agreement on the terms it sets out. WHO was tasked with the responsibility of developing a standard protocol and health workforce education assessment tool.

In view of the historical problem of not only a global health workforce shortage, but an urgent need to ensure that such a health workforce has a broader training which more accurately reflects their everyday working practices, World Health Assembly Resolution 66.23 urges Member States:

� to further strengthen policies, strategies and plans, as appropriate, through intersectoral policy dialogue among the relevant ministries that may include ministries of education, health and finance, in order to ensure that health workforce education and training contribute towards achieving universal health coverage;

� to consider conducting comprehensive assessments of the current situation of health workforce education with the application of, as appropriate, standard protocols and tools, once developed by WHO;

� to consider formulating and implementing evidence-based policies and strategies, taking into account the findings from the assessment in the previous paragraph, to strengthen and transform the health workforce education and training, including but not limited to the promotion of inter-professional, community-based and health systems-based education, linkages of pre-service education to continuous professional development, and an accreditation system to ensure quality of training institutes and competency of health workforces; with a view to better responding to the health needs of people, taking into account the special needs of some Member States that have limited economy of scale in local training;

� to provide adequate resources and political support for the implementation of policies and strategies as appropriate for the strengthening and transformation of health workforce education;

� to share best practices and experiences on health workforce education.

Broad consensus was reached on the following key issues, along with a proposed way forward.

Standard protocol: It was agreed that this is a ‘road map’ and that the assessment tools will help to assess progress on this map.

Unhelpful terminology was also reviewed in order to eliminate confusion. Further agreement was reached on the following:

� To use the term ‘health worker’ rather than ‘health professional’ so as not to exclude those health workers who are not professionals. This is in line with the term ‘health workforce’ which is used in the WHO World Health Report 2006.

There should be an assessment system which includes a library of tools, as the available information on health education assessment tools appears to be patchy.

Components: The key components of health assessment tools should address the needs of the target population as well as reflect the competencies required by health workers and the ability of institutions to impart the necessary training and education.

III A global toolkit for evaluating health workforce education

Tools

� the tools should be able to take a ‘snapshot’ of the current situation and to also facilitate transformation

� the tools should make use of the learning opportunities which are competence-based � the tools need to be practical and user-friendly and also be able to capture the presence/absence of specific competencies and skills

Actions

� Three subgroups were formed to take this work forward; Subgroup 1: to look at components, conceptual frameworks presented. Subgroup 2: to generate a set of questions to interrogate the tools; Subgroup 3: to assess the feedback from subgroup 1 and subgroup 2 and to then begin to build an assessment protocol and tool.

� A Concept Paper subgroup was also formed to allow the agreements and decisions following this first TWG meeting to be included in the present draft.

� The following timeline was agreed:

By mid-2014 There should be agreement about which countries to test the tools.October 2014 Begin country testing November 2015 The report should be readyJanuary 2016 Present the report to the World Health Assembly the report to the

WHO Executive BoardMay 2016 Present the report to the World Health Assembly the report to the

World Health Assembly.

1 A global toolkit for evaluating health workforce education

1. Introduction

The purpose of this two day meeting was to allow the newly-formed Technical Working Group on Health Workforce Education Assessment Tools to convene its first meeting during which it would address the key objectives as stated below and share ideas and agree on how to implement WHA Resolution 66.23. This consultation brought together a multidisciplinary group which includes: health education specialists, senior academics and persons responsible for governance of education institutions, policy makers, experts in health systems and international health, health economics, research and civil society representation. The meeting enabled the participants to challenge, discuss and recommend actions on how to develop a standard protocol and tool for health workforce education assessment. The meeting began by addressing the question of tool components and conceptual frameworks, looking at existing tools, and target audiences.

Further to World Health Assembly Resolution (WHA.66.23), WHO was tasked with the following :1. to develop a standard protocol and tool for assessment, which may be adapted to country

context;

2. to support Member States as appropriate in using the protocol to conduct comprehensive assessments of the current situation of health workforce education;

3. to provide technical support to Member States in formulating and implementing evidence-based policies and strategies in order to strengthen and transform their health workforce education;

4. to consult regionally in order to review the country assessment findings and submit a report providing clear conclusions and recommendations, through the Executive Board, to the Sixty-ninth World Health Assembly;

5. to develop, based on the report, global and regional approaches, which may include strategies to transform health workforce education, submitting these through the Executive Board, for consideration by the Seventieth World Health Assembly

A Technical Working Group on Health Workforce Education Assessment Tools had been recently formed to reviewand provide guidance to the WHO Human Resources for Health Unit on the scope and use of health workforce education assessment tools at national, institutional, prequalification and graduate levels.

The Technical Working Group on Health Workforce Education Assessment Tools met on 4-5 December 2013 in order to agree on the following nine objectives:1. The terms of reference for the Technical Working Group on Health Workforce Education

Assessment Tools, to ensure clarity at the outset for the group.

2. The need for, and purpose of, a standard protocol and set of tools for assessment to support Resolution WHA 66.23, to establish the foundation for future planning of the tools.

2 Transforming the Health Workforce in Support of Universal Health Coverage:

3. The targeted audience for these tools.

4. What kinds of components should be part of effective assessment tools.

5. What existing tools may be suitable to assess the health professional education continuum, in order to begin to identify what is useful and where there are gaps.

6. The key overall milestones for the Technical Working Group, with timeline.

7. How best to configure the group in order undertake the work effectively.

8. An action plan – what; how; by whom; and by when.

9. Immediate next steps.

Twenty two Technical Working Group members, in addition to three WHO personnel, and a WHO medical intern participated in the meeting. At present, the total number of members of the Technical Working Group is 41.

The target audiences will be largely Member States, their ministries of health, education and finance respectively, policy makers. The final document will be useful at all levels of the health system as well as training institutions. The information contained within it will be evidence-based and of relevance to academic institutions.

The meeting outcome was very positive with the Technical Working Group members keen to collaborate and meet the objectives set out during this first meeting. Critical issues are mentioned in the Executive Summary, as well as Key Actions.


Day 1: proceedings

The consultation began with a welcome and introductory remarks which set the context for the meeting. The focus of the first day’s proceedings was on discussion and agreement on the Terms of Reference for the Technical Working Group, the need for a standard protocol and set of assessment tools to support Resolution WHA66.23, the targeted audience for these tools, and the components of these tools. Presentations were followed by group work and feedback. The consultation was facilitated by Mr Stevie Johnston of Interaction Institute for Social Change.

Opening sessionDr Ruediger Krech, WHO Director, Office of the Assistant Director-General, Health Systems and Innovation

Dr Krech stated the importance of the discussion on how the Technical Working Group would work closely with WHO to effectively implement WHA.66.23 and reminded the group that the agenda for the two-day meeting was full and expectations were high.

The goal of the meeting was to meet the objectives set out in WHA Resolution 66.23 (see Annex 1: Agenda)

Dr Erica Wheeler, WHO Technical Officer set the context for the meeting and explained the reality of the timeline in terms of when the final report has to be ready. It was stressed that the Technical Working Group would decide how these milestones would be reached and that the Terms of Reference for the Technical Working Group is a map of how to reach the deadline of November 2015 from December 2013. It was also stated that WHO is depending on the Technical Working Group to fulfill its Terms of Reference as WHO cannot fully implement this without the active participation of the members of the group.

Addressing the agendaThe meeting began by the group addressing each of the nine points on the Agenda (Annex1) and discussing them in order to reach understanding and agreement.


1. A shared understanding of the terms of reference for the technical working group on health workforce education assessment tools

The group examined the Terms of Reference in order to make sure that the terms were fully understood and agreed upon by all. This discussion concluded with a consensus about the Terms of Reference and what methods might be used to meet them.

DISCUSSIONKey discussion points included:• Thegeneralfeelingisthatthisinitiativeislongoverdueandwillprobablytakemorethan

two years to be completed.• Althoughthereisalotofinformationoneducationassessment,ittendstobepatchy.• Thefocustendstobelargelyondoctorsandnurses,andsoitisimportanttohavea

wider view of all the relevant health professions, e.g. dentists, pharmacists and others.• Questionsaboutdataonhealthworkforceeducationassessmentshouldbeconsidered

carefully.• It is important to definewhat is actually being assessed before agreeing on the

development of tools.• Wemustaddressthefundamentalquestionaboutwhetherthisassessmentisareflection

of the past, or of a need to change for the future.• Whatshouldbethemethod of assessment?• Howwerethetoolsusedinthepastascomparedtowhatweneedtodonowlooking

towards the future? • Howdowebringaboutchangeswiththesetools?• Itisworthnotingthatahealthworkforceisnotonlyproducedbymedicalinstitutions.• Theassessmenttoolsshouldbegenericenoughtobeadaptablebutspecificenough

to be used to obtain the type of assessment at the country level required.• Roadmapsandstrategiesmustbeconsidered,notjusttools.• WHA.66.23isaskingforaglobalassessment,claritywassoughtonwhethertheTechnical

Working Group will develop the tool and WHO will perform the assessment?• Weneedtobeclearaboutwhowemeanwhenwesaythehealthworkforce.• Canonetoolapplytoallhealthprofessions?• Weneedtobeclearaboutwhetherthetoolsareonlyforhealthprofessionalsornot.• Consultingregionallyisimportant.It’sbestforregionalconsultationtobedonewhilstthe

tools are being developed.


2. The need for, and purpose of, a standard protocol and set of tools for assessment to support resolution wha66.23

During the 2013 World Health Assembly, Resolution WHA66.23 (See Annex 3: WHA Resolution 66.23) was passed, whereby WHO was tasked with the responsibility of developing a standard protocol and health workforce education assessment tool. With little information about health workforce education, the need for mechanisms to assess the competence and performance of the health workforce has continued to receive more and more attention.

The group discussed the above objective and covered the points below:

DISCUSSIONKey discussion points included:• ThepurposeoftheTechnicalWorkingGroupistoformpolicies,strategiesandplans–

intersectoral dialogue is therefore key.• LookingattheWorldHealthReport2006,thereisnosingletoolproposedthatcoversall

that is necessary.• TheLancetReportlooksatlevelsoflearningwhichaddresstransformativelearning• Weneedtotakeintoconsiderationwhataretheessentialcomponentsofthecurriculum

that are needed.• Itisimportanttoclarifywhatkindofhealthworkforcewewanttoproducealthoughwe

cannot be too prescriptive. It is therefore better to cover the whole health workforce and attempt to address the unmet needs, and thus incorporate other cadres which have not been considered.

• It’snotjustaboutproducingtools,wearelookingatanassessment system that includes tools but includes more than tools and so we need to think about what else it should include.

• Inconsideringtoolstoassesseducation–theassessmentofhealtheducationwillbevery different in 10 years’ time, so we need to look forward and not just at the current situation.

• Weneedtofindawayforinstitutionstoadaptandthisshouldincludeadaptation of their assessment procedures.

• Weneedtobeclearabouthealthprofessionalinstitutions’sphereofinfluence.Itisalsoimportant to make these health professional institutions accountable.

• Weare not just stocktaking,WHA66.23 talks about transforming healthworkforceeducation assessment. This involves stocktaking and transformation. Both are relevant.

• Aroadmapisimportantbecauseitstablisheslandmarksandmarksthestagesofourprogress.


3. Who is the target audience for these tools?

It was agreed that before beginning to consider transforming health workforce education assessment tools, it would be necessary to first think about who the target audience is.

DISCUSSIONKey discussion points included:In considering the target audience we need to be aware of the fact that some health workers do not end up doing the work they chose at the start of their training (there are graduate studies being done to throw more light on this area). We need to look at the following:• Whatdoweenvisionforthenext20-25years?Thereise-educationandonlineeducation.• Whatisthecurrentsituationofhealthworkers?• Weneedtothinknotonlyoftoolsbutofan‘assessmentprocess’.• Whatkindsofpoliciesandplansareneededcomparedtowhatcurrentlyexists?• Addabroadtitleof‘healthprofessionaleducationinstitutionsandorganizations’.

It was generally recognized that the WHA Resolution 66.23 addresses the whole health workforce, not only health professionals.

Categorization of groups1. Policy makers and advisory bodies2. Government and external entities3. Policy and decision-makers4. Intersectoral groups/mechanisms

Further discussion topics included• Theneedtoconsidergovernmentandprofessionalorganizationswhoareinvolvedin

decision-making.• Differentfunctionsaccordingtothecountrycontextmustbeconsiderede.g.intheUK,

the General Medical Council works with government in making decisions.• Politicaldecisionmakersandprofessionaldecisionmakersshouldbekeptseparate.• Theassessmenttoolsareforresearchersaswellaseducatingandtraininginstitutions.

It would be helpful to see where the gaps in training are.• Wearetalkingaboutchanging professional health education. • Nochangescanbemadeineducationwithoutinvolvingallthestakeholdersfromthe

very beginning.• A failure inplanning indeveloping thehealthworkforce is usuallydue to the lackof

coordination between education, health, and finance.


4. The components of effective assessment tools

The group firstly brainstormed to get agreement on what they thought ‘components’ were and then refined their thinking in small groups and agreed an outline list of components.

DISCUSSIONKey discussion points included:• Countrieshavedifferentprotocolsforinstitutionalassessment.• Samplingdoesn’tprovideindepthdetail.Howcanwegobeyondasamplediagnosis

and have a more accurate assessment?• Howdoyourecognizewhichschoolsareusingacompetency-basedapproach?• Itiseasiertoassesshealthstudentswhilsttheyareinschool.• Thisisasnapshot of the situation in these 5 countries in Asia.• Apilottoolassessmentisneeded.• ThisAsiansurveyshowsthatweneedtohavesuchasnapshotbutitisalsoimportant

to determine if this snapshot should be national or international.• Thenationalsituationisnotthesameasaglobalsituationandsoweneedtoconsider

whether we want the instrument to assess the global situation (to be decided as part of further discussions as the work progresses).

• Wouldthesnapshotsneedtobeverydetailed?Ifthesnapshotsareforglobalusetheyshould be globally comparative.

• Keycomponentsshouldaddresstheneedsofthepopulation,thecompetenciesrequiredby health workers, and the readiness of the institution to impart the training and education. This should direct transformational growth and cross between Education, Health and Finance.

• The area of collaboration is very important. An important consideration iswhetherinstitutions have the resources needed to develop the curriculum to meet the needs of the country?

• Whatshoulddrivethisprocess?Thedemandortheneed?Whosedemandarewetalkingabout?

• Itisimportanttolookateducationasitexiststoday,aswellashealthsystemsneed,HRHstatus, transformational guidelines; faculty development is an important part of the development, CPD, diversity and cultural components; the tools should be culturally sensitive to avoid offending other cultures.

• Theproblemsofneedsanddemandsshouldbeincludedintheassessment.• Weneedtolookattheadmissionsprocessasitrelatestohumanresourcesdemand• Inmanycountries,health institutionsarepubliclyfinancedwhichmeans that there is

accountability. • Traditionalvsnon-traditional–weneed to lookathowpeoplearebeing trained.For

instance, education programmes that result in a diploma or recognized credentials.• Therearemanyhealthworkerswhoarenotprofessionally trained,buthavereceived

training in a different way. It is important to empower this group of health workers.• Thereisaneedtogobeyondbestpracticesofthepast.


Brainstormed list:• Competencies.• Coversalldisciplines.• Promotioninter-professional,communitybasedandhealthbaseseducation.• Pre-serviceeducationlinkedtoCPD.• Existenceofmulti-stakeholdertaskgroup.• Accountability–socialandpublic.• Communication.• Visionforwhatwewanttoachieve.• Collaboration–atlocal/national/internationallevel.• Accreditationsystem.• Financing.• Privatesectorinvolvement.• Curriculum.• Institutionalsustainabilityandrobustness.• Educationoversight.

There was feedback from small groups in order to refine the components. – (n.b. the words highlighted reflect the emergent key components which participants identified in a narrowing exercise after the small groups reported back. It was also noted that participants identified the need for the tools to be able to take a Snapshot of the current situation and also to facilitate Transformation).

Group 1• ContextAssessment–educationtoday–whatarethetraininginstitutions(traditional,

non -traditional, public, non-state) doing now?• Healthsystemneeds(buildonexistingdata).• Transformational‘GuidelinesPlus’.

» Career pathways (clinical and faculty). » CPD (clinical and faculty). » MLH & CHW.

• Inter-professionalEducation.• Diversity/CultureCompetence.• Accreditationandoversightmechanisms.• Inter-sectoralCollaboration.

Group 23 Key areas 1 tool

Snapshot • CurrentSituation.• GapsassessmentinUHC.

Capacity for change• Vision.• Policydialogue.• Accreditation.


Progress for change• Keytransformativemeasures.• Unifyvisionfrompre-servicetoCPD.

Group 3

Snapshot (Needs – Competencies – Institutions).

1. Snapshot for transformational growth.2. Close the gap between education, health systems and practice.3. Address the potential mismatch between needs and health care service by focusing on

competencies (core competencies for IPE and IPP).4. Competency based assessment.5. Tools need to be practical and user friendly.6. Collaboration is key –partnerships.7. Financial sustainability (Resources).

Group 41. Keep it very simple, adaptable, comprehensive, easily analysed, reliable, valid.2. Define the need in terms of the skills mix number and distribution, of health workers.3. Curriculum: does it support the development of graduates who meet the country’s current

and future needs? » Is there a mechanism (e.g. committee of multi stakeholders) to ensure relevant

curriculum over time? Collaboration?4. Institutional Resources: Does the institution have the financial, human, material and

intellectual resources needed to ensure the delivery of the curriculum (see 3 above). Collaboration?

5. Quality Assurance and oversight. » Internal mechanism for quality improvement (e.g. a unit). » External mechanism for quality assurance. » Collaboration.

From the work of the small groups, during the plenary the participants agreed the following:

Components (i.e. the highlighted terms above).• Capacityfor/Progressforchange• Snapshot• Competency• Intersectoral• Collaboration• Interprofessional• QualityAssurance/Oversight• Resources• Curriculum• Skillsmix• Need


Participants also noted the following points:• Necessarytodefinewhat‘need’and‘demand’meanwithinthiscontext.• Admissionprocessesneedtobeconsidered.• Build onGuidelinesPlus –WHOTransforming and scaling uphealth professionals’

education and training document.• Innovationheldbackbyevidence-basedthinking?

Summary of day 1Day 1: Summary of topics

� Need � Skills-Mix � Curriculum � Resources � QualityAssurance � Inter-professional � Intersectoral collaboration � Competency-based assessment � Snapshot of health workforce education � Key change elements

The following key words were considered to be important:1. Snapshot

2. Change

3. Intersectoral Education

4. Resources

5. OversightonQualityAssurance

6. Skill mix

7. Competency

Day 1

Agreement

� Assessment tools need to be able to take a snapshot of the current situation and also facilitate transformation, therefore look towards the future.

No agreement

� Although 4 broad categories of target audiences were agreed, there was no consensus on their titles.

� There was no agreement on which specific elements should be included in the snapshot.


Day 2: proceedings

The focus of the second day’s proceedings was on the effectiveness of assessment tools, the timeframe during which the TWG and WHO must meet its deadlines, how the TWG can best undertake this work and what the next steps are.

5. The suitability of existing tools to assess the health professional education continuum

The group examined the Terms of Reference in order to make sure that the terms were understood and agreed upon by all. This discussion concluded with a consensus about the Terms of Reference and what methods might be used to meet them.

Presentations

Members of TWG were invited to give a five minute presentation on specific assessment tools that they were familiar with: (see Annex 9)

DISCUSSIONFour discussion groups were formed which concentrated on the following: what they liked about the tools that were circulated, what weaknesses there were as a body of tools and what gaps exist. The table below reflects the outcome of the discussions about the assessment tools.

Strengths Weaknesses Gaps

Group 1 • Broad range of existing tools

• Tools are contemporary an forward-looking

• Adaptable and comprehensive

• Embedded in continuum• Opportunity to work in

modules in different areas

• We need to know more about these tools

• Constraints - financial issues (didn’t know about the financial constraints regarding the tools)

• Variable stakeholders or targets

• Difficult to tell – need to be set against Guidelines +.

• Need to define Guidelines +• Link between health

education and Health Professional Workforce (HPW)to the ultimate outcome, UHC, Saving life, UN Access.

• Recommendation about tools

• Develop core and modules.• Need to translate tools into

different languages.• Need to form a group of

stakeholders• TWG –needs to form sub

groups


Strengths Weaknesses Gaps

Group 2 • What is the ultimate purpose? – How are the various existing tools contributing to this?

• Generic (inclusive) adaptability / relevant.

• Should inform a transformational process (road map).

• QI ability.• Identify key factors

affecting HPEI’s ability to influence / improve outcomes – HS/UHC/Equity/Accountability.

• Need to develop/identify research priorities.

• Advocacy (role of institutions) pressure from society (consumer).

• Need to regularize the level of health professionals

• Need to respond to the country’s need

Group 3 • All tools had positive aspects that could be used – quality improvement cycles.

• Timeframes for administration.

• Dynamic tool, adaptive, simple, workable, use by a range of people.

• Specialist competency.• Length of tool.

Recommendation:• Small group to evaluate

all tools and identify best model aspects.

• The transformative process will be applied according to the country’s resources

Group 4 • Breadth, richness and focus of tools we can build on.

• More aspirational than we were aware of.

• We can assess against what we identified through discussion/ resolution/guidelines.

• Further identify Guidelines + e.g. the way students need to learn when there is a proliferation of information – personalised education.

• Process – need to look more in depth to not appear to measure capability for change.

• Concern for the large amount of data collected.

• Ability/capacity to collect data at global level.

• Do not bring next steps.

• Need to examine these more carefully – Global context?

• Build a framework – what are the critical ‘things’ we need to know.

• Identify levels of collaboration & leadership.

• The political context must be considered when developing a tool.

• Strong advocacy is necessary.

• Pressure must come from civil society.

The group went on to discuss various aspects of developing tools.


DISCUSSIONKey discussion points included:• TheconceptofGuidelinesPluswasintroducedasawayofbroadeningtheagendato

include other cadres such as mid-level providers and community health workers who are not covered by the current WHO guidelines on Transforming and scaling up health professionals’ education and training. If we are to develop assessment tools that address the health workforce, we need to look beyond health professionals. NGOs also provide education and training of health workers in some countries and this needs to be taken into account.

• Whendevelopingthetoolsweneedtotakeintoconsiderationthedifferencebetweencomponents of tools for a snapshot and components for change leading to a transformation of education.

• Thisisaglobalissueandnotjustrelatedtocrisiscountries.• ItwassuggestedthatitwouldbeinformativetorefertothewebsiteofCOPASAH,inorder

to assist us in further thinking about assessment tools at the community level. The website address is http://www.copasah.net/

6. Key milestones for the technical working group

Sandra Pandi, from WHO presented a broad outline of the timeframe of this work up to May 2016. It was noted that the main milestones are non-negotiable as they are laid out in WHA Resolution 66.23.

Timeline

� By mid-2014, we should have agreed in which countries to test the tools. � October 2014- begin country testing � October/November 2015 – the report should be ready for the WHO Executive Board � January 2016 –The report will be submitted to the WHO Executive Board � May 2016 The report will be presented in written form to the World Health Assembly.

Questions for the technical working group:

� Is this timeline realistic? � What can realistically be done and by when? � What suggestions does the TWG group have?

Meetings

DECEMBER 2013 The first Technical Working Group on Health Workforce Education Assessment Tools (4-5 December)

JANUARY 2014 Webex discussion of the work of Subgroup 1FEBRUARY 2014 Teleconference with each of the TWG subgroups (TBD)MAY 2014 Technical Working Group meeting (21-22 May) * the same week of the World

Health Assembly which takes place 19-24 MayJULY 2014 Technical Working Group meeting (30 June to 3 July)


A decision will be made in February/March 2014 as to the content of the report which we will submit at the World Health Assembly 2014 Side Meeting.

subgroups (see Section 7)**In January 2014 the subgroups were restructured

Tools

The TWG needs to choose which tools to test by early next year. By mid-year 2014, country testing needs to be carried out, so agreement on which countries to start implementing the tools is needed.

Important points to consider in country testing: � How to choose countries? � Which countries are TWG members already working in/familiar with?

Our approach needs to be strategic as the results of country surveys will inform policies.Following the above WHO presentation, the participants discussed their concerns about the timeframe, how to go about developing a tool and the proposed meetings for 2014.

DISCUSSIONKey discussion points included:• Thetimeframefordatacollection,implementationandevaluationisJanuary-June2015.• Therewassomeconcernfromthegroupaboutthedeadlines.• OnceTWGformssubgroups,amoreaccuratepictureoftheworkaheadmaybecome

clearer and once it is agreed on how soon they can begin.• Needtoallowtimeforpiloting.• The PrinceMahidol AwardConference 2014 (PMAC)meetingmight be a useful

opportunity to move this work forward if enough members are attending.• CanwebuildonwhatsomeofTWGmembersarealreadydoing(regardingthetools)?• Scientificvalidityisimportant.Wherecanwegetscientificled-tools?FromWHO?The

tools should have the right concepts.• DoweneedaPlanB?–Whatresourcesareavailable?–(thiswillbeinfluencedbythe

parameters of the work).• ThePlanBisthatweneedtocomeupwithatoolandsomecountrieswherewehave

tested them. A tool developed and tested could run in parallel with existing tools that are in already in operation. We need evidence that we have tested in countries. The TWG needs to decide on country testing.

• Whatresourcesareneededtodothis?Manpower?Cost?• TheTWGmustdiscusswhatitiswearegoingtodo.Arewegoingtoworkwithexisting

tools? The TWG needs to set up the parameters of the work.• WHOstatedthattheTWGmustdecidehowweproceed.2016isnottheendofthework.

The timeframe is much longer.• Thetoolsneedtobeglobalandnotjustregional.WHOisworkingwiththeTWGtodecide

and agree on how to proceed. We are re-defining how to work and collaborate together.• Thetoolshouldbemodularwithacoremodule.• Whatisouroverallframework?Howdowereviewexistingtools?Weneedtheoverall

conceptual framework. This is very important. We need all the existing tools related to various concepts.


The Facilitator asked the group to clarify and discuss the following topics in more detail:1. Discussion on Guidelines Plus

2. How the TWG planned to organize the work

3. Action Planning

DISCUSSIONKey discussion points included:• Dowewanttomakeagreatimpactonhealthcaredelivery?• IntermsofGuidelinesPlus,weneedtools/indicatorsthatuniversalhealthcoveragewill

be met with the transformation of the educational assessment tools. We also need to look at where we are starting from, as the starting point is different in different countries.

• Wehavenoframeworktoputtheseissuesin.Lookingattheframeworksusedinthedifferent presentations this morning might aid us in developing a tool.

• UniversalHealthCoverage(UHC)willbeinthecentre.• GuidelinesPlusneedstofocusmoreonALLmembersofahealthteam.• Needtoconsiderothermembersofthehealthteame,g,MLPs,andhealthmanagers• Theissuesofaccountabilityshouldbecross-cutting.• WecantransformeducationandnotevendeliveronUHC.Wearetransformingeducation

but perhaps we need to also transform the health system.• It is important to linkUHCand other principles such as: equity, quality, efficiency,

partnerships, relevance, affordability.• See theUniversalHealthCoverageWorldHealthReport2013http://apps.who.int/iris/

bitstream/10665/85761/2/9789240690837_eng.pdf• Thetoolisabouthowtomodifyeducation.Ithastodocumenttowhatextentthehealth

institution is not going in the right direction.• ItisimportanttocheckwhethercountriesaremovinginlinewiththeWHOguidelines• TWGarespecialistsindifferentareaswithineducation.Whatishappeningthatwecan

build on? How do we organize ourselves?• Weshouldbethinkingaboutdevelopingacommontheme.

7. Plans to undertake this work effectively

Following discussion on how best to proceed in meeting the objectives and deadlines set out in Section 6, three subgroups were formed in order to take the TWG work forward.

Each Subgroup has formed an Action Plan.

Group 1 will look at the components, conceptual frameworks presented alongside Guidelines Plus and feedback to the TWG by 21 December 2013.

Group 2 will generate a set of questions to interrogate the tools, by the end of January 2014. An associated task is to engage a manager for the review process. Their task would be to do a basic analysis, circulate for comments, synthesize the comments and submit findings to Erica by the end of April 2014.


Group 3 will begin to build an assessment protocol and tool (First teleconference 14 Jan 2014)

A Dropbox will be established by WHO (and managed by Sandra Pandi) into which participants can add examples of other tools that they know about. It was also agreed that other audiences/networks would also be able to contribute to this Tools Dropbox.

Erica and George Zangaro also agreed to commission a literature review of the future health workforce to help guide the work of the TWG.

8. Action plan

It was agreed that this is the stage to move the TWG from where they are today to where they want to be, and that the first task would be to identify the subgroups and agree on who is going to join which group.

DISCUSSIONKey discussion points included:• Noagreementasyetonaconceptualmodel.• Competenceandeducationshouldgoinonebox.• CantheTWGadapttheFIPCompetencyFrameworkModel(seeAnnex9).• Let’sadaptitasacollectivegroup‘inprinciple’nowsothatwehavesomethingtogoonwith.• Itisimportanttorememberthatnotallthecomponentsareonthelist.• Someclarificationisneededasmanyoftheboxesonthismodelactuallyoverlap.How

does research fit into competencies?• Weneedqualityinsurance(thisistheevaluationtool)-itmustfitthismodel.• Wecancoverprogresswithchange.• Theseareimportantcomponents.• Thecircleworkswellasyouidentifypopulationhealthneeds,whatcompetenciesare

necessary to meet those needs, what is the education needed to produce those competencies.

• UHCshouldbeatthecentreofthismodel.• TheFIPmodelCompetencyModelFrameworkreferstoUHCbeingatthecentre–this

is the objective. The evaluation tool needs to fit into this model.• WHA66.23Resolutionstatesthevisionveryclearly.• The2015AgendaisaboutUHC.• TransformativeeducationisabuzzwordandsoUHCshouldnotbeatthecentreofthis

model.• Ourgoalisthetransformationofeducationandsoit’snotimportanthowweplaceiton

the model.• TheFacilitatorproposedthatagroupcouldtakethislist,addtoitandthencirculateitto

the TWG.• EveryonecouldstarttocollecttoolsandputintoaTWGToolsDropbox(Sandratomanage

this).• Itwouldbeusefultolookatfuturehealthworkforceprofessionals.

The Tools Dropbox will allow documents to be accessed, but we need to decide who is going to review these tools.


9. An agreement on next steps

The TWG considered how to best address the tasks at hand in order to meet the strict deadlines set out in the timeline.

DISCUSSIONKey discussion points included:• ItwasagreedthattoolsneedtobeaddedtoaDropbox,followingareviewofthetools,

a set of questions will be generated, circulated for comments, and then submitted to Erica at WHO.

• TheTWGmuststicktothestricttimeline.• ItwouldbebestiftheproposalwasreadybytheendofApril2014.• Aconceptualframeworkdoesn’tfitinhere.CanweincludeGuidelinesPlus?• ShouldthisbeaninternalissueorcanwethrowthenetoutsideofthisTWG?• Weneedtodefinewhatwemeanbyatool.• WHOwillworkonthedeliverablesandtheTermsofReferenceandwillcommissiona

university to do this.• Whataboutagrouptodeveloptoolsandtestthemincountries?Isanothergroupneeded

with input from Group 2?

Summary of day 1Day 2

Agreement

� The tools will have been developed by January 2014. � The tools must be global and not just regional. � Guidelines Plus was identified as being central to the development of the assessment tool. � A set of questions will have been generated by the end of January 2014. � The questions to be completed by the end of April 2014. � Teleconference scheduled for February 2014 (with the subgroups). � A Planned Side Meeting during World Health Assembly19-24 May 2014. � Next TWG meeting 30 June to 3 July 2014 (TBC). � A Tools Dropbox will be established by WHO and managed by Sandra Pandi, and TWG members are encouraged to add examples of tools that they know about. It was also agreed that other audiences/networks would also be able to contribute to this Tools Dropbox.

� WHO will work on the deliverables and the Terms of Reference and will commission a university to do this.

� Lyn Middleton will generate a set of questions – looking at the matrix – start working on the TOR and see if it is necessary to have an intern for help; locate a contact person in country; and develop an operational plan.

� Erica Wheeler and George Zangaro agreed to commission a literature review of the future health workforce to help guide the TWG.


No agreement

� Choice of countries. � The definition of a tool. � Who will actually review the tools in the Tools Dropbox. � Competency Framework. � How to develop a common theme. � What should the future health workforce look like? � The possibility of adapting the FIP Competency Framework Model.

Day 2: Summary of topic

TWG subgroups

Three subgroups were formed in order to organize the work at hand.

Group 1 will look at the components, conceptual frameworks presented alongside Guidelines Plus and present feedback to the TWG by 21 December 2013

Group 2 will generate a set of questions to interrogate the tools, by the end of January 2014. An associated task is to engage a manager for the review process. This will involve carrying out a basic analysis, circulate for comments and synthesize those comments and submit the findings to Erica by the end of April 2014.

Group 3 will begin to build an assessment protocol and tool. (First teleconference to take place on 14 January 2014).

Concept paper subgroup

Following this meeting a Concept Paper subgroup was formed to allow the agreements and decisions following this first TWG meeting to be included in this draft.

Next stepsIt was agreed that

� Teleconference scheduled for February 2014 (with the subgroups) � A Side Meeting during the World Health Assembly (19-24 May 2014) � Next TWG meeting will be 30 June to 3 July 2014 (Lisbon, Portugal)


Concluding remarks1. It was agreed to continue meeting as a main group with members present at this meeting and

other key people.

2. The TWG Tools Database will be expanded via a Dropbox.

3. The 3 subgroups will contact their group members, organize their tasks between themselves and report back to the main TWG and WHO secretariat.

4. The Conceptual Framework was positively received as a potential model for health workforce education assessment.

5. It was agreed that Guidelines Plus is central to the development of assessment tools.

Closing remarks (Dr Ruediger Krech)

The value of the discussions and agreements which took place during this consultation was noted. The participants were congratulated for their hard work during the past two days which was reflected in the progress made and agreements reached. It was agreed that immediate collaboration between the TWG members is required in the form of three subgroups in order to address the goals of data collection, implementation and evaluation being completed between January-June 2015.


Annexes

Annex 1: AgendaTechnical working group on health workforce education assessment tools

Programme

Meeting Purpose:

The purpose of this meeting is to bring members of the Technical Working Group on Health Workforce Assessment Tools together (for the first time) to begin to plan the work of the Group.

By the end of this event participants will have:1. A shared understanding of the terms of reference for the Technical Working Group on Health

Workforce Education Assessment Tools, so that all are clear on what is involved.

2. A shared understanding of the need for, and purpose of, a standard protocol and set of tools for assessment to support Resolution WHA 66.23, so that we can plan accordingly.

3. An agreement on the targeted audience for these tools.

4. An outline agreement on what kinds of components should be part of effective assessment tools

5. A shared understanding of existing tools that may be suitable to assess the health professional education continuum, so that we can begin to identify what is useful and where there are gaps.

6. An agreement on key overall milestones for the Technical Working Group, with timeline.

7. An agreement on how we organize ourselves to undertake this work effectively.

8. An agreement on an action plan – What; How; By Whom; and by When.

9. Agreement on immediate next steps


Technical working group on health workforce education assessment tools

Day 1

Time Session topic Presenter08 :30 Registration

09 :00 Opening Session- Setting the Context Dr Erica Wheeler, Technical Officer, HRH

09 :10 Welcome Address Dr Rüdiger Krech, Director, Office of the Assistant Director-General, Health Systems and Innovation

09 :15 Objectives Implementation of WHO Resolution WHA.66.23

09 :40 Terms of Reference for the Technical Working Group – Background to Resolution WHA.66.23

10 :10-10 :20 Presentation - Health Workforce Education : An Overview of Assessment Tools

Dr Sabiha Essack

10 :20 Input on Resolution WHA.66.23 and its purpose

10 :30-11 :00 Questions followed by feedback

11 :00-11 :15 Coffee Break

11 :15-12 :30 The target audiences for the assessment tools ; consensus-building around the Technical Working Group TOR -

Group discussion

12 :30-13 :30 Lunch Break

13 :30-14 :10 The target audiences for the assessment tools (continued)

14 :15-14 :30 Presentation – Thai Tools Dr Wanicha Chuenkongkaew

14 :30-15 :00 Effective Assessment Tools


15 :15-16 :00 Small Group Discussion –

What makes an effective tool?

16 :00-16 :45 Plenary feedback

Question for clarification

16 :45 Evaluation of Day 1

17 :30 CLOSE


Technical working group on health workforce education assessment toolsS

Day 2

Time Session topic Presenter09 :00 Start up and Review of Day 1

09 :15-10 :00 Input from the group on existing tools and review of existing tools.

Individual input on tools and review of WHO provisional Assessment Tools document

10 :00-11 :00 The number and nature of health workforce assessment tools available – group discussion


11 :15-11 :55 Rapporteur from each group to provide plenary feedback

11 :55-12 :30 Input on overall timeframes

Agree key milestones for the Technical Working Group

Sandra Pandi, Technical Officer, HRH

12 :30-13 :30 Lunch Break

13 :30-14 :45 Proposal from WHO on how to organize the Technical Working Group in order to undertake the work effectively

Small group discussion on WHO proposal

Feedback and agreement on proposal

Dr Erica Wheeler, Technical Officer, HRHSandra Pandi, Technical Officer, HRH


15 :15-16 :00 Participants to choose the subgroup they wish to participate in

16 :00-16 :45 Agreement on an Action Plan –

What ;How ;By Whom ; and by When ;


16 :45-17 :10 Review

Key milestones and Next Steps


17 :10 Evaluation

17 :20 CLOSING COMMENTS Dr Rüdiger Krech, Director, Office of the Assistant Director-General, Health Systems and Innovation

17 :30 CLOSE


ADJASE, EmmanuelDirectorCollege of HealthKintampo, [email protected]

BAILEY, RebeccaTeam Leader Health Workforce DevelopmentCapacityPlus, IntraHealth InternationalChapel Hill, 27517, [email protected]

BARRY, JeanNurse Consultant USAGeneva, [email protected]

BRUNO, Andreia Fradinho Project Coordinator and Researcher International Pharmaceutical Federation (FIP)UCL School of PharmacyLondon, WC1N 1AX, United [email protected]

CHUENKONGKAEW, WanichaVice President for EducationDepartment of OphthalmologyMahidol UniversityBangkok,10400, [email protected]

DARE, LolaChief Executive OfficerCentre for Health Sciences Training, ResearchCentre for Health Sciences Training, Research and Development (CHESTRAD)Ibadan,[email protected]

ESSACK, SabihaDeanSchool of Health SciencesUniversity of KwaZulu-NatalDurban,4000, South [email protected]

GORDON, DavidVisiting ProfessorWorld Federation for Medical EducationUniversity of CopenhagenCopenhagen,[email protected]

HARDEN, Ronald McGlashanGeneral SecretaryAssociation for Medical Education in Europe (AMEE)Dundee,DD2 1LR, United [email protected]

LIU, HuapingSchool of NursingPeking Union Medical CollegeBeijing,100730, [email protected]

MIDDLETON SOLOMON, LynRegional Nursing AdvisorNursing Education Partnership InitiativeColumbia UniversityPietermaritzburg,South [email protected]

MINICLIER COBB, NadiaAssistant Professor (Clinical), Director of Clinical Evaluation/Friday Clinic PlacementUtah Director of Collaboration with the college of Health, Ghana , Department of Family and Preventive MedicineUniversity of Utah Physician Assistant ProgramSalt Lake City,[email protected]

NEUSY, André-JacquesCEO THEnetTraining for Health Equity Network (THEnet)Baisy-Thy,1470, [email protected]

Annex 2: List of participants


PETRINI, MarciaProfessor and DeanWuhan UniversityWuhan,430071, [email protected]

SEWANKAMBO, NelsonPrincipal and Professor, MEPI Principal InvestigatorCollege of Health SciencesMakerere UniversityKampala,[email protected]

STRASSER, RogerDean, Faculty of MedicineNorthern Ontario School of MedicineThunder Bay,ON P7B 5E1, [email protected] SULEMAN, FatimaAssociate Professor, Discipline of Pharmaceutical Sciences,School of Health Sciences,University of Kwazulu-NatalDurban,4000, South Africasulemanf@ ukzn.ac.za

VAN LERBERGHE, WimInstituto de Higiene e Medicina TropicalUniversidade Nova de LisboaLisbon, [email protected]

VILLENA SARMIENTO, RitaPresident Latin American Region (IADR-LAR) 2011-2014International Association for Dental Research (IADR)Lima,18, [email protected]

YOO, Young IlProfessor College of NursingYonsei UniversitySeoul,120-752, South [email protected]

ZANGARO, GeorgeDirectorOffice of Performance ManagementHHS/HRSA/BHPrRockville,20857, United [email protected]

OBSERVER

FISHER, Dr Julian Peter L. Reichertz Institute for Medical InformaticsUniversity of Braunschweig - Institute of Technology and Hannover Medical SchoolHannover,30625, [email protected]

FACILITATOR

JOHNSTON, Mr Stevie Interaction Institute, IISC IrelandBelfast, [email protected]

WHO STAFF

BOERMA, TiesDirector, a.i.,Health Systems Policies and [email protected]

GOIANA DA SILVA, FranciscoInternHealth Systems Policies and [email protected]

NKOWANE, MwansaTechnical OfficerHealth Systems Policies and [email protected]

PANDI, SandraTechnical Officer, Health Systems Policies and [email protected]

WHEELER, EricaTechnical OfficerHealth Systems Policies and [email protected]


Annex 3: WHA resolution 66.23

Sixty-sixth world health assembly WHA66.23

Agenda item 17.3 27 May 2013

Transforming health workforce education in support of universal health coverage

The Sixty-sixth World Health Assembly,

� Recalling resolution WHA59.23 urging Member States to scale up health workforce production in response to the shortages of health workers that hamper the achievement of the internationally agreed health-related development goals, including those contained in the Millennium Declaration;

� Recognizing that a functioning health system with an adequate number and equitable distribution of committed and competent health workers at the primary health care level is fundamental to equitable access to health services as an important objective of universal health coverage, and was highlighted in The world health report 2006;1

� Recognizing also the need to provide adequate and reliable financial and non-financial incentives and an enabling and safe working environment for the retention of health workers in areas where they are most needed, especially in remote, hard-to-reach areas and urban slums, as recommended by WHO global guidelines;2

� Recalling resolution WHA64.9 on sustainable health financing structures and universal coverage, which, inter alia, urged Member States to continue, as appropriate, to invest in and strengthen the health delivery systems, in particular primary health care and services, and adequate human resources for health and health information systems, in order to ensure that all citizens have equitable access to health care and services;

� Concerned that in many countries, notably those in sub-Saharan Africa, there is inadequate capacity to train a sufficient number of health workers to provide the population with adequate service coverage;

� Recognizing the specific challenges of some Member States that have limited economy of scale in local health workforce education, their special needs, and the potential partnerships and collaboration with other Member States;

� Concerned also that the health workforce education challenge is global;

� Concerned further that demographic projections highlight the supply and distribution of the health workforce as issues of concern in the coming decades, irrespective of countries’ development status;

1 The world health report 2006: Working together for health. Geneva, World Health Organization, 2006.

2 Increasing access to health workers in remote and rural areas through improved retention, global policy recommendations, Geneva, World Health Organization, 2010.


� Recognizing also the need for intersectoral collaboration among the Ministry of Health, the Ministry of Education, public and private training institutions, and health professional organizations in strengthening the health workforce education system so as to produce competent health workforces that support universal health coverage;

� Concerned also that many countries lack sufficient financial means, facilities and number of educators to train an adequate, competent health workforce; and that there is a need to improve the health workforce education and training system in response to countries’ health needs;

� Mindful of the need for Member States to develop comprehensive policies and plans on human resources for health, including health workforce education as one of the elements;

� Recalling resolution WHA63.16 on the WHO Global Code of Practice on the International Recruitment of Health Personnel, in which Code, inter alia, Member States agreed to strive to create a sustainable health workforce and establish effective health workforce planning, education and training, as well as retention strategies;3

� Recognizing the Dhaka Declaration on strengthening the health workforce in the countries of the South-East Asia Region and resolution SEA/RC65/R7 adopted by the Regional Committee for South-East Asia on strengthening health workforce education and training in the Region, which urged Member States, inter alia, conduct comprehensive assessments of the current situation of health workforce education and training, based on an agreed regional common protocol, as a foundation for evidence-based policy formulation and implementation;

� Recognizing also the recommendations contained in the Global Independent Commission report on health professionals for a new century: transforming education to strengthen health systems in an interdependent world;4

� Appreciating the ongoing initiatives to strengthen health workforce education and training in various regions; including but not limited to the Medical and Nursing Education Partnership Initiative, in-service training of health workers in sub-Saharan Africa supported by Japan in line with the G8 Hokkaido Tokyo Summit Leaders Declaration, and the Asia Pacific Network for Health Professional Education Reform,

3 Article 3 – Guiding principles, paragraph 3.6.

4 Education of health professionals for the 21st century: a Global Independent Commission. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world, The Lancet, Harvard University Press, Cambridge MA, 2010.


1. URGES Member States:5

(1) to further strengthen policies, strategies and plans as appropriate, through intersectoral policy dialogue among the relevant ministries that may include ministries of education, health and finance, in order to ensure that health workforce education and training contribute to achieving universal health coverage;

(2) to consider conducting comprehensive assessments of the current situation of health workforce education with the application of, as appropriate, standard protocols and tools, once developed by WHO;

(3) to consider formulating and implementing evidence-based policies and strategies, taking into account the findings from the assessment in the previous paragraph, to strengthen and transform the health workforce education and training, including but not limited to the promotion of inter-professional, community-based and health systems-based education, linkages of pre-service education to continuous professional development, and an accreditation system to ensure quality of training institutes and competency of health workforces; with a view to better responding to the health needs of people, taking into account the special needs of some Member States that have limited economy of scale in local training;

(4) to provide adequate resources and political support for the implementation of policies and strategies as appropriate for the strengthening and transformation of health workforce education;

(5) to share best practices and experiences on health workforce education;

2. REQUESTS the Director-General:

(1) to develop a standard protocol and tool for assessment, which may be adapted to country context;

(2) to support Member States as appropriate in using the protocol to conduct comprehensive assessments of the current situation of health workforce education;

(3) to provide technical support to Member States in formulating and implementing evidence- based policies and strategies in order to strengthen and transform their health workforce education;

(4) to consult regionally in order to review the country assessment findings and submit a report providing clear conclusions and recommendations, through the Executive Board, to the Sixty-ninth World Health Assembly;

(5) to develop, based on the report, global and regional approaches, which may include strategies to transform health workforce education, submitting these, through the Executive Board, for consideration by the Seventieth World Health Assembly.

Ninth plenary meeting, 27 May 2013A66/VR/9

5 And, where applicable, regional economic integration organizations.


Annex 4: Assessment tools for health workforce education (draft)

Type of Tool Link

Supervised Learning Events (SLEs) and In-Programme Assessments

http://depts.washington.edu/uwrhrc/uploads/RHRC_PB146-5.pdf

Educational Supervisors https://www.eastmidlandsdeanery.nhs.uk/page.php?id=1713

Skills for Health workforce assessment tools http://www.alphaplusconsultancy.co.uk/alphaplus-case-studies/skills-for-health-workforce-assessment-toolshttp://www.skillsforhealth.org.uk/component/docman/doc_view/1869-public-health-skills-career-framework-03-2009.html

Support workforce - tools and resources http://www.nhsemployers.org/planningyourworkforce/supportworkforce/toolsandresources/pages/toolsandresources.aspx

Health Manager’s Toolkit http://www.msh.org/resources/health-managers-toolkit

Assessment of practitioners in mental health Best Practices for assessing competence and performance of the behavioral workforce

http://www.bbs.ca.gov/pdf/mhsa/resource/workforce/assessing_behavioral_health_wkforce_competency.pdf

Career Pathways InitiativeSix Key Elements Readiness Assessment Tool

http://www.workforceinfodb.org/PDF/CareerPathwaysToolkit2011.pdf

Educational Institution and Workforce Intermediary Self-Assessment: Organizational Readiness for Implementing a Work-Based Learning Or Career Pathways Program

http://www.jff.org/sites/default/files/EducationalInstituion_WorkforceIntermediarySelfAssess_081610.pdf

Selected Health Workforce Tools from CapacityPlus

http://www.capacityplus.org/files/resources/selected-health-workforce-tools-from-capacityplus.pdf

NHS Competency Assessment Tool (for NHS users only)

http://systems.hscic.gov.uk/icd/informspec/p3m/resource/development/assessment

Performance – a guide and toolkit for health worker training and education programs

http://www.intrahealth.org/lfp/overview.html

Jhpiego’s Approach to Pre-Service Education (an affiliate of Johns Hopkins University)

http://www.jhpiego.org/files/%20ERCO-new-infosheets/3-How-We-Do-It/3Education%20and%20Training/Jhpiego's%20approach%20to%20PSE.pdf

HRH K4 pre-service and in-service toolkitsThis eToolkit is meant to serve as a resource for those dealing with aspects of the HRH crisis, including individuals and organizations who wish to familiarize themselves with the various components of HRH.

http://www.k4health.org/toolkits/hrh

PHORCaST – Public Health Online Resource For Careers, Skills and Training

Public Health Skills and Career Frameworkhttp://www.phorcast.org.uk/page.php?page_id=44

A guide to rapid assessment of human resources for health, World Health Organization, 2004

http://www.who.int/hrh/tools/en/Rapid_Assessment_guide.pdf


Annex 5: Terms of reference for the technical working group on health workforce education assessment tools

Background

At present, the World Health Organization estimates that additional 2.4 million doctors, nurses and midwives are needed worldwide. The consequence of such an estimate is that access to health is considerably compromised. Today, over a billion people worldwide lack access to quality health services-in large part because of a huge shortage, imbalanced skill mix, and uneven geographical distribution of professionally qualified health workers such as doctors, nurses and midwives.

Worldwide, 2420 medical schools, 467 schools or departments of public health, and an indeterminate number of postsecondary nursing educational institutions train about 1 million new doctors, nurses, midwives, and public health professionals each year. Four countries (China, India, Brazil and the USA) each have more than 150 medical schools, whereas there are 36 countries which have none at all. 26 countries in sub-Saharan Africa have only one or no medical schools. With such stark imbalances, it is not surprising that medical school numbers reflect neither the country population nor the burden of disease.

Insufficient collaboration between the health and education sectors, as well as weak links between educational institutions and the health systems which employ graduates, often result in a mismatch between professional education and the realities of health service delivery. These factors limit the capacity of even those highly-qualified personnel to improve health outcomes. Fundamental global reforms are needed to increase the numbers of health professionals and to strengthen their impact on population health.

With little information about health professional education, the need for mechanisms to assess the competence and performance of the behavioural health workforce has continued to receive more and more attention.

It is generally accepted that simply adding more qualified health workers into the mix will have little impact on the burden of disease in countries. The real situation is much more complex. With insufficient numbers of health workers and many health workers not having a sufficiently broad training, a failure to provide universal health coverage would mean the internationally agreed health-related goals in addition to those included in the Millennium Declaration are unlikely to be achieved.

During this year’s World Health Assembly, Resolution WHA66.23 was passed, whereby WHO was tasked with the responsibility of developing a standard protocol and health workforce education assessment tool.

In view of the historical problem of not only a global health workforce shortage, but an urgent need to ensure that such a health workforce has a broader training which more accurately reflects their everyday working practices, The World Health Assembly Resolution 66.23 urges Member States to:

� Further strengthen policies, strategies and plans, as appropriate, through intersectoral policy dialogue among the relevant ministries that may include ministries of education, health and finance,


in order to ensure that health workforce education and training contribute towards achieving universal health coverage;

� To consider conducting comprehensive assessments of the current situation of health workforce education with the application of, as appropriate, standard protocols and tools, once developed by WHO

� To consider formulating and implementing evidence-based policies and strategies, taking into account the findings from the assessment in the previous paragraph, to strengthen and transform the health workforce education and training, including but not limited to the promotion of inter-professional, community-based and health systems-based education, linkages of pre-service education to continuous professional development, and an accreditation system to ensure quality of training institutes and competency of health workforces; with a view to better responding to the health needs of people, taking into account the special needs of some Member States that have limited economy of scale in local training;

� To provide adequate resources and political support for the implementation of policies and strategies as appropriate for the strengthening and transformation of health workforce education;

� To share best practices and experiences on health workforce education.

Objectives of technical working group

The main objective of the Terms of Reference for the Technical Working Group on Health Workforce Education Assessment Tools, is the implementation of WHA66.23.

This tool is to be reviewed by the Technical Working Group, in order to come up with the most suitable instruments for undertaking health workforce education assessment.

WHO is tasked with the following: � To develop a standard protocol and tool for assessment, which may be adapted to country context; � To support Member States as appropriate in using the protocol to conduct comprehensive assessments of the current situation in health workforce education

� To provide technical support to Member States in formulating and implementing evidence-based policies and strategies in order to strengthen and transform their health workforce education

� To consult regionally in order to review the country assessment findings and submit a report providing clear conclusions and recommendations, through the Executive Board, for consideration by the Seventieth World Health Assembly.

Discussions the Technical Working Group will cover:

1. What are the main topics/areas to be covered as part of health workforce education to ensure that the assessments to be undertaken to strengthen HRH policies, strategies, plans and practice1 will be considered comprehensive ?

2. What existing or new tools (as above) may be/are needed to contribute to this comprehensive assessment, if any ?

3. What tools are most suitable to assess the current situation of health workforce education at the institutional and programmatic2 level ?

1 Regarding the experience and guiding examples of health professionals assessments, the TWG may wish to consider including ‘practice’ as well, to ensure larger relevance and impact of any ‘plans’ that would ensue for transformative education.

2 both ‘institutional and programmatic’ included in view of the standard methodology for (a) academic ranking systems and (b) assessments of international scientific and higher education proposals for funding (calls-for-proposals).


4. What tools exist to undertake institutional and programmatic assessment ? What new tools are needed if any ?

5. What tools are most suitable to assess issues of motivation/aspirations/choice of career/specialization/incentive on entry into pre-service and other health professionals3 education?

6. What tools exist to undertake motivation/aspiration/choice of career/specialization/incentive assessment on entry into pre-service and other health professionals’ education? What new tools are needed if any ?

7. What kinds of tools are most suitable to track health professionals after graduation in terms of their pre-service and other career choices/aspirations/location/specialization and incentives ?

8. What tools exist to track health professionals after graduation in terms of their pre-service and other career choices/aspirations/location/ specialization and incentives ?

9. What new tools are needed, if any ?

10. Which tools are problematic?

Scope of work

The Technical Working Group on Health Workforce Education Assessment Tools will guide the process of developing these assessment tools. Recommendations for moving forward need to be based on evidence, hence the need for experts to identify, discuss and analyze the appropriateness of existing tools and/or the need to develop new ones. These tools will span national, institutional, prequalification and graduate levels as well as encompass graduates who have entered the labour market. It will therefore be required to:

� undertake discussions and provide feedback on the appropriateness of existing tools via electronic means and teleconferences, with availability to have face-to-face meetings either one or twice per year.

� provide written advice on the appropriateness of existing tools � commission documents or tools as required � oversee the development of any new tools required � develop a protocol for use of the assessment tools

The group is expected to function initially for a period of two years which can be extended to a third year as required.

Composition of the technical working group on health workforce education assessment tools

The multidisciplinary group will include representatives from all WHO regions composed of: health education specialists, senior academics and persons responsible for governance of education institutions, policy makers, experts in health systems and international health and civil society representation. Some members of the group also have expertise in research and health economics.

The members of the group will act in their own name, and will not necessarily represent the views of the organizations they belong to. A declaration of interest will be provided by each member of the core group at the beginning of the consultative process.

3 “other” would refer to graduate health professionals’ and medical education: GME selection/motivation/incentives/etc. increasingly seems to play a major ‘systems’ role in (a) retention aspects, (b) quality, quantity and scope of development of the health professions, and (c) decision-making power as to the under-graduate and pre-service educational standards and practice.


The following process and timeline for the next 9 months was agreed

Meetings:

DECEMBER 4-5 First Technical Working Group on Health Workforce Education Assessment Tool meeting

FEBRUARY 2014 Teleconference with each of the TWG subgroups (TBD)MAY 2014 Side meeting World Health Assembly (19-24 May)30 JUNE -3JULY 2014 Technical Working Group on Health Workforce Education Assessment Tool

meeting (Lisbon, Portugal)

Timeline:

BY MID-2014 We should have agreed which countries to test the tools inOCTOBER 2014 Begin country testingNOVEMBER 2015 The report must be readyJANUARY 2016 Present the report to the WHO Executive BoardMAY 2016 Present the report to the World Health Assembly


Annex 6: Technical working group subgroups

Three subgroups were formed in order to organize the work at hand.

Group 1: will look at the components, conceptual frameworks presented alongside Guidelines Plus and present feedback to the TWG by 21 December 2013

Group 2: will generate a set of questions to interrogate the tools, by the end of January 2014. An associated task is to engage a manager for the review process. This will involve carrying out a basic analysis, circulate for comments and synthesize those comments and submit the findings to Erica by the end of April 2014.

Group 3: will begin to build an assessment protocol and tool. (First teleconference to take place on 14 January 2014).


Annex 7: Asia Pacific Network on Health Professional Education Reform (ANHER)

Annex 1 National Level assessment

Draft as of October 6, 2011

Objectives

To conduct assessment, at the national level, on the health professional education in the current country context

Contents1

1. National basic indicators � Burden of disease [BOD] � Economic, demographic structure and trend

» Gross National Income[GNI] per capita and trend, » Population, urban, rural » Poverty incidence

� Health resources indicators » Physical resources [public /private hospital and hospital beds, basic health unit] » Human resources : Physician, nurse, by public /private,

� Health services utilization indicators » Outpatient[OP] visit per capita per year » Admission per capita per year » Antenatal care[ANC] coverage » Expanded Program on Immunization[EPI] coverage e.g. DPT3

� Inequity gap [whatever parameters available] » Inequity in income

° IncomeratiobetweenQ5andQ1 ° GINI index

» Inequity in health status ° Malnutrition (stunting, wasting), concentration index [CI] ° Low birth weight, concentration index ° Other indicator of policy makers concerns

� Health financing indicators » Total Health Expenditure[THE] per capita, % Gross Domestic Product[GDP] » General Government Health Expenditure[GGHE], % General Government Expenditure [GGE] » GGHE, % THE

2. National policy for higher education in health � The top five policies in the last decades � Communication and intersectoral coordination among ministries especially ministry of education [MOE] and ministry of health [MOH]

1 Each country should keep the content of following questions in the survey, however, their formats and sequences are adjustable.


3. Assessment of demand for HRH � Projection of demands for physicians and nurses

4. Assessment of supply of HRH � Trend of production capacities and future trends : Nurse, Doctor

5. Accreditation Institutions � Accrediting agencies: legality, present, function � Accredited institutes and trends, nursing, nurse, � Criteria and mechanisms

6. National standards for curriculum

7. Innovative education training policies and outcome

8. Soliciting the perspectives and viewpoints from � Senior peers in public /private, urban / rural hospitals on graduate’s clinical competencies, management skills, communication skill, inter-professional skills, being public mind and good attitudes and ethics [Focus group discussion ; FGD]

� Policy makers from MOH on doctors and nurses contribution to health development, relevance, and overall competencies [Interview and FGD]

� Poll survey to assess public view on satisfaction in health service2

Methods

1. Literature reviews

2. Synthesis of secondary data where data available

3. In-depth interviews and series of focus group discussion with key stakeholders3

4. Poll survey

2 Poll survey may not be applied if the country deems inappropriate or infeasible to conduct

3 The questions below are slightly complex and need accurate information from the key informants who involved in health professional education in the country. It is suggested that a several rounds of focus group discussion with 5-7 key informants each round to discuss and assess the situation. In the focus group discussion, key informants from three constituencies may be invited, (a) the health professional education institutes, (b) policy makers and (c) research institutes. Note that the spill-over effect of FGD is the entry point for the reform of health professional education when concerned stakeholders participate at the beginning.


Part 1: national context

I. National basic indicators 45

2000 2005 2010 Reference

Health needs

• Burden of diseases, Total Disability-adjusted life year(DALY) loss or per 1000 population4

• Please specify top five of BOD

» 1st rank

» 2nd rank

» 3rd rank

» 4th rank

» 5th rank

• Elderly population, >65 years, % total

Economic, demographic structure and trend

• Population, million

• Rural population, % total

• GNI per capita, PPP$ and trend

• Poverty Head count Ratio at National Poverty Line (% pop)

Health resource indicators

Physical resources [public /private hospital and hospital beds, basic health centres5]

• Total hospitals

• % hospitals in private

• Total hospital bed

• % hospitals bed in private

• Number of basic health centres

HRH indicators

• Total medical doctors

• % in private

• Total nurse+ personnel

• % in private

• Doctor per 1,000 pop: urban

• Doctor per 1,000 pop: rural

• Doctor per 1,000 pop : total

• Nurse per 1,000 pop: urban

• Nurse per 1,000 pop: rural

• Nurse per 1,000 pop : total

4 This is a country specific question. If time-series data are not available, please provide us the most updated data.

5 Basic health center defines as public health facilities which have lower capacity than hospital. In Thailand, basic health center is called “primary health center.”


2000 2005 2010 Reference

• Midwife per 1,000 pop: urban

• Midwife per 1,000 pop: rural

• Midwife per 1,000 pop : total

Health services utilization indicators

• OP visit per capita per annum

• Admissions per capita per annum

• ANC coverage

• % institutional delivery

• % DPT3 coverage

• % MCV [Measles contained vaccine]at ninth month

Inequity gap6

• Inequity income

» GINI Index

» Income ratio Q5/Q1

• Inequity in health status

» Malnutrition, concentration index

» Low birth weight, concentration index

» Other indicator of policy makers concern

Health financing indicators

• THE per capita, % GDP

• GGHE, % GGE

• GGHE, % THE6

Part 2: medical doctor education

II. National policies and strategies

Policy and strategies

1. Does your country have a national strategy or plan for medical education7?

[ ] No [ ] Yes, what years

Excerpts of major content of the strategic plan: e.g. stakeholder involved in the draft,

1.1 Does the plan cover public and private schools? [ ] No [ ] Yes

6 Country focal person may add other inequity indicators according to their policy makers’ concerns.

7 Please provide the information on a national strategy in the last decade or more.


1.2 Description of the strategic plan ° Goals

° Objectives

° Major content

° Targets

° Timeline of the strategic

2. Were there assessments of the plans?

[ ] No [ ] Yes, what are the outcomes? [ ] service coverage [ ] utilization

Effective coordination: MOE and MOH

Describe and assess 3. Is there a national forum or mechanism?

[ ] No [ ] Yes, if yes please specify [ ] regular [ ] non-regular [ ] official [ ] non-official

4. Is there an interactive dialogue between Ministry of Health, Ministry of Education, Medical Schools, and general public on the assessment of medical education in the country? Or is there the emerging health needs of the population, determinants of ill-health, and contributions by medical schools to health systems development?

[ ] No [ ] Yes, if yes please specify


Who involves in the forum? Who set and what are the key agenda?

If there are firm commitments, what is the follow up and outcome?

HRH Information systems

Focus Group Discussion5. Who are the key stakeholders: (central office, institute of public health, MOH, other)

6. Please describe the existing national HRH information systems

7. Who is responsible in producing national data that is important to HRH development, for example; national population, house hold survey, planning and monitoring. What is its reliability, timely production for decision making?

8. A critical assessment of the adequacy of HIS information for decision making and recommendations for improvement [data (content, capacity and practices, dissemination, integration and use), collection, analysis, reporting of relevant indicators]

III. Demand for HRH

Literature review (published and grey documents) on quantifying, projecting the future demands for different cadres of health workforces or health professionals


9. What methods are applied? such as health needs, service model, population ratio approaches

IV. Supply of HRH: an inventory

10. Number of medical schools in the last 10 years where data is available?

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

Number of medical schools

• Government owned medical schools

• Non-government owned medical schools

11. Portfolios of medical schools, 2010 where data available

Total number of schools

Medical schools

Public Private

1. Geographical location

• Located in the capital city

• Located outside capital city8

2. Having special program on recruitment of student from rural, or under-served area

3. Offering post-graduate medical training8

12. Medical students enrolled and graduated between 2001 and 2010

Student enrolled9 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010



Student graduated



9

8 China is grouped into eastern, middle and western regions. Hence, China respondent please report the number of schools broken down into regions.

9 Only undergraduate medical students.


V. National accreditation and standard

13. Describe if there are accreditation agencies ensuring quality and standards of medical curriculum and standard of education (The office for national education standard and quality assessment, The office of the higher education commission)

14. What are the national agencies in licensing of training institutes and certification of curriculum, legality of these agencies (Medical council, Royal college)

15. Describe the objectives, processes of the certification of curriculum, how often it needs for re-certification, when was the latest revision and what was the major change?

16. Describe the objectives, processes of licensing of training institutes, what are the criteria used? (quality assurance)

17. Discuss and analyze different tools applied by different medical schools for the measurement of competency of medical graduates [OSCE, other competency based examination].

VI. National standards for curriculum 18. Literature review of national/international framework for curriculum evaluation (content, sequence

of course, curricular element, balance between core and optional content) and gold standard for each component of the framework. Please discuss and analyze whether they are credible, functional, up-to-date or taken seriously.


VII. Innovative policies on education and training

19. Innovative policies on recruitment: implementation status and outcome

Provide evidence and description box 1-7 help support stakeholders in reaching consensus on the level of implementation and outcome of these innovative policies

Analysis of policy 7 modules by FGD

BOX 1 TARGET ENROLMENT OF MEDICAL STUDENTS FROM RURAL AREAS

Description on background how the policy emerged and evolved, major policy contents, is it applied to public and private medical schools, how the policy was supported [financial, non-financial incentives], major barriers in implementing such policies, the proportion of targeted admission to the total admissions and what are the trend over the last ten years, assessment of this policy innovation, outcome of policy in term of rural retention.

BOX 2 MEDICAL SCHOOLS OUTSIDE CAPITAL CITY10

Analysis why and how this policy emerged, who involved in pushing / resisting this policy, what is the outcome of such policy implementation? Strengths and challenges and potential solutions?

10

BOX 3 CLINICAL ROTATION TO RURAL COMMUNITY

Analysis why and how this policy emerges, who involved in supporting and not supporting this policy, what are the outcomes of implementation? Strengths and challenges and potential solutions?

BOX 4 REVISION OF CURRICULA

Analysis why and how this policy emerges, who involved in supporting and not supporting this policy, what are the outcomes of implementation? Strengths and challenges and potential solutions?

10 This topic may not be China’s interest. Hence respondent please analyze policy on other related topic for example policy on establishing or distributing medical schools in different regions etc.


BOX 5 CONTINUED MEDICAL EDUCATION

Analysis why and how this policy emerges, who involved in supporting and not supporting this policy, who implement the continued professional education problems, what are sources of finance in supporting this program, what are the outcome of implementation? Is the continued professional education a mandatory or voluntary scheme? What are incentives of up-taking continued medical education, strengths and challenges and potential solutions?

BOX 6 SCHOLARSHIP IN RETURN OF GOVERNMENT LOAN OR BONDING

Analysis why and how this policy emerges, who involved in supporting and not supporting this policy, who implement the subsidy program, what are source of finance in supporting this program, what are the outcome of implementation? Is it a voluntary scheme or apply to all? What are up-taking of subsidies, how to enforce the returning of service in rural remote areas by graduates, strengths and challenges and potential solutions?

BOX 7 SUBSIDIES TO BOOST NUMBER OF SELECTED SPECIALTIES

Analysis why and how this policy emerges, who involved in supporting and not supporting this policy, who implement the subsidy program, what are source of finance in supporting this program, what are the outcome of implementation? Is it a voluntary scheme or apply to all in that specialty? What are up-taking of subsidies, how to enforce the returning of service by post-graduates, strengths and challenges and potential solutions?

BOX 8 OTHER TYPES OF INNOVATIONS SUCH AS CURRICULAR INNOVATION AROUND PROBLEM-BASED LEARNING, INTER-PROFESSIONAL EDUCATION, INNOVATIVE USE OF TRAINING AIDS ETC.

Please describe the content of this policy. Analysis why and how this policy emerges, who involved in supporting and not supporting this policy, who implement the subsidy program, what are source of finance in supporting this program, what are the outcome of implementation? Is it a voluntary scheme or apply to all in that specialty? What are up-taking of subsidies, how to enforce the returning of service by post-graduates, strengths and challenges and potential solutions?


Based on discussion with stakeholders who involved in the implementation of these innovations in series of FGD, try to reach consensus on the level of implementation and the outcome using score 1 to 5, from least to most

20. Please score the policies above in term of implementation and outcome11

Implementation status [score 1-5]

Outcome of implementation [score 1-5]

<very limited--full implement>

NA

<Poor--excellent outcome>

NA1 2 3 4 5 1 2 3 4 5

1. Targeted admission: enrol students from rural districts into medical schools

2. Medical schools locate outside major cities or affiliate with others outside capital cities

3. Rotations to rural community during medical education

4. Revision of curricula reflecting rural health problems of the country

5. Continue medical education program meets the needs of rural doctors, accessible from where they work

6. Scholarship or other education subsidies for return to services in rural remote areas once graduated

7. Scholarships or other education subsidies or encourage for certain specialty post-graduate specialist training

11 This is a parameter for within country comparison rather than across country comparison.


21. Describe successful policies in the last five year in your country

VIII. Soliciting the perspectives and viewpoints

� Senior peers in public /private, urban / rural hospitals on graduate’s clinical competencies, management skills, communication skill, inter-professional skills, being public mind and good attitudes and ethics [FGD]

� Policy makers, PCMO, MOH on doctors and nurses contribution to health development, relevance, supply and shortfalls relative to needs and expectations, and overall competencies linked to supply assessment [Interview and FGD]

� Poll survey to assess public view on satisfaction on health professionals’ competencies and responsiveness

Part 3: nurse and midwifery

IX. National policies and strategies

Policy and strategies

22. Does your country have a National Strategy or Plan for nursing education’?

[ ] No [ ] Yes, what years

Excerpts of major content of the strategic plan: e.g. stakeholder involved in the draft,

22.1 Does the plan cover public and private schools? [ ] No [ ] Yes

22.2 Description of the strategic plan ° Goals

° Objectives

° Major content


° Targets

° Timeline of the strategic

23. Were there assessments of the plans?

[ ] No [ ] Yes, what are the outcomes? [ ] service coverage [ ] utilization

Effective coordination: MOE and MOH

Describe and assess

24. Is there a national forum or mechanism?

[ ] No [ ] Yes, if yes please specify [ ] regular [ ] non-regular [ ] official [ ] non-official

25. Is there an interactive dialogue between Ministry of Health, Ministry of Education, Nursing Schools, and general public on the assessment of nursing education in the country? Or is there the emerging health needs of the population, determinants of ill-health, and contributions by nursing schools to health systems development?

[ ] No [ ] Yes, if yes please specify

Who involves in the forum?

Who set and what are the key agenda?

If there are firm commitments, what is the follow up and outcome?


HRH Information systems

Focus Group Discussion

26. Who are the key stakeholders: (central office, institute of public health, MOH, other)

27. Please describe the existing national HRH information systems

28. Who is responsible in producing national data that is important to HRH development, for example; national population, house hold survey, planning and monitoring. What is its reliability, timely production for decision making?

29. A critical assessment of the adequacy of HIS information for decision making and recommendations for improvement [data (content, capacity and practices, dissemination, integration and use), collection, analysis, reporting of relevant indicators]

X. Demand for HRH

Literature review (published and grey documents) on quantifying, projecting the future demands for different cadres of health workforces or health professionals,

30. What methods are applied? such as health needs, service model, population ratio approaches

XI. Supply of HRH: an inventory

31. Number of nursing schools in the last 10 years where data is available?

Number of nursing schools 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

• Government owned nursing schools


Number of nursing schools 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

• Non-government owned nursing schools

32. Portfolios of nursing schools, 2010

Total number of schools

Nursing schools

Public Private

1. Geographical location

» Located in the capital city

» Located outside capital city

2. Having special program on recruitment of student from rural, or under-served area

3. Offering post-graduate nursing training

33. Nursing students enrolled and graduated between 2001 and 2010

Student enrolled 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010



Student graduated



XII. National accreditation and standard

34. Describe if there are accreditation agencies ensuring quality and standards of nursing curriculum and standard of education (The office for national education standard and quality assessment, The office of the higher education commission)

35. What are the national agencies in licensing of training institutes and certification of curriculum, legality of these agencies (Nursing council, Royal college)

36. Describe the objectives, processes of the certification of curriculum, how often it needs for re-


certification, when was the latest revision and what was the major change?

37. Describe the objectives, processes of licensing of training institutes, what are the criteria used? (quality assurance)

38. Discuss and analyze different tools applied by different nursing schools for the measurement of competency of nursing graduates [OSCE, other competency based examination].

XIII. National standards for curriculum

39. Literature review of national/international framework for curriculum evaluation (content, sequence of course, curricular element, balance between core and optional content) and gold standard for each component of the framework. Please discuss and analyze whether they are credible, functional, up-to-date or taken seriously.

XIV. Innovative policies on education and training

40. Innovative policies on recruitment: implementation status and outcome

Provide evidence and description box 1-7 help support stakeholders in reaching consensus on the level of implementation and outcome of these innovative policies

Analysis of policy 7 modules by FGD

BOX 1 TARGET ENROLMENT OF NURSING STUDENTS FROM RURAL AREAS

Description on background how the policy emerged and evolved, major policy contents, is it applied to public and private nursing schools, how the policy was supported [financial, non-financial incentives], major barriers in implementing such policies, the proportion of targeted admission to the total admissions and what are the trend over the last ten years, assessment of this policy innovation, outcome of policy in term of rural retention.


BOX 2 NURSING SCHOOLS OUTSIDE CAPITAL CITY

Analysis why and how this policy emerged, who involved in pushing / resisting this policy, what is the outcome of such policy implementation? Strengths and challenges and potential solutions

BOX 3 CLINICAL ROTATION TO RURAL COMMUNITY

Analysis why and how this policy emerges, who involved in supporting and not supporting this policy, what are the outcomes of implementation? Strengths and challenges and potential solutions

BOX 4 CONTINUED NURSING EDUCATION

Analysis why and how this policy emerges, who involved in supporting and not supporting this policy, who implement the continued professional education problems, what are sources of finance in supporting this program, what are the outcome of implementation? Is the continued professional education a mandatory or voluntary scheme? What are incentives of up-taking continued nursing education, strengths and challenges and potential solutions?

BOX 5 SCHOLARSHIP IN RETURN OF GOVERNMENT BONDING



BOX 6 SUBSIDIES TO BOOST NUMBER OF SELECTED SPECIALTIES


BOX 7 OTHER TYPES OF INNOVATIONS SUCH AS CURRICULAR INNOVATION AROUND PROBLEM-BASED LEARNING, INTER-PROFESSIONAL EDUCATION, INNOVATIVE USE OF TRAINING AIDS ETC.

Please describe the content of this policy. Analysis why and how this policy emerges, who involved in supporting and not supporting this policy, who implement the subsidy program, what are source of finance in supporting this program, what are the outcome of implementation? Is it a voluntary scheme or apply to all in that specialty? What are up-taking of subsidies, how to enforce the returning of service by post-graduates, strengths and challenges and potential solutions?

Based on discussion with stakeholders who involved in the implementation of these innovations in series of FGD, try to reach consensus on the level of implementation and the outcome using score 1 to 5, from least to most

41. Please score the policies above in term of implementation and outcome




NA


NA1 2 3 4 5 1 2 3 4 5

1. Targeted admission: enrol students from rural districts into medical schools

2. Medical schools locate outside major cities or affiliate with others outside capital cities

3. Rotations to rural community during medical education





NA


NA1 2 3 4 5 1 2 3 4 5

4. Revision of curricula reflecting rural health problems of the country

5. Continue medical education program meets the needs of rural doctors, accessible from where they work

6. Scholarship or other education subsidies for return to services in rural remote areas once graduated

7. Scholarships or other education subsidies or encourage for certain specialty post-graduate specialist training

42. Describe successful policies in the last five year in your country

XV. Soliciting the perspectives and viewpoints

� Senior peers in public /private, urban / rural hospitals on graduate’s clinical competencies, management skills, communication skill, inter-professional skills, being public mind and good attitudes and ethics [FGD]

� Policy makers, PCMO, MOH on doctors and nurses contribution to health development, relevance, supply and shortfalls relative to needs and expectations, and overall competencies linked to supply assessment [Interview and FGD]

� Poll survey to assess public view on satisfaction on health professionals’ competencies and responsiveness


Annex 2 Institutional level assessmentDraft October 6, 2011

Objectives

To assess the school capacities in term of physical, financial, curriculum, teaching methods and teaching staffs

Methods

This is a participatory joint assessment by key partners in each medical school and outside researchers. Various methods were applied including reviews of available data, in-depth interviews and focus group discussion with key informants12.

Scope of assessment depends on how large the total number of medical schools, if small, then census is useful, if very large, multi-stage random sampling or purposive sampling would be considered in view of resource and survey capacity.

Key questions

These questions should be adjusted according to the country context, while keep several core questions for cross country comparisons.

Scope of study

The scope of study varies on the country context and the feasibility of assessment. The main scope was discussed in the 5-country situation analysis workshop on “Health Professional Education for a New Century” during August, 15th -17th, 2011, Khon Kaen, Thailand.

Scope of Study for the situation analysis on “Health Professional Education for a new century”13

Doctors Nurses and midwives Public Health workers

Degree

Minimum study years/ terminology Degree


Minimum study years/ terminology

Bangladesh Bachelor 12+5+114 / Medical degree [MBBS]15

Bachelor and Diploma

• 12+4/BSc 16

Nursing• 10[12]+3+1/

Diploma Nurse-midwife

Master • MBBS+1 • Graduates+1+1

China Bachelor 12+5+1/ Medical degree [MD]17


• 12+4/ BSc Nurse

• 12+3/ Diploma Nurse

BachelorMaster

• 12+4/ BPH • Graduates +3

12 The questions are complex and require accurate information from the key informants who involved in health professional education in the country. It is suggested that a several rounds of focus group discussion (FGD) with 5-7 key informants each round to discuss and assess the situation. In the focus group discussion, key informants from three constituencies may be invited, (a) the health professional education institutes, (b) policy makers and (c) research institutes. Note that the spill-over effect of FGD is the entry point for the reform of health professional education when concerned stakeholders participate at the beginning.

13 Referring to minutes of 5-country situation analysis workshop on “Health Professional Education for a New Century” during August, 15th -17th, 2011, Khon Kaen, Thailand.


Doctors Nurses and midwives Public Health workers

Degree



Minimum study years/ terminology

India Bachelor 12+5.5/ Medical degree [MBBS]


• 12+3.5 or 4/ BSc Nurse [colleges or institutions]

• 12+3/ General nursing and midwifery

Master • Graduates +2

Thailand Bachelor 12+6 / Medical degree [MD]

Bachelor • 12+4/ Bachelor Nurse and Midwifery

BachelorMaster

• 12+4/ BPH,BSc in Public health

• 12+4 +[1 or 2] for MPH

• MSc in public health19

Vietnam Bachelor 12+6 / Medical degree [MD]


• 12+4/ Bachelor nurse

• 12+3/ Diploma nurse

BachelorMaster

• 12+4/ BPH• Graduates +2/

MPH

Japan Bachelor 12+6 / Medical degree [MD]

Bachelor • 12+3/ College nurse

• 12+3+1/ Nurse and midwife or public health nurse

• 12+4/ University nurse and midwife

Master • MD +1• Graduates +2

1415 16 17 18 19

Background information

Name of school or institute Town or City Country Survey respondent name Position E-mail Phone

14 Only for internship

15 Bachelor of medicine, Bachelor of surgery

16 Bachelor of science

17 Doctor of medicine

18 Bachelor of public health

19 These includes Bachelor of Public Health , Master of Public Health (MPH), Bachelor of Science, Master of Science (MSc), Master of Philosophy regarding public-health or public-health related education e.g. epidemiology, health economics, environmental health, occupational health, health informatics etc. The minimum study period varies across faculties.


I. Institutional Governance

To answer the following questions, please indicate x in the box [ ] or write down your reply in the appropriate space.

1. Authority and ownership

1.1 Status of the medical school [ ] affiliated to a university [ ] affiliated to an academic institute of health sciences [ ] autonomous [ ] affiliated to the central government [ ] affiliated to the local government [ ] others; please specify

1.2 The ownership of the medical school [ ] public [ ] private; non for profit [ ] private; for profit

2. Governing board and partnerships

2.1 Description of a governing body or school governing board which steers the policy and direction of the school, please attach reference if any. ° Who appoint the board or key administrators? ° Please give details about term of office, mandates and terms of reference, regularly of

meeting, key agendas. ° Has the governing board a real power of directing policy of the school?

[ ] No [ ] Yes; please give details to what extent of the directing power

° Is the governing board composed of representatives from different constituencies? [ ] No [ ] Yes; please give details of whom they represent and their numbers [ ] ministry of health persons [ ] ministry of education persons [ ] professional organization persons [ ] health care providers persons [ ] community organization persons [ ] graduate/alumni persons [ ] associated partner institutes persons [ ] civil society persons [ ] consumer groups persons [ ] others; please specify persons

2.2 Does your school have partnership/affiliation with other schools for under-graduate training? [ ] No [ ] Yes; if yes, please answer the following questions


° Please describe the objectives of such partnership ° Please indicate the ownership of the partners and describe the extent and duration of each

partner (The table can be extended aligning with the number of your school’s partners.)

Name of partners

Ownership of the partner

Extent of partnership and program activity

Duration [years]

Medical schoolHealth care

provider

Public Private Public Private

1.

2.

3.

4.

5.

° Please describe the outcomes of such partnership or collaborations

2.3 Please describe the strategies on institutional development by governing board or school leaders in terms of ° Vision and mission ° Strategic planning

3. Special concern for undergraduate medical curriculum and admission criteria

Does your school concern or have the policy on the following aspects?

Not any Yes, please specify

1. Admission preference by

• Gender

• Geography

• Socio-economic status

• Ethnicity

• Religion

2. Curricular review

3. Retention through to graduation


II. Educational Services

A. Curriculum Analysis

4. Curricular content

4.1 Please describe the historical evolution of the undergraduate medical program, how it evolves in the last 10 years, and the philosophy of the curriculum.

4.2 In the undergraduate medical curriculum, have the following topics been covered and in what form?

Not included

Yes, in a distinctive

courseYes, integrated in a course,

please specify

1. Health policy

2. Health management

3. Communication skill

4. Leadership

5. Public mind and voluntarism

6. Professional ethics and

7. Social determinants of health and diseases

8. Humanities and social justice

9. Health equity

10. Evidenced-based practice

11. Research methodology and critical thinking

12. Comparative national health systems

13. Information and technology [IT]

14. Second language

4.3 When was the recent update of your school’s curriculum and how frequent is it regularly updated? ° Last update in (year) ° Update frequency: every years


4.4 Please describe the mechanism of update/ review

5. Curriculum Method

5.1 Please describe the primary and secondary languages used in the instruction, text book and handouts, written and oral examinations ° Primary language: ° Secondary language:

5.2 Please indicate how frequent an intra-professional20 team-based learning is integrated in basic courses21 and practice courses22?

Level of integration23 Not at all Occasionally Frequently Extensively

Basic courses

Practice courses

23

5.3 Please indicate how frequent an inter-professional education24 is integrated in basic courses and practice courses?

Level of integration Not at all Occasionally Frequently Extensively

Basic courses

Practice courses

5.4 Please indicate how frequent problem-based learning (PBL25) is applied in basic courses and practice courses?

Level of integration Not at all Occasionally Frequently Extensively

Basic courses

Practice courses

20 It is an instructional approach aimed for collaborative work of students in the same professional.

21 Basic Courses refer to courses taught based on lectures and seminars without practice part.

22 Practice Courses refer to practicum, including clinical rotation

23 Note on level of integration• Occasionally:<10%oftotalwork/studycoursesassignedorpreparedtostudents• Frequently:10-20%oftotalwork/studycoursesassignedorpreparedtostudents• Extensively:>20%oftotalwork/studycoursesassignedorpreparedtostudents

24 Inter-professional education occurs when two or more professions learn about, from and/with each other to enable effective collaboration and improve health outcomes, [World Health Organization, 2010. Framework for action on inter-professional education and collaborative practice. Geneva: WHO Press.]

25 PBL is an instructional approach which uses priority health problems that practitioners are likely to encounter in their professional lives, as stimuli for learning. This approach emphasizes active self-directed learning by students individually or in small groups.


5.5 Are students required to undertake research project either in team or individual in order to graduate?

[ ] No [ ] Yes; please describe the objectives and arrangement of such program

5.6 Please order the three main characteristics of the curriculum approach by writing down “1” for the approach applied for majority of the curriculum and “2” and “3” for the second and third main approaches respectively

[ ] Lecture/ seminar-based learning [ ] Community-based learning [ ] Problem-based learning [ ] System-based learning [ ] Competency-based learning [ ] Others; please specify

5.7 Please give the percentage of time students need to spend on the following study modules26 ° Classroom and laboratory studies % ° Clinical practice % ° Community practice .% ° Self study % ° Others; please specify %

5.8 Please indicate if there are off-site exposure opportunities for students mandated by the curriculum

[ ] No [ ] Yes, please specify [ ] Local communities [ ] Rural hospitals [ ] Overseas experience [ ] Organization in related fields; specify [ ] Others; specify

5.9 Please describe if any innovative and effective mix of channels used such as, didactic faculty lectures, small student learning groups, early patient or population exposure, different worksite training bases, longitudinal relationship with patients and communities etc.

5.10 Is there any mechanism to review or update the curriculum teaching method? [ ] No [ ] Yes, please specify

B. Evaluation of Core Competencies of Students

6. Student assessment

6.1 Please indicate how students’ competencies are evaluated (may answer more than one) []MultipleChoiceQuestions(MCQ) []ModifiedEssayQuestions(MEQ)

26 This is an overall estimation throughout the medical education course. Respondents may provide in detail information by breaking down into different year of study if data are available.


[ ] Essay exam [ ] Objective Structured Clinical Examination (OSCE27) [ ] Oral exam, term paper [ ] Long case exam [ ] Close book exam [ ] Open book exam [ ] Others, please specify

6.2 Describe if any other innovative approach used for student competency e.g. interdisciplinary working competency, evidence-based practice, continuous quality improvement, use of new informatics, integration of public health, ethical principles, gender sensitivity etc.

7. Students

7.1 Based on in-depth interview or focus group discussion, and reviews of available data, and according to individual medical school context, please indicate the number of students admitted by different admission criteria in the last decade

Year

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010Criteria

National entrance examination

Previous academic performance

Special quota for talent

Community work hours

Special quota for students from under-served areas

Selection by local community

Others, please specify

7.2 Please compare attrition and completion rate of students by different types of admission criteria in the last years or any recent years if data available ° Attrition rate ° Graduate completion rate

7.3 For the way forwards, having discussed and analyzed different type of admission, what should be improved, continued, scale up and scale down and why?

27 OSCE stands for Objective Structured Clinical Examination, an assessment method that is based on objective testing and direct observation of student performance during planned clinical encounters (also called interactions or test stations). Studies have demonstrated that the OSCE is an effective tool for evaluating areas most critical to performance of health care professionals: the ability to obtain information from a patient, establish rapport and communicate, and interpret data and solve problems.


7.4 Please give the number of students admitted, completing study and passing national license examination28 between 1994 and 2004, where data are available29.

Admission year Total admissions Completing medical study

Passing national license exam

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

28 National license exam varies according to the country. For example, Thai 6th year medical students have to attend national license examination in order to complete their medical courses.

29 In this table, respondent has to provide the total number of students admitted and completing medical study in the indicated years. Nevertheless, if any faculties have more exhaustive information such as the cohort of students completing study and passing national license exam sorted by each batch, please kindly indicate herewith.


III. Workforce of Teachers

8. Availability and profiles

8.1 Please give the number of staffs30 in different age groups, stratifies by gender and their qualification in the latest year31

Age group

Gender Qualification

Male Female Bachelor Master Doctoral

Post-gradspecialist training

<30 yr

30-34 yr

35-39 yr

40-44 yr

45-49 yr

50-54 yr

55-59 yr

60-64 yr

>65 yr

Total

2004

8.2 Where data allows, please give the number of part time teaching staffs32 in the latest year, as percentage of total staffs ° Percent of part time staff %

8.3 Please discuss and analyze the trend of recruiting part-time staffs and also the advantages and disadvantages of such trend

30 “Staffs” in this question refers to those who are faculty members and does not comprise all employees.

31 Retrieve any available personnel data available, and protect confidentiality of personal information

32 Since definition of part time staffs or affiliated staffs varies across institutes and countries. Please provide reference or clear definition of part time and affiliated staffs in your country.


8.4 Does your school apply a formal appointment of health service provider and service user as adjunct faculty or teaching staff?

[ ] No [ ] Yes

9. Staff recruitment

9.1 Where data available, please compare the number of full time versus part time staff recruitment based on minimum qualifications required e.g. Bachelor, master, doctoral, postgraduate specialist training etc. for clinical and pre clinical teaching or each specific department where data available.

9.2 Are there any criteria applied for staff recruitment? [ ] No [ ] Yes, please specify [ ] rural experience [ ] secondary language fluency [ ] doctorate degree or equivalent [ ] outstanding student’s profile [ ] other criteria, please specify

10. Staff retention

Does your school have the following strategies on staff retention? If yes, please describe the detail of those strategies and also their effectiveness?

� Financial incentives [ ] No [ ] Yes ; please give the detail

� Performance based payment [ ] No [ ] Yes ; please give the detail

� Non-financial incentives such as social recognition, career advancement, professorship, social recognition, rewards etc.

[ ] No [ ] Yes ; please give the detail

� Scholarship for continuing education [ ] No [ ] Yes ; please give the detail


11. Outcome of staff retention

11.1 Review data available in the last five years, please give the number of full time teaching staffs and the turn-over rate in the past five years

Year Total recruitment

Total separation Total staffs at the end of the yearAttrition Retirement

2006

2007

2008

2009

2010

11.2 Review data available in the last five years, please give the number of full time teaching staffs and the turn-over rate in the past five years ° What are main reasons of separation such as lack of financial incentives, job un-satisfaction,

heavy workload, family reason, personal reasons, etc ? ° Where did they go, such as private hospital, continue training abroad, leave the medical

practice?

11.3 What is the retirement age of your school’s staff?

11.4 What is the policy to extend staff’s retirement? Please describe detail of the policy as well as its strengths and weaknesses

12. Continuous staff development

Please describe the continuous staff development program, its nature (e.g. mandatory or voluntary) and training profile (such as public health, medical education, social science, behavioral science etc.) [ ] mandatory development program, please describe training profiles 1. 2. 3. 4. 5.

[ ] voluntary 1. 2. 3. 4. 5.


13. Performance evaluation mechanism

Please describe if the school applies a formal evaluation mechanism for the staff performance regarding;

� technical knowledge [ ] No [ ] Yes ;please specify

� pedagogic competency [ ] No [ ] Yes ;please specify

IV. Financing

14. Financing of your institution

14.1 Generally, these are sources of revenue for health professional schools; ministry of education, ministry of health, local government, tuition fees, clinical services, domestic research grants, international research grants, domestic philanthropic donation, international philanthropic donation, etc. Please indicate the three main revenue sources of your school and estimate the proportions of those main sources to the total revenue?

1. % 2. % 3. %

14.2 Do you have the information of unit cost of medical production in your institution? [ ] No [ ] Yes, please describe

° How much does it cost? ° How frequent it is regularly updated? ° When was it recently updated?

14.3 After discussion with key informants in the medical school, please describe and solicit comments on financial situation in terms of strengths, weaknesses, sustainability, and future prospects


V. Infrastructure and Technology

15. School facilities

Please score 1-5 from non-existence to highly adequate33

<Non-existence------Highly adequate>

1 2 3 4 5

1. Building

2. Library and

» Library and information support

» Inter-library services

» teaching facilities

» Classrooms

» Teaching labs

3. Laboratory

4. IT facilities and services

» Computers per student

» Internet service

» Conference call technology

» Video conference technology

» Telemedicine/ tele-radiology link

5. Training on use of IT

» Library search training course

» Computer skill lab

» Other IT training course specify

6. Field sites

33 The scoring is either based on individual institutional standard, criteria, quality assurance system or external peer review or joint assessment. It is a parameter aimed to compare institutional physical facilities within individual countries rather than across countries.


<Non-existence------Highly adequate>

1 2 3 4 5

7. Learning materials

8. Accommodation for students , transport,

9. Internal transportation for students

10. Amenities e.g. cafeteria, convenient stores, sport and recreational facilities

16. Electronic learning (E-learning)

Are there applications of E-learning34 in your school?[ ] No [ ] Yes, please specify what modalities are used?

[ ] Web-supplemented course : courses focus on classroom-based teaching but include elements such as putting a course outline and lecture notes on line, use of e-mail and links to online resources.

[ ] Web-dependent course: courses require students to use the Internet for key elements of the program such as online discussions, assessment, or online project/collaborative work, but without significant reduction in classroom time.

[ ] Mixed mode courses: the e-learning element replace part of classroom time. Online discussions, assessment, or project/collaborative work replace some face-to-face teaching and learning. But significant campus attendance remains part of the mix.

[ ] Fully online course: students can follow courses offered by a university in one city from another town, country or time zone.

[ ] Others, please specify

VI. Information for Policy Making

17. Existing mechanism on interactive dialogue for policy making

Is there any either, regular or non-regular, official or non-official, interactive dialogue between medical school and the following institutes in order to meet the emerging health needs of the population, determinants of ill-health, and health systems development?[ ] No[ ] Yes, please indicate institutes to which your school has interactive dialogue and describe the detail of dialogue and assess its effectiveness [ ] Ministry of education, please give detail [ ] Ministry of health, please give detail [ ] local government, please give detail [ ] general public, please give detail

34 The use of information and communications technology to enhance and/or support learning in tertiary education, which covers a wide range of systems, from students using e-mail and accessing course work on line while following a course on campus to programs offered entirely online.


VII. Quality assurance in education

18. Does the school have its own quality assurance unit?

[ ] No ; if no, please skip question No. 19 and answer question No. 20[ ] Yes ; please answer question No. 19

19. Please describe the quality assurance unit in terms of

� Its organizing structure � Its resources used e.g. staffs, budgets or equipments � How it functions

20. Has the school ever performed any external evaluation?

[ ] No [ ] Yes, please describe

» Who were the external reviewers? » How frequent it is regularly updated and when was the most recent update?

24. What are the indicators used for quality assurance/improvement of the school?

26.What would be the strategies/policies and plans of quality assurance or improvement of the school in the next 5 years?


Annex 3 Graduate surveys Draft October 6, 2011

Objectives

1. To assess the relationship between socio-economic background and types of health services employment [public, private, service, teaching, administration] from the whole range of employment history including transitions and reasons for transitions.

2. To conduct a self assessment by graduates on attitudes in providing health services in rural, hardship, remote or areas where the poor and vulnerable population live, knowledge, skill and competencies

Methods

Two sampling methods First, to solicit views from the all new graduates who are about to leave the schools; it is easy to identify them. A self-administrative survey can be conducted in class with almost 100% responses rate. This sample provides good information about the young graduates and found useful for policy reorientation. There is an opportunity to assess the curriculum and training modality as they have fresh experiences. See Tool 1.

Second, to solicit views from the graduates who are employed in health service sector; samples can be identified with the application of multiple stratified sampling technique. For example, one may stratify provinces / areas into three groups of rich, moderate and poor as first strata, then randomly select provinces from these areas. In the selected province, all public and private doctors/ nurses are the samples for questionnaire survey. Sample size depend very much the capacity and resources to launch such a survey. As a nature of retrospective cohort study, this approach provides hard evidence on job transition probability between public and private and between rural and urban in relation to their socio-economic and personal background. Job transition probability and survival in professional career is vital parameter for long term project and human resource supply dynamics. See Tool 2.

Country may take first or second or both methods, depends on policy demand and capacity to conduct surveys. Second method is complex and requires efforts to ensure high response rate. However, opportunistic surveys in large gatherings or meetings of samples can be applied. Furthermore, one must be careful to prevent selection bias, by non-representation in such gathering.


Tool 1 Graduates about to leave schools

A self-administered survey questionnaire is launched to all graduates who are about to leave the school. Tool should be tested, verify its reliability, and validity for a final version. A class room type of self-administration questionnaire survey can be applied easily, without intervention by surveyors. Collection of questionnaire at the end of the class would ensure almost 100% response rate.

Key questions

Questionswouldbeadjustedaccordingtothecountrycontext,whilekeepseveralcorequestionsfor cross country comparisons.

1. About yourself

» Gender [ ] male [ ] female » Age [ ] » When you were 1-15 years, where did you spend most of your life?

° District ° province

» Where was your high school located? ° District ° province

» You were recruited into medical school through what mechanism [design according to your country context]

° national entrance exam, ° direct admission, ° special quota ° others

2. About your parents

» Where do they live now, ° District ° Province

» Main occupation of your father, [choice according to the country context] » Main occupation of your mother [choice according to the country context] » Highest education level of your father[choice according to the country context] » Highest education level of your mother [choice according to the country context]

3. Attitudes towards rural, remote or hardship areas


<Disagree ----------Agree>

1 2 3 4 5

1. Work in these areas provides opportunities to use various skills

2. There are supportive environment when working in these areas

3. Work in these areas limits communications with professional peers

4. Work in these areas provides opportunities to work independently

5. There are lack of amenities and entertainment in these areas

6. People in these areas are friendly

7. Work in these areas results in “isolation” from friend and family

8. Work as medical doctor in hospitals in these areas is the most important contribution to health of population

9. Medical school prepared me well to work in these areas

10. Medical education inspires me to work in hospitals in these areas

11. Work in hospitals in these areas is most challenging

12. Work in hospital in these areas provide opportunities for real-life problem solving

4. Job preference after graduate and reasons

4.1 You are about to graduate, where are you intending to work

4.2 Reasons [review literature on job attributes: financial incentive, indirect benefit, housing, work environment, career advancement, social recognition, their own conception of social accountability etc.]

4.3 Intention to stay or leave by 3 years, reasons

Where to go, public /private, urban/rural, hometown preference, for further training


5. Competency self-assessment

<Least confident -------- most confident>

1 2 3 4 5

1. Public health services

2. Health administration

3. Communication with community and professional peers

4. inter-professional collaboration

5. Managing internal medicines patients

6. Managing obstetric and gynecology patients

7. Managing pediatric patients

8. Managing surgical patients

9. Managing general patients such as DM, hypertension, HIV/AIDS

10. Managing difficult labour and delivery patients

11. Referring patients to upper level

12. Overall clinical competency

13. Overall public health competency

Tool 2 Survey of in-service doctors/nurses

A self-administered survey questionnaire is launched to all in-service doctors or nurses in sample area or province can be conducted through opportunistic survey when there is a gathering, workshop or conference, where most of the samples would be attending. This is a context specific that country researcher has to ensure the representativeness of sample; otherwise a conventional mail survey would be applied, though the responses rate is expected to be as low as 10-20%. A context specific solution can be devised such as a coordinator in each province to follow up and collect back questionnaire.


Key questions

Questionswouldbeadjustedaccordingtothecountrycontext,whilekeepseveralcorequestionsfor cross country comparisons.

1. About yourself

� Gender [ ] male [ ] female � Year of birth [ ] � When you were 1-15 years, where did you spend most of your life?

» District » province

� Where was your high school located? » District » province

� You were recruited into medical school through what mechanism [design according to your country context]

» national entrance exam, » direct admission, » special quota » others

2. About your parents

� Where do they live now, » District » province

� Main occupation of your father � Main occupation of your mother � Highest education level of your father � Highest education level of your mother


3. Attitudes towards rural, remote or hardship areas

<Disagree –----------Agree>

1 2 3 4 5

1. Work in these areas provides opportunities to use various skills

2. There are supportive environment when working in these areas

3. Work in these areas limits communications with professional peers

4. Work in these areas provides opportunities to work independently

5. There are lack of amenities and entertainment in these areas

6. People in these areas are friendly

7. Work in these areas results in “isolation” from friend and family

8. Work as medical doctor in hospitals in these areas is the most important contribution to health of population

9. Medical school prepared me well to work in these areas

10. Medical education inspires me to work in hospitals in these areas

11. Work in hospitals in these areas is most challenging

12. Work in hospital in these areas provide opportunities for real-life problem solving

4. Your employment and transitions

Please provide detail history of your employment from year one to the current year.

Year of your graduation from (name of school)

Your current employment, Name of hospital , District , Province

Your employment details from graduation to current year35

35 Respondent can add more tables if they have more than four major job transitions


First Second

Work place

Was the indicated work place situated in rural or urban area?

Hospital name,

• District,

• This district is your hometown? Check box Y/N

• Province

• Type, public or private

Duration of employment

• from month/year to month/year

Nature of work

Full time/ part time

Average working hour per week, hours

Your responsibilities, %

• Medical services

• Administration

• Public health function

• Research

• Teaching

Total

Your satisfaction here

• Overall job satisfaction score 1-5 [1 least satisfied, 5 most satisfied]

• Satisfaction on salary, score 1-5

• Satisfaction on additional financial incentive, score 1-5


First Second

• Satisfaction on housing benefit, score 1-5

• Opportunity for career advancement, score 1-5

• Opportunity for in-service training, or continued education, score 1-5

• Adequacy of equipment, and facilities, score 1-5

• Safe work environment, score 1-5

• Satisfaction on workload, score 1-5

• If you leave this employment to the second employment, please describe three most important reasons for your leaving

Third Fourth

Work place

Was the indicated work place situated in rural or urban area?

Hospital name,

• District,

• This district is your hometown? Check box Y/N

• Province

• Type, public or private

Duration of employment

• from month/year to month/year

Nature of work

Full time/ part time

Average working hour per week, hours

Your responsibilities, %

• Medical services


Third Fourth

• Administration

• Public health function

• Research

• Teaching

Total

Your satisfaction here

• Overall job satisfaction score 1-5 [1 least satisfied, 5 most satisfied]

• Satisfaction on salary, score 1-5

• Satisfaction on additional financial incentive, score 1-5

• Satisfaction on housing benefit, score 1-5

• Opportunity for career advancement, score 1-5

• Opportunity for in-service training, or continued education, score 1-5

• Adequacy of equipment, and facilities, score 1-5

• Safe work environment, score 1-5

• Satisfaction on workload, score 1-5

• If you leave this employment to the second employment, please describe three most important reasons for your leaving

After graduation, have you ever planned to work outside your own country?[ ] No[ ] Yes ; if yes please answer the following questions

� Which country do you plan to move to? � Which organization do you plan to work for? � Why do you plan to work outside your country? Or what are the major drives?

Before going to section 5, please describe three major challenges in your current work 1. 2. 3.


� Indicate the name of city or country that you planned to move to

� Indicate the name if the workplace you planned to work

� Do you plan to remain in the clinical field?[ ] Yes[ ] No ; if no please indicate the field of your future work

5. Job intention for next 3 years

What do you plan for your work for the next three years, say in 2015 [choose one] � Still work at the same place that you are working today � Plan to move [choose one]

[ ] Move to a public provider in the same province, [ ] Move to a public provider in another province, [ ] Move to a private provider in the same province, [ ] Move to a private provider in another province, [ ] Move for continued post-graduate study

� Quitfromprofessionalcareer

6. Competency self assessment for those who graduated after 2006 for valid assessment, for those graduated before 2006, please leave this section

For medical graduates

< Least confident --------most confident >

1 2 3 4 5

1. Public health services

2. Health administration

3. Communication with community and professional peers

4. inter-professional collaboration

5. Managing internal medicines patients

6. Managing obstetric and gynecology patients

7. Managing pediatric patients

8. Managing surgical patients

9. Managing general patients such as DM, hypertension, HIV/AIDS



1 2 3 4 5

10. Managing difficult labour and delivery patients

11. Referring patients to upper level

12. Overall clinical competency

13. Overall public health competency

Please list three competencies which you consider the most competencies for medicine graduates1. 2. 3.

For nursing graduates

For public health graduates


1 2 3 4 5

1. Describe the scope, role and functions of public health in relation to the heath system and other social sectors

2. Demonstrate professional judgment and ethical standards in addressing a wide range of health issues and quality assurance

3. Apply qualitative and quantitative methods in the conduct of public health research, policy and practice

4. Analyze, interpret, and communicate effectively existing and emerging public health issues, priorities, and trends

5. Design intervention to prevent and control communicable, non-communicable diseases, injuries and other health-related concerns based on sound risk management principles

6. Promote healthy life style policy and intervention

7. Plan, manage and evaluate public health programs and services



1 2 3 4 5

8. Contribute to formulation, implementation and advocacy of evidence-based health policy including legislation, regulation and financial measures

9. Demonstrate capacity to engage, mobilize and create partnership with communities in diverse social and cultural situations

10. Demonstrate capacity for leading and participating in effective team effort

11. Contribute to the development of human resources for public health

12. Apply appropriate information technology and computer skills effectively

13. Demonstrate capacity for continuing life-long learning and professional development

Please list three competencies which you consider as key competencies for public health graduates

6.1 Please comment if the curriculum and training in the school help equip you to a real-life clinical work? Or the in-service training which supports you in improving yourself for these works?

6.2 Comment if the curriculum is relevant to health needs of the population you are serving today? What is the major lacking?

6.3 Did you participate in Continued Professional Education in the past five years, how frequent and did you find them useful? What are your recommendations to improve the CPE?


Annex 8: Presentations by members of the Technical Working Group on Health Workforce Education Assessment ToolsPresentation by Professor Sabiha Essack

B. Pharm., M. Pharm., PhD, Dean: School of Health SciencesChair: South African Committee of Health Sciences DeansHealth Workforce Education: An Overview of Assessment Tools

UKZN INSPIRING GREATNESS

Health Workforc e E duc ation A s s es s ment T ools :

A n Overview


Health Workforc e E duc ation As s es s ment

Quantity

Quality & Relevance

Curriculum

Resources & Funding

Accreditation & Regulation

Planning & Governance


This presentation set out to explain components of health workforce assessment in terms of quantity, quality and relevance, curriculum, resources & funding, accreditation and regulation, planning & governance.

Presentation by Professor Wanicha Chuenkongkaew

Vice President for Education,Mahidol University,Thailand

This presentation explained showed how the development of assessment tools initially began with research in three countries in Asia: Thailand, China and India, in 2011. Asia Pacific Network on Health Professional Education Reform (ANHER)

Goals

� To develop and strengthen the Asian regional network of health professional training institutes as a platform for collaboration on knowledge synthesis and evidence generation for health professional education reforms in response to changing health determinants and health systems development in countries.

“ANHER Tool Development”

� August 2011 – Khon Kaen,Thailand » Protocol Development Meeting

� November 2011 – Cebu,Philippines » Protocol Finalization Meeting


Quality & RelevanceL ine of E nquiry E xis ting Tools /E videnc e

Does the existing/first entry health workforce have the necessary:• Competencies,• Knowledge & technical skills, and,• Values, attitudes and behavioursto address health systems needs?

• Jhpiego &/or HRH K4 Pre-Service & In-Service Toolkits

• PHORCaST: Public Health Skills and Career Framework

• NHS: Competency Assessment Tool• Competency Frameworks e.g. CANMEDS,

FIPEd etc.• Scopes of Practice• IntraHealth International: Learning for

Performance• Pre-registration examinations• Bashook PG. Best practices for assessing

competence & performance of the behavioural health workforce. Administration in Mental Health. 2005; 32:563-592

Is the existing/first entry health workforce able to function across the continuum of healthcare from health promotion and disease prevention to diagnostic, therapeutic & curative care to rehabilitative & palliative care?

Does the existing/first entry health workforce have the necessary graduate attributes to address health systems needs?

• Morgan JHC. Designing an assessment tool for professional attributes of medical graduates from a new medical school in Nepal. South East Asian Journal of Medical Education. 2009; 3: 2-7


� April 2012 – New Delhi, India » Presentation of National Level Findings

� November 2012 – Beijing,China » Presentation of Interim findings

� December 2012 – AAAH meeting, Bangkok,Thailand » Planning for cross – country analyses

� January 2013 – PMAC meeting, Bangkok, Thailand » Coding for cross – country analyses » Innovations and case success stories

° Rural retention ° National licensing

� June 2013 – Hanoi, VN » Plan for cross – country analyses » Plan for common protocol development of country’s Innovations

� November 2013 – Dhaka, Bangladesh » Finalize the cross – country report

� January 2014 – PMAC » Finalize cross – country publication » Plan for cross – country manuscript of country’s innovation

Asia Pacific Network on Health Professional Education Reform (ANHER)

� Since late 2011, a research team in each country conducted the SA, � Common protocol and tool development based on literature review � The scope of the study confined in medical education, nursing and public health and plan to cover other professions in further studies.

� The National survey, Institutional survey and Graduate survey, both the last year students and in–service doctors and nurses

� Both quantitative and qualitative methods were applied � The study was started in April 2012 followed the same model for each country in 5Cs network. � The Kick–off andData Collection Training Workshop of study was organized


Presentation by Dr Emmanuel Adjase

Director, College Of Health, Kintampo ,Ghana On Mid-Level Providers


Presentation by Dr Andreia Bruno

Project Coordinator and Researcher International Pharmaceutical Federation (FIP), UCL School of Pharmacy, London,WC1N 1AX, United Kingdom

This needs-based educational model is applicable to all health workers and and not just those working in the area of pharmacy and is consensus-driven. It focuses on four clusters/areas,

� Needs � Services � Competencies � Education

and has been developed as a quality assurance tool .There will be more information about the Competency Framework in August/September 2014.

International Pharmaceutical Federation (FIP) | Tools

Dr Andreia BrunoFIPE d Project Coordinator and Researcher

WHO Technical Working Group5.xii.13


Needs-Based Educational Model

Global Competency Framework | GbCFVersion 1


Guidance on use…

• Starting point, or mapping tool (development of practice and practitioner development).

• Guidance for foundation level practice (“early years”).

• Translations for ‘what’ and ‘how’ students can interact with pharmaceutical care skills during initial education.

• A map of core outcomes for initial education.

• By creating a portfolio, in synergy with other assessment tools, countries can use the GbCF for workforce development.

(the G bC F v1 does not imply that there s hould be a ‘s ingle’ global curriculum that would fit all countries ).

Global Competency Framework v1 Concept


…Users

Snippets in the FIPE d Global Education Report | pages 37 to 39.

Examples of impact:• Ireland – Cora Nestor• Univers ity of Wyoming, S c hool of P harmac y – Linda

Gore Martin• C roatia, B os nia and Herzegovina, Monte Negro and

Mac edonia – Arijana Mestrovic• S erbia – Dusanka Krajnovic• P ac ific Is land C ountries – Andrew Brown• PAHO/WHO Initiative

7

Development of Irish Competency Framework


PSI: Core Competency Framework

A Global Framework for Quality Assurance of Pharmacy Education | Version 1


2013 FIPE d 2013 Global Education Report

educ ation@ fip.orgwww.fip.org/educ ation

www.fip.org/educ ationreportswww.fip.org/pe_res ourc es


Presentation by David Gordon

Visiting Professor, World Federation for Medical Education, University of Copenhagen, Copenhagen, Denmark

Presentation: The future of medical education quoting the mission of WFME -Standards and Accreditation http://www.wfme.org/

This presentation explained how the consideration of future needs was written into the standards of WME and stated that basic medical educational standards include a page on how a medical school must adapt for the future.

Accreditation of medical education is of the highest importance to WFME and they have a programme for accreditation to ensure that medical schools are at the right standard.

Recognition of Accreditors

‘How to ensure that accreditation agencies are using the right standards, and are working in the right way? WFME is not an accrediting authority or agency. Accreditation of medical education is normally carried out by national governments, or by national agencies receiving their authority from government. It is important that all accreditation processes are working to internationally recognised standards.

The objective of the WFME programmes about accreditation has been to create a transparent and rigorous method of ensuring that accreditation of medical schools, world-wide, is always at an internationally accepted and high standard. To meet this objective, WFME has worked with ECFMG, and other experts, to develop a policy and a new process for evaluation of accreditation agencies’.

Standards

The WFME programme on definition of international standards in medical education was launched in 1997. The purpose was to provide a mechanism for quality improvement in medical education, in a global context, to be applied by institutions, organisations and national authorities responsible for medical education.

The original Trilogy of WFME Global Standards was developed by three international task forces with broad representation of experts in medical education from all six WHO - WFME Regions, and published in 2003.

Since 2004, the Standards programme has been promoted by the WHO/WFME Partnership to Improve Medical Education.

WFME Global Standards have been used in self-evaluation, peer review and other reform processes in several hundreds of medical schools, and used as a template for national and regional standards and for recognition and accreditation procedures in more than 60 countries.


European Specifications of the WFME Global Standards were developed by a joint WFME – Association of Medical Schools in Europe (AMSE) task force, under the MEDINE Thematic Network, and published in 2007.

In 2010, a joint task force of WFME, AMSE and ORPHEUS (the Organisation for PhD Education in Biomedicine and Health Sciences in the European System) began work to develop the ORPHEUS position paper “Towards Standards for PhD Education in Biomedicine and Health Sciences” into agreed standards for the PhD degree in biomedicine.


Presentation by Andre-Jacques Neusy

CEO, Training for Health Equity Network –THEnet

1

Our questions in 2007:What is the role of health professions schools in addressing health equity and the health workforce crisis?

Who is trying, are they successful and if so how?

We found schools asking this:How can we as a school help get enough of the right people, with the right competencies to work where they are most needed?

Common goal: how do we measure success?

2


THEnet SCHOOLS3

Training for Health Equity Network

• Ateneo de Zamboanga University School of Medicine (Philippines)• The School of Health Sciences in Leyte, University of the Philippines

Manila (Philippines)• The Faculty of Health Sciences, Walter Sisulu University (South Africa)• Flinders University School of Medicine (Australia)• James Cook University Faculty of Medicine, Health and Molecular

Science (Australia)• The Latin American Medical School ELAM (Cuba)• Northern Ontario School of Medicine (Canada)• Gezira University Faculty of Medicine (Sudan)• Ghent University Faculty of Medicine and Health Sciences (Belgium)• Patan Academy of Health Sciences (Nepal)• The University of New Mexico Health Sciences Center (United States)


5

Health equity oriented governance and policies Targeted recruitment from disadvantaged communities Relevant competency-driven curriculum design & delivery Service learning in context Symbiosis with health system Community Engagement hardwired into all activities Relevant research Measuring performance based on impact on health &

health inequities Mentoring approaches through community-based

practitioners

Common Principle - Different Contexts

• Focus on outcomes and impact

• Move from ivory towers to community settings

• Alignment across different professions

• Linked to health system and other sectors

• Focus on local needs

• Engaged with communities

• Focused on primary care and prevention

6

Trans forming educ ation to meet needs


THEnet’s SOCIAL ACCOUNTABILITY OPERATIONAL MODEL

7

Performance improvement tool - not a pass/fail exercise

helps schools take a critical look at performance and progress towards greater impact through social accountability

assists in establishing priority areas for research and improvement

identifies key factors affecting a school’s ability to positively influence health outcomes and health systems performance

THEnet Evaluation Framework


Key Component Aspirations

1. How does our school work?

1.1 What do we believe in? We uphold and demonstrate shared values of social accountability as defined by the Training for Health Equity Network. [equity, relevance, quality, efficiency, partnerships]

1.2 Whom do we serve and what are their needs? What our health system’s needs?

We recognize and define the population we serve, with particular reference to underserved populations and hold ourselves accountable to meet these needs. We plan to address the priority health and workforce needs of our reference populations and health systems. We are active contributors to the health system of which we are a part and play a role in advocacy and reform. Our particular emphasis is on increasing the provision of and access to comprehensive Primary Health Care (PHC) and addressing the social determinants of health.

1.3 How do we work with others?

We operate in partnership with all relevant stakeholders, with a primary focus on the priority health needs and social needs of our target populations. Our partnerships reflect our genuine commitment to meaningful collaboration with communities, health services and health care providers.

1.4 How do we make decisions?

Our strategic decision-making involves meaningful participation from all stakeholders.


2. What do we do? (1:2)

2.1 How do we manage our resources?

We allocate resources to operationalize our plans for community engagement and delivery of the program in communities where there is the greatest need for the provision of high quality health services. We encourage reciprocal contributions among the school, community and other stakeholders

2.2 What, where and how do we teach?

We have an education program that reflects the priority health, workforce and social needs of the communities we serve, as defined by community partnerships. This is evident within curriculum, pedagogy, and teaching sites. Professional roles and responsibilities reflect current and projected health and workforce needs.

2.3 Whom do we teach? We enrol and support students who reflect the socio-demographic characteristics of our reference population, especially underserved populations..



2. What do we do? (2:2)

2.4 Who does the teaching?

We recruit and support educators and other staff who: - reflect the demographics of our reference population; - reflect the balance of clinical, biomedical and social sciences; - demonstrate commitment to SA principles.

We engage and support community and community health service providers as educators in a manner which strengthens local health services.

2.5 How does our research program relate to our mission and values?

We have a research agenda that reflects priority health and health system needs of our reference population, developed and undertaken in partnership with key stakeholders with a focus on participatory methodologies

2.6 What contribution do we make to delivery of health care?

Educators and students are involved in service delivery related to changing priority health needs of reference populations and reflective of future working environments.

Educators and students are involved in community development.


3. What difference do we make? (1:2)

3.1 Where are our graduates and what are they doing?

We produce graduates technically, socially and culturally suited to address the health and social needs of reference populations and health system. This will be reflected in geographical location, career choice, and professional behaviour.

Our graduates particularly engage in comprehensive primary health care, (addressing the social determinants of health) and broader advocacy and reform.

We are involved in the continuum of medical education and support alumni beyond graduation to promote access, quality and efficiency of health care

3.2 What difference have we made to our reference population and reference health systems?

We have a positive impact on priority health and social needs of our reference population.

In partnership with stakeholders we contribute to the transformation of health systems to be more relevant to the health needs of our reference populations.

We are recognized agents of positive change by our reference population/partners and stakeholders



3. What difference do we make? (2:2)

3.3 How have we shared our ideas and influenced others?

We are engaged in a continuous process of critical reflection and analysis with others and disseminate these learning’s in many ways. We influence policymakers, education providers and other stakeholders to transform the health system

3.4 What impact have we made with other schools?

We actively engage with and support other institutions across national boundaries to achieve common social accountability goals

Research Evidence and Knowledge Generation

Through research and data collection THEnet:• identifies evidence for effective strategies • identifies what works, how and in what

context• identifies bottlenecks and barriers to reform• provides lessons learn to prevent mistakes• measures return on investment • identifies research gaps • builds a knowledge base on best practices• creates global database to track graduates• conducts research on health impact of

specific packages of education interventions• develops and strengthens tools


1. Conducting a 10-year cross-institutional Graduate Outcome Study. Theme 1: What are students/graduates characteristics upon entry, during their training process, and postgraduate?

Theme 2: Do our graduates make greater contributions to better access to services, quality of care, and health outcomes, appropriate for underserved populations in our community when compared to all medical graduates?

Theme 3: Where are graduates located, and what is their type of practice and discipline? Do these match the health workforce needs?

THEnet Research Priorities

Aims of cohort study – To analyse the origins, intentions and

practice outcomes of medical students and graduates

– To understand the link with selection strategies

– To correlate graduate outcomes with population needs


41

For more on the Training for Health Equity Network

www.thenetcommunity.org

Conclusions and implications• THEnet partner schools using a variety of

selection strategies have a student population that more closely matches reference population when compared with traditional schools.

• Confirms data that rurality of origin predicts likelihood of intending to work with rural and remote and other underserved populations.

• Selection processes important

A/P rof S arah L arkins , P rof S ara Willems , P rof S alwa E ls anous i, P rof J ehu Iputu, A/P rof K enia Monjes , Dr Marykutty Mammen, Dr R ex S ams on, A/P rof F iledito Tandinco, Dr S hambuUpadhyay, Dr J ennene G reenhill, Dr L is a G raves , Dr Mark Haus wald, Ms S imone R os s , Dr Torres Woolley, P rof Andre-J acques Neus y, Ms B jörg P als dottir; F or the Training for Health E quity Network

What difference do we make? Preliminary results from an

international graduate outcome study in socially accountable health

professional schools.TUFH, Ayuthaya, 16-20 November 2013


Presentation by Rebbeca Bailey

Team Leader, Health Workforce Development, CapacityPlus, IntraHealth International

Education Assessment Tools

Rebecca Bailey, Team Lead for Health Workforce Development, CapacityPlus Project led by IntraHealth International

For more information, please contact [email protected]

School management self assessment: management of human, f inancial, material and intellectual resources

Delivery of a relevant, learner-

centered, and competency-

based curriculum

Leadership and

Governance Strategic Planning

Financial Resources

Human Resources

Facilities and

Infrastruc-ture

Equipment and

Material Resources

Evaluation and

Knowledge Management

Student Resources

External Relations

R ec ognizes that s c hools mus t adhere to loc al s tandards , polic ies , laws and regulations

Elements of C urric ulum Management integrated into all categories


Example: Equipment and M aterial Resources

To what extent do you agree that the following are present at your institution?1. Forecasting and Procurement

– Planning for equipment and material resources is based on a clear assessment of the future needs of the institution—especially with respect to the curriculum.

– Each academic department has a budget for educational materials and equipment.

– A competitive bidding process is used for major acquisitions.

– Procurement is completed in a timely manner.

– Approval for procurement of major acquisitions is at the executive level.

Bottlenecks Assessment: Snap Shot Plus

Focuses on identifying bottlenecks to scale up within nine critical areas of health worker education.

1. Students2. Educators3. Financial Management4. Curriculum5. Materials and Equipment6. Facilities and Infrastructure7. Clinical Practice8. Quality Assurance9. Partnership and Exchange


Examples of common f indings (2)

• Shortage of qualified and motivated educators, especially for clinical practicum (with high turnover)

• Limited opportunities for staff development, especially in pedagogy

• Students poorly prepared for health sciences studies, and seldom selected based on interest and motivation (central selection processes)

• Large class sizes often exceeding the maximum capacity for new admissions and of classrooms

• Drop outs due to financial burden

• Outdated curricula

• Shortage of materials and equipment,especially textbooks, anatomic models for skills labs, and equipment for clinical practice (e.g. gloves, thermometers, etc.)

Examples of common f indings (2)

• Shortage of well-equipped classrooms and skills labs

• Lack of autonomy or control over finances (budgets, income, expenditures), and of budgets for the maintenance and renovation of infrastructure

• Poorly maintained infrastructure, with limited access to IT, inadequate libraries, computer labs, electricity, water, etc.

• Insufficient opportunities for clinical practice, with too many students per site, poor transportation options, and limited supervision

• Near absence of partnerships with health facilities and other schools, and limited exchange programs for students/faculty

• Weak systems for accreditation of schools and certification of graduates

• Lack of support to students for their transition to employment after graduation


Optimizing Performance and Quality

A performance improvement approach designed for service delivery that can be applied to education

http://www.intrahealth.org/files/media/optimizing-performance-and-quality/OPQ_FINAL.pdf


This presentation looked at those tools which are being used by CapacityPlus. Management is an important area in education, and as management skills are often lacking in these professionals, it is important to teach these skills to medical educators.

Curriculum Management is an area which did not exist before. This self-assessment tool should be available in the near future.

Some important issues to bear in mind include: � Finance – does each academic department have a budget? � Bottlenecks Assessment – Snapshot – what are the bottlenecks for schools to achieving specific goals? What are the reasons for these bottlenecks? How is a school being restricted from what it wants to achieve?

� Quality-Improvement Approach – this is also aspirational. We bring together stakeholders and identifying the gaps between the current situation and the aspirations.

� Optimizing Performance and Quality tool for service-delivery – What is the current situation? What are the gaps in relation to the aspirations? Which tool would be most cost-effective?


Presentation by Lola Dare, CEO

The Centre for Health Sciences Training, Research and Development (CHESTRAD)

System performance: Universal Access & Coverage

Timely, adequate, quality resources for

HRH action

Investments / Resources

Transformational Education

HRH for service delivery

The accountable health workforce

Tracking

Skilled & competent health team

Satisfied consumers

Monitoring & Evaluation

Skilled workforce

Public accountability

Social Accountability

Contribution to HSS performance:

Towards Universal Access & Coverage

Productivity

Actions- Policy- Finance- Education- Partnership- Leadership- Managed

migration

(Retention strategies)

Actors

- MoH- Prof.

Assoc.- MoF- CSC

Performance

Actions- Mortality &

morbidity audits

- Service delivery assessments

- Absenteeism- HRH Per

Assessment etc.

Actors

- Facility managers

- Facility committees

- Communities

Accountability for HRH Accountability of HRH

Actors

- MoH- MoE- MoF- Training

institutions- (trad/non-trad)

Production

Actions- Education &

training- Accreditation

& regulation- Financing &

sustainability- Implementat

ion, M&E- Governance

& Planning

Commitments & Guidelines

ActionsCommitments made at 3GF are reflected in plan with: Timeframes, Resources, Indicators, Responsibilities

Guidelines on: Code of Conduct, Retention and Transformational Education

Actors

- Multi-stakeholder

This presentation set out an Accountability Framework for human resources for health working on with four countries - Malawi, Nigeria, Liberia and Ghana - and a number of other partners. It is working on a cascading system of accountability rather than a clear cut divisive system of accountability.

There are several types of guidelines including guidelines on guidelines on retention and education.

Accountability

� Different actors have different accountability. � Accountability for the productivity of the health workforce. � Accountability for performance : An accountability framework is very important- (absenteeism, ghost-workers, etc)

� Public accountability for producing a dynamic health workforce. � Institutional accountability cannot be looked at in isolation.

This is part of what will be implemented in the above four countries. The tools were developed by civil society institutions looking at accountability of the health system.

The report has just been submitted to WHO.


Presentation by Professor Ronald Harden

General Secretary, Association for Medical Education in Europe (AMEE)

Education programmes can be evaluated at three levels.

Level 1 Evaluation of the education programme in relation to 10 domains (see attached figure).

Level 2 More in depth evaluation of each of the 10 domains.

» For example, in relation to educational strategies, the SPICES model has been used where six strategies are each assessed on a continuum

Student centred … Teacher centredProblem based … Information orientatedIntegrated/inter-professional … Unidisciplinary/uniprofessionalCommunity based … Hospital basedElectives… Uniform programmeSystematic …Opportunistic

Level 3 Further in-depth evaluation of individual elements from level 2.

» For example, in the case of educational strategies and inter-professional education, an evaluation of the position on the rungs of a ladder with isolation being the bottom rung and trans-professional education being at the top.

I describe elements of this approach further with references in my book, Essential Skills of a Medical Teacher, Elsevier Publishing Company.

This presentation described a set of tools used to evaluate a curriculum/educational programme using a SPICES model

Student-centred … Teacher centredProblem-based … Information orientatedIntegrated/inter-professional … Unidisciplinary/uniprofessionalCommunity-based … Hospital-basedElectives… Uniform programmeSystematic …Opportunistic

For more information, contact:World Health OrganizationDepartment of Health Workforce (HWF)Avenue Appia 201211 Geneva 27Switzerlandhttp://www.who.int/hrh/education/en/

transforming the health workforce in support of universal health coverage

Documents