who operational package for assessing, monitoring and evaluating

WHO/TCM/2007.2

WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations

Guide for coordinators and data collectors

December 2007

It is important to know… if people have access to essential medicines; if they are getting medicines that are safe, effective and of good quality; and if these medicines are being properly used.

WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors WHO/TCM/2007.2

© World Health Organization 2007 All rights reserved. Publications of the World Health Organization can be obtained from WHO Press, World Health Organization, 20 Avenue Appia, 1211 Geneva 27, Switzerland (tel: +41 22 791 3264; fax: +41 22 791 4857; email: [email protected]). Requests for permission to reproduce or translate WHO publications – whether for sale or for noncommercial distribution – should be addressed to WHO Press, at the above address (fax: +41 22 791 4806; email: [email protected]). The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of the World Health Organization concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted lines on maps represent approximate border lines for which there may not yet be full agreement. The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by the World Health Organization in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters. The World Health Organization does not warrant that the information contained in this publication is complete and correct and shall not be liable for any damages incurred as a result of its use.

- iii -

Contents

Abbreviations...........................................................................................................................vii Preface ........................................................................................................................................ ix Acknowledgements ................................................................................................................. xi Getting started ............................................................................................................................1 Pharmaceutical indicators for monitoring and assessment................................................5 The need for pharmaceutical sector indicators .......................................................................5 Who can use the results..............................................................................................................5 Indicator-based monitoring strategies .....................................................................................7 Level I core indicators................................................................................................................9 Pharmaceutical components in Level I indicators..................................................................9 Level II facility-based core indicators ..................................................................................11 Survey design and data collection (Level II - facility survey).............................................13 Selecting geographical areas....................................................................................................13 Selecting public health facilities ..............................................................................................14

Making random selections.............................................................................................15 Selecting private drug outlets..................................................................................................16 Selecting warehouses................................................................................................................16 Sampling patients for data collection .....................................................................................16

Retrospective sampling..................................................................................................16 Prospective sampling .....................................................................................................18

Summary of Level II indicators measured by facility ..........................................................19 Preparing the survey (Level II - facility survey).................................................................21 Coordination and survey coordinator....................................................................................21 Tailoring the survey forms to country situations by choosing key medicines, selecting tracer conditions and identifying treatment protocols........................................22

Selecting basket of key medicines.................................................................................23 Identifying medicines to be considered as injections and antibiotics......................24 Identifying medicines of choice to treat pneumonia in adults and children..........24 Identify standard criteria for adequate labelling and patient knowledge..............25 Identifying unit price of medicines to treat outpatient adults and children ..........25 Instructions for obtaining global and regional drug prices [paid by the patient or paid by the facility] (Survey Forms 2 and 11).....................25

- iv -

Training data collectors...........................................................................................................27 Selecting data collectors ...........................................................................................................27 Training components................................................................................................................27 Training tools needed ...............................................................................................................28 Training slides introduced.......................................................................................................28 The field test...............................................................................................................................29 Reviewing the field test and calculations ..............................................................................30 The survey .................................................................................................................................31 Notifying facilities to be visited ..............................................................................................31 Travel schedule and logistics...................................................................................................31 Data collectors............................................................................................................................32 Supervision of data collection .................................................................................................33 Data processing, analysis and reporting: what to do with the results ...........................35 Computation of Level II facility indicators............................................................................35 Quality of data and information .............................................................................................35 Analysis and interpretation of Level I and Level II facility indicators ..............................36

Limitations of the Level II facility survey....................................................................38 Indicator measure for Level II facility indicators .......................................................38 Performance standards for Level II facility indicators ..............................................39

Written report ............................................................................................................................39 Presentation and discussion of results and related issues...................................................43 Format of result presentation ..................................................................................................44 Annexes ......................................................................................................................................47 Annex 1: Level I questionnaire ............................................................................................49 Annex 2: Level II indicators .................................................................................................71 Annex 2a: Technical descriptions of Level II indicators................................71

Annex 2b: Issues, technical details, survey design and sampling for Level II indicators.......................................................................85

Annex 3: Coordinator checklist ...........................................................................................89 Annex 4: Example of budget allocation..............................................................................95 Annex 5: Guide on how to use the training slides (Level II indicators) ........................97 Annex 6: Training slides for data collection ....................................................................103 Annex 7: Level II Survey Forms 1—17 ...............................................................................121

- v -

Annexes in the CD Annex 1: Level I questionnaire

Annex 3: Coordinator checklist

Annex 4: Example of budget allocation

Annex 6a: Slides for training data collectors

Annex 6b: Slides for training survey coordinators

Annex 6c: Slides for policy-makers

Annex 7: Level II Survey Forms 1-17

Annex 7a: Regional List for prices of drugs (supplementary to SF2 & SF 11)

Annex 8: Summary Forms Annex 9: Sample report Annex 10: Letter to facilities

List of boxes

Box 1 Summary list of indicators and corresponding survey form used to collect the data ...................................................................................................... 12

Box 2 Summary of survey areas and sites ........................................................................ 14 Box 3 Selecting facilities from 5 geographical areas ....................................................... 15 Box 4 Randomly selecting public health facilities from the capital city....................... 15 Box 5 Simple method for selecting 30 patients from the general outpatient list ........ 17 Box 6 Retrospective sampling: Selecting 30 patients

from general outpatient records covering 12 months (365 days)....................... 17 Box 7 Items on Survey Forms to predefine at national level ......................................... 22 Box 8 Sample training schedule......................................................................................... 27 Box 9 Sample letter to facilities .......................................................................................... 31 Box 10 Checklist for data collectors..................................................................................... 32 Box 11 Error in sampling ...................................................................................................... 86 Box 12 Overview of the core indicator pyramid (Slide 7) .............................................. 106

List of figures

Figure 1 Levels of core indicators ........................................................................................ 1 Figure 2 Use of median and percentiles to describe the dispersion

of measurement..................................................................................................... 38 Figure 3 % of patients receiving antibiotics (distribution of facility results) ............... 39 Figure 4 Presenting national level data graphically........................................................ 45

- vi -

List of tables

Table 1 Study facilities, indicators and forms ............................................................... 19 Table 2 Model list of key medicines................................................................................ 24 Table 3 Suggested report outline..................................................................................... 41

Abbreviations

- vii -

Abbreviations

AFRO* WHO Regional Office for Africa AMRO* WHO Regional Office for the Americas ARI Acute respiratory infection DTC Drug and therapeutics committee EC European Community EDM WHO Essential Drugs and Medicines Policy Department EML Essential medicines list EMRO* WHO Regional Office for the Eastern Mediterranean EURO* WHO Regional Office for Europe FIP International Pharmaceutical Federation HAI Health Action International HIV/AIDS Human immunodeficiency virus/Acquired immunodeficiency

syndrome INN International Nonproprietary Name

INRUD International Network for the Rational Use of Drugs MOH Ministry of Health MSH Management Sciences for Health

NAF/ENSP/Fiocruz Center for Pharmaceutical Policies / Sérgio Arouca National School of Public Health / Oswaldo Cruz Foundation (Núcleo de Assistência Farmacêutica/ Escola Nacional de Saúde Pública Sérgio Arouca/ Fundação Oswaldo Cruz)

NGO Nongovernmental organization NMP National medicines policy ORS Oral rehydration salts OTC Over-the-counter RDU Rational drug use SEARO* WHO Regional Office for South-East Asia SF Survey forms STG Standard treatment guidelines TM/CAM Traditional medicine and complementary and alternative

medicines TRIPS Trade-Related Aspects of Intellectual Property Rights

UNICEF United Nations Children's Fund WHO World Health Organization WPRO* WHO Regional Office for the Western Pacific WTO World Trade Organization

WHO regional offices were used to group countries for purposes of regional data summary and analysis.

WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors

- viii -

Preface

- ix -

Preface

The complexity of the pharmaceutical sector, with multiple and cross-cutting factors that can influence access to and rational use of quality medicines, makes it is extremely important to have a systematic method for assessing the pharmaceutical situation at country, regional and global levels. Pharmaceutical sector assessment, monitoring and evaluation aim to answer the following vital questions: do people have access to essential medicines? Are people getting medicines that are safe, effective and of good quality? Are these medicines being used properly? Currently we have insufficient information to answer such questions, but as part of its commitment to assisting countries to improve access to and use of quality medicines, the World Health Organization (WHO) has been developing tools to monitor and evaluate pharmaceutical situations.

Assessment, monitoring and evaluation underpin evidence-based policy development and strategic planning, and therefore every aspect of WHO’s activities in pharmaceuticals in general and essential medicines in particular. These processes can be complex for a number of reasons: it is difficult to establish a sustainable system of regular assessment, monitoring and evaluation; resources are not consistently allocated to these tasks and there is limited advocacy for them. Furthermore, many efforts to develop tools have been exhaustive, but impractical and in the past, most tools included indicators that were difficult to collect, especially on a regular basis.

A number of indicator-based tools now exist. Indicators for Monitoring National Drug Policies,1 a WHO manual, includes approximately 120 indicators covering structure, process and outcomes of various national medicines policy (NMP) components. Another set of indicators, developed by Management Sciences for Health (MSH),2 focuses on rapid assessment of strengths and weaknesses in the pharmaceutical sector. Another WHO manual, How to Investigate Drug Use in Health Facilities,3 has been used extensively in many countries. The WHO/HAI Medicine Pricing Project,4 is also developing indicators to assess medicines regulations and access to pharmaceuticals.

Development of the set of indicators referred to in this document started in 1999 when the first Level I global survey was done. Another survey was undertaken in 2003 and the intention is to repeat surveys every four years. The Level I core indicators consist of a questionnaire used to rapidly assess existing structures and processes in a national pharmaceutical system.

Along with this, a set of core indicators - Level II facility indicators - was developed, which require systematic surveys and data gathering at health care facilities. The development process has benefited from countries doing baseline assessments, and feedback from trainers and experts who have used the survey forms. In 2002, the WHO

1 Brudon P, Rainhorn JD, Reich M. Indicators for monitoring national drug policies. Geneva, World

Health Organization, 1994. 2 Rapid pharmaceutical assessment - An indicator-based approach. Washington DC, Management

Sciences for Health, 1995. 3 How to investigate drug use in health facilities. Geneva, World Health Organization, 1993.

WHO/DAP/93.1. 4 Medicine prices: a new approach to measurement. Geneva, World Health Organization and Health

Action International, 2003. WHO/EDM/PAR/2003.2.


- x -

Operational Package for Assessing, Monitoring and Evaluating Country Pharmaceutical Situations evolved, together with the Level I indicators.

In February 2003, the first group of countries to have used the Level II facility survey tool, and the technical experts and consultants involved in the development process met in Geneva. They discussed the results of country surveys and further improved the tool based on experiences from the field. By June 2003 a working draft was developed.

In July 2006, a meeting of experts discussed the status of WHO indicator-based pharmaceutical assessment and monitoring approaches. Several recommendations for future improvements of the Level I and Level II facility survey tools were made. These recommendations are included in the report, WHO Expert Meeting on Pharmaceutical Indicators, Monitoring and Assessment 10-12 July 2006, Geneva Switzerland.

The WHO Operational Package for Assessing, Monitoring and Evaluating Country Pharmaceutical Situations is intended as a useful tool for researchers, policy-makers, planners and others who need to use standardized measurement tools to gather data and other information. The tools presented here have already been used for several years at global and country levels. In addition, the operational package can be used by international agencies and donors, by professional groups and nongovernmental organizations (NGOs). Countries can use the Level I questionnaire as a checklist to illustrate sectoral structures, strategies and approaches. Countries can also use selected forms from the Level II facility survey in their routine monitoring. The results can help them to focus their strategies, advocacy plans and information campaigns.

Acknowledgements

- xi -

Acknowledgements

Development of the tools included in this document has been ongoing for several years. The operational package has been used as a working draft since 2002 under the former WHO Department of Essential Drugs and Other Medicines (EDM). The current document is an initiative of the Department of Technical Cooperation for Essential Drugs and Traditional Medicine, with support from the Department of Medicines Policy and Standards (PSM).

Precious Matsoso, Germán Velásquez and Gilles Forte provided the leadership and support culminating in finalization of this document. Jonathan Quick guided the initial development and evolution of the WHO medicines indicator process and the assessment of pharmaceutical situations in countries. Hans Hogerzeil has also supported the development of the operational package. Dennis Ross-Degnan has provided technical advice on the document from its inception to completion as well as assisted in developing some chapters.

The development of the tools presented in this document benefited from the contributions and expertise of various individuals and institutions:

In February 2003, representatives from the first group of countries who used the Level II facility survey tool and the technical experts and consultants involved in the development process met in Geneva: Timotheo Badoy (Philippines); Majid Cheraghali (the Islamic Republic of Iran); Norma Duarte (Guatemala); Ilko Getov (Bulgaria); Jean-Marc Guimier (MSH); Alec Irwin (Millennium Development Project); Martin Auton, Simona Chorliet and Indro Mattei (consultants); Dennis Ross-Degnan (Harvard University); and from WHO, Ogori Taylor (WHO Nigeria), Bakuti Shengelia, Asghari Gholamreza, Guitelle Baghdadi, Diane Whitney, Eshetu Wondemagegnehu, Helen Tata, Edelisa Carandang, Gilles Forte, Germán Velásquez and Jonathan Quick.

In July 2006, another meeting of experts was held in Geneva. The recommendations and proceedings of the meeting are recorded in the report, WHO Expert Meeting on Pharmaceutical Indicators, Monitoring and Assessment. Participants were: John Chalker (INRUD-MSH); Dennis Ross-Degnan (Harvard Ubniversity); Simona Chorliet (Mali); Trudie Gerrits (University of Amsterdam); Vera Lucia Luiza (Brazil NAF/ENSP/Fiocruz); Helene Moller (UNICEF); Patrick Mubangizi (HAI-Africa); Javier Penalvier Herrero (Spain); Ray Skinner (Solomon Islands); Klara Tisocki (University of Kuwait); Mary Justin (United Republic of Tanzania); Marsha Macatti-Yamba (United Republic of Tanzania); Albert Wertheimer, (International Pharmaceutical Federation); Zafar Mirza (WHO, Regional Office for the Eastern Mediterranean - EMRO); Nelly Marin (WHO Regional Office for the Americas - AMRO); Ogori Taylor (WHO, Nigeria); Bekele Tefera Jembere (WHO, Ethiopia). The following are from WHO headquarters, Geneva - Precious Matsoso, Gilles Forte, Clive Ondari, Valerio Reggi, Richard Laing, Marthe Everard, Sue Hill, Kathy Holloway, Sophie Logez, Jörg Hetzke, Edelisa Carandang, Helen Tata, Martin Auton, Alain Prat, Shanti Noriega-Minichiello, David Wood and Paul Verboom.


- xii -

The development of this document, specifically the Level II Facility survey tool, would not have been possible without the contributions of researchers, data collectors and survey coordinators from nearly 50 countries who have undertaken the Level II facility survey. The WHO Regional Pharmaceutical Advisers /Acting Advisers: Jean-Marie Trapsida (WHO Regional Office for Africa - AFRO); Jorge Bermudez, Nelly Marin, Rosario D’Alessio; Christophe Rerat (WHO AMRO); Zafar Mirza, Mohammed Bin Shahna (WHO EMRO); Kees de Joncheere (WHO Regional Office for Europe - EURO); Krisantha Weerasuriya (WHO Regional Office for South-East Asia - SEARO); and Budiono Santoso (WHO Regional Office for the Western Pacific - WPRO) have organized and facilitated country surveys with WHO country offices and National Programme Officers working on essential medicines issues.

Several people have contributed at different stages of development and finalization of the main document, the survey forms - Indro Mattei, Simona Chorliet, Martin Auton, Andy Gray, Kathy Holloway, Marthe Everard, Sue Hill, Christophe Rerat, Klara Tisocki, Anita Wagner, Catherine Vialle-Valentin and Richard Laing. John Snow Inc. provided comments in the early stage of development. The facility survey tool has also benefited from feedback in the Drug Policy Issues courses held in Mumbai and Beirut.

Diane Whitney assisted in the development of the June 2003 working draft. Lalit Dwivedi assisted in reformatting the working draft. The group at NAF/ENSP/Fiocruz, Brazil, led by Vera Lucia Luiza, Thiago Azeredo, Isabel Cristina Martins Emmerick and Maria Auxiliadora Oliveira assisted in the completion and finalization of the main document, and survey forms. They also helped in completing the summary forms and various sets of training slides. Claude Da Re did the layout and supervised publication.

Edelisa D. Carandang coordinated and supervised the development and implementation of the Level I and Level II indicator tools, and the development of this document.

The European Community (EC) supported the printing of this publication as well as some country and regional training and country surveys. The Department for International Development (DFID), UK, also provided financial support for some country training and surveys. The content of this publication does not necessarily reflect the official opinion of the EC and DFID.

Getting started

- 1 -

Level I Core structure

& process indicators

Level II Core outcome/impact indicators

Access to essential medicines Rational use of medicines

Level III In-depth assessments of specific components of the pharmaceutical sector � Pricing � Traditional medicines � HIV/AIDS � Regulatory capacity � Drug supply mgt. � TRIPS

Systematic health facility and household surveys

Key informant questionnaire sent to countries

Getting started

Monitoring and assessing the pharmaceutical sector are vital to determine if the key pharmaceutical objectives are met: people have access to essential medicines, these medicines are safe, effective and of good quality, and they are used correctly. A systematic method to assess and monitor the impact of strategies and activities will provide information on issues and gaps, all-important input in the development of health policies.

The WHO process for pharmaceutical monitoring and assessment uses a hierarchical approach with three groups of indicators: Level I, Level II and Level III. This provides a standard methodology to follow progress over time and to compare situations in different facilities, districts and countries.

Figure 1: Levels of core indicators

Level I indicators provide a rapid means of obtaining information on the existing infrastructure and key processes of each component of the pharmaceutical sector. The indicators are assessed by using a short questionnaire completed at national level at a regular time interval (four years), allowing measurement of trends and comparisons of the situation over a period of time. The results provide a range of descriptions of


- 2 -

existing structures and processes, and can illustrate the country's capacity to implement policy in specific areas of the pharmaceutical sector - areas which are listed in the next section. The WHO database is updated using data and other information from the Level I survey. Analyses are also used in reports, publications and as references when giving country data. In addition, results can serve as a checklist for countries, and as a basis for doing rapid assessment of the structures and process of their pharmaceutical sector.

Level II health facility indicators provide systematic data to measure outcomes on access (affordability and availability of key medicines and geographical accessibility of dispensing facilities) and rational use of quality medicines, including some indication of the quality of medicines at health facilities and pharmacies. Data on these indicators are collected through systematic surveys of public health facilities, public and private pharmacies and public warehouses. The processes involved are detailed in this package. The results of country surveys can be used to indicate the extent to which the objectives set by the pharmaceutical sector - specifically the government and the national medicines policy - have been achieved. The results show the areas and gaps that should be addressed and which strategy can be prioritized for facilities, districts and countries. Global comparison can also be used to establish norms in access, use and, to some degree, quality of medicines made available from health facilities. Subsequent sections of this document focus on the technical aspects and logistics of doing a Level II facility indicator survey.

The Level I survey and Level II facility survey do not measure access to and use of medicines from the patient/consumer perspective. Only household surveys can provide population-based information about how pharmaceutical policies affect the well-being of individuals. A working document on household surveys has been developed and is being piloted together with Level II facility surveys in some countries. The methodology assesses whether and how people access medicines, how they use them, how much they pay for them, and how out-of-pocket payments for medicines impact on household finances.

Level I and Level II surveys (facility- and population-based surveys) should be done as a way to scope comprehensive pharmaceutical situations — they can be used to establish baseline data and to measure the impact of the strategies implemented. They can also be used to establish trends, particularly the Level I survey which can track the global pharmaceutical situation regularly to measure trends.

Information gathered specifically from Level I and II indicators is useful to reassess strategies, priorities and strengthen pharmaceutical system components, and to synchronize programmes and policies. Policy-makers and managers will have a clear picture of national and institutional problems. International agencies and donors can focus on priority areas where the best impact can be achieved. Professional groups and NGOs can focus advocacy and information campaigns. Information can be shared via databases and web pages.

Level III indicators are a more detailed and expanded list of indicators covering key components and areas such as those elaborated in several indicator documents in medicine pricing, medicine supply management, HIV/AIDS, TRIPS, rational drug use (RDU) and regulatory capacity assessment. Countries can use any of these set of indicators as baseline assessment and follow-up studies depending on needs and capabilities.

Getting started

- 3 -

The tools necessary to gather most of this information have been brought together in this document. This operational package provides the following:

• A Guide for Coordinators and Data Collectors of the Level II Facility Survey detailing operational procedures to carry out the indicator survey, with step-by-step procedures on administrative preparation (budget, training plan and schedule) and technical requirements (training and field-testing, surveying, analysis and reporting). Training slides are also provided.

• The annexes contain the technical descriptions of Level II facility indicators and the sampling process. Survey forms are included, and graphs and tables can be generated from the analysis template.

• The Level I questionnaire, which is sent to countries once every four years to update global pharmaceutical data, is included in the annexes. It can also serve as a rapid assessment and checklist for countries to check current the pharmaceutical structure and processes of their national pharmaceutical systems.

• A diskette that contains Level II facility survey forms, summary forms and training slides.

At least 40 countries have already undertaken Level II facility surveys. The current version of the operational package has benefited considerably from experiences in these countries. The indicators have also been used in international/regional courses and meetings to gather data from health facilities and pharmacies. Experience has shown that the survey can be completed without investing large resources in terms of time, people or money. Experiences with using these tools have proven that regular monitoring is not difficult (Level I survey) and can be done in a cost-efficient manner (Level I, II facility). This experience encourages allotting a portion of country support budgets, project grants and MOH budget for monitoring and assessment.

WHO published Using Indicators to Measure Country Pharmaceutical Situations: Fact Book on Level I and Level II Monitoring Indicators5 using information from surveys conducted with the above tools. This Fact Book gives the results of the assessment of Level I indicators conducted in 2003 and of Level II indicator surveys conducted between 2002 and 2004.

5 WHO, Harvard Medical School and Harvard Pilgrim Health. Using indicators to measure country

pharmaceutical situations — Fact book on WHO Level I and II monitoring indicators. Geneva, World Health Organization, 2006. WHO/TCM/2006.2.


- 4 -

Pharmaceutical indicators for monitoring and assessment

- 5 -


The need for pharmaceutical sector indicators

Given the complexity of the pharmaceutical sector, a systematic method of gathering data is very important for assessing access, quality and rational use of medicines. There are multiple, cross-cutting factors that can influence access and rational use of quality medicines, and a variety of strategies that countries can adopt and implement to improve their pharmaceutical situations.

Indicators have been developed for monitoring national medicines policies (NMPs) that enable systematic assessment, evaluation and monitoring of the formulation and implementation of pharmaceutical policies and programmes. These can be used to:

• assess country capacity, such as available infrastructure, logistics and human resources to support the pharmaceutical sector and implement NMPs;

• monitor the implementation of NMPs;

• measure the impact of implementation strategies; and

• evaluate progress towards identified objectives.

Who can use the results?

All stakeholders in the pharmaceutical sector can use indicator-based assessment of the pharmaceutical situation to inform priorities and set targets. They can also use regular monitoring of the sector through indicator-based studies to assess strengths and weaknesses of strategies to improve the provision of pharmaceuticals.

Indicators provide policy-makers and managers with a clear picture of national and institutional problems. Policy-makers and managers can refer to study results when developing strategies to strengthen the pharmaceutical sector. Results can also be used to synchronize policies. For instance:

• Low access, as measured by availability and affordability of essential medicines, could indicate that policies on health and medicines financing should be reviewed. Economic policies may be focused on joining the global economy without adequate consideration of the implications on pricing, affordability and availability of important medicines.


- 6 -

• The presence on the market of a large number of substandard medicine products could indicate the need to assess various policies and systems, including licensing and inspection of manufacturers, quality assurance or product registration.

International agencies and donors can use the results of indicator-based studies to identify where their activities will achieve the greatest impact. Professional groups and nongovernmental organizations (NGOs) can use the results to focus their advocacy and information campaigns.

Standard indicators allow informative comparisons among countries. For example an indicator-based assessment in 12 countries revealed:6

• Most of the countries surveyed have relevant structures in place, however, “it is easier to create a structure than to make it work”.

• Most of the countries have a medicines regulatory authority with a mandate to register medicines and inspect manufacturer and retail outlets, however, the enforcement of regulations is often weak.

• In most of the countries, public financing for medicines is limited.

Countries may use the information as follows:

• Advocacy to disseminate and to share the results with national stakeholders, politicians, civil society, acadaemia and key partners (awareness issue). Comparison with other countries at a similar level of economic development, within the region or globally can also be made.

• Sharing the information through national workshops, by posting the information on website links and through national brochures. These are also good ways to elicit feedback from stakeholders.

• Analysing the country situation allows gaps, probable causes of problems and unmet objectives to be identified. This in turn facilitates the identification of appropriate strategies.

• Reporting by topic (access, quality, safety and rational use of medicines), and doing the analysis by type of indicator (structure, process and output) will also guide countries to share sub-regional experiences, through informative comparisons.

• Guiding policy-makers in developing and planning national strategy, including budget planning, reallocation of funding and human resource management. which are key cross-cutting factors that can influence access to medicines.

• Providing the national medicines regulatory authorities with adequate information for the enforcement of regulations and to assess the level of regulatory authority capability.

6 Comparative analysis of national drug policies. Workshop report. Geneva, World Health Organization,

1997. WHO/DAP/97.6.


- 7 -

• Information and data can be used to:

o formulate/revise the NMP. Regular monitoring of implementation and evaluation of progress towards strategic objectives is important in aligning medicines policy to changes in the political, economic and financial environment.

o assess the quality of governance, such as systems and mechanisms that would safeguard against vulnerability to corruption.

o encourage and facilitate cooperation between the different main players in a country to achieve common strategic goals.

Indicator-based monitoring strategies

Monitoring NMPs is a complex task. While important, it is difficult to establish a sustainable system of regular monitoring. Resources are not consistently allocated to this task and there is limited advocacy for a culture of monitoring. Further, many efforts to develop monitoring tools have been exhaustive, but impractical. In the past, most monitoring tools included indicators that were difficult to collect, especially if this was done regularly.

A number of indicator-based monitoring tools now exist. Indicators for Monitoring National Drug Policies,7 a WHO manual, includes approximately 120 indicators covering structure, process and outcomes of various NMP components. Several countries have used it to monitor and evaluate their pharmaceutical situations. Another set of indicators, developed by Management Sciences for Health,8 focuses on rapid assessment of strengths and weaknesses in the pharmaceutical sector. The WHO manual How to Investigate Drug Use in Health Facilities9 has been used extensively in many countries.

This document, WHO Operational Package for Assessing, Monitoring and Evaluating Country Pharmaceutical Situations. Guide for Coordinators and Data Collectors, was developed to provide a practical indicator-based tool that can be implemented regularly without investing large amounts of human or financial resources. This package relies on a hierarchical approach to monitoring built around three groups of core indicators. The core indicators are easy to collect using standardized methodologies, small samples of data and simple survey techniques. These core indicators systematically measure the most important information needed to gain a comprehensive picture of the pharmaceutical situation in a country.

7 Brudon P, Rainhorn JD, Reich M. Indicators for monitoring national drug policies. Geneva, World

Health Organization, 1994. 8 Rapid pharmaceutical assessment – An indicator-based approach. Washington DC, Management

Sciences for Health, 1995. 9 How to investigate drug use in health facilities. Geneva, World Health Organization, 1993.

WHO/DAP/93.1.


- 8 -

Grouping monitoring indicators by level (Level I and Level II) has the following advantages:

• It offers flexibility to those interested in information on the country pharmaceutical situation:

o rapid assessment of key pharmaceutical components;

o monitoring outcome and achievement of key objectives of the pharmaceutical policy; and

o in-depth assessment of specific system components.

• It provides practical methods for regularly monitoring NMPs and their components.

• It encourages regular reporting and exchange of pharmaceutical information among facilities, districts, regions, government and nongovernmental agencies as well as international organizations.

Level I indicators provide a rapid means of obtaining information on the existing infrastructure and key processes of each component of the pharmaceutical sector. These indicators are assessed using a short questionnaire completed at the national level.

Level II indicators provide systematic data on access and rational use of quality medicines through facility-based surveys. Systematic survey processes to collect data on these indicators are contained in this package. A population-based survey has been developed as part of the current process and is discussed in a separate document, Manual for the Household Survey to Measure Access and Use of Medicines.

Level I core indicators

- 9 -

Level I core indicators

The Level I core indicators are used to assess existing structures and processes in a national pharmaceutical system. They provide a method to rapidly assess the implementation of NMPs and their components.

For Level I indicators, a knowledgeable informant can coordinate the gathering of information. Most data will be available within the Ministry of Health, although data on intellectual property rights protection may require consultation with the responsible ministry. Data collection does not require field surveys and information can easily be updated periodically, for example every two years.

Data from Level I indicators can be used to array the achievements and weaknesses of individual pharmaceutical systems and to illustrate common sectoral strategies and approaches. Many WHO Member States have submitted data from the Level I questionnaire to EDM. The WHO MedNet can be consulted to compare results over time and among countries. Comparisons among countries can be particularly interesting and convincing for policy-makers. The questionnaire on Level I indicators is included as Annex 1. The indicators in this questionnaire are summarized below.

Pharmaceutical components in Level I indicators

National Medicines Policy (NMP) - An NMP document that covers the public and private sectors, a written implementation plan and the integration of medicine and health policies provide a basic framework to organize and improve the pharmaceutical system. They also assist in coordinating the functions and strategies of each component as they are being implemented. Regular monitoring helps to inform the NMP and its implementation.

Regulatory system (Regulatory authority, Marketing authorization, Licensing, Regulatory inspection, Control of narcotics and stupifiants, Quality control, Pharmacovigilance, Counterfeit medicines, Dispensing and prescribing, Promotion and advertising) - Legislation/regulations on medicine manufacturing, promotion and advertising, sales, distribution, dispensing and prescribing must be in place. A medicines regulatory authority should be able to efficiently regulate these activities through registration of products, licensing and inspection of manufacturers, importers and pharmacies, control of counterfeiting, control of narcotics and stupifiants and monitoring of adverse medicine reactions. Legislation directed at generic prescribing, dispensing and substitution can help increase access to essential medicines in both the private and public sectors. Quality control of pharmaceuticals should cover all activities to ensure patients receive safe, efficacious and high-quality medicines. There should be a medicines quality control laboratory to test medicines prior to registration and at various points in the distribution system.


- 10 -

Medicines supply system - Access and availability of essential medicines, especially at public sector facilities, are affected by how medicines are purchased and distributed and how medicines are managed in the health system.

Medicines financing - Access and availability are also affected by how much money the government can allocate to medicines, pricing policies, financing schemes (such as insurance programmes and user fees) and medicine donations.

Production and trade - Activities ranging from repackaging to formulation of products to developing new medicines are important in assessing the pharmaceutical sector. Implementing TRIPS flexibilities in public health can increase access to medicines.

Rational use of medicines - Medicines policies can often have greater impact with effective use of strategies to improve the prescribing and dispensing practices of health workers. Key strategies include standard treatment guidelines, curricula and continuing education programmes on essential medicines concepts, medicines information centres and public education campaigns.

Level II facility based core indicators

- 11 -

Level II facility-based core indicators

The Level II facility core outcome indicators support the Level I structure and process indicators by providing specific data about important pharmaceutical outcomes. This set of indicators requires field surveys. For data to be collected accurately and reliably, attention must be paid to appropriate survey design, sampling and data gathering techniques. In selecting the core outcome indicators, consideration was given to the need to obtain the most relevant information from as limited a data collection process as possible. There are 17 survey forms to be completed during a survey (Annex 7).

These indicators measure the degree of attainment of strategic pharmaceutical objectives: improved access, quality and rational use. Access is measured in terms of the availability and affordability of essential medicines, especially to the poor and in the public sector. Measuring the actual quality of medicines by testing samples can be expensive. Instead, the presence of expired medicines on pharmacy shelves and the adequacy of handling and conservation conditions of medicines are used as indicators of quality. Finally, rational use is measured by examining prescribing and dispensing habits and the implementation of key strategies such as standard treatment guidelines (STGs) and essential medicines lists (EMLs).

Level II indicators are measured in public health facilities, private drug outlets, and in warehouses supplying the public sector.10 Surveys of 30 public health facilities and their dispensaries are used to gather information about availability of essential medicines, medicine prices, stockout duration, the adequacy of conservation conditions, affordability, geographical accessibility, prescribing and dispensing habits, and presence of guidelines. A similar survey of 5 warehouses supplying the public sector also examines availability, stockout duration, and adequacy of conservation conditions. Surveys of 30 private drug outlets assess availability, affordability, geographical accessibility and medicine prices.

Survey Forms 1—17 have been developed to obtain data from survey sites. Box 1 summarizes the Level II indicators and lists the corresponding survey forms. The indicators are described in more detail in Annex 2a. Information on data collection and calculation can also be found on the respective survey forms. The sampling framework and survey methodology are described below. Summary Forms 1—4 provide a simple method of combining facility results to obtain national indicators (a spreadsheet is supplied with this material).

10 For the purposes of the Level II survey package, a private drug outlet is a permanent retailer selling

medicines, whether a pharmacy, drug seller, drug store, or chemical seller; a warehouse is a central, regional, or district warehouse supplying the public sector; and a public health facility dispensary or public health facility pharmacy refers to the medicines dispensing area of the public health facility whether a pharmacist is present or not.


- 12 -

Box 1 Summary list of indicators and corresponding survey form used to collect the data

Indicator Survey Form

Access

1 Availability of key medicines in public health facility dispensaries, private drug outlets and warehouses supplying the public sector

1, 10, 15

2 % of prescribed medicines dispensed or administered to patients at public health facility dispensaries

6

3 Average stockout duration in public health facility dispensaries and warehouses supplying the public sector

4, 16

4 Adequate record keeping in public health facility dispensaries and warehouses supplying the public sector

4, 16

5 Affordability of treatment for adults and children under 5 years of age at public health facility dispensaries and private drug outlets

3, 12

6 Price of key medicines in public health facility dispensaries and private drug outlets 2, 11 7 Price of paediatric medicines in public health facility dispensaries and private drug

outlets 2, 11

8 Average cost of medicines at public health facilities and private drug outlets 6, 14 9 Geographical accessibility of public health facility dispensaries and private drug

outlets 6, 14

Quality

1 % medicines expired in public health facility dispensaries, private drug outlets and warehouses supplying the public sector

1, 10, 15

2 Adequacy of conservation conditions and handling of medicines in public health facility dispensaries and warehouses supplying the public sector

5, 13, 17

Rational use of medicines

1 % medicines adequately labelled at public health facility dispensaries and private drug outlets

6, 14

2 % patients knowing how to take medicines at public health facility dispensaries and private drug outlets

6, 14

3 Average number of medicines per prescription at public health facility dispensaries and public health facilities

6, 7

4 % patients prescribed antibiotics in public health facilities 7 5 % patients prescribed injections in public health facilities 7 6 % prescribed medicines on the essential medicines list at public health facilities 7 7 % medicines prescribed by generic name (INN) at public health facilities 7 8 Availability of standard treatment guidelines at public health facilities 8 9 Availability of essential medicines list at public health facilities 8 10 % tracer cases treated according to recommended treatment protocol/guide at public

health facilities 9

11 % prescription medicines bought with no prescription 14

Other information

1 % of facilities that comply with the law (presence of a pharmacist) Sections A, C 2 % facilities with pharmacist, nurse, pharmacy aide/health assistant or untrained staff

dispensing Sections A, C

3 % facilities with doctor, nurse, trained health worker/health aide prescribing Section B 4 % facilities with prescriber trained in RDU Section B


- 13 -

Survey design and data collection (Level II - facility survey)

The survey of Level II indicators is a very important part of monitoring the pharmaceutical sector because these indicators measure the outcome and impact of pharmaceutical programmes in a country. The Level II core outcome indicator surveys are designed to obtain relevant information from as simple a data collection process as possible. There are 17 survey forms to be completed (Annex 7).

Surveys of public health facilities, private drug outlets/pharmacies, and warehouses supplying the public sector are required. Small samples of data and simple survey techniques will be used to collect the quantitative and qualitative information needed for these indicators.

In order to estimate indicators accurately and reliably, it is important to follow specific procedures for drawing samples and gathering data, to reduce bias and so that the study population is more representative of the reference population.

Larger samples are more costly but give more precise results. Sample size is therefore a balance between what is desirable and what is feasible. The best sample size will be the smallest one that will result in estimates with the desired degree of precision. Sampling bias and error in sampling are discussed in Annex 2b.

For data to be collected accurately and reliably, attention must be paid to appropriate sampling and data gathering techniques.

Adequate preparation is needed and those who will gather the data must be trained. Key pointers are discussed below.

Selecting the geographical areas

The survey should be conducted in five geographical areas. A geographical area can be a district, municipality, or province, see Box 2. Selection can occur before the training or during the training with the participation of the data collectors.

If appropriate data are available, select the largest or capital city, the most rural or lowest income-generating area, and randomly select three other geographical areas from the remaining regions. If data are not available, select the capital city and randomly select four other areas from the remaining regions.

In selecting the five geographical areas, the urban/rural population split can be taken into consideration at this level of sampling or at the next level (facility sampling) depending upon the size and make-up of the geographical areas. In some countries, these areas may be predominantly urban, predominantly rural or they may be mixed depending upon the size of the geographical area sample unit. Most countries have official statistics on the degree of urbanization. However, it may be more practical to take the urban/rural split into consideration when sampling facilities within a geographical area.


- 14 -

In a number of countries major logistical constraints, such as transport, time, budget and security concerns necessitate excluding some parts of the country from the data collection process. When reporting results of the study, it is important to explicitly state any parts of the country that were excluded from the sampling in order to acknowledge possible bias.

There may also be difficulties in countries with large nomadic or mobile populations where there are very few fixed facilities that are operational or used throughout the year. Often in such areas facilities only function in some seasons or there are mobile facilities. Efforts should be made to include these regions if possible. Excluding them may exclude a part of the population for whom access to health care is difficult. The degree of significance in terms of the national picture will depend upon the proportion of the population that such groups represent. Any access difficulties this group has cannot be identified separately unless they represent the whole geographical area.

Box 2 Summary of survey areas and sites

• Identify five geographical areas o 1 should be the largest or capital city o 1 should be among the lowest income-generating areas o 3 others should be randomly selected

Selecting public health facilities

To collect most of the indicators for access, quality and rational use, a total of 30 public health facilities need to be selected from the five geographical areas identified (see Box 3). These public health facilities should cater to general outpatients and they should have a pharmacy or a medicine-dispensing area. Within each geographical area, choose six public health facilities providing outpatient services. One of the selected public health facilities should be the main (biggest) public hospital in the area. Of the other five facilities, at least one should be a primary/rural health centre or lowest-level public health facility. The remaining ones should be chosen randomly from all middle-level primary and other public health facilities. The names and affiliations of all facilities in a region can usually be determined at the central level, so this sampling can be done centrally. A number of facilities can be identified as replacements if the facility selected in sampling is not available, has too few patients, or has totally inadequate records. If replacement facilities are used, data collectors should note the reason for not surveying the first facility selected.

If resources allow, countries may choose to add surveys of private, nongovernmental or mission health facilities to the core package. If the country situation is such that at least 25% of primary health care is delivered by facilities other than public health facilities — such as missions, NGO, social security, etc. — the public health facility forms can be adapted for use at these facilities as well. Any surveys done at other facilities should be in addition to the surveys carried out at public health facilities. Likewise, drug outlets/pharmacies/dispensaries connected to these facilities can be included in the survey. The private pharmacy/drug outlet survey forms can be adapted for use in these drug outlets.


- 15 -

Box 3 Selecting facilities from 5 geographical areas

Public health facilities - Each facility should cater to general outpatients and have a medicine-dispensing area. Total for country: 30 public health facilities Per geographical area: 6 public health facilities

1. The main or biggest public hospital in the area 2. One primary/rural health centre or lowest-level public health facility 3. Four middle-level public health facilities, randomly selected

Private drug outlets Total for country: 30 private drug outlets Per geographical area: 6 private drug outlets

1. Select the private drug outlet closest to each public health facility included in the survey

Central/regional/district warehouses Total for country: 5 warehouses Per geographical area: 1 warehouse

Making random selections

Box 4 provides an example illustrating the process of randomly selecting public health facilities from the capital city. The same process is used to select public health facilities in each of the five geographical areas included in the survey.

Box 4 Randomly selecting public health facilities from the capital city

Step 1: Select the first facility. From a list of all public health facilities in the capital city, select out the main or biggest public hospital.

Step 2: Select the second facility. Identify all of the primary health centres or lowest-level public health facilities and randomly select one.

Step 3: Number the remaining middle-level primary and public health facilities that are not hospitals or lowest-level public health facilities.

Step 4: Calculate the sampling interval. For example, if there are 303 middle-level health facilities in the capital city and 4 are to be chosen, the sampling interval is calculated by dividing the total number of facilities by the number to be selected: 303 ÷ 4 = 75.75

Step 5: Identify the third, fourth, fifth and sixth facilities • Choose a random whole number between 1 and 75.75, for instance 35. • The third facility to be chosen will be the one numbered 35. • Add the sampling interval to the randomly chosen number: 75.75 + 35 = 110.75 = 111

(always round up). The fourth facility is the one numbered 111. • Continue until all 6 facilities from the capital city have been chosen, i.e. the fifth

facility is: 75.75 + 110.75 = 186.5 = 187; the sixth facility is: 75.75 + 186.5 = 262.25 = 263


- 16 -

Selecting private drug outlets

Select the closest private drug outlet to each public health facility surveyed (see Box 4). It may be possible to do this selection centrally from data from the ministry/department of health, the chief pharmacist and/or the national pharmacy association/council. However, information may be outdated and you may find that the “map and terrain” do not match. Data collectors may need to select the private drug outlets to be surveyed after arriving in the field.

Depending on the country situation, other types of drug outlets may be surveyed as well. If another sector of drug outlets (other than private drug outlets and dispensaries/pharmacies connected to public health facilities) provides at least 25% of primary health care medicines, the private pharmacy/drug outlet forms can be adapted for use in these drug outlets. Surveys of other drug outlets should be done in addition to the surveys completed at public health facility pharmacies/dispensaries and private drug outlets.

Selecting warehouses

Select one central/regional/district warehouse in each geographical area for inclusion in the survey (see Box 3).

Sampling patients for data collection

To measure the rational medicine use indicators, general outpatient encounters from health facilities should be used. The patients to be sampled should be restricted to general illness encounters, representing a mix of health problems and ages. The indicators from the facility survey have limitations when applied to well-child visits, pre- and post-natal visits, specialist consultations, or even separate clinics for adults and paediatric cases because treatment practices are different and results may be difficult to interpret.

The survey will include both retrospective and prospective sampling. In some cases only one approach will be possible, but elsewhere, there will be a choice. Prospective sampling can introduce bias due to seasons, variations in staffing, inconsistencies in the supply cycle etc.

Retrospective sampling

The survey should retrospectively sample the previous 12-month period. There are several ways of sampling. Random selection of patients is illustrated in Box 5.


- 17 -

Box 5 Simple method for selecting 30 patients from the general outpatient list

If there are 5000 patients treated during the entire period and the sample size is 30:

• Calculate the sampling interval by dividing the total number of patient encounters on the general outpatient list for the year by the number of patients to be selected. For example, if there are 5000 patient encounters on the general outpatient list covering the previous 12 months and 30 patient encounters are to be selected: 5000 ÷ 30 = 166.6. Sampling interval is 166.6. (5000 ÷ 30 = 166.6 is the sampling interval)

• Select the first patient using a table of random numbers or just point to one patient at random. This is the 1st patient, add 166.6 to get the next patient and so on until 30 patients is reached.

Another method of sampling is by the following chronological sampling method described in Box 6.

Box 6 Retrospective sampling: Selecting 30 patients from general outpatient records covering 12 months (365 days)

A. Chronological sampling method • Calculate the sampling interval by dividing the number of days covered by the

outpatient list by the number of patients to be selected: 365 ÷ 30 = 12.2 days. • Number each day covered by the outpatient list. • Randomly select one patient encounter between day 1 and day 12.2. • Each subsequent encounter is selected by adding the sampling interval (12.2) to the

previous total and rounding up. In other words, select one patient encounter from the first day that was randomly selected (example 1st patient was selected from day 3, skip the sampling interval, and select one patient from day 15 (3 + 12.2 = 15.2 16 always round up). Second patient is selected from day 16 Third encounter = 15.2 + 12.2 = 27.4 = 28th day Fourth encounter = 27.4 + 12.2 = 39.6 =40th day Fifth encounter = 39.6 + 12.2 =51.8 = 52nd day

To choose a patient encounter from the patient list for each selected day, pick a random number between 0.0 and 1.0, multiply this number by the number of patient encounters on the list, and round upwards. For example, if you choose 0.4 and there are 18 patients on the list for the selected day: 0.4 x 18 = 7.2 = 8 (always round up). Review the 8th patient encounter when completing the survey forms provided the patient meets the inclusion criteria. If the patient does not meet the inclusion criteria, select the next patient on the list who does. Repeat this process, selecting one patient encounter from each selected day until 30 patient encounters have been reviewed.

Patients for Survey Forms 7 and 9 are selected retrospectively. For retrospective sampling the following steps should be undertaken:


- 18 -

• Confirm the availability and accessibility of medical records. Possible sources of retrospective prescribing data include clinic registers, treatment logbooks, patient/family files and retained prescription forms. The data collector must know where to locate these records in the facilities and how to use them.

• Identify the study period to be covered. Because of seasonal differences, variations in staffing, inconsistencies in the supply cycle etc., the survey should, as far as possible, cover the 12-month period prior to the date of data collection. If records during this period cannot be found or do not meet the sampling requirements, the study period can be shortened, making sure that there are no missing records during the period covered. This can be done by checking that all the months and days are represented in the record.

• Select patient encounters and extract data

Survey Form 7 From general outpatient treatment records, select 30 patients with any diagnosis seen during the last 12 months. Use either the simple or chronological sampling method described in Boxes 5 and 6. If records are not available, 30 patients may be selected from current treatment records provided there are sufficient records to randomly select the patients. Prospective sampling can also be used.

Survey Form 9 From general outpatient records or under-5 ledgers, select:

• 10 patients under 5 years of age seen during the last 12 months for diarrhoea,

• 10 patients under 5 years of age seen during the last 12 months for pneumonia, and

• 10 patients of any age seen during the last 12 months for acute respiratory tract infection (ARI).

If the selected patient encounter does not meet the inclusion criteria (i.e. does not have the appropriate diagnosis) select the next patient on the list who does.

Prospective sampling11

Prospective sampling is used in Survey Form 6. Interview 30 patients leaving the dispensing area/pharmacy or leaving the facility after they have been treated and received medicines to see how many of the prescribed medicines were dispensed, if the medicines are adequately labelled (the label should contain the medicine's name and how it should be taken), if the patients know how to take their medicines (patient knows dosage and duration of all dispensed medicines), and how much the patient paid out-of-pocket for medicines and in non-diagnostic fees. Patients may be interviewed consecutively or as convenient.

It is important to know how to identify the patients to be included and to construct a system that will allow patients to be interviewed without disrupting the normal activities of the facility.

Planning the data collection is also important. Throughout the day, facilities tend to have peak times when collecting enough interviews will be easy and low periods when there will not be enough patients to carry out the surveys effectively. 11 Prospective methods for sampling encounters. In: How to investigate drug use in health facilities,

pp. 63-65. Geneva, World Health Organization, 1993.


- 19 -

Summary of Level II indicators measured by facility

Table 1 gives the overview of the facilities to be visited and indicators to be measured during the Level II survey (see Annex 7, for corresponding survey forms).

Table 1 Study facilities, indicators and forms

Indicators Public health facilities (n=30)

Private drug outlets (n=30)

Warehouses (n=5)

Access Availability of key medicines

Check availability of 15 medicines (P)* SF 1



% medicines dispensed or administered

Exit interview 30 patients SF 6

Average stockout duration

Review stock cards of 15 medicines (R) SF 4


Adequate record keeping



Affordability of treatment

Check price of medicines to treat pneumonia and another condition (P) SF 3

Check price of medicines to treat pneumonia and another condition (P) SF 12

Price of key medicines Check price of 15 medicines (P) SF 2

Check price of 15 medicines (P) SF11

Price of paediatric medicines

Check price of paediatric medicines (P) SF 2

Check price of paediatric medicines (P) SF11

Average cost of medicines



Geographical accessibility of dispensing facilities



Quality % medicines expired Check if there are

expired medicines (P) SF 1

Check if there are expired medicines (P) SF 10

Check if there are expired medicines (P) SF 15

Adequacy of conservation conditions and handling of medicines

Check conditions using checklist (P) SF 5




- 20 -

Table 1 Study facilities, indicators and forms

Indicators Public health facilities (n=30)

Private drug outlets (n=30)

Warehouses (n=5)

Rational Use of Medicines % medicines adequately labelled

Exit interview 30 patients (P) SF 6


% patients knowing how to take medicines



Average number of medicines per prescription

30 patient records (R) SF 6 & 7

% patients prescribed antibiotics

30 patient records (R) SF 7

% patients prescribed injections


% of prescribed medicines on EML


% medicines prescribed by generic name (INN)


Availability of STGs Check at 30 facilities (P) SF 8

Availability of EMLs Check at 30 facilities (P) SF 8

% tracer cases treated according to recommended treatment protocol/guide

30 cases (10 each of diarrhoea, pneumonia and ARI) (R) SF 9

Prescription medicines bought with no prescription


Other information % of facilities that comply with the law (presence of a pharmacist)

Check at 30 facilities (P) Section A

Check at 30 facilities (P) Section C

% facilities with pharmacist, nurse, pharmacy aide/ health assistant or untrained staff dispensing

Check at 30 facilities (P) Section A

Check at 30 facilities (P) Section C

% facilities with doctor, nurse, trained health worker/health aide prescribing

Check at 30 facilities (P) Section B

% facilities with prescriber trained in RDU

Check at 30 facilities (P) Section B

* (R): retrospective review; (P): prospective review.

Preparing the survey (Level II - facility survey)

- 21 -


The survey of Level II indicators is a very important part of monitoring the pharmaceutical sector because these indicators measure the outcome and impact of pharmaceutical programmes in a country. Adequate preparation is needed and data collectors must be trained.

Coordination and survey coordinator

A national coordinator should be selected to take charge of the overall coordination of the Level II survey, to oversee the survey process, data analysis, reporting and presentation of results. The coordinator should be someone who is knowledgeable about the pharmaceutical sector and experienced in conducting surveys. A thorough working knowledge of the pharmaceutical sector will help to ensure that important aspects of the sector are not overlooked in planning for the Level II survey. The results of the Level I questionnaire will also inform the planning process.

The coordinator has to go through the manual carefully, covering concepts of assessment, monitoring, indicators and how to carry out a systematic survey. The coordinator should also go through the sections of this manual on survey design and data collection, and on how to prepare for the training and conduct the actual survey.

Any group within a health ministry/department involved in pharmaceutical activities, such as a pharmaceutical service management group, medicines regulatory authority, or national medicines policy office can coordinate the Level II survey activity. It is also possible for other groups such as NGOs, professional groups and acadaemia to initiate the activity. WHO, through its country and regional offices, and other donor agencies may also be able to provide assistance. A complete coordinator checklist is provided in Annex 3. The roles and functions of the coordinator include:

• Communicating with government officials and other local agencies to gain approval for the survey and to request personnel who will do the survey;

• Communicating with officials and health facilities to be visited for the field test and for the actual survey;

• Selecting geographical areas and identifying facilities to be surveyed;

• Allocating budget and, if necessary, requesting financial (Annex 4,) and technical support. Be sure that main components, such as data collector training, the survey itself, including transport and per diem details, analysis and dissemination of results, are covered;

• Coordinating and identifying sources of information for the Level I questionnaire on structures and processes of the country pharmaceutical situation;


- 22 -

• Identifying country-specific items on the survey forms, for example, the key basket of medicines, treatment guidelines;

• Preparing all the necessary materials for training, the field test, and survey of Level II indicators;

• Providing training and arranging the field test exercise for Level II data collectors;

• Supervising the actual survey of Level II indicators;

• Checking that survey forms are completed correctly and in full;

• Checking the computations on the survey forms;

• Coordinating completion of the summary forms, analysing results, writing of the report, and presenting/giving feedback of the results to appropriate groups.

Tailoring the survey forms to the country situation through choosing key medicines, selecting tracer conditions and identifying treatment protocols

In addition to selecting geographical areas and facilities to be included in the survey, there are a number of items on the survey forms – such as the basket of key medicines, treatment protocols and tracer conditions – that need to be identified at the national level.

The survey forms used to collect data on Level II indicators are designed to be tailored to country situations (Box 7). Tailoring must be done at the national level and country-specific features pre-printed on the survey forms and discussed with data collectors during training. The survey forms also have several places where countries can indicate additional data to be collected without affecting the indicator outcomes.

Box 7 Items on survey forms to predefine at national level

Survey Form Item

1, 2, 4, 10, 11, 15, 16

Identify basket of key medicines and paediatric medicines

5, 13, 17 Review language so it fits terminology commonly used. During training ensure data

collectors are using uniform criteria for evaluating items on the checklists for conservation and handling conditions

3, 12 For affordability, identify treatment of choice for pneumonia and additional condition with treatment of choice

6, 14 Define the requirements for adequate labelling on medicine dispensed and standard patient knowledge if different from those listed

7 Define what will be considered as an antibiotic, and injection Develop guidelines for data collectors on categorizing generic versus brand medicines

8 Identify the acceptable guidelines and medicines lists; obtain 1 copy of each for each survey team

9 Define 1st-line antibiotics for mild/moderate pneumonia and identify any optional conditions and medicines that will be used to measure recommended or non-recommended practices


- 23 -

Selecting a basket of key medicines

(Survey Forms 1, 4, 10, 15 and 16)

A list of 15 key medicines used to treat common health problems must be selected to measure availability, presence of expired medicines, medicine price and stockout duration. It is important to select key medicines that are basic requirements at all levels of health care. When selecting the medicines, list the 15 most common conditions treated at primary health care level and choose medicines used to treat these conditions. The chosen medicines must be:

• on the national essential medicines list

• the most important therapeutically and based on national treatment guidelines or at least on the consensus of experts

• the most widely used of the medicines meeting the above criteria

• medicines expected to be available at all primary health care facilities at all times.

The basket of key medicines can be selected systematically applying the principles above by referring to official morbidity data for adults and children, and following this step-by-step guide:

1 List the top 15—20 morbidity's for adults and children.

2 Remove from that list conditions that would not be treated as general outpatient cases at primary health care services.

3 For each remaining morbidity, assign the most important medicine that corresponds to the applicable STG. Be sure to include medicines for both adults and children.

4 The resulting list will probably be longer than 15 medicines, prioritize and reduce the list of key medicines to reflect the principles listed above.

5 Revisit the list and ensure that important medicines used for the alleviation of common symptoms and important preventive medicines have not been excluded. If they have, make appropriate substitutions in order to have a list of 15 medicines.

Medicines that are known to be problematic should not be included in the basket of medicines as this will disproportionately reduce the value of this indicator. Instead, such medicines can be monitored separately. Likewise, other medicines that may be of interest, but do not meet all the principles above could be monitored separately. This enables additional data to be collected with very little effort and without adversely affecting the results.

A model list of key medicines is presented in Table 2. This list can be modified based on important health problems in the country. Once the medicines have been selected, pre-print them on the survey forms.


- 24 -

Table 2 Model list of key medicines

diarrhoea oral rehydration salts (ORS), co-trimoxazole tablets acute respiratory tract infection

amoxycillin, co-trimoxazole tablets, procaine penicillin injection, paediatric paracetamol tablets

malaria chloroquine tablets anaemia ferrous salt + folic acid tablets worm infestations mebendazole tablets conjunctivitis tetracycline eye ointment skin infection iodine, gentian violet or local alternative fungal skin infection benzoic acid + salicylic acid ointment pain acetylsalicylic acid or paracetamol tablets prophylactic retinol (vit. A) ferrous salt + folic acid tablets

Identifying medicines to be considered as injections and antibiotics

(Survey Form 7)

To complete this form consistently, data collectors also need to use common criteria for identifying which of the prescribed medicines to classify as injections and antibiotics. Immunizations and injectable contraceptives should not be counted as injections. Definitions of medicines considered as antibiotics must be discussed. Antimicrobial agents are not always classified in an identical way. Indicators for antibiotic use can be sensitive to certain medicines, especially in places with high incidence of parasitic infections, such as malaria or tuberculosis. Medicines such as antiprotozoals and anthelminthics are also usually placed in a different category of antibiotics. How to classify topical antibiotics widely used in areas where trachoma, bacterial conjunctivitis and bacterial skin infection are common will also need to be considered.

Identifying medicines of choice to treat pneumonia in adults and children

(Survey Forms 3 and 12)

One indicator of medicine affordability uses the price of treatment. The STGs to treat moderate pneumonia and another condition in adults and children who are not hospitalized should be identified. The STGs to be used must be produced by an unbiased organization. Ideally, these should be officially endorsed and used by the ministry/department of health.

For adults and children, identify the medicine of choice and the recommended dosage preparation based on the guidelines. For example, from the WHO Model Formulary12 and WHO guidelines13 the following medicines and dosage preparations are recommended for pneumonia (other dosage preparations can be chosen if these are not available on the market):

12 WHO, 2006. WHO model formulary. Geneva, World Health Organization 13 WHO, 2005. Pocket book of hospital care for children. Geneva, World Health Organization.


- 25 -

Adults: Amoxicillin 0.5-1g every 8 hours (capsule/tablet)

Children: Amoxicillin 25 mg/kg 2 times a day

The medicines of choice, their preparations, and the number of units needed to complete treatment should be pre-printed on the survey forms.

If the WHO-recommended guidelines will not be used, the treatment guidelines for pneumonia and another condition must be carefully specified with input from national experts. Selected treatment guidelines must be developed by an independent group and not influenced by pharmaceutical promotion.

Identify standard criteria for adequate labelling and patient knowledge


The standard minimum criteria for adequate labelling of medicines dispensed to patients have to be set before the survey is carried out and discussed with surveyors. Minimum criteria should at least be to have the name, dosage and duration of treatment written on the package given to the patient. The patient/caregiver should at least know the appropriate dosage and duration of medicines dispensed. These criteria can be adjusted as appropriate for the survey population.

Identifying unit price of medicines to treat outpatient pneumonia in adults and children


The unit price of each medicine at the facility or the price charged to patients should be recorded, including the price of a syringe, if applicable. The lowest-priced brand or generic equivalent medicine should be used. If there are flat charges paid for each medicine given to patients, then this amount should be recorded as the price of the medicine. Indicate if medicines are given free of charge.

Instructions in obtaining drug prices of global and regional drug list [paid by the patient or paid by the facility]


Attention should be paid to ensure collection of unit prices of drugs' preparation unit, indicated in column B, e.g., if the price refers to a medicines sold in 30-tablet boxes, the value of the box should be divided by 30 to obtain the price of one tablet.

All countries should obtain the prices of drugs in the global list. Countries are also encouraged to get the prices of key drugs in the regional lists (Africa, Eastern Mediterranean, South America, Central Asia, South Asia and the Western Pacific). These lists are included on the CD.


- 26 -

Training data collectors

- 27 -


Selecting data collectors

Based on experience, the most effective data collectors are those with clinical experience such as physicians, nurses, pharmacists or paramedical staff. Health ministry/department staff and temporary employees with some health-related background and experience are possible candidates. It can also be useful to select data collectors from different parts of the country to reflect language differences and enable a more diverse sampling of geographical areas–this could also resolve any concerns data collectors may have about travelling to unfamiliar areas. In all, 10 data collectors will be needed–one team of two data collectors for each geographical area.

Data collection is the most crucial part of the monitoring process. For accuracy and reliability of information and indicator measurement, the data collectors must be well trained. Training should focus on ensuring data collectors have a common understanding of the required information and know how to gather the data and complete the survey forms in a standardized fashion.

Training components

The training will be in three parts (Box 8).

Box 8 Sample training schedule

Part I: Briefing on survey forms and patient sampling (1st day - half day) Part II: Field test (2nd day - half day) Part III: Calculation, debriefing and logistical issues (3rd day)

Note: All data collectors should be trained together

• How to use Survey Forms 1—17. Copies of these forms must be available for

discussion and use during training.

• Ethical issues and confidentiality when interviewing people (professionals and users), and handling documents (administrative and clinical).

• Sampling exercises using copies of actual general outpatient lists and patient records. These can be obtained from nearby facilities. It is advisable to keep all the records and materials used in training for future use.


- 28 -

• Exercises on how to calculate stockout duration using copies of stock cards

• Field tests in public health facilities and private drug outlets to familiarize data collectors with the procedures they will use in the field.

Note that facilities visited during the field test and those that will be included in the actual survey should be contacted and the permission of the facilities and appropriate officials sought in advance. Data collectors will need to be properly introduced to the facilities through an endorsement letter from an appropriate authority, e.g. the health ministry/department.

Training tools needed

• Slides 1—33, explained below and provided as Annex 6. • Survey Forms 1—17 (data collectors will need copies for discussion during

training, for practice during the field test, and for carrying out the actual survey).

• Samples of actual general outpatient lists, patient records and stock cards. • Copies of the National Medicines List. • This manual provides all the information on the indicators and the survey

process. Sampling methods crucial in getting valid and accurate data are also described.

Training slides introduced

Several slides have been prepared and can be used by the national coordinator to train data collectors in countries. There are two other sets of slides, to train coordinators and to introduce the activity to policy-makers and other parties/stakeholders. These sets can be adapted if there are other aspects and information that need elaborating.

Slide set to train data collectors - This is used by the national coordinator to train data collectors on the details of gathering data. Details of how to use the slides to train data collectors are in Annex 5 with a Powerpoint presentation in Annex 6. The trainer should review these annexes.

Trainers and survey coordinators can adapt/adopt and include more information as needs are perceived during training. The slide set presented in the annex can be adjusted according to needs. Examples are: the need to elaborate further on the concepts of monitoring, evaluation and assessment; to discuss the definition of each indicator; and how to transfer data in the summary forms.

Introduction

Slides 1—4 introduce the activity, why monitoring is important and who can use the results. Slides 5 to 8 discuss indicators as basic tools in monitoring, pointing out that it is important to use a standardized procedure to enable comparisons.


- 29 -

Survey logistics

This covers some of the logistics involved in the surveys, and adjustments and decisions to be made at the central level between the coordinator and those who will do the survey.

Slide 9 explains the logistics of the survey. Slide 15 (key medicines) can be completed before or during the training.

Discussion of the survey forms

Slides 13-25 review Survey Forms 1-17. A complete set of Survey Forms for discussion must be provided to each of the participants.

This includes practise on techniques for conducting retrospective and prospective sampling. An exercise using patient lists to select sample encounters is necessary. Data collectors should know how to get information from patient records both for general outpatients (Survey Form 7) and for cases of diarrhoea, pneumonia and ARI (Survey Form 9).

Slide set for training national coordinators (file is in the CD) - is developed to train the key person who will coordinate and survey the process and who will train the data collectors in countries. Usually regional or subregional training is conducted to train coordinators from different countries.

In this training the national coordinators are exposed to the principles of pharmaceutical assessment, the survey methodology and survey forms. They are also trained how to organize the survey process in their respective countries. This training equips them to train data collectors.

Slide set for policy-makers (file in the CD) - This set is included for national coordinators to present the Pharmaceutical Situation Assessment Survey proposal to national policy- and decision-makers. This set includes the survey objectives and rationale as well as a guide for the main methodological decisions (geographical areas and survey sites, key medicines and tracer conditions).

The field test

As part of the training, two to three public health facilities with dispensaries must be identified for the field test. The data collectors should be divided into three teams and each team assigned to one of the facilities. Private drug outlets should be identified in advance or selected by the teams on the day of the field test depending on which procedure will be used during the actual survey.

A notice or an agreement with facilities to be visited for the field test must be obtained before the training.

Each team should have a complete set of Survey Forms 1—17 and Summary Forms 1—4. They should familiarize themselves with the sampling procedures, data-gathering processes, and appropriate completion of each form. Each data collector should practise filling in a part of each survey form and calculating the values during the field test. Any problems encountered in collecting data or completing the forms should be


- 30 -

discussed during the debriefing. The coordinator should oversee the field test and review the completed forms to identify areas needing further explanation.

During the field test data collectors should take note of the checklist in Box 10.

Review the field test and calculations

Discuss the field test. Review all the survey forms and summary forms and discuss any difficulties in collecting data or completing the forms:

• Check that all computations are done properly. Each survey form must be checked for inconsistencies in data entry.

• Review sampling.

• Check the number of days covered by the review of stock records (Survey Forms 4 and 16).

• Discuss questions arising from the field test exercise.

• Ensure data collectors have a uniform approach to gathering data and completing forms.

At this point the data collectors should know how to use the Survey Forms and Summary Forms.

The survey

- 31 -

The survey

Notifying facilities to be visited

• Obtain permission from officials responsible for public health facilities. In some cases verbal permission may be sufficient; in others it may be necessary to send a formal letter. Box 9 provides a sample letter.

• Data collectors must be properly introduced to the facility (both for the field test and actual survey) through an endorsement letter from the health ministry/department or the WHO country office.

Travel schedule and logistics

The national coordinator should:

• Arrange the schedule and survey dates, including the necessary travel time to get to the selected geographical areas:

o there should be five teams. A team of two field workers should be assigned to each geographical area;

o all indicators from each public health facility and its dispensary can be collected in one and a half days, including the visit to a nearby private drug outlet and central warehouse;

o each team can complete its portion of data gathering in approximately nine days.

• Provide the necessary logistics: survey forms, calculator, and per diem or transport costs as applicable.

Box 9 Sample letter to facilities

Dear_______________________ Pharmaceuticals are important in maintaining and providing health services to the population. It is also important to know if the population has access to essential medicines of good quality and whether or not these medicines are being used properly.

The name and location of facility has been selected as one of the facilities to be included in a national survey to assess the pharmaceutical situation in name of country .

On date of the visit , our staff will be visiting your facility to gather some information on the availability of some key essential medicines and how these are being used. While at your facility, our staff will need to access list all areas where access is needed, i.e. the last 12 months of outpatient records, the last 12 months of stock records. Thank you for your cooperation. Sincerely,


- 32 -

• Make copies of all the survey forms. The cover sheet of the survey forms' packet lists the necessary number of copies.

• Indicate on the bottom of the Summary Form 1, Public health facility pharmacies/dispensaries, the names of the geographical areas included in the survey and the numbers assigned to facilities in each area. Assign the same number to the public health facility and its dispensary and to the nearby private drug outlet. Note that each survey form has a box in the upper right-hand corner. The facility number should also be recorded in this box.

Data collectors

On arrival at the facilities, data collectors will need to speak to the person in charge of the facility and make a brief tour in order to be able to understand patient flow, where to locate the desired patients, and the existence and location of the necessary records. It is essential that this is done before any data are collected otherwise incomplete or incorrect data might be gathered, wasting time or invalidating the results. A checklist for data collectors is provided in Box 10.

Box 10 Checklist for data collectors Check if completed

At the geographical area level (region, province, district)

1 Confirm that the necessary authorization from the relevant authorities has been received and check on local logistical and security issues. Obtain any necessary additional authorization letter

2 Identify the location of the facilities and the most efficient order in which to visit them

At the facility level (public health facility and private drug outlet)

1 Take an initial tour of the facility and talk with key staff to understand the patient flow, and existence and location of records

2 Determine the best order to complete the survey forms and which data collector will complete which forms

3 Confirm the availability and accessibility of records:

• Location of general outpatient encounter records

• Patient records

• Stock records

4 Decide on retrospective or prospective sampling for survey forms based on the availability and accessibility of records

5 For retrospective sampling identify the study period to be covered

The survey

- 33 -

After completing the actual survey, data collectors should submit a brief report on the data collection process and, in particular, anything that will be important for the report writer in interpreting the results. Data collectors should also note if they had to visit more facilities in addition to the ones actually included in the survey and record the reason facilities were skipped, whether due to a lack of available records or other problems.

Supervision of data collection

Supervision of the data collectors should aim to ensure that the agreed procedures and methods are being followed, and the data collected are complete and of good quality. It should also solve problems such as incomplete answers and omissions. The handling and storing of data collection forms before they are processed and analysed is important as well.


- 34 -

Data processing, analysis and reporting: What to do with results?

- 35 -

Data processing, analysis and reporting: what to do with the results

The coordinator should oversee the analysis of the data and the writing of the report. Guidance and a suggested report outline are provided below.

Data processing should be carefully planned. Even before data collection begins, the necessary resources and a person to encode/tabulate the data and do the computations must be identified.

Computation of Level II facility indicators

During the survey, the coordinator must check the accuracy of data and information reported as far as possible in the field. These should be checked for completeness and consistency before making the final calculations and analysis. Data coding and entry must be double-checked.

The data entered on the Survey Forms should be calculated following the instructions and formulas on each of the Survey Forms. All the calculations can easily be done manually. The completed results per facility/outlet should then be transferred onto the relevant Summary Form.

For Level II facility indicators, Summary Forms 1—4 (public health facility dispensaries, public health facilities, private drug outlets, and warehouses) should be used to calculate and analyse data, make graphs and tables, and prepare reports. Summary Forms 1—4 automatically compute values, including a summary of indicators, once data are entered and can also generate several graphics automatically. The Summary Forms are included in the CD.

Quality of data and information

The quality of the information that can be generated depends on the quality of data. The data must be checked during the field survey and missing information completed. The data must be checked again for completeness and consistency before the final calculation and analysis. Inconsistent data that cannot be corrected must be excluded. How the data have been coded and subsequently entered must also be checked when entering them into the Summary Forms.

Reporting biased data can also occur. Those involved in the survey or those providing information on Level I indicators may either be motivated to report only on the success of a programme or only the negative aspects of the existing situation. Those involved in collecting, analysing or presenting the data should bear in mind that the purpose of the exercise is to measure what is really going on in order to improve the situation.


- 36 -

Analysis and interpretation of Level I and Level II facility indicators

Using the information from the Level I questionnaire and data from Level II facility Summary Forms, the country pharmaceutical situation can be analysed and discussed. The information in the Level I questionnaire can be used to discuss the existing pharmaceutical sector structures, and what activities and strategies have been and are being implemented in the country. The results of the Level II facility survey should be able to provide a glimpse of the probable impact or outcome in terms of meeting the overall objectives in pharmaceuticals. Thus, for analysis of outcomes and impact, Level I indicators can be discussed in the light of the results and outcome measures of the Level II indicators. The results and information from Level II core indicators can be correlated with the information on processes and structures obtained from the Level I questionnaire.

The Level I questionnaire gives information on:

Existence of an NMP that provides a basic framework to organize and improve the pharmaceutical system and integrates medicines and other health policies. An NMP also assists in coordinating the functions and strategies of each component of the pharmaceutical system and implementation of these strategies.

Profile of the regulatory system such as legislation/regulations on medicine manufacturing, promotion and advertising, sales, distribution, dispensing and prescribing must be in place. A medicines regulatory authority should be able to efficiently regulate activities such as generic prescribing, dispensing and substitution and can help increase access to essential medicines in both the private and public sectors. Quality control of pharmaceuticals covers all activities to ensure patients receive safe, efficacious and high-quality medicines.

Medicines supply systems are affected by how medicines are purchased and distributed, and hence how medicines are managed in the health system, which has a significant impact on access and availability of essential medicines, especially at public sector facilities. Access and availability are also affected by how much money the government can allocate to medicines, pricing policies and financing schemes (such as insurance programmes and user fees).

How medicines are used - what structures and processes are available that can help to improve the prescribing and dispensing practices of health workers? Key strategies include STGs, curricula and continuing education programmes on essential medicines concepts, medicines information centres, and public education campaigns.

The Summary Forms for Level II facility survey/indicators are grouped by point of data collection (public health service facility, public and private pharmacy/drug outlet and public warehouse). In the Summary Form, the summary indicator - Worksheet 2 presents the summary of all indicators. Outcome measures are shown by areas of achievement - access, quality and rational use of drugs.

The sampling scheme of patients at each health facility and pharmacy/drug outlet limits the potential to compare one facility with another. Extending the sample to 100 patients per facility should allow such comparison. However if applicable and provided discussion is still focused on the national significance of the survey, each of the five geographical areas can be analysed and discussed. Regional comparisons may


- 37 -

be of interest where there is especially wide variation or contrast, particularly with a group of related indicators. Regional comparisons should be done sparingly as over-emphasizing the five regions included in the study will detract from the study’s significance as a national survey. However, the characteristics of each geographical area should be described.

Other measures that do not involve patient data can be compared, including affordability, availability, conservation condition, adequacy of storage and dispenser and prescriber profiles can be discussed across facilities and geographical areas. It is important to describe the general characteristics of each facility and the geographical area where they are sampled in such discussions.

Data analysis of Level II facility surveys can be divided into three topics using the data from the worksheet of summary indicators: access, quality of drugs and how drugs are used.

Access can be discussed in relation to the physical availability of key drugs and stockout duration, affordability and prices of medicines. It can also be discussed using data on access to health facilities. Public and private sectors can be compared.

• Availability of key drugs indicates the system's ability to cater to common disease conditions. The length of stockouts can provide information on the supply logistics in relation to these key drugs. Drugs dispensed to a patient are a good measure of access, especially at public health facilities.

• Affordability to treat pneumonia expresses how much a patient has to pay for treatment of an illness in terms of the number of day's wages of the lowest-paid government worker.

• Prices of a selected basket of medicines can be compared across pharmacies and drug outlets, across geographical areas and can also be compared to prices in other countries. International price comparison is a good tool for advocacy for policy on drug prices. The Health Action International web site can be used: http://www.haiweb.org/medicineprices/.

• Distance to health facility and cost to get there can also provide information on measure of access. These will be particularly useful when comparing facilities in urban and rural areas.

The storage and handling conditions of medicines and the expired medicines on dispensing shelves can provide an indication of the quality of drugs distributed to patients. Measuring quality by testing samples is costly for this level of assessment. These indicators are measured at pharmacy dispensing and storage areas at public and private pharmacies/drug outlets as well as at public warehouses. These can also be compared.

Drug use can be assessed by measuring how health workers are prescribing, such as number of drugs prescribed, magnitude of prescribing antibiotics and injections in the primary health care setting and general outpatient setting, use of drugs in the essential drugs list, generic prescribing, compliance with treatment guides and protocols. The amount of dispensing time and adequate labelling are indicators on dispensing practices that can be measured. Patients should know how to take their drugs. Inappropriate use of prescription drugs is also a measure in this survey. As mentioned earlier comparisons of the public and private sectors can be made.

Comparisons can also be made with similar studies done in the past and with results from studies done in other countries.


- 38 -

Limitations of the Level II facility survey

The survey has been designed to provide a picture of the national pharmaceutical situation in a country. The regions and facilities selected cumulatively represent the national situation.

The sample sizes used are not statistically large enough to make inter-facility comparisons. For patient care indicators, for example, a minimum sample size of 100 would be necessary in order to make comparisons between facilities. This survey uses a sample size of 30. However, providing that the majority of the data are collected and the results are statistically different, comparisons between geographical regions can be made. Regional comparisons may be of interest where there is especially wide variation or contrasts, particularly with a group of related indicators. Regional comparisons should be done sparingly as not all geographical regions are represented. Over-emphasizing the five regions included in the study will detract from the study’s significance as a national survey.

Indicator measure for Level II facility indicators

Indicator values are computed and reported as percentages (%). They are also reported as measures of central tendency (mean and median). The two are equivalent if data are distributed normally. Mean as the average value is sensitive to outliers, weighted towards skewed value. It is the best summary of values that are normally distributed. Median the middle value is resistant to outliers, and is a good summary of any distribution. The median value (rather than the mean or average) should be used in reporting the indicator at the geographical region and at the national levels.

A convincing way of presenting the dispersion of results is to use percentiles. Percentiles are points that divide all measurements in 100 equal parts. The median is the 50th percentile, i.e. the value below which 50% of measurements are positioned. As shown in Figure 2, the 25th and 75th percentiles are the boundaries of half of the values around the median, i.e. 50 % of the values are within the 25th (lower quartile) and 75th (upper quartile) percentiles. Presenting the 25th and 75th percentiles together with the median value provides a good summary of the overall spread of values and give a better summary of skewed data.

Figure 2: Use of median and percentiles to describe the dispersion of measurement


- 39 -

0%

20%

40%

60%

80%

100%

facility

medianvalue

0%

20%

40%

60%

80%

100%

facility

medianvalue

facility

% p

atie

nts

rece

ivin

g an

tibio

tics

med

ian

valu

e

Performance standards for Level II facility indicators

The target for indicators measuring the extent of adequate labelling, generic prescribing, the proportion of prescribed medicines dispensed, adherence to treatment guidelines and availability of key medicines is ideally 100%. However, internationally valid standards for other indicators, such as the average number of medicines per prescription, and the percentage use of antibiotics and injections are more complex and have not been established empirically. The optimal indicator values largely depend on disease patterns, policies and treatment guidelines, and therefore may vary from country to country and over time.

Group norms can be used to some extent in target setting, with regions and facilities able to compare themselves to one another. However use of group norms must be carefully considered since the norm may be wrong, as in the figure below where the median value is almost 45%. Although not empirically tested, it is a possibility that antibiotics are over prescribed, if the value at primary health care level is > 30%. Such a result and its use have to be analysed carefully however. Moreover, in the figure below, in most facilities, more than 40% of patients are prescribed antibiotics.

Figure 3: % patients receiving an antibiotic - distribution of facility results

Written report

A report should be prepared in order to disseminate the information obtained. This report is an important source of information and a basis for decisions on medicine policy and strategies. It should be persuasive and well prepared. The results must be discussed in a comprehensive and systematic manner, taking into account the objectives and strategies of the medicine policy. Clear recommendations should be included, based on the study findings.


- 40 -

After all the data have been processed and all the indicators calculated, the information should be analysed and presented in a report. Analysis can be done systematically by looking at each group of indicators and correlating the structures, processes and outcomes. Some examples are given in three key reference documents, Using Indicators to Measure Country Pharmaceutical Situations — Fact Book on WHO Level I and Level II Monitoring Indicators 14 and Indicators for Monitoring National Drug Policies15 as well as in How to Investigate Drug Use in Health Facilities”.16

The core indicators represent general measures of the current pharmaceutical situation. Reasons for improvements, deterioration or stagnation in implementation of the NMP can be deduced from the results. This will help to identify the inputs that have had a real impact and to focus attention, resources and efforts on areas requiring improvement. More detailed investigation and further analysis of a particular pharmaceutical component can be done if needed (many Level III approaches are available on the WHO Medicines website: http://www.who.int/medicinedocs/).

The data and results in themselves do not suggest the implementation of specific interventions, but can be used as a good take off point to discuss problems in more detail so that a focused strategy can be identified. The more accurately data were collected the more useful they will be. This can also direct the responsible group in identifying follow-up activities that can be done.

A report communicating the information obtained should be prepared by the monitoring unit. This report is an important source of information and a basis for decisions on medicine policy and strategies. It should be persuasive and well prepared. The results must be discussed in a comprehensive and systematic manner, taking into account the objectives and strategies of the medicine policy. The report should also be published.

The written report should not be long. It should be objective and summarize information in tables and graphics. The results should be presented in a logical order with a logical link between findings, analysis and recommendations. Data included in the report should name the source of information, i.e. exit interview of patients, retrospective sampling of records, household interview, etc. While not necessary to include in the report, raw data should be kept for reference. See Table 3 for a suggested outline.

14 Harvard Medical School and Harvard Pilgrim Health. Using indicators to measure country

pharmaceutical situations — Fact book on WHO Level I and II monitoring indicators. Geneva, World Health Organization, 2006. WHO/TCM/2006.2.

15 Brudon P, Rainhorn JD, Reich M. Indicators for monitoring national drug policies. Chapter 5, pp 46-47. Geneva, World Health Organization, 1993. WHO/EDM/PAR/99.3.

16 How to investigate drug use in health facilities, Selected drug use indicators. Chapter 4, pp 37-43. Geneva, World Health Organization, 1993. WHO/DAP/93.1.


- 41 -

Table 3 Suggested report outline

Section Notes 1 Table of contents • Include list of graphs and tables. 2 Acknowledgements 3 List of

abbreviations • Sort alphabetically.

4 Executive summary

• Summarize survey findings and recommendations.

5 Introduction Country profile • Basic geography (regions, provinces, districts).

• Data on population. • Basic economic indicators (presented in a table).

Structure of health and pharmaceutical systems (results from Level I can be used here)

• Brief overview of the health situation including key health indicators (presented in a table) the structure of the health system and how the basic mechanisms of medicine distribution, financing, etc. operate.

• Description of all pharmaceutical sectors: public, private, not-for-profit, NGO, informal.

• Total size (or best estimate) of the pharmaceutical market, split by public and private sectors.

• Inventory and overview of any previous pharmaceutical sector surveys.

• Brief overview of the NMP (when approved, summary of content, achievements/constraints).

6 Study design and methodology

• Brief summary including any country-specific considerations and a reference to this manual.

Study purpose and indicators

• Clear and realistic within the limitations of the design of the survey package.

Scope and limitations of the data

• General scope and limitations of the package and any additional limitations that have been introduced at the implementation level.

Sampling procedure • Concise, to the point description of the process, including any exclusions from the sampling process that might limit the interpretation of the results (e.g. exclusion of a region in the sampling due to security reasons or difficulty of travel; exclusion of the not-for-profit sector).

Problems experienced

• In carrying out the survey especially those which might have affected the data collection and placed limitations on the interpretation of the results. Include general problems identified in carrying out the survey at the national level and reports from the data collectors (e.g. difficulties in finding sufficient patients for the exit interviews; access to private drug outlets refused).


- 42 -

Table 3 Suggested report outline

Section Notes 7 Results and

analysis (any findings from Level I not already presented and results from Level II) • Policy • Access • Quality and

safety • Rational use

• Report results concisely and clearly • Use tables and graphs • Use headings and appropriate lead sentences to enable the reader

to quickly find information, e.g. “62% of the 15 key medicines were found to be available in the public health facilities”. More description and discussion can then follow

• The survey is primarily designed as a survey of the national situation and the report should reflect this. The report should not focus on reporting differences between the 5 geographic areas. Comparisons between the 5 areas should only be used where they are particularly interesting. Any geographic comparison should also include the national median

• See Analysis above for examples of how to graphically present national results

8 Interpretation of the results

• Make logical inferences based on the results of the survey and taking into account its limitations

• Link appropriate access, quality and rational medicine use indicators

9 Conclusions and recommended interventions and/or next steps

• Conclusions should be focused and clear • Recommendations and interventions should be realistic, limited

and focus on those areas where greatest impact can be achieved. They should identify the problem to be addressed and the proposed activity to address the problem

• Next steps should be as specific as possible listing: o Expected outcome o Timeline o Necessary (policy) decisions required and by whom o Who is responsible and who will carry out the

recommendation o Necessary budget and other resources

• Recommendations and next steps should reflect consultation with the Ministry of Health and input from the stakeholder workshop where possible

10 Annexes Completed level 1 questionnaire

Summary forms • Facilities' names should not be reported as the survey was not designed for interpretation of results by facility

• Original survey forms should be retained for archiving purposes. Interpretation of the coding should be retained with the forms

Sampling and country-specific items

• Lists of geographic areas, facilities (without coding), key medicines, definitions of adequate labelling, adequate knowledge, and other specifications to the survey forms made at the national level


- 43 -

General notes

• The report should be as concise as possible, presenting the information in a way that is understandable to a moderately informed reader. It should be easy to locate the methodology, key findings and recommendations quickly.

• Tables and graphs should be used to avoid long complicated narrative descriptions.

• General country, health and pharmaceutical data should be as up-to-date as possible. Generally data over 5 years old is not very informative unless part of a time-trend where more up-to-date data are also presented.

Presentation and discussion of results and related issues

The core indicators represent general measures of the current pharmaceutical situation. The results should be presented in meetings, seminars and other fora. Presentations should be comprehensive enough so that the audience will see how a problem in one component can affect another, but specific enough that the study’s findings can be used as a launching point for discussing the audience’s experiences and knowledge of problem areas and for identifying focused strategies to address problems.

The monitoring and assessment report should be presented to all parties involved in formulating policies and implementing pharmaceutical strategies such as medicines inspectors, central medical stores, private drug outlets, and local industry. NGOs, professional organizations and academia should also know the results. The discussion should be non-judgmental, pointing out both the positive and negative findings and recommendations. The presentation focus must be suited to the type of audience.

It is advisable to present the report separately to national, district/regional managers, and health facility levels. When presenting to a specific audience with a specific area of interest, provide a brief general summary of the overall results so the audience will know the entire picture and relate how a problem in one component can affect the other. The majority of the presentation, however, should be limited to issues that will be of interest and relevant to the specific audience. The presentation should:

• Present the issues clearly.

• Recognize what is well done and encourage further improvement in performance.

• Focus on the part the audience is responsible for, can influence or is directly concerned with so the discussion, consensus and recommendations are directed to specific actions, activities and strategies.

• Give enough time to discuss what did not work and potential changes that need to be implemented.

Health workers should be presented with the findings and given specific feedback on how they are performing and how/where they can make improvements. They must be encouraged to participate in the discussion and give their comments. As soon as the completed forms are received, it would be useful if an individual one-page summary of data was organized and sent to each health facility. Also, if the team of data collectors is skilled enough, they could provide the individual rapid report before leaving the research area.


- 44 -

A meeting with key managers in the district, regional and administrative areas should also be held.

Policy-makers at the national level can use the results to emphasize issues that can be addressed by a national strategy, to update current strategies, to define new ones, to reallocate resources or to adjust plans and targets. The discussion can also lead to decisions to put appropriate structures in place, to strengthen enforcement of policies and laws and to motivate/direct different levels and agencies of the health care system to take action or to improve performance.

Wide dissemination of the results is important to create awareness at different levels. The results can also be discussed with other stakeholders–civil society, professional organizations and academia–that can help in revising or selecting appropriate pharmaceutical strategies and can contribute to effective implementation plans.

Format of result presentation

Reports always have greater impact when results are presented clearly. Tables and graphs should complement narrative description. On first examination, it might appear that there are limited ways of presenting national results for a single country graphically. However, there are a number of ways of presenting interesting and meaningful graphs. The Summary Form automatically generates several graphics.

Graphics such as those in Figure 4 can be developed:.

• Bar charts comparing the same indicator in different sectors, e.g. availability of the basket of medicines in public facilities, private drug outlets and warehouses.

• Bar charts showing variability of facilities, e.g. variability of stockout duration in public facilities.

• Comparison of a group of outcome indicators such as rational drug use, e.g. comparison of rational medicine use indicators: % of patients prescribed antibiotics, % of patients prescribed injections and % of medicines prescribed on the essential medicines list.

• Presentation of the results of similar indicators in a table, e.g. prescribing of various medicines.


- 45 -

Figure 4: Presenting national level data graphically

Prescribing of drugs by diagnosis

Diagnosis

% prescribing of Diarrhoea Pneumonia

Respiratory Tract

Infections

Oral rehydration salts (ORS) 76.1

Antibiotics 50.24 55.6 75.5

Antidiarrhoeals 4.76

Availability of the basket of drugs in different sectors

0

20

40

60

80

100

public health facility public sectorw arehouse

private drug outlet

Variability ofstockout duration in public facilities

0

4

8

12

16

20

24

0 1-30 31-60 61-180 >180Duration of stock out (days)

Num

ber o

f fac

ilitie

s

Comparison of rational drug use indicators

0.0

20.0

40.0

60.0

80.0

100.0

% patientsreceiving antibiotics

% patientsreceiving injections

% drugs on EML


- 46 -

Annexes

- 47 -

Annexes

Annex 1: Level I questionnaire ............................................................................................49 Annex 2: Level II indicators .................................................................................................71 Annex 2a: Technical descriptions of Level II indicators................................71

Annex 2b: Issues, technical details, survey design and sampling for Level II indicators.......................................................................85

Annex 3: Coordinator checklist ...........................................................................................89 Annex 4: Example of budget allocation..............................................................................95 Annex 5: Guide on how to use the training slides (level II indicators)..........................97 Annex 6: Training slides for data collection ....................................................................103 Annex 7: Level II Survey Forms 1—17 ...............................................................................121

Annexes in the CD

Annex 1: Level I questionnaire



Annex 6a: Slides for training data collectors

Annex 6b: Slides for training survey coordinators

Annex 6c: Slides for policy-makers

Annex 7: Level II Survey Forms 1-17

Annex 7a: Regional List for prices of drugs (supplementary to SF2 & SF 11)

Annex 8: Summary Forms Annex 9: Sample report Annex 10: Letter to facilities


- 48 -

Annex 1 Level I Questionnaire

- 49 -

Annex 1: Level I Questionnaire Questionnaire on structures and processes of country pharmaceutical situations

Country: Date (dd/mm/yyyy):

Name of coordinator/principal respondent : E-mail address:

Position : Postal address :

Questions Responses Explanations

1. NATIONAL MEDICINES (DRUG) POLICY (NMP)

Please consult the health ministry, medicines regulatory authority and/or medicine service in answering the questions in this section.

1.1. Is there a National Medicines Policy (NMP) document? If no, skip to 1.4.

Yes No Don’t Know A national medicines (drug) policy document is a written expression of the government’s medium- to long- term goals and priorities for the pharmaceutical sector and the main strategies for attaining them.

a) If yes, is it an official or draft document?

Official Draft Don’t Know Mark “official” if the NMP document has been endorsed or officially adopted by the government otherwise mark “draft”.

b) What year was it last updated?

Year Indicate the year of last update whether the document is still in draft form or has been officially adopted.

1.2. Is there an NMP implementation plan that sets activities, responsibilities, budget and timeline?

Yes No Don’t Know

a) If yes, when was it last updated? Year

1.3. Is the NMP integrated into or included in the published/official national health policy/plan?

Yes No Don’t Know The national medicines policy is considered to be integrated into the national health policy/plan if the pharmaceutical sector priorities and strategies are specified in the health plan.

a) If yes, when was the national health policy/plan last updated?

Year

1.4. Has a national assessment/indicator study been conducted?

Yes No Don’t Know

a) If yes, which topics have been studied and when was the most recent study covering each topic conducted:

Overall pharmaceutical situation: Yes No DK Year

Rational use/prescription audit:

Yes No DK Year

Access (i.e. prices, affordability and/or availability) to medicines:

Yes No DK Year


- 50 -

Questions Responses Explanations 1.5. Is there a code of conduct that applies

to public officials and staff involved in pharmaceutical related activities or posts, such as persons working in pharmaceutical services, medicines regulation, procurement and supply of medicines and other pharmaceutical divisions of the health ministry?

Yes No Don’t Know This means an officially adopted/promoted document aimed at ensuring accountability and appropriate conduct, including transparency and good governance, that applies to public/government officials and staff involved in pharmaceutical related activities or posts, such as those persons working in pharmaceutical services, medicines regulation, procurement and supply of medicines and other pharmaceutical divisions of the health ministry.

2. REGULATORY SYSTEM

Please consult the medicines regulatory authority in answering the questions in this section. Specific information regarding medicines tested for quality control purposes and monitoring of adverse drug reactions may need to be obtained from the quality control laboratory or the responsible agency/department.

Regulatory authority

2.1. Are there legal provisions establishing the powers and responsibilities of the medicines regulatory authority?

Yes No Don’t Know This question is asking if there are legal provisions (or legislation) that describe the legal conditions under which the medicines regulatory authority should be organized and operate.

2.2. Is there an existing formal medicines regulatory authority?

Yes No Don’t Know This question is asking if there is a formal regulatory body with existing staff and a specific budget for conducting relevant medicines (drug) regulatory functions.

Mark “no” if medicines regulatory functions, such as registration and licensing, are performed on an ad-hoc basis by an office, group or department that performs other pharmaceutical service functions, such as supply management and procurement.

2.3. What are the sources of funding for the medicines regulatory authority:

Regular budget from the government: Yes No Don’t Know

Fees from registration of medicines:

Yes No Don’t Know

Other: Yes No Don’t Know

2.4. Are there legal provisions requiring transparency and accountability and promoting a code of conduct in regulatory work?

Yes No Don’t Know This question is asking whether there are legal provisions (or legislation) requiring the regulatory authority to:

• Define its policies and procedures in writing and publish the written documentation

• Give reasons for decisions to affected parties

• Account for its conduct and actions to individuals or groups and ultimately to the public and

• Follow a code of conduct in conducting its regulatory functions.


- 51 -

Questions Responses Explanations 2.5. Is the medicines regulatory authority

involved in regional/international harmonization initiatives?

Yes No Don’t Know Regional/international harmonization initiatives include the International Conference on Harmonization, regional harmonization on registration of medicines, etc.

Regulatory authorities are considered to be involved in these initiatives if they participate in meetings that discuss common regulatory matters with representatives of regulatory authorities from other countries, use a regional harmonized regulatory standard, etc.

2.6. Is there a medicines regulatory authority website providing publicly accessible information on any of the following: legislation, regulatory procedures, prescribing information (such as indications, contraindications, side effects, etc.), authorized companies, and/or approved medicines?

Yes No Don’t Know

Marketing authorization

2.7. Are there legal provisions for marketing authorization?

Yes No Don’t Know This question is asking if there are legal provisions (or legislation) that describe the legal conditions under which marketing authorization should be conducted.

Marketing authorization is an official document issued by the medicines regulatory authority for the purpose of marketing or free distribution of a product after evaluation for safety, efficacy and quality and/or after registration of a product for marketing.

2.8. How many medicinal products have been approved to be marketed? (count total number of unique dosage forms and strengths)

Number Tablets, capsules, injections, elixirs and suppositories should be counted in different strengths. For example, if Paracetamol (Brand X) 250 mg and 500 mg have been approved to be marketed, they count as two medicinal products because they are two unique strengths. Paracetamol (Brand Y) 250 mg and 500 mg are another two unique products.

2.9. Is a list of all registered products publicly accessible?

Yes No Don’t Know Registered products are medicine products that have been evaluated for quality, safety and efficacy and thence authorized for marketing. In order to be publicly accessible, a list should be available on the web or to anyone contacting the responsible authority.

2.10. Is there a computerized registration system that facilitates retrieval of information on registered products?

Yes No Don’t Know This refers to a systematic medicines registration process that makes the information on registered products more readily accessible and more easily updated by the responsible officials.


- 52 -

Questions Responses Explanations 2.11. Is the WHO Certification Scheme

certificate required as part of the marketing authorization process?

Yes No Don’t Know The WHO Certification Scheme on the Quality of Pharmaceutical Products Moving in International Commerce guarantees, through the issue of a WHO-type certificate by the certifying authority, the quality of pharmaceutical products entering international commerce. It is a simple administrative procedure that enables importing countries to obtain information on whether a product has been authorized to be placed on the market in the exporting country, and assure that the manufacturer has been found to comply with WHO standards of good manufacturing practices.

2.12. Is the INN used in the registration of medicines?

Yes No Don’t Know An INN (international nonproprietary name) is a common, generic name selected by designated experts to identify a new pharmaceutical substance unambiguously. INNs are recommended for worldwide use, destined to be unique and public property (nonproprietary).

2.13. Is there a functional formal committee responsible for assessing applications for registration of pharmaceutical products?

Yes No Don’t Know Mark “yes” if there is a committee composed of experts and specialists who review dossiers and applications for registration of medicines.

Licensing

2.14. Are there legal provisions for licensing of the following:

This question is asking if there are legal provisions (or legislation) that describe the legal conditions under which manufacturers, wholesalers and distributors, and importers and exporters are subject to evaluation against a set of requirements and issued a permit to operate (licence) authorizing them to undertake specific activities.

Manufacturers: Yes No Don’t Know

Wholesalers or distributors:

Yes No Don’t Know A wholesaler is a company that buys goods from a manufacturer or importer and sells them to retailers. The wholesaler may be an agent for one company only or deal with products from several companies. Manufacturers may also be wholesalers for their own products. In some countries, pharmacies may also have a wholesaler license. Distributors include wholesalers, retail pharmacies and medicine outlets.

Importers or exporters of medicines: Yes No Don’t Know


- 53 -


Regulatory inspection

2.15. Are there legal provisions to inspect premises and collect samples?

Yes No Don’t Know This question is asking if there are legal provisions (or legislation) that describe the legal conditions under which physical inspections of premises are conducted, including the legal procedures, guidelines and criteria for inspections. Physical inspection includes collection of samples and testing/calibrating equipment.

2.16. Are the following types of facilities inspected to check compliance with applicable requirements and are there written national guidelines/checklists for the inspection:

Facilities inspected Written national guidelines/ checklists

Manufacturers: Yes No DK Yes No DK Wholesalers/distributors: Yes No DK Yes No DK

Importers/exporters: Yes No DK Yes No DK Retail distributors/pharmacies:

Yes No DK Yes No DK A retail distributor is a company that

sells goods to consumers, e.g. a pharmacy or other medicines outlet. Many low and middle income countries have at least two different types of shops in which medicines can be purchased: pharmacies with a registered pharmacist and medicines stores, chemists or medicines outlets with paramedical or lay staff.

Control of narcotics and stupifiants

2.17. Are there legal provisions for the control of narcotics, psychotropic substances and precursors?

Yes No Don’t Know This question is asking if there are legal provisions (or legislation) that describe the legal conditions under which narcotics, psychotropic substances and precursors are controlled.

2.18. Is your country a signatory to the international conventions on the control of narcotics, psychotropic substances and precursors?

Yes No Don’t Know

Quality control

2.19. Is there a quality management system in place?

Yes No Don’t Know This question is asking if there is an officially defined protocol for ensuring the quality of medicines, including testing of medicines to be registered, collection and testing of samples, reporting results, corrective actions to be taken when poor results are found and preventative measures to be taken to reduce future incidence of poor results.

2.20. Are medicine samples tested for the following regulatory purposes:

Medicines registration: Yes No Don’t Know

Post-marketing surveillance:

Yes No Don’t Know Post-marketing surveillance is testing medicine samples to assess the quality of medicines that have already been licensed for public use.


- 54 -

Questions Responses Explanations 2.21. In which of the following laboratories

are samples tested:

This question is asking where medicine samples are tested. Mark “yes” if medicine samples are collected by or submitted to the listed laboratories for quality testing whether the testing is for purposes of registration, post-marketing surveillance and/or because the samples are suspected counterfeit/sub-standard medicines.

Government quality control laboratory:

Yes No Don’t Know

Local academic institutions: Yes No Don’t Know

Private laboratory: Yes No Don’t Know

Mini laboratories (district, regional): Yes No Don’t Know

Quality control laboratory in another country:

Yes No Don’t Know

2.22. What is the total number of samples quality tested in 2006?

Number This should include all samples tested whether in a quality assurance laboratory within the country or outside the country.

2.23. What is the total number of samples tested in 2006 that failed to meet quality standards?

Number This should include all samples tested that failed to meet quality standards whether the testing was done in a quality assurance laboratory within the country or outside the country.

2.24. Are there regulatory procedures to ensure quality control of imported medicines?

Yes No Don’t Know This question is asking if there are standard operating procedures for ensuring the quality of imported medicine, such as reviewing dossiers, product evaluation and testing of imported medicine products. This may include donated medicines.

2.25. Are there legal procedures for the recall and disposal of defective products?

Yes No Don’t Know This question is asking if there are legal provisions (or legislation) and procedures under which defective products are recalled and disposed of. Defective products are those that are found to be of poor quality and/or inadequately labelled.

Pharmacovigilance

2.26. Are adverse drug reactions (ADRs) monitored?

Yes No Don’t Know Monitoring adverse drug reactions is the science and activities relating to the detection, assessment, understanding and prevention of adverse effects or any other medicines-related problems.

a) If yes, at which of these health system levels are adverse drug reactions (ADRs) monitored:

Local level:

Yes No Don’t Know Mark “yes” next to local level if data are submitted to the hospital or community level for assessment and action.

Regional level:

Yes No Don’t Know Mark “yes” next to regional level if data are submitted to the regional or district level for assessment and action.

Central level:

Yes No Don’t Know Mark “yes” next to central level if data are submitted to the national level for assessment and action.


- 55 -

Questions Responses Explanations 2.27. Does your country report ADRs to an

international network or to the WHO Collaborating Centre for International Drug Monitoring?

Yes No Don’t Know Mark “yes” if your country has reported cases of ADR to the WHO Collaborating Centre for International Drug Monitoring (Uppsala Monitoring Centre) or other international network at least once in the last five years.

Counterfeit medicines

2.28. Are there any laws, regulations, programmes or procedures for detecting and combating counterfeit medicines?

Yes No Don’t Know Programmes and procedures for detecting and combating counterfeit medicines may include active regular surveillance, official agreement with police authorities, involvement in national or international networks, etc.

Counterfeit medicines are medicines, whether branded or generic, that are deliberately and fraudulently mislabelled with respect to identity and/or source or that have fake packaging. Counterfeit products may contain the correct ingredients or the wrong ingredients or may lack any, or sufficient, active ingredients. If your country has an alternative definition of counterfeit medicines, use that definition.

2.29. What sources of information are used to detect and combat counterfeit medicines:

Reports from national authorities:

Yes No Don’t Know “National authorities” refers to authorized groups providing data on a regular basis specifically for official use, such groups might include health facilities, professional organizations, etc.

Reports from specific/ad hoc studies:

Yes No Don’t Know An ad hoc study is a “one-off” study completed outside a regular system of collecting information.

Reports from the pharmaceutical sector:

Yes No Don’t Know “Reports from the pharmaceutical sector” primarily refers to reports from the pharmaceutical industry, whether brand or generic, local or multinational.

Reports from civil society/NGOs:

Yes No Don’t Know “Reports from civil society/NGOs” refers to reports produced by civil society for its own purposes rather than specifically for official use.

Civil society/NGOs (nongovernmental organizations) are non-profit organizations, networks and voluntary associations including charities, community groups, faith-based organizations, professional associations, academia and trade unions.


- 56 -


Dispensing and prescribing

2.30. Are there legal provisions for the following:

This question is asking if there are legal provisions (or legislation) that describe the legal conditions under which prescribers and the practice of pharmacy are licensed.

Licensing is a system that subjects all persons to evaluation against a set of requirements before they may be authorized to prescribe medicines/practice pharmacy. It may include issuing an official permit and granting authorization to prescribe medicines/practice pharmacy by either the governing authority or the body regulating the exercise of the profession.

Licensing and practice of prescribers: Yes No Don’t Know

Licensing and practice of pharmacy: Yes No Don’t Know

2.31. Is prescribing by generic name obligatory in the:

A generic name (international non-proprietary name - INN) is a non-proprietary or approved name rather than a proprietary or brand name under which a generic medicine is marketed. If prescribing by generic name is obligatory then prescribers are required to prescribe by generic name.

Public sector: Yes No Don’t Know

Private sector: Yes No Don’t Know

2.32. Is generic substitution permitted at:

Generic substitution is the practice of substituting a product, whether marketed under a trade name or generic name, by an equivalent product, usually a cheaper one, containing the same active ingredient at the dispensing level. Mark “yes” if either generic substitution is required or if the dispenser is allowed to make a generic substitution in at least some instances.

Public pharmacies: Yes No Don’t Know

Private pharmacies: Yes No Don’t Know

2.33. Are there incentives to dispense generic medicines at:

Incentives may include dispensing fees or mark-ups which provide financial incentive for dispensers to dispense lower-priced generic medicines.

Public pharmacies: Yes No Don’t Know

Private pharmacies: Yes No Don’t Know


- 57 -


Promotion and advertising

2.34. Are there provisions in the medicines legislation/regulations covering promotion and/or advertising of medicines?

Yes No Don’t Know This question is asking if there are legal provisions (or legislation) that describe the conditions under which the promotion and/or advertisement of medicines may be conducted.

Promotion and advertisement are activities that provide health workers and consumers with information about medicine products, particularly with the intent of encouraging health workers and consumers to use a particular product.

2.35. Who is responsible for regulating promotion and/or advertisement of medicines?

Industry (self-regulation) only

Government or national regulatory authority only

Co-regulation Don’t Know

Mark “industry (self-regulation) only” if the pharmaceutical industry bears responsibility for regulating itself to ensure companies’ practices of promoting and/or advertising medicines are in line with appropriate standards, whether the standards are explicitly defined or not, and government regulation is minimal. For example, in some countries companies police each other in terms of compliance to industry set guidelines.

Mark “government or national authority only” if the government or national regulatory authority is responsible for enforcing written legal restrictions on the promotion and/or advertising of medicines and company (self-regulation) is minimal.

Mark “co-regulation” if both industry (self-regulation) and the government or national regulatory authority contribute to regulating the promotion and/or advertisement of medicines.

a) If regulated by government, do regulations include any of the following:

Pre-approval for advertisements and/or promotional materials:

Yes No Don’t Know Pre-approval for advertisements and/or promotional materials means official approval of advertisements and/or promotional material must be obtained before they can be used.

Prohibition on advertising prescription medicines to the public:

Yes No Don’t Know

Guidelines on advertising of non-prescription medicines:

Yes No Don’t Know Guidelines on advertising of non-prescription medicines means there may be some restrictions on advertising non-prescription medicines to the public, such as in promoting therapeutic claims.


- 58 -

Questions Responses Explanations 2.36. Are civil society/NGOs included in

surveillance of promotion and/or advertisement of medicines?

Yes No Don’t Know Mark “yes” if civil society/NGOs actively monitor and report unethical/illegal practices regarding the promotion and/or advertisement of medicines and that this involvement is recognized/requested by the medicines regulatory authority, e.g. though a written memorandum of understanding. Normal consumer/public complaints that are not part of a government request for or recognition of civil society/NGO involvement should not be counted in this question.

Civil society/NGOs (nongovernmental organizations) are non-profit organizations, networks and voluntary associations including charities, community groups, faith-based organizations, professional associations, academia and trade unions.

3. MEDICINES SUPPLY SYSTEM

Please consult the agency/department responsible for the procurement and supply of medicines in answering the questions in this section.

3.1. Is public sector procurement pooled at the national level (i.e. there is centralized procurement for the regions/provinces)?

Yes No Don’t Know

Mark “yes” if public sector procurement is centralized and medicines are procured for the entire public sector by a national procurement body even if in some instances, such as cases of stock outages, public sector facilities procure medicines through other means.

3.2. Who is responsible for public sector medicines procurement and distribution:

Procurement

Distribution

Ministry of Health: Yes No DK Yes No DK Nongovernmental organization

(NGO):

Yes No DK Yes No DK Mark “yes” for nongovernmental organization (NGO) if government funds or foreign contributions are allocated to NGOs to procure or distribute medicines for the public sector.

Nongovernmental organizations (NGOs) are nongovernmental, non-profit organizations, networks and voluntary associations including charities, community groups, faith-based organizations, professional associations, academia and trade unions.

Private institution contracted by the government:

Yes No DK Yes No DK Mark “yes” for private institution contracted by the government if the government contracts or makes an agreement with a private entity to procure or distribute medicines for the public sector, e.g. if an agreement is made with a private company to distribute medical items and supplies to public sector district warehouses and health facilities.

Individual health institutions: Yes No DK Yes No DK


- 59 -

Questions Responses Explanations 3.3. What type of tender process is used

for public sector procurement and what is the percentage of the total cost for each:

Percentage of total cost

Competitive tender is a procedure for procuring medicines which puts a number of suppliers into competition. Purchasing is done on the basis of quotations submitted by suppliers in response to a public notice.

National competitive tender:

Yes No DK % National competitive tender is open to all or a limited number of local suppliers only.

International competitive tender:

Yes No DK % International competitive tender is open to all or a limited number of local and international suppliers though sometimes conditions give preference to either local or international suppliers.

Negotiation/direct purchasing:

Yes No DK % In negotiation/direct purchasing the buyer approaches one or a small number of suppliers and either buys at the quoted prices or bargains for a specific service arrangement.

3.4. Is there a tender board/committee overseeing public sector procurement?

Yes No Don’t Know This question is asking if there is a board or committee responsible for overseeing public sector procurement which is composed of members who are not staff/officials of the government procurement agency. A tender is a procedure for procuring medicines by seeking quotations from suppliers in response to a public notice.

a) If yes, are the key functions of the procurement office and those of the tender committee clearly separated?

Yes No Don’t Know

3.5. Does public sector medicines procurement use the WHO Prequalification system?

Yes No Don’t Know The WHO Prequalification Programme is a service provided by the World Health Organization (WHO) to facilitate access to medicines that meet unified standards of quality, safety and efficacy for HIV/AIDS, malaria and tuberculosis. The growing list of products (i.e. medicines) and manufacturers that have been found to meet the set requirements can be used by anyone bulk purchasing medicines, including countries.

3.6. Is public sector procurement limited to medicines on the Essential Medicines List (EML)?

Yes No Don’t Know An Essential Medicines List (EML) is a government-approved selective list of medicines or national reimbursement list.

Essential medicines are those that satisfy the priority health care needs of the population. They are selected with due regard to disease prevalence, evidence on efficacy and safety, and comparative cost-effectiveness. Essential medicines are intended to be available within the context of functioning health systems at all times in adequate amounts, in the appropriate dosage forms, with assured quality, and at a price the individual and the community can afford.


- 60 -

Questions Responses Explanations a) If yes, are there provisions for

purchasing medicines not on the Essential Medicines List?

Yes No Don’t Know This question is asking if public sector procurement is in general limited to medicines on the EML, are there guidelines to procure non-EML medicines (e.g. emergency medicines, high cost medicines, medicines for special conditions etc).

3.7. Did your country participate in a pooled procurement scheme with at least one other country for at least one of the last two procurement cycles?

Yes No Don’t Know

4. MEDICINES FINANCING

Please consult the budget/ finance division of the health ministry and/or the pharmaceutical supply group in answering the questions in this section. The hospital/health facility service and/or the national social and insurance services may also need to be consulted.

4.1. What is the total public or government expenditure for medicines in US$ for the most recent year for which data are available?

US$ Year This question is asking for the total amount the government has spent on medicines, including government allotment, health ministry expenditure, donor contributions channeled through the government, etc.

4.2. Is there a national policy to provide at least some medicines free of charge (i.e. patients do not pay out-of-pocket for medicines) at public primary care facilities?

Yes No Don’t Know If medicines are provided for free but patients must pay service fees, mark “yes” here.

If some facilities provide medicines for free but there is not a consistent national policy that applies to all primary public health facilities, mark “no” here.

If there is a national policy to provide medicines for free at primary public health facilities, but facilities are not required to abide by the policy and not all facilities provide medicines for free, mark “no” here.

a) If yes, which of the following are free at public primary care

All medicines: Yes No Don’t Know

Malaria medicines: Yes No Don’t Know

Tuberculosis medicines: Yes No Don’t Know

Sexually transmitted diseases medicines: Yes No Don’t Know

HIV/AIDS-related medicines: Yes No Don’t Know

At least one vaccine: Yes No Don’t Know

b) Which of the following types of patients receive medicines for free:

Patients who cannot afford them: Yes No Don’t Know

Children under 5 years of age: Yes No Don’t Know

Older children:

Yes No Don’t Know Mark “yes” for “older children” if children over 5 years of age receive medicines for free, regardless of the age limit, for example mark “yes” if children under 12 receive medicines for free.

Pregnant women: Yes No Don’t Know

Elderly persons: Yes No Don’t Know


- 61 -

Questions Responses Explanations 4.3. Which fees are commonly charged in

public primary care facilities:

Registration/consultation fees:

Yes No Don’t Know Registration and consultation fees are fees patients must pay for seeing a health professional for a health check-up and/or diagnosis regardless of whether or not medicines are prescribed.

Dispensing fees:

Yes No Don’t Know A dispensing fee is a fixed fee that pharmacies are allowed to charge per prescribed item or per prescription instead of or in addition to a percentage mark-up. The dispensing fee is paid to the dispenser and is in addition to the cost of the medicine. Both the dispensing fee and the cost of the medicine may be paid in part or whole by the patient, insurer or government.

Flat fees for medicines:

Yes No Don’t Know Mark “yes” for “flat fees” if either a flat fee for medicines or a flat fee per medicine item is commonly charged. A flat fee for medicines is a fee which remains the same irrespective of the number of medicines or the quantity of each medicine dispensed. Thus, for example, a patient receiving 3 medicines would pay the same as one receiving 1 medicine. Also a patient receiving 20 tablets of one medicine would pay the same as a patient receiving 100 tablets each of 2 medicines. A fee per drug item is a fee where the patient pays one set fee per each medicine irrespective of the number of units (tablets) of that medicine dispensed. Thus, for example, a patient receiving one medicine would pay US$1 and a patient receiving 2 medicines would pay US$2 and a patient receiving 3 medicines would pay US$3 and so on. However, a patient receiving 10 tablets of one medicine would pay the same as a patient receiving 100 tablets of one medicine.

Flat rate co-payments for medicines:

Yes No Don’t Know A flat rate co-payment is a fixed amount that a patient must pay either per medicine or per prescription to cover part of the cost of medicines, the other part being paid by an insurer or government.

Percentage co-payments for medicines:

Yes No Don’t Know A percentage co-payment is a fixed percentage of the cost of prescribed medicines that a patient must pay to cover part of the cost of medicines, the other part being paid by an insurer or government. The amount a patient pays will depend on the medicine and the number of units of that medicine prescribed.


- 62 -

Questions Responses Explanations 4.4. Is revenue from fees or the sale of

medicines used to pay the salaries or supplement the income of public health personnel in the same facility?

Always Frequently Occasionally Never DK

Mark “yes” if any percentage of collected fees or medicines sales is used to pay salaries, expenses and/or in any way supplement the income of public health personnel in the same facility.

Public sector Private sector4.5. Do prescribers dispense medicines? Always

Frequently Occasionally Never DK


In answering this question, mark the degree of frequency doctors or other authorized prescribers dispense medicines in the public and private sectors irrespective of laws permitting or disallowing authorized prescribers to dispense medicines.

4.6. What proportion of the population has health insurance?

All None Some DK

All None Some DK

Health insurance is any prepayment scheme for health care costs additional to but excluding subsidies funded through the health ministry budget. The purpose of questions 4.6 and 4.7 are to identify how much protection the population has against exposure to the cost of medicines at the time people are sick. This includes:

Prepaid financing and Public funding through the (prepaid) health ministry budget.

4.7. Are medicines covered by health insurance?

All None Some DK

All None Some DK

Public sector Private sector NGO4.8. Is there a policy covering medicine prices that applies to the public sector, the private sector, or non-governmental organizations?

Yes No DK

Yes No DK

Yes No DK

In some countries, NGOs, such as faith-based missions, provide non-profit or not-for-profit health services. The third column should be completed by ticking any policies applicable to this sector.

Nongovernmental organizations (NGO) are nongovernmental non-profit organizations, networks and voluntary associations including charities, community groups, faith-based organizations, professional associations, academia and trade unions.

a) If yes, which of the following policies covering medicine prices apply:

Maximum wholesale mark-up:

Yes No DK

Yes No DK

Yes No DK

A wholesale mark-up is a certain percentage added to a purchasing price to cover the cost and profit of the wholesaler.

Maximum retail mark-up:

Yes No DK

Yes No DK

Yes No DK

A retail mark-up is a certain percentage added to a purchasing price to cover the cost and profit of the retailer.


- 63 -

Questions Responses Explanations Duty on imported raw

pharmaceutical materials:

Yes No DK

Yes No DK

Yes No DK

A duty/tax on imported raw pharmaceutical materials is a fee assessed by customs or other responsible national authority on imported starting materials, reagents, intermediates, process aids, and solvents intended for use in the production of intermediates or active pharmaceutical ingredients.

Duty on imported finished pharmaceutical products:

Yes No DK

Yes No DK

Yes No DK

A duty/tax on imported finished pharmaceutical products is a fee assessed by customs or other responsible national authority on medicinal products that require no further processing and are already in their final containers.

4.9. Is a national medicine prices monitoring system for retail/patient prices in place?

Yes No DK

Yes No DK

Yes No DK

A national medicine prices monitoring system for retail/patient prices is any means of regularly tracking and comparing over time retail/patient medicine prices in the public, private and/or NGO sectors.

4.10. Are there regulations mandating retail/patient medicine price information to be made publicly accessible?

Yes No DK

Yes No DK

Yes No DK

In order for retail/patient medicine price information to be considered publicly accessible, one or more of the following or similar measures should be taken: prices should be available on the web or to anyone contacting the responsible authority, prices should be periodically published in national newspapers or official publications, prices should be posted in health facilities/pharmacies, etc.

4.11. Are there official written guidelines on medicine donations that provide rules and regulations for donors and provide guidance to the public, private and/or NGO sectors on accepting and handling donated medicines?

Yes No DK

Yes No DK

Yes No DK

Countries may have differing definitions for medicine donations which may include not only products but also monetary gifts earmarked for a particular product from a named source (e.g. manufacturer, organization or other country).

5. PRODUCTION AND TRADE

Please consult the medicines regulatory authority, the patent office and/or the trade ministry in answering the questions in this section.

5.1. What is the medicines production capability in the country:

Mark “yes” next to each of the types of medicine production that currently occurs in your country.

Research and development of new active substances:

Yes No Don’t Know

Production of pharmaceutical starting materials:

Yes No Don’t Know

Formulation from pharmaceutical starting materials:

Yes No Don’t Know

Repackaging of finished dosage forms:

Yes No Don’t Know


- 64 -

Questions Responses Explanations 5.2. Are patents granted on

pharmaceutical products by the national patent office?

Yes No Don’t Know A patent is an exclusive right awarded to an inventor to prevent others from making, selling, distributing, importing or using the invention, without license of authorization, for a fixed period of time, such as 20 years. This may be granted by the national patent office or, for smaller countries, a regional patent office representing several countries may grant the patent.

5.3. If your country is a member of the World Trade Organization (WTO), has national legislation been modified to implement the TRIPS Agreement?

Yes No Don’t Know Country not a member of WTO

Countries who are members of the World Trade Organization (WTO) should adapt national legislation to include the minimum standards of protecting and enforcing intellectual property rights, including those for patents, required under the Agreement on Trade-Related Aspects of Intellectual Property Rights (TRIPS Agreement).

a) If a WTO member, has your country used the following available transitional periods to implement the TRIPS Agreement:

TRIPS provides transitional periods during which countries are required to bring their national legislation and practices into conformity with its provisions.

Article 65:

Yes No Don’t Know Under Article 65 of the TRIPS Agreement: Transitional Arrangements: No Member shall be obliged to apply the provisions of the Agreement before the expiry of a general period of one year following the date (WIPO note 1 January 1995) of entry into force of the WTO Agreement. A developing country Member is entitled to delay for a further period of four years from the date of application, as defined in paragraph 1, of the provisions of this Agreement other than Articles 3, 4 and 5.

Article 66:

Yes No Don’t Know Country not an LDC

Under Article 66 of the TRIPS Agreement: Least-Developed Country Members: In view of the special needs and requirements of least-developed country Members, their economic, financial and administrative constraints, and their need for flexibility to create a viable technological base, such Members shall not be required to apply the provisions of this Agreement, other than Articles 3, 4 and 5, for a period of 10 years from the date of application as defined under paragraph 1 of Article 65. The Council for TRIPS shall, upon duly motivated request by a least-developed country Member, accord extensions of this period.

Doha declaration (Article 7):

Yes No Don’t Know Under the Doha Declaration, the transitional period is extended until 2016.


- 65 -

Questions Responses Explanations 5.4. Which of the following TRIPS

flexibilities have been incorporated into national legislation as applies to pharmaceuticals:

CBD = Currently being discussed

Compulsory licensing provisions:

Yes No CBD DK Compulsory licensing is when the judicial or administrative authority is allowed by law to grant a license, without permission from the holder, on various grounds of general interest (absence of working, public health, economic development, and national defense).

Government use:

Yes No CBD DK Government use is when the administrative authority is allowed to grant a license, without permission from the holder, for public health needs.

Parallel importing provisions:

Yes No CBD DK Parallel importation is importation, without the consent of the patent-holder, of a patented product marketed in another country either by the patent-holder or with the patent-holder’s consent. Parallel importation enables promotion of competition for the patented product by allowing importation of equivalent patented products marketed at lower prices in other countries.

The Bolar exception:

Yes No CBD DK The Bolar exception is an early working provision whereby generic pharmaceutical manufacturers are allowed to use patented inventions for the purpose of obtaining marketing approval prior to patent expiration.

6. RATIONAL USE OF MEDICINES

Please consult the health ministry (hospital division), professional bodies and/or the education ministry in answering the questions in this section.

6.1. Is there a national Essential Medicines List (EML)?

Yes No Don’t Know A national Essential Medicines List is a government-approved selective list of medicines or national reimbursement list from which most prescriptions should be made. Essential medicines are those that satisfy the priority health care needs of the population. They are selected with due regard to disease prevalence, evidence on efficacy and safety, and comparative cost-effectiveness.

a) If yes, how many unique medicine formulations does the national EML contain?

Number: Count similar formulations registered or approved as different products as one formulation, for example Brand X 500 mg Paracetamol tablets and Brand Y 500 mg Paracetamol tablets are counted as one formulation whereas Brand X 250 mg Paracetamol tablets and Brand X 500 mg Paracetamol tablets are counted as two formulations.


- 66 -

Questions Responses Explanations b) How many paediatric formulations

are included in the:

National EML: Number:

Separate Paediatric EML:

Number: No separate paediatric EML

Some countries have a separate list of essential medicines for paediatrics.

c) When was the national EML last updated?

Year:

Is the national EML being used in thefollowing:

Mark “yes” if the EML is currently being used.

Public sector procurement: Yes No Don’t Know

Public insurance reimbursement: Yes No Don’t Know

Private insurance reimbursement: Yes No Don’t Know

d) Is there a committee responsible for the selection of products on the national EML?

Yes No Don’t Know This refers to a formally recognized committee with members of different expertise and from different agencies/organizations.

National STG

Hospital-level STG

Primary care STG

6.2. Are the following types of standard treatment guidelines (STGs) produced by the health ministry for major conditions?

Yes No DK

Yes No DK

Yes No DK

Mark “yes” if the health ministry or similar national authority produces a collection of treatment guidelines covering prevalent/common disease conditions in the country for use at the national, hospital or primary care levels.

If treatment guidelines are produced separately for each disease/condition or organ system, mark “no”.

a) If yes, when were the STGs last updated?

Year Year Year

6.3. Are there standard treatment guidelines for key paediatric illnesses?

Yes No Don’t Know

Mark “yes” if the health ministry or similar national authority produces a collection of treatment guidelines covering prevalent/common paediatric disease conditions, whether it is a stand alone document or a chapter/section of the national, hospital or primary care STG.

Mark “no” if paediatric treatment guidelines are produced separately for each disease or condition.

6.4. Is there a National Medicines Formulary Manual?

Yes No Don’t Know

A formulary is a manual containing clinically oriented summaries of pharmacological information about selected medicines. The manual may also include administrative and regulatory information pertaining to the prescribing and dispensing of medicines. A national formulary generally concentrates on available and affordable medicines that are relevant to the treatment of diseases in a particular country.

a) If yes, when was it last published/reviewed?

Year:

b) Does it cover only medicines on the national EML?

Yes No Don’t Know


- 67 -

Questions Responses Explanations 6.5. Are the following prescribing issues

part of the basic curricula in most health training institutions for:

Essential Medicines List (EML)

Standard Treatment Guidelines (STGs)

Problem-based pharmaco-therapy

Rationalpresc- ribing

This question is asking whether most or major government-recognized institutions training health professionals require students to be instructed in the following areas: An Essential Medicines List is a government-approved selective list of medicines or national reimbursement list from which most prescriptions should be made. Standard Treatment Guidelines are developed by the health ministry or similar national authority. They are a collection of treatment guidelines covering prevalent/common disease conditions in the country for use at the national, hospital or primary care levels. Problem-based pharmacotherapy is a problem-based practical approach to teaching prescribing. Rational prescribing, or appropriate prescribing, assures that patients receive medications appropriate to their clinical needs, in doses that meet their own individual requirements, for an adequate period of time, and at the lowest cost to them and their community.

Doctors: Yes No DK

Yes No DK

Yes No DK

YesNo DK

Nurses: Yes No DK

Yes No DK

Yes No DK

YesNo DK

Pharmacists: Yes No DK

Yes No DK

Yes No DK

YesNo DK

Pharmacy assistants: Yes No DK

Yes No DK

Yes No DK

YesNo DK

Paramedical staff: Yes No DK

Yes No DK

Yes No DK

YesNo DK

6.6. Are there obligatory, non-commercially funded continuing education programmes that include use of medicines for:

A continuing education programme is a programme based on regular workshops, seminars and/or in-service training which provides all prescribers and dispensers with refresher courses on medicine issues.

Doctors: Yes No Don’t Know

Nurses/midwives/paramedical staff: Yes No Don’t Know

Pharmacists: Yes No Don’t Know

Pharmacy aides/assistants: Yes No Don’t Know


- 68 -

Questions Responses Explanations 6.7. Is there a public or independently

funded, nationally accessible (e.g. by phone) medicines information centre or service that provides information on demand to:

A medicines information centre or service is an organization within or outside the health ministry which collects and provides objective information on medicines to health personnel and the public. Objective information should be understood as information produced by independent scientific sources without any support from the pharmaceutical industry or private firms involved in the medicines sector. The medicines information centre/service may perform additional tasks.

Prescribers: Yes No Don’t Know

Dispensers: Yes No Don’t Know

Consumers: Yes No Don’t Know

6.8. Have there been any public education campaigns about rational medicines use in the previous two years conducted by the health ministry, a nongovernmental organization, or academia on the following topics:

Mark “yes” if there has been any effort to inform the public (e.g. through radio programmes, newspaper articles/ announcements, public gatherings, etc.) about any of the listed rational medicines use issues. The focus of these efforts may include the importance of patients receiving medications appropriate to their clinical needs, the dangers of over-prescribing, the dangers of inappropriate dosing, etc.

Use of antibiotics: Yes No Don’t Know

Use of injections: Yes No Don’t Know

Other rational medicine use topics/issues: Yes No Don’t Know

6.9. How often do the following personnel prescribe prescription-only medicines at the primary health care level in the public sector:

Doctors: Always Frequently Occasionally Never DK

Nurses/midwives/paramedical staff: Always Frequently Occasionally Never DK

Pharmacists/pharmacy aides/assistants: Always Frequently Occasionally Never DK

Personnel with <1 month formal health training:


Formal health training includes any government-recognized training programme on health issues.

6.10. Are there a national programme and/or multidisciplinary body, involving government, civil society and professional bodies, which monitor and promote the rational use of medicine?

Yes No Don’t Know


- 69 -

Questions Responses Explanations 6.11. Is there a mandatory requirement to

organize/develop drug and therapeutics committees?

Yes No Don’t Know Mark “yes” if there is a mandatory requirement for facilities/areas to organize/develop drug and therapeutics committees and positive incentives exist for those that have drug and therapeutics committees and disincentives exist for those that do not. If there is a requirement, but no incentives or disincentives mark “no” here.

A drugs and therapeutics committee is a group of people established and officially approved by the health ministry and/or health facility management that promotes the safe and effective use of medicines in the area or facility under its jurisdiction.

6.12. What proportions of hospitals and regions have drug and therapeutics committees:

In answering this question, please use the following approximations: All = more than 90% Most = 61-90% Half = 40-60% Few = 10-39% None = less than 10%

Referral hospitals:

All Most Half Few None DK

Referral hospitals are tertiary care centres which include major hospitals offering a full range of services, including specialty units, and specialty hospitals dedicated to specific types of patients, e.g. paediatrics, or specific range of conditions, e.g. oncology. While referral hospitals (tertiary care centres) may provide secondary or even primary care, their main function is to provide a referral service for secondary care centres (general hospitals) in all the main subspecialties.

General hospitals:


General hospitals are secondary care centres, which usually include acute medical, surgical, paediatric and obstetric services but do not include specialist services such as oncology, cardiac or neurological surgery etc. While general hospitals may have 1-2 units providing sub-specialist tertiary care, they do not have a full range of such services, and their main function is to provide a referral service for the primary health care centres and a direct service to the population under their jurisdiction and they do not generally have specialist units.

Regions/provinces:


For regions/provinces, mark the proportion of regions/provinces with a drugs and therapeutics committee that promotes the safe and effective use of medicines in all facilities (hospital and primary health care) in the region/province.


- 70 -

Questions Responses Explanations 6.13. Is there a national strategy to contain

antimicrobial resistance?

Yes No Don’t Know A national strategy is a written plan supported/spearheaded by a national/central authority and officially endorsed by the government.

6.14. Is there a national reference laboratory to coordinate epidemiological surveillance of antimicrobial resistance?

Yes No Don’t Know Mark “yes” if there is a national laboratory to which health facilities submit reports on antimicrobial resistance that monitors and reports on antimicrobial resistance in the country and coordinates and reports on any response. If there is no national laboratory, but reports on antimicrobial resistance are submitted to a regional laboratory, also mark “yes”.

6.15. Is there a funded national inter-sectoral task force to coordinate the promotion of appropriate use of antimicrobials and prevention of spread of infection?

Yes No Don’t Know This question is asking if there is a formal committee or other body of representatives of public, private and NGO or mission sectors that oversees efforts to improve appropriate use of antimicrobials and prevent spread of infection. This may include developing and evaluating codes of practices at health facilities and/or education programmes for health professionals and/or consumers.

6.16. How frequently are the following types of medicines sold over the counter without any prescription:

This question is asking how often antibiotics and injections which require a prescription to be dispensed are sold without a prescription, regardless of laws prohibiting such practice.

Antibiotics: Always Never

Frequently Occasionally DK

Injections: Always Never

Frequently Occasionally DK

Annex 2 Level II Indicators

- 71 -

Annex 2: Level II Indicators

Annex 2a: Technical descriptions of Level II indicators

Access

1. Availability of key medicines in public health facility dispensaries, private drug outlets and warehouses supplying the public sector (Survey Forms 1, 10 and 15)

Purpose To measure current availability of key medicines to treat common health

problems in public health facility dispensaries, private drug outlets and warehouses. Essential medicines to treat common diseases should be available in all these facilities, especially in public sector facilities providing health services for the poor. Physical availability is a basic measure of access to essential medicines. See pages 29—30 for guidance on selecting key medicines for this study.

Prerequisites List of 15 key essential medicines to treat the most common health problems. Source of data Survey of 30 public health facility dispensaries, 30 private drug outlets, and

5 warehouses. Process Go through the shelves and identify which of the listed essential medicines are

available at the facility at the time of the survey. Only count in stock medicines in the facility at the time of the visit regardless of whether or not they are available at an offsite storage facility.

Calculation % of key medicines available in a facility = % in stock = number of key

medicines available in a facility ÷ number of key medicines reviewed x 100 National average = sum of % of key medicines available for all facilities ÷ number of facilities sampled.

2. % of prescribed medicines dispensed or administered to patients at public health facility

dispensaries (Survey Form 6) Purpose To measure the degree to which facilities are able to provide needed medicines. Source of data Sample of 30 prospective outpatient encounters at 30 public health facility

dispensaries. See section on sampling. Process Interview patients leaving the dispensing area or leaving the facility after they

have been treated and received medicines. Patients can be interviewed consecutively or as convenient. Count how many chemical entities were prescribed and dispensed.

Calculation % of medicines dispensed = number of medicine dispensed ÷ number of

medicines prescribed x 100. National average = sum of % of medicine dispensed for all public health facility dispensaries ÷ number of public health facility dispensaries sampled.


- 72 -

3. Stockout duration at public health facility dispensaries and warehouses supplying the public sector (Survey Forms 4 and 16)

Purpose To measure availability over the past 12 months of key medicines to treat

common health problems. An adequate logistic system ensures that essential medicines remain in stock at all times.

Prerequisite List of 15 key essential medicines to treat the most common health problems

Adequate medicine stock recording system (able to access stock records for the previous 12 months).

Source of data Survey of 30 public health facility dispensaries and 5 warehouses. Process Using existing data on stock cards, record the number of days each key

medicine is not available (when the stock is zero). A medicine is in stock if any dose of any equivalent product is available in either branded or generic form. Stockout should be measured based upon the chemical entity rather than any one brand or dosage form. The review period should, as far as possible, cover one year and never less than six months.

Calculation Equivalent number of days per year each medicine is out of stock = number of

days each medicine is out of stock x 365 divided by number of days covered by the review for that medicine. Average stockout duration in each facility = average number of stockout days = sum of the equivalent number of days per year all medicines are out of stock divided by number of key medicines reviewed. National average = sum of average stockout duration in all facilities number of facilities sampled.

Limitations Results on this indicator will be influenced by the quality of stock records.

4. % of adequate record keeping at public health facility dispensaries and warehouses supplying the public sector (Survey Forms 4 and 16)

Purpose To determine the extent to which stock records are maintained. The presence of

adequately maintained and accurate stock records contributes to proper management, estimation of needs and the reorder of medicines.

Prerequisite List of 15 key essential medicines to treat the most common health problems. Source of data Survey of 30 public health facility dispensaries and 5 warehouses. Process For each of the key medicines, examine the data on the stock card and identify

those medicines for which there are records of quantities of receipt, issue and stock-on-hand for at least 6 months in the previous 12 months.

Calculation % adequate stock records = number of incidences where there are adequate

records for at least 6 months ÷ number of key medicines reviewed National average = sum of % adequate stock records for all facilities divided by number of facilities sampled.

Limitations This indicator does not permit verification of whether the records are accurate.


- 73 -

5. Affordability of treatment for adults and children under 5 years of age at public health facility dispensaries and private drug outlets (Survey Forms 3 and 12)

Purpose To measure affordability of basic pharmaceutical treatment as an indicator of

access to essential medicines. In most developing countries, a majority of the population pays for treatment out-of-pocket. Affordability is expressed as the ratio of the cost of treating moderate pneumonia or another condition to a standard unit of measure. For this survey, the lowest daily government salary is used. Countries may also identify an optional second unit of measure (e.g. poverty line, basket of food, etc.).

Prerequisite Lowest government daily wage or the lowest daily wage of any appropriate

majority group. Standard treatment for moderate pneumonia and another condition (where no hospitalization is required) for adults and children.

Source of data Survey of 30 public health facility dispensaries and 30 private drug outlets. Process Identify the treatment of choice (medicine/generic name, dosage form and

strength) for adults and children based on standard treatment guidelines. Identify the number of units needed to complete the treatment. Identify the unit price (the price charged to patients) for the lowest priced brand or generic equivalent medicine in each pharmacy, include any applicable charges that patients would pay to receive treatment such as dispensing fees and/or syringes. If there are flat charges paid for each medicine given to patients, then this amount should be recorded as the price of the medicine. Indicate “0” if medicines are given free of charge.

Calculation Total cost of treatment = number of units needed to complete the treatment x unit price. Equivalent number of day’s wages = total cost of treatment ÷ lowest daily government salary. Ratio of cost of treatment and optional standard unit of measure = total cost of treatment ÷ optional standard unit of measure. National average = sum of equivalent number of day’s wages at all facilities ÷ number of facilities sampled. National average = sum of ratio of cost of treatment and optional standard unit of measure at all facilities divided by number of facilities sampled.

6. Price variation of key medicines in public health facility dispensaries and private drug outlets (Survey Forms 2 and 11)

Purpose To assess affordability based on the variation of price among pharmacies in the

country. Prerequisite List of predetermined medicines to treat relevant health problems and their

respective preparations. Source of data Survey of 30 public health facility dispensaries and 30 private drug outlets. Process Go through stock cards corresponding the listed medicines and take note of the

most recent price per smallest dispensing unit. If there is a list of selling price, do the same. If there is not, ask a person in charge of the pharmacy. Be sure that the consulted source if the same used to charge the users.


- 74 -

Calculation Average minimum price of a specific medicine = sum of unit prices in each facility divided by number of facilities where the price could be recorded.

7. Average cost of medicines at public health facilities and private drug outlets (Survey Form 6 and 14)

Purpose To measure average cost paid by patient for medicines at public health facilities

and private drug outlets as an indicator of access to essential medicines. In most developing countries, a majority of the population pays for treatment out-of-pocket.

Prerequisite Patient consent to the interview. Source of data Sample of 30 prospective outpatient encounters at 30 public health facility

pharmacies and pharmacies and 30 private drug outlets. See section on sampling .

Process Interview patients leaving the dispensing area/pharmacy or leaving the facility

after they have been treated and received medicines. Patients can be interviewed consecutively or as convenient. Ask how much each patient paid out-of-pocket for the medicines received at the facility.

Calculation Average cost = amount paid for medicines divided by number of patients reviewed

National average = average cost for all public health facility dispensaries ÷ number of public health facility dispensaries sampled.

8. Geographical accessibility of facilities

8.1. % patients taking more than one hour to travel to the facility (Survey Forms 6 and 14)

Purpose To assess geographical accessibility in terms of how long does it takes the

patients to get to the pharmacy where they get their medicines. Prerequisite Patient consent to the interview. Source of data Sample of 30 prospective outpatient encounters at 30 public health facility

pharmacies and 30 private drug outlets. Process Interview patients leaving the dispensing area/pharmacy or leaving the facility

after they have been treated and received medicines. Patients can be interviewed consecutively or as convenient. Ask how long it took the patient to get to the facility.

Calculation % of patients/ consumers taking more than one hour to get to the facility = number of patients/consumers who answered that it took more than one hour for them to get to the facility (category 3, > 1h) divided by number of interviewed patients/costumers. National average = % of patients/ consumers taking more than one hour to get to the facility in all public health facility dispensaries/private drug outlets divided by number of public health facility dispensaries/private drug outlets sampled.


- 75 -

Limitations For some patients, the departure point may be different from their house (they may be using a health facilities near their work, for example).

8.2. Average transportation cost to the facility (Survey Forms 6 and 14) Purpose To assess geographic accessibility in terms of how much does it cost to the

patients to get to the pharmacy where they get their medicines. Prerequisite Patient consent to the interview. Source of data Sample of 30 prospective outpatient encounters at 30 public health facility

pharmacies and 30 private drug outlets. Process Interview patients leaving the dispensing area/pharmacy or leaving the facility

after they have been treated and received medicines. Patients can be interviewed consecutively or as convenient. Ask how much it cost the patient to get to the pharmacy.

Calculation Average cost = amount spent in transportation to get to the facilitydivided by number of patients/costumers interviewed. National average = average cost for all public health facility dispensaries/private drug outlets divided by number of public health facility dispensaries/private drug outlets sampled.

Note: The average cost can be compared to the minimum government daily salary.

Quality

1. Presence of expired medicines in public health facility dispensaries, private drug outlets and warehouses supplying the public sector (Survey Forms 1, 10 and 15)

Purpose To determine if expired medicines are being distributed or sold. In some

countries, expired medicines are distributed or medicines are allowed to go out of date on pharmacy shelves. See section on selecting key medicines for this study.

Prerequisite List of 15 essential medicines to treat the most common health problems. Source of data Survey of 30 public health facility dispensaries, 30 private drug outlets and

5 warehouses. Process Go through stock on shelves and check the expiry dates of all of the generic and

branded forms of each of the essential medicines. If any of the strengths has an expiry problem, the answer for that medicine should be “yes”. If expired medicines are listed and kept in a designated location in the store to be destroyed, availability of expiry should not be registered.

Calculation % of expired key medicines in stock = number of key medicines with any

samples beyond expiry date divided by number of key medicines in stock x 100 National average = sum of % of expired key medicines in stock at all facilities divided by number of facilities sampled.


- 76 -

2. Adequacy of conservation conditions and handling of medicines in public health facility pharmacies/dispensaries, private drug outlets and central/regional/district warehouses supplying the public sector (Survey Forms 5, 13 and 17)

Purpose To determine status of conservation conditions and handling of medicines in

public sector facilities, both of which are factors that affect quality of medicines. Prerequisite Checklist of minimum criteria for adequate conservation conditions and

handling of medicines at facilities. Source of data Survey of 30 public health facility dispensaries, 30 private drug outlets and 5

warehouses. Process Use the checklist to rate the conservation conditions and handling of medicines.

Only indicate “true” if all conditions included in the statement are true. If any condition of the statement is false, indicate “false”.

Calculation Conservation condition and handling of medicines = total number of “true”

responses to items on the conservation condition and handling of medicines checklist ÷ 10 x 100. National average = total score of all facilities ÷ number of facilities sampled.

Rational use of medicines

1. % of medicines adequately labelled at public health facility dispensaries (Survey Forms 6 and 14)

Purpose To assess quality of dispensing practice. If medicines are to be used properly, they should be labelled appropriately by the person dispensing them.

Prerequisite An adequate label includes the name of the medicine, how muchis to be taken

and the frequency of administration. Source of data Sample of 30 prospective outpatient encounters at 30 public health facility

dispensaries. See page 28 for sampling instructions. Process Interview patients leaving the dispensing area or leaving the facility after they

have been treated and received medicines. Patients can be interviewed consecutively or as convenient. Check if each medicine label conforms to all requirements for adequate labelling. Count a medicine as adequately labelled only if all requirements are met.

Calculation % of medicines adequately labelled = total number of medicines adequately labelled ÷ total number of medicines dispensed x 100. National average = sum of % of medicines adequately labelled at all public health facility dispensaries ÷ number of facilities sampled.


- 77 -

2. % of patients knowing how to take medicines at public health facility dispensaries (Survey Forms 6 and 14)

Purpose To assess if patients have adequate knowledge about how to take their

medicines. Prerequisite Adequate knowledge includes knowing the appropriate dosage and duration of

each medicine. Source of data Sample of 30 prospective outpatient encounters at 30 public health facility

dispensaries. See page 28 for sampling instructions. Process Interview patients leaving the dispensing area or leaving the facility after they

have been treated and received medicines. Patients can be interviewed consecutively or as convenient.

Check if the patient knows both the appropriate dosage and duration of each medicine (i.e., how much, how often and for how long he or she should take each medicine). Count the patient as having adequate knowledge only if both criteria are met for all medicines dispensed to the patient.

Calculation % of patients knowing how to take medicines = number of patients who know how to take medicines ÷ number of patients sampled x 100. National average = sum of % patients knowing how to take medicines at all public health facility dispensaries ÷ number of facilities sampled.

3. Average number of medicines prescribed in public health facilities (Survey Forms 6 and 7)

Purpose To determine the prevalence of polypharmacy, which is one measure of

unnecessary prescribing. Prerequisite Outpatient treatment records covering the past 12 months or, if unavailable,

current treatment records sufficient to randomly select 30 outpatient encounters.

Source of data Sample of 30 outpatient encounters (retrospective or prospective) at 30 public

health facilities. See pages 27—28 for sampling instructions. Process Survey form6- Interview patients leaving the dispensing area or leaving the

facility after they have been treated and received medicines. Patients can be interviewed consecutively or as convenient. Survey fprm7- Request all available records for the past 12 months before beginning sampling. Consider only encounters for a single disease, complaint, or symptom.List the number of medicines given per encounter. Combination products are counted as one medicine. The same product prescribed consecutively in different forms (i.e. injection and tablet) should be counted as one medicine, however if the same product is prescribed simultaneously in different forms, each form should be counted separately.

Calculation Average number of medicines per encounter = total number of medicines prescribed ÷ number of patient encounters reviewed. National average = sum of average number of medicines per encounter from all public health facilities ÷ number of facilities sampled.


- 78 -

Note: Survey Form 6 collects data from patient interviews and Survey Form 7 collects data from patient records. Results may therefore differ for Survey Forms 6 and 7..

4. % of patients prescribed antibiotics in public health facilities (Survey Form 7) Purpose To determine the prevalence of antibiotic prescribing, since over-prescribing of

antibiotics is one common type of inappropriate medicine use. Prerequisite Understanding of which medicines should be counted as antibiotics. Definitions

of medicines considered as antibiotics must be agreed upon at the national level. Antimicrobial agents are not always classified in an identical way. Indicators for antibiotic use can be sensitive to certain medicines, especially in places with high incidence of parasitic infections, such as malaria or tuberculosis. Medicines such as antiprotozoals and anthelminthics are also usually placed in a different category of antibiotics. How to classify topical antibiotics widely used in areas where trachoma, bacterial conjunctivitis and bacterial skin infection are common will also need to be considered.


health facilities. See pages 27—28 for sampling instructions. Process Request all available records for the past 12 months before beginning sampling.

Determine encounters where at least one antibiotic has been prescribed.

Calculation % of patients prescribed antibiotics = number of encounters in which one or more antibiotics is prescribed ÷ number of patient encounters reviewed x 100.

National average = sum of % of patients prescribed antibiotics in all public

health facilities ÷ number of facilities sampled.

5. % of patients prescribed injections in public health facilities (Survey Form 7) Purpose To determine the prevalence of injection use, since over-prescribing of injections

is one common type of inappropriate medicine use. Source of data Sample of 30 outpatient encounters (retrospective or prospective) at 30 public


Determine encounters where an injection has been prescribed. Do not count immunizations and injectable contraceptives.

Calculation % of patients prescribed injections = number of encounters in which one or more injection is prescribed ÷ number of patient encounters reviewed x 100 National average = sum of % of patients prescribed injections in all public health facilities ÷ number of facilities sampled.

6. % of prescribed medicines on the essential medicines list at public health facilities (Survey Form 7)

Purpose To measure the degree to which prescribing practice conforms to the national

essential medicines list (EML). The essential medicines concept is one of the main strategies being promoted in medicines policy. More and more countries


- 79 -

are formulating national EMLs. For most, this should be the basis for all public medicines procurement and prescribing.

Prerequisite A current national essential medicines list officially endorsed by the ministry of

health. If there is no current officially endorsed EML, then this indicator should not be measured.


health facilities. See pages 27—28 for sampling instructions. Process If there is a current officially endorsed national EML, a copy should be

provided to each survey team. Request all available records for the past 12 months before beginning sampling. Determine how many of the prescribed medicines are included on the EML, even if they are not prescribed under an internationally recognized name.

Calculation % of prescribed medicines included on the EML = number of prescribed

medicines included on the EML ÷ total number of medicines prescribed x 100. National average = sum of % prescribed medicines included on the EML at all public health facilities ÷ number of facilities sampled.

7. % of medicines prescribed by generic name (INN) at public health facilities (Survey Form 7)

Purpose To measure the degree to which prescribing practice conforms to the principles

of generic prescribing. Prerequisite A clear understanding of what is meant by the term generic medicine. Source of data Sample of 30 outpatient encounters (retrospective or prospective) at 30 public


Determine the encounters where a generic medicine has been prescribed. Calculation % of medicines prescribed by generic name = number of medicines prescribed

by generic name ÷ total number of medicines prescribed x 100. National average = sum of % medicines prescribed by generic name at all public health facilities ÷ number of facilities sampled.

8. Availability of standard treatment guidelines at public health facilities (Survey Form 8) Purpose To determine if prescribers have available to them the key source of therapeutic

information they need in daily practice. Prerequisite Identify STGs for pneumonia and another condition officially endorsed by the

government, WHO or other international, academic or professional organization. A copy should be provided to each survey team.

Source of data Survey of 30 public health facilities. Process Ask to see a copy of the relevant STGs. Only count a facility as having each STG

if the facility is able to produce the current version. If the current version of the document is not physically available, mark “no”.


- 80 -

Calculation National average = number of facilities with both STGs available ÷ number of facilities sampled.

9. Availability of EML at public health facilities (Survey Form 8) Purpose To determine if prescribers and/or dispensers have available to them the key

source of pharmaceutical information that should be the basis for all medicine prescribing and dispensing.

Prerequisite A current essential medicines list officially endorsed by the ministry of health,

region, district or health facility as appropriate. Source of data Survey of 30 public health facilities. Process If there is a current officially endorsed national EML, a copy should be

provided to each survey team. Ask to see a copy of the current applicable EML for that facility. Only count a facility as having an EMLif the facility is able to produce the current version. If the current version of the document is not physically available or if it has been more than five years since the EML was last updated, mark “no”.

Calculation National average = number of facilities with at least one current EML available ÷ number of facilities sampled.

10. % of tracer cases treated according to recommended treatment protocol/guide (Survey Form 9)

Purpose To measure quality of care for common conditions with clear recommended

treatment protocols. Adherence to recommended protocols can be measured by checking if tracer diseases are treated appropriately. Such recommendations might include use of ORS for watery diarrhoea in children, use of the recommended antibiotic for mild pneumonia or non-use of antibiotics for simple ARI. The survey form has space for countries to track additional conditions, if desired.

Prerequisite Ability to sample cases retrospectively by diagnosis.

Existing STG with clear treatment recommendation for any additional conditions evaluated.

Source of data Sample of 10 outpatients under 5 years of age with non-bacterial diarrhoea, 10 outpatients under 5 years of age with mild/moderate pneumonia, and 10 outpatients of any age with non-pneumonia (non-bacterial) acute respiratory tract infection at 30 public health facilities. See page 27—28 for sampling instructions.

Process Request all available records for the past 12 months before beginning sampling.

Select encounters from outpatient records or ledgers of under 5 . If possible choose only single diagnosis encounters.

Determine if patients received any of the treatments listed on the survey form. Calculation % of cases prescribed each medicine = number of cases prescribed each

medicine ÷ number of cases x 100. National average = sum of % of cases prescribed each medicine at all facilities ÷ number of facilities sampled.


- 81 -

Limitations This indicator is one of the most valuable measures of quality of care, but

problems exist in terms of obtaining enough records where the above conditions are correctly diagnosed.

11. % of prescription medicines bought without prescription (Survey Form 14) Purpose To determine if consumers are purchasing and dispensers are selling

prescription medicines without prescription. The existence of a prescription (and therefore a medical encounter) as the source of (prescription) medicine seeking behaviour should be the basis for all medicine dispensing as a way to promote rational use of medicines.

Prerequisite Patient consent to the interview. Source of data Sample of 30 prospective outpatient encounters at 30 private drug outlets. Process Interview customers leaving the pharmacy after they have received medicines.

Cosumers can be interviewed consecutively or as convenient. List the number of prescription medicines bought without prescriptions.

Calculation % of prescription medicines bought without prescription = number of prescription medicines bought without prescription ÷ number of medicines purchased x 100. National average = sum of % of prescription medicines bought without prescription in all private drug outlets ÷ number of private drug outlets sampled.

Other information

1. % of facilities that comply with the law (presence of a pharmacist) Purpose To determine if facilities comply with the law (presence of a pharmacist where

the law requires). Prerequisite Knowbefore data collection if the law requires a pharmacist to be present

during hours of operation of public/government pharmacies/drug outlets. Source of data Survey of 30 public health facility dispensaries and 30 private drug outlets. Process Identify if there is a pharmacist present at the time of the visit.

Calculation National percentage = % of facilities that comply with the law = Sum of facilities with pharmacist ÷ number of facilities sampled x 100.

2. % of facilities with pharmacist, nurse, pharmacy aide/ health assistant or untrained staff dispensing

Purpose To determine the profile of the health professionals who dispense medicines in

health facilities and private drug outlets.


- 82 -

Prerequisite Verify in the field which professional is dispensing. Source of data Survey of 30 public health facility dispensaries and 30 private drug outlets. Process Identify who are the health professionals dispensing during the time of visit. Calculation National percentage = % of facilities with a pharmacist dispensing during the

visit = Sum of facilities where there was a pharmacist dispensing during the visit ÷ number of facilities sampled x 100.

National percentage = % of facilities with a nurse dispensing during the visit = Sum of facilities where there was a nurse dispensing during the visit ÷ number of facilities sampled x 100.

National percentage = % of facilities with a pharmacy aide/health assistant dispensing during the visit = Sum of facilities where there was a pharmacy aide/ health assistant dispensing during the visit ÷ number of facilities sampled x 100.

National percentage = % of facilities with untrained staff dispensing during the visit = Sum of facilities where there was untrained staff dispensing during the visit ÷ number of facilities sampled x 100.

3. % of facilities with doctor, nurse, trained health worker/health aide prescribing Purpose To determine the profile of the health professionals prescribing in health

facilities. Prerequisite Verify in the field which professionals are prescribing. Source of data: Survey of 30 public health facilities. Process Identify who are the health professionals prescribing during the time of visit. Calculation National percentage = % of facilities with a doctor prescribing during the visit =

Sum of facilities where there was doctor prescribing during the visit ÷ number of facilities sampled x 100.

National percentage = % of facilities with a nurse prescribing during the visit = Sum of facilities where there was nurse prescribing during the visit ÷ number of facilities sampled x 100.

National percentage = % of facilities with a trained health worker/health aide prescribing during the visit = Sum of facilities where there was trained health worker/health aide prescribing during the visit ÷ number of facilities sampled x 100.

4. % facilities with prescriber trained in RDU Purpose To determine if the most senior health professionals prescribing in health

facilities were trained in RDU. Prerequisite Identify who is the most senior prescriber in health facilities during the time of

the visit. Source of data: Survey of 30 public health facilities.


- 83 -

Process Identify if the most senior prescriber has attended RDU-related training within the last year. (RDU curriculum can include any of the following: rational prescribing, essential medicine concept, use of IMCI or other clinical guidelines.)

Calculation National percentage = % of facilities with prescriber trained in RDU = Sum of

facilities where the most senior prescriber was trained in RDU ÷ number of facilities sampled x 100.


- 84 -


- 85 -

Annex 2b: Issues, technical details, survey design and sampling for Level II indicators

Sampling

In order to estimate indicators accurately and reliably, it is important to follow specific procedures for drawing samples and gathering data. In this way, major types of selection bias are reduced and the study population is more representative of the reference population–in this case the national population.

Larger samples are more costly but give more precise results. Sample size is therefore a balance between what is desirable and what is feasible. The best sample size will be the smallest one that will result in estimates with the desired degree of precision.

Experience with similar methodologies1 has shown that individual health providers tend to have consistent practices over time. Therefore a sample drawn at one point in time will provide roughly similar results as a sample that covers a longer period. However since data will generally be collected over a short period, they may suffer from bias due to seasons, variations in staffing, inconsistencies in the supply cycle etc. Where possible, the survey should use records to sample retrospectively over the previous 12 month period rather than sampling cases only on the day of the data collection.

Because the treatment practices of individual providers are consistent and similar amongst providers within the same facility (i.e., correlated), within-facility variation tends to be reduced. Because of this, after a certain point, adding prescriptions to a sample within a facility adds little new information. The principal source of variation in the country situation will tend to be differences in practices between health facilities. Increasing the number of facilities in a sample is the best way to improve accuracy. It is generally unwise to generalize about a group of facilities if that sample includes fewer than 20 facilities.

If resources allow, increasing the number of geographical areas and facilities/outlets above the minimum numbers quoted will increase the accuracy of the survey, however the sample sizes recommended have been shown to be sufficiently accurate and balance accuracy and investment of funds and time to carry out the survey.

Experience with measuring prescribing indicators in similar types of health facilities has shown that samples similar to those recommended in this survey result in estimates that are accurate 95% of the time within a range of plus or minus 7—9% in the worst case. Facility-specific indicators, such as estimates of availability of medicines, have a sample size equal to the number of health facilities or pharmacies, so they will tend to have a somewhat wider range of error.

1 How to investigate drug use in health facilities. Geneva, World Health Organization, 1993.

WHO/DAP/93.1.


- 86 -

Margin of error (half of 95% confidence interval) around a proportion of 0.5 (worst case) estimated from sample sizes ranging from 50 to 500

Mar

gin

of e

rror

Sample size

Error due to simple random sampling Box 11 Error in sampling

As shown in the figure to the right, the margin of error around an estimate from a simple random sample gets smaller as sample size increases. A percentage estimated as 50% in a sample of 100 units has a 95% confidence interval of +/- 10%, which means that 95 random samples out of 100 drawn from the same population would yield estimates between 40% and 60%. If the sample size were increased to 300, the margin of error around the same estimate of 50% would be +/- 6%. An estimate of 50% is the worst case; percentages greater or smaller than 50% have somewhat narrower margins of error. Error due to clustering within a sample

When sample units (e.g., households, patients in a clinic) are drawn in clusters, as most large samples are, the calculation of the margin of error around a sample estimate is more difficult. Units within a cluster (e.g. households in the same village, patients treated at the same health facility) are often more alike than units drawn randomly from other clusters. In this case, the characteristics or behaviours of these units are said to be correlated. Examples of correlated behaviours might be where people in a village purchase their medicines, or which patients in the same facility receive the same treatment for a particular diagnosis. In the worst case, where the behaviours are perfectly correlated, knowing the value from one unit allows you to predict the value for other units in the cluster, so sampling additional units yields no new information.

The effect of within-cluster correlation is to reduce the effective sample size below the actual number of units sampled. To know how much the effective sample size is reduced requires knowing the value of the within-cluster correlation for a particular characteristic to be estimated; this is rarely known in advance. All in all, increasing the number of clusters and reducing the number of units within each cluster is the best way to increase the reliability of estimates. However, sample design will often be a balance between adding more clusters (which increases logistic complexity and cost) and getting a convenient number of units per cluster (to allow for work to be carried out in multiples of an entire day).

For the types of indicators measured in medicines use surveys, it is not unusual for the effective sample size to be reduced by half, depending on the actual within-cluster correlation and the cluster size. Using the figure above, this would mean that for a sample size of 300 (drawn as 20 clusters of 15 units), the effective sample size might actually be 150, and the margin of error (95% confidence interval) roughly +/- 7—9%.

This is a conservative estimate of the margin of error for most of the patient indicators in the Level II survey.


- 87 -

Issues to address in identifying sectors (public/private) to include in the sampling

Level II indicators measure selected aspects of the delivery of pharmaceutical services for routine outpatient primary health care. Prior to sampling, careful consideration needs to be given to where patients access the pharmaceutical sector for routine outpatient care. This will ensure a study population representative of the reference population–which is the national population.

Consideration also needs to be given to the structure of the pharmaceutical sector in order not to exclude parts of the sector from the sampling process. In particular, is publicly-supported health care provided only in public facilities or also in the not-for-profit or private sectors, where is primary health care delivered, and which warehouses supply the public sector?

Among the main considerations in drawing the sample of facilities are:

• Is publicly-supported health care provided only in public facilities or also in the not-for-profit or private sector?

• Is primary health care delivered in general outpatient clinics of hospitals?

• Are public sector supplies obtained from public or private warehouses?

• Are medicines purchased from private pharmacies and drug outlets?

• Are medicines sold in the formal and informal sectors?

These issues will be discussed further below. As a general rule, if non-public sector sources represent greater than 25% of publicly-funded service delivery, then they should be included in the survey.

• Is publicly supported health care provided by not-for-profit or private sectors?

The survey is primarily looking at the delivery of pharmaceutical services in the public sector. The "public sector" indicators are intended to measure issues around access, quality and rational use of medicines that patients obtain from the public sector. The delivery of public sector services may vary from country to country. Sometimes a significant amount of publicly supported care may either officially or in practice be delivered by not-for-profit, NGO or mission providers or even the private sector. In such a country, these sectors should be sampled to represent the proportion of care they provide.

• Is primary health care delivered in general outpatient clinics of hospitals?

Primary health care is delivered in health centres and clinics, but hospitals may also have large primary care clinics, especially in urban areas. For this reason, 5 hospitals (1 per geographical area) have been included in the general survey methodology. The hospital-based survey must be in the department(s) serving general outpatients and not specialist clinics; this should apply not only to patient interviews, but also to the retrospective and prospective surveys of patient records and prescriptions.

Patient samples should be restricted to general illness encounters, representing a mix of health problems and ages. These indicators have limitations when applied to well-child visits, pre- and post-natal visits, specialist consultations or even separate


- 88 -

clinics for adults and paediatric cases because treatment practices are different and results may be difficult to interpret.

If general primary health care services are delivered primarily in hospitals in a given country, the proportion of hospitals in the survey might need to be increased. Alternatively, the proportion may need to be reduced if hospitals provide an insignificant proportion of primary health care services.

• Are public sector supplies obtained from public or private warehouses?

The survey requires collection of data from 5 public sector central/regional/district warehouses. The intention is to measure the availability of medicines in the supply chain at the level(s) above the health facility, regardless of whether these warehouses are managed and owned by the public sector, private sector or a public-private partnership. The sample should be drawn from the locations from which the public sector facilities obtain their medicine supply.

Annex 3 Coordinator checklist

- 89 -


Coordinator Checklist

Task Completed

Organize the Survey Process

Approval 1. Seek approval for the survey from government officials and necessary local agencies.

Meeting 2. Many of the pre-training tasks may need to be done in consultation with government officials, expert groups and other interested parties. Consider arranging a meeting of these persons to complete them.

Budget 3. Draw up a budget and, if necessary, request financial and technical assistance.

Schedule 4. Schedule completion of Level I Questionnaire, Level II training and survey period, data analysis, report preparation, and workshops to discuss results.

Level I 5. Coordinate and identify sources of information for Level I Questionnaire.

Level II 6. Select geographical areas for Level II surveys.

Site and Personnel Selection

7. Select facilities for Level II survey sites. As a minimum, the 30 public health facilities and 5 warehouses should be selected centrally. The 30 private drug outlets may be selected centrally or by data collectors once they are in the field.

8. Select 3 public health facilities with dispensaries near the training site for field tests. If the private drug outlets will be selected centrally for the actual survey, then they should be selected centrally for the field test as well. Otherwise the drug outlets can be selected by the data collectors once they are in the field. Facilities and outlets visited during the field test must be different from those selected for the actual survey.

9. Prepare and send letters seeking permission from officials responsible for selected facilities to conduct the field test and actual survey. Letters must include details of all areas where access is needed. Depending on the facility, this is the last 12 months of outpatient records, the last 12 months of under-fives paediatric ledgers and/or the last 12 months of stock records. Depending on the quality of records, it may be necessary to access current treatment records.

10. Select 10 data collectors.


- 90 -


Task Completed

11. Assign data collectors into teams and assign teams to geographic areas. Fill in assignments on Slide 27 (Data collectors training).

12. Identify who will encode/tabulate the data and do the computation.

Tailor Level II

13. Identify 15 key medicines (use morbidity data to aid selection) and pre-print on Survey Forms 1, 5, 10, 11 and 15

Survey Forms

14. Review language on Survey Forms 5, 13 and 17 to ensure it fits with normal terminology. During training ensure data collectors are using uniform criteria for evaluating items on the checklists.

15. Identify an additional condition and treatment of choice to treat pneumonia and the additional condition in children and adults and pre-print medicine, preparation, and number of units needed to complete treatment on Survey Forms 3 and 12.

16. Identify lowest daily government salary and pre-print on Survey Forms 3 and 12.

17. Determine the requirements for adequate labelling and patient knowledge if different from those listed. Modify criteria printed on forms if changes are made.

18. Identify current approved STG and EML to be checked when completing Survey Form 8 and obtain copies of each for each survey team.

19. Define 1st -line antibiotics for mild/moderate pneumonia and identify any optional conditions and medicines that will be used to measure recommended or non-recommended practices for Survey Form 9 and pre-print on the form.

20. Determine if any additional information is to be gathered by data collectors.

21. Define what data collectors should categorize as antibiotics, make a reference list of antibiotics and develop guidelines for data collectors to categorize generic versus branded medicines.


- 91 -


Task Completed

Prepare for Level II Training

Official Letters

1. Prepare official letter of introduction for data collectors.

Copies 2. Print and photocopy all materials for training session and field test:

• One copy per data collection team of the official letter of introduction

• Two copies per person of a complete set of survey forms, if possible, copy survey forms to be completed at public health facilities, private drug outlets, and warehouses on different coloured paper

• Two copies per data collector of a complete set of summary forms

• One copy per data collector of this Manual

• One copy per data collection team of the national EDL and STGs required for Survey Form 8

• One copy per data collector of antibiotic definition and reference list

• One copy per data collector of guidelines on categorizing generic versus branded medicines

• Additional materials such as pens, calculators and others, as indicated on Slide 29 (Data collectors` training)

Packets 3. Staple survey forms to be completed at the same facility together in order to ensure none are missed


- 92 -


Task Completed

4. Assemble field test packets for all three data collection teams (advise data collectors to retain these documents for reference during the actual survey). Each team should receive:

• Official letter of introduction to the facilities and households to be surveyed

• One copy per person of Survey Forms 1—17

• One copy per person of this Manual

• One copies per team of the National EDL and STGs required for Survey Form 8

• One copy per person of the antibiotic definition and reference list

• One copy per person of the guidelines on categorizing generic versus branded medicines

• One calculator per person

• Two pencils with erasres per person

• Necessary transport costs

5. Assemble discussion packets for data collectors:

• Field test packets

• One copy per person of a complete set of survey forms and summary forms

6. Prepare information on transport, distanceand security for each data collector

Training Slides

7. Obtain sample outpatient lists, patient records, and sample stock cards and copy onto a transparency or photocopy for distribution to data collection teams (these will be used during the discussion)

8. Fill in Slide 27 (team assignments)

9. Fill in Slide 15 (list of key medicines)

Other Materials

10. Compile one copy of a training reference packet including: completed Level I questionnaire, current national medicines policy document, relevant laws and regulations on medicines, information on enforcement and implementation of medicines laws, and information on pharmaceutical inspections


- 93 -

During Level II Training

Note Taker

1. During training arrange for someone to take notes of all decisions on how to interpret forms, changes in instructions, solutions to problems experienced in field test, etc. and prepare a list for distribution to the data collectors before they go into the field

Review 2. Review materials from field test to ensure proper completion of forms

After Level II Training

Official Letter

1. Prepare an official letter of introduction to the local health authorities, facilities to be surveyed

Survey Forms

2. Ensure key medicines are pre-printed on Survey Forms 1, 2, 3, 4, 10, 11, 12, 15 and 16; preparation and unit are pre-printed on Survey Forms 2 and 11; second condition, medicines of choice, their preparation, number of units needed to complete treatment and lowest daily government salary are pre-printed on Survey Forms 3 and 12; second condition is pre-printed on Survey Form 8, and, if additional tracer conditions are selected, the appropriate drugs are pre-printed on Survey Form 9

3. Pre-print geographic name and investigators

Copies 4. Print and photocopy all materials for actual survey: • One copy per data collection team of the official letter of

introduction to the local health authorities and facilities to be surveyed

• Six copies per team of Survey Forms 1—17. If possible, copy survey forms to be completed at public health facilities, private drug outlets, and warehouses on different coloured paper. Spare copies of forms should also be provided to data collectors

• One copy per data collector of the notes on decisions made during the training session (see point 1 of During Level II Training, above)

And, depending on whether changes were made and whether or not data collectors retained the documents from the training session:

• One copy per data collector of this manual • One copy per data collection team of the national EDL

and STGs required for Survey Form 8 • One copy per data collection of the antibiotic definition

and reference list • One copy per data collection of the guidelines on

categorizing generic versus branded medicines


- 94 -

Packets 5. Staple together survey forms to be completed at the same

facility in order to ensure none are missed

6. Assemble Actual Survey Packets for each data collection team: • One copy of the official letter of introduction to the local

health authorities, facilities to be surveyed, and village/hamlet leaders

• Transport costs • Per diem • Six copies of Survey Forms 1—17 • Two copies of the notes on decisions made during the

training session (see point 1 of During Level II Training, above)

And, depending on whether changes were made and whether or not data collectors retained the documents from the training session:

• Two copies of this Manual • Two copies of the National EDL and STGs required for

Survey Form 8 • Two copies of the antibiotic definition and reference list • Two copies of the guidelines on categorizing generic

versus branded medicines • Two calculators • Four pencils with erasers

Supervise Level II

7. Supervise the actual Level II survey, ensure that all data points are collected, that sampling is random, and survey forms are completed correctly

Survey 8. After the survey, check again that all data points have been collected, random sampling was used, and that the forms were completed correctly. Also check the computations

9. Collect written reports from data collectors on the data collection process and in particular anything that will be important in interpreting the results

Analyse Data

10. Provide copy of Survey Forms to data analyser to complete Summary Forms 1—4

Report Writing

11. Provide copy of Summary Forms, graphs, and data analysis and notes from data collectors to report writer

12. Oversee writing of report

Present Results

13. Coordinate presentation/feedback of results

Annex 4 Example of budget allocation

- 95 -


Item Cost (US$)

Training to do the survey (10 people at US$10.00 each for 2 days’ snacks)

200.00

Field test allowance (10 people at US$15.00 each) 150.00

Actual survey * (US$50.00 per day per diem for 10 people for 6 days each)

3000.00

Transportation and communication 600.00

Materials (printing costs, paper, pens, calculator, etc.) 500.00

Analysis and computation 600.00

Other expenses and fees** 2000.00

TOTAL 7050.00

* For the actual survey, the rates should be computed based on the sites to be visited bearing in mind the need to provide expenses for board, lodging and transportation

** Include professional and coordinator fees and other services that may be needed


- 96 -

Annex 5 Guide how to use training slides (level II indicators)

- 97 -

Box 12 Overview of the core indicator pyramid (Slide 7)

Level I is a questionnaire on the existing infrastructures and key processes of each component of the pharmaceutical sector. The completion of the questionnaire can be accomplished in a relatively short time after identifying sources of accurate information. For Level II indicators, a systematic survey has been developed to assess the degree of access to and rational use of quality drugs. The operational procedure to carry out the survey is the focus of this package. Level III indicators provide detailed evaluation and analysis of the pharmaceutical system and different key components.

Annex 5: Guide on how to use training slides (level II indicators)

Introducing indicators for monitoring and assessing the pharmaceutical situation (Slides 1—8)

Slides 1—4 introduce the activity, why monitoring is important and who can use the results. Go through the slides and emphasize how the results can be used to improve the pharmaceutical sector.

Slides 5 to 8 discuss indicators as basic tools in monitoring, pointing out that it is important to use a standardized procedure to enable comparisons. WHO has developed a hierarchical approach to monitoring and assessing country pharmaceutical situations. This approach is built around three groups of core indicators: Levels I, II and III. Following the standard methodologies developed to assess these indicators, comparisons can be made in the pharmaceutical situation over time and among different facilities, districts and countries.

The core indicators can be collected easily using the sampling design, survey forms and survey process described in this manual. Monitoring and assessment using indicators must be done at least every three years to assess trends and progress towards achieving the objectives of the pharmaceutical sector. Explain the pyramid (slide 7), noting that to get information on the pharmaceutical situation at the national level, a questionnaire on Level I indicators will be used. This training focuses on how to get information for Level II indicators.

The survey sites (Slide 9)

Using slide 9, explain the logistics of the survey.

Indicators (Slides 10-12)

Here indicators are presented according to the surveyed site.

The survey forms (Slides 13-25)

How the survey teams carry out the survey affects the accuracy and validity of the data. Consistency in data gathering and completing the survey forms is very important. The next discussion will review Survey Forms 1—17. A complete set of survey forms for discussion must be provided to each of the participants. Go through


- 98 -

all the survey forms and discuss the notes at the bottom of each form. It is important that all members of the survey team have the same understanding of each column and how and where to get the information. They should also know how to calculate the values on each of the forms. The survey forms have been designed to facilitate manual computation in the field.

Slide 13 gives the key topics to be discussed.

Slide 14 presents the basic criteria used for the previous selection of the key essential medicines used in this survey. The selected essential medicines should be for common health conditions and should be ones that are expected to be always available at public facilities providing primary health care. List the selected key medicines on Slide 15 and review them with the participants. Caution data collectors to be careful when completing the survey forms as the same chemical entity can have several different product names.

Slide 16 reviews how to verify compliance with the law, availability of key essential medicines and the presence of expired medicines (Survey Forms 1, 10 and 15). The key essential medicines that have been selected for evaluation in this survey should be pre-printed on the survey forms.

Slide 17 reviews how to verify medicines prices (Survey Forms 2 and 11). The key medicines list should be pre-printed on the survey forms.

Slide 18 corresponds to Survey Forms 5, 13, 17. The survey team should have a common understanding of how to check conservation conditions. Go through the checklist and discuss the local situation. Ensure that everyone has the same understanding of when to indicate "true" or "false", for instance decide on the appropriate methods to control temperature for the climate.

Slide 19 and the notes on the bottom of Survey Forms 3 and 12 explain how to measure affordability to treat pneumonia and another condition. The formula is on the survey form.

Slide 20 explains how to select patients for the exit interview required for Survey Forms 6 and 14. The exit interview of patients will obtain information on patient care and cost of treatment. It should be done after patients have been treated and received medicines, either near the dispensing area or as they are leaving the facility. It is important to construct a system that will allow the patients to be interviewed without disrupting the normal activity in the facility. Note that patient interviews should be timed to match peak hours at facilities in order to have enough patients to complete the survey forms.

Thirty (30) consecutive patients or any 30 patients after they have been treated and have received their medicines must be interviewed for the following information:

• The number of medicines prescribed and dispensed and, in the private drug outlets, the number of prescription medicines purchased without a prescription.

• Whether each of the dispensed medicines is appropriately labelled with at least the name, dosage and duration written on the package given to the patient.

• Whether the patient knows both the appropriate dosage and duration of each medicine.


- 99 -

• The amount paid out-of-pocket for medicines and non-diagnostic fees, i.e. visit or injection fees, but not lab or x-ray fees.

• How long it took to reach the facility that day.

• How much did it cost to reach the facility that day.

Data collectors should be trained how to identify patients, how to conduct appropriate and respectful interviews to determine whether patients know how to take the medicines and the amount they paid out of pocket for medicines and fees and how to determine whether medicines are adequately labelled. The logistics of intercepting patients for exit interviews should be practised.

Slide 21 is an example of a sampling procedure that can be used to select patient records for completing Survey Form 7 (see also Sampling patients, below). Data collectors should be able to do the following:

• Confirm the availability and accessibility of medical records.

• Identify possible sources of retrospective prescribing data, including clinic registers, treatment logbooks, patient/family files, and retained prescription forms. The data collector must know where to locate these records in the facilities and how to use them.

• Identify the study period to be covered.

In order to complete Survey Form 7 consistently, data collectors also need to use common criteria for identifying which of the prescribed medicines to classify as injections and antibiotics. Immunizations and injectable contraceptives should not be counted as injections. Definitions of medicines considered as antibiotics must be discussed. Antimicrobial agents are not always classified in an identical way. Indicators for antibiotic use can be sensitive to certain medicines, especially in places with high incidence of parasitic infections, such as malaria or tuberculosis. Medicines such as antiprotozoals and antihelminthics are also usually placed in a different category of antibiotics. How to classify topical antibiotics widely used in areas where trachoma, bacterial conjunctivitis and bacterial skin infection are common will also need to be considered.

Sampling patients

Techniques for conducting retrospective and prospective sampling should be practised. An exercise using patient lists to select sample encounters is necessary. Data collectors should know how to get information from patient records both for general outpatients (Survey Form 7) and for cases of diarrhoea, pneumonia and ARI (Survey Form 9).

Retrospective sampling

Because of seasonal differences, variations in staffing, inconsistencies in the supply cycle etc., the survey should, as much as possible, cover the 12-month period prior to the date of data collection. If records during this period cannot be found or do not meet the sampling requirements, the study period can be shortened, making sure that there are no missing records during the period covered. This can be done by checking that all the months and days are represented in the record.


- 100 -

To complete Survey Form 7, data collectors should select 30 patients with any diagnosis seen during the last 12 months from general outpatient treatment records. Either the chronological or alternative sampling method described in Box 7 may be used. Both sampling methods should be practiced during training. If records are not available, 30 patients may be selected from current treatment records provided there are sufficient records to randomly select the patients. If medical records are not available or too difficult to extract, it may be preferable to use prospective sampling methods (see description under slide 14 above).

Retrospective and prospective samplings are described in further detail in the main document section on sampling patients for data collection. .

Slide 22, discuss criteria to identify medicines to be considered as injections and antibiotics.

Slide 23, discuss with participants which Standard Treatment Guidelines (STG) to use for the survey. The most current national STGs and Essential Medicines Lists (EML) should be distributed to data collectors to use for comparison purposes at the facilities. Survey Form 8 notes that the facilities must be able to show the documents to the survey team. Not knowing where the documents are located is an indication that they are not being used.

Use Slide 24 to discuss how to select patients with specific conditions from an outpatient list. Each record will be reviewed to assess compliance to recommended treatment protocols in Survey Form 9. Thirty (30) patients seen during the previous 12 months should be sampled: 10 cases of non-bacterial diarrhoea in children under age 5, 10 cases of moderate pneumonia in children under age 5, and 10 cases of non-pneumonia acute respiratory tract infection in patients of any age. See Box 7 and Retrospective sampling above for sampling instructions.

Use Slide 25 to explain how to perform the retrospective sampling to selecting 30 patients from general outpatient records covering the past 12 months (365 days).

The survey team (Slides 26 and 27)

Slide 26 give a resume of the survey sites and data collectors’ fieldwork organization. Slide 27 can be filled out before or during the training so that the assignments may be discussed.

Final considerations and general recommendations (Slides 28 — 36)

Slide 28 gives a resume of all survey forms and their respective utilization.

Use Slide 29 to explain materials needed in the field work and how to use them.

Use Slide 30 to explain how to start working in each kind of facility surveyed.

Slide 31 summarizes general instructions on conducting the field work.

Use Slide 32 to emphasize the care needed to conduct appropriate, good quality data collection. Data collectors must conduct themselves appropriately during the field work. They must be respectful with all those interviewed and always ask for their permission to conduct the interview. They should stimulate people to collaborate with the survey but respect if they do not agree. They should have a good time control to complete all tasks within the estimated time. Per diem and all the logistics involved are forecast according to this. Forms must be completed in full out and reasons for missed data should be adequately indicated. Data must be recorded as they are, good or bad. Data collectors should not divulge administrative or clinical data they have accessed during field work.


- 101 -

The summary forms

Calculations indicated at the bottom of each Survey Form must be accurate. Briefly explain that all data will be typed on Summary Forms 1—4. If data are incomplete, the quality of the analysis and the entire work and investment will have been wasted.


- 102 -

Annex 6 Training slides for data collection

- 103 -

Annex 6: Training slides for data collection

A guide to fieldwork:Monitoring and Assessing theMonitoring and Assessing thePharmaceutical Situation in Pharmaceutical Situation in CountriesCountries

Target audience: Data collectorsTarget audience: Data collectors

A guide to fieldwork:Monitoring and Assessing theMonitoring and Assessing thePharmaceutical Situation in Pharmaceutical Situation in CountriesCountries

Target audience: Data collectorsTarget audience: Data collectors

2

OutlineOutline

1st Day1st DayIntroductionDiscussion of the survey forms

2nd Day2nd DayFieldtestComputing the results of the fieldtest

3rd Day3rd DayDiscussing fieldtest and data input in the summary formsSurvey logistics

2

OutlineOutline

1st Day1st DayIntroductionDiscussion of the survey forms

2nd Day2nd DayFieldtestComputing the results of the fieldtest

3rd Day3rd DayDiscussing fieldtest and data input in the summary formsSurvey logistics


- 104 -

3

Why monitor and assess?Why monitor and assess?

To assess country capacity (infrastructures and resources)

To review implementation strategies so adjustments can be made

To measure outcome of pharmaceutical objectives (access and rational use of quality medicines); and

To evaluate progress towards identified objectives

• Introduction

• Discussion of the survey forms

• Survey logistics

4

Who can use the results from Who can use the results from assessment and monitoring?assessment and monitoring?

§ Countriesto focus action, prioritize, measure achievement

§ National policy-makersto synchronize health and economic policies

to get a clear picture of national problems and identify and prioritize strategies

to present the performance of the pharmaceutical sector to donors and other governmental agencies

§ Health facilitiesto be aware of the institutional problems and identify strategies to improve the situation

§ International agenciesto assess the structure and capability of countries when developing new projects

to assess the progress and accomplishment of current projects

to assess the impact of aid

§ Professional groups, NGOs and acadaemiato focus their activities on advocacy activities and informationcampaigns

IntroductionDiscussion of the survey formsSurvey logistics

4

Who can use the results from Who can use the results from assessment and monitoring?assessment and monitoring?

§ Countriesto focus action, prioritize, measure achievement

§ National policy-makersto synchronize health and economic policies

to get a clear picture of national problems and identify and prioritize strategies

to present the performance of the pharmaceutical sector to donors and other governmental agencies

§ Health facilitiesto be aware of the institutional problems and identify strategies to improve the situation

§ International agenciesto assess the structure and capability of countries when developing new projects

to assess the progress and accomplishment of current projects

to assess the impact of aid

§ Professional groups, NGOs and acadaemiato focus their activities on advocacy activities and informationcampaigns



- 105 -

5

Challenges for monitoring and assessment Challenges for monitoring and assessment of public policiesof public policies

§ It is difficult to establish a sustainable system of regular monitoring

§ Resources are not consistently allocated to this task § There is limited advocacy for a culture of monitoring § Many efforts to develop monitoring tools have been

exhaustive, but impractical§ Most monitoring tools include indicators that are difficult to

collect, especially if done regularly


5

Challenges for monitoring and assessment Challenges for monitoring and assessment of public policiesof public policies

§ It is difficult to establish a sustainable system of regular monitoring

§ Resources are not consistently allocated to this task § There is limited advocacy for a culture of monitoring § Many efforts to develop monitoring tools have been

exhaustive, but impractical§ Most monitoring tools include indicators that are difficult to

collect, especially if done regularly


6

Why is it important to use Why is it important to use indicators?indicators?

Monitoring and evaluation of standard indicators facilitates:

comparing the performance of facilities, districts, provinces and the overall situation in the country

seeing trends over time

setting targets

• Introduction




- 106 -

7

Core indicators to monitor national Core indicators to monitor national medicine policymedicine policy

• Introduction



Level ICore structure


Level IICore outcome/ impact indicators• Access to essential medicines• Rational Use of Medicines

Level IIIIn depth assessments of specific components of the Pharmaceutical sector

- Pricing - Traditional Medicines- HIV/ AIDS - Regulatory Capacity- Drug Supply - TRIPS

Key informantquestionnaire sent to countries

Systematic survey (health facilities and

households)

Diverse methodologies

7

Core indicators to monitor national Core indicators to monitor national medicine policymedicine policy

• Introduction



Level ICore structure


Level IICore outcome/ impact indicators• Access to essential medicines• Rational Use of Medicines

Level IIIIn depth assessments of specific components of the Pharmaceutical sector

- Pricing - Traditional Medicines- HIV/ AIDS - Regulatory Capacity- Drug Supply - TRIPS

Key informantquestionnaire sent to countries

Systematic survey (health facilities and

households)

Diverse methodologies

8

Core indicators for monitoring and Core indicators for monitoring and assessing country pharmaceutical assessing country pharmaceutical situationssituations

Developed to provide a indicator-based tool intending to be:

practical regularly implemented without investing large amounts of human or financial resources.

The core indicators can be easily collected using:

standardized methodologiessmall samples of data simple survey techniques

These core indicators measure systematically the most important information needed to gain a comprehensive picture of the pharmaceutical situation in a country.

• Introduction




- 107 -

9

Survey sitesSurvey sitesFrom each of the five geographic the following are selected:

6 public health facilities treating general outpatients and with a medicines dispensing unit:

1 public hospital (should be the largest facility with general public outpatient services in the area)1 primary/rural health centre or lowest level public health facility4 health facilities randomly selected from all middle level public health facilities, i.e. excluding primary/rural health units andhospitalsprivate, non-governmental or mission health facilities may be added if they represent more than 25% of service delivery

6 private drug outlets

Select the private drug outlet closest to each public health facility included in the survey

1 warehouse

• Introduction• Discussion of the survey forms


10

Level II indicators and survey formsLevel II indicators and survey formsPublic health

facilities and their

pharmacies

Private pharmacies/drug outlets

Warehouse Indicator

Survey Form Access 1 Availability of key medicines 1 10 15 2 % of prescribed medicines dispensed or administered to

patients 6

3 Average stockout duration 4 16 4 Adequate record keeping 4 16 5 Affordability of treatment for adults and children under 5

years of age 3 12

6 Price of key medicines 2 11 7 Price of paediatric medicines 2 11 8 Average cost of medicines and related fees 6 14 9 Geographical accessibility of dispensing facilities 6 14

• Introduction




- 108 -

11

Level II indicators and survey formsLevel II indicators and survey forms

Public health


pharmacies


Warehouse Indicator

Survey Form Quality 1 % medicines expired 1 10 15 2 Adequacy of conservation conditions and handling of

medicines 5 13 17

• Introduction



12

Level II indicators and survey formsLevel II indicators and survey formsPublic health


pharmacies


Warehouse Indicator

Survey Form Rational use of medicines 1 % medicines adequately labeled 6 14 2 % patients know how to take medicines 6 14 3 Average number of medicines per prescription 6, 7 4 % patients prescribed antibiotics 7 5 % patients prescribed injections 7 6 % prescribed medicines on the essential medicines list 7 7 % medicines prescribed by generic name (INN) 7 8 Availability of standard treatment guidelines 8 9 Availability of essential medicines list 8 10 % tracer cases treated according to recommended treatment

protocol/guide 9

11 % costumers purchasing prescription medicines without prescription

14

• Introduction




- 109 -

13

Level II indicators and survey formsLevel II indicators and survey forms

Public health


pharmacies


Warehouse Indicator

Section

Other information

1 % of facilities that comply with the law (presence of a pharmacist)

A C

2 % facilities with pharmacist, nurse, pharmacy aide/ health assistant or untrained staff dispensing

A C

3 % facilities with doctor, nurse, trained health worker/health aide prescribing

B

4 % facilities with prescriber trained in RDU B

14

Data collectionData collection

Completing the survey (use notes on survey forms as guide)

checking key medicines (1, 10, 15) checking prices of key medicines (2, 11)getting treatment costs (3, 12)using stock records to measure stock out duration (4, 16)using a checklist for inspection (5, 13, 17)selecting patients for exit interview (6, 14)sampling patients from outpatient list at health facility (7, 9)checking availability of guidelines (8)

Calculating and filling in indicator values for the 17 survey forms and 4 summary formsCompleting Summary formsReportingSharing lessons learned

• Introduction




- 110 -

15

Diarrhoea Oral rehydration salts, cotrimoxazole tabletsAcute respiratory tractinfection

Amoxycillin, cotrimoxazole tablets, procaine penicillininjection, paediatric paracetamol tablet

Malaria Choloquine tabletsAnaemia Ferrous salt + folic acid tabletsWorm infestations Mebendazole tabletsConjunctivitis Tetracycline eye ointmentSkin infection Iodine, gentian violet or local alternativeFungal skin infection Benzoic acid + salicylic acid ointmentPain Acetylsalicylic acid or paracetamol tabletsProphylactic drugs Retinol (Vit.A) ferrous salt + folic acid tablets

Select 15 key drugs for common health problems

These must be drugs that should always be available at all facilities providing primary health careThese must be on the National Essential Medicines List

These must be the most important therapeutically, based on national treatment guidelines or at least on the consensus of experts

The most widely used of the drugs meeting the above criteria

Model list of key drugs

Selecting key Essential Drugs• Introduction

• Discussion of the survey forms• Survey logistics

16

Diseases Key drugs 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15.

List of key medicines ________________List of key medicines ________________(name of country)(name of country)

• Introduction



This must be filled up during the discussion


- 111 -

17

Section A – General information

Observe if there is a pharmacist present at the moment of the visit and register on the form

Survey Forms 1, 10, 15

Check which of the key essential medicines listed are available at the facilityCheck if there are expired key medicines on the shelves

Verifying law compliance, availability of Verifying law compliance, availability of key essential medicines and presence of key essential medicines and presence of expired medicinesexpired medicines

• Introduction



18

Survey Forms 2, 11

Determine the lowest price the facility and patient (out-of-pocket) pay for each key medicine

Price should correspond to the exact preparation and unit indicated in column B

Medicines needed to calculate indicators on SF3 and SF12 are included in this form

Medicine pricesMedicine prices• Introduction




- 112 -

19

Survey Forms 5, 13, 17

Establish a common definition and understanding of each item on the checklist

Conditions must be defined within the context of actual situations at storage and dispensing areas

Conservation conditions and handling of Conservation conditions and handling of medicines in dispensing and storage areasmedicines in dispensing and storage areas

• Introduction



20

Survey Forms 3 and 12 - Calculating the cost of treating uncomplicated pneumonia and another condition

This following information are to be already pre printed on the forms:

The treatment of choice for adults and childrenThe lowest daily salary of a government workerA second standard of measure, if desired

In the field, data collectors will:

Record the unit price, including syringe if applicableCalculate the total cost of treatment and the affordability of the treatment using the formulas indicated

Affordability of treatmentAffordability of treatment• Introduction




- 113 -

21

Survey Form 6, 14

Interview 30 consecutive patients or any patient who has been treated after medicines have been dispensed

Check each medicine label for the medicine name, dosage and duration. An adequate label must have all three written on it.

Ask the patient how to take each medicine. The patient must know the dosage and duration of all dispensed medicines.

Ask the patient how much was paid out-of-pocket for medicines and in other fees.

Patient care Patient care -- how to select how to select patients for exit interviewpatients for exit interview

• Introduction



22

Prescribing practice Prescribing practice -- how to do systematic how to do systematic sampling of outpatients for retrospective reviewsampling of outpatients for retrospective review

Survey Form 7

Simple method for selecting 30 patients from the general outpatient list

If there are 5000 patients treated during the entire period and the sample size is 305000 ÷ 30 = 166.6 is the sampling intervalselect the first patient using table of random number or just point to one patient at ran dom. This is the 1st patient add 166.6 to get the next patient and so on.

Notes: Cover one year period or current treatment records sufficient to randomly select 30 patients. Prescribing indicators have limitations when applied to well-child visits, pre and post-natal visits, specialist consultations, or even separate clinics for adults and paediatric cases because treatment practices are different and results may be difficult to interpret.

• Introduction




- 114 -

23

Retrospective sampling: Selecting 30 Retrospective sampling: Selecting 30 patients from general outpatient records patients from general outpatient records covering 12 months covering 12 months (1)(1)

Chronological sampling method ( SF 7)

Calculate the sampling interval by dividing the number of days covered by the outpatient list by the number of patients to be selected: 365 ÷ 30 = 12.2 days

Number each day covered by the outpatient list

Randomly select one patient encounter from the between day 1 and day 12.2

Each subsequent encounter is selected by adding the sampling interval (12.2) to the previous total and rounding up. In other words, select one patient encounter from first day that was randomly selected

Example 1st patient was selected from day 3, Second patient is selected by adding the sampling interval (3 + 12.2 = 15.2 =16 always round up) thus select one patient from day 16Third encounter = 15.2 + 12.2 = 27.4 = 28th dayFourth encounter = 27.4 + 12.2 = 39.6 =40th dayFifth encounter = 39.6 + 12.2 =51.8 = 52th day

• Introduction



24

Retrospective sampling: Selecting 30 Retrospective sampling: Selecting 30 patients from general outpatient records patients from general outpatient records covering 12 months (2)covering 12 months (2)

To choose a patient encounter from the patient list for each selected day, pick a random number between 0.0 and 1.0, multiply this number times the number of patient encounters on the list, and round upwards.

For example, if you choose 0.4 and there are 18 patients on the list for the selected day: 0.4 x 18 = 7.2 = 8 (always round up). Review the 8th patient encounter when completing the survey forms provided the patient meets the inclusion criteria. If the patient does not meet the inclusion criteria, select the next patient on the list who does. Repeat this process, selecting one patient encounter from each selected day until 30 patient encounters have been reviewed

• Introduction




- 115 -

25

Identifying medicines to be considered Identifying medicines to be considered as injections and antibiotics (SF 7)as injections and antibiotics (SF 7)

Immunizations and injectable contraceptives should not be counted as injections.

Discussion on definitions of medicines considered as antibiotics

Antimicrobial agents are not always classified in an identical way

medicines such as antiprotozoals and antihelminthics are also usually placed in a different category of antibiotics

How to classify topical antibiotics widely used in areas where trachoma, bacterial conjunctivitis and bacterial skin infection are common will also need to be considered

• Introduction



26

Availability of standard treatment guidelines (STG) and essential medicines lists (EML) at the facility

Survey Form 8Check for presence of guidelines. The facility must be able to show the documents to count them as present.

Selected treatment guidelines may be included in the national STG or disease specific STGs.

Identify current national EML. Check for presence of each EML. Only count EMLs that have been updated within the last 5 years.

• Introduction




- 116 -

27

Survey Form 9

Randomly select 30 outpatients seen at different dates during a one year period using previous sampling method as appropriate

10 outpatient children under 5 years of age with non-bacterial diarrhoea; 10 outpatient children under 5 years of age with pneumonia; and 10 outpatients of any age with non-pneumonia, non-bacterial acute respiratory tract infection.

For each case, check whether the treatment corresponds to the description in the column [A]

Treatment of tracer diseases Treatment of tracer diseases --how to select patients and how to how to select patients and how to do the assessmentdo the assessment

• Introduction

• Discussion of the survey forms• Survey logistics

28

5 geographic areas of survey (adjustment can be made based on the number of facilities) totaling:

30 public health facilities and their pharmacies30 private pharmacies5 central/district warehouses

In each area, one team can complete:

In one and a half day: surveys of one public health facility and its pharmacy, the private pharmacy nearest to the public health facility, and one warehouse

With 6 teams of 2 persons each, the entire survey package can be completed in 8 days

The survey areas and the survey teamThe survey areas and the survey team

• Introduction




- 117 -

29

TEAM/NAME GEOGRAPHIC AREA FACILITY TYPE NUMBERS Team 1 Public facility 1 - 6 1. Private pharmacy 1 - 6 2. Warehouse 1 Team 2 Public facility 7 - 12 1. Private pharmacy 7 - 12 2. Warehouse 2 Team 3 Public facility 13 - 18 1. Private pharmacy 13 - 18 2. Warehouse 3 Team 4 Public facility 19 - 24 1. Private pharmacy 19 - 24 2. Warehouse 4 Team 5 Public facility 25 - 30 1. Private pharmacy 25 - 30 2. Warehouse 5

Team assignmentsTeam assignments• Introduction



30

Tailoring the survey forms to country Tailoring the survey forms to country situation situation

Items on survey forms to be predefine at national level Survey Form Item

All The names of the selected geographic areas and the numbers assigned to each facility should be pre-assigned and pre-filled on all SF

1, 4, 10, 15, 16 Identify basket of key medicines and paediatric medicines 2, 11 To determine drug prices, identify dosage forms and strengths of key medicines and

any optional medicines (as suggest by WHO) 5, 13, 17 Review language so fits with normal terminology. During training ensure data

collectors are using uniform criteria for evaluating items on the checklists for conservation and handling conditions.

3, 12 For affordability, identify treatment of choice for pneumonia and additional condition with treatment of choice

6, 14 Define the requirements for adequate labeling on drug dispensed and standard patient knowledge if different from those listed

7 Define what will be considered as an antibiotic, and injection Develop guidelines for data collectors on categorizing generic versus brand medicines

8 Identify the acceptable guidelines and medicines lists; obtain 1 copy of each for each survey team

9 Define 1st line antibiotics for mild/moderate pneumonia and identify any optional conditions and medicines that will be used to measure recommended or non-recommended practices

• Introduction




- 118 -

31

Materials neededMaterials needed

Survey forms (the cover sheet of the survey forms packet lists the necessary number of copies)

Pen, pencils, erasers, rulers

Calculator

Briefcase or folder (if possible, waterproof)

Endorsement letter

Identification card

Additional materials could be helpful: National Medicines List, Brand name ? Generic List, National Pharmaceutical Guidelines cover photocopy

Per diem or transport costs

• Introduction



32

In the fieldIn the fieldOn arrival at the facilities, data collectors will need:

to speak to the person in charge of the facility

to take a brief tour in order to be able to understand:

patient flowwhere to locate the desired patientsthe existence and location of the necessary records

It is essential that this is done before any data is collected otherwise incomplete or incorrect data might be gathered wasting time or invalidating the results

Report on data collection process data collectors should be asked to present it after completing the actual survey, specially anything that will be important for the report writer in interpreting the results

• Introduction




- 119 -

33

Checklist for data collectorsChecklist for data collectorsAt the geographic area level (region, province, district)1. Confirm that the necessary authorisation from the relevant authorities has been received

and check on local logistical and security issues. Obtain any necessary additional authorization letter

2. Identify the location of the facilities and the most efficient order in which to visit them

At the facility level (public health facility and private drug outlet)1. Take an initial tour of the facility and talk with key staff to understand the patient flow and

existence and location of records

2. Determine the best order to complete the survey forms and which data collector will complete which forms

3. Confirm the availability and accessibility of records

Location of general outpatient encounter records

Patient records

Stock records4. Decide on retrospective or perspective sampling for survey forms based on the

availability and accessibility of records

5. For retrospective sampling identify the study period to be covered

• Introduction



34

Additional instructionsAdditional instructions

Data collector behavior

Approaching interviewed people (professionals and patients)

Time control

Completeness and adequate filling the forms

Reporting data how they are: good or bad

Ethics and confidentiality


- 120 -

35

Preparation for the field testing to Preparation for the field testing to practice how to use survey formspractice how to use survey forms

Familiarise with the forms and identify where data for each survey forms can be obtained

Be ready for introducing yourself and the purpose of the field test

Purpose of the whole activity assessment- to measure access and use of medicinesThis particular filed test is to check the adaptability of the tool before the launch of the formal survey

GoodGood luckluck !!!!!!!!!!!!

Annex 7 Level II Survey Forms 1-17

- 121 -

Annex 7: Level II Survey Forms 1—17


- 122 -

Survey Forms 1–17 Number of copies needed for:

Survey Forms Training*

Field Test*

Survey **

Total number of copies needed

Section A - Public health facility pharmacies/dispensaries 10 10 30 50 SF 1 % key medicines available

% medicines expired 10 10 30 50

SF 2 Price of key medicines Price of paediatric medicines

10 10 30 50

SF 3 Affordability of treatment for adults and children under 5 years of age (pneumonia with no hospitalization)

10 10 30 50

SF 4 Average stockout duration Adequate record keeping

10 10 30 50

SF 5 Adequate conservation conditions and handling of medicines at storeroom and dispensing area

10 10 30 50

SF 6 Average number of medicines per prescription % medicines dispensed or administered % medicines adequately labelled % patients knowing how to take medicines Average cost of medicines Geographical accessibility of dispensing facilities

10 10 30 50

Section B - Public health facilities 10 10 30 50 SF 7 Average number of medicines per prescription

% patients prescribed antibiotics/injections % prescribed medicines on Essential Medicines List % medicines prescribed by generic name (INN)

10 10 30 50

SF 8 Availability of Standard Treatment Guidelines Availability of Essential Medicines List

10 10 30 50

SF 9 % tracer cases treated according to recommended treatment protocol/guide

10 10 30 50

Section C - Private pharmacies/drug outlets 10 10 30 50 SF 10 % key medicines available

% medicines expired 10 10 30 50


10 10 30 50


10 30 50

SF 13 Adequate conservation conditions and handling of medicines in the storeroom and dispensing area

10 10 30 50

SF 14 Average number of medicines purchased % prescription medicines bought without prescription % medicines adequately labelled % patients knowing how to take medicines Average cost of medicines Geographical accessibility of dispensing facilities

10 10 30 50

Section D - Central/regional/district warehouses supplying the public sector

SF 15 % key medicines available % medicines expired

10 10 5 25


10 10 5 25

SF 17 Adequate conservation conditions and handling of medicines in the storeroom

10 10 5 25

* Note each data collector should be provided with one copy of each survey form for use during training and another copy of each form for use during the field test

** Copies of survey forms for the actual survey should not be completed until after the country-specific items have been introduced (see section on tailoring the survey forms to country situation)


- 123 -

Country pharmaceutical situation — Level II Summary of instructions for field procedures

(print on the other side of page 1 — cover)

Materials required in field

• Survey forms (The cover sheet of the survey forms packet lists the number of copies to be reproduced)

• Pen, pencils, erasers, rulers • Calculator • Briefcase or folder (if possible, waterproof) • Endorsement letter • Identification card • Additional materials that could be helpful: National Medicines List, Brand

name ↔ Generic List, National Pharmaceutical Guidelines (STG) or photocopy of the cover

• Per diem or transport costs • Personal items: waterproof coat, repellent and etc.

Main procedures At the geographical area (region, province, district)

1. Confirm that all necessary authorizations/endorsement letter from the relevant authorities have been received. Check on local logistics and security issues.

2. Identify the location of the facilities and the most efficient way and sequence to cover all of them.

At the facility level (public health facility and private drug outlet)

1. Take an initial tour of the facility and talk with key staff to understand the flow of patients and the existence and location of records to review.

2. Determine the best order to complete all the the survey forms and the allocation of data collector to complete the forms.

3. Decide whether to do retrospective or prospective sampling for survey forms based on the availability and accessibility of records. Survey Form 6 and Survey Form 14 will always use prospective sampling.

4. Confirm the availability and accessibility of records Location of general outpatient encounter records Patient records Stock records

5. For retrospective sampling identify the study period to be covered Additional instructions are needed:

Data collector behaviour Approaching and interviewing people (professionals and patients) Time control Completeness and accuracy in filling the forms Objective reporting of data (good or bad) Ethics and confidentiality


- 124 -

Section A – Survey Forms 1-6

Survey Forms

A - Public health facility pharmacies/dispensaries SF 1 % key medicines available

% medicines expired




SF 5 Adequate conservation conditions and handling of medicines in the storeroom and dispensing area

SF 6 Average number of medicines per prescription % medicines dispensed or administered % medicines adequately labelled % patients knowing how to take medicines Average cost of medicines Geographical accessibility of dispensing facilities

General information: Public health facility pharmacy/dispensary Facility Date Region Investigator

1) Does the law require a pharmacist to be present during hours of operation

of public/government pharmacies/drug outlets?

Yes No 2) Is a pharmacist present at the time of the visit?

Yes No

Assessment 1 complies with the law (items 1 and 2 are both Yes) 2 does not comply with the law (item 1 Yes and item 2 No) 3 no requirement for pharmacist presence (item 1 No) 3) Who is dispensing during the time of visit? (check all that apply)

Pharmacist (1=Yes; 0=No) Pharmacy aide/ health assistant (1=Yes; 0=No) Nurse (1=Yes; 0=No) Untrained staff (1=Yes; 0=No)


- 125 -

Survey form 1: Public health facility pharmacy/dispensary Indicator: % key medicines available % medicines expired

Facility Date Region Investigator

Key medicines to treat common conditions In stock Yes=1, No=0

Expired medicines on shelves

Yes=1, No=0 [A] [B] [C]

1. 2.

3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15.

[B1] = Sum of B = [C1] = Sum of C = [B2] = % in stock = B1 ÷ 15 x 100 =

[C2] = % expired = C1 ÷ B1 x 100 =

Notes: [A] The lists of 15 key medicines identified at the national level and pre-printed on this survey forms. [B] Mark “1” if any quantity of any dosage form of the medicines is in stock in the facility on the day of the visit. Mark “0” if the

medicine is not available in stock. Add the total at the bottom [B1]. Calculate the percentage in stock [B2] by dividing the total in stock [B1] by 15 and multiplying by 100.

[C] For all medicines in stock, check if any of the stock is expired. If any amount of a medicine has an expiry problem, mark “1” for yes. Do not count expired medicines stored in a separate area for destruction. Add the total at the bottom [C1]. Calculate the percentage expired [C2] by dividing the total expired [C1] by the total number of medicines in stock [B1] and multiplying by 100.

Public Health Facility

Pharmacy

Facility #____ (1-30)


- 126 -

Survey form 2: Public health facility pharmacy/dispensary Indicator: Price of key medicines

Price of paediatric medicines*


Key medicines to

treat common conditions Preparation unit Lowest unit

price of the medicine

paid by patient

Lowest unit price of the

medicine paid by facility

[A] [B] [C] [D] 1. Amitriptyline 25 mg cap/tab 2. Amoxicillin 500 mg cap/tab 3. Atenolol 50 mg cap/tab 4. Captopril 25 mg cap/tab 5. Ceftriaxone 1g/vial injection 6. Ciprofloxacin 500 mg cap/tab 7. Co-trimoxazole 8 + 40 mg/ml cap/tab 8. Diazepam 5 mg cap/tab 9. Diclofenac 50 mg cap/tab 10. Glibenclamide 5 mg cap/tab 11. Omeprazole 20 mg cap/tab 12. Paracetamol 25 mg/ml syrup/susp 13. Salbutamol 0.1mg/dose inhaler 14. Simvastatin 20 mg cap/tab Paediatric medicines 1. Cotrimoxazole 40+200 mg/5 ml

(8+40 mg/ml) Suspen-sion

2. Vitamin A 100,000 unit capsule 3. Isoniazid 50 mg tab Medicines identified in Survey Form 3 to calculate affordability (Indicate the INN and preparation (dosage strength and form) Notes: [A&B] Medicines must be the exact preparation listed in column B. Note only those currently available in stock.

(Paediatric medicines include those that are difficult to obtain because of cost or poor availability of paediatric formulation). Indicate also all the medicines that will be identified in SF 3.

[C] For each medicine that is actually available in stock during the visit, determine the lowest unit price in the local currency paid out-of-pocket by a patient. If patients pay flat charges for each medicine dispensed, this amount should be recorded as the price of the medicine. Indicate a “0” if medicines are given free. If medicine or price data are not available, mark N/A.

[D] From procurement or purchasing records at the facility, determine the lowest unit price in the local currency paid by the facility in the last year. If facilities generally receive the medicine for free from the Ministry of Health, record the last price paid when the medicine was actually purchased. If the medicine has not been purchased in the last year or if price data are not available, mark N/A.


Pharmacy Facility #____

(1-30)


- 127 -

Survey form 3: Public health facility pharmacy/dispensary Indicator: Affordability of treatment - moderate pneumonia for adults and

children under 5 years of age (equivalent number of days’ wages)


Medicine/INN and Preparation

Number of units needed to complete treatment

Unit price (one vial, tablet,

or capsule)

Total cost of treatment

[D] = B x C

Equivalent number of days wages[F] = D ÷ E

[A] [B] [C] [D] [F]

Moderate pneumonia (without hospitalization): Adult treatment of choice:

[F1] =

Child <5 treatment of choice:

[F2] =

Other adult condition: (without hospitalization): Adult treatment of choice:

[F3] =

Other paediatric condition: (without hospitalization): Child <5 treatment of choice:

[F4] =

[E] = Lowest daily government salary (divide weekly salary by 7 or monthly salary by 30) =

Notes: [A] Using standard treatment guidelines, identify at the national level and pre-print on the form the treatment of choice and the recommended

preparation (dosage strength & form) for moderate pneumonia (no hospitalization). Do not include medicines used only for relief of mild symptoms, e.g. paracetamol or cough syrup. Other important disease (e.g. malaria, asthma, hypertension, or diabetes) in adults and children can be identified if the survey group deemed it is necessary, thus it is optional.

List the identified medicines in the space provided in SF2. [B] The number of units of each medicine needed for the duration of treatment (based on standard treatment guidelines) should be identified

at the national level and pre-printed on the survey forms. If the other condition is a chronic illness, include the number of units in a month’s supply.

[C] Indicate in local currency the unit price or the price the facility charges patients for each medicine. The lowest-priced branded or generic equivalent medicine should be used. If there are flat charges paid for each medicine given to patients, this amount should be recorded as the price of the medicine. Indicate “0” if medicines are given free. Add cost of syringes to the unit price, if applicable.

[D] Calculate total cost of treatment [D] by multiplying the number of units needed [B] by unit price [C]. If patients are charged a flat fee for treatment course, record this as total cost of treatment.

[E] Identify at the national level and pre-print on the form the lowest daily government salary. If the weekly salary is known, divide this by 7 to obtain the daily salary. If the monthly salary is known, divide this by 30 to obtain the daily salary.

[F] Calculate the number of days wages needed to pay for treatment by dividing the cost of treatment [D] by the lowest daily government salary [E].

Example:

Medicine/INN and Preparation Number of units

needed to complete treatment

Unit price (one vial, tablet, or

capsule)


[D] = B x C

Equivalent number of days wages [F] = D ÷ E

[A] [B] [C] [D] [F]

Moderate pneumonia (without hospitalization): Adult treatment of choice: Procaine penicillin: 1g 1 mill IU

3 injections 280 (injection plus syringe)

840 11.2

Child <5 treatment of choice: Amoxicillin: 25 mg/ml suspension in 100 ml bottle

1 bottle 220 per bottle 220 2.93

[E] = Lowest daily government salary (divide weekly salary by 7 or monthly salary by 30) = 75



(1-30)


- 128 -

Survey form 4: Public health facility pharmacy/dispensary Indicator: Average stockout duration Adequate record keeping


Only collect data for medicines with records covering at least 6 months within the past 12

Key medicines to treat common conditions

Records cover at least 6 months within the

past 12 months Yes=1, No=0

Number of days out of

stock

Number of days covered by the review (at least

6 months)

Equivalent number of days

per year [E] = C x 365 ÷ D

[A] [B] [C] [D] [E] 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13.

[B1] = Sum of B = [E1] = Sum of E =

[B2] = % adequate records = B1 ÷ 15 x 100 =

[F] = Average number of stockout days = E1 ÷ B1 =

Notes: [A] The list of 15 key medicines identified for Survey Form 1 should also be pre-printed on this form. [B] Go through the stock cards and indicate which medicines have records covering at least 6 months within the previous 12 months. Add the

total at the bottom [B1]. Calculate the percentage of medicines with adequate records [B2] by dividing the number of medicines with records covering at least 6 months [B1] by 15 and multiplying by 100.

[C] The review should cover 6-12 months. Go through the stock cards covering the review period. Indicate the number of days each medicine was not available or marked “0” on the card. A medicine is considered in stock if any quantity of it is available in generic or branded form.

[D] Indicate the number of days actually reviewed for each medicine. [E] Compute the equivalent number of stockout days per year for each medicine by multiplying the number of days out of stock [C] by 365 and

dividing by the number of days covered by the review [D]. Add the total number of stockout days [E1]. [F] Calculate the average number of stockout days by dividing the total number of stockout days [E1] by the total number of medicines reviewed [B1].

Example:

Only collect data for medicines with records covering at least 6 months in the past 12 months

Key medicines to treat common

conditions

[A]

Records cover at least 6 months within

the past 12 months Yes=1, No=0

[B]

Number of days out of stock

[C]

Number of days covered by the review

[D]

Equivalent number of days per year

[E] = C x 365 ÷ D [E]

Cotrimoxazole 1 90 180 182.5 Paracetamol 1 30 365 30 amoxicillin 0

[B1] = Sum of B = 2

[B2] = % adequate records = B1 ÷ 3 x 100 =

[E1] = Sum of E =212.5

[F] = Average number of stockout days = E1 ÷ B1 = 106.25



(1-30)


- 129 -

Survey form 5: Public health facility pharmacy/dispensary Indicator: Adequate conservation conditions and handling of medicines

in storeroom and dispensing area Facility Date Region Investigator

Checklist Storeroom True=1, False=0

Dispensing Area/Room

True=1, False=0 [A] [B]

1. There is a method in place to control temperature (e.g. roof and ceiling with space between them in hot climates, air conditioners, fans, etc.).

2. There are windows that can be opened or there are air vents.

3. Direct sunlight cannot enter the area (e.g. window panes are painted or there are curtains/blinds to protect against the sun).

4. Area is free from moisture (e.g. leaking ceiling, roof, drains, taps, etc.).

5. There is a cold storage in the facility.

6. There is a regularly filled temperature chart for the cold storage.

7. Medicines are not stored directly on the floor.

8. Medicines are stored in a systematic way (e.g. alphabetical, pharmacological).

9. Medicines are stored first-expiry-first out (FEFO).

10. There is no evidence of pests in the area.

11. Tablets/capsules are not manipulated by naked hand.

[A1] = Sum of A [B1] = Sum of B = [A2] = Score = A1 ÷ 10 x 100 =

[B2] = Score = B1 ÷ 10 x 100 =

Notes: [A] Indicate “1” if all parts of the statement are true for the storeroom and “0” if any part of it is false. Sum the total

number of true statements in [A1]. Calculate the score for the storeroom [A2] by dividing the sum of true statements [A1] by 10 and multiplying by 100.

[B] Indicate “1” if all parts of the statement are true for the dispensing room and “0” if any part of it is false. Sum the total number of true statements in [B1]. Calculate the score for the dispensing room [B2] by dividing the sum of true statements [B1] by 10 and multiplying by 100.

* It may be necessary to look elsewhere in the facility for some of the criteria (e.g. refrigerator).



(1-30)


- 130 -

Survey form 6: Public health facility pharmacy/dispensary patient care exit interview Indicators: Average number of medicines per prescription % patients know how to take medicines % medicines dispensed or administered Average cost of medicines % medicines adequately labelled Geographical accessibility of facilities


Patient sex

M/F

Age 1) less than 5 y. 2) older children 3) adults 4) more than 60

Number of medicines prescribed

Number of medicines

dispensed or administered

Number of medicines adequately

labelled

Patient knows how to take medicines

Yes=1, No=0

Amount patient paid for

purchased medicines

How long did it take the patient to get to the health

facility today? 1. <30min; 2. 31min-1h; 3. > 1h

How much did it cost him/her to come here

[A] [B] [C] [D] [E] [F] [G] [H] [I] 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30.



(1-30)


- 131 -

Patient sex

M/F

Age 1) less than 5 y. 2) older children 3) adults 4) more than 60


Number of medicines

dispensed or administered

Number of medicines adequately

labelled


Yes=1, No=0

Amount patient paid for

purchased medicines

How long did it take the patient to get to the health

facility today? 1. <30min; 2. 31min-1h; 3. > 1h


[A] [B] [C] [D] [E] [F] [G] [H] [I] [I1] = Sum of I =[A1] =

Sum cases [C1] = Sum of C =

[D1] = Sum of D = [E1] = Sum of E = [F1] = Sum of F = [G1] = Sum of G = [I2] = Average transport cost = I1 ÷ total responses = [A2]= Sum

females= [A3] = % females = A2 ÷ A1 x 100 =

[B1]=Sum of 1= [B2]=Sum of 2= [B3]=Sum of 3= [B4]=Sum of 4=

[C2] = Average number of medicines = C1 ÷ A1 =

[D2] = % dispensed = D1 ÷ C1 x 100 =

[E2] = % adequately labeled = E1 ÷ D1 x 100 =

[F2] = % know how to take medicines =F1 ÷ A1 x 100 =

[G2] = Average cost = G1 ÷ total patient =

[H1]=Sum of 1= [H2]=Sum of 2= [H3]=Sum of 3=

[I3] = Average transport cost to minimum daily salary = [I2] ÷ [J]

[J] = Lowest daily government salary (divide weekly salary by 7 or monthly salary by 30) =

Notes: [A&B] Interview 30 patients leaving the dispensing area/pharmacy. Obtain the sex and age of the patient, not those of the person obtaining the medicine. Use the number of

patients/cases able to respond to corresponding questions as denominators for (G, H, I, J) [A] Record the number of cases [A1] and the number of females [A2]. Calculate the percentage of females by dividing the total number of females [A2] by the total number of

cases [A1] and multiplying by 100. [B] Record the age of the patient. Indicate 1) less or equal to 5 years of age, 2) for older children, 3) for adults & 4) if equal or more then 60. Sum the total of patients in each

category [B1-4]. [C] Record the number of medicines prescribed for each patient. Combination medicines in one dosage form count as one medicine. Sum the number of medicines prescribed

for all patients [C1]. Calculate average number of medicines prescribed [C2] by dividing number of medicines prescribed [C1] by number of cases [A1]. [D] Record the number of medicines dispensed or administered to each patient. Sum the total number [D1]. Calculate the percentage of medicines dispensed [D2] by dividing

the number of medicines given to all patients [D1] by the total number of medicines prescribed [C1] and multiplying by 100. [E] Record the number of medicines labelled with at least the name of the medicine and how to take it*. Count only medicines meeting both criteria. Sum the total [E1].

Calculate the percentage of medicines adequately labelled [E2] by dividing the total number of adequately labelled medicines [E1] by the total number of medicines dispensed [D1] and multiplying by 100.

[F] Determine if patient (or an adult accompanying a paediatric patient) knows how to take all medicines dispensed (patient knows dosage and duration of all dispensed medicines*). Mark “1” only if patient can correctly state how ALL medicines should be taken and “0” otherwise. Sum the total [F1]. Calculate the percentage of patients who know how to take all medicines [F2] by dividing the total number who know how to take all medicines [F1] by the total number interviewed [A1] and multiplying by 100.

* Criteria for [E] and [F] can be adjusted as relevant to the surveyed population. [G] Record the amount each patient paid out-of-pocket for all medicines received at the facility. Check with a receipt if possible. Sum the total amount [G1]. Calculate the

average medicines cost by dividing the amounts paid for medicines [G1] by the total number interviewed able to respond. [H] Record the time it took the patient to get to the facility. Indicate the codes 1-3. Sum the total of patients in each category [1-3]. [I] Note travel cost in local currency. Sum the total amount [I1]. Calculate the average transport cost [I2] by dividing the amounts paid for transport [J1] by the total number of

interviewed persons able to respond. To calculate the = Average transport cost to minimum daily salary[I3], divide the average transport cost by the minimum daily salary [J]


- 132 -

Section B – Survey Forms 7–9

Survey Forms

Public health facilities SF 7* Average number of medicines per prescription % patients prescribed antibiotics/injections % prescribed medicines on Essential Medicines List % medicines prescribed by generic name (INN)

SF 8 Availability of Standard Treatment Guidelines Availability of Essential Medicines List

SF 9 % tracer cases treated according to recommended treatment protocol/guide

* For SF 7- Use only general outpatient records. Do not select patients from well-child visits, pre-

and post-natal visits, specialist consultations, or even separate clinics for adults and paediatric cases because treatment practices are different.

General information: Public health facility Facility Date Region Investigator

1. Who is prescribing during the time of visit? (check all that apply) * doctor (1=Yes; 0=No) nurse (1=Yes; 0=No) trained health worker/health aide (1=Yes; 0=No)

1.1) Who is the most senior prescriber? doctor (1=Yes; 0=No) nurse (1=Yes; 0=No) trained health worker/health aide (1=Yes; 0=No)

2. Has the most senior prescriber named in #1.1 attended RDU-related training within

the last year? (Note: RDU curriculum can include any of the following: rational prescribing, essential medicines concept, use of IMCI or other clinical guidelines.)

Yes (=1) No (=0) * If there are several prescribers, interview one, choosing the most senior prescriber.


Facility #____

(1-30)


- 133 -

Survey form 7: Public health facility: Rational medicine use - Prescribing indicator form Indicators: Average number of medicines per prescription % prescribed medicines on EML % patients prescribed antibiotics/injections % medicines prescribed by generic name


Type R/P

Age 1) < 5 year 2) older children 3) adults 4) more than 60

Patient sex M/F


Antibiotic prescribed

Yes=1, No=0

Injection prescribed

Yes=1, No=0

Number of prescribed medicines on EML

Number of medicines

prescribed by generic name

(INN) [A] [B] [C] [D] [E] [F] [G]

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. [A]=Sum of cases [B1] = Sum

females = [C1] = Sum of C =

[D1] = Sum of D =

[E1] = Sum of E = [F1] = Sum of F =

[G1] = Sum of G =

[A1+2]=Sum of paediatric cases [AP]= % of paediatric cases=A1+2 ÷A x 100=

[B2] = % females = B1

÷ A1 x 100 =

[C2] = Average number of medicines = C1 ÷ A1 =

[D2] = % receiving antibiotics = D1 ÷ A1 x 100 =

[E2] = % receiving injections = E1 ÷ A1 x 100 =

[F2] = % EML = F1 ÷ C1 x 100 =

[G2] = % INN = G1 ÷ C1 x 100 =

Notes: [A] From outpatient treatment records, select 30 patients seen within the last 12 months (R = retrospective sampling). If records are not available, select 30

patients currently being treated (P = prospective sampling). Sample can combine R and P. Mark “R” if patient was selected retrospectively and “P” if patient was selected prospectively. Record the number of cases [A] and the number of paediatric cases [A1+2]. Calculate the percentage of paediatric cases by dividing the total number of paediatric cases [A1+2] by the total number of cases [A] and multiplying by 100.

[B] Record the number of females [B1]. Calculate the percentage of females by dividing the total number of females [B2] by the total number of cases [A1] and multiplying by 100.

[C] Record number of medicines (chemical entity, INN, generic) prescribed. Combination medicines in one dosage form count as one medicine. Total the number of medicines prescribed [C1]. Calculate average number of medicines prescribed [C2] by dividing number of medicines prescribed [C1] by number of cases [A1].

[D] Record “1” if patient was prescribed any antibiotics and “0” otherwise. Total the cases receiving antibiotics [D1]. Calculate percentage of cases with antibiotics [D2] by dividing number of cases with antibiotics [D1] by number of cases [A1] and multiplying by 100.

[E] Record “1” if patient was prescribed any injections and “0” otherwise. Total the cases receiving injections [E1]. Calculate percentage of cases receiving injections [E2] by dividing number of cases with injections [E1] by number of cases [A1] and multiplying by 100.

[F] Record number of prescribed medicines on the national Essential Medicines List (EML). Total the number of prescribed medicines on the EML [F1]. Calculate the percentage of prescribed medicines on the EML [F2] by dividing the number of medicines on the EML [F1] by the number of medicines prescribed [C1] and multiplying by 100.

[G] Record number of medicines prescribed by INN. Total the number of medicines prescribed by INN [G1]. Calculate percentage of medicines prescribed by INN [G2] by dividing number of medicines prescribed by INN [G1] by number of medicines prescribed [C1] and multiplying by 100.


Facility #____

(1-30)


- 134 -

Survey form 8: Public health facility: Essential medicine information Indicators: Availability of Standard Treatment Guidelines (STG) Availability of Essential Medicines List (EML) Facility Date Region Investigator

Standard Treatment Guidelines (STG) available Yes=1, No=0

[A] STG for pneumonia (as part of combined STG publication or disease-specific STG document)

STG for ___________________(as part of combined STG publication or disease-specific STG document)

[A1] =Both STGs are present =

Essential Medicines List (EML) updated within last 5 years available Yes=1, No=0

[B] National EML Provincial/District EML Facility-specific EML Other EML (describe):

[B1] =At least one current EML is present =

Notes: [A] Identify the second required STG at the national level and pre-print on the form. This should be for an important

disease in the region, e.g. malaria in endemic areas or hypertension. Check to see if there is a copy of each of the STGs either as part of a combined STG publication or a disease-specific STG document. Record “1” if the facility is able to present a copy of the document and “0” if the facility is unable to present the document. If both STGs are present record “1” in [A1] otherwise record “0”.

[B] Record “1” next to each type of EML updated within the last 5 years that is physically present in the facility. If the facility is unable to present the document or if the EML presented has not been updated in the last 5 years, record “0”. If any current EML is available, mark “1”in [B1], otherwise record “0”.


Facility #____

(1-30)


- 135 -

Survey form 9: Public health facility Indicator: % of tracer cases treated according to recommended treatment

protocol/guide Facility Date Region Investigator

Use of medicines by case

Yes=1, No=0 [B]

Tracer conditions and medicines prescribed

[A] 1 2 3 4 5 6 7 8 9 10

Total number of cases

[C]

Number of cases

prescribed medicine

[D]

% of cases prescribed medicine

[E] = D ÷ C x

100 [E]

Non-bacterial diarrhoea in children under age 5 ORS Antibiotic Antidiarrhoeal and/or antispasmodic

Mild/moderate (outpatient) pneumonia in children under age 5 [A1] 1st line antibiotic(s) in national STG: Any 1st line antibiotic Prescribed >1 antibiotic

Non-pneumonia acute respiratory tract infection (ARI) in patients of any age Any antibiotic [A2] Optional tracer condition 1:

[A2] Optional tracer condition 2:

Notes: [A] At the national level, identify and pre-print on the form the firs-line antibiotic(s) mentioned in the national STG

for pneumonia [A1]. If data on treatment of other important local conditions is desired, pre-print on the form the optional tracer conditions [A2] and the medicines that will be used to measure recommended or non-recommended practices.

[B] From general adult or paediatric outpatient records, select 10 patient encounters with each target condition. If possible, choose only single diagnosis encounters. Write “1” or “0” for each case selected to indicate whether or not each target medicine was prescribed.

[C] Sum the total number of cases in each row. [D] Sum the total number of cases in each row that were prescribed the target medicine. [E] For each row, calculate the percentage of patients receiving each medicine [E] by dividing the total number of

cases that were prescribed each medicine [D] by the total number of cases [C] and multiplying by 100.


Facility #____

(1-30)


- 136 -

Section C – Survey Forms 10–14

Survey Forms


SF 10 % of key medicines available % medicines expired


SF 12 Affordability of treatment for adults and children under 5 years of age (moderate pneumonia with no hospitalization)

SF 13 Adequate conservation conditions and handling of medicines conditions in storeroom and dispensing area SF 14 Average number of medicines purchased

% prescription medicine bought without prescription % medicines adequately labelled % patients know how to take medicines Average cost of medicines Geographical accessibility of dispensing facilities

General Information: Private pharmacy/drug outlet Facility Date Region Investigator

1. Does the law require a pharmacist to be present during hours of operation of private pharmacies/drug outlets? Yes No 2. Is a pharmacist present at the time of the visit Yes No Assessment 1 complies with the law (items 1 and 2 are both Yes) 2 does not comply with the law (item 1 Yes and item 2 No) 3 no requirement for pharmacist presence (item 1 No) 3. Who is dispensing during the time of visit? (check all that apply) Pharmacist (1=Yes; 0=No) Pharmacy aide/ health assistant (1=Yes; 0=No) Nurse (1=Yes; 0=No) Untrained staff (1=Yes; 0=No)

Private Pharmacy

Facility #____

(1-30)


- 137 -

Survey form 10: Private pharmacy/drug outlet Indicator: % key medicines available

% medicines expired



In stock Yes=1, No=0

Expired medicines on shelves

Yes=1, No=0

[A] [B] [C]

1. 2.

3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15.

[B1] = Sum of B = [C1] = Sum of C =

[B2] = % in stock = B1 ÷ 15 x 100 =

[C2] = % expired = C1 ÷ B1 x 100 =

Notes: [A] The same lists of 15 key medicines used for Survey Form 1 should be pre-printed on the survey forms. [B] Mark “1” if any quantity of any dosage form of the medicine is available in the pharmacy on the day of the

visit. Mark “0” if the medicine is not physically available. Add the total at the bottom [B1]. Calculate the percentage in stock [B2] by dividing the total in stock [B1] by 15 and multiplying both by 100.

[C] For all medicines in stock, check if any of the stock is expired. If any amount of a medicine has an expiry problem, mark “1” for yes. Do not count expired medicines stored in a separate area for destruction. Add the total at the bottom [C1& F1 ] . Calculate the percentage expired [C2 ] by dividing the total expired [C1] by the total number of medicines in stock [B1] and multiplying by 100.

Private Pharmacy

Facility #____

(1-30)


- 138 -

Survey form 11: Private pharmacy/drug outlet Indicator: Price of key medicines

Price of paediatric medicines


Key medicines to treat common conditions Preparation Unit Lowest unit price of the medicine

paid by patient [A] [B] [C]

1. Amitriptyline 25mg cap/tab 2. Amoxicillin 500 mg cap/tab 3. Atenolol 50 mg cap/tab 4. Captopril 25 mg cap/tab 5. Ceftriaxone 1g/vial injection 6. Ciprofloxacin 500 mg cap/tab 7. Co-trimoxazole 8 + 40 mg/ml cap/tab 8. Diazepam 5 mg cap/tab 9. Diclofenac 50 mg cap/tab 10. Glibenclamide 5 mg cap/tab 11. Omeprazole 20 mg cap/tab 12. Paracetamol 25mg/ml syrup/susp 13. Salbutamol 0.1mg/dose inhaler 14. Simvastatin 20mg cap/tab

* Paediatric medicines 1. Co-trimoxazole 40+200 mg/5 ml

(8+40mg/ml) suspension

2. Vitamin A 100,000 unit capsule 3. Isoniazid 50 mg tab Medicines identified in Survey Form 3 to calculate affordability (Indicate the INN and preparation (dosage strength and form)

Notes: [A&B] Note: Medicines must be the exact preparation listed in column B. Note only those currently available in stock.

(Paediatric medicines include those that are difficult to obtain because of cost or poor availability of paediatric formulation). Indicate also all the medicines that will be identified in SF 3.

[C] For each medicine that is actually available in stock during the visit, determine the lowest unit price in the local currency paid out-of-pocket by a patient. If patients pay flat charges for each medicine dispensed, this amount should be recorded as the price of the medicine. Indicate a “0” if medicines are given free. If medicine or price data are not available, mark N/A.

[D] From procurement or purchasing records at the facility, determine the lowest unit price in the local currency paid by the facility in the last year. If facilities generally receive the medicine for free from the Ministry of Health, record the last price paid when the medicine was actually purchased. If the medicine has not been purchased in the last year or if price data are not available, mark N/A.



(1-30)


- 139 -

Survey form 12: Private pharmacy/drug outlet Indicator: Affordability of treatment for adults and children under 5 years of age Facility Date Region Investigator

Medicine/INN and Preparation

Number of units needed to

complete treatment

Unit price (one vial, tablet,

or capsule)


[D] = B x C

Equivalent number of days wages [F] = D ÷ E

[A] [B] [C] [D] [F] Moderate pneumonia (without hospitalization): Adult treatment of choice:

[F1] =

Child <5 treatment of choice:

[F2] =

Other adult condition: (without hospitalization): Adult treatment of choice:

[F3] =

Other paediatric condition: (without hospitalization): Child <5 treatment of choice:

[F4] =

[E] = Lowest daily government salary (divide weekly salary by 7 or monthly salary by 30) =

Notes: [A] The treatment of choice for pneumonia and the other selected condition identified for Survey Form 3 should also be pre-

printed on this form. [B] The number of units of each medicine needed for the duration of treatment identified for Survey Form 3 should also be pre-

printed on this form. List the identified medicines in the space provided in Survey Form 11.

[C] Indicate in local currency the unit price or the price the pharmacy charges patients for each medicine. The lowest-priced branded or generic equivalent medicine should be used. Add the cost of syringes to the unit price, if applicable.

[D] Calculate total cost of treatment [D] by multiplying the number of units needed [B] by the unit price [C]. [E] The lowest daily government salary identified for Survey Form 3 should also be pre-printed on this form. [F] Calculate the number of days wages needed to pay for treatment by dividing the cost of treatment [D] by the lowest daily

government salary [E].

Example:

Medicine/INN and Preparation Number of units

needed to complete treatment

Unit price (one vial, tablet, or

capsule)


[D] = B x C

Equivalent number of days wages

[F] = D ÷ E

[A] [B] [C] [D] [F]

Moderate pneumonia (without hospitalization):

Adult treatment of choice: Procaine penicillin: 1g 1 mill IU

3 injections 280 (injection plus syringe)

840 11.2

Child <5 treatment of choice: Amoxicillin: 25 mg/ml suspension in 100 ml bottle

1 bottle 220 per bottle 220 2.93

[E] = Lowest daily government salary (divide weekly salary by 7 or monthly salary by 30) = 75

Private Pharmacy

Facility #____

(1-30)


- 140 -

Survey form 13: Private pharmacy/drug outlet Indicator: Adequate conservation conditions and handling of medicines Facility Date Region Investigator

Checklist Storeroom True=1, False=0

Dispensing Area/Room

True=1, False=0 [A] [B]

1. There is a method in place to control temperature (e.g. roof and ceiling with space between them in hot climates, air conditioners, fans, etc).





6. There is a regularly filled temperature chart for the cold storage.



9. Medicines are stored first-expiry-first our (FEFO).


11. Tablets/capsules are not manipulated by naked hand.

[A1] = Sum of A [B1] = Sum of B [A2] = Score = A1 ÷ 10 x 100 =

[B2] = Score = B1 ÷ 10 x 100 =

Notes: [A] Indicate “1” if all parts of the statement are true for the storeroom and “0” if any part of it is false.

Sum the total number of true statements in [A1]. Calculate the score for the storeroom [A2] by dividing the sum of true statements [A1] by 10 and multiplying by 100.

[B] Indicate “1” if all parts of the statement are true for the dispensing room and “0” if any part of it is false. Sum the total number of true statements in [B1]. Calculate the score for the dispensing room [B2] by dividing the sum of true statements [B1] by 10 and multiplying by 100.


Private Pharmacy

Facility #____

(1-30)


- 141 -

Survey form 14: Private pharmacy/drug outlet - Exit interview

Patient sex M/F

F=1, M=0

Age 1) Less than 5

yrs. 2) 5 – 14years 3) 15 – 59years 4) more than 60

Number of medicines purchased

Number of prescription medicines


purchased with no prescription

Number of medicines

adequately labelled


Yes=1, No=0

Amount patient paid

for purchased medicines

How long did it take to the patient to get to the health

facility today? 1) < 30min; 2) 31min-1h; 3) > 1h


[A] [B] [C] [D] [E] [F] [G] [H] [I] [J] 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18.

Indicators Average number of medicines purchased % medicines adequately labelled % patients know how to take medicines

% prescription medicines bought without prescription

Average cost of medicines Geographical accessibility of facilities



- 142 -

Patient sex M/F

F=1, M=0

Age 1) Less than 5

yrs. 2) 5 – 14years 3) 15 – 59years 4) more than 60

Number of medicines purchased



purchased with no prescription

Number of medicines

adequately labelled


Yes=1, No=0

Amount patient paid

for purchased medicines

How long did it take to the patient to get to the health

facility today? 1) < 30min; 2) 31min-1h; 3) > 1h


[A] [B] [C] [D] [E] [F] [G] [H] [I] [J] 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30.

[C1] = Sum of C = [D1] = Sum of D = [E1] = Sum of E = [F1] = Sum of F = [G1] = Sum of G = [H1] = Sum of G = [J1] = Sum of I = [A1] = Sum customers=

[A2]= Sum females=

[J2] = Average transport cost = I1 ÷ total responses =

[A3] = % females = A2 ÷ A1 x 100 =

[B1]=Sum of 1=

[B2]= Sum of 2=

[B3]= Sum of 3=

[B4]= Sum of 4=

[C2] = Average number of medicines purchased by customers= C1 ÷ A1 =

[D2] = Average number of prescription medicines purchased by customers

[E2] = % prescription medicines bought without prescription = E1 ÷ D1 x 100 =

[F2] = % adequately labelled = E1 ÷ C1 x 100 =

[G2] = % know how to take medicines = G1 ÷ A1 x 100 =

[H2] = Average cost = H1 ÷ total patient A1 =

[I1] Sum of 1=

[I2] Sum of 2=

[I3] Sum of 3= [J3] = Average transport cost to minimum daily salary = [J2] ÷ [K]=

[K] = Lowest daily government salary (divide weekly salary by 7 or monthly salary by 30) = D 17.78


- 143 -

Notes:

Interview 30 patients leaving the dispensing area/pharmacy (only patients older then 16 years). Ask if the interviewed is looking for medicines for his use or for another person. If he/she is looking medicines for another person ask for whom, trying to identify the king of link and if the interviewed person is the caregiver and helps the other with medication. If (1) the interviewer is the patient itself of (2) The interviewer is the caregiver, tell he/she briefly the purpose of the interview, needed time to complete it (3-5 minutes) what will be required from him/her (look into the prescription, look the medicines and ask some questions). If the interviewed agrees, follow with the interview. In any case, be kind, respectful and thanks.

Obtain the sex and age of the patient, not those of the person obtaining the medicine. Use the number of patients/cases able to respond to corresponding questions as denominators for (G, H, I)

[A] Record the number of cases [A1] and the number of females [A2]. Calculate the percentage of females by dividing the total number of females [A2] by the total number of cases [A1] and multiplying by 100.

[B] Record the age of the patient. Indicate 1) less or equal to 5 years of age, 2) for older children, 3) for adults & 4) if more then 60. Sum the total of patients in each category [B1-4].

[C] Record the number of medicines purchased by each customer. Combination medicines in one dosage form count as one medicine. Sum the total number [C1]. Calculate average number of medicines purchased [C2] by dividing number of medicines purchased [C1] by number of customers [A1].

[D] Record the number of prescription medicines purchased. Note: these are mainly antibiotics, antihypertensive, anti-diabetics, asthma and other medicines that should only be bought with prescription .

[E] Record the number prescription medicines (antibiotics, antihypertensive, medicines for diabetes, asthma, and etc) bought without prescription. Sum the number of prescription medicines bought without prescription [E1]. Calculate % of prescription medicines bought without prescription [E2] by dividing total number of prescription medicine bought without prescription [E1] by total number of medicines purchased [D1] and multiplying by 100.

[F] Record the number of medicines labelled with at least the name of the medicine and how to take it. Count only medicines meeting both criteria. Sum the total [E1]. Calculate the percentage of medicines adequately labelled [F2] by dividing the total number of adequately labelled medicines [F1] by the total number of medicines purchased [C1] and multiplying by 100.

[G] Determine if the customer who has purchased the medicines (or an adult accompanying a paediatric patient) knows how to take all medicines dispensed. Mark “1” only if customer can correctly state how ALL medicines should be taken and “0” otherwise. Sum the total [G1]. Calculate the percentage of customers who know how to take all medicines [G2] by dividing the total number who know how to take all medicines [G1] by the total number interviewed [A1] and multiplying by 100.

[H] Record the amount each patient paid out-of-pocket for all medicines received at the facility. Check with a receipt if possible. Sum the total amount [H1]. Calculate the average medicines cost by dividing the amounts paid for medicines [H1] by the total number interviewed able to respond.

[I] Record the time it took to the patient to get to the facility. Indicate the codes 1-3. Sum the total of patients in each category [I1-3].

[J] Note travel cost in local currency. Sum the total amount [J1]. Calculate the average transport cost [J2] by dividing the amounts paid for transport [J1] by the total number interviewed persons able to respond. To calculate the average transport cost to minimum daily salary[J3], divide the average transport cost by the minimum daily salary [H]


- 144 -

Section D – Survey Forms 15–17

Survey Forms

Central/regional/district warehouses supplying the public sector

SF 15 Availability of key medicines

% medicines expired

SF 16 Average stockout duration

Adequate record keeping

SF 17 Adequate conservation conditions and handling of medicines

Central/district warehouse

Facility #____

(1-5)


- 145 -

Survey form 15: Central/regional/district warehouse supplying the public sector Indicator: % key medicines available

% medicines expired


Key medicines to treat common conditions In stock

Yes=1, No=0 Expired medicines

on shelves Yes=1, No=0

[A] [B] [C] 1. 2.

3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15.

[B1] = Sum of B = [C1] = Sum of C =

[B2] = % in stock = B1 ÷ 15 x 100 =

[C2] = % expired = C1 ÷ B1 x 100 =

Notes: [A] The same lists of 15 key medicines used for Survey Form 1 pre-printed on the survey forms. [B] Mark “1” if any quantity of any dosage form of the medicines is in stock in the facility on the day of the visit.

Mark “0” if the medicine is not available in stock. Add the total at the bottom [B1]. Calculate the percentage in stock [B2] by dividing the total in stock [B1] by 15 and multiplying both by 100.

[C] For all medicines in stock, check if any of the stock is expired. If any amount of a medicine has an expiry problem, mark “1” for yes. Do not count expired medicines stored in a separate area for destruction. Add the total at the bottom [C1 ]. Calculate the percentage expired [C2] by dividing the total expired [C1] by the total number of medicines in stock [B1] and multiplying by 100.


Facility #____

(1-5)


- 146 -

Survey form 16: Central/regional/district warehouse supplying the public sector Indicator: Average stockout duration Adequate record keeping


Only collect data for medicines with records covering at least 6 months within the past 12 months


Records cover at least 6 months within the past

12 months Yes=1, No=

Number of days out of

stock

Number of days covered by the

review (at least 6 months)

Equivalent number of days

per year [E] = C x 365 ÷ D

[A] [B] [C] [D] [E] 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15.

[B1] = Sum of B = [E1] = Sum of E = [B2]= % adequate records = B1 ÷ 15 x 100 =

[F] = Average number of stockout days = E1 ÷ B1 =

Notes: [A] The list of 15 key medicines and optional additional medicines identified for Survey Form 1 should also be pre-printed on this form. [B] Go through the stock cards and indicate which medicines have records covering at least 6 months within the previous 12 months. Add the total

at the bottom [B1]. Calculate the percentage of medicines with adequate records [B2] by dividing the number of medicines with records covering at least 6 months [B1] by 15 and multiplying by 100.

[C] The review should cover 6-12 months. Go through the stock cards covering the review period. Indicate the number of days each medicine was not available or marked “0” on the card. A medicine is considered in stock if it is available in generic or branded form.

[D] Indicate the number of days actually reviewed for each medicine. [E] Compute the equivalent number of stockout days per year for each medicine by multiplying the number of days out of stock [C] by 365 and

dividing by the number of days covered by the review [D]. Add the total number of stockout days [E1]. [F] Calculate the average number of stockout days by dividing the total number of stockout days [E1] by the total number of key medicines

reviewed [B1]. Example:

Only collect data for medicines with records covering at least 6 months in the past 12 months

Key medicines to treat common

conditions

Records cover at least 6 months within the past 12 months

Yes=1, No=0

Number of days out of stock

Number of days covered by the review

Equivalent number of days per year [E] = C x 365 ÷ D

[A] [B] [C] [D] [E] Co-trimoxazole 1 90 180 182.5 Paracetamol 1 30 365 30 Amoxicillin 0

[B1] = Sum of B = 2 [B2] = % adequate records = B1 ÷ 3 x 100 = 66.7

[E1] = Sum of E =212.5

[F] = Average number of stockout days = E1 ÷ B1 = 106.25


Facility #____

(1-5)


- 147 -

Survey form 17: Central/regional/district warehouse supplying the public sector Indicator: Adequate conservation conditions and handling of medicines Facility Date Region Investigator

Checklist Storeroom

True=1, False=0 [A]

1. There is a method in place to control temperature (e.g. roof and ceiling with space between them in hot climates, air conditioners, fans, etc.).





6. There is a regularly filled in temperature chart for the cold storage.



9. Medicines are stored first-expiry-first out (FEFO).


[A1] = Sum of A = [A2] = Score = A1 ÷ 10 x 100 =

Notes: [A] Indicate “1” if all parts of the statement are true for the storeroom and “0” if any part of it is false.

Sum the total number of true statements [A1]. Calculate the score for the storeroom [A2] by dividing the sum of true statements [A1] by 10 and multiplying by 100.



Facility #____

(1-5)

who operational package for assessing, monitoring and evaluating

Documents