appropriate evidence website - ceg

Appropriate evidence

Ethical questions concerning the use of Evidence in health care policy

Prof. D.W. Willems

Prof. R. Vos

G. Palmboom

P. Lips, general practitioner

Monitoring Report Ethics and Health 2007

Centre for Ethics and Health

Appropriate evidence. Ethical questions concerning the use of evidence in health care policy Publication of the Centre for Ethics and Health

ISBN 978-90-78823-05-6

All rights reserved

This publication can be cited as follows: Council for Public Health and Health Care. Appropriate evidence. Ethical questions concerning the

use of evidence in health care policy. Monitoring Report Ethics and Health 2007/4. The Hague:

Centre for Ethics and Health, 2007

You can download this publication via www.ceg.nl / www.rvz.net

Appropriate evidence 3

Tot the Minister of Health, Welfare and Sport

Dear Minister,

Scientific evidence is playing an increasingly important role in both

health care and health care policy. Yet is this emphasis on scientific

rigour appropriate for every form of care? That is the key question

posed in this monitoring report that we are pleased to present you with.

The monitoring report has been written by four authors on behalf of the

Centre for Ethics and Health (CEG). The Council for Public Health and

Health Care, which works together with the Health Council of the

Netherlands in the CEG, approved the monitoring report on 20

September 2007.

The advance of evidence-based medicine has led to many positive

changes. The structured approach of weighing up scientific evidence

and making this available in standards and protocols is helping to

increase the quality of medical care provided. Currently this method is

also being used to decide what forms of care should be covered in the

basic health insurance package.

However, it is not all good news. The emphasis on scientific evidence

can lead to a reduced appreciation for forms of care where such

evidence is difficult to produce and to the neglect of elements such as

attention, trust and 'presence'. There is a risk that the ethical question

as to what constitutes good care could be reduced to what is proven

effective care. The effect of a cholesterol inhibitor can be determined

with extensive and often expensive research. The effect of

physiotherapy in a nursing home, with elderly clients who often suffer

from several complaints is, however, far more difficult to investigate.

Parnassusplein 5

2511 VX The Hague

P.O. Box 19404

2500 CK The Hague

Tel +31 70 340 5060

Fax +31 70 340 7575

E-mail [email protected]

URL www.ceg.nl

Date

22 January 2008

Your letter of

--

Your reference

--

Our reference

CEG/U08-002

Direct line

+31 70 340 5815

Subject

Monitoring Report

Appropriate evidence

4 Appropriate evidence

If we fail to take this difference into account then important values in

health care could be threatened. Certain forms of care could suffer a

loss of status, and eventually resources, with negative consequences

for both health care professionals and patients.

We need to ask ourselves which type of evidence is appropriate for

each form of care. Hard evidence should be obtained wherever possi-

ble. Yet for forms of care where this is not a realistic option, other re-

quirements need to be set. To summarise: appropriate evidence must

be sought.

The same applies to the supplementary health insurance package.

There we observe that patient preferences are the decisive factor. That

puts health care providers in a difficult position: they are frequently

required to work in an evidence-based manner and then all of a sud-

den not. Therefore, in such cases, the search for appropriate evidence

is also relevant.

Yours sincerely,

Rien Meijerink, PieterVos,

chair RVZ general secretary RVZ


Contents

Executive summary 6

1 Appropriate use of evidence? 9

1.1 The complexities of decisions about health care 9

1.2 Ethical explorations 10

1.3 Report outline 11

2

Use in health care practice and policy 12

2.1 Development of evidence-based medicine 12

2.2 Use in current health care practice 15

2.3 Use in current policy 17

2.4 Conclusion 19

3

Possibilities and limitations of evidence 21

3.1 Prioritising evidence 21

3.2 Gaps in the availability of evidence 22

3.3 Consequences of the differences in levels of evidence 25

3.4 Conclusion 29

4

Consequences for the health insurance package 30

4.1 Dilemmas in compiling the basic health insurance package 30

4.2 Dilemmas in compiling the supplementary health care package 32

4.3 Conclusion 34

5

Key considerations for policy and research 36

References 38

Appendices 42


Executive summary

The advance of scientif ic evidence

Research results are playing an increasingly important role in the decisions taken about medical

treatments. In recent years an extensive system has been developed for this purpose in which results

from different types of research are awarded a different value from 'hard' to 'soft’ evidence. If there is

enough hard evidence for the effectiveness of a treatment then this will be included in a professional

standard.

This standard will subsequently serve as the guiding principle, despite the continuing importance of

the patient's input and the fact that it is the health care professional who will ultimately decide whether

or not the protocol should be used for the patient concerned. Therefore scientific evidence alone is

not enough but instead forms one of the three aspects of what is termed evidence-based medicine.

Yet as the term suggests, evidence obtained from research plays an important role. Scientific

knowledge has a high status, the number of publications is still rising enormously and the system of

evidence-based medicine used in health care is also starting to penetrate public health policy, where it

is also increasingly being used to support decisions with scientific knowledge.

An example of this is the drawing up of the basic health insurance package. Yet interestingly, scientific

knowledge scarcely plays a significant role in the drawing up of the supplementary health insurance

package. There the wishes of the patient are the decisive factor: health insurers provide packages that

meet consumer demands. Therefore in practice these contain quite a number of treatments for which

no evidence whatsoever is available.

These developments raise the question as to what is an appropriate role for scientific evidence within

the current health care system and current public health policy. Is evidence according to the criteria

of evidence-based medicine the best way of substantiating the value of all forms of care? And how

should the three elements of evidence-based medicine – the best available knowledge, the

judgement of health care professionals and the preferences and needs of patients – relate to each

other to ensure an optimal quality of health care and good health care policy? That is the key question

considered in this monitoring report.

The pr ice of evidence

There is much to be said for trying to support medical decisions with scientific evidence. In the history

of evidence-based medicine, fine examples can be found of enormous quality improvements realised

as a result of working with tried and tested methods. Methods have been developed for this approach,

and protocols ensure that the latest findings are made available for health care professionals; with the

current growth in knowledge it is impossible for individuals to keep abreast of the latest develop-

ments, let alone assess these on the basis of their merits. The evidence-based approach means that

patients can be assured consistency in their treatment, irrespective of the health care professional

treating them.


In public health policy, a legitimate effort is also being made to support decisions with scientific

evidence. Policy makers also want to do what works and to discard things that are pointless or

counterproductive. Information about the effectiveness of a measure can be used to determine

whether the benefits justify the costs. The approach of evidence-based medicine provides leads for

this.

However, there are catches in the quest for increasingly more and harder evidence. One problem is

that not all forms and elements of health care can be easily investigated according to the methods

prescribed for evidence-based medicine. The obstacles can be methodological but also financial and

ethical.

For example, the efficacy of a specific medicinal product can often easily be tested in a randomised

controlled trial (RCT) with a large group of study subjects and a control group. Such investigations

are very expensive but can usually be financed by the pharmaceutical industry (even if that in turn is

reflected in high costs for medicinal products – evidence is anything but cheap). This is not the case

where preventive measures are concerned which do not bear any immediate effect, or for the treatment

of elderly patients who suffer from several conditions at once. Then it is difficult to select a stable study

group and there are many factors that cannot be controlled but which can influence the outcomes.

Finding an interested party willing to finance the research can also prove difficult.

Another completely different area is the elements of health care that are usually referred to as guidance,

attention and 'presence'. In these cases the nature of the 'intervention' demands a different form of

evidence than that generally accepted in evidence-based medicine.

There is a risk that where it can be easily proven that certain forms of care and measures result in

health gain then these will receive greater attention, whereas others will suffer a loss of status, and

eventually resources as well, due to a lack of hard evidence. That danger will definitely be present if

the methodology of evidence-based medicine is used in an increasing number of contexts. Then not-

proven interventions will increasingly come under fire.

Moreover, practice has taught that people easily forget that there is a fundamental difference between

'not proven that it works' and 'proven that it does not work'. Further it should also be remembered that

only 10 to 25% of current medical treatments are based on evidence. At present, efforts to obtain

evidence seem to outweigh the actual evidence obtained.

If despite these nuances the focus still comes to lie on hard evidence then, and certainly where that

cannot be obtained, important values within health care could be lost. In some cases searching for

evidence of health gain is not even relevant. Paying attention, not being in a hurry, a kind word: we still

value such things, even if their 'effectiveness' can never be proven.

The variable role of evidence in health care insurance pack-

ages

What does all of this mean for the decisions about treatments that are included in the basic health

insurance package, and the treatments in the supplementary package? Here we find the remarkable

situation that evidence is increasingly becoming the guiding principle in the basic health insurance

package, whereas for supplementary packages the wishes of the consumer play the decisive role.

In the basic health insurance package there is the danger that the hard form of scientific evidence will

acquire a great deal of weight, even though 'proven efficacy in a research situation' is not the same as

' practically applicable in the complex daily reality'. Such a scenario also limits the judgement of the


health care professional and the preference of the patient. And with this the quality of care provided

will eventually be put at risk.

The solution is to search for types of evidence that are appropriate to each form of care. In many cases

that can be hard evidence, according to the methodology of evidence-based medicine. However, other

types of research can also yield appropriate evidence. Furthermore, these can help to prevent the

development of a competition for evidence that pushes up the costs of health care even further still,

as hard evidence is expensive whereas an effective intervention can often be investigated in another

manner. After all, a significant clinical effect can be demonstrated even with a small group of patients.

Whereas in the basic health insurance package there is a danger of too great an emphasis on a certain

type of evidence, in the case of supplementary packages evidence scarcely gets a look in. And that is

hardly surprising. After all these packages are aimed at enabling people to insure themselves for care

that does not satisfy the criteria for the basic package. Accordingly these packages contain a wide

variety of care forms, from dentistry to health resort trips, and for a considerable number of the

treatments reimbursed, there is no evidence that these work. This applies, for example, to the

alternative therapies included in virtually every supplementary package but also for the medical check-

ups that are increasingly being offered.

Of course people are free to take out insurance for non-proven treatments. Perhaps there are

occasions when the supplementary package is a useful parking space for non-proven treatments that

are rightly or otherwise – further research will need to show this – excluded from the basic package.

On the other hand, the fact that insurance can be obtained for some forms of care unduly accords

these a status: 'if I can obtain insurance for it then it must work'.

An even more difficult problem is where treatments are offered despite evidence that these do not

work. Such cases also illustrate how the differences in levels of evidence between the basic package

and the supplementary package can place the health care professional in a difficult situation.

Evidence-based working, which is further the creed of clinical practice, would not appear to apply in

certain cases. That leads to a strange fragmentation, which could ultimately lead to the confidence

placed in the health care professional being damaged, even though a good therapeutic relationship is

crucial for the quality of care provided.

Solving this problem therefore requires a greater role for evidence in the drawing up of supplementary

packages. Such a use of evidence should lead to the exclusion for treatments that have clearly been

proven not to work. Combining the sale of worthwhile and not-proven treatments is equally undesirable.

Towards the use of appropriate evidential value

The differences between the basic and supplementary packages in terms of the importance attached

to evidence could be reduced. However it remains obvious that in the basic package, where solidarity

continues to be an important element, other, heavier advice will be considered more appropriate than in

the supplementary package, where the consumer's choice is the decisive factor. Yet for the basic pack-

age the current definition of appropriate evidence can be extended. This would allow justice to be

done to the complexity and value of treatments that are more difficult to capture in terms of hard fig-

ures.


1. Appropriate use of evidence?

1.1 The complexit ies of decisions about health care

Three types of in formation are needed

Two parties have always been involved in decisions about medical treatments: the medical practitioner

and the patient. More recently a third 'party' has become involved in this process: the scientist, who

supplies knowledge about the efficacy and suitability of interventions – commonly referred to as

'evidence'. Evidence-based medicine (or EBM) is now a widely understood term. The concept arose in

clinical practice, where three types of information are needed for a well-grounded decision.

Firstly, information is needed about the requirements and wishes of individual patients. What do they

need and what is the treatment objective? For example, there is a considerable difference between

someone who hopes to be cured and someone who is in the last phase of his or her life and wishes to

maintain as good a quality of life as possible. Secondly, there is a need for research results that

provide information about the effectiveness of a certain treatment, in other words: what is the

probability that the patient will benefit from the treatment concerned. Thirdly, there is a need for

professional expertise. Professionals can assess what is the best form of treatment in view of the

available scientific knowledge on the one hand and the treatment objectives, wishes, and possibilities

of a patient on the other.

The increasing importance of evidence-based medicine is certainly a welcome development. After all it

signals a move towards decisions made on the basis of scientific arguments and aimed at realising as

high a quality of care as possible. However in practice, finding the right balance between the three

types of evidence and the values underlying these is far from easy. A lack of information, differences

in the degree of certainty and conflicting findings and interests are some of the problems encountered.

And what should one do then? Which aspect should be given the most weighting?

This question not only arises in concrete cases, for example, when a nursing home physician has a

protocol for the treatment of a condition yet needs to consider whether it in the interests of a dementia

patient to apply this. The implications are far more general. How much value does our society attach to

professional judgements, the wishes of the patient and scientific information?

The advance of scienti f ic evidence

The answer to the above question has changed over the course of time. And the emphasis might

well change again in the future. For example, several decades ago professional judgement carried

more weight than at present. Nowadays, patients have a far greater say. And in recent years an

increasing amount of weight has been attached to scientific evidence.

The term evidence-based medicine clearly indicates the role of science. Although, in principle,

three aspects are distinguished, that is not clear from the term used. That is why a proposal was

recently made in the international literature to replace the term evidence-based medicine with the

term evidence-informed practice (Cox, 2004). This term would more accurately reflect the use of


evidence in health care practice. Or as epidemiologist Paul Glasziou puts it: "I suggest we start with

evidence-informed medicine and add a little wisdom. It is clearly time to change 'evidence-based

medicine' to 'evidence-informed practice.' Although 'EBMers' have emphasised the importance of

patients' values in decision making, this is missed in most discussions." (Glasziou 2005).

However, scientific knowledge is still clearly gaining ground. And within the influential approach of

evidence-based medicine certain types of research are preferred to others. Experimental and

quantitative research are the preferred options, whereas less value is attached to qualitative

investigations.

Evidence-based medicine, and within this the increasingly stronger emphasis on hard evidence, is

not only making advances within health care practice. The concept is also increasingly being used

to justify choices within health care policy. For example, it already plays a role in deciding what to

include in the basic health care insurance package. The approach of evidence-based medicine is,

therefore, increasingly being used outside of the context for which it was originally intended.

Yet the same is not true for the supplementary health insurance package. There scientific evidence

frequently plays a subservient role in decisions about what should and should not be reimbursed. The

contents of a package are primarily determined by what consumers are prepared to insure

themselves for, whether or not the care in question is worthwhile.

1.2 Eth ica l exp lorat ions

These developments call into question the role of scientific evidence in the current health

care system and health care policy and how the three forms of information that together lead

to evidence-based actions should relate to each other to ensure an optimum level of care.

To investigate this matter, the following questions will be answered in this monitoring report:

1. How has evidence-based medicine developed?

2. What role does the criterion 'proven effectiveness' currently play in clinical practice and in

government policy?

3. What are the advantages and limitations of 'proven effectiveness' as a criterion in health care and

health care policy?

4. What are the ethical questions associated with using ' proven effectiveness' as a criterion for

drawing up the basic health insurance package and yet not using this for drawing up the

supplementary package?

5. Which concerns in the area of evidence-based medicine, and the role of ' proven effectiveness'

within this, deserve a place on the ethical policy agenda?

These questions will mainly be answered using examples in the area of treatment; diagnostics plays

a smaller role. The reason for this is pragmatic: the discussion about evidence-based medicine and

health insurance packages has until now primarily focused on forms of treatment. However, the

conclusions will be relevant to both areas.

The monitoring report was written by four authors on behalf of the Centre for Ethics and Health (CEG).

The text has been reviewed and approved by the Council for Public Health and Health Care (RVZ),

which together with the Health Council of the Netherlands is responsible for monitoring reports that

are published under the name of CEG.

The findings are based on information from two sources. The first was a literature study, which focused

on historical and ethical publications and on policy documents about the composition of health insurance

packages. The literature study was based on an electronic search in the medical and medical ethics


literature, using the key words 'evidence-based', 'ethics' and 'policy' and the search filter 'bioethics'.

The websites of the most important health insurers were also consulted.

The second source of information was interviews with 17 'stakeholders' 1. All interviews had an open

character and started with the question as to what role evidence-based actions played within the

discipline concerned. Subsequently more detailed questions were asked about what the respondent

had stated. Sometimes the visions of earlier interviewees were presented in order to gain a reaction.

On average, each interview took one hour.

1.3 Repor t out l ine

Chapter 2 gives a brief history of developments within the area of evidence-based medicine. How the

method is currently applied in treatment and policy is then reviewed. Chapter 3 contains a discussion

of the pros and cons associated with using proven effectiveness as a criterion. Chapter 4 considers

the ethical questions that can arise if scientific evidence plays the decisive role, as is increasingly

the case in the drawing up of the basic health care insurance package, or a subservient role, as is

the case for the drawing up of the supplementary health care insurance packages. Chapter 5 details

the points that are important for the ethical policy agenda.

1 A general practitioner, nursing home physician, company doctor, nurse, physiotherapist, social worker, health lawyer, palliative care

researcher, patient representative, philosopher/ethicist, physican-epidemiologist, pharmaco-epidemiologist, medical advisor health care

insurer, the Cochrane Collaboration, two health care inspectors, a policy officer from the Ministry of Health, Welfare and Sport and an

employee of the Care Insurance Board (see appendix 2).


2. Use in health care practice and policy

How did evidence-based medicine arise? How has it developed? And how is this system used to

weigh up scientific evidence for treatments and policy? This chapter will provide a brief history of

evidence-based medicine and then detail its current status. That will provide a good starting point for

the subsequent analysis.

2.1 Development o f evidence-based medicine

Origins in medical practice

The term evidence-based medicine arose in the 1980s and 1990s, but of course efforts to make

well-founded choices have a much longer history. However, the approach used to support choices

has changed over the years. For example, up into the eighteenth century the phenomenological

approach to diseases was common practice. The patient stated the nature and progression of his

complaints and then the physician made his diagnosis; diseases were mainly classified according

to their symptoms. Observation and physical examination played a subordinate role in this process

(Timmermans & Berg, 2003). By the start of the nineteenth century, major advances were being made

in the disciplines of anatomy and pathology. These went hand in hand with the increasing use of

experiments and the development of statistical methods.

An important impulse for the development of evidence-based medicine came during and after the

Second World War. The explosive increase in new medicines led to a growing demand for studies

into their effectiveness. For example, in 1943 the British Medical Research Council (MRC) started

the first double-blind and controlled study into a potential product against the common cold. The

product was found not to work. Three years later the MRC investigated the effectiveness of a

tuberculosis treatment with Streptomycin. This was the first occasion on which the study groups were

randomised. Besides demonstrating the effectiveness of the antibiotic, this study also showed that it

was feasible to conduct randomised, controlled and double-blind research.

Besides the boost given by the emergence of the pharmaceutical industry and effectiveness

studies, the post-war years' search for evidence was also stimulated by the increase in

internationalisation. The growing exchange of knowledge and expertise was not just limited to the

political and economic arenas but took place within medical disciplines as well. This was facilitated by

the standardisation of research and publication techniques (Wiersma, 1999).

Moreover, individual researchers sometimes made important new steps possible. For example, in 1973

Cochrane published Effectiveness and efficiency, in which he called for a greater role for evidence in

medical diagnosis and treatment. The epidemiologists Sackett and Feinstein also made an important

contribution: they opened up the use of statistics to scientists and medics.

‘The Dartmouth Atlas of Health Care' had an impact similar to Cochrane's book. For example, the

atlas clearly showed that in the case of breast cancer, some American states carried out radical


breast amputation 33 times more often than others. Since 1973, a new edition of this American review

mapping the number of medical interventions per region has been regularly published.

New societal impulses

Twenty years ago the development of evidence-based medicine suddenly accelerated. Several factors

played a role in this.

First of all there was a growing demand to legitimise medical practice. This call came from both

patients and medical professionals (Sackett et al, 1996). The emancipation of the patient and changes

in the balance of authority between the physician and the patient allowed a more critical attitude to

develop towards medical practice. Medical professionals therefore saw the developments in evidence-

based medicine as a possibility for legitimising medical practice and maintaining their position of

authority (Timmermans & Berg, 2003).

Secondly, the considerable developments that took place within information technology have also

contributed to the increasing importance of evidence-based medicine. Since the emergence of

Internet, physicians and patients have had increasingly easier access to a wealth of information.

The evidence-based medicine approach forms a framework in which information can be sifted and

prioritised.

A third impulse arose when the approach started to make inroads in areas outside of clinical practice.

The growth in diagnostic and therapeutic possibilities led to spiralling health care costs in the

Western world. Therefore an increasingly urgent need arose for a method to assess the efficiency and

effectiveness of treatments. Evidence-based medicine provided governments and insurance

companies with an instrument to combat non-effective and unnecessarily expensive treatments.

'A scarcity of financial resources and a wide choice of interventions, yet you can only spend your

money once: this combination was an important driving force for EBM.' (Interview with physician-

epidemiologist)

'Ultimately EBM has emerged from the requirements that governments have long imposed with respect

to the safety and efficacy of medicines. First regulation of the safety, later regulation of the

effectiveness and quality and yet later still the efficiency. These government regulations are the basis

from which EBM has so strongly developed.’(Interview with policy officer from the Ministry of Health,

Welfare and Sport)


The d e f i n i t i o n of EBM

Ideally, evidence-based-medicine concerns the integration of individual clinical expertise with the best

external evidence available from systematic research. The preferences, wishes and expectations of the

patient play a key role in the decision-making process (Offringa et al, 2003).

Three items play an important role in good care: the purpose of the treatment (what patients find impor-

tant/need/would like), the available evidence concerning the effectiveness in patient populations (the

chance that a patient will benefit from a treatment), and the clinical experience concerning the appli-

cability of that evidence to the specific patient facing the health professional. Clinical experience

uses what philosopher Annemarie Mol referred to in a recent book as the logic of care, a logic in

which the application and harmonisation (calibration) of knowledge for the individual patient is crucial

(Mol, 2006). Therefore the first question is: what do patients and health professionals want to achieve

with the care (cure? best possible quality of life? support?); the second: what provides the biggest

chance of realising that objective; and the third: to what extent is that also helpful for this patient and

what should be done if this is not the case (for example due to comorbidity or other specific charac-

teristics and preferences)?

Extending to other areas

Evidence-based interventions are now firmly rooted within medical practice. Initially, some people

thought the evidence of effectiveness would be enough to guide and legitimise the actions of health

professionals. Yet nowadays, leading advocates of evidenced-based interventions are also calling

for a balance: '[We] now see clinical expertise as the ability to integrate research evidence and

patients' circumstances and preferences to help patients arrive at optimal decisions' (Haynes et al,

2002a & 2002b; see Offringa, 2004 also).

The concept is also making increasing inroads outside of strictly medical disciplines. This applies,

for example, to paramedical care where evidence-based practice (EBP) has already emerged. Over

the past ten years the first books about evidence-based nursing, physiotherapy and occupational

therapy have been published. Via PubMed, 333 articles can be retrieved from the past five years using

the search terms 'physiotherapy AND evidence based'. Ten years ago, just 11 articles had been

published with these search terms over a period of five years.

It seems likely that the same factors that formed a stimulus in medicine in the 1980s have also played

an important role in the paramedical sector. Here the same need arose for proven efficiency and

effectiveness, there was a need for legitimacy and people were swamped with information.

Wishes and

needs of pa-

tients

Clinical ex-

pertise

Proof of ef-

fectiveness

(evidence)


A further expansion is that into the area of policy. We have already seen that policy makers and health

insurers use the principles of evidence-based medicine to justify decisions, for example, about

allocations and what should be included in the health insurance package. In turn, this expansion has

also given a considerable impulse to the development of the system itself.

2.2 Use in current heal th care pract ice

It took many years for evidence to gain a foothold within medical practice, but developments over

the past two decades have been rapid. With this, certain shifts have taken place in the relationship

between the three types of information that contribute to evidence-based medical interventions: the

needs of the patient, scientific knowledge and the professional's judgement. And that search for the

right balance is an ongoing process. What is the current status of scientific evidence in the context

where evidence-based medicine originates from: clinical practice? What is the relationship there be-

tween the three 'parties': patient, scientist and professional?

Guide l ines as a response to sc ienti f ic evidence

A detailed methodology has been developed to utilise the results of scientific research for evidence-

based interventions in health care. Guidelines and standards play a key role in this. They are the

result of a deliberation process within the profession about the scientific knowledge regarding the

most appropriate treatment for specific disorders, complaints and risk factors.

Within the Netherlands, pioneers in that area were the Dutch College of General Practitioners (more

than 80 guidelines have now been published for GPs) and the Dutch Institute for Healthcare

Improvement (which has published several hundred guidelines). Now a variety of professional

organisations are busy compiling evidence-based guidelines. A major player within this field is the

Dutch Cochrane Collaboration, which was founded in 1997.

Guidelines contain recommendations for the care of certain patient populations, based on the

complaint or diagnosis. The strength of the guidelines differs, dependent on the type of research that

is available. A ranking of research types makes it possible to sort the evidence and to determine how

important it is to comply with a guideline. This means that more value is attached to a research

outcome from a particular type of research than a result achieved from a different research method.

Use of guidel ines in everyda y pract ice

Legally speaking, guidelines form a professional standard. Consequently they form an important

element within the Dutch Medical Treatment Contracts Act: ‘In carrying out his duties, the health

professional must pay due regard to the care provided by a good health professional and in doing so

acts according to the responsibility he bears, arising from the relevant professional standard for health

professionals’ (WGBO Art 153). However, guidelines are not binding in the legal sense of the word.

'The guideline is not the be all and end all. People can indicate why they have deviated from a

guideline. Jurisprudence also recognises that guidelines may always be deviated from on the basis of

good arguments. Yet on the other hand, following the guideline to the letter does not exempt a

practitioner from guilt in the situation where it was clearly evident that following the guideline would

constitute a poor medical intervention.' (Interview with health lawyer)

All guidelines contain a qualifier: there can be good reasons to deviate from it. However, in such a

case the proof of evidence is the responsibility of the professional concerned: he/she must, amongst

other things, document in the patient's records why a different treatment was chosen - for example,

because in this case the professional did not consider the guideline to be applicable or because the

patient did not agree with the treatment indicated in the guideline. Therefore following the guideline


is the norm and deviating from it the exception. Consequently it could be stated that the elements

'clinical expertise' and ' patient preferences' carry less weight than the element ‘evidence’.

There is another qualification, however. Although considerable importance is attached to evidence that

is as strong as possible, guidelines always contain clearly non-scientific elements. Consensus about

such elements is achieved during the formulation of the guideline, for example, when hard evidence

is lacking. Further, there are often normative principles for which scientific research provides no

definitive answers. Yet scientific knowledge always forms the basis for a guideline. If such evidence is

lacking then a guideline is abandoned.

'EBM is particularly important as an educational/didactic instrument. The practice of EBM is more

important than the result. Doctors and pharmacists must be trained in EBM so that they assimilate this

form of thinking. Just practising EBM without further reflection is not good. As soon as the

“bookkeepers” become involved (insurers; inspectors; policy makers) then a very important didactic

concept will be misused.' (Interview with pharmaco-epidemiologist)

Intended resul ts

The primary aim of using evidence-based medicine is to satisfy the standards for good care.

Confidence in the physician is a first element in this. If he or she states that the chances of a cure

are 20 percent then there must be a good reason for this. According to the approach of evidence-

based medicine, such a statement can be supported by a good historical group analysis. If the same

physician says ‘This treatment has a 10% higher chance of cure but more side effects’ then

according to the rules of evidence-based medicine, knowledge from a randomised-controlled trial

(RCT) is usually needed.

Another element that contributes to the quality of care is health professionals having the most up-to-

date knowledge at their finger tips. Use of the evidence-based medicine approach helps them in this

respect (Evidence-Based Medicine Working Group, 1992; Goodman, 2003) and is even considered to

be the most important exercise: the system must help health professionals to be able to take important

decisions, sometimes even about life and death, in a situation of uncertainty.

That happens by ensuring order in the wealth of information from medical research. Some 40,000

medical journals publish more than two million articles each year. Between 1990 and 1995, no less

than 14,000 articles were published about the role of calcium channel blockers in the treatment of

high blood pressure. Specialists must read between fifteen and thirty articles per day to remain up to

date in their discipline; for generalists such as general practitioners this number is far higher (Offringa

et al, 2003). The levels of evidence used to rank studies on the basis of argumentation, and the

guidelines and protocols compiled on the basis of this consideration, are essential for practising health

professionals to keep on top of this enormous flow of information.

By making this flow of information manageable, evidence-based medicine ought to simplify the

execution of complex tasks. For example, since the emergence of evidence-based practice, dozens of

physiotherapy guidelines have been developed for frequently-occurring complaints. That has

important advantages:

'[...] streamlining the individual steps leads to an increase of the overall complexity of the overall task

without a concurrent increase of the complexity of the individual steps.' (Timmermans & Berg, 2003).

Furthermore, the approach is intended to banish ineffective or harmful interventions. For example,

within general practice EBM has, according to a general practitioner interviewed, led to a drastic

change in the treatment policy for a sprained ankle. Previously, the physician or nurse requested an


X-ray and bed rest and a plaster cast were prescribed; now the ankle is bandaged and the patient is

given the advice to walk.

'Thanks to EBM we have discovered how little we really know. Science cannot prove any truth.

However, you can use EBM to demonstrate what does not work. ‘Exposing lies.’ Research should be

focused on undermining your prejudices. (Popper's falsification principle) Valuable concepts from

EBM are ‘Numbers Needed to Treat’ and ‘Numbers Needed to Harm’. Those are terms that you can

explain to everyone, and that is a major advance.' (Interview with physician-epidemiologist)

Another intended effect of evidence-based working in health care practice is the safeguarding of

quality. The uniform scientific methods make it possible to objectify medical knowledge, which in turn

increases the transparency of medical practice. Where you live and who treats you should no longer

determine the treatment you receive.

With this the old-fashioned medical paradigm is rejected, as this would not lead to satisfactory

medical intervention:

'EBM de-emphasizes intuition, unsystematic clinical experience and pathophysiologic rationale as

sufficient grounds for clinical decision making and stresses the examination of evidence from clinical

research.' (EBM Evidence-Based Medicine Working Group, JAMA 1992).

Not everyone considers this to be an unequivocal advantage: a less important role for intuition and

clinical experience is perhaps an advantage for many patients, yet in some cases it can also prove to

be detrimental.

'What do you hope to achieve with a blood transfusion for an incompetent demented man with

anaemia, which is making him tired and short of breath? If you do nothing then sooner or later the

patient will develop heart failure and die from this. Perhaps that is desirable because then at least he

dies from this and his dementia will not have to get any worse. Who decides what is good? That is

often expressed in terms of quality of life. If you give him a transfusion you prevent heart failure

without knowing what the cause of the anaemia is. Perhaps a tumour is the cause. Such an

approach leads to one uncertain decision after the other. A carefully made decision needs to be

justified as well as possible. There are no concrete empirical bases for this. I make a detailed

decision-analysis: what will I miss if I do not carry out any further investigations? what could that

mean for the patient? You explore different options and the extent to which these can influence your

decision, taking the ethical aspects into account and discussing the possibilities with the family.’

(Interview with nursing home physician)

2.3 Use in current policy

How is the approach of evidence-based medicine currently used in health care policy? Policy

documents from the Ministry of Health, Welfare and Sport scarcely contain the term 'evidence-based’.

However the concept does play an implicit role, namely where the quality of care is in the spotlight.

Interestingly, recent policy memorandums have paid considerable attention to the position of the

patient; in current government policy there appears to be a greater focus on the role of the patient than the

role of evidence.

In one area evidence and effectiveness are, however, playing an increasingly important role: in

decisions about what to include in the basic health insurance package. Several important reports and

recommendations devote a considerable amount of attention to the role of scientific evidence in this. A

review of this follows.


Choices in heal th care ( 1 9 9 1 )

In the final report of the Choices in Care Commission published in 1991, proven effectiveness is the

second filter in a series of four. The filters are intended to select care that should be placed in the

basic health insurance package:

1. Is it necessary care, viewed from a perspective of solidarity? (if no, not in the basic health

insurance package)

2. Has the effectiveness been demonstrated? (idem)

3. Is the care functional? (idem)

4. Can the care remain the responsibility of and be paid for by the patient? (if yes, not in basic health

insurance package)

The report explains the functioning of the second filter as follows: 'The second filter selects on the basis

of effectiveness, according to a scale that runs from proven and documented effectiveness, via

assumed and weakly documented effectiveness, to not demonstrated and proven and documented

ineffectiveness. The commission is of the opinion that only care for which the effectiveness has been

proven and documented should be included in the basic health insurance package. Assumed

effectiveness is that which must still be demonstrated. Care for which the effectiveness has not been

proven or that has been proven to be ineffective, must remain outside of the basic health care

package.'

The advice is clear: only evidence-based care should be included in the basic health insurance

package. A proven ineffective treatment is not included, even if it could still be worthwhile in some

individual cases. The situation is different for care for which the effectiveness is assumed but not

(yet) proven. This is probably the largest group. In that case further research is prescribed, but the

treatment is provisionally included in the basic health insurance package.

Contours o f the basic heal th insurance package (2003)

The advisory report Contours of the Basic Health Care Benefit Package that the Health Council of the

Netherlands published in 2003, partly builds further on the work of the Choices in Care Commission.

The main new aspect is the emphasis on defining and measuring disease burden as a criterion for the

necessity of care (the first filter). The Health Council draws attention to the differences in scientific

evidence that can be provided for different types of care and treatment: '[...] distinction between cure

and care activities is not necessary. Both are focused on reducing disease burden. Currently it is

easier to subject a purely curative form of treatment to scientific assessment than help which consists

almost entirely of nursing and care. The problems that arise in trying to define, standardise and

measure, for example, the need for help therefore deserve greater scientific attention.'

Consequently it is difficult to draw a clear distinction between care and cure. Nursing, care, time and

attention also play an important role within curative care and purely 'curative forms of treatment' are

difficult to find. This means that for the time being, curative care is also difficult to assess in scientific

terms. Consequently the conclusion from the advisory report ought to apply to care elements in

curative care as well: if research into the effectiveness is difficult then effectiveness should be given less

weighting than necessity.

Package management in everyda y pract ice (2006)

In 2006, the Care Insurance Board (CVZ) published the report Package management in everyday

practice. In line with the advisory report from the Health Council of the Netherlands, this report proposed

that the criteria of the Choices in Care Committee should be used in a weighted system and not

hierarchically. However in the CVZ report necessity and effectiveness are still the most heavily

weighted factors: ‘yet if, for example, a form of care is not in the slightest bit necessary or transpires


to be completely ineffective then this principle will be decisive in the assessment.' On the other hand,

this vision does permit a form of care for which the efficiency and effectiveness have not been

sufficiently demonstrated, but is nevertheless necessary and not easy for patients to pay for, to be

included in the basic health insurance package.

According to the CVZ, conclusions about efficacy should preferably be based on evidence from at last two

RCTs and if that is lacking on evidence from a lower level. The CVZ report also devotes considerable

attention to the lack of effectiveness measures in care, an issue raised by the Health Council of the

Netherlands. For the time being, decisions about what to include in the package will have to be made

despite a lack of evidence or weak evidence, says the CVZ.

The CVZ emphasises that in these cases the assessment should not just be based on current scientific

knowledge but also everyday practice. This applies to 'all forms of care and therefore to the care and

services for which less or no scientific status is present or needed'. Examples of facilities that must be

primarily assessed against everyday practice are the 'welfare-related' accessories, for example those

aimed at facilitating communication.

Therefore the CVZ is clearly interested in more than just proven effectiveness alone. Yet assessing

against everyday practice is not applicable to 'forms of care for which the (cost) effectiveness can be

demonstrated by means of scientific research' (p. 6). With this scientific evidence remains the guiding

principle.

‘[Equal assessment criteria] are of course not completely possible, although this is the aim. Also

where palliative care is concerned, and the question as to whether or not a guide dog should be

funded from the health insurance package, the same principles should apply as much as possible.'

(Interview with employee of CVZ)

Worthwhile and sustainable care (2006)

The Council for Public Health and Health Care published the advisory report Worthwhile and

sustainable care in 2006. In this advisory report, the first of two reports on this subject, the Council

considers how the criteria of necessity, effectiveness and justification can be used together in a

weighted system. The key idea in this report is that the acceptability of costs per quality of life year

gained is dependent on the disease burden (as defined in Contours of the Basic Health Care Benefit

Package). According to the Council, a public debate is needed to determine how high the maximum

reimbursements from the basic health insurance package may be.

Just like the CVZ, the RVZ recommends that effectiveness, necessity and justification need to be

viewed in conjunction with each other. It might therefore be reasonable that a moderately effective

treatment for a grave illness is more entitled to reimbursement than a highly-effective treatment for a

non-debilitating disease. Determining the effectiveness of care interventions also requires different

outcome measures than those used for cure interventions. Therefore a shift has taken place since the

publication of the report Package management in everyday practice. Whereas in the case of grave

illness, the CVZ would be satisfied with less strong evidence, the RVZ proposes that in such

cases a less strong (but nevertheless proven) effect must be the decisive factor.

2.4 Conclusion

What is the role of scientific evidence painted by the overview presented in this chapter? We have

seen that evidence-based medicine has developed within the medical profession, with the emergence

of new treatments and research methods. Over the past twenty years in particular, the approach has


developed enormously and the result is a hierarchic system for weighing up the importance of research

results.

This instrument now helps medical professionals to cope with the enormous quantity of research, to use

the best knowledge from this and in doing this to safeguard the quality of care provided. However, this

has led to a shift in practice: the health care professional no longer makes a judgment on the basis of

knowledge acquired or that has been passed on, but instead adheres to the guidelines of the

professional association that have been compiled on the basis of the best scientific knowledge

available. Yet there are still countless situations in which the scientific knowledge is uncertain or

lacking. Moreover, the professional still needs to determine whether adherence to the guideline is

indeed the best choice in each individual case. The patient's interest is the guiding principle here.

When it comes to policy, the trend in recent years has been a stronger emphasis on input from the

patient. Yet for decisions concerning what should be included in the basic health insurance package,

evidence is an important criterion. Subtle changes are gradually taking place, however. For example,

there is a growing realisation that it is difficult to draw a distinction between care and cure. Both contain

elements that are difficult to investigate using the methods that have acquired the highest status within

the approach of evidence-based medicine. Consequently, there is increasing attention for other methods

to assess the utility of allocations, despite the ever-present pursuit for proven effectiveness.


3. Possibilities and limitations of evidence

In the previous chapter we saw how the approach of evidence-based medicine is used in health care

practice and policy. Yet it became clear that it is not always easy to provide the required proof of ef-

fectiveness for all forms of care. So what are the possibilities and limitations for using scientific re-

search to determine the effectiveness of care (Goldenberg 2006)? What evidence is required? And

can this requirement always be satisfied?

3.1 Prior i t is ing evidence

Guide l ines of di f fer ing strength

As we saw in the previous chapter, it is now customary in clinical practice to compile guidelines for

professionals based on the available evidence and to prescribe these to a greater or less extent de-

pendent on the strength of the evidence concerned. Four levels of evidence are distinguished.

Conclusion based on

1 Study at level A1 or at least 2 level A2 studies performed independently of each other

2 1 study at level A2 or a t l e a s t 2 s t u d i e s o f l e v e l B p e r f o r m e d i n d e p e n d e n t l y o f e a c h o t h e r

3 1 study at leve l B or C

4

4

Exper t op in ion

Source: Dutch Institute for Healthcare Improvement (CBO).

Proofs w ith di f fer ing s tatus

This means that evidence-based medicine depends to a significant extent on the hierarchical priority

of the evidence. The outcomes of certain types of research are given more weight than results

obtained using less valuable research methods.


Right from the start, the randomised clinical experiment has been considered to supply the highest

level of evidence. Preferably, several such experiments should have been performed for a treatment,

which together provide a consistent picture of the effectiveness. All other forms of research, such as

those where no control group is used (as is the case for qualitative research), yield less strong

evidence. The lowest status is the form of evidence based solely on expert opinion.

Classi f icat ion of methodological qual i t y of ind ividual studies

Intervention

Diagnostic accuracy study

Harm or side effects, aetiology, prognosis*

A1 Systematic review of at least two A2 level studies performed independently of each other

A2 Randomised,

double-blind

comparative clinical

study of good quality

and sufficient size

Study compared to a reference test

(a '’gold standard') with predefined

normative values and independent

assessment of the results of the

test and gold standard and

containing a sufficiently large

series of successive patients all of

whom have had the index and

reference test

Prospective cohort study of

sufficient size and follow-up for

which confounding has been

adequately controlled and

selective follow-up sufficiently

excluded

B Comparative study,

but not with all of the

characteristics stated

under A2 (this

category includes

patient-control

studies, cohort

studies)

Study compared to a reference

test, but not with all of the

characteristics stated under A2.

Prospective cohort study, but

not with all of the characteristics

stated under A2 or a

retrospective cohort study or

patient-control study

C Non-comparative studies

D Expert opinion

Source: Dutch Institute for Healthcare Improvement (Dutch acronym: CBO).

This overview indicates which forms of evidence should ideally be available to conclude that

interventions are effective. However, the real question is what happens if the desired evidence is not

available. Three situations will be considered: proof of effectiveness is lacking at present, only

evidence of a lower order is available, and proof of effectiveness is not relevant for the form of care

concerned. Then the consequences of each scenario will be described.

3.2 Gaps in the avai lab i l i ty o f evidence

Hard evidence is lacking for the t ime being

In the paramedical professions, preventive care and nursing, too little research is being done at

present into the effectiveness of interventions to provide the type of evidence preferred within the

approach of evidence-based medicine. Speech therapy, physiotherapy and occupational therapy are

also health care disciplines in which sufficient evidence for the effectiveness of treatments is often not

yet available (Kalf, 2004; Kinebanian, 2006).


However, that evidence can often be obtained. For example, the effectiveness of many forms of nursing

care could be thoroughly investigated. At present simply not enough research is being done to gather

that evidence and render it usable.

'Nurses must be familiar with the hard side (quantitative, scientific evidence). However, here lies a

problem. Also for doctors. Sometimes they manage without the hard evidence that is available. And

the art is to make good use of this knowledge without neglecting the relational aspects. That is

difficult.' (Interview with nurse)

Why is so little evidence available in some disciplines despite the fact that this could be supplied

with the right research methods? The most important reason is probably the lack of funding. Setting

up a randomised experiment is very expensive and so the possibility to do that is strongly dependent

on commercial parties, such as the pharmaceutical industry. An estimated 20,000 clinical trials are

carried out worldwide each year, and pharmaceutical companies finance about 90% of these.

Commercially interesting interventions therefore have an advantage over their less commercially

interesting counterparts.

Apart from a lack of funding, the absence of the necessary infrastructure can also play a role. In the

clinical setting, various parties have usually invested in the infrastructure needed to carry out strictly

controlled studies from a methodological viewpoint. This means that the expertise (for example, in the

form of epidemiological knowledge and knowledge about biostatistics and research methods) is

present as well as a certain attitude (a way of working, for example with protocols). The required

facilities (such as assistance and hardware) and materials (equipment and space) are also present. In

paramedical care settings, on the other hand, far less has been invested in the infrastructure and so

the research culture, organisation and coordination are missing.

'A student nurse had a proposal to interview cancer patients who received a treatment which had such

unpleasant side effects (skin problems; pain) that they scarcely dared to meet other people. They also

experienced problems with intimacy. She wanted to interview the patients about their experiences.

What was the impact of the treatment and did they really want to undergo this? The professor said:

"Nonsense! Measure something, do something and measure again. Then you've finished!” However,

the student did succeed in ‘mapping out’ the suffering of these patients. And all of these patients

agreed that they did want to go through with the treatment. That simply makes it stronger!' (Interview

with nurse)

Onl y ' l o w e r ' evidence can be obta ined

A second possibility is that the highest form of evidence is not possible. This can be seen in some

aspects of preventive health care. For example, in lifestyle programmes there are complex interactions

between many variables. If reliable results are to be obtained in such a situation then the size of study

required is so large that it can scarcely be funded and if there is no commercial interest involved, a

company will certainly not want to invest in research.

'Evidence for screening requires expensive, large studies, but can be commercially interesting. [...] Yet

for research into malaria vaccines you need the Bill Gates Foundation.' (Interview with physician-

epidemiologist, Cochrane Center)

That therefore limits the possibilities for providing foundational arguments, whereas prevention is one

of the focal points of Dutch government policy. Preventing disease would be better for citizens and the

costs of health care would be significantly reduced. Some preventive treatments can, however, be

properly tested in RCTs in terms of both design and funding. These are commercially interesting


cases, such as cholesterol inhibitors to prevent cardiovascular diseases, where only a limited

number of variables need to be included.

'The same demand for the right outcome measures plays a role in the assessment of preventive

health care. This concerns both the lack of hard evidence and how population screening should be

carried out – for example, in the case of mammography there are too many axes that cannot be

tested in a randomised trial. This calls for observational methodology, questions about accuracy, test

validity, test quality and other endpoints – such as normative questions – when is "adding years to

life" a better alternative than "adding life to years"? Value judgements cannot be tested in EBM.'

(Interview with physician-epidemiologist)

In other forms of care and certain health care disciplines, the problem of the highest standard of

evidence (therefore with repeated RCTs) not being appropriate also arises (Ashcroft, 2004).

Consequently only evidence of a lower order can be expected, even if the funding, infrastructure and

research culture are, in principle, present.

In such cases methodological problems play an important role. For example, the patient group can be

highly complex (e.g. multimorbidity in the case of the very old, and terminal patients). Sometimes the

outcomes in the sectors concerned are difficult to capture in hard figures. Obtaining reliable results

can also be difficult if the context is complex and the intervention concerned cannot be investigated

independently of this. Then there is a real risk of bias (Vliet Vlieland, 2002).

'In the case of occupational health it is hardly ever possible to allow the intervention to change and the

rest to stay the same. For better or worse this problem is often resolved by carrying out RCTs anyway

but often with smaller numbers of patients and more clustered designs. Randomisation then takes

place at the level of the group and not the individual. Subsequently you need to adjust for a range of

influencing factors in your outcomes.' (Interview with company doctor)

Level of sc ienti f ic evidence is not relevant

A third possibility is that there are forms of care for which proof of effectiveness is not relevant, or is

even pointless. That is the case for forms of care we still find valuable even if their effectiveness can

never be proven. Examples are qualities such as being present, having time and giving attention.

Some of the people we interviewed compared this to policing and education: professions where many

interventions and actions are not called into question, despite the lack of hard evidence for their

efficacy.

'It is just accepted that people need to be cared for. No one asks how "evidence-based" the local

police officer or education is. The effect and result of education and expertise can often be called into

question, yet nobody doubts its usefulness. But education is not dealt with as harshly as EBM and

physiotherapy.' (Interview with philosopher)

The sociologist Duyvendak considered the theme of provability using social work as a case study.

He said the following about this: ‘Measurements can contribute to knowledge but because each

discipline is different, the creed “measuring equates to knowledge” only applies when the measurement

method is adapted to the specific type of results that can be achieved in the area concerned. (...) In

particular, social objectives for the longer term, such as increasing social cohesion, can scarcely be

captured in “hard” units that can be applied anywhere. This in the case for welfare, where quantitative

data do not give the full picture. More often it concerns “unique” data, the stories behind the figures.

Neither the inseparability of the variables from the context nor the complexity of converting the results

into figures and narratives facilitate the effort to benchmark’ (Duyvendak, 2003). What Duyvendak

noted about the difference between hard and soft evidence in welfare work is also applicable to health


care.

3.3 Consequences of the d if ferences in levels o f evidence

Confus ion about 'n o t proven ef fect ive ’ and 'proven not e ffect ive'

What are the consequences of a lack of evidence? Firstly, there is the danger that sight will be lost of

the distinction between 'not proven effective’; and 'proven not effective', and that forms of care will

unjustly be labelled 'proven not effective'.

In practice, the conclusion that there is no evidence for the effectiveness is sometimes

misunderstood. For example, at the start of the 1990s the report Effectiveness of physiotherapy

(Beckerman & Bouter, 1991) led to heated discussions, as this drew the conclusion that for a number

of applications within physiotherapy (such as electrotherapy) little evidence, in other words 'no

evidence of effect', was available. 'No evidence of effect' means that there is no prove of effectiveness

but also no evidence that the intervention is not effective (Van Weel & Knottnerus, 1999). However in

the media, 'no evidence of effect' is all too quickly interpreted as 'evidence of no effect'. Politicians

and health insurers subsequently used this to make cuts in physiotherapy financing.

Less considerat ion for important questions

Within health care, the difficulty of not being able to investigate effectiveness adequately is more

important in some sectors than in others. This applies, for example, to prevention and in particular

interventions in the area of lifestyle. Furthermore, in sectors that care for patients with multiple

conditions, such as nursing home care and palliative care, it is notoriously difficult to set up good

scientific experiments. And in occupational health the strong effect of circumstances is a problem.

Consequently, in such cases there is a danger that only things which can be investigated according

to the accepted methodology will be studied, and important questions that require a different

approach will not be tackled.

Other forms of knowledge are devalued

Further a hierarchy of evidence can lead to a devaluation of other forms of knowledge, such as

clinical experience or the familiar 'it is not right’ feeling. Ashcroft (2004), for example, has expressed

concerns about this. Evidence-based medicine, he says, is far too focused on objective and

quantitative parameters, whereas qualitative and narrative data, an essential element of the

professional expertise of health care providers, are used too little. A similar concern was expressed in

The Lancet several years ago by Dutch professors of general practice Van Weel and Knottnerus:

'With the RCT (randomised clinical trial) methodology of evidence-based medicine, the convention is to

ignore the consequences of the personal dimension, and instead to focus on the assessment of

evidence exclusively in terms of the disease intervention.' (Van Weel & Knottnerus, 1999).

Also the experts interviewed for this monitoring report emphasised that other forms of evidence are

becoming undervalued. They also believe that lower level evidence is sometimes sufficient to guide

medical decisions. If, for example, rare fatal side effects are observed in unnecessary interventions

then these form an important argument for suspending the intervention. No RCT is needed for this.

Stronger still: rare fatal side effects often do not manifest in trials because the numbers involved are

too small for this. However postmarketing surveillance, following the effects of a drug now marketed,

does provide this information.

That is why the ethicist Goodman (2003) calls for a broad use of the term evidence that in his opinion

also ought to cover clinical experience and theoretical knowledge: 'Maybe we should construe


"evidence" more broadly, to include scientific evidence, theoretic evidence, practical evidence, expert

evidence and judicial evidence'.

Development of orphaned f ie lds of care

If an upward spiral of increasingly stringent requirements on research and scientific credibility arises

then these could lead to an increasingly stricter selection, with the consequence that valuable forms of

care for which no evidence is yet available could become marginalised. Vos et al refer to these as

'orphaned fields' within medicine: areas that are not interesting for well-designed studies and therefore

acquire the same status as orphan drugs (Vos et al, 2004).

'The greater the quantity of evidence that can be gathered, the more you can know and the more you

can demand to know. This therefore means increasingly stringent data requirements. It remains to be

seen whether this is an ethical problem but it is certainly an economic problem: nowadays increasingly

larger studies are necessary that require even higher investments. In principle, a good starting point is

to clearly establish the added value of the new therapy. And should one be cautious about offering

new therapies anyway? All of this means that it is becoming increasingly difficult to market new

therapies. Consequently potentially good therapies might never be marketed. That could pose an

ethical problem.' (Interview with policy officer, Ministry of Health, Welfare and Sport)

Innovat ive treatments under pressure

Further the emphasis on evidence can also inhibit innovative treatments and methods. For example, Biller-

Adorno et al (2004) show how evidence-based medicine can influence hospital managers who are under

budgetary pressure. They can be inclined to so strongly limit the use of not-proven treatments that no more

new treatments are developed.

Infr ingement of the autonom y of the heal th professional and patient

Another disadvantage can be that evidence-based medicine is increasingly used in so-called disease

management and in the use of performance indicators, say Biller-Adorno et al (2004). This can damage

the autonomy of the physician and patient.

One of the people we interviewed for this monitoring report drew a distinction in this context between

evidence-informed practice (leads to ‘client-specific care’) and evidence-based medicine as a policy

instrument (leads to ’moderated care’).

'I continue to state aloud, also to the Health Care Insurance Board and the Dutch Healthcare Authority:

remember that for everything which comes onto the market and for which you as policy makers think

"now we finally have some hard evidence that something works", you only have a half truth. The truth

only becomes fully evident in the light of everyday practice. Especially in the case of illnesses with a

high disease burden, such as cancer and cardiovascular diseases, patient-specific care should be

given; not measured care. In the case of moderated care, EBM is a very dangerous instrument if it is

used in the sense of the ultimate proof for the policy maker or financier.' (Interview with former general

practitioner, patient representative)

Unfai r ef fect possible

There might be a societal effect as well. One of the objectives of evidence-based working is that the

care will become more transparent and therefore fairer as well. The Australian ethicist Rogers (2002,

2004) is, however, concerned about whether care will indeed become fairer. 'Hard evidence is very

seductive,' she says. Consequently, considerations such as necessity can be pushed into the

background as a result of which, in the words of Rogers, we no longer do what is necessary but that for

which there is evidence.


She adds a political consideration to this: as the majority of evidence is gathered about medical

treatments that the most wealthy people undergo, this type of research will be invested in and not

research into the treatment of diseases that mostly occur in the lower socio-economic classes.

Preventive measures will also come under pressure. After all, the majority of evidence comes from

commercially-sponsored experiments. Or, as Saarni and Gylling (2004) say in the same issue of the

Journal of Medical Ethics: evidence-based medicine may appear to be an apolitical way of making

choices in care but it still has a political effect.

Loss of important values in health care

And then there is another effect: the emphasis on evidence as a quality criterion can lead to medical

disciplines for which less proof of effectiveness is available (for example, in nursing or medical social

work) receiving a lower status within medicine as a whole, with the consequence that the values

underlying these will also be considered less important.

'The need for nursing home care is not unequivocal. Do you need physicians in the nursing home

setting? Can this be proved? [...] Evidence can play a role in safeguarding the discipline. Not only

with respect to other medical disciplines but also with respect to health care insurers. Health care

insurers will request nursing home physicians to demonstrate their usefulness in hard outcome

measures. Preparations for this are now being made within the profession.' (Interview met nursing

home physician)

If certain disciplines or specific elements of care within medicine for which the effectiveness is not

measurable come under pressure then that could mainly have consequences for forms of care

where the emphasis is on elements such as 'presence' (Van Heijst, 2005; Baart, 2004). 'Presence' is

about patients feeling heard and being important. Whether or not these objectives are achieved cannot

be established by doing research into clearly definable and measurable effects.

This does not mean that research would be pointless in such cases. However, a different focus would

be needed. Instead of investigating the effectiveness of interventions, the research question would

have to be whether or not patients are satisfied. This applies, for example, to medical social work.

Although it might be possible to measure effectiveness in this discipline, it is questionable whether this

is necessary. After all the aim is broader than achieving health gain by means of an intervention.

'Do I perform intervention “a” or not? Then you always have the alternative of not doing it. It does not

work like that in health care. People who need care need to be cared for. And that is that. Only once

this basic point has been accepted can discussions about the efficiency of the health care provided

take place. And if you start talking about minutes of attention then I think: what on earth are you talking

about: shouldn’t nurses always be able to take the time just to have a simple chat with a patient? I do

not want to waste time finding evidence-based reasons for whether a chat is necessary.' (Interview

with physician-epidemiologist)

Therefore investigating such qualities requires a method other than the randomised experiment. How

can this aspect of health care gain a place within the current hierarchy of evidence? Other forms of

evidence are more suitable in the area of presence, for example, based on the degree to which the

care provided satisfies the patient's expectations.

'Physiotherapy does not prevent any deaths and does not have any spectacular interventions. As a

physiotherapist working in geriatric care you ensure that people can get outside again. Such outcome

measures are rarely included in research. Our work concerns, for example, aspects such as well-

being, empathy and being able to enjoy your twilight years. Those are not exactly sexy outcomes


measures. Evidence-based practice tends to look far too hard towards quantitative research:

systematic reviews, RCTs, guidelines. With this, another type of evidence is all too easily dismissed.

[...] Instead, the outcomes measures important for the patient should be determined. Outcome

measures in which the patient can more easily see his own story. [...] The problem is that such

measures are not considered to be reliable enough compared to the hard outcome value of “muscle

strength”, for example.’ (Interview met physiotherapist)

Impetus for new developments in research

A last consequence is that the increasing demand for proof of efficiency and effectiveness as part of

the pursuit towards evidence-based medicine provides a welcome impetus to the development of the

infrastructure and research culture - certainly in areas where research is indeed possible yet has not

been performed up until now.

A recent example is the development of the protocol Dyslexia Diagnostics & Treatment, published at

the end of 2006; testing and assessment took place in cooperation with many expert practitioners in

the area of psychology and pedagogics and in consultation with professional groups and the patient

organisation SDN. Therefore in this case, evidence-based medicine did not work as a selection

system but helped to get the process going.

It is also plausible that other types of research will be developed that are more appropriate to the care

disciplines which are currently difficult to investigate using the methods demanded by evidence-based

medicine.


3.4 Conclusion

In many forms of care, the hierarchy of evidence that forms the basis of evidence-based medicine is

suitable for determining which treatment interventions are effective, in other words: proven

contributions to health gain. Yet not all forms of care can be tested in this manner. Although

randomised controlled trials are considered to be the gold standard, these are not always feasible and

are sometimes not even possible. The reasons can be methodological but also financial or ethical. For

example, there might be too few patients for an RCT or within a patient group the differences might be

too large to obtain an unequivocal result.

In some cases no research has been done yet, even though this could, in principle, be performed.

Therefore the research area concerned is still developing. In other cases only research with a low

status is possible even if the financial or infrastructural starting point were to be further improved. And

under some circumstances other types of research are more relevant, for example, because not

effectiveness but patent satisfaction is an appropriate outcome measure.

The type of evidence required within the evidence-based medicine system has negative

consequences for a number of health care disciplines. Disciplines where treatment interventions that

have a specific and testable objective are used less, run the risk of losing status and importance. That

applies, for example, to nursing home care, palliative care, medical social work and preventive health

care.

This brings the risk of an increasingly polarised image, as a result of which it might be forgotten that in

many other disciplines with so-called 'hard results' there are elements which are not so easily

measured. Because even after a high-tech operation a patient still needs to be washed and a kind

word during a night of anxiety or pain remains indispensable.

If this trend continues then this could lead to problems in health care sectors where evidence is

more difficult to collect for methodological or financial reasons (for example, paramedical care).

Also certain aspects of the care given could disappear (whether high tech or high touch) such as

dignity, attention, time and involvement. That will eventually undermine the quality of care provided.

Consequently, greater attention needs to be paid to forms of evidence that are appropriate to

different disciplines. And if that is a smile or the offer to give someone’s hair a quick wash then looking

for proof of effectiveness is, perhaps, improper.


4. Consequences for the health insurance package

The previous chapters have described how the system of evidence-based medicine has won increasingly

more ground. An instrument originally developed in clinical practice has now penetrated health care policy

as well, and, in particular, decisions about what to include in the basic health insurance package. However,

we also saw the negative consequences that arise if the system is used to evaluate all forms of care, even

those where proof cannot be obtained or is difficult to obtain. How does all of this affect the role that

evidence-based medicine can play in decisions made about reimbursements? That is the focus of this

chapter. We will start with the basic health insurance package, where evidence still seems to be gaining

ground. However, that is certainly not the case for supplementary health insurance, where patient

preferences are the decisive factor.

4.1 D i lemmas in compi l i ng the bas ic hea l th i nsurance package

Reimbursement decis ions are increasingly made on the basis of ev idence

The NESS Handmaster is a forearm-wrist hand splint to which electrostimulation (ES) has been added.

Since 1996, users had been eligible for a reimbursement for this 'reinforced splint, rectifying or corrective

device' (Medical Devices Regulation, 1996). In 1999 the Commission for Appeal Cases of the Health

Insurance Funds Council confirmed that the NESS Handmaster was indeed eligible for reimbursement.

However, in subsequent years there was much discussion about this. In 2003, the State Secretary for

Health, Welfare and Sport informed the Health Care Insurance Board that she wanted to exclude the

NESS Handmaster from being reimbursed. The Board responded with a provisional standpoint of not

recommending this exclusion. Meanwhile a study had been started that was presented at the end of 2006.

This time a different judgement emerged: the Health Care Insurance Board was ‘of the opinion that the

Handmaster (and other ES medical devices) were not in line with current scientific knowledge and

everyday practice. The Health Care Insurance Board is of the opinion that two well-performed RCTs with

sufficient study subjects and with a long intervention period are necessary to be able to make a statement

about the effectiveness of the long-term use of ES medical devices (including the Handmaster)' (Health

Care Insurance Board report 2006, p. 17). With the hierarchy of evidence as the guiding principle, the

NESS Handmaster was seen as an 'experimental' albeit highly-promising development.

This is an example of a reimbursement decision where use was made of the system of evidence-based

medicine. That is not an isolated example. Evidence also plays an important role in the presentation of

the basic health insurance package by the various insurance companies. An example of this is the costs

for rehabilitation, which are only reimbursed if 'this care for the policy holder is indicated as the most

effective' (website of Dutch health insurer FBTO Art 22).

Man y dec isions are s t i l l taken on other grounds

On the other hand, for many forms of care (for example, oral hygiene) the following formulation is used: ‘The

amount of care to be provided is limited by what [health care providers] tend to offer’. Here common practice

and not evidence is the criterion.


That is partly because for much care, also that considered to be standard practice and which forms part of the

basic health insurance package, the effectiveness has yet to be proven. According to Goodman (2003) that is

the most important ethical problem of evidence-based medicine. He estimates that 10-25% of medical

interventions are based on good evidence.

Since the publication of the Choices in Care Commission’s report, there would appear to be more

openness towards including forms of care in the basic health insurance package for which the

effectiveness has not yet proven. This is probably related to the transfer of some of the care covered

under the Exceptional Medical Expenses Act (AWBZ) to the basic health insurance package. Yet this freedom

is only granted for so long as no evidence is found. All advisory reports clearly indicate that the development

of methods for obtaining evidence and the search for evidence are vitally important, not only in the cure, but

also in the care.

Searching for a s e n s i b l e balance

Over the past few decades the role of evidence in clinical practice has already been put into perspective: it is

just one of the elements of good care (Guyatt et al, 2004). After all evidence-based medicine originated in

clinical practice. In this context, it is crucial for the quality of care that the needs and wishes of patients, the

expertise of health care professionals and the evidence present, together form the basis for good decisions.

Moreover, ‘proven effectiveness’ is not the same as ‘practically applicable’. Even if there is hard evidence for

the effectiveness of a certain intervention that does not necessarily mean that this should be the preferred

option. All sorts of factors can hinder the translation into everyday practice: an intervention might require

more expertise than is available, there might be logistical problems, or the implementation might be too

expensive.

'Stenting versus thrombolysis in the case coronary disease: trials consistently demonstrate that stenting is

better than thrombolysis. Yet in reality, for stenting you need an experienced service that is available 24/7. A

crucial factor is the time between the infarct and the intervention. (‘Time to needle' is always shorter than 'time

to balloon'). In the complex reality of everyday practice, thrombolysis is always better than stenting. That

means you need to give an injection straightaway, if you are certain an infarct has taken place. Just about

anyone can give such an injection, but stenting is far more complex. From an EBM perspective, all

cardiologists want centres for stenting, but that would cost a fortune and would also have logistical

consequences. A better approach is emergency thrombolysis – that buys you some time – and after that

selective stenting.' (Interview with physician-epidemiologist)

This illustrates an important problem of assuming that clinical experiments provide the best evidence: the

problem of translating outcomes from the artificial and controlled situation of the experiment to the complex

reality of everyday practice. 'Clinical studies are usually performed on a homogeneous study population and

exclude clinically complex cases for the sake of internal validity' (Knottnerus & Dinant, 1997).

Such a qualification of the importance of scientific evidence ought to resound in decisions about health

insurance packages as well. Otherwise the situation could arise where reimbursements in the basic health

insurance package are awarded on the basis of levels of evidence and that via this path the freedom for

professional insights and patient preferences will become even more limited still. And with this the quality of

care provided will eventually be put at risk. This means that the approach of evidence-based medicine

cannot be blindly used as a criterion for policy decisions about appropriate care and care that should be

reimbursed.


Searching for appropriate evidence

Yet it is not just a matter of balancing the three elements: patient preferences, professional judgement and

scientific knowledge. Within the last element, efforts can also be made to find greater variety in the evidence

provided. Is it really desirable to pursue a strictly defined form of evidence in all cases? In Chapter 3 we saw

that such evidence cannot always be obtained. The 10 to 25 percent of proven effective treatments referred

to by Goodman provides another reason for adjusting the high expectations.

Therefore the lack of evidence can be viewed in a more nuanced manner. Sometimes research for 'hard'

evidence needs to be stimulated, sometimes less hard evidence needs to be accepted as the best

feasible, and sometimes criteria other than proven effectiveness need to be accepted. An important fear

which emerged in Chapter 3 is that some forms of care will suffer if the same form of evidence is required

of these as for medicines. One solution to this is to determine an appropriate form of evidence for each type

of care and care activity.

For each form of care investigations need to be made into what form of evidence is meaningful and

achievable. This would lead to the development of a characterisation of forms of care and the type of

evidence appropriate to each of these. This has already been proposed for medical practice, for example, in

the scheme below adapted from Knottnerus and Dinant (1997):

Relation between clinical practice clinical research designs

Clinical practice Appropriate design for clinical research

Exploration of hypotheses Qualitative research

History taking Case control study

Diagnostic testing Cross-sectional study

Treatment experience Randomised clinical trial

Individual trial and error n of 1 trial

Following clinical course Cohort study

Record keeping Systematic registry based (computer supported) research

Individual peer review Quality of care research/process evaluation

A similar characterisation could be developed for non-medical care. That could also yield financial advan-

tages. After all, considerable costs are associated with providing evidence according to the strict require-

ments of evidence-based medicine. We have already seen that finding financiers for such studies of non-

medicinal interventions or prevention programmes is far from easy. In particular, experiments with complex

interventions, such as prevention programmes, could become so expensive that the care would be less effi-

cient if such a proof of effectiveness had to be provided.

This trend which is already visible in the case of medicines, namely that the costs of obtaining evidence are

becoming so high (or are made so high) that a treatment is by definition ineffective, could occur to a greater

extent still in the case of highly complex interventions. On the other hand, evidence for an intervention that can

make a real difference does not need to be very expensive, as a clinical effect can often be demonstrated with

a small group of patients (Horrobin 2003).

4.2 Dilemmas in compiling the supplementary package

A different role for evidence

Besides the basic health insurance package, there are supplementary packages. These differ per insurer.

They are made to measure for individual clients or matched to the needs of groups of consumers. What all


supplementary packages have in common is that they enable people to insure themselves for care that

does not satisfy the criteria for care included in the basic health insurance package.

Accordingly, the supplementary package concerns by definition care not considered to be vital, care demon-

strated not to be effective or for which the effectiveness has yet to be demonstrated, care that is not appro-

priate or care that, in principle, the client should pay for themselves but for which they can choose to take out

insurance.

In July 2007, there were about twenty health insurance companies in the Netherlands which all reimbursed

care from the basic health insurance package and, in addition to this, offered supplementary packages

(www.medischestartpagina.nl). Although there is a plenty of choice, most of the supplementary packages

reimburse roughly the same care, which often includes alternative medicine, physiotherapy, skin treatments,

eye corrections, preventive care, dental treatment, treatment for menopausal complaints, contraception and

stays at health resorts.

Examples o f re imbursements

Freedom has clearly been created within the system to insure care that does not satisfy the criteria of the

basic health insurance package but which people still want to receive. As this monitoring report concerns the

role of scientific evidence, we will focus on that criterion. To what extent do supplementary health insurance

packages contain care proven to be ineffective and to what extent do these reimburse treatments that have not

yet been proven to work? In other words: does scientific evidence play a significant role in the supplementary

health insurance package? Several examples will be considered.

Medical check-ups are reimbursed in many supplementary packages, mostly for people older than 40 years;

they may undergo a check-up once every two years. The composition of this check-up differs somewhat

between packages. VGN Zorgverzekeraars reimburse, for example, the following: a questionnaire to be

completed online, a physical examination in which the heart rate, weight and body mass index are

determined, an exertion test and blood and urine tests. If necessary, the lung function can also be examined

and an electrocardiogram made (source: www.vgz.nl, consulted on 12/7/2007). That is a whole battery of

tests, even though there is no evidence that regularly investigating these aspects has a preventive effect.

Many supplementary insurance policies also provide a partial reimbursement for stays at health spas.

Although such a stay is undoubtedly pleasant, there is no evidence that these have a positive effect on

health.

The most striking aspect, however, are the frequently generous reimbursements for alternative medicine. That

usually includes all alternative medicine practices available in the Netherlands. For example, health insurer

DSW reimburses acupuncture, anthroposophic medicine, chiropractice, natural medicine, neural therapy,

osteopathy and homeopathy (source www.dsw.nl, consulted 12/7/2007). None of these forms of treatment

have ever been demonstrated to be effective according to the approach of evidence-based medicine.

'On the insurance market EBM is but one factor. As an insurer you want to remain large and preferably to

grow. Then you are prepared to do “everything” to gain new clients. And EBM then? So what? Consumers are

attracted to alternative medical practices that are not proven.' (Interview met medical advisor health insurance

company)

Consequences of a lack of evidence

Providing insurance for care that people are deemed capable of paying themselves (and that is therefore

not included in the basic package) is not a problem in the supplementary package. Citizens are free to

spend their money in this manner, just as they can insure themselves against the loss or theft of their mobile


phone, even though, in principle, they can buy a new phone should that prove necessary. Also the

insurance of non-essential care, in other words care for complaints with a moderate disease burden,

would not appear to pose ethical problems.

However, a more complex situation is where care is included in the supplementary package which although

effective is not cost-effective, in other words: if the care is too expensive for the severity of the complaint.

This raises the question as to whether such care is rightly excluded from the basic health insurance

package or whether it should be included in that package in so far as it satisfies the efficiency criteria while

the rest is, if necessary, covered by supplementary health insurance. A detailed consideration of that

subject can be found in the Council for Public Health and Health Care advisory report Rechtvaardige en

zinnige zorg (Fair and sustainable care, Council for Public Health and Health Care, 2007).

Then there is the inclusion of clearly ineffective care (assuming that it is has been demonstrated in an

appropriate manner that such care is not effective). For example, some insurers try to attract clients by

offering waiting list negotiation as an extra service: ‘With us you only have to wait for one week’. Yet in

many cases there is evidence that ‘watchful waiting’ is more effective; therefore referral within one week

might be anything but sensible. This evidence conflicts with the wishes of consumers and the attempts of

insurance companies to profile themselves. Must and can health care professionals give due consideration to

this, in other words, must they (partially) match their care to the patient's insurance?

'Professionals want to treat patients according to the professional standard and not according to the

conditions of the insurance policy. Also, from a practical viewpoint it is difficult to draw distinctions in your

internal organisation.' (Interview with medical advisor health insurance company)

With this, the limitations of the distinction between the basic and supplementary health insurance packages

come into view. That distinction mostly concerns the position of the health care professional. Health care

professionals are expected to give the patient the best available care and in so doing to make use of the

knowledge about what will work for this patient. That starting point applies irrespective of whether the

patient is willing to pay, or for what he/she is insured. It forms the basis of the confidence placed in the

person treating them (Council for Public Health and Health Care, 2007).

From the perspective of this responsibility, the reimbursement of ineffective care in the supplementary

package can lead to a dilemma for the health care professional. Of course a health professional is not

required to provide care that he or she considers to be pointless. Yet the pressure to provide patients with

that which they have insured themselves for in the supplementary package can be felt. It gives rise to an

awkward tension. Evidence-based working, which is further the creed of clinical practice, is sometimes left out

of the equation.

This tension can also lead to questions about the integrity of the professional in general, who at one moment

appeals to his professional expertise and at another succumbs to the argument 'but I am insured for that'.

This could damage the confidence placed in the health care professional, whereas a good treatment

relationship is crucial to the quality of care provided.

4.3 Conclusion

In the basic health insurance package, the availability of evidence is an important criterion – even if there

are still far more treatments for which the efficiency and effectiveness have not been proven than there are

treatments that can boast hard evidence. Yet the pursuit of proven effectiveness is clearly present. That is

not the case for the supplementary package. There the patient's preferences are the decisive factor, and

insurers primarily focus on how they can distinguish themselves from the competition.


Both approaches have their own distinct disadvantages. Too great an emphasis on hard evidence could result

in valuable forms of care being excluded from the basic health insurance package. During the compilation of

the basic health insurance package (and also during the drawing up of professional guidelines) high

expectations of evidence can result in other important values being put under pressure. Using demonstrated

effectiveness as a criterion for decisions about the basic health insurance package is, on the face of it, not

open to question. It can, however, have a limiting effect, particularly if hard research results are increasingly

accorded more weight. Consequently, care for which research results about effectiveness are lacking could be

viewed as less valuable, whereas the value based on ethical or societal reasons could in fact be high.

In the supplementary package on the other hand, we can see that ineffective care can also be reimbursed if

people are willing to pay the premium for this. The inclusion of proven ineffective treatments is, in general,

more problematic than the inclusion of treatments that do not satisfy the other three criteria. The insurability

can all too easily be interpreted as a sign of effectiveness.

This duplicity creates, moreover, a scarcely comprehensible discrepancy, certainly for health care

professionals who during their training are convinced of the importance of evidence-based medicine. They

will wonder why on the one hand they are appraised via performance agreements for the extent to which they

work in an evidence-based manner, whereas at the same time patients can claim care for which there is no

evidence whatsoever and purely because they are insured for it.

Therefore should people be allowed to take out supplementary insurance for care, such as regular check-ups,

which has been proven to be ineffective using appropriate evidence? In a previous advisory report (Medical

diagnosis, 2006), the Council for Public Health and Health Care has already drawn attention to the undesirable

situation that whoever wants to take out insurance for dental care, for example, by and large cannot escape

being insured for treatments proven to be ineffective. That advisory report also called for flexible packages so

that this form of tied sale would disappear.

Furthermore, it would be advisable to exclude treatments from packages when there is appropriate evidence

that these do not work. However, it remains obvious that in the basic package, where solidarity continues to

be an important element, other, heavier advice will be considered more appropriate than in the supplementary

package, where the consumer's choice is the decisive factor.


5 Key considerations for policy and research

The emphasis on evidence could lead to the demise of unproven but valuable forms of

care

An important ethical dilemma associated with the use of evidence-based medicine in health care policy, and

especially with respect to decisions about reimbursements in the basic health insurance package, is that a

scarcity of resources can justify a demand for demonstrated effectiveness. However, this could also mean

that valuable forms of care and treatment for which there is no proof of effectiveness could disappear from

the health insurance package.

That can be prevented by di f fe rent iat ing the use of evidence

Therefore the first challenge is to ensure a differential use of evidence in health care policy. For the basic

health insurance package this also means accepting other forms of knowledge, and the dominant position of

evidence obtained from several RCTs being qualified. The types of care need to be characterised, as equally

the forms of evidence most suited to supporting each of these. In this process, due consideration needs to

be given to methodological possibilities and limitations. Not all treatments allow their value to be expressed

in hard figures.

Furthermore, the input of professiona ls and pat ients continues to be of vi ta l impor-

tance

The second challenge within health care policy is to give balanced consideration to all three elements of

evidence-based medicine as opposed to solely focusing on scientific evidence. This means that the need for

care is the focal point, and that professionals continually assess whether following a protocol based on

scientific insights is also in the interest of the individual patient they are currently treating.

Discipl ines where knowledge is not develop ing fast enough need a st imulus

There are forms of treatment and even entire professional disciplines where research into the effectiveness

is scarcely taking place, partly due to a lack of funding and partly due to the lack of a suitable infrastructure

in which such research can be performed. Some of the obstacles can be overcome. The current inequality in

financial opportunities to collect evidence must disappear.

Where hard evidence is impossible other cr i te r ia must apply

Sometimes hard evidence for the effectiveness is impossible, cases in point being support, attention and being

present. Then other criteria for determining the value apply. These can be normative considerations. The

satisfaction of users could also play a role in such cases.


Measuring w ith two standards is confusing for professionals

The distinction between the basic health insurance package and supplementary packages leads to an equivocal

attitude towards evidence. Whereas for the compilation of the basic health insurance package proven

effectiveness is, next to necessity, a decisive criterion, scientific evidence does not play a significant role in the

case of the supplementary package. This can put professionals in a position where they are pressurised to

provide care that is neither evidence-based nor evidence-informed but to which users claim a right because,

quite simply, they have paid for it. Accordingly the role of evidence as a criterion within the supplementary

package needs to be strengthened.

The pursui t o f evidence w il l not make care any cheaper

Good evidence is expensive. Evidence as the foremost criteria for meaningful care is more likely to increase

than to reduce the costs, as setting up the experiments required for this is very expensive. This consequence

of emphasising evidence within health care policy, which is receiving little attention at present, merits further

consideration.

New research methods are needed to provide more appropriate evidence

Good evidence is appropriate evidence. However, at present there are not enough methods available to

demonstrate the effects of certain types of care and, where necessary, to improve the care provided by these.

The development of new methods should therefore be encouraged.


References

Ashcroft R.E. Current epistemological problems in evi-

dence based medicine. Journal of Medical Ethics 2004;

30 (2): 131-135

Baart A. Een theorie van de presentie. Utrecht: Lemma,

2004

Beckerman H., Bouter L.M. Effectiviteit van de fysiothe-

rapie. Een literatuuronderzoek. Maastricht: Rijksuniver-

siteit Maastricht, 1991

Bensing J. Bridging the gap. The separate worlds of

evidence-based medicine and patient-centered medi-

cine. 2000; 39 [1]: 17-25

Biller-Adorno N., Lie, R.K, Ter Meulen, R. Evidence-

Based Medicine as an Instrument for Rational Health

Policy. Health Care Analysis 2002: 10(3): 261-275

College voor Zorgverzekeringen. Pakketbeheer in de

praktijk (2006a). Diemen: CVZ, 245.2006

College voor Zorgverzekeringen. NESS Handmaster

(2006b). Diemen: CVZ, 2006

Commissie Keuzen in de Zorg (Commissie Dunning)

Kiezen en delen. Den Haag, Ministerie van VWS, 1991

Cox K. Evidence Based Practice voor verpleegkundi-

gen: methodiek en implementatie. Utrecht: Lemma,

2004

DSW Zorgverzekeraar, www.dws.nl geraadpleegd 17

mei 2007

Duyvendak J.W. Accountability: de trend. Lezing lus-

trum Verwey-Jonkerinstituut. Utrecht, 2003

Evidence-Based Medicine Working Group. Evidence-

based medicine. A new approach to teaching the prac-

tice of medicine. Evidence-Based Medicine Working

Group. 1992; JAMA 268 (17): 2420-2425

Freeman A. C., Sweeney K. Why general practitioners

do not implement evidence: qualitative study. British

Medical Journal 2001; 323 [7321]: 1100-1102

Gezondheidsraad. Contouren van het basispakket.

2003/02. Den Haag: Gezondheidsraad 2003

Glasziou P. Evidence based medicine: does it make a

difference? Make it evidence informed practice with a

little wisdom. British Medical Journal 2005; 330 (7482):

92

Goldenberg M.J. On evidence and evidence-based

medicine: lessons from the philosophy of science. So-

cial Science 2006; 62 (11): 2621-2632

Goodman K.W. Ethics and evidence-based medicine:

fallibility and responsibility in clinical science. New

York: Cambridge University Press, 2003

Haynes R. B., Devereaux P. J., Guyat G.H. Clinical

expertise in the era of evidence-based medicine and

patient choice. 2002a. ACP Journal Club 2002; 136 [2]:

A11-A14

Haynes R. B., Devereaux P. J., Guyatt G. H. Physi-

cians' and patients' choices in evidence based practice.

2002b. British Medical Journal 2002; 324 [7350]: 1350

Horrobin D.F., Are large clinical trials in rapidly lethal

diseases usually unethical? Lancet 2003:(9358): 695-

697


Jorland G., Opinel A., Weisz G. Body counts. Medical

quantification in historical and sociological perspective.

Montreal: McGill-Queen's University Press, 2005

Kalf H., De Beer J. Evidence Based Logopedie. Logo-

pedisch handelen gebaseerd op wetenschappelijke

evidentie. Houten: Bohn Stafleu Van Loghem, 2004

Kinebanian A., Granse M.l. Grondslagen van de Ergo-

therapie. Maarssen: Elsevier Gezondheidszorg, 2006

Knottnerus A., Dinant G.J. Medicine based evidence, a

prerequisite for evidence based medicine. British Medi-

cal Journal 1997; 315: 1109-10

Kwaliteitsinstituut voor de gezondheidszorg CBO,.

www.cbo.nl/product/richtlijnen/handleiding. Geraadpleegd

14 mei 2007.

Lambert H., Gordon E. J., Bogdan-Lovis, E. A. Introduc-

tion: gift horse or Trojan horse? Social science per-

spectives on evidence-based health care. Social Sci-

ence and Medicine 2006; 62[ 11]: 2613-2620

Mol A. De logica van het zorgen. Amsterdam: Van

Gennep, 2006

Offringa M., Assendelft W.J.J., Scholten R.J.P.M. Inlei-

ding in evidence-based medicine. Klinisch handelen

gebaseerd op bewijsmateriaal. Houten/Antwerpen:

Bohn Stafleu van Loghem 2003

Raad voor de Volksgezondheid en Zorg. Medische

diagnose. Zoetermeer, RVZ 2006

Raad voor de Volksgezondheid en Zorg. Zinnige en

duurzame zorg. Zoetermeer: RVZ, 2006

Raad voor de Volksgezondheid en Zorg. Rechtvaardige

en duurzame zorg. Zoetermeer: RVZ, 2007

Rogers W.A. Is there a tension between doctors' duty of

care and evidence-based medicine? Health Care

Analysis 2002; 10 (3): 277-287

Rogers W.A. Evidence based medicine and justice: a

framework for looking at the impact of EBM upon vul-

nerable or disadvantaged groups. Journal of Medical

Ethics 2004; 30 (2): 141-5

Saarni S.I., Gylling H.A. Evidence based medicine

guidelines: a solution to rationing or politics disguised

as science? Journal of Medical Ethics 2004; 30 (2):

171-5

Sackett D.L., Rosenberg W.M., Gray J.A., Haynes R.B.,

Richardson W.S. Evidence based medicine: what it is

and what it isn't. British Medical Journal 1996; 312

(7023): 71-2

Timmermans S., Berg M. The gold standard. The chal-

lenge of evidence-based medicine and standardization

in health care. Philadelphia, Pa: Temple University

Press, 2003

Van Heijst A. Menslievende zorg. Een ethische kijk op

professionaliteit. Kampen: Klement, 2005

Van Houten D. De standaardmens voorbij. Over zorg,

verzorgingsstaat en burgerschap. Maarssen: Else-

vier/De Tijdstroom, 1999

Van Weel C., Knottnerus, J.A. Evidence-based inter-

ventions and comprehensive treatment. Lancet 1999;

353 (9156): 916-918

Vandenbroucke J.P. Niveaus van bewijskracht schieten

tekort. 2006; 150: 2485

VGZ Zorgverzekeraar. www.vgz.nl geraadpleegd 12 juli

2007

Vliet Vlieland T. P. Managing chronic disease: evi-

dence-based medicine or patient centred medicine?

Health Care Analysis 2002; 10 [3]: 289-298

Vos R., Houtepen R., Horstman K. Evidence-based

medicine and power shifts in health care systems.

Health Care Analysis 2004; 10 (3): 319-328

Vos R., Willems D., Houtepen R. Coordinating the

norms and values of medical research, medical practice

and patient worlds-the ethics of evidence based medi-

cine in orphaned fields of medicine. Journal of Medical

Ethics 2004; 30 (2): 166-170

Wiersma T. Twee eeuwen zoeken naar medische be-

wijsvoering. Amsterdam: Boom, 1999


Other cons ul ted l i terature

Armstrong D. Clinical autonomy, individual and collec-

tive: the problem of changing doctors' behaviour. Social

Science and Medicine 2002; 55 [10]: 1771-7

Ashcroft R., Ter Meulen R. Ethics, philosophy, and

evidence based medicine. Journal of Medical Ethics

2004; 30 (2): 119

Bossuyt P., Kortenray J. Schaatsen op dik ijs. Evidence

based medicine. Amsterdam: Boom, 2001

De Vries R., Lemmens T. The social and cultural shap-

ing of medical evidence: case studies from pharmaceu-

tical research and obstetric science. Social Science and

Medicine 2006; 62 (11): 2694-706

Fischer P.M. Evidentiary medicine lacks humility. Jour-

nal of Family Practice 1999; 48 (5): 345-6

Kulkarni A.V. The challenges of evidence-based medi-

cine: a philosophical perspective. Medicine Health Care

and Philosophy, 2005; 8 (2): 255-60

Mol A. Proving or improving: on health care research as

a form of self-reflection. Qualitative Health Research

2006;

16 (3): 405-414

Sehon S.R., Stanley D.E. A philosophical analysis of

the evidence-based medicine debate. BMC Health Ser-

vices 2003; 3 (1): 14


Appendix 1

Counci l for Publ ic Health and Health Care

Composi t ion o f the Counci l for Public Health and Heal th Care (Dutch acronym:

RVZ)

The activities of the CEG/RVZ fall under the responsibility of the Council for Public Health and Health

Care. The monitoring report Appropriate evidence. Ethical questions concerning the use of evidence in

health care policy was approved by the RVZ during its meeting of 20 September 2007.

CHAIR

M.H. Meijerink

MEMBERS

A.M. van Blerck-Woerdman

H. Bosma

Prof. D.D.M. Braat, vice chair

Prof. W.N.J. Groot

J.M.G. Lanphen, general practitioner

Prof. J.P. Mackenbach

A.A. Westerlaken

Prof. D.L. Willems

GENERAL SECRETARY

P. Vos

Composi t ion o f the Forum of the Centre for Ethics and Hea lth (CEG)/RVZ

The forum has been appointed to support the monitoring task of the CEG/RVZ. The forum functions as a

consultative body and has a monitoring, advisory and initiating role in the realisation of the monitoring

report.

MEMBERS

G. Abrahamse-van den Bosch, policy officer health care, Protestant Christian Association for the

Aged, Zwolle

H. van Dartel, university lecturer in ethics and law of health care, Leiden University Medical Center,

Leiden

Prof. D.P. Engberts, professor of normative aspects of medicine, head Section Ethics and Law of Health

Care, Leiden University Medical Center, Leiden

Prof. G. Glas, visiting professor Christian philosophy in the reformed tradition, and psychiatrist, Leiden

University, Leiden and Zwolse Poort, Zwolle


Prof. M.H.F. Grypdonck, professor of nursing science, University Medical Centre Utrecht

M. Morskieft, Kantel Konsult, Nijmegen

Dr A. Pool, nurse and psychologist, Krimpen aan den IJssel

F.H. Stegehuis, manager health care provision, Menzis Zorg en Inkomen, Zwolle

Prof. M.J. Trappenburg, visiting professor of patient perspective in care, Erasmus University Rotterdam,

Rotterdam

M.I. Verstappen, general director municipal health service, Amsterdam

Prof. G.A.M. Widdershoven, professor of health ethics, Maastricht University, Maastricht

FROM THE RVZ

Prof. I.D. de Beaufort, professor of health ethics, Erasmus Medical Centre, Rotterdam, member RVZ and

chair Forum CEG/RVZ (to 1 January 2007)

Prof. D.L. Willems, professor of medical ethics, Academic Medical Centre, Amsterdam, member RVZ and

chair Forum CEG/RVZ (since 1 January 2007)

J.M.G. Lanphen, general practitioner, member RVZ and vice chair Forum CEG/RVZ

Dr A.J. Struijs, senior advisor and project coordinator CEG/RVZ

G. van Dijk, advisor CEG/RVZ (to 1 January 2006)

I. Doorten, advisor CEG/RVZ

L. Romein, project secretary CEG/RVZ

F.I. de Mooij, temporary secretary CEG/RVZ


Appendix 2

Preparat ion for the moni tor ing repor t Appropr iate evidence

AUTHORS

Prof. D.L. (Dick) Willems, professor of medical ethics, Faculty of General Practice Medicine, Division of

Clinical Methods and Public Health, Academic Medical Centre, Amsterdam

Prof. R. (Rein) Vos, professor of health ethics and philosophy, Faculty of Health Sciences, Maastricht

University, Maastricht

G.G. (Ger) Palmboom, researcher medical ethics section, Academic Medical Centre, Amsterdam

P. (Paulus) Lips, general practitioner and trainee ethicist, Amsterdam

CONSULTATIVE GROUP

Prof G.H Blijham, chair Board of Directors, University Medical Centre Utrecht

Prof. M.H.F. Grypdonck, professor of nursing science, University Medical Centre Utrecht

Prof. R. ter Meulen, professor of ethics in medicine, Director Center for Ethics in Medicine, University

of Bristol

Dr C. Smit, patient organisations VSOP/NPCF

Prof. J.C.J. Dute, professor of health law, Erasmus Medical Centre, Rotterdam

EXPERTS INTERVIEWED

Dr A. Boer, Board of Directors, Care Insurance Board, Diemen

Dr L.G.A. Bonneux, physician/epidemiologist, Netherlands Interdisciplinary Demographic Institute,

The Hague

B.E.M. Burema-Kuster, inspector, Dutch Health Care Inspectorate, Zwolle

A. Eliens, senior policy advisor/nurse, V&VN, professional association of health care professionals,

Utrecht

C.R. Geerincks-Vercamman, researcher social work, Leiden University Medical Center, Leiden

Prof. J.K.M. Gevers, professor of health law, Department of Health Law/Social Medicine,

Academic Medical Centre, Amsterdam

Dr H.G.L.M. Grundmeijer, general practitioner and education coordinator general practice medicine,

Division of Clinical Methods & Public Health, Academic Medical Centre, Amsterdam

Dr C. Hartogh, nursing home physician/researcher Nursing home Naarderheem, Naarden

Dr A. van der Heide, epidemiologist/researcher, Department of Public Health, Erasmus Medical Centre,

Rotterdam

Dr R. Houtepen, university lecturer and philosopher, Section Health Ethics and Philosophy, Maastricht

University, Maastricht

R.B. Jansen, senior policy officer, Ministry of Health, Welfare and Sport, Directorate General Medicines

and Medical Technology, The Hague

Prof. H.G.M. Leufkens, professor of pharmaco-epidemiology, Faculty of Natural Sciences, Utrecht

University, Utrecht

E.L. Lodeweges-Hulscher, inspector/lawyer, Dutch Health Care Inspectorate, Zwolle


Dr R.W.J.G. Ostelo, physiotherapist/epidemiologist, VU University Medical Center/EMGO Institute,

Amsterdam

F.G. Schaafsma, company doctor/researcher Academic Medical Centre, Amsterdam

Dr R.J.P.M. Scholten, physician-epidemiologist, Clinical Epidemiology and Biostatistics, Academic

Medical Centre, Amsterdam

K. van Bezooijen, patient representative and retired general practitioner, Breda

P.C.M. van Dijk, advisory physician Agis and Department of General Practice Medicine, Academic

Medical Centre, Amsterdam

IN 2005 AN INITIAL EXPLORATION TOOK PLACE VIA INTERVIEWS WITH

Prof. W.J.J. Assendelft, professor of general practice medicine, Leiden University Medical Center, Leiden

T.A. van Barneveld, Dutch Institute for Healthcare Improvement (Dutch acronym: CBO)

Prof. L. Bouter, professor of epidemiology EMGO institute, VU University Medical Center/EMGO

Institute, Amsterdam

Prof. MHF Grypdonck, professor of nursing science, University Medical Centre Utrecht

Prof. R. ter Meulen, professor of ethics in medicine, Director Center for Ethics in Medicine, University of

Bristol

Prof. H.G.M. Rooijmans, chair Advisory Council on Health Research (Dutch acronym: RGO)

and emeritus professor of clinical psychiatry, Leiden University Medical Center, Leiden


Appendix 3

Publ ications CEG to date

Reports

2003 ETHICS AND HEALTH REPORTS:

Council for Public Health and Health Care

- Eisend gedrag en agressie van zorgvragers

- Drang en informele dwang in de zorg

- Culturele eigenheid en zelfbeschikking van allochtone zorgvragers

- Zelfbeschikking en eigen verantwoordelijkheid van mensen met een ver-

standelijke handicap

Health Council

- Handelingen met geslachtscellen en embryo’s

- Screening van pasgeborenen op aangeboren stofwisselingsziekten

- Geneesmiddelen voor kinderen

- De maakbare mens


Health Council

- ‘Vruchtbaarheidsverzekering’: medische en niet-medische redenen

- Terminale sedatie

- Bestrijdingsmiddelen, cosmetica, verf: de bescherming van proefpersonen

in blootstellingsonderzoek

- Geavanceerde thuiszorgtechnologie: morele vragen bij een ethisch ideaal


- Intermezzo

- Geavanceerde thuiszorgtechnologie: morele vragen bij een nieuwe zorg-

praktijk

- Mantelzorg, kostenbeheersing en eigen verantwoordelijkheid

- Economisering van zorg en beroepsethiek


Health Council

- Embryonale stamcellen zonder morele pijn?

- Ethische aspecten van kostenutiliteitsanalyse

- Nu met extra bacteriën! Voedingsmiddelen met gezondheidsclaims

Health Council/Council for Public Health and Health Care

- Opsporing verzocht? Screening in de huisartspraktijk


- Zorgverlener èn opsporingsambtenaar?

- Ethiek in zorginstellingen en zorgopleidingen



Health Council/Council for Public Health and Health Care

- Vertrouwen in verantwoorde zorg? Effecten van en morele vragen bij het

gebruik van prestatie-indicatoren

Health Council

– Testen van bloeddonors op variant Creutzfeldt-Jakob?


Health Council

– Overwegingen bij het beëindigen van het leven van pasgeborenen


– Formalisering van informele zorg.

Over de rol van ‘gebruikelijke zorg’ bij toekenning van professionele zorg

– Financiële stimulering van orgaandonatie. Een ethische verkenning

– Passend bewijs. Ethische vragen bij het gebruik van evidence in het zorg

beleid

Background studies

- Economisering van zorg en beroepsethiek, 2004

- Ethiek in zorgopleidingen en zorginstellingen, 2005

Inves t igat ions

- De vertwijfeling van de mantelmeeuw, 2004

- Over keuzevrijheid en kiesplicht, 2006

The CEG is currently translating its publications into English. So far, the following moni-

toring reports are available:

− Considerations pertaining to neonatal life termination (2007)

− Should blood donors be tested for Variant Creutzfeldt-Jakob disease? (2006)

− Embryonic stem cells without moral pain? (2005)

− Advanced home care technology: moral questions associated with an ethical ideal

(2004)

− Terminal sedation (2004)

− Economisation of health care and professional ethics (2004)

− Human enhancement (2003)

− Confidence in responsible care? The effects of performance indicators and moral

issues associated with their use (2006)

− Advanced home care technology: moral issues surrounding a new healthcare practice

(2004)

These publications can be downloaded via www.ceg.nl


Appendix 4

Acron yms

AWBZ Exceptional Medical Expenses Act

CBO Dutch Institute for Healthcare Improvement

CEG Centre for Ethics and Health

CVZ Care Insurance Board

EBM Evidence-Based Medicine

EBP Evidence-Based Practice

ES Electrostimulation

MRC (British) Medical Research Council

NHG Dutch College of General Practitioners

RCT Randomised Controlled Trial

RVZ Council for Public Health and Health Care

VGN Netherlands Association for Care for the Disabled

VWS Ministry of Health, Welfare and Sport

UMC University Medical Centre

Bijlagen 49

appropriate evidence website - ceg

Documents