12 evidence based period ontology systematic reviews
Post on 06-Apr-2018
214 Views
Preview:
TRANSCRIPT
-
8/3/2019 12 Evidence Based Period Ontology Systematic Reviews
1/17
Evidence-based periodontology,systematic reviews and research
qualityIA N N E E D L E M A N, DAVI D R. M O L E S & HE L E N WO R T H I N G T O N
Periodontology has a rich background of research
and scholarship. A simple MEDLINE search of Peri-
odontal Diseases OR Periodontitis alone from 1966
to 2003 brings up more than 45,000 hits. Therefore,efficient use of this wealth of research data needs to
be a part of periodontal practice. Evidence-based
periodontology aims to facilitate such an approach,
accelerating the introduction of the best research into
patient care.
This chapter will review the concepts of evidence-
based periodontology, introduce the systematic
review as a research tool and examine how evidence-
based periodontology can both inform on and benefit
healthcare in periodontology. Finally, we will exam-
ine the strengths and limitations of different research
designs and their appraisal. We hope that the infor-
mation in this chapter will provide a basic under-
standing of the concepts that will be relevant to
reading and enjoying the other chapters in this vol-
ume of Periodontology 2000.
What is evidence-basedperiodontology?
Evidence-based periodontology is the application of
evidence-based health care to periodontology. Auseful definition of evidence-based health care has
been proposed by Muir Gray: An approach to
decision making in which the clinician uses the
best evidence available, in consultation with the
patient, to decide upon the option which suits that
patient best (8). Therefore, evidence-based period-
ontology is a tool to support decision making and
integrating the best evidence available with clinical
practice.
The highest quality evidence will be used if it exists,
but if it does not, lower levels of evidence will be
considered. Lower levels of evidence usually means
research designs more prone to bias and thereforewith less reliable data. However, the nature, strengths
and weaknesses of the evidence will be made clear to
the reader. In addition, wherever possible, the data
presentation supplies more clinically relevant infor-
mation, including the probability of achieving a cer-
tain effect such as a benefit, and considering possible
adverse effects.
What evidence-based
periodontology is notEvidence-based periodontology is not simply sys-
tematic reviews of randomized controlled trials,
although this can be an important aspect. Evidence-
based periodontology is an approach to patient-care
and nothing more. The expectations that are some-
times laid on it can be inappropriate. It cannot pro-
vide answers if research data do not exist (other than
using expert opinion) and it cannot substitute for
highly developed clinical skills. Therefore, it can
never be cookbook healthcare or use statistics in
isolation to drive clinical care. Instead it is the com-prehensive integration of appropriate research evi-
dence, patient preference and clinical expertise
(Fig. 1).
This can be illustrated with data from a recent
systematic review on periodontal plastic surgery for
root surface coverage in localized Miller Class I and II
defects (25). The data from the systematic review
demonstrated that connective tissue grafts were sig-
nificantly better than guided tissue regeneration in
12
Periodontology 2000, Vol. 37, 2005, 1228
Printed in Denmark. All rights reserved
Copyright Blackwell Munksgaard 2005
PERIODONTOLOGY 2000
-
8/3/2019 12 Evidence Based Period Ontology Systematic Reviews
2/17
reducing recession (mean difference 0.43 mm,
95%CI [0.62,0.23], chi square for heterogeneity 7.8
(df 5) P 0.17). This indicates that the pooled
difference between six studies included in the review
is 0.43 mm, with a 95% confidence interval from 0.62
to 0.23. The chi-square test indicates that there is no
evidence of any heterogeneity between the studies
(they could theoretically all be measuring the samedifference). So, does this mean that only connective
tissue grafts should be used in the treatment of
localized recession defects? Clearly, this would not be
appropriate. The data show that both GTR and con-
nective tissue grafts can work. For the selected out-
come, which was recession reduction, connective
tissue grafts produce 0.43 mm greater effect; the
result is both reasonably precise (judged by the
confidence interval) and the studies from which
the data were taken were similar (no evidence of
heterogeneity).
However, recession reduction might not be theonly outcome of interest. The two surgical proce-
dures are very different. One requires the harvesting
of a soft tissue graft from the palate and the other
does not. There are no data available examining
patient preferences, but it is likely that some indi-
viduals will prefer a procedure that does not involve
two surgical sites, even if it does not reduce
recession to the same extent. It is also possible that
aesthetics are different following the two procedures
and this might inform on the decision. However,
surprisingly, no data are available on patient views
on aesthetics comparing the two procedures.
Therefore, this evidence-based approach to man-
agement of recession has produced the best avail-
able evidence, shown how precise this estimate
actually is, and highlighted the limitations of the
evidence, in this case the lack of data on some
outcomes that are relevant to the decision makingprocess.
Clinical relevance
One of the barriers to the application of research
findings in clinical practice is the way that results are
often presented. Typically, a mean value will be
published, based on a statistical analysis comparing
experimental groups. Such a value in conjunction
with its associated 95% confidence interval is useful
to determine whether there is a statistically signifi-
cant difference between groups and will often be a
requirement of a study designed for regulatory
approval. However, this type of analysis is not
designed to provide information about the probabil-
ity of achieving a certain outcome were the reader to
apply it in practice. Such an outcome could include
achieving a health benefit or preventing further dis-
ease. For instance, in a meta-analysis from a sys-
tematic review on guided tissue regeneration (GTR)
for periodontal infrabony defects, the additional
benefit of using GTR over access flap surgery was a1.1 mm gain in clinical attachment (21, 22). This
should, however, not be interpreted as the additional
benefit to be expected every time that GTR is used
instead of access flap surgery.
One approach to analysing and presenting data in a
more clinically useful format is to calculate the
number needed to treat (NNT). This is the number of
patients that would need to be treated to achieve a
stated benefit (NNTb) or to avoid a stated harm
(NNTh). It is derived from a dichotomous outcome
such as the proportion of sites achieving at least
2 mm gain in attachment. For the GTR meta-analysis,and using this benefit, the NNTb is eight. In other
words, for every eight patients treated with GTR, you
can expect one to have at least 2 mm more gain in
clinical attachment than if you had used an access
flap (95% confidence interval [4,33]). For detailed
guidance regarding the use and calculation of
the NNT the reader is recommended to the elec-
tronic journal Bandolier: http://www.jr2.ox.ac.uk/
bandolier/booth/painpag/NNTstuff/numeric.htm.
Cliniciansskills
Best evidenceavailable
Patientpreferencesand views
Evidence-basedperiodontology
Fig. 1. How evidence-based periodontology fits into
healthcare. Reproduced with permission from Clarkson,
J, Harrison, JE, Ismail, AI, Needleman, IG, Worthington, H,eds. Evidence Based Dentistry for Effective Practice.
London: Martin Dunitz, 2003 (20).
13
Evidence-based periodontology
-
8/3/2019 12 Evidence Based Period Ontology Systematic Reviews
3/17
Evidence-based periodontology vs.traditional periodontology
High quality research and the use of evidence are
fundamental to both evidence-based periodontology
and traditional periodontology. The differences
between these approaches emanate from how
research informs clinical practice. Evidence-basedperiodontology uses a more transparent approach to
acknowledge both the strengths and the limitations
of the evidence. An appreciation of the level of
uncertainty or imprecision of the data is essential in
order to offer choices to the patient regarding
treatment options. Evidence-based periodontology
also attempts to gather all available data and to
minimize bias in summarizing the data. These
aspects are key to decision making and are high-
lighted in Table 1.
Furthermore, evidence-based periodontology
acknowledges explicitly the type or level of research
on which conclusions are drawn. The research hier-
archy is discussed in more detail later in this chapter.
However, one aspect that influences the reliability of
the data is the control of bias. Bias is a collective term
for factors that systematically distort the results of
research away from the truth. Different research
designs offer different possibilities for the control of
bias and therefore vary in their reliability and will be
discussed further below.
The components of evidence-basedperiodontology
An overview of the components is given in Fig. 2.
Evidence-based periodontology starts with therecognition of a knowledge gap. From the know-
ledge gap comes a focussed question that leads on
to a search for relevant information. Once the rele-
vant information is located, the validity of the
research needs to be considered in two broad areas.
Firstly, is the science good (internal validity)? Inter-
nal validity focuses on the methodology of research.
Secondly, can the findings be generalized outside of
the study (external validity)? External validity might
be affected by the way treatment was performed. For
instance, if the time spent on treatment was exten-
sive it might not be practical to provide this therapy
outside of a research study. Another example could
relate to the use of many specific inclusion criteria
in a trial which could make it difficult to generalize
the findings to a wider group of patients. The
question the reader should ask is whether their types
of patients are so different from the study that it is
Table 1. Comparison of evidence-based periodon-
tology vs. traditional periodontologyEvidence-based
periodontology
Traditional
periodontology
Similarities
High value of clinical
skills and experience
Fundamental importance
of integrating evidence
with patient values
Differences
Uses best evidence
available
Systematic appraisal
of quality of evidence
More objective, more
transparent and less
biased process
Greater acceptance
of levels of uncertainty
Unclear basis
of evidence
Unclear or absent
appraisal of quality
of evidence
More subjective, more
opaque and more
biased process
Greater tendency to
black and white
conclusions
Reproduced with permission from Clarkson, J, Harrison, JE, Ismail, AI,Needleman, IG, Worthington, H, eds. Evidence Based Dentistry for EffectivePractice. London: Martin Dunitz, 2003 (20).
Reject
if invalid
or poor
Evaluate the effects
Integrate into practice
Evaluate the evidence
Search for evidence
Develop into a focussed question
Recognize clinical knowledge gap
Fig. 2. The steps of evidence-based periodontology.
Reproduced with permission from Clarkson, J, Harrison,
JE, Ismail, AI, Needleman, IG, Worthington, H, eds. Evi-
dence Based Dentistry for Effective Practice. London:
Martin Dunitz, 2003 (20).
14
Needleman et al.
-
8/3/2019 12 Evidence Based Period Ontology Systematic Reviews
4/17
reasonable to expect differences in outcomes. After
locating and appraising the research, the results
then need to be applied clinically, or at least inclu-
ded in a range of options. Finally, the results in
clinical practice need to be evaluated to reveal
whether the adopted technique achieved the
expected outcome.
The example of gingival recession mentioned ear-
lier can be used to illustrate this approach. Theuncertainty might relate to whether to change from
using connective tissue grafts for recession defects to
guided tissue regeneration and can be translated into
a focussed question. Here the patient or problem
group could be refined more closely to localized
recession defects and perhaps Miller Class I or II, as
we might reasonably expect these lesions to respond
differently from more advanced lesions. The inter-
vention is guided tissue regeneration and the com-
parison, connective tissue grafts. The outcomes would
include change in recession or possibly the chance of
achieving complete root coverage. Since the proce-
dure is primarily for aesthetics, a patient-centred
assessment of aesthetics should be an outcome. As
always, there must be a consideration of adverse
effects and these might include pain, postoperative
infection, and severe bleeding postoperatively.
Reassembling this structure into a focussed
question would lead to In patients with localized
Miller Class I or II recession defects, what is the
effect of guided tissue regeneration vs. connective
tissue grafts on change in recession, chance of
complete defect coverage and aesthetics and whatare the adverse effects? For this particular research
question, the randomized controlled trial is best able
to address the change in recession outcome. For the
other outcomes, other research designs might have
been used, such as observational studies. Preferably,
we would like to find a systematic review that will
have completed the searching and study appraisal
for us.
The search quickly identifies a systematic review
(25). The review has a research question that is
appropriate to our question and demonstrates a
statistically superior effect of connective tissue graftscompared with guided tissue regeneration. The
review also acknowledges certain limitations. In
terms of the validity of the meta-analysis, the
reviewers urge caution as publication bias could be
affecting the overall result, but this could not be
tested due to the low number of studies. Publication
bias is discussed later in this chapter. Another limi-
tation was that there were no data on aesthetics or
adverse effects.
Therefore, having reviewed the data, it is clear that
there is good evidence to indicate that connective
tissue grafts have a greater effect on change in
recession than guided tissue regeneration, although
there are several limitations to this evidence. Clinical
recommendation is tempered by the lack of data on
aesthetics and adverse effects and the possible
exaggeration of benefit through publication bias. This
information can then inform on the case presenta-tion to the patient and a choice of options discussed
and agreed. The outcome of treatment can then be
evaluated to see whether the desired endpoint was
achieved and this helps to refine the case presenta-
tion discussion in future.
Systematic reviews
One important element of evidence-based period-
ontology is the systematic review. Systematic reviews
are a research design termed research synthesis.
That is, they use research methodology to pool data
from multiple studies that address a particular
hypothesis. A systematic review can be defined as a
review of a clearly formulated question that attempts
to minimize bias using systematic and explicit
methods to identify, select, critically appraise and
summarize relevant research.
The description of systematic reviews as providing
the highest level of evidence is widespread but also
raises expectations that may or may not be fulfilled. A
realistic understanding of what a systematic reviewcan provide is important for the appropriate use of
this type of evidence (Table 2). More detailed infor-
mation on systematic reviews exist (5, 19), and guides
to conducting them are freely available (1, 13).
As with all research, a systematic review starts from
an hypothesis. This is derived from a focussed ques-
tion which is set to answer a particular area of
uncertainty. For instance, for the systematic review
on smoking and periodontal therapy in the chapter by
Labriola et al. in this volume, the focussed question
was: In patients with chronic periodontitis, what is
the effect of smoking or smoking cessation on theresponse to nonsurgical periodontal therapy in terms
of clinical and patient-centred outcomes?(14). The
question has set the types of patients (individuals
with chronic periodontitis undergoing nonsurgical
therapy), type of exposure (cigarette smoking) and
types of outcomes (clinical and patient-centred) to be
investigated, each aspect being defined in more detail
within the protocol. As this is a prognostic research
question, where exposure (smoking) cannot be
15
Evidence-based periodontology
-
8/3/2019 12 Evidence Based Period Ontology Systematic Reviews
5/17
randomized, the cohort study is the research design
of choice to incorporate into this investigation.
These components help in the design of the search
strategy that aims to be comprehensive. Usually,
searching of multiple electronic databases is carried
out together with searching other sources. The most
commonly searched databases include MEDLINE
(strong on English-language studies), EMBASE(strong on other European languages), and CENTRAL
(the Cochrane Collaboration register of trials
records). Searching only electronic databases can
miss important data, as records on the database may
not be appropriately coded. To supplement the
electronic search, other approaches are used.
Typically, this will include checking for publications
in the bibliographies of retrieved studies and review
articles, hand-searching of journals for missed
reports, and contacting researchers, industry and
journals for unpublished data.
The search strategy aims for high sensitivity, i.e. the
greatest chance of finding all relevant studies. The
downside of this approach is low precision, i.e. in
addition to the relevant studies, the search will
identify many irrelevant hits (probably more than
90% of hits from the search will not be relevant). Forexample, in a systematic review on systemic anti-
microbials, the search identified 1,300 hits. Screening
of the title and abstracts (if available) indicated that
158 papers might be relevant. Once the full text of the
studies had been reviewed, 25 trials were judged
relevant and could be included (9). At first sight,
rejecting 1,275/1,300 studies would appear to be
wasting potentially useful data. However, the delib-
erately inclusive search identifies a large number of
Table 2. The potential of systematic reviews
What a high quality systematic review can do
Find and summarize all available studies.
A comprehensive search will identify all relevant studies up to the point of the date of the completion of the search.
This should give the reader greater confidence that bias in selecting studies has been minimized.
Provide an objective assessment of the quality or research and in particular the degree of protection from bias within
the original studies.
Components of methodological quality that have evidence of affecting bias can be evaluated. It may be much harder
to evaluate the impact of otherquality
issues if there is no consensus on how to measure them. An example could be
the quality of the treatment provided.
Estimate research effects across multiple studies with meta-analysis.
Meta-analysis is valid only if studies are similar in their research question and design.
Meta-analysis can estimate uncertainty and precision of the effect.
Meta-analysis may generate hypotheses for differential effects across subgroups of the population tested.
If the effect is consistent across multiple studies (with small differences in design), then it may more readily possible to
generalise the results to clinical practice than the results from a single study.
Overcome limitations of underpowered studies in detecting a true difference if such a true difference really exists.
What a high quality systematic review cannot do
It cannot be used in isolation to dictate clinical practice.
It is a synthesis of available research and must be used in context with clinical judgement and patient preference.
Produce strong conclusions if the research base is weak in quality.
The value of the review will then be to present a comprehensive objective summary of the strength of the data and toidentify the design of research to answer important gaps.
Overcome limitations of narrowly designed clinical research.
If the clinical studies only investigate the effect of an intervention in highly selected individuals, the conclusions
cannot be generalised outside of these conditions. However, the objective communication of any limitations in the
research base will help to set the degree of uncertainty and indicate the priorities for future research.
Exclude relevant studies.
Although the majority of hits from the search will be excluded, this is due to the deliberate strategy of achieving high
sensitivity (likelihood of finding all relevant studies) but low precision (likelihood of only finding relevant studies).
Therefore, it is common to find that more than 90% of the search records are totally irrelevant to the question and
must be excluded. The alternative approach of aiming for high precision also carries a high risk of missing relevant
studies, although it will appear as if few studies are being excluded.
Be a miracle research design.
All research has strengths and limitations/weaknesses. Systematic reviews are no different from other research
designs in this respect.
16
Needleman et al.
-
8/3/2019 12 Evidence Based Period Ontology Systematic Reviews
6/17
irrelevant papers, including veterinary medicine,
review papers, duplicate reporting of research and
laboratory studies. The systematic review screens the
search findings against prestated criteria. These cri-
teria aim to exclude studies irrelevant to answering
the question, but do not attempt to exclude on the
basis of the quality of the study. Instead, the quality of
relevant studies is critically appraised using objective
criteria that could influence the study outcome.The dimension of quality can be incorporated into
a systematic review in a number of ways. If the
studies are similar enough to be combined in a meta-
analysis, the impact of quality on the overall result
can be estimated (through sensitivity analyses or
meta-regression). If meta-analysis is not possible, the
quality of studies can be summarized in narrative
tables, in particular for those elements designed to
protect against bias. Whilst this may not be as
powerful as the use of meta-analysis, it will highlight
limitations to be placed upon the conclusions. Fol-
lowing pooling of the data with meta-analysis or
qualitative methods, the conclusions from the
investigation can be drawn and related to the data
derived from the review.
The systematic review will not be appropriate for
some questions. For instance, to address the ques-
tion, Which indices have been used to measure gin-
givitis? a descriptive survey will be more appropriate.
However, systematic methods should be adopted for
some aspects, in particular to ensure that the search
is both comprehensive and contemporary. This
might form an important initial stage to answering aquestion such as Which gingivitis indices have been
validated? This is a research question answerable by
a systematic review.
The development of evidence-based periodontology
Evidence-based periodontology is built upon devel-
opments in clinical research design throughout the
18th, 19th and 20th centuries (15, 20, 23, 28).
Evidence-based medicine has only been known for just over a decade and the term was coined by the
clinical epidemiology group at McMaster University
in Canada (4).
The influence of the McMaster group spread far.
One of the earliest to take up the challenge in peri-
odontology (in fact in oral health research overall)
was Alexia Antczak Bouckoms in Boston, USA. Ant-
czak Bouckoms and colleagues challenged the
methods and quality of periodontal clinical research
in the mid 1980s (3) and set up an Oral Health Group
as part of the Cochrane Collaboration in 1994. The
editorial base of the Oral Health group subsequently
moved to Manchester University in 1997 with Bill
Shaw and Helen Worthington as co-ordinating edi-
tors (http://www.cochrane-oral.man.ac.uk/). The
first Cochrane systematic review in periodontology
was published in 2001 and researched the effect of
guided tissue regeneration for infrabony defects (21).Many individuals have been active in the critical
analysis of the periodontal literature. These include
Jan Egelberg, Loma Linda University, Noel Claffey,
Trinity College Dublin, and Gary Greenstein, Uni-
versity of Medicine and Dentistry of New Jersey.
There have been many notable events in evidence-
based periodontology. The 1996 World Workshop in
Periodontology held by the American Academy of
Periodontology included elements of evidence-based
healthcare, supported by Michael Newman at UCLA
(2). The 2002 European Workshop on Periodontology
became the first international workshop to use rig-
orous systematic reviews to inform the consensus.
The workshop was organized by the European Acad-
emy of Periodontology for the European Federation of
Periodontology, under the chairmanship of Professor
Klaus Lang. Sixteen focussed and rigorous systematic
reviews formed the basis of intense consensus dis-
cussions. A similar approach was used subsequently
by the American Academy of Periodontology for the
Contemporary Science Workshop in 2003.
Many other groups are now using similar methods
in healthcare and research. Most recently, the Inter-national Center for Evidence-Based Oral Health was
launched in 2003 (http://www.eastman.ucl.ac.uk/
iceboh) to produce high quality evidence-based
research with an emphasis on, but not limited to,
periodontology and implants and to provide generic
training in systematic reviews and research methods.
Study designs and critical appraisal
Different study designsDifferent clinical research questions require evalua-
tion through different study designs. A study to
determine the effectiveness of surgical therapy com-
pared with nonsurgical debridement deals with the
effectiveness of a treatment option and would be best
answered by a randomized controlled trial (RCT) or,
ideally, a systematic review of RCTs. However, it must
be noted that although RCTs and systematic reviews
of RCTs may well be the gold standard upon which
17
Evidence-based periodontology
-
8/3/2019 12 Evidence Based Period Ontology Systematic Reviews
7/17
to base decisions on the effectiveness of interven-
tions, they are not necessarily appropriate, or ethical,
to answer all questions. An RCT would obviously not
be helpful in answering the question posed on the
epidemiological evidence of plaque in the etiology of
periodontitis. For such questions regarding prognosis
or etiology, cohort studies would be more appropri-
ate. Table 3 illustrates the types of study designs
most suitable for different types of research questionsarising in periodontology. The most appropriate
source of information will depend upon the type of
study design being sought.
Critical appraisal: Why, what and how?
Why critically appraise?
Evidence-based periodontology, as its name implies,
is periodontology that is based on evidence, but not
just any so-called evidence. Richards wrote a toolbox
article for the journal Evidence-based Dentistry enti-
tled Not all evidence is created equal (24). We have
already seen in this chapter that the quality of evi-
dence may vary according to study design and that
this has led to the concept that there can be a hier-
archy of evidence. One hierarchy is illustrated in
Table 4 and is specific to studies on therapy, pre-
vention, etiology, and harm. Other suggested levels
for different types of research question can be found
at the Center for Evidence-Based Medicine: (http://
www.cebm.net/levels_of_evidence.asp#levels).
The publication of research in a high-rankingjournal may not be an absolute guarantee of quality.
Within the medical literature there are methodolo-
gical studies which have empirically shown that
quality is not merely a hypothetical concept but also
affects study outcomes. As examples of this, the re-
views of Schulz et al. (26), Moher et al. (17) and Juni
et al. (12) showed that in studies in which there was
inadequate concealment of treatment allocation, the
treatment effects were exaggerated by about 40%
compared to trials of higher quality.
Quality assessment of trials in periodontology and
implantology
Two recent studies have investigated the methodo-
logical quality of RCTs in periodontology and
implantology as assessed by their publications. Both
studies targeted RCTs for investigation due to the
importance of the RCT in providing evidence for
the effect of interventions and also because of the
empiric data indicating the effect of key domains of
methodology on bias.
Montenegro et al. (18) conducted a systematic
review of the quality of RCTs of periodontal ther-
apy, published in Journal of Periodontology, Journal
of Clinical Periodontology or Journal of Periodontal
research over a 3-year period from 1996 to 1998.
From the electronic search, 283 papers were poss-
ibly relevant and 177 studies met the inclusion
criteria of being an RCT, performed on humans and
for which a full text article was available. Screeningand data abstraction were performed independently
and in duplicate to minimise error and bias. The
evaluation was not performed blind to author
affiliation identity of the RCTs as the evidence
suggests that this has a minimal impact on out-
come (16). In view of the empirical data described
above, the quality components chosen were those
demonstrated to be important for protection from
bias: adequacy of method of generation of the
random sequence, adequacy of method of con-
cealing the allocation sequence from the patient
recruitment, examiner blinding (where it was
judged possible to achieve), and handling of losses
and withdrawals.
The results indicated that 29/177 (17%) of RCTs
employed a clearly adequate method of generating
the random number sequence, and that 12/177 (7%)
of studies described adequate allocation conceal-
ment (Fig. 3). Furthermore, where examiner blinding
was possible, 97/177 (55%) of studies reported an
adequate method. Clear accounting for study sub-
jects was present in 100/177 (55%) of reports. Since
the study was conducted on trial reports, it is notclear how much of the inadequacy was due to
incomplete reporting rather than inadequate study
methods. If the data do reflect study conduct, then
bias and exaggeration of the effect of the test inter-
ventions could be a problem with some trials in
periodontology.
Similar results were found when investigating
RCTs of oral implants (6). This study searched for
RCTs up to the end of 1999 in multiple databases.
Seventy-four publications were located and 43 RCTs
were quality assessed as many studies were pre-
sented in multiple publications. Although themethods and criteria were a little different for this
study compared with the quality appraisal of peri-
odontal studies, the results are broadly comparable.
A clearly adequate method of randomization/con-
cealment of allocation was present in 1/43 (2%)
papers. Blinding was described in 12/43 (28%)
studies and the reasons for withdrawals and losses
to follow-up were specified in 33/43 (77%) of
reports.
18
Needleman et al.
-
8/3/2019 12 Evidence Based Period Ontology Systematic Reviews
8/17
Table3.Studydesignsand
thetypesofquestionstheyaddress
Definitionofstudydesign
Usedfor(examplesgiveninitalics)
Experimentalstudies
Randomized-controlledtrial:parallelgroupdesignagroupof
participants(orotherunitofanalysis,e.g.teeth)israndomizedinto
differenttreatmentgroups.Thesegroupsarefollowedupforthe
outcomesofinterest
Randomized-controlledtrial:split-mouthdesigneachpatientis
his/herowncontrol.Apairofsimilarteeth,orgroupsofteeth
(quadrants),maybeselected
andrandomlyallocatedtodifferent
treatmentgroups.
Non-randomizedcontrolledtrialallocationofparticipantsunder
thecontroloftheinvestigator
,butthemethodfallsshortof
genuinerandomization.
Evaluatingtheeffec
tivenessofanintervention
Randomizedcontrolledtrialcomparingtheeffectivenessofsurgicaltherapy
andnonsurgicaldebridement.
Controlledtrialcom
paringtwomethodsoftreatingperiod
ontalintrabony
defectsusingpairso
fsiteswheretheLHSisalwaysgroupAandtheRHSgroupB.
Observationalstudies
Cohort:alongitudinalstudy,identifyinggroupsofparticipants
accordingtotheirexposure/interventionstatus.Groupsarefollowed
forwardintimetomeasureth
edevelopmentofdifferentoutcomes.
Case-Control:astudythatide
ntifiesgroupsofparticipants
accordingtotheirdisease/outcomestatus.Groupsareinvestigated/
questionedtodeterminetheirexposurestatus
Cross-sectional:astudy(survey)undertakenonadefinedpopulatio
n
atasinglepointintime(snap-shot).Subjectsareobservedonjust
oneoccasionandarenotfollowedup.
Measuringtheincidenceofadisease;lookingatthecaus
esofdisease;
determiningprognosis.
Cohortstudylookin
gattheprogressofperiodontitisovertimeandrelatingthisto
externalfactorssuc
hassmokingorplaque.
Identifyingpotentia
lriskfactorsforadisease;lookingatthepossiblecausesof
disease.
Case-controlstudyl
ookingattheprevalenceofperiodontitisandrelatingthisto
factorssuchasgene
ticmarkers.
Measuringtheprev
alenceofadiseaseorriskfactorinadefinedpopulationata
specifictime.
Across-sectionalstu
dytodeterminethecurrentperiodontaltreatmentneedsin
aspecificpopulation.
19
Evidence-based periodontology
-
8/3/2019 12 Evidence Based Period Ontology Systematic Reviews
9/17
Improving the quality of reporting ofclinical research in periodontology
The adequacy of reporting of clinical research is
crucial if the reader is to evaluate the quality and
possible impact of studies. The importance of sev-
eral of the quality issues that we have described has
not been thoroughly appreciated until relatively
recently. Therefore, it is unfair to judge the past
from the standpoint of current knowledge. In
addition, the pressure on page numbers in paper-
based journals can restrict detail. Hopefully this
aspect will be alleviated by initiatives in electronic
publication.
Guidelines are available to help the publication of
clinical research. These guidelines are well accepted
by high impact biomedical journals and offer guid-
ance not only to authors but also to editors and
reviewers. These guidelines include CONSORT(Consolidated Standards of Reporting Trials) for
reporting randomized controlled trials and STARD
(Standards for Reporting of Diagnostic Accuracy) for
reporting studies on diagnostic tests (http://consort-
statement.org/). In addition, three guidelines for
reporting systematic reviews are available: QUOROM
(Quality of Reporting of Meta-analyses) (http://
consort-statement.org/), MOOSE (Meta-analysis Of
Observational Studies in Epidemiology) (27), and
QUADAS (Quality Assessment of studies of Diagnos-
tic Accuracy included in Systematic reviews) (29). For
clarification, it should be remembered that system-atic reviews are termed meta-analyses by some in
North America, whereas the term meta-analysis is
usually reserved only for the statistical combining of
data which may or may not be part of a systematic
review.
The format of these guidelines is similar. Each
presents a checklist of items for incorporation into
the research report. The selection of items is evi-
dence-based as far as possible and otherwise derived
Table 4. Center for Evidence-Based Medicine hier-archy of evidence for studies on therapy, prevention,etiology or harm (http://www.cebm.net/levels_of_evidence.asp#levels)
Level Type of evidence
Ia Systematic review (with homogeneity*) of
randomized controlled trials (RCT).
1b Individual RCT (with narrow confidenceinterval, see notes below).
2a Systematic review (with homogeneity*) of
cohort studies.
2b Individual cohort study (including low quality
RCT; e.g. < 80% follow-up).
2c Outcomes research; Ecological studies.
3a Systematic review (with homogeneity*) of
case-control studies.
3b Individual case-control study.
4 Case-series (and poor quality cohort and
case-control studies)
5 Expert opinion without explicit critical
appraisal, or based on physiology, bench
research or first principles.
Users can add a minus-sign to denote the level of thatfails to provide a conclusive answer because of:
EITHER a single result with a wide Confidence Interval(such that, for example, an absolute risk reduction in anRCT is not statistically significant but whose confidenceintervals fail to exclude clinically important benefit orharm);
OR a Systematic Review with troublesome (and statisti-cally significant) heterogeneity.
Such evidence is inconclusive.*A systematic review that is free of worrisome variations(heterogeneity) in the directions and degrees of resultsbetween individual studies. Not all systematic reviews
with statistically significant heterogeneity need be wor-risome, and not all worrisome heterogeneity need bestatistically significant. As noted above, studies display-ing worrisome heterogeneity should be tagged with a at the end of their designated level.
Poor quality cohort study: one that failed to clearly de-fine comparison groups and/or failed to measure expo-sures and outcomes in the same (preferably blinded),objective way in both exposed and nonexposed individ-uals and/or failed to identify or appropriately controlknown confounders and/or failed to carry out a suffi-ciently long and complete follow-up of patients. Poor
quality case-controlstudy:one that failedto clearly definecomparison groups and/or failed to measure exposuresand outcomes in the same (preferably blinded), objective
way in both cases and controls and/or failed to identify orappropriately control known confounders.
0%
10%
20%
30%
40%
50%
60%
Randomization
method
Allocation
concealment
Examiner blinding Accounting for all
subjects
Fig. 3. Quality of reporting of randomized controlled tri-
als in periodontology (18). Percentage of studies with
adequate method.
20
Needleman et al.
-
8/3/2019 12 Evidence Based Period Ontology Systematic Reviews
10/17
(a)
PAPERSECTIONAnd topic
Item Description Reportedon
Page #TITLE &
ABSTRACT1 How participants were allocated to interventions
(e.g., "random allocation", "randomized", or"randomly assigned").
INTRODUCTIONBackground
2 Scientific background and explanation of rationale.
METHODSParticipants
3 Eligibility criteria for participants and the settingsand locations where the data were collected.
Interventions 4 Precise details of the interventions intended foreach group and how and when they were actuallyadministered.
Objectives 5 Specific objectives and hypotheses.Outcomes 6 Clearly defined primary and secondary outcome
measures and, when applicable, any methodsused to enhance the quality of measurements(e.g., multiple observations, training of assessors).
Sample size 7 How sample size was determined and, whenapplicable, explanation of any interim analysesand stopping rules.
Randomization --sequencegeneration
8 Method used to generate the random allocationsequence, including details of any restrictions(e.g., blocking, stratification).
Randomization --allocation
concealment
9 Method used to implement the random allocationsequence (e.g., numbered containers or centraltelephone), clarifying whether the sequence was
concealed until interventions were assigned.Randomization --Implementation
10 Who generated the allocation sequence, whoenrolled participants, and who assignedparticipants to their groups.
Blinding(masking)
11 Whether or not participants, those administeringthe interventions, and those assessing theoutcomes were blinded to group assignment.When relevant, how the success of blinding wasevaluated.
Statisticalmethods
12 Statistical methods used to compare groups forprimary outcome(s). Methods for additionalanalyses, such as subgroup analyses andadjusted analyses.
RESULTS
Participant flow
13 Flow of participants through each stage (adiagram is strongly recommended). Specifically,for each group report the numbers of participantsrandomly assigned, receiving intended treatment,completing the study protocol, and analyzed for
the primary outcome. Describe protocol deviationsfrom study as planned, together with reasons.
Recrui tment 14 Dates def ining the periods of recrui tment andfollow-up.
Baseline data 15 Baseline demographic and clinical characteristicsof each group.
Numbersanalyzed
16 Number of participants (denominator) in eachgroup included in each analysis and whether theanalysis was by intention-to-treat. State theresults in absolute numbers when feasible (e.g.,10/20, not 50%).
Outcomes andestimation
17 For each primary and secondary outcome, asummary of results for each group, and theestimated effect size and its precision (e.g. 95%confidence interval).
Ancillaryanalyses
18 Address multiplicity by reporting any otheranalyses performed, including subgroup analyses
and adjusted analyses, indicating those pre-specified and those exploratory.Adverse events 19 All important adverse events or side effects in
each intervention group.DISCUSSIONInterpretation
20 Interpretation of the results, taking into accountstudy hypotheses, sources of potential bias orimprecision and the dangers associated withmultiplicity of analyses and outcomes.
Generalisabil ity 21 Generalisabil ity (external validity) of the trialfindings.
Overall evidence 22 General interpretation of the results in the contextof current evidence.
Fig. 4. a) CONSORT Checklist of items to include when reporting a randomized trial. b) CONSORT Flow chart. Available
from: http://www.consort-statement.org/.
21
Evidence-based periodontology
-
8/3/2019 12 Evidence Based Period Ontology Systematic Reviews
11/17
by a Delphi approach to consensus. In addition to the
checklist, a chart is used to illustrate the flow of pa-
tients through the study. The checklist and chart for
CONSORT are illustrated in Fig. 4a, b. The checklist
should accompany the manuscript in its journal
submission but not be part of the final paper. The
intention with the chart, however, is that it should be
published as part of the paper. Whilst the checklist
has numerous items, each can be concisely ad-
dressed and is unlikely to be the main cause forexcessive length of a publication.
At the time of writing, oral health journals that have
adopted CONSORT as editorial policy and their dates
of adoption are: British Dental Journal(1999), Journal
of Orthodontics (2000), International Journal of End-
odontics(2003) and Journal of Dental Research (2004).
The British Dental Journal is the only one that has
adopted QUOROM (2002).
What should be appraised?
Given that some evidence is better than other evi-dence, it seems reasonable to place greater emphasis
on good than on poor quality evidence when mak-
ing clinical decisions. The problem arises as to how
exactly we decide what constitutes good quality evi-
dence. This process is critical appraisal. The validity of
published evidence is potentially affected by the
quality of every stage of the experimental process
from aims and objectives, through design, execution,
analysis, interpretation, and finally publication. Al-
though deliberate deception is always a possibility,
the majority of problems that arise are in fact unin-
tentional. Most methodological errors may be classi-
fied as being the result of bias, confounding, or
chance. Therefore, for the purpose of this chapter,
quality will be discussed in relation to these meth-
odological issues. Other aspects of study conduct may
well be critical to the validity of a study but will not be
considered in this chapter as they will be specific for aparticular study. Such factors could include how well
treatment or supportive maintenance was provided.
Bias
Bias is a systematic error. It leads to results which are
consistently wrong in one or another direction. Bias
leads to an incorrect estimate of the effect of a risk
factor or exposure (e.g. smoking) on the development
of a disease or outcome of interest (e.g. response to
periodontal therapy). The observed effect will be
either above or below the true value. Many types of
bias have been identified, however, the main types
relate to:
how subjects were selected for inclusion in a study
(selection bias);
provision of care (performance bias);
assessment of outcomes (detection/measurement
bias);
occurrence and handling of patient attrition
(attrition bias).
Selection bias occurs when there is a systematic
difference between the characteristics of the subjectsselected for a study and the characteristics of those
who were not. For instance, selection bias will often
occur with volunteers (self-selection bias). People
who volunteer to participate in a study tend to be
different from the general population. Similarly, it is
important to consider whether people might have
selectively withdrawn from the study before its
completion (attrition bias). They may have with-
drawn at random, or because of some factor related
to the study, e.g. the treatment they were receiving
was ineffective or uncomfortable in comparison with
the alternative treatment. It is necessary to decidewhether the results of the investigation were likely to
have been compromised if one group of subjects had,
on average, a shorter follow-up as a result of more
people dropping-out.
The avoidance of selection bias is a major concern
in the design of case-control studies. In this type of
study it is essential to ensure that controls are rep-
resentative of the population from which the cases
originated. Suppose a group of researchers is con-
Assessed foreligibility (n= ... )
Excluded (n = ... )
Not meetinginclusion criteria(n = ... )
(n = ... )Refused to participate
Other reasons (n = ... )
Randomized (n = ... )
Allocated to intervention(n = ... )
intervention (n = ... )
(give reasons) (n = ... )
(give reasons) (n = ... ) (give reasons) (n = ... )
(n = ... ) (give reasons)
intervention
Received allocated
Did not receive allocated
Allocated to intervention(n = ... )
intervention (n = ... )
(give reasons) (n = ... )intervention
Received allocated
Did not receive allocated
(give reasons)Lost to follow up (n = ... )
Discontinued intervention(n = ... ) (give reasons)
(give reasons)Lost to follow up (n = ... )
Discontinued intervention
Excluded from analysis Excluded from analysis
Analysed (n = ... ) Analysed (n = ... )
Analysis
Allocation
Enrollment
Followup
(b)
Fig. 4. Continued.
22
Needleman et al.
-
8/3/2019 12 Evidence Based Period Ontology Systematic Reviews
12/17
ducting a case-control study to assess the effect of
cigarette smoking on the development of aggressive
periodontitis. In our hypothetical example, cases are
patients referred to a dental hospital with aggressive
periodontitis and controls are non-dental patients
admitted to a nearby hospital with chronic bronchi-
tis. A standard questionnaire is administered to both
cases and controls that includes questions on lifetime
smoking habits. The researchers may find no evi-dence from this study of an association between
cigarette smoking and aggressive periodontitis. Can
we accept this conclusion? The problem with this
study is that the choice of controls is biased, as the
prevalence of smoking among patients admitted with
chronic bronchitis is likely to be much higher than
among the general population resident in the catch-
ment area of the hospitals from which the cases and
controls originated. Consequently, the strength of
the association between smoking and aggressive
periodontitis will most likely be under-estimated in
this study.
Randomized controlled trials are less likely to be
affected by selection bias if the randomization is
properly conducted. Randomization is a two-stage
process. The first stage is the generation of a true
random sequence. Typically, this is achieved through
computer software or a random number table. Whilst
a tossed coin is theoretically acceptable, we suggest
using a method that can be audited later on for the
purposes of quality assessment, such as a computer
generated list.
The second stage of randomization is less wellunderstood or carried out. Once the random
sequence is generated, it must be concealed from
those selecting patients for a study until the indi-
vidual has been recruited into the trial. If not,
despite the sequence being random, the researcher
will be aware of whether the patient will be
entered into test or control group. This knowledge
provides the opportunity for selection bias, whether
intentional or not. This second stage of randomi-
zation is termed allocation concealment (see Fig. 5
for an outline of this process). The key question to
ask is, Was the recruitment of patients into a trialentirely unpredictable with respect to test or con-
trol group?
Performance bias occurs when different study
groups do not receive therapy in the same fashion or
to the same standard. This may occur if the people
providing the therapy are aware of which groups the
participants have been allocated to. Depending on
the nature of the investigation, it may be either a
relatively simple or a difficult task to ensure that
therapists remain masked (blinded) to the treatment
allocation. The use of placebo, where appropriate,
greatly facilitates masking; placebo controlled trials
are usually easy to organize in such a way as to leave
the therapist masked to the treatment allocation.
However, if the interventions to be compared are
quite dissimilar in their delivery (e.g. surgical vs.
nonsurgical therapy), then masking becomes con-
siderably more challenging. Under these circum-
stances the best available option might be to ensure
that the therapist remains masked until the last
possible moment to ensure that all therapy prior to
that point has been undertaken as even-handedly as
possible. So, for example, in a split-mouth study
comparing scaling and root planning vs. scaling and
root planning plus an adjunctive locally delivered
antimicrobial, it might be possible to complete themechanical therapy at all appropriate sites prior to
the therapist finding out which particular sites are to
receive the adjunctive therapy. However, the risk of
carry-over effects of the local antibiotic affecting the
scaling and root planning-only sites should not be
ignored.
Measurement (information) bias occurs when the
measurements of exposure and/or outcome are not
valid (i.e. they do not measure correctly what they
are supposed to measure). Errors in measurement
may be introduced by the observer (observer bias),
by the study individual (responder bias), or by theinstruments (instrument bias) used to make the
measurements (e.g. a badly designed questionnaire).
As a result of measurement errors, study partici-
pants will be misclassified in relation to their
exposure and/or outcome status. This misclassifi-
cation has particularly serious implications if the
errors in exposure measurement are related to the
participants outcome status. Ideally the person
undertaking the examination should be blinded
Step 1
Generate a true random sequence
- computer generated is best
- tossed coin is acceptable
Step 2
Allocation concealment- conceal the sequence for study recruitment
- sequentially numbered truly opaque envelopes/drug
containers, centrally kept randomization accessed by
telephone, e.g. pharmacy
Fig. 5. The two stages of randomization.
23
Evidence-based periodontology
-
8/3/2019 12 Evidence Based Period Ontology Systematic Reviews
13/17
(masked). This is more important for measures in
which there is the potential for subjectivity (e.g.
pocket depth, colour change) than for objective
measures (e.g. tooth loss).
Bias is a consequence of defects in the design or
execution of a study. Bias cannot be controlled dur-
ing the statistical analysis of the data and cannot be
eliminated by increasing the size of the study.
Publication bias
Publication bias refers to the greater likelihood of
publication of studies with positive results than those
with neutral or negative results (5). The risk with this
type of bias is that interventions appear to perform
better than they will in clinical practice. For instance,
publication bias might mean that although several
studies were published and the data available to be
included in a meta- analysis, a larger number of
studies were actually conducted but not published.
Of these missing studies, some may show no dif-
ference between the intervention group and the
control group, or even the control group performing
better. If these additional studies had been published,
the results of the meta-analysis could have been
different. Therefore, the sample of published studies
is a biased sample and does not represent the com-
plete population of all research on this question.
Graphic and formal statistical tests are available to
investigate publication bias but need approximately
10 or more studies to have adequate power. Figure 6
illustrates this situation using hypothetical data.
Confounding
Confounding is a term that describes the situation
where an estimate of the association between an
exposure and the disease is mixed up with the real
effect of another exposure on the same disease, the
two exposures being correlated. It is a difficult con-
cept that may be illustrated with the help of the fol-
lowing example. Suppose we find that coffee drinkers
have a poorer response to periodontal therapy than
noncoffee drinkers. Does it mean that coffee drinking
affects the response to therapy? The problem here isthat there is an alternative explanation. Smoking may
be an independent risk factor for poor treatment
response and it is possible that people who drink
coffee are more likely to smoke than those who do
not. Perhaps the observed association is actually due
to smoking habits, not coffee drinking (Fig. 7).
Age and sex are the most common confounding
variables in health-related studies because these two
variables are not only associated with most exposures
we are interested in, such as diet, smoking habits,
health beliefs, etc., but are also independent risk
factors for many diseases.
Confounding can be dealt with at the design stage
of an investigation by:
Randomization By randomly allocating subjects
to study groups it is hoped that confounders are
distributed equally between the groups. This is
usually the most effective way of minimizing theproblem of confounding. If randomization is
done properly, it has the advantage that it con-
trols for both known and unknown confounders
provided the sample size is sufficiently large.
Restriction This limits participation in a study to
specific groups which are similar to each other
with respect to the confounder (e.g. if smoking is
likely to be a confounder then only nonsmokers
will be included in the study).
Matching This selects comparison groups with
similar backgrounds (e.g. nonsmokers are matched
with other nonsmokers, while smokers are mat-
ched with other smokers).
Confounding can also be controlled for in the ana-
lysis by:
Stratification Here the strength of the association
is measured separately in each well-defined sub-
group (e.g. in the smokers and the nonsmokers
separately). The results are then pooled together
using basic statistical techniques to obtain an
overall summary measure of the association adjus-
ted or controlled for the effects of the confounder.
Statistical modelling These are more sophisti-cated mathematical techniques that can simulta-
neously take into consideration the effects of
several possible confounders that have been
recorded by the investigators.
It is only possible to control for confounders in the
analysis if data on them were collected during the
study. Obviously, the extent to which confounding
can be controlled for will depend on the accuracy of
these data. However, in some situations it may be
virtually impossible to gain complete and accurate
information on confounders. Some confounders
may be so difficult to assess that even attempting toadjust for them in a statistical model will not com-
pletely control for their effect. For example, Hujoel
et al. have argued that the confounding effect of
smoking is virtually impossible to measure with
sufficient precision in studies that attempt to look at
the association between periodontal diseases and
systemic health and that such studies may only
provide valid results if they are restricted to non-
smokers (10).
24
Needleman et al.
-
8/3/2019 12 Evidence Based Period Ontology Systematic Reviews
14/17
Chance
Chance (sampling error) plays a role in most studies
of humans since it is rarely if ever possible to include
an entire population in an investigation. We therefore
attempt to infer information about the population on
the basis of information obtained from representative
samples drawn from that population. The extent to
which the sample results reflect the likely result in the
population is assessed by performing statistical sig-
nificance tests and, more importantly, by calculating
confidence intervals. A proper discussion of thesemethods is beyond the scope of this chapter, but in
general, studies with small sample sizes will be more
prone to sampling error and will provide less robust
estimates than studies with larger samples.
Interpretation
It is worth noting that authors may also fail to
interpret their experimental results correctly. So,
mean
2 1 0 1
Combined
study 15
study 14
study 13
study 12
study 11
study 10
study 9
study 8
study 7
study 6
study 5
study 4
study 3
study 2
study 1
(b)
1/standard
error
mean difference
.4 .2 0 .2 .4
0
5
10
15
20
25
30
35
40
45
50
(c)
(a)
mean
2 1 0 1
Combined
study 15
study 14study 13
study 12
study 11
study 10
study 5
study 9study 8
study 7
study 4
study 6
study 5study 3
study 4
study 2
study 1
study 3
study 2
study 1
(d)
1/standard
error
mean difference
.4 .2 0 .2 .4
0
5
10
15
20
25
30
35
40
45
50
Favours control
Favoursintervention
Favours control Favoursintervention
Fig. 6. Illustration of publication bias. a) Forest plot
showing the results of a meta-analysis of 15 studies. A verysmall improvement is indicated in favour of the test
intervention since the diamond shape (representing the
95% confidence interval for the pooled result) does not
cross the zero (no-effect) line. b) Funnel plot for these
studies. In the absence of publication bias, it is anticipated
that the plot would form a funnel shape. As no funnel
shape was produced, this indicates the possibility of
missing studies. These studies would be expected to pro-duce data points that lie somewhere within the shaded
area. c) Another Forest plot, including the data from these
extra, previously missing studies. The 95% confidence
interval for the pooled result now crosses the line of no
effect, indicating no evidence that the intervention is any
more effective than the control.
spurious associationCoffee drinking Poor treatment response
association risk factor
Smoking habits
(confounder)
Fig. 7. An example of confounding.
25
Evidence-based periodontology
-
8/3/2019 12 Evidence Based Period Ontology Systematic Reviews
15/17
even if the study has been well conducted and
appropriately analyzed, there is still the potential to
draw incorrect conclusions from the results.
How to critically appraise?
When appraising quality it is necessary to consider
those factors that may affect the outcome of a study.These will inevitably vary according to both the topic
of the original research and the study designs
employed, so it is not possible to devise a single
system that will be appropriate for every occasion. As
a general rule, the aforementioned domains of bias,
confounding and chance will all have to be ap-
praised.
Some reviewers have attempted to devise com-
posite scales that give scores for the various quality
domains (11). These scores are then summed to an
overall summary measure for the study as a whole.
There are problems with this approach. Many
quality items may not be based on empirical evi-
dence and the scores attached to each item will
inevitably be subjective. It is also doubtful whether a
single summary score is likely to provide anadequate overall assessment of the quality of a
particular study. When different composite scales
are applied to the same studies, differing scores and
rankings may occur. For these reasons, composite
scales have largely gone out of favour. An alternative
approach is to appraise each quality component
separately (12).
Table 5. Quality assessment checklist for randomized controlled trials in periodontology used by Montenegro
et al. (18)
Item Classification Definition
Randomization Adequate If generated by random number
table (computer generated or not);
tossed coin; and shuffled cards.
Unclear Study refers to randomization but
either does not adequately explain
the method or no method was reported.
Inadequate Methods include alternate assignment,
hospital number, and odd/even birth date.
Allocation concealment Adequate Methods included central randomization
(e.g. by telephone to a pharmacy or trialoffice), pharmacy sequentially numbered/
coded containers, and sequentially
numbered opaque envelopes.
Unclear If the study referred to allocation
concealment but either did not
adequately explain the method or
no method was reported.
Inadequate Involved methods where randomization
could not be concealed, such as alternate
assignment, hospital number,
and odd/even birth date.
Blinding of patient, caregiver,
and examiner were considered
separately
Recorded as adequate, inadequate,
unclear, or for examiner blinding,
not applicable if the study
design precluded the
possibility of blinding.
Withdrawals and drop outs Were all patients who entered the
trial properly accounted for at the end?
Where dropouts occurred, the use of
analyses to allow for losses (such as
intention to treat) was noted.
26
Needleman et al.
-
8/3/2019 12 Evidence Based Period Ontology Systematic Reviews
16/17
For rigorous systematic reviews, independent
reviewers usually undertake quality appraisal in
duplicate and checklists are frequently employed for
this purpose. Two such checklists that have been
used previously are reproduced here as examples
(Tables 5 and 6) (7, 18). These checklists are based on
a combination of factors that have been shown
empirically to affect quality (such as allocation
concealment) and also topic specific factors deemed
important by the reviewers. Other checklists cover a
broad range of types of research and can be
found on the excellent Critical Appraisal Skills
Programme website (http://www.phru.nhs.uk/casp/
appraisa.htm).
The use of checklists with objective criteria helps
to safeguard the quality of the quality appraisal
process itself. The process of devising the check-list helps to ensure that all relevant quality issues
are included in the assessment. Written, piloted
checklists reduce, but can never completely elim-
inate individual subjectivity in decisions. Having a
written list means that it is more likely that the
quality assessors will be both consistent and
repeatable.
The results of the quality appraisal are used to
assess the value of the evidence and to aid clinicians
and reviewers in their efforts to place the evidence
into context. This might be a part of the formal pro-
cess of undertaking a systematic review or the
informal act of reading and assessing recently pub-
lished literature as part of everyday periodontal
practice.
Conclusions
The principles of evidence-based healthcare provide
structure and guidance to facilitate the highest levels
of patient care. There are numerous components to
evidence-based periodontology including the pro-duction of best available evidence, the critical
appraisal and interpretation of the evidence, the
communication and discussion of the evidence to
individuals seeking care and the integration of the
evidence with clinical skills and patient values. This
volume of Periodontology 2000 is mainly concerned
with the first component, i.e. the generation of best
evidence and, alone, is not enough to practise evi-
dence-based healthcare. However, an understanding
of the principles should help to underpin the latter
aspects. Evidence-based healthcare is not an easier
approach to patient management, but should provide
both clinicians and patients with greater confidence
and trust in their mutual relationship.
References
1. Alderson P, Green S, Higgins JPT, eds. Cochrane Reviewers
Handbook 4.2.2[updated. ]. In: TheCochraneLibrary,Issue1.
Chichester: John Wiley & Sons, 2004.
Table 6. Quality assessment checklist for systematicreviews in dentistry used by Glenny et al. (7)
Question (possible categories)
A. Did review address a focused question?
(yes, no, cant tell)
B. Did authors look for appropriate papers?
(yes, no, cant tell)
C. Do you think authors attempted to identify all
relevant studies?
(yes, no, cant tell)
D. Search for published and unpublished literature
(yes, no, cant tell)
E. Were all languages considered?
(yes, no, cant tell)
F. Was any hand searching carried out?
(yes, no, cant tell)
G. Was it stated that the inclusion criteria were
carried out by at least two reviewers?
(yes, no, cant tell)
H. Did reviewers attempt to assess the quality of the
included studies?(yes, no)
I. If so did they include this in the analysis?
(yes, no, cant tell, not applicable)
J. Was it stated that the quality assessment was
carried out by at least two reviewers?
(yes, no, not applicable)
K. Are the results given in a narrative or pooled
statistical analysis?
(narrative, pooled, not applicable)
L. If the results have been combined was it
reasonable to do so?
(yes, no, cant tell, not applicable)
M. Are the results clearly displayed?
(yes, no, not applicable)
N. Was an assessment of heterogeneity made and
reasons for variation discussed?
(yes, no, not applicable)
O. Were results of review interpreted appropriately?
(yes, no, cant tell, not applicable)
27
Evidence-based periodontology
-
8/3/2019 12 Evidence Based Period Ontology Systematic Reviews
17/17
2. American Academy of Periodontology. Proceedings of the
1996 World Workshop in Periodontics. Lansdowne, Vir-
ginia, July 13-17, 1996. Ann Periodontol 1996: 1: 1947.
3. Antczak-Bouckoms A, Tang J, Chalmers TC. Quality
assessment of randomized controlled trials in dental
research. J Periodontal Res 1986: 21: 305314.
4. EBM Working Group. Evidence-based medicine. JAMA
1992: 268: 24202425.
5. Egger M, Davey-Smith G, Altman DG. Systematic Reviews in
Health Care, 2nd edn, London: BMJ Books, 2001.
6. Esposito M, Coulthard P, Worthington H, Jokstad A. Quality
assessment of randomized controlled trials of oral
implants. Int J Oral Maxillofac Implants2001: 16: 783792.
7. Glenny AM, Worthington H, Esposito M, Coulthard P. The
assessment of systematic reviews in dentistry. Eur J Oral
Sci 2003: 111: 8592.
8. Gray JAM. Evidence-based Healthcare. Edinburgh: Churchill
Livingstone, 1997.
9. Herrera D, Sanz M, Jepsen SJ, Needleman IG, Roldan S. A
systematic review on the effect of systemic antimicrobials
as an adjunct to scaling and root planning in perio-
dontitis patients. J Clin Periodontol 2002: 29 (Suppl 3):
136159.
10. Hujoel PP, Drangsholt M, DeRouen TA. Periodontitis
systemic disease associations in the presence of smo-
king causal or coincidental? Periodontol 2000 2002: 30:
5160.
11. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ,
Gavaghan DJ, et al. Assessing the quality of reports of
randomized clinical trials: is blinding necessary? Control
Clin Trials 1996: 17: 112.
12. Juni P, Altman DG, Egger M. Assessing the quality of con-
trolled clinical trials. Br Med J 2001: 323: 4246.
13. Khan KS, ter Riet G, Glanville J, Sowden AJ, Kleijnen J.
Undertaking systematic reviews of research on effective-
ness. Http://www.York.Ac.Uk/Institute/crd/report4.htm [2nd],
2001.
14. Labriola A, Needleman I, Moles DR. Systematic review ofthe effect of smoking on nonsurgical periodontal therapy.
Periodontol 2000 2005: 37: 124137.
15. Mathews JR. Quantification and the Quest for Medical
Certainty. Princeton: Princeton University Press, 1995.
16. Moher D, Cook DJ, Jadad AR, Tugwell P, Moher M, Jones A,
et al. Assessing the quality of reports of randomized trials:
implications for the conduct of meta-analyses. Health
Technol Assess 1999: 3: 198.
17. Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M,
et al. Does quality of reports of randomized trials affect
estimates of intervention efficacy reported in meta-analy-
ses? Lancet 1998: 352: 609613.
18. Montenegro R, Needleman I, Moles D, Tonetti M. Quality
of RCTs in periodontology a systematic review. J Dent Res
2002: 81: 866870.
19. Needleman IG. A guide to systematic reviews. J Clin
Periodontol2002: 29 (Suppl. 3): 69.
20. Needleman IG. Introduction to evidence based dentistry.
In: Clarkson, J, Harrison, JE, Ismail, AI, Needleman, IG,
Worthington, H, eds. Evidence Based Dentistry for Effective
Practice. London: Martin Dunitz, 2003: 117.
21. Needleman IG, Giedrys-Leeper E, Tucker RJ, Worthington
HV. Guided tissue regeneration for periodontal infra-bony
defects (Cochrane Review). The Cochrane Library. Oxford:
Update Software. http://www.update-software.com/clibhome/
clib.htm, 2001.
22. Needleman IG, Giedrys-Leeper E, Tucker RJ, Worthington
HV. Guided tissue regeneration for infrabony defects
A systematic review. J Periodontal Res 2002: 37: 380
388.
23. Rangachari PK. Evidence-based medicine: old French wine
with a new Canadian label? J R Soc Med 1997: 90: 280284.
24. Richards D. Not all evidence is created equal so what is
good evidence? Evid Based Dent 2003: 4: 1718.
25. Roccuzzo M, Bunio M, Needleman I, Sanz M. Periodontal
plastic surgery for treatment of localized gingival reces-
sions A systematic review. J Clin Periodontol 2002: 29
(Suppl 3): 178194.
26. Schultz KF, Chalmers I, Hayes RJ, Altman D. Empirical
evidence of bias: Dimensions of methodological quality
associated with estimates of treatment effects in controlled
trials. JAMA 1995: 273: 408412.
27. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD,
Rennie D, et al. Meta-analysis of observational studies in
epidemiology: a proposal for reporting. Meta-analysis of
Observational Studies in Epidemiology (MOOSE) group.JAMA 2000: 283: 20082012.
28. Swales J. The troublesome search for evidence: three
cultures in need of integration. J R Soc Med 2000: 93: 402
407.
29. Whiting P, Rutjes AW, Reitsma JB, Bossuyt PM, Kleijnen J.
The development of QUADAS: a tool for the quality
assessment of studies of diagnostic accuracy included in
systematic reviews. BMC Med Res Methodol 2003: 3: 25.
Needleman et al.
top related