editorial on cpg

7/28/2019 Editorial on CPG

1/4

The Jou rna l of Ma nual & Ma nip ulaTive Ther apy n voluMe 16 n nuMber Two 69]

What Are Clinical Prediction Rules?

A clinical prediction rule (CPR) is a com-bination o clinical ndings that have sta-tistically demonstrated meaningul pre-dictability in determining a selectedcondition or prognosis o a patient whohas been provided with a specic treat-ment1,2. CPRs are created using multi-

variate statistical methods, are designedto examine the predictive ability o se-lected groupings o clinical variables3,4,and are intended to help clinicians makequick decisions that may normally besubject to underlying biases5. Te rulesare algorithmic in nature and involvecondensed inormation that identiesthe smallest number o indicators that arestatistically diagnostic to the targetedcondition6. Te number o derived or

validated CPRs is increasing6, specicallyin rehabilitation medicine where pre-

scriptive studies have been developed ormusculoskeletal interventions or lowback pain7,8, cervical pain9,10, and kneedysunction11,12.

Clinical prediction rules may best beclassied into three distinct groups: 1)diagnostic, 2) prognostic, and 3) pre-scriptive1,13. Studies that ocus on predic-tive actors related to a specic diagnosisare known as diagnostic CPRs. Clinicalprediction rules that are designed to pre-dict an outcome such as success or ailureare considered prognostic. Clinical pre-

diction rules designed to target the mosteective interventions are identied as

prescriptive, and these require prospec-tive, longitudinal, randomized controlledtrials that compare outcomes aer se-lected interventions or subjects whomeet a similar score on the CPR1.

Clinical prediction rules are gener-ally developed using a 3-step method14.First, CPRs are derived prospectively us-

ing multivariate statistical methods toexamine the predictive ability o selectedgroupings o clinical variables3. Te sec-ond step involves validating the CPR in arandomized controlled trial to reduce therisk that the predictive actors developedduring the derivation phase were selectedby chance14. Te third step involves con-ducting an impact analysis to determinethe extent that the CPR improves care,reduces costs, and accurately denes thetargeted objective14.

Although there is little debate thatcareully constructed CPRs can improveclinical practice, to my knowledge, thereare no guidelines that speciy method-ological requirements or CPRs or inu-sion into all clinical practice environ-ments. Guidelines are created to improvethe rigor o study design and reporting.Te ollowing editorial outlines potentialmethodological pitalls in CPRs that may

signicantly weaken the transerability othe algorithm. Within the eld o reha-bilitation, most CPRs have been prescrip-tive; thus, my comments here are reec-tive o prescriptive CPRs.

Methodological Pitfalls

CPRs are designed to speciy a homoge-nous set o characteristics rom a hetero-geneous population o prospectively se-lected consecutive patients5,15. ypically,the resulting applicable population is a

small subset o a larger sample and mayonly represent a small percentage o theclinicians actual daily caseload. Te set-ting and location o the larger sampleshould be generalizable15,16, and subse-quent validity studies require assessmento the CPR in dierent patient groups, indierent environments, and with a typi-cal patient group seen by most clini-cians16. Because many CPRs are devel-

oped based on a very distinct group, thatmay or may not be reective o a typicalpopulation o patients, the spectrumtransportability17 o many current CPRalgorithms may be limited.

Clinical prediction rules use out-come measures to determine the eec-tiveness o the intervention. Outcomemeasures must have a single operationaldenition5 and require enough respon-siveness to truly capture appropriatechange in the condition14; in addition,these measures should have a well con-structed cut-o score16,18 and be collectedby a blinded administrator15. Te selec-tion o an appropriate anchor score ormeasurement o actual change is cur-rently debated19-20. Most outcome mea-sures use a patient recall-based question-naire such as a global rating o changescore (GRoC), which is appropriate whenused in the short term but suers rom

recall bias when used in long-term analy-ses19-21. Other studies may use minimallydetectable change scores that were origi-nally validated using the GROC and alsomay be aected by both recall bias anddierences in sample severity or pathol-ogy. Lastly, outcome measures that usescores that are inuenced by administra-tive actors (discharge date, length o stay,patient charges), socio-demographic ac-tors, or internal behavioral characteris-tics (changes in ear avoidance or attitude)are not consistent among populations5.

A potential drawback or CPRs is theailure to maintain the quality o the testsand measures used as predictors in thealgorithm. Te prospective test and mea-sures should be independent o one an-other during modeling16; each should beperormed in a meaningul, acceptablemanner4; and clinicians or data adminis-trators should be blinded to the patientsoutcomes measures and condition22. Fur-

Editorial

P P Cc Prc R


2/4

70 The Jou rna l of Ma nual & Ma nipulaTive T her apy n voluMe 16 n nuMber 2

thermore, the tests should demonstrateacceptable reliability (> 0.60)15 and re-quire administration within an accept-able timerame o the outcome mea-sure22; equivocal or indeterminableresults necessitate reporting22. Recog-nizing the likelihood o a true positivending in the absence o any inorma-

tion will avoid the representative heuris-tic pitall that may compel us towardidentiying a clinical test as positive sim-ply because the result ts the pattern oother ndings23. CPRs that use tests andmeasures with reliability or agreementbelow 0.60 may result in variable nd-ings depending on the clinician whoperorms the examination and depend-ing on the ndings o other tests andmeasures.

It is my impression that the mostrequent current pitall o CPRs is asso-

ciated with the ailure to meet statisticalassumptions during regression model-ing. CPRs are typically underpoweredalling below the suggested require-ments o 10 to 15 subjects or each pro-spective predictor variable24. Validationcohorts require sampling sizes o 100 orgreater with use o logistic regression(used as a standard or CPR assess-ment)25. Rarely is the statistical signi-cance o the model reported in the reha-bilitation-based CPRs, nor is the R2 or

R

2

-equivalent o the model identied

5

.An R2 or R2-equivalent outlines thestrength o association o the predictor

variables (both independently and as agroup) in explaining the variance o theoutcome measure. Low R2 or R2-equiva-lents may suggest that other variablesmore accurately predict the outcome othe study5 and generally suggest a loweect size o the independent variablesidentied and retained in the analyses26.Most CPRs do report condence inter-

vals, and when reported, wide con-

dence intervals imply poor precision ortoo small o a sample size15.

Once a CPR is developed, it is im-portant to recognize the true benet othe tool. It has been suggested that ortrue impact on clinical practice, CPRsshould provide a LR+ o 5 or greater27.CPR derivations perormed on high-risk groups, where ailure to provide theappropriate intervention is highly unde-sirable, should have sensitivity values

that are greater than specicity values28.Tis indicates that the nal algorithmwill accurately provide all o the besttreatment(s) possible versus assuringthat only those specic to the problemare used28.

CPRs should have clinical sensibil-ity. Clinical sensibility implies that the

tool makes inherent clinical sense, thatits easy to use, that the tests and mea-sures are truly related to the outcome,and that clinician perception does notoverly alter the ndings o the tool15.Consequently, tests and measures that

vary in clinical interpretation (e.g.,spring tests o the spine) or that are po-tentially explained by actors beyond theoriginal scope o the examination (e.g.,hip osteoarthritis when addressing hipprocedures that aect the knee) may notbe as useul as actors that are more ex-

plicit during clinical assessment.Lastly, most rehabilitation-related

CPRs are derivation studies, which arethe initial steps in the development oclinical decision rules. Derivation stud-ies lack validation and require ollow-upstudies in diverse centers with dierentpopulations o patients and dierent cli-nicians. Whether the ndings rom aderivation study stand up to the scrutinyo urther assessment is unknown15. Inessence, adoption o a derivation-only

CPR runs the risk o improper treat-ment. Careul attention should be madebeore blindly adopting derivation stud-ies or basing treatment pathways onthese tools.

Summary

Is this editorial an attack on clinical pre-diction rules? Actually, its quite the con-trary. Prescriptive CPRs are useul toolsor a select and discrete population opatients. As manually oriented clini-

cians, we have long realized that sub-setso the population benet rom manualtherapy more so than others. CPRs allowus to isolate a sub-set o desired patientcharacteristics and to dene which tech-niques are most useul or that popula-tion. Te current rehabilitation-basedCPRs have opened the door or addi-tional research to improve our accuracyas clinicians. Unortunately, many o thepresent rehabilitation-based CPRs may

have methodological weaknesses thatmay allow questioning o the utility othe instrument. Although there is nosuch thing as a perect study, betterand more rigorous designs should pro-

vide additional, proound and clinicallyapplicable ndings. As a clinician and aresearcher, I am an advocate o CPRs.

REFERENCES

1. Beattie P, Nelson R. Clinical prediction

rules: What are they and what do they tell

us?Aust J Physiother2006;52:157163.

2. Randolph A, Guyatt H, Calvin JE, Doig G,

Richardson WS. Understanding articles de-

scribing clinical prediction tools. Crit Care

Med1998;26:16031612.

3. Hier DB, Edlestein G. Deriving clinical pre-

diction rules rom stroke outcome research.

Stroke 1991;22:14311436.

4. Kuijpers , van der Heijden GJMG, Ver-

gouwe Y, et al. Good generalizability o a

prediction rule or prediction o persistent

shoulder pain in the short term. J Clin Epi-

demiol2007;60:947953.

5. Wasson JH, Sox HC, Nef RK, Goldman L.

Clinical prediction rules: Applications and

methodological standards. New Engl J Med

1985;313:793799.

6. Brehaut JC, Stiell IG, Visentin L, Graham ID.

Clinical decision rules in the real world:

How a widely disseminated rule is used in

everyday practice. Acad Emerg Med2005;

12:948956.

7. Childs JD, Fritz JM, Flynn W, et al. A clin-

ical prediction rule to identiy patients with

low back pain most likely to benet rom

spinal manipulation: A validation study.

Ann Intern Med2004;141:920928.

8. Hicks GE, Fritz JM, Delitto A, McGill SM.

Preliminary development o a clinical pre-

diction rule or determining which patients

with low back pain will respond to a stabili-

zation exercise program. Arch Phys Med

Rehabil2005;86:17531762.

9. Cleland JA, Childs JD, Fritz JM, WhitmanJM, Eberhart SL. Development o a clinical

prediction rule or guiding treatment o a

subgroup o patients with neck pain: Use o

thoracic spine manipulation, exercise, and

patient education. Phys Ter2007;87:923.

10. seng YL, Wang W, Chen WY, Hou J,

Chen C, Lieu FK. Predictors or the im-

mediate responders to cervical manipula-

tion in patients with neck pain. Man Ter

2006;11:306315.

editoRial: Potential Pitfalls of CliniCal PRediCtion Rules


3/4

The Jou rna l of Manua l & Mani pulaTive Therapy n voluMe 16 n nuMber 2 71

11. Lesher JD, Sutlive G, Miller GA, Chine NJ,

Garber MB, Wainner RS. Development o a

clinical prediction rule or classiying pa-

tients with patelloemoral pain syndrome

who respond to patellar taping. J Orthop

Sports Phys Ter2006;36:854866.

12. Currier LL, Froechlich PJ, Carow SD, et al.

Development o a clinical prediction rule to

identiy patients with knee pain and clinicalevidence o knee osteoarthritis who demon-

strate a avorable short-term response to

hip mobilization. Phys Ter2007;87:1106

1119.

13. Reilly BM, Evans A. ranslating clinical

research into clinical practice: Impact o us-

ing prediction rules to make decisions.Ann

Intern Med2006;144:201209.

14. Childs JD, Cleland JA. Development and ap-

plication o clinical prediction rules to im-

prove decision-making in physical therapist

practice. Phys Ter2006;86:122131.

15. Laupacis A, Sekar M, Stiell IG. Clinical pre-

diction rules: A review and suggested modi-

cations o methodological standards.

JAMA 1997;277:488494.

16. Knottnerus JA. Diagnostic prediction rules:

Principles, requirements, and pitalls. Prim

Care 1995;22:341363.

17. Justice AC, Covinsky KE, Berlin JA. Assess-

ing the generalizability o prognostic inor-

mation.Ann Intern Med1999;130:515524.

18. McConnochie KM, Roghmann KJ, Paster-

nack J. Developing clinical prediction rules

and evaluating observational patterns using

categorical clinical markers. Med Decis

Making1993;13:3042.

19. Norman GR, Stratord P, Regehr G. Meth-odological problems in the retrospective

computation o responsiveness to change:

Te lesson o Cronbach. J Clin Epidemiol

1997;50:869879.

20. Schmitt JC, Di Fabio RP. Reliable change

and minimum important diference (MID)

proportions acilitated group responsive-

ness comparisons using individual thresh-

old criteria.J Clin Epidemiol2004;57:1008

1018.

21. Schmitt JC, Di Fabio RP. Te validity o pro-

spective and retrospective global change

criterion measures.Arch Phys Med Rehabil

2005;86:22702276.

22. Whiting P, Rutjes AV, Reitsma JB, Bossuyt

PM, Kleijnen J. Te development o QUA-

DAS: A tool or the quality assessment o

studies o diagnostic accuracy included in

systematic reviews. BMC Med Res Methodol

2003;10(3):25.

23. Klein JG. Five pitalls in decisions about di-

agnosis and prescribing. BMJ 2005;330:

781783.

24. Concato J, Feinstein AR, Holord R. Te

risk o determining risk with multivariate

methods. Ann Intern Med 1993;118:201

210.25. Vergouwe Y, Steyerberg EW, Eijkemans MS,

Habbema J. Substantial efective sample

sizes were required or external validation

studies o predictive logistic regression

models.J Clin Epidemiol2005;58:475483.

26. Cohen J. Statistical Power Analysis for the

Behavioral Sciences. 2nd ed. Hillsdale, NJ:

Erlbaum, 1988.

27. Jaeschke R, Guyatt GH, Sackett DL. Users

guide to the medical literature. III. How to

use an article about a diagnostic test. What

are the results and will they help me?JAMA

1994;271:703707.

28. McGinn G, Guyatt GH, Wyer PC, et al.

Evidence-based medicine working group.

Users guides to the medical literature. XXII.

How to use articles about clinical decision

rules.JAMA 2000;284:7984.

editoRial: Potential Pitfalls of CliniCal PRediCtion Rules


4/4

[72] The Jou rna l of Manua l & M ani pulaTive Therapy n voluMe 16 n nuMber 2

Manual thErapy awards

o encourage research in manual therapy, Cardon Rehabilitation Products, TerEx, OPP andTe Journal of Manual & Manipulative Terapyare sponsoring prizes to be awarded based on scientic merit in threeareas:

1. experimental studies,2. case studies and3. review articles.

Awards will be selected rom all papers published in JMM in 2008. Award winners will be announced inVolume 17, Number 1, 2009.

The Cardon Award For Excellence in a Published Research Article

o encourage research in manual therapy, Cardon Rehabilitation Products and JMM are sponsoring a prize in theamount o $1000 or an outstanding experimental study. Te goal o the experimental study is to evaluate a manualtherapy assessment or treatment technique. Te concept o validity is central to the experimental study. Tus withinthe connes o the study, the results are accurate, the method and analysis used can bear up under scrutiny and the

interpretation o the ndings is supported by the data collected. In addition, the conclusions reported can be general-ized to practice settings and subjects outside those described in the study. Te ormat or the experimental study alsorequires that the researchers make their assumptions clear, their methods repeatable and their interpretations clearlyseparate rom the methods and results. Experimental papers provide a orum or presenting ones own ndings andconclusions and or arguing or or against competing hypotheses in manual therapy.

The TherEx Award For Excellence in a Published Case Study

o encourage case studies in manual therapy, TerEx and JMM are sponsoring a prize o $750 or an outstandingcase presentation. Te goal o the case study is to report on a patient or a small group o patients that a manual thera-pist eels should be brought to the attention o colleagues. Specically, the case study method integrates basic scienceknowledge with patient assessment and treatment techniques. Within each case, emphasis is placed on the mostclinically relevant aspects o each musculoskeletal condition. In addition, a selective review o the literature is also

included or each case study. In all, the cases provide the reader with inormation about clinical decision making. Teintent is to acilitate critical thinking and to promote proessional growth.

The OPTP Award For Excellence in a Published Review of the Literature

o encourage reviews o the literature in manual therapy, OPP and JMM are sponsoring a prize o $750 or an out-standing review article. Te goal o the review article is to present a large amount o inormation on a subject compre-hensively and efciently. In addition to a command o the literature in a specic area o manual therapy, the writermust also apply critical appraisal skills to material that is being reviewed. Specically, it is not enough or a review tosummarize the ndings o research studies and case reports. Some comment should be made on the research designand methodological quality o the work being reviewed. By combining content review with methodological critique,review articles are intended to bring clinicians and researchers up to date on the state o the art in manual therapy.

editorial on cpg

Documents