oxford talk smmr paper july 2011 final

8/6/2019 Oxford Talk SMMR Paper July 2011 Final

1/30

DrGillianLancaster

PostgraduateStatisticsCentreLancasterUniversity

[email protected]

TrialsinPrimaryCare:design,conductandevaluationofcomplexinterventions

CentreforExcellenceinTeachingandLearning


2/30

RSSPrimaryHealthCareStudyGroupCoauthorsofSMMRpaper:

MikeCampbell,Sheffield

SandraEldridge,QueenMaryLondon

AmandaFarrin,LeedsMauriceMarchant,EastSussexPCT

SaraMuller,Keele

RafaelPerera,OxfordTimPeters,Bristol

TobyPrevost,KingsCollege

GretaRait,UCL


3/30

1.Introduction

ResearchinPrimaryCareistimeconsumingandoftenchallenging

Itrequiresextensiveplanning&prep

Interventionsareoftencomplexand

presentarangeofproblemseg.

Workinginhealthcaresetting

Sensitivitytolocalcontext

Logisticsofapplyingexperimentalmethods


4/30

Whatmakesanintervention

complex?

Interactionsbetweencomponentsinexperimentalandcontrolarms

Difficultyofbehavioursrequiredbythosedeliveringorreceivingtheintervention

Organisationallevelstargetedbytheintervention

Variabilityofoutcomes

Degreeofflexibility/tailoringofinterventionpermitted

Willitworkineverydaypractice?

NB.takenfromMRCguidelines


5/30

Guidance

MRC

documentDevelopingandEvaluatingComplexInterventions

www.mrc.ac.uk/complexinterventionsguidance

CraigP.etal.BMJ2008,337:a1655

BMJpaper(CampbellNCetal.2007,334:4559)DesigningandEvaluatingComplexInterventionsto

improvehealthcare

Casestudies
http://www.mrc.ac.uk/complexinterventionsguidancehttp://www.mrc.ac.uk/complexinterventionsguidance


6/30

KeystatisticaldesignissuesIPhases

given in MRCguidanceframework

Key elements in

designing andevaluating complexinterventions

General points to consider Key statistical design

issues addressed in ourpaper

Development Background andcontext(For more informationand examples see MRC

and Campbell et al.)

Socio-economic background;Underlying cultural

assumptions;Health service system;

Government initiatives;Preventative policies

Defining andunderstanding theproblem(See above docs)

Prevalence of condition;Population most affected;How condition is

caused/sustained;Potential for intervention and

improvementConceptualising theproblem(See above docs)

Levels of complexity of healthproblem and co-morbidity;

Risk factors and factorsinfluencing changes overtime;

Patient beliefs, symptoms andadherence to treatment

Gathering evidence Systematic reviews;Epidemiological research;Qualitative research;Expert opinion

Using evidence from primarystudies, systematic reviewsand qualitative studies toinform study design

Developing theintervention

Identify key processes andmechanisms for delivery;

Potential beneficial effect;

Define target group;Optimise best treatment

combinations

Conducting primary careresearch in the UK:complying with research

governance and assessingquality of care using theQuality and OutcomesFramework


7/30

KeystatisticaldesignissuesII

Phases given inMRC guidanceframework

Key elements indesigning andevaluating complexinterventions

General points to consider Key statistical designissues addressed in ourpaper

Evaluation Developing and

optimising trialparameters

Testing the feasibility and

integrity of the trial protocol;Consideration of appropriate

primary/secondaryendpoints;

Recruitment and retentionstrategies;

Method of randomisation to

minimise imbalance;Sample size considerations

Pilot studies and pre-trial

modelling;Selection of outcome

measures for effectivenessand quality;

Recruitment of practices andparticipants;

Choosing the method of

randomisation;Sample size and between

trial variationData collection andanalysis

Data collection forms;Design of database;Monitoring procedures;Awareness of issues of data

analysis for different studydesigns

Choosing the method of

analysis: cluster specificversus marginal modelsImplementation Getting evidence into

practice(See new MRCguidance document)

Publication and disseminationstrategy;

Stakeholder involvement;Benefits, harms, costs for

decision making;Recommendations


8/30

2.Usingevidencefromprimary

studies,systematicreviewsand

qualitativestudiesinthedesign

Muchhighqualityresearchlacksgeneralisability (externalvalidity)

Interventionmaynotbeeasilyimplementedinpractice(Who?How?Duration?)

Strongargumentforcarryingoutresearchin

themostappropriatecontextandsettingEg.Canwetrustestimateofeffectsizewhen

interventionstudiestolowerBPafterstroke

aremostlycarriedoutinsecondarycare?(Mant etalBMJ2006)


9/30

Usefulbecausebasedonclearlyformulatedresearchquestionsandmethodology

Quality

of

included

papers

has

been

appraised Summary(pooled)estimateofeffectsize

Feasibility,acceptabilityanduptakeof

intervention

can

be

measured

by

level

of

attrition

ofparticipants

Eg.RelativeattritionhasbeenusedtocomparelevelsofattritionacrossoralanticoagulationandDiabetestypeIItrials(Hennekens etal.BMCRes.Methods2007)

Systematicreviewsofdiagnostictestandmethodcomparisonstudiesalsousefulforselectinganappropriatemeasurementmethod ortechnique

SystematicreviewsofRCTs


10/30

Especiallyusefulwhenplanningorevaluatingacomplexintervention

Canbeused:

Before thetrialtoexploreissuesofdesigneg.barrierstorecruitment;acceptabilityof therandomisationfromapatientsperspective

During thetrialtounderstandandunpacktheprocessesofimplementation andchange

After thetrialtoexplorereasonsforthefindingseg.arefindingsinlinewithunderlyingtheory;acceptabilitytodeliverersandreceivers;comparisonswithpatientreportedoutcomes;

thevalueoftheintervention asanevaluativeassessmentandtoaidinterpretation

Qualitativestudies


11/30

3.

Conducting

primary

care

researchintheUK

Publicgenerallytrustsacademicresearch Researchgovernance ensuresresearchintegrity

toupholdthepublicsconfidence,toprotect

participantsfromabuse,andtoprotectresearchersfromaccusationsofmisconduct

Widerangeoflegalrequirementseg.

European

Clinical

Trials

Directive DataProtectionAct

EthicalapprovalNB.muchdebateaboutwhetherRECsshould

examinestatisticalissues&methodologicalrigour


12/30

GPsareusuallyselfemployedorworkwithina

limitedcompany;contracttoNHS

NHSPrimaryCareTrusts(PCTs)commissionGPs

serviceswithintheirgivenarea

PCTsfacilitateresearchlocallytoensure

researchintegrity;researchreviewcommittees

PrimarycareresearchofteninvolvesseveralGP

centresacrossmultiplePCTs

verytimeconsumingtoobtainapproval;

honorarycontracts;CRBchecksetc.(eg.6months)

NIHRhaverecentlyintroducedguidanceanda

ResearchPassportSystemtohelptheprocess

Researchgovernance


13/30

Tomonitorqualityofcareofpatients,financial

incentives(upto30%ofGPincome)havebeen

introducedthroughtheQualityandOutcomes

Framework(QOF)

QOFhas5domainsofincentivisation

oClinicalcare

oOrganisation

oPatientexperience

o

Education

and

trainingoOtheradditionalservices

Pointsareawardedaccordingtotheworkload

neededtoachievetargetsandprevalenceofdisease(age,sex,deprivation)inthearea

ResearchpotentialofQOF


14/30

TouseQOFindicatorsinresearcheg.toassess

differencesinqualityofcare,therearecertain

problemstoovercome:

o Exclusionseg.failuretoattendforassessment,

frailtyofcondition,refusetreatment

o DifferencesbetweenGPPracticeseg.how

conditionsarerecorded,howinterventionsare

assessed,compositionandskillsofpracticestaff

QOFisprimarilypaymentdrivenandnotcreated

forresearchpurposes

Researchdatabaseshavebeencreatedeg.GPRD,

Qresearch,usingsamplesofGPpractices

ResearchpotentialofQOF


15/30

4.Useofpilotstudies

Important

pre

requisite

for

funding Oftenadhocsmallstandalonestudies

Subjecttopublicationbias

Isthereadifferencebetweenafeasibility

andapilotstudy?

Pilotstudiesaddresstheintegrityofthestudyprotocol

Needclearlistofkeyobjectives


16/30

Keyobjectivesofapilotstudy

Testintegrityofstudyprotocol

Samplesizecalculation

Recruitmentandconsentrates Developandtestimplementationanddelivery

oftheintervention

Acceptabilityoftheintervention Trainstaffindeliveryandassessment

Selectionofmostappropriateoutcome

measures

(endpoints) Randomisationprocedure

Pilotdatacollectionforms/questionnaires

Prepareandplandatacollectionandmonitoring


17/30

Example UKBEAMtrial

UKBackPain,Exercise,ActivemanagementandManipulationtrial(Farrin etal.2005)

Totesttheintegrityofthestudyprotocolusingaseriesofsubstudies

Plannedasclusterrandomisedtrial

3treatments activemanagement(practicelevel);spinalmanipulationandexercise(patientlevel)

Findings: Majorityofmethodsweresuccessful

Problemwithdifferentialrecruitmentbetween

practices changedtononclustereddesign


18/30

Pretrialmodelling

Example Fallspreventiontrial(Eldridgeetal.2005)

Toinformdesignandtestlikelihoodofthe

interventionbeingviableandeffective Costeffectivenessmodelusingpilotdata

o UsedprobabilitytreeandMarkovsimulation

Findings: Interventionwouldreduceproportionfalling

byonly2.8%over12months

Ifpolicymakerswerewillingtospend30,000perQALYgained,therewasstillonlya40%chancetheinterventionwouldbecost

effective


19/30

5.Selectionofappropriate

outcomemeasure(s)

Distinguishbetweenprimaryandsecondary

outcomemeasures Validandreliable(repeatable&reproducible)

Directlymeasuredvs patientreported

o Includeadditionalobjectivemeasureswhenselfreportingmaybeunreliableeg.selfassessedsmokingcessationandbiochemicalmeasure

o HRQL usegenericanddiseasespecificmeasure

Selectmostappropriateoutcomeforevaluatingtheeffectivenessoftheintervention

eg.levelofkneepain, kneefunction,abilitytowork,satisfactionwithtreatment

Individuallevelvs group(cluster)level


20/30

6.

Recruitment

Successfulrecruitmentrequiresacoordinatedapproachandgoodpilotwork

Importanttoengagepracticesearlyono IsresearchquestionimportantforPrimaryCare?

o Whatisitsprioritycomparedtootherissues?

o Howdoesitimpactonpatientdoctorrelationship?

o IsGPconfidenttoraiseresearchissuewithinasensitiveconsultation?

Timeconstraintsareamajorissue

Needtofindefficientwaystoidentifythesampleandgainconsent

Complexinterventionscanhavedifferentlevelsofrecruitment(practices&patients)


21/30

Principlesofgoodrecruitment Engagewithallstakeholders(GPs,practicestaff

andparticipants) Brandfortrial(eg.BEAM,PANDA,SCAMPS)

Welldevelopedmarketingstrategy,goodPReg.BellsPalsytrialusedlocalcelebrityinmedia

Wellwrittenpatientinformationdocuments

InvitationtotakepartcomingfromownGP

UsetrainedstaffotherthanGPstoidentifyandconsentparticipantseg.practicenurses

Providestafftrainingindiseasetopicandresearch

GetsupportfromlocalPCRNinfrastructure ResearchReadyaccreditationscheme

ePCRN (www.ePCRN.org)

Reimbursepracticesfortakingpart NB.Participantsareallowedtooptout


22/30

7.Methodofrandomisation

Byindividualorbyclustereg.GPpractices,households,nursinghomes

o relativecostsandjustification

Relativelyfewerclustersthanindividualsare

usually

available

higher

prob.

of

imbalanceo inthesizeofeachtreatmentarm

o inbaselinecovariatedistributionsatindividuallevel

Complexinterventionsinprimarycaremayhavemultiplecomponentseg.simpleparalleldesignvs factorialdesign


23/30

Imbalanceinsizeoftrt groups

Tooptimisepowerneedtoensure

o

an

equal

number

of

clusters

in

each

treatment

armo anequalnumberofpeopleineachtreatmentarm

Toensurebalanceinnumbersofpeopleineach

armcanuseblockingo interventionsareassignedrandomlywithineachblock

o varyingblocksizesreducespredictabilityofnext

assignment Allocationconcealmentisharderinclustertrials

o eachclustergetssameallocation

o useofplacebosisnotusuallyfeasible


24/30

Imbalanceinbaselinecovariates

Imbalancemayaffectface validityofcomparisonsandoverallconclusions

Waystominimiseimbalance: Adjustmentbyanalysis mayresultindifferent

unadjustedandadjustedestimatesoftreatmenteffectso byeffectsizeandsignificance

o difficultiesininterpretation

Atthedesignstage byidentifyingselected

covariateswhichmaybeimportantpredictorsofoutcomeo Randomiseusingstratification prepareaseparate

randomisationscheduleforeachstrata

o Useminimisation handleslargernumberofselectedvariables


25/30


26/30

Samplesize

Identifyprimaryoutcomemeasureand

calculatesamplesizeforindividualtrial

FindestimateofIntraclusterCorrelation

Coefficient(ICC)

o Fortrialsrandomisinggeneralpracticeswith

patientleveloutcomes,ICCsusuallyaround0.05.

o PapershavebeenpublishedprovidinglistsofICCs

Multiply(inflate)samplesizebydesigneffecto 1+(m1)xICC wheremisclustersizeassumingall

clustersizesareequal

Pre2000manyCRTswereunderpowered


27/30


28/30

Example DESMONDtrialComparisonof4methodsofanalysis:outcomeistheproportionof

patientswithanHbA1cbelow7%, interventionisstructurededucn

Model OddsRatio

StandardError

z P > | z | (95% Confidence Interval)

Cluster

specific

1.085539 0.166037 0.54 0.592 (0.804362, 1.465007)

Population averaged:

Robust 1.161681 0.271156 0.64 0.521 (0.735194, 1.835573)

Independent

errors

1.161681 0.162818 1.07 0.285 (0.882643, 1.528933)

Exchangable

errors

1.079769 0.160086 0.52 0.605 (0.807480, 1.443876)


29/30

10.Conclusion

Presentedmainstatisticalissuesforconductingcomplexinterventions

ProvidesaflavouroftheissuescoveredinourPHCSGmeetingsoverpast8years

Balancebetweenmethodologicalissuesandmorepracticalissuesofrealliferesearch

o manyissuesnotuniquetoprimarycaresetting

Challengeremainsofmaintainingandexpandingthecapacityofbothmethodologicalandappliedexpertiseinprimarycare


30/30

Reference LancasterG.A.,CampbellM.C.,EldridgeS.E.,Farrin

A.,

Marchant M.,

Muller

S.,

Perera R.,

Peters

T.J.,

PrevostA.T.,Rait G.(2010).TrialsinPrimaryCare:statisticalissuesinthedesign,conductandevaluationofcomplex

interventions.StatisticalMethodsinMedicalResearch19:34977.

Facultyof1000publication.

Primstat data

archive www.jiscmail.ac.uk/primstat

presentationsandsummariesofdiscussionsfrommeetingsofPHCSG
http://www.jiscmail.ac.uk/primstathttp://www.jiscmail.ac.uk/primstat

oxford talk smmr paper july 2011 final

Documents