Introduction toIntroduction toSystematic Reviews of Systematic Reviews of

Disability and Rehabilitation Disability and Rehabilitation InterventionsInterventions

3.0

DefinitionsDefinitions

A Systematic Review isA Systematic Review is

““The application of procedures that limit bias in The application of procedures that limit bias in the assembly, critical appraisal, and synthesis of the assembly, critical appraisal, and synthesis of all relevant studies on a particular topic. Meta all relevant studies on a particular topic. Meta analysis may be but is not necessarily part of the analysis may be but is not necessarily part of the process.” process.” (Chalmers et al. 2002)(Chalmers et al. 2002)

DefinitionsDefinitions

A meta-analysis is defined as:A meta-analysis is defined as:

““The statistical synthesis of the data from The statistical synthesis of the data from separate but comparable studies leading to separate but comparable studies leading to a quantitative summary of the pooled a quantitative summary of the pooled results.” results.” (Chalmers et al. 2002.)(Chalmers et al. 2002.)

DefinitionsDefinitions

A Systematic Review Production Model is A Systematic Review Production Model is defined as:defined as:

The systematic and replicable organization of The systematic and replicable organization of human and financial resources to achieve human and financial resources to achieve efficiency, cost effectiveness, and economies of efficiency, cost effectiveness, and economies of scale in producing systematic reviews. scale in producing systematic reviews.

(Turner et al., 2006(Turner et al., 2006) )

MethodsMethods

• Developed Target List of OrganizationsDeveloped Target List of Organizations

• Inclusion Criteria: SR as core or importantInclusion Criteria: SR as core or important

• Personnel CommunicationPersonnel Communication

- Phone calls- Phone calls

- Emails- Emails

• Literature SearchLiterature Search

ResultsResults

ResultsResults

• Identified 18 Identified 18 organizationsorganizations

• Divided data collection Divided data collection into two Roundsinto two Rounds

• Report results from Report results from Round OneRound One

Organizations & Data CollectionOrganizations & Data Collection

Round 1Round 1 Round 2Round 2

1. Cochrane Collaboration (UK)1. Cochrane Collaboration (UK) 12. Policy Hub (UK)12. Policy Hub (UK)

2. EPPI Centre (UK)*2. EPPI Centre (UK)* 13. UK Home Office (UK)13. UK Home Office (UK)

3. CRD (UK)3. CRD (UK) 14. Dept of ED. & Skills (UK)14. Dept of ED. & Skills (UK)

4. Campbell Collaboration (US)4. Campbell Collaboration (US)

Nordic Campbell Center (NOR)Nordic Campbell Center (NOR)

15. Social Science Institute 15. Social Science Institute for Excellence (UK)for Excellence (UK)

5. WWC (USA)5. WWC (USA) 16. NICE (UK)16. NICE (UK)

6. CSLP (Canada)6. CSLP (Canada) 17. GAO (USA)17. GAO (USA)

7. Blueprints for Violence (US)7. Blueprints for Violence (US) 18. CDC (USA)18. CDC (USA)

8. NREPP (US)8. NREPP (US) 19.19.

9. SCTA (Sweden)9. SCTA (Sweden) 20. 20.

10. CERM (US)10. CERM (US) 21.21.

11. Joanna Briggs Institute (AU)*11. Joanna Briggs Institute (AU)* 22.22.

Organizations IdentifiedOrganizations Identified

CountryCountry NN

United KingdomUnited Kingdom 3 (5)3 (5)

United StatesUnited States 5 (2)5 (2)

CanadaCanada 11

NorwayNorway 11

AustraliaAustralia 11

SwedenSweden 11

NoteNote. Numbers in parenthesis count organizations that will be . Numbers in parenthesis count organizations that will be included in round two of data collection.included in round two of data collection.

Development of the Field of Development of the Field of Systematic ReviewingSystematic Reviewing

19801980 20002000

C2C2

19881988

CSLPCSLP

19931993

C1C1

EPPIEPPI

19941994

CRDCRD

19951995

JBIJBI

19991999

CERMCERM

20022002

WWCWWC

2006200619871987

SCTASCTA

Outside US:Outside US:

(Sweden, CA, UK, AU)(Sweden, CA, UK, AU)11

Inside USInside US

BVP(US)BVP(US)

1Not shown are organizations that will be included in round 2 of data collection: CDC GAO, Policy Hub, UK Home Office, DE&S, SSIE, and NICE.

Types of OrganizationsTypes of Organizations

Review OrganizationsReview Organizations(n = 11)(n = 11)

ContractContract(n = 8)(n = 8)

InterestInterest(n = 3)(n = 3)

HealthHealth(n = 4)(n = 4)

SocialSocial(n=4)(n=4)

HealthHealth(n = 2)(n = 2)

Social Social (n = 1)(n = 1)

• Most organizations were government fundedMost organizations were government funded

• Most organizations conduct contract reviewsMost organizations conduct contract reviews

• Cochrane, Campbell, and Briggs conduct Cochrane, Campbell, and Briggs conduct “interest” reviews“interest” reviews

Typical Review TeamTypical Review Team(whether Interest or Contract)(whether Interest or Contract)

• Information SpecialistInformation Specialist

• At Least Two ReviewersAt Least Two Reviewers

• Users*Users*

• Statistical Consultant* Statistical Consultant*

Accountability

Support

The General Stages of a ReviewThe General Stages of a Review

1.1. Formulate review questionsFormulate review questions

2.2. Define inclusion and exclusion criteria Define inclusion and exclusion criteria **3.3. Locate studiesLocate studies4.4. Select studiesSelect studies

5.5. Assess study quality Assess study quality **6.6. Extract dataExtract data

7.7. Analyze and present results Analyze and present results **8.8. Interpret resultsInterpret results

* In Stages 2, 5, and 7 there is substantial variability across (and within) organizations.

The NC2 Project Plan:The NC2 Project Plan: Ingredients for a Pricing ModelIngredients for a Pricing Model

ProcessProcess

Review Review

TeamTeam

Research Research AssistantsAssistants

Title RegistrationTitle Registration 4 weeks4 weeks 40 hours40 hours 4 hours4 hours

Protocol DevelopmentProtocol Development 14 weeks14 weeks 28 hours28 hours 18 hours18 hours

Protocol ReviewProtocol Review 8 weeks8 weeks 130 hours130 hours 0 hours0 hours

Locate, code, & screen Locate, code, & screen studiesstudies

6 weeks6 weeks 60 hours60 hours 177 hours177 hours

AnalysisAnalysis 10 weeks10 weeks 100 hours100 hours 0 hours0 hours

InterpretationInterpretation 172 hours172 hours 4 hours4 hours

ApprovalApproval 8 weeks8 weeks 120 hours120 hours 0 hours0 hours

TotalTotal 50 weeks50 weeks 650 hours650 hours 203 hours203 hours