the campbell collaboration education panel session comments and points for discussion sandra jo...

The Campbell Collaboration www.campbellcollaboration.orgThe Campbell Collaboration www.campbellcollaboration.org

Education Panel SessionComments and Points for Discussion

Sandra Jo WilsonVanderbilt University

Editor, Campbell Collaboration Education Coordinating Group

The Campbell Collaboration www.campbellcollaboration.org

Truancy Interventions• Focused on students already exhibiting

attendance problems.• Broad inclusion criteria in terms of study

designs.• Statistically significant effects on attendance

outcomes.– Also, significant variability in those effects.– And, some question about the clinical

significance.


What moderates the effects of truancy interventions?• A variety of moderators was examined, including

publication type and research design, participant characteristics, and intervention characteristics.

• No moderator was significantly associated with larger or smaller treatment effects.

• Small sample makes it difficult to detect significant moderators.

• But, interesting that different modalities had similar effects, and that participant characteristics were not influential.


Volunteer Tutoring• Update of a previous review• Intervention scope is similar to the attendance

intervention review• Participant scope is broader• Broader array of outcomes• Inclusion regarding research designs more stringent• Statistically significant affects on most of the

achievement outcomes– Comprehension and mathematics not significant


Updating a Previous Review

• Including newly available literature and broadening the geographic scope

• This should give us greater opportunities for examining variability

• New innovations in meta-analysis methodology make updating reviews attractive from a methodological standpoint, in addition to the obvious policy relevance.


Discussion Points• What challenges did you face in narrowing down

your question?• What challenges did you face in gathering

necessary data from source studies?• What areas of support would make it easier for

you to do a review like this in the future?• What were the difficult choices you had to make

during the study collection process or the review process?


Scoping a Review• The research question

–Are you interested in whether a particular program is effective vs. are you interested in what best improves some problem?

–What research designs would be the best for answering such a question? • What designs are actually used.


Other Scoping Considerations• Your constituents

– Your funding agency– The users

• Budget


Challenges in the Conduct of a Review• Definitional and language challenges

– Has implications for your search, for the scope, for how you combine (or not) outcomes in a meta-analysis, and for moderator analysis• What is truancy, how is it different from school refusal

or attendance problems?• Programs with different labels but the same content?

• Missing or vaguely reported data in primary studies


Support and Training for Conducting a Review

• Creating the ideal review team• Content and methods experts• Making contact with other groups who may

have access to different resources– Can help with the search and with

definitional issues


Other Discussion Points?• Reporting relevant and useful reviews

– How do we create useful systematic reviews?

• Questions for our review authors?• Other topics?

The Campbell Collaboration www.campbellcollaboration.orgThe Campbell Collaboration www.campbellcollaboration.org

Thank you to Brandy and SarahAnd our audience!

Sandra Jo Wilsonemail: [email protected]

the campbell collaboration education panel session comments and points for discussion sandra jo...

Documents

significant slide

significant effects

review process

review definitional

attendance outcomes

research designs

users budget slide

research question