the research challenges of large scale rcts with volunteers and volunteering organisations...
TRANSCRIPT
The research challenges of large scale RCTs with volunteers and volunteering organisations
Facilitator: Dr Matt RyanPresenter: Professor Peter John
RCTs, trials, randomised evaluations, experiments• Randomly allocate something to a group and compare with a group
that does not get it• Needs to be staged and for certain things to be in place –
randomisation, and tying of outcomes to random allocation• See diagram• Unbiased estimate of a programme• Considered to be the gold standard• They are hard to do - worth it, and lots of leaning and interactions
with researchers to the benefit of all
Figure 7.1 Simple Random Allocation Design
Simple Random Allocation Design
Key distinguishing features of RCTs
• Research needs to deliver or have delivered an intervention at the same time as research
• So more activities to carry out as well as measurement• Need for effective coordination of activities• Real world constraints need to be anticipated • Need to ensure procedures follow special conditions of a trial –
attrition, selection etc.
Interactive design: The Glass Bead Game• Step 1: Assessment of fit for a RCT• Step 2: Recruitment of partner(s) • Step 3: Selection of the research site and subjects• Step 4: Determination of the treatments• Step 5: Specification of control condition• Step 6: Calculation of sample size• Step 7: Agreement on procedure for random allocation• Step 8: Collection of data on the subjects• Step 9: The preparation of research plans• Step 10: Assessment of ethical principles
Threats to experiments
• Problem 1: Loss of subjects • Problem 2: Failed randomization• Problem 3: Treatment not given or modified• Problem 3: Interference between treatment and control• Problem 4: Unavoidable confounds• Problem 5: Poor quality data• Problem 6: Loss of cooperation of partners• Problem 7: Unexpected logistical challenges and expenses
Volunteering experiments
• We are the first of these (bar one)• Sector has had a lot of attention but with giving e.g. experiments on
matching (see BIT), e.g. on JustGiving• Volunteering is different – more lumpy, differentiated, hard to match
demand and supply• Need to recruit volunteering organisations – really enthusiastic and
interested in the methods – issue was the practical constraints• RCTS need:
• enough numbers• good data and its management• those who understand experiments
Some reactions to the offer:no fear of white coats!
Politicophobia?
• Concerns about mentioning politicians, especially with governing boards
Data protection and management
• Need to ensure transfer of the data is legal, sensitive – university example
• Managing data
12
Recruitment to the project, volunteering• High attrition to staying in the study • Just getting volunteers to volunteer!
Measuring outcomes
• Hard to find records of the volunteers, information not routinely kept or easily found
Summing up
• It is possible to get the experiments to work, but new grounds – so needed our great partners to make it happen
• Might be easier now if this knowledge disseminates• Also experience generally is growing about RCTs >> helps volunteering
experiments• Can we make it easier, get better quality data