designing data collection the two types of study!
Post on 22-Dec-2015
221 views
TRANSCRIPT
![Page 1: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/1.jpg)
Designing data collection
The two types of study!
![Page 2: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/2.jpg)
• Objectives and hypotheses always (really?) reduce to…– …there is a difference between…– …there is an association between…– …change X leads to response Y…
![Page 3: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/3.jpg)
Implications for methods
Primary response
What changes it?
What needs to be held constant?
What may be confounding it?
Experiment or survey?
![Page 4: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/4.jpg)
Individual Population Research question Measurement tool
farmer Farmers with < 3ha in Embu district, Kenya
Does the proportion of income derived from fruit depend on distance to a tarred road?
Questionnaire, map
tree Baobabs in Kibwesi Do the trees fall into groups with different flowering dates?
Visual observation
wood trader Wood traders selling timber in Kampala
What proportion of timber currently comes from farms?
Questionnaire
tree Grevillea trees in upper Embu
How does pruning affect growth rate?
Tape measure
mother Women around Zomba who have planted sclerocarya
How do the farm grown fruit compare with those in the wild?
Bao board and stones
![Page 5: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/5.jpg)
Question What changes?
Unit observed Measurement tool
Can time to germination for species x be reduced by using a higher temperature?
temperature Dish of 100 seeds Visual count
Can reducing the density of potting mix increase root growth rates?
Potting mix density
seedling Ruler, balance
How much variation in growth is there between 6 provenances of x?
Provenance Line plot of 8 trees Tape measure
What advantages are there of planting timber trees mixed with crops rather than along boundaries?
Planting position
10 trees on a farm Questionnaire
![Page 6: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/6.jpg)
Surveys Experiments
Define units Define units
Select sample Select sample
Impose changes (treatments)
Control variation
Take measurement Take measurement
Conclude:
1. Status of population
2. Difference between groups
3. Relationship between variables
Conclude:
Treatments cause observed changes
![Page 7: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/7.jpg)
“Cigarette smoking is harmful to your health”
• How do we know?
Experiment:– Take 2000 young people who have not
smoked– Randomly allocate to smoking and non-
smoking groups– Monitor health outcomes for 40 years
![Page 8: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/8.jpg)
• Survey (prospective)– Sample of smokers and non-smokers– Monitor health– Compare two groups
• Survey (retrospective)– Sample of sick and health people– Elicit smoking history– Compare two groups
![Page 9: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/9.jpg)
• Result clear
Health
Smoking
F
?
•BUT maybe there are other factors (F)?
- lifestyle - diet - genetics - …
![Page 10: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/10.jpg)
• Reaction: control for F– eg F=overweight. – Do the study for overweight and normal
people
• Hypothesis: it is F2– Do a study controlling for F and F2
• ….
![Page 11: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/11.jpg)
• Look for further supporting evidence– Surveys in very different contexts that gave the same
results– Experiments on animals and in vitro
• considered ethical
– Causal explanations• biochemical
• Eventually evidence is considered overwhelming (even by tobacco companies?)
![Page 12: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/12.jpg)
Your problem
• You have to probably 1 chance to do the study
• Therefore:– Think of those alternative explanations and
control for them– Collect the supporting evidence– Get familiar with the causal/theory
explanations
![Page 13: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/13.jpg)
Common design principles
Principle What? Example problems
Objectives Drive all of design
Too vague
Not internally consistent
Units The objects being studied
Mixed within one study
Wrong level used
Comparison Differences or relationships
No variation in key variable
![Page 14: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/14.jpg)
Principle What? Example problems
Confounding In ability to separate effects of two variables
Failure stratify
Management varying between treatments
Repeatability and uncertainty
Consistent results shown by repeating
No repetition (case study)
Repetition at the wrong level
Measurement Variables to measure
Many measurements not related to objective
Key measurements missing
![Page 15: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/15.jpg)
Any more?
Principle What? Example problems
Protocol Written details of all procedures
Not done
Not reviewed and critiqued
Not kept uptodate
![Page 16: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/16.jpg)
Design hierarchy of unitsHow many?
Which ones?
Which treatments?
Who decides?
Site
Community/ farmer group
Farm
Field
Plot
Tree
Sample
![Page 17: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/17.jpg)
Comparison…
• …of treatments• …of niches• …of farmers• …of farm types• …of communities• …of farmer groups• …of group typesThe more which are planned the better!
![Page 18: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/18.jpg)
Uncertainty
• Aim to make some generalisable conclusions Variation and uncertainty make this difficult.• Answer: repeat observations to spot general
patterns = Replication– decreases the uncertainty
• 40 out of 50 farmers or 4 out of 5 farmers?– quantify the uncertainty
• 40 out of 50 means 67% to 93%
• Replicate at the appropriate 'layer‘• And remember the domain to which conclusions
must apply
![Page 19: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/19.jpg)
Bias
• Bias in allocation – randomise– measure
• Bias in selection– Randomise measure
• Bias in measurement– use improved tool– triangulate
= consistent under or over estimate of real effect
Awareness
Training
![Page 20: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/20.jpg)
Confounding
Solution:• Check objectives - it may not be a problem• Control the NEV• Include extra plots of other combinations
= confusingSystemcontinuous IF
variety
Local
improved
Eg: compare continuous cropping with improved fallow
![Page 21: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/21.jpg)
The protocol
written plan of the activity
![Page 22: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/22.jpg)
A protocol should be...
• prepared for every research activity, however small
• written, not in your head• shared with others who can help improve it
– experience from similar problems, methods, species, ecozones,...
• detailed enough for someone else to take over the study
![Page 23: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/23.jpg)
• kept up to date– plans change during the execution– a record of what actually done
• archived with the data
![Page 24: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/24.jpg)
The protocol contains...• Identification (name of trial, people,...)• Justification• Objectives• Methods
– Type– Location– Treatments– Field layout (sites, blocks, plots)– Management– Measurements– ….– Analysis methods
• Implementation
![Page 25: Designing data collection The two types of study!](https://reader035.vdocuments.site/reader035/viewer/2022062516/56649d7f5503460f94a63048/html5/thumbnails/25.jpg)
Experiments with Farmers:
Checklist for Preparing Protocols
- A list of items to think about
- Not a form to fill in