![Page 1: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/1.jpg)
Analyzing the Results of an Experiment…
• -not straightforward..
– Why not?
![Page 2: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/2.jpg)
Variability and Random/chance outcomes
![Page 3: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/3.jpg)
Inferential Statistics
• Statistical analysis appropriate for inferring causal relationships and effects.
• Many different formulas…which one do you use?
![Page 4: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/4.jpg)
Inferential Stat selection
• -Determine that you are analyzing the results of an experimental manipulation, not a correlation
• Identify the IV and DV.
• The IV Will always be nominal on some level, even when it may seem to be continuous..low, medium and high doses of a drug
![Page 5: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/5.jpg)
Inf. Stat Selection
• What is the scale of the DV?
– Scale of DV -Statistic to use
Nominal Chi-squared
Ordinal Mann-Whitney U-test
Continuous T-test or ANOVA
![Page 6: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/6.jpg)
t-test or ANOVA?
How many levels of the IV are there?
2 levels more than 2 levels
T-test or ANOVA ANOVA
![Page 7: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/7.jpg)
There are different forms of T-tests and ANOVA’s:
Did the Study Use a Within Group or Between group Experimental Design?
Between Group Within Group
Only 2 levels of the IV Unpaired t-tests (or “t for independent samples”).
“Paired t-tests ( or “t for dependent samples”)
Or…ANOVA ( the basic ANOVA is fitted for between group designs)
Or…Within group ANOVA (often referred to as a “repeated measures ANOVA”)
More than 2 levels of the IV
ANOVA Repeated Measures ANOVA
![Page 8: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/8.jpg)
In some ways all inferential Stats are similar.
• They calculate the probability that a result was due to the IV as opposed to random variability…
• Let’s focus on the Basic ANOVA since it is likely to be the statistic you may use most commonly.
![Page 9: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/9.jpg)
ANOVA
• ANOVA produces an F-value.
• F values are the ratio of overall between group Variability to the Mean within group variability
Between Var. (+ chance) /Mean within grp.
Variability (+ chance)
What does this mean?
![Page 10: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/10.jpg)
Lets suppose:
• Experiment- IV marijuana– Control– Placebo control– Low dose– High dose
![Page 11: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/11.jpg)
Dependent Variable is:
• Performance on a short term memory task measured number correct out of 10 test items.
• 9 subjects in each group
![Page 12: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/12.jpg)
Possible out come 1
![Page 13: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/13.jpg)
Possible Outcome 1
Control Placebo Low dose High dose
• 4 2 2 2• 5 3 3 3• 6 4 4 5• 5 6 4 3• 5 5 5 4• 6 5 4 4• 4 4 5 4• 3 4 6 6• 7 3 3 5
![Page 14: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/14.jpg)
Distribution of scores for control sample
0
.5
1
1.5
2
2.5
3
3.5
Cou
nt
0 2 4 6 8 10 12control
![Page 15: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/15.jpg)
Placebo scores
0
.5
1
1.5
2
2.5
3
3.5C
ount
0 2 4 6 8 10 12placebo
![Page 16: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/16.jpg)
Low dose scores
0
.5
1
1.5
2
2.5
3
3.5C
ount
0 2 4 6 8 10 12low
![Page 17: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/17.jpg)
High dose scores
0
.5
1
1.5
2
2.5
3
3.5
Cou
nt
0 2 4 6 8 10 12high
![Page 18: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/18.jpg)
The population distribution of scores
0
2
4
6
8
10
12C
ount
0 1 2 3 4 5 6 7 8 9 10 11population
![Page 19: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/19.jpg)
F value relatively low
Highlow placebo
control
Between grp. Var
w/in grp. var
![Page 20: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/20.jpg)
Now consider this: Possible Outcome 2
Control Placebo Low dose High dose
• 4 2 2 2• 5 3 3 3• 6 4 4 5• 5 6 4 3• 5 5 5 4• 6 5 4 4• 4 4 5 4• 3 4 6 6• 7 3 3 5
![Page 21: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/21.jpg)
Distribution of scores for control sample
0
.5
1
1.5
2
2.5
3
3.5C
ount
0 2 4 6 8 10 12control
![Page 22: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/22.jpg)
Placebo scores
0
.5
1
1.5
2
2.5
3
3.5C
ount
-2 0 2 4 6 8 10 12placebo
![Page 23: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/23.jpg)
Low dose scores
0
.5
1
1.5
2
2.5
3
3.5C
ount
0 2 4 6 8 10 12low
![Page 24: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/24.jpg)
High dose scores
0
.5
1
1.5
2
2.5
3
3.5C
ount
0 2 4 6 8 10 12high
![Page 25: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/25.jpg)
F value relatively High
Highlow placebo
control
Between grp. Var
w/in grp. var
![Page 26: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/26.jpg)
The high F value reflects
• Logic!
• Distribution of score are much more obviously separated, and in this case are completely non-overlapping
• Low F values indicate highly overlapping score distributions
![Page 27: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/27.jpg)
So how do we decide if an F value is large enough to consider the result as causal?
• We consult a table of established probabilities of different F values, within the context of Degree of freedom terms:
![Page 28: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/28.jpg)
![Page 29: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/29.jpg)
![Page 31: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/31.jpg)
![Page 32: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/32.jpg)
![Page 33: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/33.jpg)
Where is/are the difference (s)?
0
10
20
30
40
50
60
70
Neutral Positive Negative Sex Drug Taboo
Neutral
Positive
Negative
Sex
Drug
Taboo
![Page 34: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/34.jpg)
Inferential Statistics
![Page 35: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/35.jpg)
The story of “Scratch”
![Page 36: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/36.jpg)
Why not jus use repeated t-tests? Probability pyramiding
• 15 t-tests required for this data set
• Post-hocs include compensations for repeated testing of a large data set
0
10
20
30
40
50
60
70
Neutral Positive Negative Sex Drug Taboo
Neutral
Positive
Negative
Sex
Drug
Taboo
![Page 37: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/37.jpg)
After all this where so we stand?We can still be wrong.
![Page 38: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/38.jpg)
![Page 39: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/39.jpg)
![Page 40: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/40.jpg)
Factors that affect “power.”Sample size
![Page 41: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/41.jpg)
One vs two-tailed testing
![Page 42: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/42.jpg)
![Page 43: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/43.jpg)
• Effect size
![Page 44: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/44.jpg)
![Page 45: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/45.jpg)
![Page 46: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/46.jpg)
![Page 47: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/47.jpg)
![Page 48: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/48.jpg)
![Page 49: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/49.jpg)
![Page 50: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/50.jpg)
![Page 51: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/51.jpg)
![Page 52: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/52.jpg)
![Page 53: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/53.jpg)
![Page 54: Analyzing the Results of an Experiment… -not straightforward.. –Why not?](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649f355503460f94c52c29/html5/thumbnails/54.jpg)