issues in experimental design for the study of atypical language development
DESCRIPTION
TRANSCRIPT
![Page 1: Issues in experimental design for the study of atypical language development](https://reader034.vdocuments.site/reader034/viewer/2022051207/540029968d7f7261088b49fa/html5/thumbnails/1.jpg)
08/10/2013
1
Issues in experimental design for the study of
atypical language development
David Saldaña
DESIGNING YOUR STUDY: CONTROL
![Page 2: Issues in experimental design for the study of atypical language development](https://reader034.vdocuments.site/reader034/viewer/2022051207/540029968d7f7261088b49fa/html5/thumbnails/2.jpg)
08/10/2013
2
Controlled experiments This type of experiment is conducted in a well-controlled environment – not necessarily a laboratory – and therefore accurate measurements are possible.
Easier to replicate
Precise control of extraneous and independent variables.
The artificiality of the setting and lack of generalization
Demand characteristics or experimenter effects may bias the results
McLeod, S. A. (2012).
Field experiments The experimenter still manipulates the independent variable, but in a real-life setting (so cannot really control extraneous variables).
More likely to reflect real life because of it natural setting
Less likelihood of demand characteristics affecting the results
Can be used in situations in which it would be ethically unacceptable to manipulate the independent variable, e.g. researching stress.
Less control over extraneous variables
Less replicable
McLeod, S. A. (2012).
![Page 3: Issues in experimental design for the study of atypical language development](https://reader034.vdocuments.site/reader034/viewer/2022051207/540029968d7f7261088b49fa/html5/thumbnails/3.jpg)
08/10/2013
3
Natural experiments Natural Experiments are conducted in the everyday environment of the participants but here the experimenter has no control over the IV as it occurs naturally in real life.
More likely to reflect real life because of it natural setting
Less likelihood of demand characteristics affecting the results
Can be used in situations in which it would be ethically unacceptable to manipulate the independent variable, e.g. researching stress.
Less likelihood of demand characteristics affecting the results, as participants may not know they are being studied.
No control over extraneous variables
Less replicable
Possible more expensive
McLeod, S. A. (2012).
A perhaps “silly” research question
Does vocabulary influence the comprehension of questions more than the comprehension of imperatives?
![Page 4: Issues in experimental design for the study of atypical language development](https://reader034.vdocuments.site/reader034/viewer/2022051207/540029968d7f7261088b49fa/html5/thumbnails/4.jpg)
08/10/2013
4
A question that is a little more complicated
Do children with SLI understand questions better than typically developing children?
Do children with SLI understand questions better than imperatives?
DESIGNING YOUR STUDY: CONDITIONS
![Page 5: Issues in experimental design for the study of atypical language development](https://reader034.vdocuments.site/reader034/viewer/2022051207/540029968d7f7261088b49fa/html5/thumbnails/5.jpg)
08/10/2013
5
How is your IV distributed?
Independent Measures
Repeated Measures
Matched Pairs
Independent measures (between-group)
• Each group gets one condition
• Different participants in each group • Avoids practice
• More people needed
• Participant variables could affect results.
![Page 6: Issues in experimental design for the study of atypical language development](https://reader034.vdocuments.site/reader034/viewer/2022051207/540029968d7f7261088b49fa/html5/thumbnails/6.jpg)
08/10/2013
6
Task 1
• Measurement
Task 2
• Measurement
Repeated measures (within-group)
- Less participants - Precision determined by variation within same
subject - May be the only design that answers the
questions of interest. - Order effects: practice and fatigue effect
Counterbalancing
Moment 1
A
B
Moment 2
B
A
![Page 7: Issues in experimental design for the study of atypical language development](https://reader034.vdocuments.site/reader034/viewer/2022051207/540029968d7f7261088b49fa/html5/thumbnails/7.jpg)
08/10/2013
7
Back to our “silly” study
Lets design the tasks and the data collection
Does vocabulary influence the comprehension of questions more than the comprehension of imperatives?
A little bit more interesting (perhaps) research question
Does vocabulary influence the comprehension of questions more than the comprehension of imperatives in SLI?
![Page 8: Issues in experimental design for the study of atypical language development](https://reader034.vdocuments.site/reader034/viewer/2022051207/540029968d7f7261088b49fa/html5/thumbnails/8.jpg)
08/10/2013
8
Task 1
• Measurement
Task 2
• Measurement
Mixed-design
Task 1
• Measurement
Task 2
• Measurement
Gro
up
1
Gro
up
2
Some imaginative results
70
77 73 75
90 90 95
99
92 90
100
90
0
20
40
60
80
100
120
IQ Vocabulary ComprehensionQuestions
Word Reading SES ComprehensionImperatives
Stan
dar
d S
core
s
SLI Typical
![Page 9: Issues in experimental design for the study of atypical language development](https://reader034.vdocuments.site/reader034/viewer/2022051207/540029968d7f7261088b49fa/html5/thumbnails/9.jpg)
08/10/2013
9
MATCHING
Matching your groups: why?
70
77 73 75
90 90 95
99
92 90
100
90
0
20
40
60
80
100
120
IQ Vocabulary ComprehensionQuestions
Word Reading SES ComprehensionImperatives
Stan
dar
d S
core
s
SLI Typical
![Page 10: Issues in experimental design for the study of atypical language development](https://reader034.vdocuments.site/reader034/viewer/2022051207/540029968d7f7261088b49fa/html5/thumbnails/10.jpg)
08/10/2013
10
Matching your groups: on what?
Imp.Comp
6 8 10 2 4 6 8
24
68
68
10
IQ
QuestComp
510
15
24
68
SES
2 4 6 8 5 10 15 4 6 8 10
46
810
Voc
Your groups: three group design
• Group of interest
• Control group, matched on chronological age
• Control group, matched on variable of interest
![Page 11: Issues in experimental design for the study of atypical language development](https://reader034.vdocuments.site/reader034/viewer/2022051207/540029968d7f7261088b49fa/html5/thumbnails/11.jpg)
08/10/2013
11
Some issues
• Match on variables of interest to the study
• Match average and distribution
• Study carefully your exclusions
Recruit clinical sample
Decide variables of
matching
Recruit age controls
Recruit “level” controls
Let’s do it! Matching exercise
Some issues
• Match on variables of interest to the study
• Match average and distribution
• Study carefully your exclusions
Recruit clinical sample
Decide variables of
matching
Recruit age controls
Recruit “level” controls
Check and report
![Page 12: Issues in experimental design for the study of atypical language development](https://reader034.vdocuments.site/reader034/viewer/2022051207/540029968d7f7261088b49fa/html5/thumbnails/12.jpg)
08/10/2013
12
What statistical test?
An ideal world
Facon et al. (2011)
What statistical test?
An ideal world
![Page 13: Issues in experimental design for the study of atypical language development](https://reader034.vdocuments.site/reader034/viewer/2022051207/540029968d7f7261088b49fa/html5/thumbnails/13.jpg)
08/10/2013
13
What statistical test?
An ideal world The real world
Non-normal distributions Different range and variance Different distributions
What statistical test?
None: look at the picture first!
Back-to-back histogram Back-to-back stem-and-leaf
![Page 14: Issues in experimental design for the study of atypical language development](https://reader034.vdocuments.site/reader034/viewer/2022051207/540029968d7f7261088b49fa/html5/thumbnails/14.jpg)
08/10/2013
14
What statistical test?
What statistical test?
Mann-Whitney?
Two-sample Kolmogorov-Smirnov test
For small sample sizes: exact version
• Does not assume prior “shape” in distributions
• Tests for the differences in distribution
![Page 15: Issues in experimental design for the study of atypical language development](https://reader034.vdocuments.site/reader034/viewer/2022051207/540029968d7f7261088b49fa/html5/thumbnails/15.jpg)
08/10/2013
15
Matching your groups: how much? The easy-peasy way
Matching your groups: how much?
http://intuitor.com/statistics/T1T2Errors.html
A brief recall of significance testing
Match No Match
I say No Match
I say Match
![Page 16: Issues in experimental design for the study of atypical language development](https://reader034.vdocuments.site/reader034/viewer/2022051207/540029968d7f7261088b49fa/html5/thumbnails/16.jpg)
08/10/2013
16
“Casual acceptance of the null hypothesis” (Harcum, 1990)
< .2 too low
.2 to .5 ambiguous
> .5 fine
Frick (1995)
http://intuitor.com/statistics/T1T2Errors.html
Some things to remember • Population comparisons are group comparisons
(Paradis, 2010)
![Page 17: Issues in experimental design for the study of atypical language development](https://reader034.vdocuments.site/reader034/viewer/2022051207/540029968d7f7261088b49fa/html5/thumbnails/17.jpg)
08/10/2013
17
Some things to remember
• Each group must be there for a reason
(Paradis, 2010)
An example: why each group?
OCS 20 adolescents (13-21) task with sound
OSS 20 adolescents (13-21) task with no sound
NO 20 children (6-11) oral language equivalent to SAL
Two groups of deaf adolescents
SAL 16 deaf (13-21) good oral language
SBL 16 deaf (13-21) poor oral language
Three groups of hearing adolescents
Torres (2013)
![Page 18: Issues in experimental design for the study of atypical language development](https://reader034.vdocuments.site/reader034/viewer/2022051207/540029968d7f7261088b49fa/html5/thumbnails/18.jpg)
08/10/2013
18
Some things to remember
• Check matching while you are recruiting
(Paradis, 2010)
Keep your lab notes up to date
•Some important decisions to record: •Changes and discussions on criteria •Subjects in – subjects out. •Transcriptions and criteria if using MLU
(Paradis, 2010)