introduction to multilevel analysis2 aim of this session •what‘s the problem about multilevel...
TRANSCRIPT
![Page 1: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/1.jpg)
Introduction to Multilevel Analysis
Prof. Dr. Ulrike Cress
Knowledge Media Research Center
Tübingen
![Page 2: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/2.jpg)
2
Aim of this session
• What‘s the problem about multilevel data?
• Options to handle multilevel data in CSCL
Caution: After this presentation you will not be able to do or fully understand
a HLM model
– but you will be aware of all the mistakes you can do!
give you some take-home messages
![Page 3: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/3.jpg)
3
Into to topic ....
„Extraverted children perform better in school“
What may be the reason for that?
What may be the processes behind?
What does this mean statistically?
![Page 4: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/4.jpg)
4
What is the problem about multi-level
data?
![Page 5: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/5.jpg)
5
Example: Effect of Extraversion on Learning Outcome
IV: Extraversion
DV: performance
![Page 6: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/6.jpg)
6
First view on the data
5 2 4
9 14 7
Extraversion 7 8 6
Performance 13 14 13
4 5 4 5
12 12 11 10
Pooled (n=10) r = .26
Aggregated (Mean of the groups; n=3) r = .99
Mean correlation (n=3)
r=.86 r=-.82 r=-.30 r = -0.08
2,00 4,00 6,00 8,00
pre
8,00
10,00
12,00
14,00
post
2,00 4,00 6,00 8,00
pre
8,00
10,00
12,00
14,00
post
2,00 4,00 6,00 8,00
pre
8,00
10,00
12,00
14,00
post
![Page 7: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/7.jpg)
7
Hierarchical data
Individual observations are not independent
![Page 8: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/8.jpg)
8
Question
• What does it statistically mean, if the variance within the
groups is small?
• with regard to standard-deviation?
• with regard to F?
• with regard to alpha?
![Page 9: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/9.jpg)
9
Impact on statistics
• Analysis of Variance: heavily leans on the
assumption of independence of observations
• Underestimation of the standard error
• Large number of spuriously “significant” results
• Inflation of Alpha
within
between
Var
VarF
![Page 10: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/10.jpg)
10
Alpha-Inflation
no. of groups
group size
![Page 11: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/11.jpg)
11
1st take-home message
you are not allowed to use standard statistics
with multi-level data
![Page 12: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/12.jpg)
12
Stochastic Non-Independency
…. is caused by
1. Composition: people of the groups are already
similar before the study even begins
is a problem if you can not randomize
![Page 13: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/13.jpg)
13
Stochastical Non-Independency
…. is caused by
2. Common fate caused through shared experiences
during the experiment
is always a problem in CL
![Page 14: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/14.jpg)
14
Stochastic Non-Independency
…. is caused by
3. Interaction & reciprocal influence
![Page 15: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/15.jpg)
15
Hierarchical data
Intra-class correlation
![Page 16: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/16.jpg)
16
2nd take-home message: Relevance for Learning Sciences
• CL explicitly bases on the idea of creating non-
independency
• We want people to interact, to learn from each
others, etc.
• CL should even aim at considering effects of non-
independency
• if you work on CL-data, you have to consider the
multi-level structure of the data not just as noice
but as an intended effect
![Page 17: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/17.jpg)
17
How to do this adequately?
Possible solutions
1. Working with fakes
2. Groups as unit of analysis
3. Slopes as outcomes
4. Hierarchical linear analysis (HLM)
5. Fragmentary (but useful) solutions
![Page 18: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/18.jpg)
18
Solution 1: Working with fake
confederates bogus feedback
fake
fake
fake
classical experiment: conformity study Asch (1950)
![Page 19: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/19.jpg)
19
Solution 1: Working with fake
Pros:
• well established method in social psychology
• high standardization
• situation makes people behaving like being in a group, but it leads
to statistically independent data
• causality
• sometimes easy to do in CSCL anonymity
![Page 20: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/20.jpg)
20
Solution 1: Working with fake
Cons:
• artificial situation
• no flexibility
• only simple action-reaction pairs can be faked. No real process of
reciprocal interaction
• non dynamics
![Page 21: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/21.jpg)
21
Solution 2: Unit of Analysis
• Group level: Aggregated data
M(x) M(y) M(x) M(y) M(x) M(y) M(x) M(y)
Pros:
• statistically independent measures
Cons:
• need of many groups
• waste of data
• results not valid for individual level Robinson - Effect
![Page 22: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/22.jpg)
22
Robinson-Effect (1950)
• illiteracy level in nine geographic regions (1930)
• percentage of blacks (1930)
regions r = 0.95
individuals r = 0.20
Ecological Fallacy: inferences about the nature of
specific individuals are based solely upon aggregate
statistics collected for the group to which those
individuals belong.
Problem: Unit of analysis
![Page 23: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/23.jpg)
23
3rd take-home message
You can use group-level data
- but the results just describe the groups, not the individuals
![Page 24: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/24.jpg)
24
Solution 2: Unit of Analysis
• Individual level: centering around the group mean
/ standardization elimination of group effects
x-M(x) ...
y-M(y) …
M(x) M(y) M(x) M(y) M(x) M(y) M(x) M(y)
x-M(x) ...
y-M(y) …
x-M(x) ...
y-M(y) …
x-M(x) ...
y-M(y) …
![Page 25: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/25.jpg)
25
Solution 2: Unit of Analysis
Pros:
• easy to do
• makes use of all data of the individual level
Cons:
• works only, if variances are homogeneouos
(centering)
• loss of information about heterogeneous
variances (standardization)
• differences between groups are just seen as
error-variance
![Page 26: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/26.jpg)
26
Solution 3: Slopes as Outcomes
performance y
.
.
.
.
.
.
.
.
.
.
. .
.
.
.
Team 2 y=ax+b
Team 1 y=ax+b
Extraversion x
Burstein, 1982
![Page 27: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/27.jpg)
27
Solution 3: Slopes as Outcomes
Pros:
• uses all information
• focus is on interaction effects between group-
level (team) and individual-level variable
Cons:
• descriptive
• just comparing the groups which are given no
random-effects are considered
![Page 28: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/28.jpg)
28
4th take-home message
Consider the slopes of the different groups.
They show group effects!
e.g. it is a feature of the group, if extraverted members are more
effective
slopes describe groups
slopes are DVs
![Page 29: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/29.jpg)
29
Solution 4: Hierarchical Linear Model Bryk & Raudenbush, 1992
the groups (you have data from) represent a randomly
choosen sample of a population of groups! (random effect
model)
Two Main ideas
The slopes and intercepts are systematically varying
variables.
![Page 30: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/30.jpg)
30
Solution 4: Hierarchical Linear Model Bryk & Raudenbush, 1992
performance y
.
.
.
.
.
.
.
.
.
.
. .
.
.
.
Team 2
y=ax+b
Team 1
y=ax+b
extraversion x
variation of slopes variation of intercepts predicted with 2nd level variables
![Page 31: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/31.jpg)
31
Equation system of systematically varying regressions
Level 1: Yij = β0j + β1jXij+ rij
Solution 4: Hierarchical Linear Model Bryk & Raudenbush, 1992
β0j = intercept for group j
b1j = regression slope group j
rij = residual error
y
.
.
.
.
.
. .
. .
. . .
. .
.
x
![Page 32: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/32.jpg)
32
Level 1: Yij = β0j + β1jXij+ rij
Level 2: β0j = g00 + g01Wj + u0j
β1j = g10+ g11Wj + u1j
W = explanatory variable on level 2 e.g. teacher experience
HLM: Equation system
y
.
.
.
.
.
. .
. .
. . .
. .
.
x
![Page 33: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/33.jpg)
33
Level 1: Yij = β0j + β1j Xij+ rij (1)
Level 2: β0j = g00 + g01Wj + u0j (2)
β1j = g10+ g11Wj + u1j (3)
Put (2) and (3) in (1)
Yij = (g00 + g01Wj + u0j) + (g10Xij+ g11WjXij + u1jXij) + rij (4)
Yij = (g00 + g01Wj + g10Xij + g11WjXij ) + (u1jXij+ u0j + rij) (5)
Total model
Fixed part Random (error) part
![Page 34: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/34.jpg)
34
y=performance
W = -1
W = 0
W = +1
g00
g01
Yij = (g00 + g01Wj + g10Xij + g11WjXij ) + (u1jXij+ u0j + rij)
g10
1
g11
1
0
W = group
predictor (e.g.
teacher
experience)
performance
at x=0
influence
teacher exper.
influence
extraversion
cross-level
interaction
random part of
slopes
random
intercept
individuum
residuum
x=extraversion
![Page 35: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/35.jpg)
35
How to do?
Iterative testing of different models
![Page 36: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/36.jpg)
36
Baseline model: null model, intercept-only model
Yij = (g00 + g01Wj + g10Xij + g11WjXij ) + (u1jXij+ u0j + rij)
Yij = g00 + + u0j + rij
Grand Mean Variance
between
groups
residuum
![Page 37: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/37.jpg)
37
g00
randomly varying intercepts
Yij = g00 + u0j + r1ij
u0j
rij
Baseline model: null model or intercept-only model
![Page 38: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/38.jpg)
38
Baseline model: null model, intercept-only model
Yij = (g00 + g01Wj + g10Xij + g11WjXij ) + (u1jXij+ u0j + rij)
Yij = (g00 + + u0j + rij
Grand Mean Variance
between
groups
residuum
which amount of variance is explained through
the groups?
Intraclasscorrelation ICC = Var (uo)
Var (uo)+ Var (rij)
![Page 39: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/39.jpg)
39
2nd model: Random intercept model with first level predictor
Yij = (g00 + g01Wj + g10Xij + g11WjXij ) + (u1jXij+ u0j + rij)
Yij = (g00 + g10Xij + u0j + rij)
first level predictor
We predict the individual measures with a first-level
predictor
![Page 40: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/40.jpg)
40
g00
• randomly varying intercepts; •sampe slope for all groups
2nd model: Random intercept model with first level predictor
u0j rij
g10
1
Yij = g00 +g10Xjj + u0j + rij
![Page 41: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/41.jpg)
41
3rd model: Random intercept model with second-level predictor
Yij = (g00 + g01Wj + g10Xij + g11WjXij ) + (u1jXij+ u0j + rij)
Yij = (g00 + g01Wj + g10Xij + + u0j + rij)
2nd level predictor
We predict the the intercepts with a second-level
predictor
![Page 42: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/42.jpg)
42
3rd model: Random intercept model with second-level predictor
g00
• randomly varying intercepts; • intercepts predicted by W •sampe slope for all groups
g01 rij
g10
1 W
W
W
Yij = g00 + g01Wj + g10Xij + u0j + rij
![Page 43: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/43.jpg)
43
4. Random coefficient-model
Yij = (g00 + g01Wj + g10Xij + g11WjXij ) + (u1jXij+ u0j + rij)
Yij = (g00 + g01Wj + g10Xij + u1jXij + u0j + rij)
Heteroscedasticity
![Page 44: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/44.jpg)
44
4. Random coefficient-model
g01
rij
g10
1
W
W
W
• randomly varying intercepts; • intercepts predicted by W •slope • randomly varying slopes •Variation of the slopes is not predicted
Yij = g00 + g01Wj + g10Xjj + u1jXjj + u0j + rij
g00
u1j
![Page 45: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/45.jpg)
45
5. Context model: cross-level interaction
Cross-level interaction
Yij = (g00 + g01Wj + g10Xij + g11WjXij ) + (u1jXij+ u0j + rij)
Yij = (g00 + g01Wj + g10Xij + g11WjXij) + (u1jXij + u0j + rij)
![Page 46: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/46.jpg)
46
5. Context model: cross-level interaction
g01
rij
g10
1
W
W
W
• randomly varying intercepts; • intercepts predicted by W •slops predicted by W • randomly varying slopes •Variation of the slopes predicted by W
g00
Yij = g00 + g01Wj + g10Xij + g11WjXij+ u1jXij + u1j + rij
g11
![Page 47: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/47.jpg)
47
Pros and Cons of multilevel model
Pros
• deals with ML data
• allows to test group-level influences
• allows to test cross-level interactions
• method would optimally fit to many questions of CL
Instruction group level
collaboration interaction between group members ICC as goal
learning learning as individual variable
IV DV Process
![Page 48: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/48.jpg)
48
Pros and Cons of multilevel model
Cons
• sometimes difficult to specify
• needs many data
bottleneck for CL
![Page 49: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/49.jpg)
49
5th take-home message
Do not test the whole model, but do it iteratively
(1) test, if the groups significantly differ
(2) explain the difference of the intercepts with group-level predictors
(3) test if the slope significanly differ
(4) explain the difference of the slopes with group-level predictors
(5) test if there is a cross-level interaction
![Page 50: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/50.jpg)
50
Required sample size
• 30/30 rule (Kreft, 1996): ok for interest in fixed
parameters
• accurate group level variance estimates: 6-12 groups
(Brown & Draper, 2000)
• 10 groups: variance estimates are much too small
(Maas & Hox, 2001)
• if interest is in cross-level interactions: 50/20
• if interest is in the random part: 100/20
see Hox, J. (2002), p. 175
![Page 51: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/51.jpg)
51
Multilevel Articles in CSCL
• Strijbos, Martens, Jochems, & Broers, Small Group Research 2004
33 students (10 groups); usefulness of roles on group efficiency
• Schellens, Van Keer & Martin Valcke, Small Group Research, 2005
286 students (23 groups); measurement occasions within
students; roles in groups
• Piontkowski, Keil & Hartmann, Analyseebenen und Dateninter-
dependenz in der Kleingruppenforschung am Beispiel netzbasierter
Wissensintegration; Zeitschrift für Sozialpsychologie, 2006
120 students (40 groups); sequenzing tool; amount of discussion
in a group
![Page 52: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/52.jpg)
52
Take home messages
• be aware of group effects
• think about working with fakes
• think about groups as unit of analysis
• look for the variances! heterogeneous variances
can be a sign for group effects
• look for different slopes!
• try to explain slopes
• look for the ICC
![Page 53: Introduction to Multilevel Analysis2 Aim of this session •What‘s the problem about multilevel data? •Options to handle multilevel data in CSCL Caution: After this presentation](https://reader034.vdocuments.site/reader034/viewer/2022050518/5fa1dcf37540bd1fc31c526d/html5/thumbnails/53.jpg)
53
Questions?