sem 2: structural equation modeling - sacha...

Introduction Causal Modeling Expectation Algebra Covariance Algebra

SEM 2: Structural Equation ModelingWeek 1 - Causal modeling and SEM

Sacha Epskamp

03-05-2018


Course Overview

• Tuesdays: Lecture

• Thursdays: Unstructured practicals

• Three assignments• First two 20% of final grade, last 10% of final grade

• Final project• Presentations and report, 50% of final grade


Schedule“Week 1” – Introduction to Structural Equation Modeling

• Thursday May 3 – Lecture + practical

• Tuesday May 8 – Lecture + practical

Week 2 – Causality and equivalent models

• Tuesday May 15 – Lecture + deadline assignment 1

• Thursday May 17 - Practical

Week 3 – Time-series analysis and network models

• Tuesday May 22 – Lecture + deadline assignment 2

• Thursday May 24 - Free time to work on final project

Week 4 – Wrap-up and presentations

• Tuesday May 29 – Recap + practical + (presentations) +deadline assignment 3

• Thursday May 31 - Presentations

• Friday June 1 - deadline final project report


Individual AssignmentsEach week, the assignment will be made available 15:00 onTuesday, and will be due 15:00 the next Tuesday. Each assignmentwill contribute to 20% or 10% (last week) of your grade.

• Work on the assignments alone.

• Hand in a PDF file and an .R file (in case R was used). If youuse Jasp, hand in the Jasp object as well as a screenshot ofthe options used.

• Make sure your PDF report is as standalone readable aspossible. E.g., if you are asked to report a factor loadingmatrix, then report it in the PDF and not just say “look at .Rfile”.

• Assignments are due before 15:00. If you do not hand in anassignment before 15:00, you will get a 1.

• If you encounter any problems, or have any feedback, pleaselet me know before the deadline, as then I can take it intoaccount or help you.


Final Project

Three options:

1. Perform a SEM analysis on your own data and write a report(individual)

2. Write a manual for semPlot, Onyx, Jasp or Lavaan (individualor with a partner)

3. Research an area or a topic of SEM in more detail and teachfellow students about it

See syllabus on blackboard

• Claim your project using the discussion board on blackboardas soon as possible!

• If you have another idea on a project not listed above, talk tome


Causal modeling

• This course will introduce structural equation modeling (SEM)

• In SEM, we will discuss modeling complex causal hypotheses

• Again, all variables are assumed normally distributed and allassociations are assumed linear

• Causal hypotheses can be specified between observed andlatent variables

• CFA is a special case of SEM


Causal models

X Y

X causes Y


Endogenous and exogenous

• Exogenous (independent) variables are variables of which thecausal origin are not modeled

• Exogenous variables have a variance (sometimes not drawn)• Exogenous variables, except residuals, are allowed to covary

(sometimes not drawn)• Latents: ξ (xi); observed: x (x is also used for indicators of

latent exogenous variables)• Residuals are exogenous

• Endogenous (dependent) variables are variables of which thecausal origin are modeled

• Simply stated: endogenous variables have incoming arrows• Endogenous variables do not have a variance by themselves• Latents: η (eta); observed: y• The causal equation for endogenous variables can be derived

from the path diagram by summing all incoming edges


X Y

X is exogenous, Y is endogenous


X Y

yi = xi

Causal effect goes from right hand side to left hand side.Experimentally changing x will change y , experimentally changingy will not change x


βX Y

yi = βxi


βX Y ε

yi = βxi + εi


Exogenous variables have a variance (often not drawn)

βσx2 θX Y ε

yi = βxi + εi

x ∼ N(µx , σx)

ε ∼ N(0, θ)


yi = βxi + εi

x ∼ N(µx , σx)

ε ∼ N(0, θ)

Three observations (variance of x and y and covariance between xand y), three unknowns. Solvable!

Var(x) = σ2x

Var(y) = β2σ2x + θ

Cov(x , y) = βσ2x

In addition, we can derive that the expected value (mean) of ybecomes:

E(y) = βE(x).

But why?


Expected Values

Suppose someone offers you to play the following game: You throwa dice, if you throw a 6 you win 6 Euro, but if you throw any othernumber you pay 1 Euro. Would it, statistically, be smart to playthis game?

Given a 1/6 probability to win 6 Euro and a 5/6 probability to lose1 Euro, we expect to win:

6× 1

6− 1× 5

6= 0.166 . . .

We should play this game!


Expected Values

Suppose someone offers you to play the following game: You throwa dice, if you throw a 6 you win 6 Euro, but if you throw any othernumber you pay 1 Euro. Would it, statistically, be smart to playthis game?Given a 1/6 probability to win 6 Euro and a 5/6 probability to lose1 Euro, we expect to win:

6× 1

6− 1× 5

6= 0.166 . . .

We should play this game!


Expected Values

Let E(x) denote the expected value (mean) of a random variable xthat can take K different states (e.g, K = 2 when you can win orlose). We can obtain the expected value as follows:

E(x) =K∑

k=1

xk Pr(x = xk).


If you bet on a number and the ball falls on that number, you win35 times your bet! Should you play this game?

Suppose we bet 1 Euro, the expected value is:

1

37× 35− 36

37× 1 = −0.027

So you expect to lose 2.7 cent on average for every bet. This istrue for every possible bet you can place!


If you bet on a number and the ball falls on that number, you win35 times your bet! Should you play this game?Suppose we bet 1 Euro, the expected value is:

1

37× 35− 36

37× 1 = −0.027

So you expect to lose 2.7 cent on average for every bet. This istrue for every possible bet you can place!


Expected ValuesIf x is continuous, k →∞ and Pr(x = xk)→ 0, but we can stillcompute the expected value by using the density function f (x) andintegration:

E(x) =

∫Rxf (x) dx .

More general, for any function g(x), we can obtain:

E(g(x)) =

∫Rg(x)f (x) dx .

With this we can e.g. proof that if x ∼ N(µ, σ2x):

E (x) = µ

Var(x) = E((x − µ)2

)= σ2x

For example, see: https://www.statlect.com/probability-distributions/normal-distribution


Expected Values

From:

E(g(x)) =

∫Rg(x)f (x) dx .

we can derive the following rules:

E(α) = α

E(αx) = αE(x)

E(αx + β) = αE(x) + β

E(x + y) = E(x) + E(y)

E(αx + βy) = αE(x) + βE(y)

Where α and β are constants (parameter) and x and y are randomvariables.


Example, the sample mean is defined as:

x̄ =1

n

n∑i=1

xi

we can now derive that, if y ∼ N(µ, σ2x), this is a good estimatefor µ:

E (x̄) = E

(1

n

n∑i=1

xi

)

=1

nE

(n∑

i=1

xi

)

=1

nnE (x)

= µ


βσx2 θX Y ε

yi = βxi + εi ; x ∼ N(µx , σx); ε ∼ N(0, θ)

We can now derive:

E (y) = E (βx + ε)

= E (βx) + E (ε)

= βE (x)


Expected ValuesMultivariate generalizations are straightforward:

E(AAA) = AAA

E(AAAxxx) = AAAE(xxx)

E(xxxBBB) = E(xxx)BBB

E(AAAxxx +BBB) = AAAE(xxx) +BBB

E(xxx + yyy) = E(xxx) + E(yyy)

Where AAA and BBB are constant (parameter) matrices. For example,given E(ηηη) = ααα:

yyy i = λλληηηi + εεεi

E(yyy) = E(λλληηη + εεε)

E(yyy) = λλλE(ηηη) + E(εεε)

E(yyy) = λλλααα


Given that Var(x) = E((x − µx)2

)and E (x) = µx = −0.027 for

betting 1 Euro in Roulette. What is the variance and standarddeviation of our bet?

Var (bet 1 Euro) =1

37× (35−−0.027)2 +

36

37× (1−−0.027)2

= 34.19

SD (bet 1 Euro) =√Var (bet 1 Euro) = 5.85


Given that Var(x) = E((x − µx)2

)and E (x) = µx = −0.027 for

betting 1 Euro in Roulette. What is the variance and standarddeviation of our bet?

Var (bet 1 Euro) =1

37× (35−−0.027)2 +

36

37× (1−−0.027)2

= 34.19

SD (bet 1 Euro) =√

Var (bet 1 Euro) = 5.85


Covariance Algebra

Let Var(x) indicate “the variance of x” and Cov(x , y) indicate“the covariance between x and y”. Given thatCov(x , y) = E ((x − µx)(y − µy )), the following rules can bederived:

Var(x) = Cov(x , x)

Cov(x , α) = 0

Cov(x , y) = Cov(y , x)

Cov(αx , βy) = αβCov(x , y)

Cov(x + y , z) = Cov(x , z) + Cov(y , z)

Where α and β are constants (parameter) and x , y , and z arerandom variables.


Covariance Algebra

Some consequences:

Cov(αx + βy , z) = Cov(αx , z) + Cov(βy , z)

= αCov(x , z) + βCov(y , z)

Var(x + y) = Var(x) + Var(y) + 2Cov(x , y)

Var(βx) = β2Var(x)

Where α and β are constants (parameter) and x , y , and z arerandom variables.


Matrix Covariance Algebra

Let Var(xxx) indicate “the variance–covariance matrix of vector xxx”and Cov(xxx ,yyy) indicate “the covariance matrix between xxx and yyy”.Then the following rules can be derived:

Var(xxx) = Cov(xxx ,xxx)

Cov(AAAxxx ,BBByyy) = AAACov(xxx ,yyy)BBB>

Var(BBBxxx) = BBBVar(xxx)BBB>

Cov(xxx + yyy ,zzz) = Cov(xxx ,zzz) + Cov(yyy ,zzz)

Where AAA and BBB are constant (parameter) matrices.


yi = βxi + εi

Var(x) = σ2x

Var(y) = Var(βx + ε)

= Cov(βx + ε, βx + ε)

= Cov(βx , βx + ε) + Cov(ε, βx + ε)

= Cov(βx , βx) + Cov(βx , ε) + Cov(ε, βx) + Cov(ε, ε)

But since x is not correlated with the residuals, Cov(x , ε) = 0 andthus:

Var(y) = β2Cov(x , x) + Cov(ε, ε)

= β2Var(x) + Var(ε)


yi = βxi + εi

Cov(x , y) = Cov(x , βxi + εi )

= Cov(x , βxi ) + Cov(x , εi )

= βCov(x , xi )

= βVar(x)


Path analysis

β1 β2

θ1 θ2

x y1 y2

x is exogenous, and both y1 and y2 are endogenous. θ1 is thevariance of ε1. Causal model for y2:

yi2 = β2yi1 + εi2

yi2 = β2(β1xi + εi1) + εi2


β1 β2

θ1 θ2

x y1 y2

Number of parameters: 2 regressions +2 residual variances +1exogenous variance (not drawn) = 5, number of observations: 3variances and 3 covariances. 1 degree of freedom!


β1 β2

θ1 θ2

x y1 y2

Implied covariance between x and y2:

Cov (x , y2) = Cov (x , β2(β1xi + εi1) + εi2)

= Cov (x , β2β1x + β2ε1 + ε2)

= Cov (x , β2β1x) + Cov (x , β2ε1) + Cov (x , ε2)

= β1β2Cov (x , x)

= β1β2σx


Practical: identify exogenous (x) and endogenous (y) variables,and derive expressions for expected values and (co)variancesof/between all variables, in terms of parameters andexpectations/(co)variances of exogenous variables (x andresiduals):

β1

β2

β1

β2

β1

β2

β3

Also think about the final project!

sem 2: structural equation modeling - sacha...

Documents