Judea Pearl

Computer Science Department

UCLA

www.cs.ucla.edu/~judea

DIRECT AND INDIRECT EFFECTS

2005

QUESTIONS ASKED

• Why decompose effects?• What are the semantics of direct and indirect

effects (in nonlinear and nonparametric models)?• What are the policy implications of direct and

indirect effects?• When can direct and indirect effects be estimated

consistently from experimental or nonexperimental data?

• Can these conditions be verified from accessible causal knowledge, i.e., graphs?

1. Direct (or indirect) effect may be more transportable.2. Indirect effects may be prevented or controlled.

3. Direct (or indirect) effect may be forbidden

WHY DECOMPOSEEFFECTS?

Pill

Thrombosis

Pregnancy

+

+

Gender

Hiring

Qualification

EFFECT-DECOMPOSITIONIN LINEAR MODELS

X Z

Y

ca

b

effect Indirect effect Direct effect Total

a bc

Definition: ))(),(|( zxYEx

a do do

CAUSAL MODELS AND COUNTERFACTUALS

Definition: A causal model is a 3-tupleM = V,U,F

(i) V = {V1…,Vn} endogenous variables,(ii) U = {U1,…,Um} background variables (unit)(iii) F = set of n functions,

The sentence: “Y would be y (in unit u), had X been x,”denoted Yx(u) = y, is the solution for Y in a mutilated model Mx, with the equations for X replaced by X = x. (“unit-based potential outcome”)

),( uvfv ii

COUNTERFACTUALS:STRUCTURAL SEMANTICS

Notation: Yx(u) = y Y has the value y in the solution to a mutilated system of equations, where the equation for X is replaced by a constant X=x.

u

Yx(u)=y

Z

W

X=x

u

Y

Z

W

X

Probability of Counterfactuals:

FunctionalBayes Net

))(|()())(()( xdoyPu

uPyuxYPyxYP

)(uM,P

tindependen- ))(),(|(

))(|(

DETEIE

ZzdoxdoYEx

DE

xdoYEx

TE

TOTAL, DIRECT, AND INDIRECT EFFECTS HAVE CONTROLLED-BASED

SEMANTICS IN LINEAR MODELS

X Z

Y

ca

b z = bx + 1

y = ax + cz + 2

a + bc

bc

a

z = f (x, 1)y = g (x, z, 2)

????

))(),(|(

))(|(

IE

zdoxdoYEx

DE

xdoYEx

TE

X Z

Y

CONTROLLED-BASED SEMANTICS NONTRIVIAL IN NONLINEAR MODELS(even when the model is completely specified)

Dependent on z?

Void of operational meaning?

``The central question in any employment-discrimination case is whether the employer would have taken the same action had the employee been of different race (age, sex, religion, national origin etc.) and everything else had been the same’’

[Carson versus Bethlehem Steel Corp. (70 FEP Cases 921, 7th Cir. (1996))]

x = male, x = femaley = hire, y = not hirez = applicant’s qualifications

LEGAL DEFINITIONOF DIRECT EFFECT

(FORMALIZING DISCRIMINATION)

NO DIRECT EFFECT

',' ' xxxx YYYYxZxZ

z = f (x, u)y = g (x, z, u)

X Z

Y

NATURAL SEMANTICS OFAVERAGE DIRECT EFFECTS

Average Direct Effect of X on Y:The expected change in Y, when we change X from x0 to x1 and, for each u, we keep Z constant at whatever value it attained before the change.

In linear models, DE = Controlled Direct Effect

][001 xZx YYE

x

);,( 10 YxxDE

Robins and Greenland (1992) – “Pure”

POLICY IMPLICATIONS(Who cares?)

f

GENDER QUALIFICATION

HIRING

What is the direct effect of X on Y?

Is employer guilty of sex-discrimination given data on (X,Y,Z)?

X Z

Y

CAN WE IGNORE THIS LINK?

tYYE xZxx

][0

01

SEMANTICS AND IDENTIFICATION OF NESTED COUNTERFACTUALS

Consider the quantity

Given M, P(u), Q is well defined

Given u, Zx*(u) is the solution for Z in Mx*, call it z

is the solution for Y in Mxz

Can Q be estimated from data?

Experimental: nest-free expressionNonexperimental: subscript-free expression

)]([ )(*uYEQ uxZxu

entalnonexperim

alexperiment

)()(*uY uxZx

z = f (x, u)y = g (x, z, u)

X Z

Y

NATURAL SEMANTICS OFINDIRECT EFFECTS

Indirect Effect of X on Y:The expected change in Y when we keep X constant, say at x0, and let Z change to whatever value it would have attained had X changed to x1.

In linear models, IE = TE - DE

][010 xZx YYE

x

);,( 10 YxxIE

POLICY IMPLICATIONS(Who cares?)

f

GENDER QUALIFICATION

HIRING

What is the indirect effect of X on Y?

The effect of Gender on Hiring if sex discriminationis eliminated.

X Z

Y

IGNORE

Theorem 5: The total, direct and indirect effects obeyThe following equality

In words, the total effect (on Y) associated with the transition from x* to x is equal to the difference between the direct effect associated with this transition and the indirect effect associated with the reverse transition, from x to x*.

RELATIONS BETWEEN TOTAL, DIRECT, AND INDIRECT EFFECTS

);*,()*;,()*;,( YxxIEYxxDEYxxTE

Is identifiable from experimental data and is given by

Theorem: If there exists a set W such that

EXPERIMENTAL IDENTIFICATION OF AVERAGE DIRECT EFFECTS

zw

xzxxz wPwzZPwYEwYEYxxDE,

** )()|()|()|()*;,(

Then the average direct effect

)(,*;, ** xZx YEYEYxxDEx

xzWZY xxz and all for |*

HOW THE PROOF GOES?

)(,*;, ** xZx YEYEYxxNDEx

xzWZYW xxz and all for If |*

Proof:

)()|(

),|()(

*

*, *

wWPwWzZP

wWzZYEYE

x

w zxxzZx x

)()|(

)|()(

*

, *

wWPwWzZP

wWYYEYE

x

w zxzZx x

Each factor is identifiable by experimentation.

Example:

Theorem: If there exists a set W such that

GRAPHICAL CONDITION FOR EXPERIMENTAL IDENTIFICATION

OF DIRECT EFFECTS

zw

xzxxz wPwzZPwYEwYEYxxDE,

** )()|()|()|()*;,(

)()|( ZXNDWWZYXZG and

then,

GRAPHICAL CONDITION FOR NONEXPERIMENTAL IDENTIFICATION

OF AVERAGE NATURAL DIRECT EFFECTS

Identification conditions1. There exists a W such that (Y Z | W)GXZ

2. There exist additional covariates that render all counterfactual terms identifiable.

zw

xzxxz wPwzZPwYEwYEYxxDE

,** )()|()|()|(

)*;,(

Corollary 3:The average direct effect in Markovian models is identifiable from nonexperimental data, and it is given by

where S stands for any sufficient set of covariates.

IDENTIFICATION INMARKOVIAN MODELS

X ZExample:S =

Y

s z

sPsxzPzxYEzxYEYxxDE )()*,|()*,|(),|()*;,(

z

xzPzxyEzxYEYxxDE *)|()*,|(),|()*;,(

Y

Z

X

W

x*

z* = Zx* (u)

Nonidentifiable even in Markovian models

GENERAL PATH-SPECIFICEFFECTS (Def.)

)),(*),(();,(* ugpagpafgupaf iiiii

*);,();,( **gMMg YxxTEYxxE

Y

Z

X

W

Form a new model, , specific to active subgraph g*gM

Definition: g-specific effect

SUMMARY OF RESULTS

1. Formal semantics of path-specific effects, based on signal blocking, instead of value fixing.

2. Path-analytic techniques extended to nonlinear and nonparametric models.

3. Meaningful (graphical) conditions for estimating direct and indirect effects from experimental and nonexperimental data.

4. Estimability conditions hold in Markovian models.5. Graphical techniques of inferring effects of

policies involving signal blocking.