the power of certainty - carnegie mellon …deswaran/talks/sdm17-netconf-talk.pdf · the power of...

A DIRICHLET-MULTINOMIAL APPROACH TO BELIEF PROPAGATION

THE POWER OF CERTAINTY

Dhivya Eswaran* CMU

[email protected]

Stephan Guennemann TUM

[email protected]

Christos Faloutsos CMU

[email protected]

mailto:[email protected]?subject=

mailto:[email protected]?subject=

mailto:[email protected]

PROBLEM

MOTIVATION

NETCONF

GUARANTEES

EXPERIMENTSEswaran, Guennemann & Faloutsos

PROBLEM

ADS PLACEMENT

ALICE

BOB

CAROL

SMITH

RECOMMENDATION

ALICE

BOB

CAROL

SMITH

JOHN

THE POWER OF CERTAINTY A DIRICHLET-MULTINOMIAL MODEL FOR BELIEF PROPAGATION

Eswaran, Guennemann & Faloutsos

GRAPH LABELING / NODE CLASSIFICATIONEXPERIMENTS

4

GIVEN

‣ a graph of nodes & edges

‣ labels for a few nodes

‣ label compatibility

FIND

‣ labels of all nodes

ALICE BOB

CAROL

SMITH

JOHN

H0.8 0.2

0.2 0.8

H0.2 0.7 0.1

0.7 0.2 0.1

0.1 0.1 0.8



NODE CLASSIFICATION IS HARDEXPERIMENTS

5

SMITH

4 x

1 x

JOHN

30 x

15 x

Who is more likely to buy an Android phone?

“Higher fraction of android friends”

“Higher number of android friends”

PROBLEM

MOTIVATION

NETCONF

GUARANTEES


MOTIVATION



CLASSIFICATION BY “PROPAGATION”EXPERIMENTS

7

INITIALIZE:

‣ Set nodes to random/given values.

PROPAGATE:

‣ Update each node’s value based on the values of its neighbors.

CONVERGENCE:

‣ If no value changes, terminate.

‣ Else continue propagation.

SMITH

ALICE

BOB

CAROL

Q1. What are values here?

Q2. How are they updated?



“BELIEF” PROPAGATIONEXPERIMENTS

8

SMITH

ALICE

BOB

CAROL

A1. Values : beliefs

A2. Update : 2 stages

(probability vectors)

(i) Each neighbor sends a message

(ii) Node updates its belief based on messages

[0.4, 0.6]

[0.5, 0.5]

[0.8, 0.2]

[0.73, 0.27] bu(i) eu(i)Y

v2N (u)

mvu(i)

mvu(i) kX

j=1

H(i, j)ev(j)Y

v2N (v)\u

mwu(i)

DETAILS!



BP LEADS TO COUNTER-INTUITIVE RESULTSEXPERIMENTS

9

Who is more likely to buy an Android phone?

110 x

100 xJOHN

13 x

3 xSMITH

INTUITION

BP



MAIN IDEAEXPERIMENTS

10

“belief distributions” as values

10.5

prob

abilit

y

0

belief / leaning(ratio of android : apple neighbors)

certainty / confidence

(absolute count of

neighbors)

PROBLEM

MOTIVATION

NETCONF

GUARANTEES


NETCONF



NODE CLASSIFICATION WITH CERTAINTYEXPERIMENTS

12

GIVEN

‣ a graph of nodes & edges

‣ belief distributions for a few nodes

‣ label compatibility

FIND

‣ belief distributions of all nodes

SUBJECT TO

‣ theoretically-grounded algorithm

‣ fast & scalable implementation

H0.8 0.2

0.2 0.8

ALICE BOB

CAROL

SMITH

JOHN



DIRICHLET BELIEF DISTRIBUTIONS (1/2)EXPERIMENTS

13

2D Dirichlet distribution (Beta distribution)

p(x;↵+ 1,� + 1) / x

↵(1� x)�

Belief/Leaning : ↵+ 1

↵+ � + 2

BACKGROUND

# #



DIRICHLET BELIEF DISTRIBUTIONS (2/2)EXPERIMENTS

14

Certainty:↵+ �

EXTENDS TO ANY NUMBER OF DIMENSIONS!!

Image source: UBC Wiki

BACKGROUND

Example: 3 dimensions (talkative / silent / confused)



MULTINOMIAL MESSAGE DISTRIBUTIONSEXPERIMENTS

15

BP belief update rule

parameters of belief distribution

parameters of message distribution

NETCONF belief update rule

MULTINOMIAL DISTRIBUTION!

DIRICHLET DISTRIBUTION

DETAILS!

bu(i) eu(i)Y

v2N (u)

mvu(i)

bu eu +X

v2N (u)

mvu



MESSAGES FROM BELIEFSEXPERIMENTS

16

JOHNSMITH

(a, b)

(a, b)

perfect homophily

(b, a)

perfect heterophily

M =k

k � 1

✓H� 1

k

◆+

(0, 0)

no network effects



NETCONF BELIEF & MESSAGE UPDATE RULESEXPERIMENTS

17

message update

belief update

M

bu eu +X

v2N (u)

mvu

mvu M

0

@eu +X

v2N (v)\u

mwu

1

A

muvbu



NETCONF MATRIX BELIEF UPDATEEXPERIMENTS

18

modulation

B E+ (ABM�DBM2)(I�M2)�1

prior belief distribution

final belief distribution

~~ ~~~~

diagonal degreeadjacency

~~ ~~~~ ~~

~ ~~ ~

~ ~~ ~

PROBLEM

MOTIVATION

NETCONF

GUARANTEES


GUARANTEES



KEY THEORETICAL QUESTIONSEXPERIMENTS

20

UNIQUENESS

CONVERGENCE

Is the steady state solution unique?

Can we predict convergence?



NETCONF HAS CLOSED-FORM SOLUTIONEXPERIMENTS

21

ITERATIVE UPDATE

CLOSED FORM

Roth’s Column Lemma

DETAILS!

vec(B) =⇣I� (MM)T ⌦A+ (M2M)T ⌦D

⌘�1vec(E)

B E+ (ABM�DBM2)(I�M2)�1



PRECISE CONVERGENCE GUARANTEES

22

GRAPH STRUCTURE LABEL COMPATIBILITY

CLOSED FORM

NECESSARY &

SUFFICIENT CONDITION

DETAILS!

⇢⇣(MM)T ⌦A+ (M2M)T ⌦D

⌘< 1

vec(B) =⇣I� (MM)T ⌦A+ (M2M)T ⌦D

⌘�1vec(E)

GUARANTEES



KEY THEORETICAL GUARANTEESEXPERIMENTS

23

CLOSED-FORM

CONVERGENCE

Closed-form solution and unique fixed point!

Necessary and sufficient conditions for convergence!

PROBLEM

MOTIVATION

NETCONF

GUARANTEES


EXPERIMENTS



KEY QUESTIONS FOR EXPERIMENTSEXPERIMENTS

25

EFFECTIVENESS

SCALABILITY

INTERPRETABILITY

Improves accuracy & precision?

Fast and scalable?

Are final results interpretable?



DATAEXPERIMENTS

26

POLBLOGS

2 classes

1.5K nodes, 19K edges

DBLP

4 classes

28K nodes, 67K edges

POKEC

10 classes

1.6M nodes, 30M edges



NETCONF IS ACCURATE AND PRECISEEXPERIMENTS

27

HIGHER ACCURACY %

DATASET BP NETCONF

POLBLOGS 91.38 92.40

DBLP 76.26 81.89

POKEC 73.78 75.02

BETTER PRECISION

Ideal



NETCONF IS FAST AND SCALABLEEXPERIMENTS

28

30M edges in ~7 seconds!

Linear scaling with graph size



NETCONF GIVES INTERPRETABLE RESULTSEXPERIMENTS

29

AUTHOR H-index

Michael J Carey 48

Rakesh Agrawal 96

Jiawei Han 139

Hamid Pirahesh 40

David J Dewitt 81

Serge Abiteboul 77

AUTHOR H-index

Jiawei Han 139

Annie W Shum -

Werner Keibling -

Xiaofang Zhou 36

Bertram Ludascher 45

Amarnath Gupta -

TOP DATABASES AUTHORS IN DBLP

Many papers &high H1 indices!

BPNETCONF



KEY EXPERIMENTAL FINDINGSEXPERIMENTS

30

EFFECTIVENESS

SCALABILITY

INTERPRETABILITY

Improves accuracy & precision!

Scales linearly with graph size!

Certainty scores reflect intuition!



SUMMARY: NETCONFSUMMARY

31

✓Theoretically grounded

✓Closed-form solution

✓Convergence guarantees

✓Improved performance

✓Fast and scalable

✓Interpretable results

Questions? [email protected]

mailto:[email protected]

the power of certainty - carnegie mellon …deswaran/talks/sdm17-netconf-talk.pdf · the power of...

Documents