topic and role discovery in social networks
DESCRIPTION
Topic and Role Discovery In Social Networks. Review of Topic Model. Review of Joint/Conditional Distributions. What do the following tell us: P( Z i ) P( Z i | {W,D}) P( Z i , Z j | {W,D}). Extending The Topic Model. Topic Model spawned gobs of research e.g., visual topic models - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/1.jpg)
Topic and Role DiscoveryIn Social Networks
![Page 2: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/2.jpg)
Review of Topic Model
![Page 3: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/3.jpg)
Review of Joint/Conditional Distributions
What do the following tell us:
P(Zi)
P(Zi | {W,D})
P(θ | {W,D})
![Page 4: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/4.jpg)
Extending The Topic Model
Topic Model spawned gobs of researchE.g., visual topic models
Bissacco, Yang, Soatto, NIPS 2006
![Page 5: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/5.jpg)
Today’s Class
Extending topic modeling to social network analysis
Show how research in a field progresses
Show how Bayesian nets can be creatively tailored to tackle specific domains
Convince you that you have the background to read probabilistic modeling papers in machine learning
![Page 6: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/6.jpg)
Social Network Analysis
Nodes of graph are individuals or organizationsLinks represent relationships (interaction, communication)
Graph properties connectedness / distance to other nodes natural clusters / bridge points
Examples interactions among blogs on a topic communities of interest among faculty spread of infections within hospital
![Page 7: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/7.jpg)
9/11 Hijacker Analysis
![Page 8: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/8.jpg)
Indadequacy of Current Techniques
Social network interaction Capture a single type of relationship No attempt to capture the linguistic content of the interactions
Statistical language models (e.g., topic model) Don't capture directed interactions and relationships between individuals
![Page 9: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/9.jpg)
Author Model (McCallum, 1999)
Documents: research articlesad: set of authors associated with document
z: a single author sampled from set (each author discusses a single topic)
![Page 10: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/10.jpg)
Author-Topic Model (Rosen-Zvi,Griffiths, Steyvers, & Smyth, 2004)
Documents: research articlesEach author's interests are modeled by a mixture of topics
x: one author
z: one topic
![Page 11: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/11.jpg)
Can Author-Topic Model Be Applied To Email?
Email: sender, recipient, message bodyCould handle email if
Ignored recipients But discards important information about connections between people
Each sender and recipient wereconsidered an author But what about asymmetry ofrelationship?
![Page 12: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/12.jpg)
Author-Recipient-Topic (ART) Model(McCallum, Corrado-Emmanuel, & Wang, 2005)
Email: sender, recipient, message bodyGenerative model for a word
pick a particular recipient from rd
chose a topic from multinomialspecific to author-recipient pair sample word from topic-specificmultinomial
![Page 13: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/13.jpg)
Review/Quiz
What is a document?
How many values of θ are there?
Can data set be partitioned into subsetsof {author, recipient} pairs and eachsubset is analyzed separately?
What is α?
What is β?
What is form of P(w|z,φ1, φ2, φ3,… φT)?
![Page 14: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/14.jpg)
Author-Recipient-Topic (ART) Model
joint distribution
marginalizing over topics
![Page 15: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/15.jpg)
MethodologyExact inference is not possible
Gibbs Sampling (Griffiths & Steyvers, Rosen-Zvi et al.) variational methods (Blei et al.) expectation propagation (Griffiths & Steyvers,Minka & Lafferty)
McCallum uses Gibbs sampling of latent variables latent variables: topics (z), recipients (x) basic result:
![Page 16: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/16.jpg)
Derivation
Want to obtain posterior over z and x given corpus
![Page 17: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/17.jpg)
nijt: # assignments of topic t to author i with recipient j
mtv : # occurrences of (vocabulary) word v to topic t
is conjugate prior of
is conjugate prior of
![Page 18: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/18.jpg)
Data Sets
Enron 23,488 emails 147 users 50 topics
McCallum email 23,488 emails 825 authors, sent or received by McCallum 50 topics
Hyperpriors α = 50/T β = .1
![Page 19: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/19.jpg)
Enron DataHuman-generated label
three author/recipient pairswith highest probability for discussing topic
Hain: in house lawyer
![Page 20: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/20.jpg)
Enron Data
Beck: COODasovich: Govt RelationsSteffes: VP Govt. Affairs
![Page 21: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/21.jpg)
McCallum's Email
![Page 22: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/22.jpg)
Social Network Analysis
Stochastic Equivalence Hypothesis Nodes that have similar connectivity must have similar roles e.g., in email network, probability that one node communicates with other nodes
How similar are two probability distributions? Jensen-Shannon divergence = measure of dissimilarity
1/JSDivergence = measure of similarity
For ART, use recipient-marginalized topic distribution
DKL
![Page 23: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/23.jpg)
Predicting Role EquivalenceBlock structuring JS divergence matrix
SNA
ART AT
#9: Geaccone: executive assistant#8: McCarty: VP
![Page 24: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/24.jpg)
Similarity Analysis With McCallum Email
![Page 25: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/25.jpg)
Role-Author-Recipient Topic (RART) Model
Person can have multiple roles e.g., student, employee, spouse
Topic depends jointly on roles of author and recipient
![Page 26: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/26.jpg)
New Topic!
If you have 50k words, you need 50k free parameters to specify topic-conditioned word distribution.
For small documents, and small data bases, the data don’t constrain the parameters.
Priors end up dominatingCan we exploit the fact that words aren’t just strings of letters but have semantic relations to one another?
Bamman, Underwood, & Smith (2015)
![Page 27: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/27.jpg)
Distributed Representations Of Words
Word2Vec scheme for discovering word embeddings
Count # times other words occur in the context of some word W
Vector with 50k elements
Do dimensionality reduction on these vectors to get compact, continuous vector representation of W
Captures semantics
![Page 28: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/28.jpg)
Distributed Representations Of Words
Perform hierarchical clustering on word embeddingsLimit depth of hierarchical clustering tree
(Not exactly what authors did, but this seems prettier.)
![Page 29: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/29.jpg)
Distributed Representation Of Words
Each word is described by a string of 10 bitsBits are ordered such that most-significant bit represents root of hierarchical clustering tree
![Page 30: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/30.jpg)
Generative Model For Word
P(W) = P(B1)P(B2|B1)P(B3|B1:2) … P(B10|B1:9)
where the distributed representation of W is(B1, …, B10)
How many free parameters are required to represent word distribution?
1023 vs. 50k for complete distribution
![Page 31: Topic and Role Discovery In Social Networks](https://reader035.vdocuments.site/reader035/viewer/2022070504/5681682e550346895dddd06d/html5/thumbnails/31.jpg)
Generative Model For Word
P(W|T) = P(B1|T)P(B2|B1,T)P(B3|B1:2,T) … P(B10|B1:9,T)
Each topic will have 1023 parameters associated with the word distribution.
What’s the advantage of using the bitstring representation instead of simplyspecifying a distribution over the 1024leaf nodes directly?
Leveraging priors