discourse coherence neural networks...
TRANSCRIPT
Neural Networks for Discourse Coherence
Roy Aslan, Dwayne Campbell, Chris Kedzie
Task
1. blah blah... 2. blah blah...3. blah… 4. ...5. ...6. ...
2. blah blah...4. ...5. ...1. blah blah... 3. blah… 6. ...
rank( ) rank( )
Task
1. blah blah... 2. blah blah...3. blah… 4. ...5. ...6. ...
2. blah blah...4. ...5. ...1. blah blah... 3. blah… 6. ...
>
Task 1. blah blah... 2. blah blah...3. blah… 4. ...5. ...6. ...
Task 1. blah blah... 2. blah blah...3. blah… 4. ...5. ...6. ...
s1 s2 s3
s2 s3 s4
s3 s4 s5
s4 s5 s6
Task 1. blah blah... 2. blah blah...3. blah… 4. ...5. ...6. ...
s1 s2 s3
s2 s3 s4
s3 s4 s5
s4 s5 s6
log p(y1= coherent | s1,s2,s3 )
log p(y3= coherent | s3,s4,s5 )
log p(y4= coherent | s4,s5,s3 )
log p(y2= coherent | s2,s3,s4 )
Task 1. blah blah... 2. blah blah...3. blah… 4. ...5. ...6. ...
s1 s2 s3
s2 s3 s4
s3 s4 s5
s4 s5 s6
log p(y1= coherent | s1,s2,s3 )
log p(y3= coherent | s3,s4,s5 )
log p(y4= coherent | s4,s5,s3 )
log p(y2= coherent | s2,s3,s4 )+
+
+
= log p( = coherent)
Task 1. blah blah... 2. blah blah...3. blah… 4. ...5. ...6. ...
s1 s2 s3
s2 s3 s4
s3 s4 s5
s4 s5 s6
log p(y1= coherent | s1,s2,s3 )
log p(y3= coherent | s3,s4,s5 )
log p(y4= coherent | s4,s5,s3 )
log p(y2= coherent | s2,s3,s4 )+
+
+
= log p( = coherent) ≜ rank( )
Task
s1 s2 s3
p(Y | s1,s2,s3 ) ⇒
h1
o
Task
s1 s2 s3
p(Y | s1,s2,s3 ) ⇒
h1
o
● 3 models implemented with this framework
● models vary differ at the sentence layer
Model 1: CBOW Model
w1 w2 w3 w4 wn
Model 1: CBOW Model
w1 w2 w3 w4 wn+ + + ...( )n = s
Model 2: Recurrent Model
Model 3: Recursive Model
Results
Results
See our final paper :)
Why is this useful?Discourse might be helpful for non-factoid Question Answering.
``Empirically we show that modeling answer discourse structures is complementary to modeling lexical semantic similarity and that the best performance is obtained when they are tightly integrated.’’
Jansen, Peter, Mihai Surdeanu, and Peter Clark.
"Discourse Complements Lexical Semantics for Non-factoid Answer Reranking."
Application to QA
query
Information RetrievalSystem
Candidate Answer Passage 3
Candidate Answer Passage 2
Candidate Answer Passage 1
Application to QA
query
Information RetrievalSystem
Candidate Answer Passage 3
Candidate Answer Passage 2
Candidate Answer Passage 1
Application to QA
query
Information RetrievalSystem
Candidate Answer Passage 3
Candidate Answer Passage 2
Candidate Answer Passage 1
Application to QA
query
Information RetrievalSystem
Answer Re-Ranker
Candidate Answer Passage 3
Candidate Answer Passage 2
Candidate Answer Passage 1
Application to QA
query
Information RetrievalSystem
Answer Re-Ranker
lexical similarity
Candidate Answer Passage 3
Candidate Answer Passage 2
Candidate Answer Passage 1
Application to QA
query
Information RetrievalSystem
Answer Re-Ranker
lexical similarity
discoursestructure
Candidate Answer Passage 3
Candidate Answer Passage 2
Candidate Answer Passage 1
Application to QA
query
Information RetrievalSystem
Answer Re-Ranker
lexical similarity
discoursestructure
Candidate Answer Passage 1
Candidate Answer Passage 3
Candidate Answer Passage 2
Candidate Answer Passage 3
Candidate Answer Passage 2
Candidate Answer Passage 1
Application to QA
query
Information RetrievalSystem
Answer Re-Ranker
lexical similarity
discoursestructure
Candidate Answer Passage 1
Candidate Answer Passage 3
Candidate Answer Passage 2
Candidate Answer Passage 3
Candidate Answer Passage 2
Candidate Answer Passage 1
Application to QA
query
Information RetrievalSystem
Answer Re-Ranker
lexical similarity
discoursestructure
Candidate Answer Passage 1
Candidate Answer Passage 3
Candidate Answer Passage 2
Discourse Structure
Features● Explicit discourse markers
○ “because”, “however”, …
● RST Parse
Discourse Structure (NNET features)
1. blah blah... 2. blah blah...3. blah… 4. ...5. ...
s1 s2
s2 s3
s3 s4
s4 s5
query
query
query
query
Results
See our final paper :)
The End
Thanks!