coling 2016 citation count: 1 · •7220 conversations (q1 a1 q2 r1) •134k/65k words for...
TRANSCRIPT
![Page 1: COLING 2016 Citation count: 1 · •7220 conversations (Q1 A1 Q2 R1) •134K/65K words for input/output sequence text. Experiment. SIGIR 2017 Citation count: 2. Methodology. Offline](https://reader033.vdocuments.site/reader033/viewer/2022060322/5f0d65247e708231d43a2410/html5/thumbnails/1.jpg)
COLING 2016Citation count: 1
![Page 2: COLING 2016 Citation count: 1 · •7220 conversations (Q1 A1 Q2 R1) •134K/65K words for input/output sequence text. Experiment. SIGIR 2017 Citation count: 2. Methodology. Offline](https://reader033.vdocuments.site/reader033/viewer/2022060322/5f0d65247e708231d43a2410/html5/thumbnails/2.jpg)
Motivation
• Incomplete questions do not make sense to conversation system
![Page 3: COLING 2016 Citation count: 1 · •7220 conversations (Q1 A1 Q2 R1) •134K/65K words for input/output sequence text. Experiment. SIGIR 2017 Citation count: 2. Methodology. Offline](https://reader033.vdocuments.site/reader033/viewer/2022060322/5f0d65247e708231d43a2410/html5/thumbnails/3.jpg)
Methodology
![Page 4: COLING 2016 Citation count: 1 · •7220 conversations (Q1 A1 Q2 R1) •134K/65K words for input/output sequence text. Experiment. SIGIR 2017 Citation count: 2. Methodology. Offline](https://reader033.vdocuments.site/reader033/viewer/2022060322/5f0d65247e708231d43a2410/html5/thumbnails/4.jpg)
Challenge
• How to handle OOV in small dataset (7k)
![Page 5: COLING 2016 Citation count: 1 · •7220 conversations (Q1 A1 Q2 R1) •134K/65K words for input/output sequence text. Experiment. SIGIR 2017 Citation count: 2. Methodology. Offline](https://reader033.vdocuments.site/reader033/viewer/2022060322/5f0d65247e708231d43a2410/html5/thumbnails/5.jpg)
Syntactic Sequence Model
![Page 6: COLING 2016 Citation count: 1 · •7220 conversations (Q1 A1 Q2 R1) •134K/65K words for input/output sequence text. Experiment. SIGIR 2017 Citation count: 2. Methodology. Offline](https://reader033.vdocuments.site/reader033/viewer/2022060322/5f0d65247e708231d43a2410/html5/thumbnails/6.jpg)
Problem
• Fails to utilize the meaning of each word, only the syntactic information
![Page 7: COLING 2016 Citation count: 1 · •7220 conversations (Q1 A1 Q2 R1) •134K/65K words for input/output sequence text. Experiment. SIGIR 2017 Citation count: 2. Methodology. Offline](https://reader033.vdocuments.site/reader033/viewer/2022060322/5f0d65247e708231d43a2410/html5/thumbnails/7.jpg)
Semantic Sequence Model
• Assign each OOV a category number by k-means algorithm
• Use word embedding as clustering features
![Page 8: COLING 2016 Citation count: 1 · •7220 conversations (Q1 A1 Q2 R1) •134K/65K words for input/output sequence text. Experiment. SIGIR 2017 Citation count: 2. Methodology. Offline](https://reader033.vdocuments.site/reader033/viewer/2022060322/5f0d65247e708231d43a2410/html5/thumbnails/8.jpg)
Dataset
• Collected from Amazon Mechanical Turk
• 7220 conversations (Q1 A1 Q2 R1)
• 134K/65K words for input/output sequence text
![Page 9: COLING 2016 Citation count: 1 · •7220 conversations (Q1 A1 Q2 R1) •134K/65K words for input/output sequence text. Experiment. SIGIR 2017 Citation count: 2. Methodology. Offline](https://reader033.vdocuments.site/reader033/viewer/2022060322/5f0d65247e708231d43a2410/html5/thumbnails/9.jpg)
Experiment
![Page 10: COLING 2016 Citation count: 1 · •7220 conversations (Q1 A1 Q2 R1) •134K/65K words for input/output sequence text. Experiment. SIGIR 2017 Citation count: 2. Methodology. Offline](https://reader033.vdocuments.site/reader033/viewer/2022060322/5f0d65247e708231d43a2410/html5/thumbnails/10.jpg)
SIGIR 2017Citation count: 2
![Page 11: COLING 2016 Citation count: 1 · •7220 conversations (Q1 A1 Q2 R1) •134K/65K words for input/output sequence text. Experiment. SIGIR 2017 Citation count: 2. Methodology. Offline](https://reader033.vdocuments.site/reader033/viewer/2022060322/5f0d65247e708231d43a2410/html5/thumbnails/11.jpg)
Methodology
![Page 12: COLING 2016 Citation count: 1 · •7220 conversations (Q1 A1 Q2 R1) •134K/65K words for input/output sequence text. Experiment. SIGIR 2017 Citation count: 2. Methodology. Offline](https://reader033.vdocuments.site/reader033/viewer/2022060322/5f0d65247e708231d43a2410/html5/thumbnails/12.jpg)
Offline �Question Template Library
• 6420 → 5451
![Page 13: COLING 2016 Citation count: 1 · •7220 conversations (Q1 A1 Q2 R1) •134K/65K words for input/output sequence text. Experiment. SIGIR 2017 Citation count: 2. Methodology. Offline](https://reader033.vdocuments.site/reader033/viewer/2022060322/5f0d65247e708231d43a2410/html5/thumbnails/13.jpg)
Problem
• Template library size is still too large (5k)
• Make online inference slow
![Page 14: COLING 2016 Citation count: 1 · •7220 conversations (Q1 A1 Q2 R1) •134K/65K words for input/output sequence text. Experiment. SIGIR 2017 Citation count: 2. Methodology. Offline](https://reader033.vdocuments.site/reader033/viewer/2022060322/5f0d65247e708231d43a2410/html5/thumbnails/14.jpg)
Candidate �Question Template Selector
![Page 15: COLING 2016 Citation count: 1 · •7220 conversations (Q1 A1 Q2 R1) •134K/65K words for input/output sequence text. Experiment. SIGIR 2017 Citation count: 2. Methodology. Offline](https://reader033.vdocuments.site/reader033/viewer/2022060322/5f0d65247e708231d43a2410/html5/thumbnails/15.jpg)
Re-ranking with Language Model
• Template selector seq2seq score:
• Language model score:
• Total score:
![Page 16: COLING 2016 Citation count: 1 · •7220 conversations (Q1 A1 Q2 R1) •134K/65K words for input/output sequence text. Experiment. SIGIR 2017 Citation count: 2. Methodology. Offline](https://reader033.vdocuments.site/reader033/viewer/2022060322/5f0d65247e708231d43a2410/html5/thumbnails/16.jpg)
Same Dataset
• Collected from Amazon Mechanical Turk
• 7220 conversations (Q1 A1 Q2 R1)
• 134K/65K words for input/output sequence text
![Page 17: COLING 2016 Citation count: 1 · •7220 conversations (Q1 A1 Q2 R1) •134K/65K words for input/output sequence text. Experiment. SIGIR 2017 Citation count: 2. Methodology. Offline](https://reader033.vdocuments.site/reader033/viewer/2022060322/5f0d65247e708231d43a2410/html5/thumbnails/17.jpg)
Experiment
![Page 18: COLING 2016 Citation count: 1 · •7220 conversations (Q1 A1 Q2 R1) •134K/65K words for input/output sequence text. Experiment. SIGIR 2017 Citation count: 2. Methodology. Offline](https://reader033.vdocuments.site/reader033/viewer/2022060322/5f0d65247e708231d43a2410/html5/thumbnails/18.jpg)
Window Size of Pre�fix Tree
![Page 20: COLING 2016 Citation count: 1 · •7220 conversations (Q1 A1 Q2 R1) •134K/65K words for input/output sequence text. Experiment. SIGIR 2017 Citation count: 2. Methodology. Offline](https://reader033.vdocuments.site/reader033/viewer/2022060322/5f0d65247e708231d43a2410/html5/thumbnails/20.jpg)
Conclusion
• Syntactic information is important
• Template method works well on the dataset
• Use language model to further enhance performance