challenges & design patterns for conversational ai fileincomplete sentences grammatical errors...
TRANSCRIPT
![Page 1: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/1.jpg)
Challenges & Design Patterns for Conversational AI
Peter Skomoroch, Head of Data Products
![Page 2: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/2.jpg)
Introductions
Peter Skomoroch@peteskomoroch
• Co-Founder and CEO of SkipFlag, Enterprise AI startup acquired by Workday
• Co-Host of O’Reilly AI Bots Podcast
• Principal Scientist and early member of the data team at LinkedIn
• Machine Learning and Search at MIT, AOL
![Page 3: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/3.jpg)
Challenges & Design Patterns for Conversational AI
![Page 4: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/4.jpg)
• Didn't understand request
• Wrong interpretation of request
• No results found
• No memory of past conversations
• No knowledge of user’s identity
• No grasp of slang, typos, jargon
• Entity disambiguation errors
Common Scenarios in AI Conversations
Credit: @JamieSkella
![Page 5: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/5.jpg)
• Rule Based Bots & Heuristics
• Slot Filling & Intent Classification
• Generative Models
• Retrieval Based Models
Common Approaches to Conversational AI
![Page 6: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/6.jpg)
Rule Based Bots & Heuristics
![Page 7: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/7.jpg)
Slot Filling & Intents
![Page 8: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/8.jpg)
Generative Models
Smart Reply: Automated Response Suggestion for Email (Kannan et al)
![Page 9: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/9.jpg)
Retrieval Based Models
https://rajpurkar.github.io/SQuAD-explorer/
![Page 10: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/10.jpg)
Narrow Domains vs. Unconstrained Conversations
![Page 11: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/11.jpg)
Knowledge Graphs & Conversational AI
![Page 12: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/12.jpg)
SkipFlag: A Knowledge Base That Builds Itself
• Smart Knowledge Base
• Expert Identification
• Instant Answers
![Page 13: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/13.jpg)
Content is auto-organized into a Knowledge Graph
![Page 14: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/14.jpg)
Entity Understanding and Linking
Job DescriptionKnow python and django, and have some experience with docker
PythonHigh-level programming language
DockerComputer program
DjangoSoftware
![Page 15: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/15.jpg)
Fact Extraction from Text with Linked Entities
Workday was founded by David Duffield, founder and former CEO of ERP company PeopleSoft, and former PeopleSoft chief strategist Aneel Bhusri. It is an on-demand (cloud-based) financial management and human capital management software vendor.
<Workday, Inc.> <founded by> <David Duffield>
<Workday, Inc.> <founded by> <Aneel Bhusri>
![Page 16: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/16.jpg)
Good Training Data is Often the Bottleneck
Credit: @mrogati
![Page 17: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/17.jpg)
Entity Understanding Training Data
Common Crawl: ~4B pages monthly
![Page 18: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/18.jpg)
Challenge: Workplace Dialogue and Internal Jargon
Product ManagerJob Title
Agora ProjectInternal Project
Site AnalyticsInternal Team
Workplace Conversations
● Short messages
● Incomplete sentences
● Grammatical errors
● Alternating speakers
● Meandering topics
● Internal jargon
● Overlapping chat conversations
Conover et. al., “Pangloss: Fast entity linking in noisy text environments”, KDD 2018 (to appear)
![Page 19: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/19.jpg)
Question Answering Training Data
![Page 20: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/20.jpg)
• Don’t assume building a bot for a messaging platform is easier than an app. If you are training conversational AI, it’s much harder.
• User retention issues will cause most bots to fail, unless platforms let them be ambient and contextual.
• Distribution and discovery are still challenging on messaging platforms. You need users to get the conversation data flywheel going.
• Google and Alexa Assistants are becoming a higher level discovery layer that delegates requests to 3rd party skills or conversational agents.
Parting thoughts: Platform Level Challenges
![Page 21: Challenges & Design Patterns for Conversational AI fileIncomplete sentences Grammatical errors Alternating speakers Meandering topics Internal jargon Overlapping chat conversations](https://reader031.vdocuments.site/reader031/viewer/2022021811/5c9e9e6888c9934b2d8be824/html5/thumbnails/21.jpg)
Q&A