spoken dialogue systems and the galaxy architecture 29 october 2000 advanced technology laboratories...
TRANSCRIPT
![Page 1: Spoken Dialogue Systems and the GALAXY Architecture 29 October 2000 Advanced Technology Laboratories 1 Federal Street A&E Building 2W Camden, New Jersey](https://reader036.vdocuments.site/reader036/viewer/2022072013/56649e4c5503460f94b41508/html5/thumbnails/1.jpg)
Spoken Dialogue Systems and the Spoken Dialogue Systems and the GALAXY ArchitectureGALAXY Architecture
29 October 200029 October 2000
Advanced Technology Laboratories
1 Federal Street • A&E Building 2W • Camden, New Jersey 08102
Jerry FrankeJerry FrankeSenior Member, Engineering StaffSenior Member, Engineering [email protected] [email protected] 856.338.3341
![Page 2: Spoken Dialogue Systems and the GALAXY Architecture 29 October 2000 Advanced Technology Laboratories 1 Federal Street A&E Building 2W Camden, New Jersey](https://reader036.vdocuments.site/reader036/viewer/2022072013/56649e4c5503460f94b41508/html5/thumbnails/2.jpg)
September 99
Talk OutlineTalk Outline
• Spoken Language DevelopmentSpoken Language Development
• GALAXY II SystemGALAXY II System
• SUMMIT (speech recognition)SUMMIT (speech recognition)
• TINA (natural language parsing)TINA (natural language parsing)
• GENESIS (natural language generation)GENESIS (natural language generation)
• Turn ManagementTurn Management
• GALAXY II DomainsGALAXY II Domains
• Overview/DemonstrationOverview/Demonstration
![Page 3: Spoken Dialogue Systems and the GALAXY Architecture 29 October 2000 Advanced Technology Laboratories 1 Federal Street A&E Building 2W Camden, New Jersey](https://reader036.vdocuments.site/reader036/viewer/2022072013/56649e4c5503460f94b41508/html5/thumbnails/3.jpg)
September 99
Spoken Language DevelopmentSpoken Language Development
• UniversitiesUniversities
– MIT, CMU, ColoradoMIT, CMU, Colorado
• Basic research labsBasic research labs
– ATT, SRIATT, SRI
• Software developersSoftware developers
– Nuance, SpeechWorksNuance, SpeechWorks
• Domain developersDomain developers
– Lockheed Martin - ATLLockheed Martin - ATL
![Page 4: Spoken Dialogue Systems and the GALAXY Architecture 29 October 2000 Advanced Technology Laboratories 1 Federal Street A&E Building 2W Camden, New Jersey](https://reader036.vdocuments.site/reader036/viewer/2022072013/56649e4c5503460f94b41508/html5/thumbnails/4.jpg)
September 99
GALAXY II SystemGALAXY II System
• Developed by MIT Spoken Language Systems groupDeveloped by MIT Spoken Language Systems group
• Multiple servers performing parts of the dialogue processMultiple servers performing parts of the dialogue process
SpeechRecognition
LanguageUnderstanding
TurnManagement
LanguageGeneration
SpeechSynthesis
Sound
Sound
SUMMIT TINA GENESIS
![Page 5: Spoken Dialogue Systems and the GALAXY Architecture 29 October 2000 Advanced Technology Laboratories 1 Federal Street A&E Building 2W Camden, New Jersey](https://reader036.vdocuments.site/reader036/viewer/2022072013/56649e4c5503460f94b41508/html5/thumbnails/5.jpg)
September 99
SUMMIT (speech recognition)SUMMIT (speech recognition)
• Three elements: vocabulary, language models, acoustic modelsThree elements: vocabulary, language models, acoustic models
• Pause words stripped outPause words stripped out
• AM: segment-based models and boundary-based diphone AM: segment-based models and boundary-based diphone modelsmodels
• LM: Forward Viterbi search with a class bigram model, followed LM: Forward Viterbi search with a class bigram model, followed by a backward A* search with a class trigram modelby a backward A* search with a class trigram model
• Produces N-best list or word graph of possible utterancesProduces N-best list or word graph of possible utterances
• Models trained on domain corpusModels trained on domain corpus
• Models achieve speaker-independenceModels achieve speaker-independence
![Page 6: Spoken Dialogue Systems and the GALAXY Architecture 29 October 2000 Advanced Technology Laboratories 1 Federal Street A&E Building 2W Camden, New Jersey](https://reader036.vdocuments.site/reader036/viewer/2022072013/56649e4c5503460f94b41508/html5/thumbnails/6.jpg)
September 99
TINA (natural language parsing)TINA (natural language parsing)
• Selects from N-best list depending on grammatical parseSelects from N-best list depending on grammatical parse
• Grammars reflect both syntactic and semantic structureGrammars reflect both syntactic and semantic structure
• Result is a semantic frameResult is a semantic frame
• Example:Example:
““Where is the library in Swain Hall?”Where is the library in Swain Hall?”
Clause: LOCATEClause: LOCATE
Topic: PUBLIC-BUILDINGTopic: PUBLIC-BUILDING
Quantifier: DEFQuantifier: DEF
Name: libraryName: library
Predicate: INPredicate: IN
Topic: HALLTopic: HALL
Name: SwainName: Swain
![Page 7: Spoken Dialogue Systems and the GALAXY Architecture 29 October 2000 Advanced Technology Laboratories 1 Federal Street A&E Building 2W Camden, New Jersey](https://reader036.vdocuments.site/reader036/viewer/2022072013/56649e4c5503460f94b41508/html5/thumbnails/7.jpg)
September 99
GENESIS (natural language generation)GENESIS (natural language generation)
• Processes semantic framesProcesses semantic frames
• Embeds semantic frame components into context-dependent Embeds semantic frame components into context-dependent message templatesmessage templates
• Two types of output:Two types of output:
– natural language messagesnatural language messages– messages are sent to some speech synthesis modulemessages are sent to some speech synthesis module
– possibility of output in multiple languagespossibility of output in multiple languages
– keyword-value pairskeyword-value pairs– useful structure for the turn management backenduseful structure for the turn management backend
• Can be used to map between (translate) languagesCan be used to map between (translate) languages
![Page 8: Spoken Dialogue Systems and the GALAXY Architecture 29 October 2000 Advanced Technology Laboratories 1 Federal Street A&E Building 2W Camden, New Jersey](https://reader036.vdocuments.site/reader036/viewer/2022072013/56649e4c5503460f94b41508/html5/thumbnails/8.jpg)
September 99
Turn ManagementTurn Management
• Manages the system’s part of the dialogueManages the system’s part of the dialogue
• Fuses current utterance with dialogue history for full contextFuses current utterance with dialogue history for full context
• Five main tasks:Five main tasks:
– Answer user’s requests (information retrieval)Answer user’s requests (information retrieval)
– Initiate sub-dialogues to clarify the user’s requestInitiate sub-dialogues to clarify the user’s request
– Track progress through the dialogueTrack progress through the dialogue
– Control response to the userControl response to the user
– Provide assistance in using the system when neededProvide assistance in using the system when needed
![Page 9: Spoken Dialogue Systems and the GALAXY Architecture 29 October 2000 Advanced Technology Laboratories 1 Federal Street A&E Building 2W Camden, New Jersey](https://reader036.vdocuments.site/reader036/viewer/2022072013/56649e4c5503460f94b41508/html5/thumbnails/9.jpg)
September 99
GALAXY II DomainsGALAXY II Domains
From MIT:From MIT:
– Jupiter - weather forecastsJupiter - weather forecasts
– Pegasus - airline schedulingPegasus - airline scheduling
– Voyager - Cambridge, Massachusetts city guideVoyager - Cambridge, Massachusetts city guide
– Dinex - Boston restaurant guideDinex - Boston restaurant guide
– Wheels - automobile classified adsWheels - automobile classified ads
– Mercury - airline flight bookingMercury - airline flight booking
From Lockheed Martin - ATL:From Lockheed Martin - ATL:
– DARPA Communicator - airline flight, hotel, car rental bookingDARPA Communicator - airline flight, hotel, car rental booking
– DARPA LCS-Marine/Marine Small Unit Logistics - supply DARPA LCS-Marine/Marine Small Unit Logistics - supply requestrequest
– DARPA LCS-Army - data collection during equipment testsDARPA LCS-Army - data collection during equipment tests
![Page 10: Spoken Dialogue Systems and the GALAXY Architecture 29 October 2000 Advanced Technology Laboratories 1 Federal Street A&E Building 2W Camden, New Jersey](https://reader036.vdocuments.site/reader036/viewer/2022072013/56649e4c5503460f94b41508/html5/thumbnails/10.jpg)
September 99
Overview/DemonstrationOverview/Demonstration
• Booking airline flights (round trip and one way)Booking airline flights (round trip and one way)
• Hotel, car rentalHotel, car rental
• Uses user profile to fill in some information about reservationUses user profile to fill in some information about reservation
• Uses some real-world knowledgeUses some real-world knowledge
• Information retrieval via mobile agentsInformation retrieval via mobile agents
![Page 11: Spoken Dialogue Systems and the GALAXY Architecture 29 October 2000 Advanced Technology Laboratories 1 Federal Street A&E Building 2W Camden, New Jersey](https://reader036.vdocuments.site/reader036/viewer/2022072013/56649e4c5503460f94b41508/html5/thumbnails/11.jpg)
September 99
Try It YourselfTry It Yourself
• CMU 1-412-268-1084CMU 1-412-268-1084
• MIT 1-877-527-8255MIT 1-877-527-8255
• Colorado http://communicator.colorado.edu/Colorado http://communicator.colorado.edu/