![Page 1: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/1.jpg)
Explainable AI CS891 - Introduction to Emerging Technologies
Old Dominion UniversityDepartment of Computer ScienceKritika Garg, [email protected]
![Page 2: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/2.jpg)
Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations
Upol Ehsan Brent HarrisonGeorgia Institute of Technology University of Kentucky, Atlanta, GA, USA Lexington, KY, USA
Larry ChanGeorgia Institute of Technology, Atlanta, GA, USA
Mark O. RiedlGeorgia Institute of Technology, Atlanta, GA, USA
![Page 3: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/3.jpg)
Basic Terminologies
![Page 4: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/4.jpg)
I propose to consider the question, ‘Can machines think?’— Alan Turing, 1950
● Turing's paper "Computing Machinery and Intelligence" (1950)
● Turing Test
![Page 5: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/5.jpg)
What is AI?
The concept of AI is based on the idea of building machines that can think and learn like humans.
![Page 6: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/6.jpg)
What is ML ?● An approach to AI where a machine learns by itself to solve a given
problem.● Algorithms that learns to make predictions based on input data.
https://christophm.github.io/interpretable-ml-book/images/programing-ml.png
![Page 7: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/7.jpg)
Training
Testing
Input Data
https://image.slidesharecdn.com/dataphillycoreychiversintromlv2-160219135616/95/intro-to-machine-learning-7-638.jpg?cb=1455890361https://www.google.com/about/main/machine-learning-qa/img/cat-dog-flow-horizontal.gif
Predictions
![Page 8: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/8.jpg)
"Black box" in machine learning
![Page 9: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/9.jpg)
https://christophm.github.io/interpretable-ml-book/images/programing-ml.png
![Page 10: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/10.jpg)
https://www.sciencemag.org/news/2017/07/how-ai-detectives-are-cracking-open-black-box-deep-learning
Neural Network
![Page 11: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/11.jpg)
https://miro.medium.com/max/500/1*ZhYNqU2y96_f3QkWq9oiWQ.jpeg
![Page 12: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/12.jpg)
https://artificialintelligence-news.com/2018/02/21/experts-ai-clear-present-danger/
![Page 13: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/13.jpg)
A.I. Is Everywhere
https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2019/08/01213414/AI_app_architecture.jpg
![Page 14: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/14.jpg)
Sometimes AI system can behave differently than what was expected
![Page 15: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/15.jpg)
https://www.nytimes.com/interactive/2018/03/20/us/self-driving-uber-pedestrian-killed.html
![Page 16: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/16.jpg)
The Accountability( who is to blame? ) and Safety becomes a factor when there are more critical decisions involved. For example:
● Driving an autonomous car
● Making medical diagnostics
You need to know “what AI is thinking” not just “what the results its giving”.
![Page 17: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/17.jpg)
https://www.darpa.mil/ddm_gallery/xai-figure1-inline-graphic.png
![Page 18: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/18.jpg)
https://www.darpa.mil/ddm_gallery/xai-figure2-inline-graphic.png
Explainable AI(XAI)
![Page 19: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/19.jpg)
Explainable A.I is hard?
![Page 20: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/20.jpg)
Data
Neural Net
Neural Net Decision Making
https://www.sciencemag.org/news/2017/07/how-ai-detectives-are-cracking-open-black-box-deep-learning
![Page 21: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/21.jpg)
https://www.sciencemag.org/news/2017/07/how-ai-detectives-are-cracking-open-black-box-deep-learning
![Page 22: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/22.jpg)
![Page 23: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/23.jpg)
![Page 24: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/24.jpg)
Relationalization
● Form of explainable AI● Act as a proxy for real explanation on what AI
is really thinking?● It provides human-like explanation.
![Page 25: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/25.jpg)
Other Forms of Explanation● Accurate account of each
step in decision making process
● Requires an expert to interpret the decisions
Rationalization
● Insights into the agents behaviour explained in a human like manner.
● Can be easily understood by a non-expert.
![Page 26: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/26.jpg)
Benefits over other explainability techniques
● Can be easily understood by non-expert.● Establish a trust between machine and
human.● Aid in human-machine collaboration, as it
is faster in conveying explanation.● Useful for time-critical decision making.
https://hbr.org/2018/07/collaborative-intelligence-humans-and-ai-are-joining-forces
![Page 27: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/27.jpg)
https://www.sciencemag.org/news/2017/07/how-ai-detectives-are-cracking-open-black-box-deep-learning
Related Works
![Page 28: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/28.jpg)
ApproachUsed a Game playing environment to train the AI.
● Experimental domain: Frogger
Generated Realizations for AI behaviour in the Frogger Game environment
![Page 29: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/29.jpg)
The Agent had to reach the top while avoiding the obstacles in the environment
![Page 30: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/30.jpg)
Issues
● Sequential Decision making
● Dynamic environment
![Page 31: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/31.jpg)
Hypothesis
1. Encoder-Decoder network can accurately generate realizations
![Page 32: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/32.jpg)
Hypothesis
1. Encoder-Decoder network can accurately generate realizations
2. Realizations are more satisfactory than other form of explanations, for humans.
![Page 33: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/33.jpg)
Methodology for Generation of Rationalization 1. Training Corpus2. Grammar Creation3. Training and testing data set4. Training and Testing the network5. Results6. Discussion
![Page 34: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/34.jpg)
Training Corpus
● Videos of 12 humans playing frogger while explaining their actions out loud.
● Recorded state of frog, after every action.● Used online speech transcription service.● Self validated the data by assigning action to their
utterances.● 225 action-rationalizations.
Fig1: AI Rationalization
![Page 35: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/35.jpg)
Training corpus consisted of state-action pairs annotated with natural language.
● State: Coordinates of agent on grid layout, (x,y).
● Action ○ Movements
○ Standing still
● Natural Language
![Page 36: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/36.jpg)
Fig1: AI Rationalization
![Page 37: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/37.jpg)
Grammar Creation
Action-rationalization annotations
Set of rules based on behaviour of frogger agent
Grammar
![Page 38: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/38.jpg)
State-action representation: tripple
(s1, a, s2)
Initial State
State, after action is performed
Action
![Page 39: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/39.jpg)
Ground Truth Realizations Grammar was used to construct Ground Truth Realizations.
Semi- Synthetic Language
Sate- Action Representation Internal to System
![Page 40: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/40.jpg)
Test Set● For every triple, a realization is generated● These examples were grouped according to their associated grammar rule.
E1E4
E3 E1E2
E5 E7E6E8 E9
Rule1 Rule2 Rule3
![Page 41: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/41.jpg)
Test Set
E5 E7 E6
E8 E9
20% from each cluster was taken as a testing data.
This was to ensure that there is an example associated with every grammar rule in the testing dataset.
![Page 42: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/42.jpg)
Training Set
E5 E7 E6
E8 E9
Rest 80% from each cluster was taken as a training data.
To help in training, they made duplicates of these 80% examples until they became 1000 in quantity.
![Page 43: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/43.jpg)
Training the network
![Page 44: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/44.jpg)
Approach1. Used Neural network to translate the two languages:
a. Code of the Gameb. Natural language
2. Then imported it back to the game playing network.
![Page 45: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/45.jpg)
Encoder-Decoder Network
Fig1: AI Rationalization
● 2-layered LSTM ● Node size of 300● Trained the network
for 50 epochs
![Page 46: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/46.jpg)
Evaluate The Accuracy● Requires to associate each generated sentence with a grammar rule.● To determine which rules associates the most with the output sentence:
○ Calculated sentence similarity between output sentence and the sentences generated by the grammar
○ The sentences with the highest similarity were recorded.○ Scores below 0.7 BLEU scores were not considered.○ The Rules of output and recorded sentences were compared, to check
whether they match or not.
Accuracy was calculated based on percentage of output sentences that matched with their associated test example.
![Page 47: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/47.jpg)
Three different environments Implemented the generated rationalizations on three different maps, to evaluate their approach.
![Page 48: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/48.jpg)
25%
![Page 49: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/49.jpg)
25% 50%
![Page 50: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/50.jpg)
25% 50% 75%
![Page 51: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/51.jpg)
Comparing Results in different environment
![Page 52: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/52.jpg)
Baseline Models
1. Random Model: Selected any random sentence as rationalization.
2. Majority-Vote Model: Selected sentences that are associated with the most common Rule in a particular map.
![Page 53: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/53.jpg)
Comparing models using chi-square test
![Page 54: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/54.jpg)
Figure 4. The Frogger agent rationalizing its actions.
![Page 55: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/55.jpg)
Methodology for rationalization satisfaction1. Participants2. Procedure3. Results4. Analysis5. Findings and Discussion
![Page 56: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/56.jpg)
Participants● Fifty-three adults (age range = 22 – 64 years, M = 34.1, SD = 9.38) were
recruited from Amazon Mechanical Turk (AMT).● Women: 21%● Participants were from three countries. ● 91% of people reported that they live in the United States.
![Page 57: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/57.jpg)
![Page 58: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/58.jpg)
Procedure● Participants were introduced to a hypothetical high-stakes scenario.● Scenarios were just “re-skinned” version of Frogger.● To avoid effects of preconceived notions, bots were named to A, B and C.
“Robot A” for the rationalizing robot, “Robot B” for the action-declaring robot, and “Robot C” for the numerical robot.
● Participants were randomly assigned to watch 6 videos: three depicting the agents succeeding and three showing them failing.
● Participants ranked their satisfaction and justified their choices.
![Page 59: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/59.jpg)
![Page 60: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/60.jpg)
Thematic Analysis● Analyzed the justifications that participants provided for their rankings.● Developed a set of codes that represented the different justifications.● Clustered these codes under emergent themes● Distilled the most relevant themes into insights that can be used to
understand the “whys” behind satisfaction of explanations.
![Page 61: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/61.jpg)
Finding
The rationalizing agent’s explanations were rated higher than were those of the other two agents.
![Page 62: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/62.jpg)
Attributes
Four attributes emerged from our thematic analysis that can be used to distinguish the rationalizing robot from the action declaring robot.
1. Explanatory power2. Relatability3. Ludic nature4. Adequate detail
![Page 63: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/63.jpg)
Explanatory Power
Ability to explain its actions.
“. . . what it’s doing and why” (P6) enabled them to “. . . get into [the rationalizing robot’s] mind” (P17)
“[The action-declaring robot] explained almost nothing. . .which was disappointing.” (P38)
![Page 64: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/64.jpg)
Relatability
Personality expressed through Robot’s explanation.
“ [The rationalizing robot] was relatable. He felt likea friend rather than a robot. I had a connection with[it] that would not be possible with the other 2 robotsbecause of his built-in personality. (P21) ”
![Page 65: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/65.jpg)
Lucid Quality
Perceived Playfulness.
“[Therationalizing robot] was fun and entertaining. I couldn’t wait to seewhat he would say next!” (P2).
![Page 66: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/66.jpg)
Adequate Detail
“[the rationalizing robot] talks too much” (P47)
“ the action-declaring robot is “nice and simple” (P48)
“would like to experience a combination of [the action-declaring robot] and [the rationalizing robot]” (P41).
![Page 67: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/67.jpg)
Future Work
● To investigate the types of rationalizations.● To judge the extent of confidence people have on the rationalization robot.
![Page 68: Explainable AI - cs.odu.edujbrunelle/cs891/student... · Kritika Garg, kgarg001@odu.edu. Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations](https://reader030.vdocuments.site/reader030/viewer/2022040503/5e2bec403f2cb804f46982fd/html5/thumbnails/68.jpg)
Conclusions● Used Crowd-Source data to construct action-rationalization pairs.● Construct ground-truth realizations using Grammar.● Generated Rationalizations through neural machine translation techniques.● Evaluated the model over three different maps● The Model with 75% obstacle map performed with 80% accuracy.● Analysed that humans find Rationalization more satisfactory than other form
of explanations.