Download - Haitham Elmarakeby. Speech recognition
![Page 1: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/1.jpg)
Sequence to Sequence Learning
Haitham Elmarakeby
![Page 2: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/2.jpg)
Sequence to Sequence
Speech recognition
http://nlp.stanford.edu/courses/lsa352/
![Page 3: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/3.jpg)
Sequence to Sequence
Machine translation
Welcome to the deep learning class
درس في بكم مرحباالعميق التعلم
![Page 4: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/4.jpg)
Sequence to Sequence
Question answering
![Page 5: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/5.jpg)
Statistical Machine Translation
Knight and Koehn 2003
![Page 6: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/6.jpg)
Statistical Machine Translation
Knight and Koehn 2003
![Page 7: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/7.jpg)
Statistical Machine Translation
Components Translation model Language Model Decoding
![Page 8: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/8.jpg)
Statistical Machine Translation
Translation model
Learn the P(f | e)
Knight and Koehn 2003
![Page 9: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/9.jpg)
Statistical Machine Translation
Translation model Input is Segmented in Phrases Each Phrase is Translated into English Phrases are Reordered
Koehn 2004
![Page 10: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/10.jpg)
Statistical Machine Translation
Language Model
Goal of the Language Model: Detect good English P(e)Standard Technique: Trigram Model
Knight and Koehn 2003
![Page 11: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/11.jpg)
Statistical Machine Translation
DecodingGoal of the decoding algorithm: Put models to work, perform the actual translation
Koehn 2004
![Page 12: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/12.jpg)
Statistical Machine Translation
DecodingGoal of the decoding algorithm: Put models to work, perform the actual translation
Koehn 2004
![Page 13: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/13.jpg)
Statistical Machine Translation
DecodingGoal of the decoding algorithm: Put models to work, perform the actual translation
Koehn 2004
![Page 14: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/14.jpg)
Statistical Machine Translation
DecodingGoal of the decoding algorithm: Put models to work, perform the actual translation
Koehn 2004
![Page 15: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/15.jpg)
Statistical Machine Translation
DecodingGoal of the decoding algorithm: Put models to work, perform the actual translation
Koehn 2004
![Page 16: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/16.jpg)
Statistical Machine Translation
DecodingGoal of the decoding algorithm: Put models to work, perform the actual translation
Koehn 2004
![Page 17: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/17.jpg)
Statistical Machine Translation
DecodingGoal of the decoding algorithm: Put models to work, perform the actual translation
Prune out Weakest Hypotheses by absolute threshold (keep 100 best) by relative cutoff
Future Cost Estimation compute expected cost of untranslated words
![Page 18: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/18.jpg)
Sutskever et al.,2014
Sequence to Sequence Learning with Neural Networks
![Page 19: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/19.jpg)
Neural Machine Translation
Model
A B C
W X Y Z
![Page 20: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/20.jpg)
Neural Machine Translation
Model
Sutskever et al. 2014
![Page 21: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/21.jpg)
Neural Machine Translation
Model- encoder
Cho: From Sequence Modeling to Translation
![Page 22: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/22.jpg)
Neural Machine Translation
Model- encoder
Cho: From Sequence Modeling to Translation
![Page 23: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/23.jpg)
Neural Machine Translation
Model- encoder
Cho: From Sequence Modeling to Translation
![Page 24: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/24.jpg)
Neural Machine Translation
Model- encoder
Cho: From Sequence Modeling to Translation
![Page 25: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/25.jpg)
Neural Machine Translation
Model- encoder
Cho: From Sequence Modeling to Translation
![Page 26: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/26.jpg)
Neural Machine Translation
Model- decoder
Cho: From Sequence Modeling to Translation
![Page 27: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/27.jpg)
Neural Machine Translation
Model- decoder
Cho: From Sequence Modeling to Translation
![Page 28: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/28.jpg)
Neural Machine Translation
Model- decoder
Cho: From Sequence Modeling to Translation
![Page 29: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/29.jpg)
Neural Machine Translation
RNN
![Page 30: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/30.jpg)
Neural Machine Translation
RNNVanishing gradient
Cho: From Sequence Modeling to Translation
![Page 31: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/31.jpg)
Neural Machine Translation
LSTM
Graves 2013
![Page 32: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/32.jpg)
Neural Machine Translation
LSTMProblem: Exploding gradient
![Page 33: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/33.jpg)
Neural Machine Translation
LSTMProblem: Exploding gradient Solution: Scaling gradient
![Page 34: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/34.jpg)
Sequence to Sequence
Reversing the Source Sentences
Welcome to the deep learning class
![Page 35: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/35.jpg)
Sequence to Sequence
Reversing the Source Sentences
Welcome to the deep learning class
![Page 36: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/36.jpg)
Sequence to Sequence
ResultsBLEU score (Bilingual Evaluation Understudy)
Candidate the the the the the the the
Reference 1 the cat is on the matReference 2 there is a cat on the mat
P = m/w= 7/7 = 1
Papineni et al. 2002
![Page 37: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/37.jpg)
Sequence to Sequence
ResultsBLEU score (Bilingual Evaluation Understudy)
Candidate the the the the the the the
Reference 1 the cat is on the matReference 2 there is a cat on the mat
P = 2/7
Papineni et al. 2002
![Page 38: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/38.jpg)
Sequence to Sequence
Results
Sutskever et al. 2014
![Page 39: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/39.jpg)
Sequence to Sequence
Results
Sutskever et al. 2014
![Page 40: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/40.jpg)
Sequence to Sequence
Model Analysis
Sutskever et al. 2014
![Page 41: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/41.jpg)
Sequence to Sequence
Long sentences
Sutskever et al. 2014
![Page 42: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/42.jpg)
Sequence to Sequence
Long sentences
Cho et al. 2014
![Page 43: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/43.jpg)
Bahdanau et al.,2014
Neural Machine Translation by Jointly Learning to Align and Translate
![Page 44: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/44.jpg)
Sequence to Sequence
Long sentences
Fixed length representation maybe the cause
![Page 45: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/45.jpg)
Jointly Learning to Align and Translate Attention mechanism
![Page 46: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/46.jpg)
Jointly Learning to Align and Translate Attention mechanism
![Page 47: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/47.jpg)
Jointly Learning to Align and Translate Attention mechanism
![Page 48: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/48.jpg)
Jointly Learning to Align and Translate Attention mechanism
![Page 49: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/49.jpg)
Jointly Learning to Align and Translate Attention mechanism
![Page 50: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/50.jpg)
Jointly Learning to Align and Translate Attention mechanism
![Page 51: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/51.jpg)
Jointly Learning to Align and Translate Attention mechanism
![Page 52: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/52.jpg)
Jointly Learning to Align and Translate
Long sentences
Cho et al. 2014
![Page 53: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/53.jpg)
Vinyals et al., 2015
Grammar as a Foreign Language
![Page 54: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/54.jpg)
Grammar as a Foreign Language
Parsing tree
![Page 55: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/55.jpg)
Grammar as a Foreign Language
Parsing tree
![Page 56: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/56.jpg)
Grammar as a Foreign Language
Parsing tree
![Page 57: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/57.jpg)
Grammar as a Foreign Language
Parsing tree
![Page 58: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/58.jpg)
Grammar as a Foreign Language
Parsing tree
John has a dog .
![Page 59: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/59.jpg)
Grammar as a Foreign Language
Converting tree to sequence
![Page 60: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/60.jpg)
Grammar as a Foreign Language
Converting tree to sequence
![Page 61: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/61.jpg)
Grammar as a Foreign Language
Model
![Page 62: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/62.jpg)
Grammar as a Foreign Language
Results
![Page 63: Haitham Elmarakeby. Speech recognition](https://reader036.vdocuments.site/reader036/viewer/2022081516/5697bfeb1a28abf838cb817c/html5/thumbnails/63.jpg)