4190.408 2016-spring deep learning for nlp · 2016-04-17 · b io i ntelligence 4190.408 artificial...

57
Bio Intelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak Zhang Biointelligence Lab Seoul National University 1

Upload: others

Post on 17-Jun-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring)

4190.408 2016-Spring

Deep Learning for NLP

Byoung-Tak Zhang

Biointelligence Lab

Seoul National University

1

Page 2: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 2

Page 3: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 3

[Yoshua Bengio, 2009, Deep Learnng for Speech and Language]

Page 4: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 4

[Richard Socher, Stanford, Deep Learning for NLP]

Page 5: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 5

[Richard Socher, Stanford, Deep Learning for NLP]

Page 6: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 6

[Richard Socher, Stanford, Deep Learning for NLP]5 Reasons to Explore Deep Learning

Page 7: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 7

[Richard Socher, Stanford, Deep Learning for NLP]5 Reasons to Explore Deep Learning

Page 8: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 8

[Richard Socher, Stanford, Deep Learning for NLP]5 Reasons to Explore Deep Learning

Page 9: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 9

[Richard Socher, Stanford, Deep Learning for NLP]5 Reasons to Explore Deep Learning

Page 10: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 10

[Richard Socher, Stanford, Deep Learning for NLP]5 Reasons to Explore Deep Learning

Page 11: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 11

[Richard Socher, Stanford, Deep Learning for NLP]5 Reasons to Explore Deep Learning

Page 12: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 12

[Richard Socher, Stanford, Deep Learning for NLP]5 Reasons to Explore Deep Learning

Page 13: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 13

[Richard Socher, Stanford, Deep Learning for NLP]5 Reasons to Explore Deep Learning

Page 14: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 14

[Richard Socher, Stanford, Deep Learning for NLP]5 Reasons to Explore Deep Learning

Page 15: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 15

[郭江, 2014, Deep Learning for NLP]Statistical NLP – Challenges

Page 16: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 16

[郭江, 2014, Deep Learning for NLP]Statistical NLP – Challenges

Page 17: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 17

[郭江, 2014, Deep Learning for NLP]Statistical NLP – Challenges

Page 18: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 18

[郭江, 2014, Deep Learning for NLP]Statistical NLP – Challenges

Page 19: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 19

[郭江, 2014, Deep Learning for NLP]Statistical NLP – Challenges

Page 20: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 20

[郭江, 2014, Deep Learning for NLP]Statistical NLP – Challenges

Page 21: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 21

[郭江, 2014, Deep Learning for NLP]Statistical NLP – Challenges

Page 22: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 22

[郭江, 2014, Deep Learning for NLP]Statistical NLP – Challenges

Page 23: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 23

[郭江, 2014, Deep Learning for NLP]Statistical NLP – Challenges

Page 24: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 24

[Yoshua Bengio, NLP – Applications of Deep Learning]

Page 25: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 25

[Yoshua Bengio, NLP – Applications of Deep Learning]

Page 26: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 26

[Yoshua Bengio, NLP – Applications of Deep Learning]

Page 27: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 27

[Yoshua Bengio, NLP – Applications of Deep Learning]

Page 28: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 28

[Yoshua Bengio, NLP – Applications of Deep Learning]

Page 29: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 29

[Yoshua Bengio, NLP – Applications of Deep Learning]

Page 30: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 30

[Yoshua Bengio, NLP – Applications of Deep Learning]

Page 31: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 31

[Yoshua Bengio, NLP – Applications of Deep Learning]

Page 32: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 32

[Yoshua Bengio, NLP – Applications of Deep Learning]

Page 33: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 33

[Yoshua Bengio, NLP – Applications of Deep Learning]

Page 34: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 34

[Yoshua Bengio, NLP – Applications of Deep Learning]

Page 35: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 35

[Yoshua Bengio, NLP – Applications of Deep Learning]

Page 36: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 36

[Yoshua Bengio, NLP – Applications of Deep Learning]

Page 37: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 37

[郭江, 2014, Deep Learning for NLP]

Page 38: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 38

[郭江, 2014, Deep Learning for NLP]

Page 39: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 39

[郭江, 2014, Deep Learning for NLP]

Page 40: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring)

4190.408 2016-Spring

Deep Learning for Speech Recognition

Byoung-Tak Zhang

Biointelligence Lab

Seoul National University

40

Page 41: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 41

Page 42: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 42

Page 43: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 43

Page 44: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 44

Page 45: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 45

Page 46: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 46

Page 47: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 47

Page 48: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 48

Page 49: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 49

Page 50: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 50

Page 51: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 51

Page 52: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 52

Page 53: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 53

Page 54: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 54

Page 55: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 55

Page 56: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 56

Page 57: 4190.408 2016-Spring Deep Learning for NLP · 2016-04-17 · B io I ntelligence 4190.408 Artificial Intelligence (2016-Spring) 4190.408 2016-Spring Deep Learning for NLP Byoung-Tak

Bio

Intelligence4190.408 Artificial Intelligence (2016-Spring) 57