edinburgh mt lecture 10: discriminative training

DiscriminativeLearning

Moses off the shelf:

-1286.916461


-1286.916461

13.2 14.0 10.8 11.2 13.0 14.6


-1286.916461

13.2 14.0 10.8 11.2 13.0 14.619.3 19.8 24.0 24.0 24.6 23.0


-1286.916461

13.2 14.0 10.8 11.2 13.0 14.619.3 19.8 24.0 24.0 24.6 23.026.2


-1286.916461

13.2 14.0 10.8 11.2 13.0 14.619.3 19.8 24.0 24.0 24.6 23.026.2

10

15

20

25

30

-1500 -1425 -1350 -1275 -1200


-1286.916461

13.2 14.0 10.8 11.2 13.0 14.619.3 19.8 24.0 24.0 24.6 23.026.2

Great example of fortuitous search error: bad search fixes a bad model.

It’s important to get the model right, then the search algorithm.

10

15

20

25

30

-1500 -1425 -1350 -1275 -1200

Learning

Problem: why maximize likelihood if we care about BLEU?

argmax

�

1

Zexp

(X

k

�khk(English, alignment, Chinese)

)

Learning

Solution: maximize BLEU instead

argmax

�BLEU

arg max

Eng,align

X

k

�khk(English, alignment, Chinese)

!

The Noisy Channel

-log p(g | e)

-log p(e)

As a Linear Model

-log p(g | e)

-log p(e)

~w

As a Linear Model

-log p(g | e)

-log p(e)

~wg

Improvement 1:

change to find better translations~w

As a Linear Model

-log p(g | e)

-log p(e)

~w

As a Linear Model

-log p(g | e)

-log p(e)

~w

Improvement 2:

Add dimensions to make points separable

14

h1

h2

~w

K-Best List Example

14

h1

h2

~w

#2#1

K-Best List Example

#3

#4#5#6

#7

#8

#9#10

15

h1

h2

#2#1

K-Best List Example

#3

#4#5#6

#7

#8

#9#10

0.8 � < 1.0

0.6 � < 0.8

0.4 � < 0.6

0.2 � < 0.4

0.0 � < 0.2

~w

Training as Classification• Pairwise Ranking Optimization

• Reduce training problem to binary classification with a linear model

• Algorithm

• For i=1 to N

• Pick random pair of hypotheses (A,B) from K-best list

• Use cost function to determine if is A or B better

• Create ith training instance

• Train binary linear classifier

16

17

h1

h2

#2 #1

#3

#4#5

#6

#7

#8

#9#10

0.8 � < 1.0

0.6 � < 0.8

0.4 � < 0.6

0.2 � < 0.4

0.0 � < 0.2

h1

h2

18

h1

h2

#2 #1

#3

#4#5

#6

#7

#8

#9#10

0.8 � < 1.0

0.6 � < 0.8

0.4 � < 0.6

0.2 � < 0.4

0.0 � < 0.2

h1

h2

Worse!

19

h1

h2

#2 #1

#3

#4#5

#6

#7

#8

#9#10

0.8 � < 1.0

0.6 � < 0.8

0.4 � < 0.6

0.2 � < 0.4

0.0 � < 0.2

h1

h2

Worse!

20

h1

h2

#2 #1

#3

#4#5

#6

#7

#8

#9#10

0.8 � < 1.0

0.6 � < 0.8

0.4 � < 0.6

0.2 � < 0.4

0.0 � < 0.2

h1

h2

21

h1

h2

#2 #1

#3

#4#5

#6

#7

#8

#9#10

0.8 � < 1.0

0.6 � < 0.8

0.4 � < 0.6

0.2 � < 0.4

0.0 � < 0.2

h1

h2

Better!

22

h1

h2

#2 #1

#3

#4#5

#6

#7

#8

#9#10

0.8 � < 1.0

0.6 � < 0.8

0.4 � < 0.6

0.2 � < 0.4

0.0 � < 0.2

h1

h2

Better!

23

h1

h2

#2 #1

#3

#4#5

#6

#7

#8

#9#10

0.8 � < 1.0

0.6 � < 0.8

0.4 � < 0.6

0.2 � < 0.4

0.0 � < 0.2

h1

h2

Worse!

24

h1

h2

#2 #1

#3

#4#5

#6

#7

#8

#9#10

0.8 � < 1.0

0.6 � < 0.8

0.4 � < 0.6

0.2 � < 0.4

0.0 � < 0.2

h1

h2

Better!

25

h1

h2

#2 #1

#3

#4#5

#6

#7

#8

#9#10

0.8 � < 1.0

0.6 � < 0.8

0.4 � < 0.6

0.2 � < 0.4

0.0 � < 0.2

h1

h2

26

h1

h2

Fit a linear model

27

h1

h2

Fit a linear model

~w

28

h1

h2

#2#1

K-Best List Example

#3

#4#5#6

#7

#8

#9#10

0.8 � < 1.0

0.6 � < 0.8

0.4 � < 0.6

0.2 � < 0.4

0.0 � < 0.2~w

Optimizing for BLEU

Notice:

score(English|Chinese) =

..�

i

�ihi(Chinese,English)

Optimizing for BLEU

Notice:


�xhx(Chinese,English) +.�

i/x

�ihi(Chinese,English)

Optimizing for BLEU

Notice:


a�x + b

Optimizing for BLEU

Notice:


a�x + b

just a line!

Optimizing for BLEU

�x

mod

el sc

ore

Optimizing for BLEU

�x

mod

el sc

ore

BLEU

�x

Optimizing for BLEU

�x

mod

el sc

ore

BLEU

�x

Minimum Error Rate Training (Och 2003)

Max Blues

Optimizing for BLEU

•Lots of alternative learning algorithms!

•Margin-infused Relaxation (Chiang ’08)

•Ramp Loss Minimization (Gimpel & Smtih ’12)

•Online Rank Learning (Watanabe ’12)

•many others…

iat lat pippat eneat hilat oloat at-yurp .

ok-voon ororok sprok .

at-voon bichat dat . ok-drubel ok-voon anok plok sprok .

at-drubel at-voon pippat rrat dat .

erok sprok izok hihok ghirok .

totat dat arrat vat hilat . ok-voon anok drok brok jok .

at-voon krat pippat sat lat . wiwok farok izok stok .

totat jjat quat cat . lalok sprok izok jok stok .

wat dat krat quat cat .

lalok farok ororok lalok sprok izok enemok .

wat jjat bichat wat dat vat eneat . lalok brok anok plok nok .

iat lat pippat rrat nnat . wiwok nok izok kantok ok-yurp .

totat nnat quat oloat at-yurp . lalok mok nok yorok ghirok clok .

wat nnat gat mat bat hilat . lalok nok crrrok hihok yorok zanzanok .

wat nnat arrat mat zanzanat . lalok rarok nok izok hihok mok .

wat nnat forat arrat vat gat .

















pippat : anokquat : izokrrat : ploksat : drok

totat : eroktotat : wiwok

vat : izokwat : lalok

zanzanat : zanzanok??? : crrrok

arrat : hihokat-drubel : ok-drubel

at-voon : ok-voonat-yurp : ok-yurp

bat : clokbichat : ororok

cat : stokdat : sprok

eneat : enemokforat : rarok


gat : mokhilat : ghirok

iat : lalokjjat : farokkrat : joklat : brok

mat : yoroknnat : nok

oloat : kantok

language model +translation model +

other features






























oloat : kantok


other features

Decoder






























oloat : kantok


other features


lalok brok anok enemok ghirok kantok ok-yurp .

Decoder






























oloat : kantok


other features



Decoder

lalok brok anok ghirok enemok kantok ok-yurp .






























oloat : kantok


other features



Decoder


accuracy = 0.83






























oloat : kantok


other features



Decoder


accuracy = 0.83

feature weights〈0.2, 0.4, 0.1, ...〉






























oloat : kantok


other features



Decoder


accuracy = 0.83

feature weights〈0.2, 0.4, 0.1, ...〉

Fairly reasonable approximationto how Google Translate and

Bing Translator work

•Key ingredients in Google Translate:

•Phrase-based translation models

•... Learned heuristically from word alignments

•... Coupled with a huge language model

•... Very tight pruning heuristics

•... And minimum error rate training for BLEU.

edinburgh mt lecture 10: discriminative training

Technology