bangla machine translation

8/8/2019 Bangla Machine Translation

1/47

Master Thesis

English to Bangla Phrase-Based Statistical Machine

Translation

Md. Zahurul Islam2519786

Supervisors:

Dr. Andreas Eisele

PD. Dr. Valia Kordoni

Prof. Hans Uszkoreit

Department of Computational linguistics

Saarland University

Germany

&

Dr. Jrg TiedemannDepartment of Humanities Computing

University of Groningen

The Netherlands

Submitted to

Department of Computational linguistics

atSaarland University

Erasmus Mundus European Masters Program in

Language and Communication Technologies (LCT)

August 2009


2/47

Master Thesis

English to Bangla Phrase-Based Statistical Machine

Translation

Md. Zahurul Islam

1788213

Supervisors:

Dr. Jrg Tiedemann

Department of Humanities Computing

University of Groningen

The Netherlands

&

Dr. Andreas Eisele

DFKI GmbH, Language Technology Lab andDepartment of Computational linguistics

Saarland University

Germany

Submitted to

Faculty of Arts

atUniversity of Groningen

Research Master Linguistics

Erasmus Mundus European Masters Program in

Language and Communication Technologies (LCT)

August 2009


3/47


4/47


5/47


6/47


7/47


8/47


9/47


10/47


11/47


12/47

S > N P+ V P


13/47

!"#$ %' !"(


14/47

)*#+ ,-!

P r (w)

P r (e|w)

P r (j|e)

P r (k|j)

P r (o|k)


15/47

P r (w) .P r (e|w) .P r (j|e) .P r (k|j) .P r (o|k)

P r (w)


16/47

eI1

= e1 . . . ei . . . eI fJ1 = f1 . . . f j . . . f J

fj1

= argminJ,fJ1

Jfj

1

P r

fJ

1 |eI1

.L

fJ1

, fJ

1

L

fJ1

, fJ

1

fJ1

fJ

1

L

fJ1 , fJ

1

=

0 fJ1 = f

J

1

1

= 1

fJ1 , fJ

1


18/47

fJ1

eI1

e1, e2 . . . eI e

I1

fJ1

fJ1|eI1

fJ1 eI1

d (aj bj1) aj ith ei bj1 i 1th ei1 d (aj bj1) =

|ajbj11|

P r

fJ1 |eI1

=

I,Ji,j=1

(fj , ei) d (aj bj1)


19/47

e = eI

1

f = fJ1

f = argmaxf {P r (f|e)}

f = argmaxf {P r (e|f) .P r (f)}

P r (e|f) P r (f)

P r (e|f) =a

P r (e, a|f)

a a = aI

1= a1 . . . ai . . . aI, ai

{0, . . . J } ai = 0 p (ai|ai1, J) ai aj1 p ((|ai ai1|) |J)


20/47


21/47


22/47

./0#1. 2345,67,893:

)"#;


23/47

%/W .;*/^F %/W

_,&*/F

'PQF _,&*/F

#'/Y )( `F #'/Y &/F .aE*F &/F

.#a .$*. ,-! )*#+ )*#+/! ,b .aE* .aE*b

c*;* deFQ c*/;eFQ )f" gh.) )f"h.)

!"#$ `)i %#$ %$F*


24/47


25/47


26/47


27/47


28/47


29/47


30/47

`;#1 `j^ %&'*F %('W! K#f;*F 'E*ME )E).*

'E*ME )E).* )*fE!*$k^;O*/) `;#1'!"' U!*iU( .*f*Fl #$m K/n*)F g#'UU

`( 'b #D.*/) ;#o' `(#1 #D/./) p#' `)i;F/! -.1* M*b#' M/qr &*/F's

D/b/t

T!E*uE*! -M `1*

;*v'Zw* ;*v'Zw*

-.x#P %F) #;iy$ -.x#P %F)

$z!" . $#FU*. -;*{*#F;* -;*{*#F;*

#|;* |Q.


31/47

K#=. Z' #O;*F. )*', #;,$"'

`;#1 U&F `j/^* %&'*F %('W! K#f;*F 'E*ME )E).*

'E*ME )E).* )*fE!*$k^;O*/) `;#1

'!"' U!*iU(.*f*Fl #$m Z' #O;*F. K/n*)Fg#'UU

`( 'b #D.*/) ;#o' `(#1 #D/./) p#' `)i;F/! -.1* M*b#' M/qr &*/F's

$*$y D$P'ZP D/b/t T!E*uE*! -M -}*^;~ `1* #}q s

)*', #;,$"'


32/47

F)!


33/47

`;#1 `$' #'/P


34/47

`;#1 -U*&&*F `$' #'/P


35/47


36/47

n


37/47

ACC = 1N

Ni=1

{1 ri = ci, 0 }

Top J =1

N

Ni=1

{1 ci,j ci,j = ri, 0 }

LCS(c, r) = 1/2 (length (c) + length (r) editDistance (c, r))

ith

Ri =LCS(ci, ri)

length (ri)Pi =

LCS(ci, ri)

length (ci)Fi = 2

Ri Pi

Ri + Pi


38/47


39/47


40/47


41/47

`;#1 FY*^'* ;/F `;#1 *F F #'/P


42/47


43/47


44/47


45/47


46/47


47/47

bangla machine translation

Documents