using monolingual crowds to improve translationbederson/images/pubs_pdfs/aaas-poster.pdf ·...

Using Monolingual Crowds to Improve Translation Yakov Kronrod1, Chang Hu2, Olivia Buzek2, Alex Quinn2 University of Maryland (1) Dept. of Linguistics (2) Dept. of Computer Science

UMD HCI Lab (cs.umd.edu/hcil) Comp. Ling. and Info. Processing Lab (umiacs.umd.edu/labs/clip) MonoTrans Project Page: (http://www.cs.umd.edu/hcil/monotrans)

1

ParaTrans: Targeted Paraphrasing   Monolingual speakers improving translation quality

  Human participants perform the following tasks:

1.  Identify mistranslated spans in target language

2.  Create paraphrases for spans in source language responsible for the target error spans.

•  Produces gains of 0.82 and 1.32 for adequacy and fluency on a 5-point scale

•  Improves 65.3% of sentences examined

•  Yields increases of +0.9 BLEU to +3.8 BLEU, depending on size of error span, in an oracle study.

Monolingual  Human  Par.cipa.on  

Quality  

Afforda

bility  

Space of Available Translation Options

• Online system for collaborative monolingual translation • Asynchronous edits, short tasks, minimal context

repeat …

Target Language Source Language MT

Sentence   Sentence  

1   Vote  2   Identify error spans  

3   Create new translation candidates  1   Vote  

2   Paraphrase  

3   Explain errors  

Crowd Tasks:  

New  candidate  1  2  3  

MT and

word alignment

…

Crowd Tasks:  

The Bigger Picture: MonoTrans Features Results

Translations and Human Judgments

Original  Google  Translate  output  (GT),  with    transla.ons  produced  by  the  targeted  paraphrase  transla.on  process  (TP),  selected  to  show  a  range  from  strong  to  weak  improvements  in  fluency.    Fluency  and  adequacy  (preserva.on  of  meaning)  were  rated  on  a  5-­‐point  scale  by  independent  judges  not  familiar  with  this  research.  

Paraphrases from the Crowd

68% of sentences translated perfectly with no bilingual humans