Using Monolingual Crowds to Improve Translation Yakov Kronrod1, Chang Hu2, Olivia Buzek2, Alex Quinn2 University of Maryland (1) Dept. of Linguistics (2) Dept. of Computer Science

UMD HCI Lab (cs.umd.edu/hcil) Comp. Ling. and Info. Processing Lab (umiacs.umd.edu/labs/clip) MonoTrans Project Page: (http://www.cs.umd.edu/hcil/monotrans)

1

ParaTrans: Targeted Paraphrasing   Monolingual speakers improving translation quality

  Human participants perform the following tasks:

1.  Identify mistranslated spans in target language

2.  Create paraphrases for spans in source language responsible for the target error spans.

•  Produces gains of 0.82 and 1.32 for adequacy and fluency on a 5-point scale

•  Improves 65.3% of sentences examined

•  Yields increases of +0.9 BLEU to +3.8 BLEU, depending on size of error span, in an oracle study.

Monolingual  Human  Par.cipa.on  

Quality  

Afforda

bility  

Space of Available Translation Options

• Online system for collaborative monolingual translation • Asynchronous edits, short tasks, minimal context

repeat …

Target Language Source Language MT

Sentence   Sentence  

1   Vote  2   Identify error spans  

3   Create new translation candidates  1   Vote  

2   Paraphrase  

3   Explain errors  

Crowd Tasks:  

New  candidate  1  2  3  

MT and

word alignment

…

Crowd Tasks:  

The Bigger Picture: MonoTrans Features Results

Translations and Human Judgments

Original  Google  Translate  output  (GT),  with    transla.ons  produced  by  the  targeted  paraphrase  transla.on  process  (TP),  selected  to  show  a  range  from  strong  to  weak  improvements  in  fluency.    Fluency  and  adequacy  (preserva.on  of  meaning)  were  rated  on  a  5-­‐point  scale  by  independent  judges  not  familiar  with  this  research.  

Paraphrases from the Crowd

68% of sentences translated perfectly with no bilingual humans