MT and Post-Editing in Master's Level Translation Education
Constantine the Philosopher University (UKF) Nitra, SlovakiaDaša Munková, Jakub Absolon
User conference MT@Work 2015 (4 December — Brussels)
MT a PEMT in Current System of Translation Studies
Goals:• Make know principals of
MT• Basic of Post-Editing of
Machine Translation• Errors typology, most
frequent errors for EN-SK made by MT engines
• PEMT practice is a part of internship program
Courses with content connected with PEMT
• Identify differences in job description of Translator vs. Post-Editor• Trainability of PE skills and
competences• Necessary workload for PEMT
training• Recommended Methodology for
PEMT (e.g. split techniques)Post-Editor
Interpreter
Translator
Literary Translator
Challenges and Questions
Pilot Study Goals
• Find out the attitude of future translators to PEMT process (before and after pilot study)• Find out increase/decrease of productivity
comparing PEMT with HT• Find out if the productivity increase will be
the same for fast and slower translators (correlation between HT and PEMT)• Find out if there will be significant increase
of productivity after short exercises.
MT, format and text characteristics• We use Google Translate API integrated into Memsource CAT• We use exported tables for revision in word format• We use three types of texts:
HT and PEMT AMT type: statistical MTMT Specification: generalMT: Google TranslateDocument Type: IT, SW user manualReadability Index (Flesch-Kincaid): 55.7Average Grade Level: 10.6
EVALUATIONMT type: statistical MTMT Specification: generalMT: Google TranslateDocument Type: IT, SW user manualReadability Index (Flesch-Kincaid): 12.7Average Grade Level: 20.3
PEMT BMT type: statistical MTMT Specification: generalMT: Google TranslateDocument Type: IT, SW user manualReadability Index (Flesch-Kincaid): 54.5Average Grade Level: 11.3
Activities in Pilot Study
PEMT basics MT Quality and usability evaluation (IT text)
HT(IT text)
MT Quality and usability evaluation (adhesives leaflet)
PEMT(IT text)
HT(IT text)
PEMT(human rights text)
PEMT(IT text)
MT Quality and usability evaluation (HR text)
PEMT(human rights text)
1.
2.3.4.5.
6.
7.8.9.
10.
Who were our respondents?
39 first year MA university students of Translation Studies in UKF (language pairs EN-SK)
Do you have any experience with commercial translation?
• No I do not translate (only at school) - 79.4% • I am translating at least 5 pages per
week - 20.6 %
Have you ever metwith demand for PEMT?
• I've never heard of PEMT 17.6%• Personally, no, but I heard about
PEMT 52.9%• Yes, but I have no practical
experience 26.5%• I have practical experience with
PEMT 2.9%
How Do You Use MT?
• I translate unknown words 55.9%• I translate a whole sentence and then
edit them 32.4%• I only use MT for unknown languages
11.8%• I do not use it at all 0%
What Do You Think About the Quality of MT?• It is a high quality translation 0%• MT quality varies, but is a great help
55.9%• MT helps, but it must not be
automatically copied to target 35.3%• MT slows down the process of
translation 8.8%• MT is absolutely unacceptable 0%
Translation Studies students attitude toward PEMT before and after pilot study• There was significant
improvement in students attitude to PEMT although initial level was very high in comparison to average translator
before after
Switch on MT
Switch on MT
Switch-off MT Switch-off
MT
Find out increase/decrease of productivity comparing PEMT with HT
0.0% 50.0% 100.0% 150.0% 200.0% 250.0% 300.0%0
0.2
0.4
0.6
0.8
1
1.2
1.4
PEMT vs. HT acceleration
Possible reason:• Using MT is very natural for young people• Well selected text for MT• Students were not experienced translators
Find out if the productivity increase will be the same for fast and slower translators (correlation between HT and PEMT)
We found no linear dependence between • speed of HT speed and speed of PEMT• HT quality and speed of PEMT
Possible reasons:• There are other KPIs (further research needed)
higher quality lower quality
Find out if there will be significant increase of productivity after short exercises
We found no significant differencePossible reasons:• Not enough amount of
exercises (further research needed)
Added value of Study for students• Familiarizing students with the theory and current practice in the
PEMT• We gave them the opportunity of direct comparison and HT PEMT• We familiarize them with the system of human assessment PEMT• We taught them to identify major MT errors, to be able to react more
quickly on them.• We gave them possibility to measure their potential in PEMT, to be
able to decide if the PEMT will be one of their future professional activity.
Future plans• Continue in PEMT activities• Enhance research focused on identification
of KPI for PE, creating specific competency based model of PE, which can be used for PE recruitment and selection• Identification of trainable skills and
competencies• Creation of methodology for PEMT training• Creation of error typology specific for MT in
EN SK language combination
Our Goal:Carefully selected
and well trained Post-Editor
Arnold Turned His Biggest Weakness His Calves Into A
Showcase Muscle
Pilot Study Results
Based on our study we find out:• Rýchlosť prekladu (predpokladaná šikovnosť) rastie priamo úmerne
s rýchlosťou PEMT, nepotvrdil sa teda náš predpoklad, že najväčšie zrýchlenie dosiahnu tí najpomalší v preklade• Všetci testovaní dosiahli v PEMT vyššiu rýchlosť oproti HT• Priemerné zvýšenie rýchlosti oproti HT bolo 2,5 násobok čo predstavu
vyššiu hodnotu než sme predpokladali na základe výskumu (xxx)• Zvýšilo sa pozitívne vnímanie PEMT (z 91% na 99%, už na začiatku sa
prejavil mladý vek testovanej vzorky v pozitívnom postoji k MT)
Skóre:1 – 100 percente akceptovateľný preklad2 – akceptovateľný preklad (vyžaduje si menšie opravy , ale oplatí sa mať vetu predpreloženú)3 – čiastočne akceptovateľný preklad (vyžaduje si opravy, a neoplatí sa mať vetu predpreloženú)4 – neakceptovateľný preklad5 – neviem posúdiť Typológia chýb podľa DGT:slovosled - Wgramatické chyby - Gterminologické chyby- Tchyby vo význame - Mostatné – O