developing a specialized corpus for ata translator ...ruttan/capstone/lectures/translation.pdf ·...
TRANSCRIPT
![Page 1: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/1.jpg)
Developing a Specialized Corpus for ATA Translator Certification Examinations
Geoffrey S. Koby Kent State University,Ohio, USA
![Page 2: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/2.jpg)
Slide 2 of 41
American Translators AssociationTranslator Certification Exam
• Examinations currently handwritten(transitioning to computerized)
• Data starting with 2006 testing year
• Design of certification exam determines corpus design
![Page 3: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/3.jpg)
Slide 3 of 41
Exam Design• 23 language pairs• Language > English or English >
language• Three source texts - about 250 words • Candidates translate two passages:• Mandatory general passage (labeled
“A”)• Either a semi-technical passage (“B”)
or a commercial/legal passage (“C”)
![Page 4: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/4.jpg)
Slide 4 of 41
Annual Statistics• In 2006, 535 candidates• Spanish>English/English>Spanis
h about 55%• 1070 passages (x two graders)• 2140 marked passages• Estimated 535,000 words for
2006• 7,585 examinations 2001-2011• Corpus grows 690,000 words
per year
![Page 5: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/5.jpg)
Slide 5 of 41
American Translators AssociationTranslator Certification Exam
• Graders mark passages• Error codes indicate
category/severity • Target text shows scope, type
and severity of various translation errors
![Page 6: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/6.jpg)
Slide 6 of 41
Sample Marked Target Text Error scope
marked explicitly
Error type and severity marked
explicitly
![Page 7: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/7.jpg)
Slide 7 of 41
Unique features of the corpus
• Multidirectional parallel corpus• All exam texts selected according
to same criteria• Common language is English• One-to-many alignment between
one source and multiple target texts
• Range of quality• Explicit error markings
(scope, severity, category)• Designing a database tool:
![Page 8: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/8.jpg)
Slide 8 of 41
Op
enin
g S
creen
![Page 9: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/9.jpg)
Slide 9 of 41
En
ter
New
Sou
rce
![Page 10: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/10.jpg)
Slide 10 of 41
Revie
w S
ou
rce T
exts
![Page 11: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/11.jpg)
Slide 11 of 41
Revie
w S
ou
rce T
ext
![Page 12: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/12.jpg)
Slide 12 of 41
New
Exam
Reco
rd
![Page 13: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/13.jpg)
Slide 13 of 41
Targ
et
Text
Qu
eue
![Page 14: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/14.jpg)
Slide 14 of 41
En
ter
Targ
et
Text
Target Passage Input
Submit Target
![Page 15: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/15.jpg)
Slide 15 of 41
Targ
et
Text
Qu
eue
![Page 16: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/16.jpg)
Slide 16 of 41
Revie
w T
arg
et
Pass
ag
eTarget Passage Input
Submit Target
![Page 17: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/17.jpg)
Slide 17 of 41
Gra
din
g Q
ueu
e
![Page 18: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/18.jpg)
Slide 18 of 41
Grading Screen
![Page 19: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/19.jpg)
Slide 19 of 41
Entering an Error
![Page 20: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/20.jpg)
Slide 20 of 41
Enter Error Category
![Page 21: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/21.jpg)
Slide 21 of 41
Enter Error Severity
![Page 22: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/22.jpg)
Slide 22 of 41
ATA
Flo
wch
art
![Page 23: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/23.jpg)
Slide 23 of 41
Normal/Disjunct Error
![Page 24: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/24.jpg)
Slide 24 of 41
Vie
win
g G
rad
ed
Exam
s
![Page 25: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/25.jpg)
Slide 25 of 41
List
of
Gra
ded E
xam
s
![Page 26: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/26.jpg)
Slide 26 of 41
Viewing Graded Exam
![Page 27: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/27.jpg)
Slide 27 of 41
Viewing Error List
![Page 28: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/28.jpg)
Slide 28 of 41
Analy
sis
Op
tion
s
Already coded queriesUser-guided queriesStatistical analysesKeyword in Context
![Page 29: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/29.jpg)
Slide 29 of 41
Analysis Options - Preliminary
Pass-fail analysis Average score analysis
Error analysis
![Page 30: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/30.jpg)
Slide 30 of 41
Sele
ct S
ou
rce L
an
gu
age
![Page 31: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/31.jpg)
Slide 31 of 41
Sele
ct T
arg
et
Lan
gu
age(s
)
![Page 32: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/32.jpg)
Slide 32 of 41
Pass
/Fail
An
aly
sis
![Page 33: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/33.jpg)
Slide 33 of 41
Avera
ge S
core
An
aly
sis
![Page 34: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/34.jpg)
Slide 34 of 41
Err
or
An
aly
sis
by P
art
![Page 35: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/35.jpg)
Slide 35 of 41
Err
ors
by Q
uart
er
![Page 36: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/36.jpg)
Slide 36 of 41
Keyw
ord
in C
on
text
Not yet implemented
![Page 37: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/37.jpg)
Slide 37 of 41
Handel, Clearing und Settlement sehr zu begrüßen und ökonomisch sinnvoll, so dass mit
trade, clearing and settlement is a very welcome trend and makes economic sense to both GID024:1A
trading, clearing, and settlement is very much to be greeted and economically makes sense
GID024:4MT-GID156:4MT
trading, clearing and settlement is quite welcome and makes economic sense to large inves
trade, clearing and settlement welcoming and economically sensible, which means that furt
GID023:MT4-GID165:4MT
trading, clearing, and settlement are very welcome and make economic sense so that ad
GID023:2MT/MU/G-GID024:1G GID024:1P
trade, clearing and settlement is very welcome and economically sensible, such that further GID023:2MT-GID200:2MT-GID165R:2MT GID023:1U-GID200:1U-GID165R:U1
Trading, Clearing, and Settlement is very appreciated and economically sensible, so that aGID156:1C-GID200:1C GID156:2MT-GID200:1U+2MT GID200:1L
trade, clearing and settlement is very desirable and makes economic sense for both for the
GID 023:MT8 Candidate lost his place, destroyed sense of text. – GID165: Mixed up reading of source text 4 MU
commerce , Clearing and Settlement is highly appreciated and economically useful , so thGID165:2MT|GID165:1C|GID165:1C|GID156:2MT-GID165:2MT GID156:2MT-GID165:2MT
KW
IC w
\Mark
ed
Err
ors
![Page 38: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/38.jpg)
Slide 38 of 41
Summary and Outlook
• Multidirectional parallel corpus of translation exams
• One-to-many alignment between one source and multiple target texts
• Future research possibilities
![Page 39: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/39.jpg)
Slide 39 of 41
Future Research Possibilities 1
• Sorting/analysis of translation errors as marked by the graders.
– error type, error severity, language pair, etc.
• Patterns of errors, types of errors, etc.
• Analyze terms by Part of Speech• Search, view, analyze by various
tags• Pass/fail rates & score ranges by
language pair and for the corpus as a whole
![Page 40: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/40.jpg)
Slide 40 of 41
Future Research Possibilities 2
• Clustering of score ranges above and below the pass/fail boundary
• Inter-grader/intra-grader consistency and variability in error discovery/assessment
• Congruity in discovery of translation errors
– (i.e., the ratio of errors marked by both graders to those marked by only one of the two)
• Investigation of congruity in marking of severity and category of translation errors
![Page 41: Developing a Specialized Corpus for ATA Translator ...ruttan/capstone/lectures/translation.pdf · Corpus for ATA Translator Certification Examinations Geoffrey S. Koby Kent State](https://reader036.vdocuments.site/reader036/viewer/2022081523/5fd8f8a53675d356ec24e6be/html5/thumbnails/41.jpg)
Slide 41 of 41
Thank You for your Attention! • Dr. Geoff Koby – [email protected]
• For a complete description of the ATA Examination Program, see:
Koby, Geoffrey S. and Gertrud G. Champe. “Welcome to the Real World. Professional-Level Translator Certification.” Translation & Interpreting 5:1 (2013): 156-173