The focus of this controlled eye-tracking and key-logging study is to analyze the behaviour of translation professionals at the European Commission's Directorate-General for Translation (DGT) when detecting and correcting errors in neural machine translated texts (NMT) and their post-edited versions (NMTPE). The experiment was informed by quality analyses of an authentic DGT parallel corpus (Vardaro, Schaeffer, and Hansen-Schirra 2019), consisting of English source texts and corresponding German NMT, NMTPE and revisions (REV). To identify the most characteristic error categories in NMT and NMTPE, we used the automatic error annotation tool Hjerson (Popovic 2011) and the more fine-grained manual MQM framework (Lommel 2014). Results show that quality assurance measures by post-editors and revisors at the DGT are most often necessary for lexical errors. More specifically, if post-editors correct mistranslations, terminology or stylistic errors in an NMT sentence, revisors are likely to correct the same type of error in the same sentence, suggesting a certain transitivity between the NMT system and human post-editors. In this study, carried out in Translog II (Carl 2012), participants' eye movements and typing behavior for test sentences where the error categories mistranslation, terminology, function words and stylistic errors are included will be compared to control sentences without errors. 30 language professionals from the DGT post-edited 100 English-German machine translated sentences from the DGT corpus. We examine the three error types' effect on early (first fixation durations, first pass durations) and late eye movement measures (e.g., total reading time and regression path duration) and on typing behaviour. Statistical regression analyses predict the temporal, technical, and cognitive effort during the DGT post-editing and revision process which will be corelated to the recognition and correction of said error categories. In addition, the behavioural data of the DGT translation professionals will be compared to those of a group of 30 translation students. Behavioural differences in the two groups will allow for further predictions regarding the effect of expertise on the post-editing process.in.
展开▼
