• Aiman Eid Al-Rawajfeh

  Author

Educational facilities are turning to automatic translation for multilingual students; translation quality becomes crucial for the effectiveness of teaching processes. However, most common evaluation criteria tend to favor general fluency rather than domain relevance. In order to fill this gap, the paper offers an evaluation of machine translation (MT) quality in an educational context based on a multi-criteria, theoretically based approach that takes into account factors such as functional adequacy and reader comprehensibility. Three different systems (a general neural engine, System A; a general cloud service, System B; and an education-specific system, System C) have been tested on the basis of the 60-passage corpus (360 translations in total) covering both science and humanities. Human raters with specific training evaluated the output of translation according to the following criteria (rating from 1 to 5): adequacy, fluency, terminology accuracy, cultural relevance, and readability. However, System C performed better in terms of criteria essential for learning effectiveness, obtaining the best results in adequacy (4.4 compared to 4.1 for A and 3.8 for B), terminological accuracy (4.5 compared to 3.6 for A and 3.9 for B), cultural appropriateness (4.2 compared to 3.4 for A and 3.2 for B), and readability (4.3 compared to 4.0 for A and 3.7 for B). For all the systems, the criterion of cultural appropriateness had the lowest score. Thus, it can be concluded that fluency in itself cannot serve as a good measure for educational translation.

• PDF