The assessment of translation accuracy of the Lexica Machine Translation System

Abstract

This article focuses on the research of the English–Afrikaans language pair development team and aims to establish an assessment method and procedure that will assess the translation accuracy of the Lexica Machine Translation (MT) System in an easily repeatable, scientifically acceptable way. Lexica is a transfer system that is used to carry out morphological, syntactic, semantic and contextual analysis and can be used for the following language pairs: Afrikaans, Tswana, Swahili and Portuguese to English; and English to Xhosa, Zulu and Afrikaans. The research has shown that there are no universally accepted and reliable methods and measures, and that assessment methodology has been the subject of much discussion in recent years. To assess the accuracy of translation of the Lexica MT System, diagnostic assessment was determined as the most suitable mode of assessment, as the focus of such assessment is on the identification of lim- itations, errors and deficiencies, which may then be corrected or improved by the development team. A method and procedure were developed according to which marks were awarded in terms of the following two aspects: (a) preservation of meaning, (b) grammatical correctness.