Translation quality assessment: a critical methodological review

ABSTRACT

Translation quality assessment (TQA) represents one of the most productive lines of research in Translation Studies (TS), with scholars proposing different TQA models. In parallel to these theoretical models is the emergence of a diverse array of assessment methods trialled and used to evaluate translation quality in different settings (e.g., error analysis, corpus-based evaluation, rubric scoring, calibration of dichotomous items, Thurstonian comparative judgement). Despite these developments, it seems that little effort has been made to bring to the fore methodological aspects of the TQA practices. Against this background, this methodological review sets out to track the developmental trend of the TQA methods, hoping to bring clarity to the nature of the methodological pluralism, driving forces behind methodological evolution, as well as promises and constraints of each methodological choice. The review also attempts to interrogate the TQA practices, focusing on three overarching issues of reliability, validity and practicality. By doing so, we hope to highlight their merits and demerits, and occasionally challenge their legitimacy. Finally, the review describes some gaps in TQA practice and research, and suggests potential directions to move forward the enterprise of TQA.

KEYWORDS:

• Translation quality assessment
• assessment methods
• validity
• reliability
• practicality

Disclosure statement

No potential conflict of interest was reported by the author.

Notes

1. We want to differentiate two approaches in TQA: one is to compare different versions of translation of the same source text; the other is an individual translation where there is no other translation to compare it with than that in the mind of the assessor. Both approaches can be practised in TQA. For example, the assessor may have a reference/exemplar translation with which renditions by different translators are compared.

2. We are aware that the issue of reliability does not concern raters only; it is, in fact, an issue of score generalizability/dependability as a function of multiple assessment facets including persons, tasks, raters, and assessment occasions. Rater reliability is discussed here, because TQA is essentially rater-mediated, and because the rater facet has been a traditional concern in translation assessment.

3. Note that both reliability and validity are relative concepts, meaning that they are not a matter of yes or no but a matter of degree (a continuum). Requirements on the degree of reliability and validity could vary, depending on the stakes involved in an assessment.

Additional information

Funding

This work was supported by the China National Social Sciences Foundation [18CYY010].

Notes on contributors

Chao Han

Dr.Chao Han’s research interests include testing and assessment of translation and interpreting (T&I), evidence-based T&I studies, and methodological aspects of T&I studies.