Evaluating Fairness and Justice of External English Language Test Score Interpretation and Use for Japanese University Admission

ABSTRACT

The study evaluated a recently postponed national test policy on the use of external agencies’ English tests measuring four skills for Japanese university admission purposes. Using Kunnan’s principles of fairness and justice, we generated and evaluated three claims regarding the test policy: vocabulary coverage of the tests, the justifiability of the Common European Framework of References for Languages (CEFR)-based concordance table, and tests’ impact on teaching and learning productive skills. Scrutinizing the backing and rebuttal evidence from the test agencies’ voices to the key literature, we arrived at a partial acceptance of all three claims with the need for further research, calling into question a optimistic outlook underlying the test policy.

要旨

本研究の目的は、日本国内の共通大学入学試験で予定され延期されることとなった、英語外部試験を利用するテスト政策の評価である。ここでは Kunnan の提案する「公正・正義の原則」を利用して、この政策が前提とする主張3点を評価した。その3点とはテスト語彙範囲の適切性、外部試験と CEFR 対照表の正当性、話す・書く技能の指導と学習への波 及効果である。試験運営機関へのアンケート調査と文献等による裏付けと反証を精査し た結果、3つの主張はいずれも部分的に支持された。しかし、この外部試験利用のテスト 政策を支える楽観的な見通しには疑問を呈さざるをえず、さらなる研究が求められる。

Acknowledgement

An earlier version of this paper was presented at the symposium in the 22nd Conference of Japan Language Testing Association held in Niigata. Sincere appreciation goes to our discussant, Dr. Antony Kunnan, for his insightful comments and answering many questions of ours and to Yo In’nami for his contribution in the early stages of this study. We also thank Tatsuro Tahara, Hiroya Tanaka, and LAQ reviewers for their insightful comments. However, the authors are solely responsible for any errors.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

¹ Actual wording in an official English document reads “thinking capacity/decisiveness/ expressiveness” (MEXT, 2011).

² In addition, it was reported that the education minister’s gaffe and other political blunders prompted the administration’s decision to put off the introduction of the new test policy.

³ Toulmin (2003) calls this “data,” but we use “grounds,” in line with Kunnan (2018) to avoid confusion.

⁴ In Toulmin’s (2003) model, qualifiers are derived from rebuttals. The present study adopts Kunnan’s modified model (Kunnan, 2018).

⁵ The Eiken representative acted as a spokesperson for both Eiken and TEAP.

⁶ This website was discontinued after the decision to postpone the introduction of the external four-skills tests was made by MEXT.