The Minimal English Test: a new method to measure English as a Second Language proficiency

Abstract

Maki, Wasada, and Hashimoto developed the Minimal English Test (MET), which is a five-minute English as a Second Language (ESL) test. The Maki Group has, since the development of the MET, investigated correlations between the scores on the MET and the scores on other ESL tests, the reliability of the MET, and what kind of ESL proficiency the MET measures. They found that there were statistically significant, relatively strong correlations between the scores on the MET and the scores on other ESL tests, that the scores on the MET were reliable, and that the MET, which appears to be a listening comprehension test, actually measures both reading and listening proficiency, rather than one or the other of them. The findings in this paper suggest that the MET, a radically time-saving test, can be used as a useful ESL test in ESL research as well as ESL classes.

Keywords:

• validity
• reliability
• testing

Acknowledgements

Earlier versions of this paper were presented at the following conferences: the European Second Language Association 2005 Conference in Dubrovnik, Croatia; the 2006 KALS-KASELL International Conference on English and Linguistics in Pusan, Korea; the 36th Chubu English Language Education Society Conference in Wakayama, Japan, in 2007; the 37th English Language Education Society of Japan held in Tokyo, Japan, in 2007; the 42nd Annual TESOL International Convention in New York, USA., in 2008; and the 18th International Congress on Linguistics in Seoul, Korea, in 2008. We would like to thank the audience at the meetings for their suggestions on earlier versions of this paper.

We are also indebted to two anonymous reviewers of Evaluation & Research in Education, Chunhua Bai, Masahiko Date, Jessica Dunton, Eiko Esaka, Alexandra von Fragstein, Yumiko Fukunaga, Shinya Furukawa, Michiyo Hamasaki, Megumi Hasebe, Ekuko Hashimoto, Tomohiro Hashimoto, Yohey Hashimoto, Takaki Hiraishi, Kenyu Ichihara, Hirotaka Imamaki, Akane Ishikawa, Takane Ito, Hironobu Kasai, Dae-Jin Kim, Hee-Won Lee, Myung-Hwan Lee, Sakae Mitsui, Yoichi Miyamoto, Tamami Morishima, Yuka Morita, Kosuke Nagasue, Fumikazu Niinuma, Masao Ochi, Akina Okada, Satoshi Oku, Kenjiro Tagawa, Bill Perry, Lynne Roecklein, Naoki Sugino, Kazushige Takahashi, Takaharu Takamura, Ryoko Tsuruta, Asako Uchibori, Yukiko Ueda, Hiroaki Wasada, Jeong-Hee Yoo, and Yoko Yumoto for their useful comments which helped to improve this paper.

We are also grateful to William Edmunds, Masahiro Goto, Norio Hirota, Hidetoshi Inoue, Jeong-Seok Kim, Yuko Nagao, Kazuo Nakagawa, Kenichi Nakamura, Norio Nakamura, Yasuo Nishizawa, Ping Qu, John Russell, Toshiya Sato, Yasuko Setozaki, Hong-Qiang Sheng, Junko Shibata, Yoko Sugiyama, Masaru Uchida, Ying-Ling Yang, and Yong-Shun Yin for cooperating to collect data.

Research by the second author was supported in part by the 2002 and 2003 Gifu University Research Grants (Kasseeka Kenkyuu Hi: Hooga Kenkyuu) #1 and #13, and the 2004 Gifu University Exchange Program Grant provided by Taguchi Fukujukai.

All errors are our own.

Notes

1. We follow Yanai (1998) in interpreting values of correlation coefficients. She assumes the following correspondence between correlation coefficients and their characteristics.

2. Lado (1961) offers the following interpretations of the reliability coefficient:

 In general, written tests may be expected to show a higher coefficient of reliability than oral and auditory tests. Good vocabulary, structure and reading tests are usually in the 0.90–0.99 range. Auditory comprehension tests are more often in the 0.80–0.89 range.…

Therefore, since the MET measures both reading and listening ESL proficiency, as we will seen above, the reliability coefficient of the MET is considered to be in the 0.80–0.99 range.

3. Bachman (2004, p. 173) provides an explicit explanation of the CI.

 The three most commonly used probabilities for CIs are 0.68, 0.95 and 0.99. We choose the appropriate level of probability depending on how confident we want to be in our estimate of the test taker's ‘true scores’. In high-stakes situations … we would choose the 0.99 probability level … On the other hand, for low-stakes situations, where we may be willing to accept a lower degree of confidence, we might choose the 0.68 probability, which means that we would be 68% certain that the test taker's ‘true score’ is within the CI.

(68% CI) CI_0.68=X±1.00 SEM

(90% CI) CI_0.90=X±1.96 SEM

(95% CI) CI_0.95=X±2.58 SEM

4. We did not use the scores on the UEE 2002–2005, the CSAT, and the SE in this survey, since the scores on each of the listening and the reading parts were not obtained. The UEE 2002–2005 and the SE contain just the reading part and do not have the listening part. Also, although the CSAT consists of both the reading and the listening parts, the scores on each part were not available.

5. One of the reviewers of this paper raised this issue: ‘Considering that, as mentioned here, the ratio of the scores between the reading and the listening parts is 4:1, it might indicate that the testees were completing highly predictable answers on the basis of reading rather than listening’. However, this does not seem to be the case. This is because no statistically significant differences were found between the two correlation coefficients for the scores on the Reading section and for those on the Listening section, on both the UEE 2006 and the UEE 2007, by the Fisher r-to-z transformation provided by Vassar Stats: Web Site for Statistical Computation (2009). (According to Vassar Stats: Web Site for Statistical Computation (2009), the Fisher r-to-z transformation calculates a value of z that can be applied to assess the significance of the difference between two correlation coefficients, r _a and r _b, found in two independent samples.) The significance level was set at 0.05.

First, there was no statistically significant difference between the two correlation coefficients for the scores on the Reading section and for those on the Listening section on the UEE 2006 (p two-tail = 0.30), as shown in Table 5.

Second, there was no statistically significant difference between the two correlation coefficients for the scores on the Reading section and for those on the Listening section on the UEE 2007 (p two-tail = 0.50), as shown in Table 6.