Epilogue to the Special Issue on the Nature and Measurement of Second Language Listening

This special issue is another important step to further our understanding of some of the key areas relevant for L2 listening assessment and pedagogy. The individual studies in the issue addressed a variety of topics in different contexts and the findings are relevant for language researchers and teachers alike. In this epilogue, I will summarize the studies and draw conclusions with regards to practical recommendations and future research in this area.

Papers in the Special Issue

In the first paper, Suvorov and He presented a synthesis of research on the use of visuals in L2 listening assessment, an important topic considering the increasing impact of technology on the listening construct. The synthesis was based on detailed inclusion and exclusion criteria and considered not only peer-reviewed journal articles, but also book chapters, conference proceedings, and PhD theses. Overall, the authors collaboratively coded 45 studies in terms of methodological aspects such as research design, study and instrument characteristics, and test administration procedures. Through this analysis Suvorov and He identified several key areas for improving future research. Their findings show that previous studies often lacked methodological rigor, such as incomplete information about participants, failure to report reliability of test instruments, lack of information about which types of visuals or items were used, or missing details about test administration. Other aspects reported by the authors are the need for more studies using eye-tracking technology, the application of advanced statistical methods, the recruitment of diverse learner samples, and increased data transparency. Suvorov and He’s study is important as their advice, if heeded by researchers, will help make research findings more comparable across contexts. The results are not only relevant for studying the use of visuals in L2 listening assessment, but they are applicable for listening research more generally.

A different topic was addressed in the paper by In’nami and Koizumi, whose study adds to the body of research on metacognitive awareness and how it influences L2 listening success. Similar to the issue of video-based listening, the role of individual difference factors such as metacognition has received renewed interest in research on listening. In’nami and Koizumi studied metacognitive awareness in the Japanese university context by comparing responses to the Metacognitive Awareness Listening Questionnaire (MALQ, Vandergrift et al., 2006) with results on the Test of English as a Foreign Language (TOEFL). Their study is unique in that they included two similar groups of learners, used two different versions of the TOEFL (the paper-based ITP and the computer-based iBT), and analyzed the data through random forest analysis. The authors report that across the two groups of learners and tests, the only dimension of the MALQ consistently related to listening performance was person knowledge. Another interesting finding of the study was that the 5-factor structure of the MALQ proposed by Vandergrift et al. (2006) was not supported, which suggests that the validity of widely used research instruments such as the MALQ should not be taken for granted when using them in new contexts but needs to be reestablished.

In the third paper, Fujita investigated the impact of noise on test takers’ use of contextual information. The issue of background noise is a controversial topic in language testing, however considering that some level of noise is present in most real-world listening situations, it seems that including noise in L2 listening tests would increase authenticity and thereby enhance construct validity. Fujita’s findings support this argument. By analyzing think aloud protocols and interview data of 7 Japanese undergraduate students who completed a sentence-level noise test as well as an English proficiency test, Fujita found that participants used contextual information more often in low-to-moderate noise conditions compared to conditions with no-noise and extreme noise. Although the study was small-scale, these findings are important, as they show that different parts of the construct are tested depending on the level of background noise in the recordings. Fujita also reported strong individual differences between learners, so a larger study including more participants may be necessary to generalize findings and to identify underlying patterns.

The study by Sok and Shin looked at the interplay of aptitude and metacognitive awareness and their effects on L2 listening performance. Research on the role of aptitude in L2 listening is sparse, which makes this an important investigation. An advantage of the study over previous research is that the authors used regression analyses and bootstrapping techniques to analyze the data, which arguably leads to more robust findings than simple correlational analyses. In addition, studying how different individual difference variables interact to predict L2 listening success is a neglected area of research (Holzknecht & Brunfaut, in press). Sok and Shin found that aptitude and metacognitive awareness are interdependent constructs, but also that aptitude explained 36% of the variance in L2 listening performance with a medium effect size, while metacognitive awareness only accounted for 5% with a small effect size. This suggests that aptitude may play an important role in L2 listening success, particularly for young beginner learners in formal settings, which was the context of the study. The authors also observed that the MALQ may not fully capture young learners’ metacognitive awareness due to its dependence on self-reported data.

The two studies in the special issue on metacognitive awareness by In’nami and Koizumi and Sok and Shin are also good examples of how different learner groups and research contexts can influence a study’s outcomes. Although both studies used the MALQ and were based on a similar research design, In’nami and Koizumi investigated Japanese university students and their performance on two different TOEFL listening tests, whereas Sok and Shin focused on 12-year-old Korean learners taking the Wechsler Individual Achievement listening test. While In’nami and Koizumi report that only person knowledge on the MALQ was related to listening success across both learner groups, Sok and Shin found that problem solving, mental translation, and planning and evaluation on the MALQ correlated significantly with L2 listening success. This suggests that the relationship between L2 listening and metacognitive strategies is content-specific and depends on the features of test takers and assessment instruments.

Finally, another individual difference factor in L2 listening was the focus of the paper by Von Zansen, Hilden, and Laihanen, who investigated to what extent English high stakes multimodal listening test items function differentially for female and male test takers. While the influence of test takers’ identity on their L2 listening performance is under-researched, gender is the identity variable that has received most attention from listening researchers in the past. Von Zansen et al.’s investigation adds to this body of studies by looking at gender-based DIF in a new context (the Finnish matriculation exam), on a large sample size (over 20,000 students), and with a multimodal listening test (video-based and picture-based tasks). The authors’ detailed Rasch analyses revealed several items displaying statistically noticeable gender based DIF; however, the observed differences were small. Still, the authors rightly suggest investigating these items further to avoid introducing bias in the test results.

Conclusion

In sum, the special issue covered a wide range of topics on the nature and measurement of L2 listening, thereby illustrating the complexity and multi-facedness of the listening construct, and at the same time highlighting that more research is needed to help us gain a deeper understanding of this crucial language skill. The major themes addressed in the issue included the use of visuals in L2 listening assessment (Ruslan Suvorov and Shanshan He), the effects of background noise on L2 listening test takers’ performance (Ryoko Fujita), and three papers on individual difference factors in L2 listening including the role of metacognitive awareness across learner groups and tests (Yo In’nami and Rie Koizumi), the interplay of language aptitude and metacognitive awareness and their impact on L2 listening performance (Sarah Sok and Hye Won Shin), and the effects of test takers’ gender on test outcomes (Anna von Zansen, Raili Hilden, and Emma Laihanen). Based on the findings presented in the special issue, classroom teachers and test developers should consider the inclusion of visuals and background noise in recordings, rethink the role of aptitude and metacognitive strategy instruction for different learner groups, and continue to scrutinize test items to avoid bias.

In terms of theoretical contribution, the individual papers showed that applying established research instruments and methods in new contexts and with new learner groups, as well as studying the interaction of individual difference factors, can lead to important novel insights. Suvorov and He’s paper also emphasized that research on L2 listening should strive toward increased methodological rigor to improve comparability across contexts and should not shy away from using new technologies such as eye-tracking. Seeing that the listening construct has changed markedly through the impact of technology, this advice seems crucial for advancing previous research conducted in more traditional settings.

Future research on L2 listening should also think about the application of advanced methods such as neuroimaging techniques, and first studies in this realm show promising results (Aryadoust et al., 2021). Although brain imaging has been used to some extent in psycholinguistics research on L2 listening, mostly in controlled experimental setups to study individual variables in isolation, its application for investigating learners’ processes while they are engaged in complex real-world listening tasks such as L2 listening tests is only just beginning. Studying learners’ listening processes through neuroimaging may offer new insights, particularly in triangulation with more established methods.

Real-world listening demands are likely to keep changing over the coming years due to ongoing technological advances across domains. Researchers will need to stay at the forefront of these developments to ensure that students are taught L2 listening effectively, and that assessment instruments stand up to the scrutiny of validity arguments.

Disclosure Statement

No potential conflict of interest was reported by the author(s).