The Evolution of Assessment in English Pronunciation: The Case of Hong Kong (1978-2018)

ABSTRACT

This study tracked the development of Hong Kong’s assessment practices for English pronunciation over the past four decades, with reference to the nativeness and intelligibility principles in L2 pronunciation research and pedagogy. Specifically, it evaluated changes in assessors’ comments on candidates’ English pronunciation performance in school-exit public examinations between 1978 and 2018. Qualitative and quantitative content analyses were conducted on the examination report for each year to identify themes related to candidates’ pronunciation ‘problems’, including ‘word-based’ features (word pronunciation, word stress, segmentals), ‘discourse-based’ features (suprasegmentals) and ‘delivery’ (clarity, fluency, loudness, naturalness, pacing). In the examination reports, candidates’ problems with word-based features (particularly word pronunciation) received the most attention across the decades. Most of the comments in later reports were aligned with the intelligibility principle, particularly at the segmental level (e.g., missing consonants, simplification of consonant clusters, word pronunciation). These assessment practices were potentially influenced by the teaching methods recommended in the different ELT curricula over time (i.e., from an oral-structural to a communicative/task-based language teaching approach), and also by the assessors’ judgements. The paper concludes by proposing a research agenda for the promotion of an evidence-based approach that can inform future assessment practices.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.

Notes

¹ The ‘three circles’ model was put forward by Kachru (1985, p.366–337), who classified Englishes according to their sociolinguistic profile. The inner circle refers to the ‘traditional cultural and linguistic bases of English’ (e.g., the USA and UK). The outer circle represents ‘the institutionalised non-native varieties’ in regions that ‘have passed through extended periods of colonisation’ (e.g., Singapore, the Philippines). The expanding circle includes ‘the regions where the performance varieties of the language used essentially in EFL (English-as-a-foreign-language) contexts’ (e.g., China, Korea).

² The relationship between WE and ELF research has been controversial in the literature. For example, Bolton (2018) considered ELF as an extension of broad WE, an ‘approach to international English focusing on those contexts such as universities and international business, where English is used as a common language by speakers of different nationalists and linguistic backgrounds’ (p.6). Jenkins (2018) argued that ‘ELF and WE are not competing, but complementing, paradigms’ (p.12). This paper recognises the complementary relationship between ELF and WE, despite their respective research foci on international communication and the outer circle.

³ The notion of teachability has been controversial in the literature as it is difficult to measure and it focuses more on the teacher than on the learner (Levis, 2018). Levis argued that the teachability (or learnability) issue has greater importance if it is seen as an argument that ‘certain features are not acquirable in the long run, no matter what we do to teach effectively or no matter how much effort learners put into learning’ (p.213).

⁴ Sewell (2017) offered a detailed discussion of the concept of FL, which moves ‘beyond the narrow sense of minimal pairs’ to the ‘dynamic, broad senses’ linked to wider ‘functionalist approaches to language and communication, and to an awareness of frequency effects and emergent structure’ (p.74). He suggested that on the one hand, FL places the LFC findings ‘on a firmer theoretical foundation by providing tentative explanations for some of the observed patterns and commonalities’, but on the other hand, considering functional FL also ‘tends to subvert the idea that lingua franca communication is substantially different from its implied other, non-lingua communication’(p.74).

⁵ Although task-based language teaching can be regarded as a variety of communicative language teaching, there has been a shift in the focus of Hong Kong’s ELT curricula from the communicative approach (CDC, 1999) to task-based learning and teaching approach (CDC, 2002/2007) over the years (see Chan, 2019b for a detailed discussion). Nevertheless, these curricula also recognise the relationship between the two approaches.

⁶ This is a benchmark examination introduced in 2001 to ensure that local English teachers’ language proficiency meets the required standard.

⁷ As there was a structural change in the secondary education system in Hong Kong in 2007 from a five-year (secondary 1–5) to a six-year (secondary 1–6) framework, HKDSE has replaced the former HKCEE as the public school-exit examination since 2012.

⁸ It should be acknowledged that the assessment process may also be affected by the examiners’ backgrounds and prior training, and by the marking procedures (Yan & Ginther, 2018). Nevertheless, this study focused on a chronological analysis of speaking papers, the examination reports and ELT curricula, all of which are publicly available. It is known that the examiners were both local (L2) and NS teachers in Hong Kong, but there is little information about the marking process in these documents.

⁹ Given the large number of candidates taking the examination each year, using multiple versions of the speaking papers avoids overlapping content in different periods of the same year.

¹⁰ Although the examination report published each year includes comments on different skills (e.g., writing, reading, speaking, listening), only the section about the speaking paper was analysed in this study.

¹¹ The coding process included frequent revisions to the list of emerging themes, categorisations and counts after each meeting. Initially, the two researchers analysed samples of examination reports in each period and discussed the emerging themes and categories. Nevertheless, as the format and content of the examination reports were often based on previous reports, our on-going analysis found a rather systematic pattern of the examiners’ foci (and wordings) over different periods. The subsequent examination reports were first analysed by one researcher and checked by another. There were few disagreements between the two researchers apart from some problems in feature categorisation (e.g., word pronunciation), which were resolved through discussion, negotiation and revisiting previous themes and categories.

¹² The category of ‘mispronounced word’ may encompass different pronunciation ‘problems’ such as word stress, vowels and consonants, but these were not always specified in the examination reports. As our analysis was based on the wordings in the reports (e.g., ‘Even common words like… were mispronounced’), these instances were not subcategorised into word stress, vowel or consonant problems.

Additional information

Funding

This work was supported by the Seed Funding Programme for Basic Research, The University of Hong Kong and Standing Committee on Language Education and Research (SCOLAR), Hong Kong SAR government [EDB(LE)/P&R/EL/175/13].