Second language pragmatics aptitude

Abstract

The present study aimed at investigating whether language pragmatics aptitude is a separate component of the construct of language aptitude distinct from the already conceived components. To do so, the Second Language Pragmatics Aptitude Test and the LLAMA test were administered to 200 native English-speaking adults, within the age range of 18 to 38, who entered the study through availability sampling. An exploratory factor analysis was run and the results showed loadings on two components indicating that pragmatics aptitude is distinct from the other components of language aptitude measured through the currently used language aptitude batteries. In order to confirm this finding, a subset of the data from 40 randomly-selected participants (out of the original 200) was subjected to paired-samples t-test. The results demonstrated a significant difference between test scores implying that the two tests neither overlap with each other nor yield redundant results. This proves the SLPAT and the LLAMA measure two distinct constructs suggesting that there is an aptitude for second language pragmatics learning, which could be measured to predict future success in learning the pragmatics of a second language.

Keywords:

• language aptitude
• language aptitude batteries
• language pragmatics aptitude
• LLAMA
• pragmatics aptitude test

1. Introduction

Numerous studies (Granena & Long, 2013; Li, 2015, 2016, 2017, 2018; Skehan, 2012; Wen et al., 2019) have taken serious issues with the current second language aptitude batteries such as the Modern Language Aptitude Test (MLAT), the Pimsleur Language Aptitude Battery (PLAB), the Defense Language Aptitude Battery (DLAB) and more recently the LLAMA and have called for updating both the construct and the tests (Li, 2019; Li & DeKeyser, 2021; Turker & Reiterer, 2021; Wen & Skehan, 2021).

One criticism is that language aptitude tests are outdated and lag behind the state of the art in the field of SLA. In fact, such tests pertain to Audiolingual teaching and Behaviorist view of learning (Doughty, 2018; Ellis, 2015; Li, 2015, 2017, 2018; Sparks et al., 2005). Therefore, it is open to question if they can predict future language learning success in more communicative approaches of language teaching and learning (Li, 2015).

Further, a number of studies (Li, 2016, 2018; Robinson, 2005) assert that the currently used language aptitude batteries such as the MLAT suffer from problems of construct validity. Li (2018) highlights the fact that the concept of language learning aptitude has been empirically, not theoretically, validated. In other words, the tests are not developed based on a theory of second language learning. Additionally, according to Li (2015); also see, (Zhao & Li, 2021), although Carroll and Sapon (1958, 2002, as cited in Li, 2015) hypothesize three aptitude components, i.e., phonetic coding, language analytic ability and memory, the MLAT is composed of five subtests. This lack of correspondence between the number of language aptitude components and the number of subtests makes the meaningful interpretation of the related findings difficult.

Language aptitude tests have also been criticized on the grounds that they are not a complete representation of the construct of L2 aptitude and do not embody a comprehensive definition of it. Research has revealed that the predictive potential of the current aptitude batteries decreases when the instruction is more communicative in nature (Li, 2015, 2017; Safar & Kormos, 2008; Skehan, 2012; Winke, 2013). This means that the constellation of factors that contribute to successful language acquisition should be further expanded beyond the constructs represented by the current aptitude tests (Wen & Skehan, 2021; Winke, 2013).

However, the most serious issue taken with such tests is that they fall short of measuring aptitude for learning the communicative aspects of language such as pragmatics (Ellis, 2015; Li, 2017; Safar & Kormos, 2008; Sedaghatgoftar et al., 2019; Skehan, 2012; Taguchi & Roever, 2017; Wen, 2017). Li (2017) argues that the currently used tests measure only the abilities that contribute to learning the formal aspects of language and consider language learning as learning discrete items. In other words, such tests do not account for communicative language learning and how the pragmatic features of a language are learned (Li, 2017; Skehan, 2012).

The literature often emphasizes the fact that these tests fail in predicting high-level L2 use involving pragmatic abilities (Li, 2017; Lundell & Sandgren, 2013; Taguchi & Roever, 2017). Robinson (2005) believes that currently available aptitude tests are insufficient to capture the range of abilities drawn on by communicative language learning. Li (2017) also holds that the abilities measured through the present language aptitude batteries play a role only in reaching preliminary but not advanced levels of L2. Besides, several studies have indicated that the available aptitude tests are not good indicators of oral proficiency and speaking ability although they are good predictors of reading and writing abilities (Brecht, Davidson & Ginsberg, 1995; Dörnyei, 2005; Ehrman, 1998; Granena, 2013b).

Embedded in his model of aptitude complexes, Robinson (2005) implied the notion of pragmatics aptitude and asked the question: “What are the aptitude components that predict acquisition of pragmatic abilities?” (Robinson, 2005, p. 50). However, research which has seriously investigated the existence of such a thing as pragmatics aptitude is scarce to date.

Sedaghatgoftar et al. (2019) seems to be the only study which has ventured to develop a second language pragmatics aptitude test. The results indicated that the instrument developed in their study has met the criteria for reliability and validity.

The present study aimed at investigating whether Second Language Pragmatics Aptitude Test (the SLPAT developed by Sedaghatgoftar et al., 2019) yields results which are distinct from the results collected through current aptitude batteries. In other words, this study endeavors to shed light on whether the results from the pragmatics aptitude test are redundant to the results from the currently used aptitude tests or not, hence ensuring whether language pragmatics aptitude is a component of the construct of second language learning aptitude distinct from the already conceptualized components of this construct.

In what follows, first the present body of literature on the subject is briefly outlined and then the methodology and the results of the present study are discussed and final conclusions are presented.

2. Literature review

Although several attempts have been made, in recent years, to develop new, theoretically updated aptitude test batteries (e.g., the Cognitive Ability for Novelty in Acquisition of Language-Foreign or CANAL-F by Grigorenko et al., 2000; LLAMA by Meara, 2005; High Level Language Aptitude Battery, shortly called Hi-LAB, by Linck et al., 2013), they have not reconceptualized aptitude in any significant manner (Skehan, 2012). Besides, aptitude for learning pragmatics does not seem to have been dealt with in any significant way in these batteries (Sedaghatgoftar et al., 2019).

A momentous contribution to aptitude conceptualization and testing in the past decade has been the development of the Hi-LAB (Linck et al., 2013; Wen et al., 2017). The Hi-LAB aims at predicting language learning at the most advanced levels of SLA. Nine constructs are identified as tentative components of language aptitude. The battery consists of a Language History Questionnaire (LHQ) as well as eleven computer-driven cognitive tasks. The test is administered in a single three-hour session. Self-report demographic information on whether the participants’ have any experiences with languages other than English is gathered through the LHQ in either paper-and-pencil or electronic format (Linck et al., 2013, p. 539). Although Hi-LAB has yielded first-hand substantiation of the plausible cognitive predictors of successfully learning a second language to advanced proficiency levels, i.e., conceivable candidate components of the construct of high-level language aptitude, it does not engage in measuring pragmatics aptitude in any significant way. However, access to this test is often withheld; hence, it cannot be really explored and criticized (Ameringer et al., 2018).

One of the latest attempts to produce a foreign language aptitude test resulted in the production of the CANAL-F battery (Grigorenko et al., 2000). The test is based on a simulation in which the participants are expected to learn elements of a new artificial language. This new language which the researchers called Ursulu characterizes aspects of different existing languages but does not resemble any particular language or linguistic group. A number of linguists were invited to ensure the consistent internal structure and stable properties of Ursulu. The participants are gradually exposed to Ursulu during the simulation, so that they have no initial knowledge of the language; nevertheless, by the end of the test, they will have mastered enough lexical, morphological, semantic and syntactic knowledge to cope with a story in Ursulu (Grigorenko et al., 2000).

The CANAL-F consists of nine sections, five of which deal with immediate recall and four of which are identical to those five sections except that they are presented later and involve delayed recall (Grigorenko et al., 2000). However, none of the sections of the test seem to measure pragmatics aptitude.

A progressively popular alternative to the MLAT instrument is Meara’s (2005) quite newly developed LLAMA series of tests (Singleton, 2017). The battery is composed of a suite of exploratory tests designed to gauge foreign language learning aptitude. Although the design of the tests has considerably differed from the first version, they are still loosely based on pivotal work by Carroll, the MLAT. Nevertheless, unlike the MLAT, the LLAMA is language-independent (Meara, 2005). Further, the LLAMA tests use pictorial and verbal stimuli adapted from a native British-Columbian language and a Central-American language (Granena, 2013a). The battery consists of four tests (Meara, 2005, p. 5): LLAMA B, LLAMA D, LLAMA E and LLAMA F.

However, all of these tests suffer from serious (earlier-mentioned) issues such as falling short of measuring the ability for learning the pragmatics of an L2 or second language pragmatics aptitude (Sedaghatgoftar et al., 2019). In what follows, it is illustrated that L2 pragmatics aptitude is a distinct component of the construct of L2 learning aptitude and that the SLPAT (Sedaghatgoftar et al., 2019) is a good measure of this component.

3. Materials and methods

The present study, which consisted of two phases, was a quantitative piece of research mainly relying on inferential statistics.

4. Participants

Two hundred (116 female and 84 male) native English-speaking adults took the second language pragmatics aptitude test along with the LLAMA language aptitude test. The age of the participants ranged from 18 to 38 (M = 27.7, SD = 6.29). As the instruments were computerized, online tests, the participants joined the study online through availability sampling. Social media such as Facebook was used in order to invite individuals to join the study. To make sure that the participants were native speakers of English, following large-scale studies such as Linck et al. (2013), the demographic part of the SLPAT enquired about their native language and their answers were trusted in.

In the second phase of the study, a subset of the data from 40 (22 female and 18 male) randomly-selected participants out of the original 200, within the age range of 18 to 38 (M = 27.4, SD = 6.9), was subjected to paired-samples t-test.

It is worth noting that, following other language aptitude test development studies (Carroll & Sapon, 1958, as cited in Grigorenko et al., 2000; Li, 2015; Linck et al., 2013), this study also recruited native speakers and this makes sense because aptitude tests are often administered before learners embark on learning a second language. In fact, a language aptitude test is run in order to predict the future course of progress of learners in learning an L2, and to measure their ability for acquiring an L2 better, faster or more effectively than others do.

5. Instrumentation

Two instruments were employed to gather data. The first one was the SLPAT developed and validated by Sedaghatgoftar et al. (2019) which was claimed to predict language learners’ ability to learn the pragmatics of a second language. The test is composed of three sections: memory for pragmatic rule learning which measures the ability to remember pragmatic rules from other languages that are culturally different from English, mind-reading from films which assesses recognition of complex emotions and mental states using social scenes from films, and mind-reading from voices which gauges the ability to recognize complex emotions and mental states from voices (one sample item from each subtest of the SLPAT is provided in the Appendix, for a review of the full test please see, Sedaghatgoftar et al., 2019). Despite its novelty, this new measure has demonstrated high internal consistency and a unitary structure. Further, this instrument has met the criteria for reliability and validity. Therefore, it could be used to tap into pragmatics aptitude (Sedaghatgoftar et al., 2019).

The other instrument used was the LLAMA test of language aptitude, which is a progressively popular alternative to the MLAT. It is a quite newly developed suite of tests designed by Meara (2005). The reason why this test was chosen for this study is that it does not suffer from the drawbacks of other language aptitude batteries such as commercial use, military-only access or paper-and-pencil format (Meara, 2005) and this might be the reason why it has been increasingly applied in a number of studies (Abrahamsson & Hyltenstam, 2008; Granena, 2013a; Rizvanovic, 2018; Yilmaz, 2013). A number of advantages associated with employing the LLAMA include being computer-driven, user-friendly and language-independent. The battery is composed of a set of exploratory tests devised loosely based on the MLAT, is available for free download and takes 25 minutes to administer (Granena, 2013a; Meara, 2005).

The LLAMA comprises four subtests: LLAMA B to measure rote, associative memory, LLAMA D to assess the ability to recognize patterns in spoken language, LLAMA E to gauge the ability to associate sounds with symbols, and LLAMA F to rate inductive language learning ability (Kourtali & Revesz, 2019).

The Statistical Package for the Social Sciences (SPSS) version 22, for Windows, was employed to analyze the data. Exploratory factor analysis was conducted to ensure that the pragmatics aptitude test and the LLAMA language aptitude test assess two different constructs. Further, a paired-samples t-test was run to demonstrate that the results from the two tests do not overlap and are not redundant.

6. Results and discussion

This study aimed at investigating whether the pragmatics aptitude test adds to the currently used language aptitude tests and does not yield redundant results, hence making sure that pragmatics aptitude is a separate language aptitude component distinct from the currently conceptualized constituents language aptitude measured via the currently available tests. Consequently, a set of exploratory factor analysis was administered. Further, a paired-samples t- test was administered. Table 1 shows the Correlation Matrix which indicates that the variables are well correlated and internally consistent.

Table 1. Correlation matrix of the pragmatics aptitude test and the LLAMA

		LLAMA B	LLAMA D	LLAMA E	LLAMA F	Pragmatic rule learning	Mindreading videos	Mindreading voices
Correlation	LLAMA B	1.000	.910	.856	.873	.186	.135	.166
	LLAMA D	.910	1.000	.942	.952	.192	.163	.188
	LLAMA E	.856	.942	1.000	.943	.139	.111	.145
	LLAMA F	.873	.952	.943	1.000	.177	.156	.176
	Pragmatic rule learning	.186	.192	.139	.177	1.000	.783	.937
	Mindreading videos	.135	.163	.111	.156	.783	1.000	.814
	Mindreading voices	.166	.188	.145	.176	.937	.814	1.000

Table 2 displays the results of Kaiser-Meyer-Olkin (KMO) and Bartlett’s test of Sphericity. These two statistical measures help assess the factorability of the data. The KMO determines the adequacy of the data for factor analysis. The Bartlett’s test of Sphericity measures probability that the correlation matrix contains significant correlations among at least some variables in a dataset, which is a prerequisite for factor analysis.

Table 2. KMO and bartlett’s test of sphericity

Kaiser-Meyer-Olkin Measure of Sampling Adequacy		.814
Bartlett’s Test of Sphericity	Approx. Chi-Square	192 3.484
	df	21
	Sig.	.000

KMO test result was .81, well above .6, indicating that the sample was quite suitable for factor analysis. Further, Bartlett’s test result was significant (.000 < .005) suggesting that the variables did not have equal variances and were appropriate for structure detection and factor extraction. The Total Variance Explained can be seen in Table 3.

Table 3. Total variance explained of the pragmatics aptitude test and the LLAMA

Component	Initial Eigenvalues			Extraction Sums of Squared Loadings
	Total	% of Variance	Cumulative %	Total	% of Variance	Cumulative %
1	3.982	56.885	56.885	3.982	56.885	56.885
2	2.450	34.997	91.882	2.450	34.997	91.882
3	.249	3.554	95.436
4	.163	2.322	97.758
5	.062	.885	98.643
6	.053	.764	99.407
7	.042	.593	100.000

As can be seen, only two factors recorded eigenvalues above 1. The Component Matrix (Table 4) and the Scree Plot (Figure 1) confirm this result indicating that the variables load on two components one of which could be considered as the LLAMA test and the other as the pragmatics aptitude test. The results revealed that the pragmatics aptitude test is distinct from the components measured through the LLAMA test.

Figure 1. Scree plot for the pragmatics aptitude test and the LLAMA.

Table 4. Component matrix of the pragmatics aptitude test and the LLAMA

	Component
	1	2
LLAMA D	.937
LLAMA F	.925	−.307
LLAMA E	.906	−.343
LLAMA B	.892
Mindreading voices	.473	.846
Pragmatic rule learning	.474	.834
Mindreading videos	.428	.805

Once the number of factors was determined, they were rotated in order to interpret them and to present the pattern of loadings. To do so, Varimax rotation technique was employed. Table 5 presents the Rotated Component Matrix which clearly indicates that the variables have loadings on two components.

Table 5. Rotated component matrix of the pragmatics aptitude test and the LLAMA

	Component
	1	2
LLAMA D	.978
LLAMA F	.970
LLAMA E	.967
LLAMA B	.936
Mindreading voices		.966
Pragmatic rule learning		.954
Mindreading videos		.909

As shown in Tables 6 and 7, there was a significant difference [t(39) = 38.60, p = 0.000 < 0.005] between the pragmatics aptitude test scores (M = 23.17, SD = 10.43) and the LLAMA test scores (M = 85.12, SD = 1.69) demonstrating that the two tests generate different results. In other words, the participants performed differently on the two tests. The eta squared statistic (.97) indicated a large effect size.

Table 6. Paired samples statistics

		Mean	N	Std. Deviation	Std. Error Mean
Pair 1	LLAMA	85.1250	40	1.69747	.26839
	Pragmatics aptitude test	23.1750	40	10.43880	1.65052

Table 7. Paired samples test

		Paired Differences					t	df	Sig. (2-tailed)
		Mean	Std. Deviation	Std. Error Mean	95% Confidence Interval of the Difference
					Lower	Upper
Pair 1	LLAMA—pragmatics aptitude test	61.95000	10.15003	1.60486	58.70386	65.19614	38.601	39	.000

The results showed that the second language pragmatics aptitude test yields results different from the results gained through the present language aptitude batteries, suggesting that pragmatics aptitude does not overlap with the already known components of language learning aptitude. In other words, the results from the pragmatics aptitude test do significantly differ from the results gained through the current language aptitude batteries such as the LLAMA. Thus, it can be argued that second language pragmatics aptitude is a distinct component of the construct of second language learning aptitude which neither overlaps with the other components of language learning aptitude (e.g., rote, associative memory, the ability to recognize patterns in spoken language, the ability to associate sounds with symbols, and inductive language learning ability), nor yields redundant results. The findings confirm Bardovi-Harlig’s (2012, also see, Eslami & Eslami-Rasekh, 2008) assertion that grammatical proficiency does not guarantee L2 pragmatic proficiency implying that aptitude for learning pragmatics differs from aptitude for learning grammar or other aspects of an L2. Further, Taguchi and Roever (2017) assert that individual differences in cognitive abilities have effects on second language pragmatics learning implying that there is an aptitude for second language pragmatics learning, which could be measured to predict future success in learning the pragmatics of a second language, which is substantiated in this study.

Having indicated that pragmatics aptitude is a conceivable component of the construct of language learning aptitude, the findings further reinforce the componential view of the nature of language aptitude as prevailing in the literature (e.g., Dörnyei, 2010; Doughty, 2018; Granena, 2013b; Hu et al., 2013; Jilka, 2009; Suzuki & DeKeyser, 2017; Wen et al., 2017).

Moreover, the findings reiterate the literature (Ellis, 2015; Li, 2018; Safar & Kormos, 2008; Skehan, 2012; Winke, 2013; Yamashita, 2008) suggesting that the constructs underlying the currently used language aptitude batteries do not represent a complete definition of L2 aptitude. Li (2017) believes that currently available aptitude tests are insufficient to capture the range of abilities drawn on by communicative language learning. Similarly, Winke (2013) contends that the cognitive constructs underlying L2 aptitude need to be expanded. Li (2018) adds that there are many abilities involving speaking and interacting in an L2, such as pragmatics ability, almost none of which could be measured by the present language aptitude tests. In fact, this study confirmed the insufficiency of the present language aptitude batteries through indicating that pragmatics aptitude could be regarded as a new component of the construct of second language learning aptitude.

In other words, the findings of the present study corroborate a number of studies (Li, 2015; Sternberg, 2002; Wen et al., 2017) claiming that the currently used language learning aptitude batteries such as the MLAT and the LLAMA are not comprehensive measures of the componential construct of language learning aptitude and that there is a need to supplement traditional measures of language learning aptitude and to include a broader range of component processes and abilities in the structure of the construct of language aptitude, such as pragmatics (Robinson, 2005).

The existence of a distinct aptitude for learning the pragmatics of a second language presupposes the fact that a number of language learners can acquire L2 pragmatics better, faster or more effectively than others do. This finding is consistent with Bardovi-Harlig (2012) and Ang et al. (2007) holding that some individuals are more effective than others in culturally diverse situations.

7. Conclusion

Numerous empirical studies have proved the predictive power of language aptitude on L2 learning (Yang & Cao, 2020). To date, a number of tests have been developed to measure the construct of language learning aptitude (Sedaghatgoftar et al., 2019). Although none of the currently used tests measures pragmatics aptitude, Sedaghatgoftar et al. (2019) seems to be the only study which has ventured to develop a language pragmatics aptitude test meeting the criteria for reliability and validity.

The present study aimed at investigating whether language pragmatics aptitude is a component different from the already conceived components of language aptitude (e.g., rote, associative memory, the ability to recognize patterns in spoken language, the ability to associate sounds with symbols, and inductive language learning ability), demonstrating no overlap or yielding no redundant results.

Thus, the SLPAT (developed by Sedaghatgoftar et al., 2019) and the LLAMA (developed by Meara, 2005) were administered to the subjects and the results were subject to exploratory factor analysis and paired-samples t-test. The results indicated that the two tests measure two distinct constructs and yield different results. Consequently, it was concluded that pragmatics aptitude is a new component of the construct of language aptitude which can’t be measured through the currently used aptitude tests such as the LLAMA.

The findings of this study are advantageous to language learners in demonstrating how important the identification of their aptitude profiles is. The students are advised to pinpoint the best ways, techniques and tasks which match their aptitude profiles in order to learn the language more successfully. Knowing about one’s aptitude profile could be profitable in that the learner will be mindful of his/her strengths and weaknesses in learning. Therefore, future learning may be augmented if the learners perpetuate their strengths and remedy their weaknesses. Besides, it will enhance the learners’ self-esteem because they will feel more confident and prepared to engage in the learning challenge. Also, it gives students the self-assurance needed to achieve their goals. Teachers should be aware that the componential view of aptitude implies that people may vary in separate components, for instance, high phonemic coding ability does not imply high memory ability. So they should combine specific method types with specific aptitude profiles in order to create optimal learning conditions. Using aptitude information, teachers can adapt instruction to be more appropriate for individual learners. Teachers should assist learners in discovering their own aptitude profiles and supply them practical feedback about the merits and demerits of various aptitude profiles. Further, teachers should take into consideration the learners’ aptitude strengths and weaknesses and encourage developing their aptitude profiles, while, at the same time, bringing about opportunities for students to try a variety of learning methods.

What would A most probably say?

1. I do apologize. I need a letter of recommendation. Could you please write me one?

2. I need a letter of recommendation to apply for a part-time job. Could you give me one?

3. I really appreciate it if you could write me a letter of recommendation for a part-time job.

Sample item from Mind-reading from Films subtest of the SLPAT

1. At the end of the scene, how is the old man feeling?

a. Pleased b. smug c. wanting d. kind

Sample item from Mind-reading from Voices subtest of the SLPAT

The transcript of the Persian recording: /Oonja chikar mikardin? Chera rafte boodin?/

The English equivalent (not included in the test): What were you doing there? Why did you go?

8. How is she feeling?

a. suspicious b. terrified c. awkward d. perplexed

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The authors received no direct funding for this research

Notes on contributors

Nasrin Sedaghatgoftar

Nasrin Sedaghatgoftar is a PhD in applied linguistics at Yasouj University, Iran. Her research interests include psycholinguistics, individual differences in second language learning, and language learning aptitude.

Susanne Maria Reiterer

Susanne M Reiterer is a professor for cognitive neuroscience of second language acquisition with a special focus on language aptitude and individual differences in the psychology of language learning, University of Vienna, Austria.

Appendix

Sample item from Pragmatics Rule learning subtest of the SLPAT

1. A is a 26-year-old Japanese male. He’s a graduate student. He’s going to apply for a part-time job to gain some experience in his field. He intends to ask his professor to write a letter of recommendation for him.