Validating motivated strategies for learning questionnaire and invariance test across gender and caste groups in India

Abstract

Non-cognitive competencies play critical role in education. Interventions targeted at these competencies require access to valid measures. Within Indian context, no such validated measure exists at present, and teachers depend on their subjective evaluation of students’ non-cognitive competencies represented via grade or comment on report card. This study examined convergent validity and measurement invariance of seven sub-scales of the Motivated Strategies for Learning Questionnaire (MSLQ), a widely used tool but not yet validated within Indian context. MSLQ, a measure of several non-cognitive competencies, was validated across four castes and two gender groups (N = 6423 elementary school students), the two categories highly relevant in the Indian context. For establishing convergent validity, multi-group confirmatory factor analysis was conducted where all the subfactors were allowed to correlate with each other. For measurement invariance, configural, metric, and scalar invariance were assessed by sequentially imposing more restrictive conditions. We found evidence for strong configural, metric, and scalar invariance across the gender and caste groups, and across all the seven sub-scales of MSLQ. Results also indicated good model fit for all caste and gender groups, thus establishing the convergent validity of all the sub-scales. The implications of the findings are discussed.

Keywords:

• MSLQ
• non-cognitive competencies
• convergent validity
• measurement invariance

PUBLIC INTEREST STATEMENT

Non-cognitive competencies like metacognition, self-efficacy, goal orientation etc. play a crucial role in student learning outcomes but are rarely a focus of intervention in schools within developing nation context. In India, first such recognition was visible in National Curriculum Framework-2005. Now, teachers subjectively comment on students’ life-skills, attitudes, and values in their grade sheets. Yet, in absence of objective measures that systematically analyze and evaluate these competencies, interventions to make changes remain absent or ineffective. This study used Motivated Strategies for Learning Questionnaire (MSLQ), a widely used tool for measuring a set of non-cognitive competencies and validated it for use within Indian context with special focus on caste (a social grouping of family) and gender, two identities that play crucial role in determining life chances (including educational attainment). Our results make MSLQ accessible as a valid tool to Indian teachers and education administrators for use with Indian school student population.

1. Introduction

The role played by non-cognitive competencies in academic and life outcomes of students has been well established (e.g., Gabrieli et al., 2015; Macklem, 2015; Moffitt et al., 2011; Rosen et al., 2010). Working on non-cognitive competencies that are amenable to development within school environments could improve learning outcomes, especially among children belonging to socio-economically disadvantaged groups. Inspite of the realization of the importance of non-cognitive competencies at the policy level (including in the National Curriculum Framework-2005) (e.g., see NCERT, 2005), there have been no sustained interventions focused on such skills in developing country contexts (Krishnan & Krutikova, 2013). A few non-cognitive competency measures have appeared in student report cards (as life skills, attitudes, and values) in India as part of continuous and comprehensive evaluation (NCERT, 2005) but these are scored subjectively by teachers. Any efforts focused on non-cognitive skill development in the future will benefit from valid measures of non-cognitive competencies.

This paper examines a group of non-cognitive competency measures that are part of the Motivated Strategies for Learning Questionnaire (MSLQ, a set of 15 scales) developed by Pintrich et al. (1991). The scope of this paper is concerned with their application within the Indian context. Although MSLQ’s wide use and validation in diverse contexts and geographies make it an attractive candidate for use within Indian schools, one needs to re-establish its validity and reliability in any new cultural context (Chen, 2008). This is especially crucial because measures may have varying meaning-making across the social and cultural groups relevant within the specific context (caste¹ and gender² in our case). This paper examined the validity and reliability of seven (out of fifteen) sub-scales of MSLQ and tested measurement invariance across the four caste and two gender groups.

1.1. Using MSLQ for diverse groups

In developing nations such as India, poor learning levels among school students, especially in the public system, have become matters of public debate (ASER, 2018). The free public schools play an important role in providing educational access to children belonging to socio-economically disadvantaged groups (officially classified as Scheduled Castes (SC), Scheduled Tribes (ST), and Other Backward Classes (OBC) for purposes of affirmative action, that is, reservation in higher education institutions and government jobs). Thus, improving academic achievement levels in these public schools would also have positive social effects.

Several studies have highlighted the critical role of non-cognitive (NC) competencies in predicting short-term academic outcomes (grades) and long-term life outcomes (chances of graduation, employability, income, health) (for a detailed list of such studies, see Gabrieli et al., 2015; Macklem, 2015; Moffitt et al., 2011; Rosen et al., 2010). Non-cognitive competencies include “a range of personality and motivational habits and attitudes that […] may complement direct efforts to improve academic learning” (Rosen et al., 2010, p. 1). In this paper, we are concerned with a subset of non-cognitive competencies (seven out of fifteen³) measured within the academic context using the Motivated Strategies for Learning Questionnaire (MSLQ) (Pintrich et al., 1991). These included two motivation subscales: Intrinsic goal orientation (IGO); and Self-efficacy for learning and performance (SE); and five metacognitive strategies subscales: Rehearsal (RHR); Elaboration (ELB); Organization (ORG); Critical thinking (CTHINK); and Metacognitive strategies for self-regulation (MCSR).

Since its development, the questionnaire has been adapted for use in several contexts (school or university students, pre-service teachers, online classrooms), across several countries (including several developed and developing nations), and has been translated into multiple languages (Credé & Phillips, 2011; Duncan & McKeachie, 2005). MSLQ has been validated in diverse contexts and geographies, for example, Artino (2005) (Spanish and Chinese versions; low reliability of three subscales with only 4-items in each was highlighted); Credé and Phillips (2011) (meta-analysis of 59 studies that broadly supported the factor-structure of MSLQ); Erturan Ilker et al. (2014) (Turkish adaptation); Rao et al. (2000) (Chinese adaptation); Taylor (2012) (review of 70 studies that found the mean reliability scores to be very close to those reported in the MSLQ manual). MSLQ has generally been found to be a reliable measure of constructs but certain studies have also identified some challenges. For example, Credé and Phillips (2011) found some non-trivial construct overlaps and redundancy (e.g., between elaboration and organization). Erturan Ilker et al.’s (2014) established the validity of Turkish adaptation of MSLQ, but Rao et al. (2000) found inconsistencies in the factor structure of their Chinese adaptation of MSLQ. The motivational factors showed similar structure, but self-regulation had only one factor in the Chinese version compared to two factors in the original version.

As evident from the above discussion, the different sub-scales of MSLQ have been found to be valid and reliable across diverse contexts. Yet, its use in an entirely new context would still call for re-establishment of the reliability and validity. Chen (2008) argues that comparisons across cultural or social groups could be inappropriate if there is measurement variance across these groups. Such groups may show differences in factor loadings (metric invariance) and/or intercept (scalar invariance) due to their differing understanding of the construct, differences in the pattern of their response to extreme items, or differences in social desirability (Sass, 2011). MI determines if differences in observed construct means across groups are attributed only to the construct level variances (Meredith, 1993). Hence, measurement invariance (MI) needs to be established for different socially or culturally significant groups present in the sample for valid group comparisons.

Individual subscales of MSLQ have even been tested for measurement invariance (MI) in a few studies. Kraft and van der (2014) tested MI for self-efficacy and control of learning beliefs sub-scales of MSLQ for students from community colleges and research institutions and found evidence of strong invariance. Tock and Moxley (2018) used a revised nine-item metacognitive self-regulation sub-scale (MSR-R) of MSLQ with a sample including 347 college students aged 17 to 27 to test for MI across gender groups. The study found evidence of full configural and metric invariance, but only partial scalar invariance could be established. This means that the factor loadings were similar but item-level intercept equivalence could not be fully established across the two genders. Consequently, the evidence from this study makes accesible a readily available tool for school teachers and administrators to objectively assess non-cognitive competencies of their students, work to improve them, and track them over time.

2. The present study

Researchers using MSLQ assume that the respondents understand the items and interpret them in similar ways, and that their responses on the Likert scale are circumscribed by the same frame of reference. This assumption has not been tested in the Indian context where caste membership (see footnote 1 for the role played by caste within the Indian social context) and gender shape the life experiences of children. Several studies have identified caste membership and gender as significant in determining access to education and subsequent attainment levels. Chaudhury’s (2017) research points to the disparities in educational attainment among general caste and SC students, and among general caste and ST students. They attribute these differences to “endowments of personal, household, and community-related factors” (p. 86). Deolalikar’s (2010) analysis showed that SC, ST, and Muslim children’s reading, writing, and arithmetic capabilities were significantly lower compared to upper-caste children. Borooah (2012) also found SC, ST, and Muslim children to be at a disadvantage compared to other upper castes, and other religious minorities (Sikhs/Jains/Christians). Khalid (2008) and Rammohan and Vu (2018) also highlighted the role of gender in defining educational outcomes. Husain (2010) also found that gender played a big role in predicting probabilities of completing schooling, but there were significant regional differences. Joshi (2010) pointed to the relative disadvantage of tribal children (ST) in accessing schools and completing schooling, with gender being the worst affected stakeholders. Given the patriarchal nature of Indian society with a strong preference for male child, there remains a growing interest in studying student outcomes by gender groups. Saha’s (2013) study identified significant differences in resource allocation by families on education of boys and girls, with girls receiving much lesser share of the money. It is for these reasons we considered caste affiliation (SC, ST, OBC, and unreserved (includes members of other religions) and gender as the variables to examine measurement invariance in this study.

MSLQ has been used in India in a few studies (e.g., Bhattacharyya, 2007; Dangwal & Gope, 2011; Jain & Dowson, 2009) but none has attempted to establish MI across gender or caste groups. The purpose of this study was to test the measurement invariance of seven (out of fifteen) MSLQ sub-scales (see “Instruments Used” below) across four caste groups (SC, ST, OBC, which are eligible for affirmative action, and the residual category termed “General”) and two gender groups (male and female), and examine the convergent validity of these scales. More specifically, the current study examined the following research questions:

1. Is there evidence of configural invariance, metric invariance, and scalar invariance for the seven sub-scales of MSLQ across males and females?

2. Is there evidence of configural invariance, metric invariance, and scalar invariance for the seven sub-scales of MSLQ across the four caste groups (General, SC, ST, OBC)?

3. Is there evidence of the convergent validity of the seven MSLQ sub-scales for males and females?

4. Is there evidence of the convergent validity of the seven MSLQ sub-scales for the four caste groups (General, SC, ST, OBC)?

3. Method

3.1. Participants and procedure

The study was carried out in Gujarat, a province in western India. A total of 6423 students from grades five to eight of 346 government schools constituted the sample. In each school, the survey was administered to students from only one grade 48.5% of the sample was male (OBC-67%, ST-18%, SC-6%, General-9%) and 51.5% female (OBC-64%, ST-19%, SC-6%, General-11%).

MSLQ with 43-items spread across seven sub-scales was administered at the beginning and at the end of the academic year 2015–16 (referred here as pre and post data, respectively). A trained research team went to each classroom and conducted the pen and paper survey in the absence of the teacher.

3.2. Measures

Seven of the fifteen sub-scales from Motivated Strategies for Learning Questionnaire (MSLQ) (Pintrich et al., 1991) were used for the survey. The full MSLQ deals with motivation (6 scales, 31 items), cognitive and metacognitive strategies (5 scales, 31 items), and resource management strategies (4 scales, 19 items). Based on the extensive past experience of working with school children in the project area, one scale in the Value component of Motivation, Intrinsic Goal Orientation, and one scale in the Expectancy component, Self-efficacy, were considered as more relevant to the context and chosen for the study. Extrinsic goal orientation, task value, control of learning, and test anxiety (affective component), were not chosen. However, given the practical focus on improving learning in public schools, all five scales in the cognitive and metacognitive strategies component, were chosen. Resource management strategies were not the focus of the study and so were not selected.

The survey contained two motivation subscales: Intrinsic goal orientation (IGO) (4-items), and Self-efficacy for learning and performance (SE) (8-items), and five learning strategies subscales: Rehearsal (RHR) (4-items), Elaboration (ELB) (6-items), Organization (ORG) (4-items), Critical thinking (CTHNK) (5-items), and Metacognitive strategies for self-regulation (MCSR) (12-items) with items in the same sequential order as given in Pintrich et al.’s (1991) manual.⁴ All 43 were Likert scale items and students self-reported their response on a seven-point scale where 1 indicates not at all true of me and 7 indicates very true of me. Two of the items in MCSR (32 and 39) were reverse coded. The survey instrument was translated from English to the local language of students (Gujarati) and back-translated into English by another translator to ensure the accuracy of the content (both versions available on request).

3.3. Analytical procedure

The data analysis for the current study was performed using SPSS (version 22) and Mplus (version 7.31) software (Muthen & Muthen, 1998–2016). The analytic approach adopted in this study is similar to the one demonstrated in Shukla et al. (2019). For the convergent validity of the seven subscales, Multi-Group Confirmatory Factor Analysis (MGCFA) was conducted where all the subfactors were allowed to correlate with each other. For metric invariance (MeI), the configural invariance (CI) for the caste and gender groups was examined by allowing all the model parameters to vary freely across the groups as illustrated by Sass (2011). If CI was established, more restrictive conditions were imposed by keeping unstandardized factor loadings equal (across the four caste groups and two gender groups separately) to determine MeI. For scalar invariance (SI), along with unstandardized factor loadings, even the intercepts were forced to be equal across the groups being compared.

The degree of model fit in CI was based on the criteria suggested by Hu and Bentler (1999) who argue for using CFI (Comparative Fit Index), SRMR (Standardized Root Mean Square Residual), and RMSEA (Root Mean Square Error of Approximation) for the purpose. They suggested values of CFI, RMSEA, and SRMR, respectively, as >0.95, <0.06, and <0.08 for good model fit. Kline (2011) argues that chi square is sensitive to sample size, especially for very large samples (>5000). Byrne (2006) argues for use of CFI, RMSEA, and SRMR for complicated models and large samples (as in our study). Moreover, the likelihood ratio test which are often employed to evaluate competing nested models tend to be overly sensitive to large samples (Meade et al., 2008; Sass, 2011). Accordingly, methodological literature suggests the use of changes in the goodness of fit indices (ΔCFI, ΔRMSEA, and ΔSRMR values) compared to baseline model in such situations. Cheung and Rensvold (2002) suggest ΔCFI ≤ 0.01. Chen (2007) suggest ΔCFI ≤ 0.005 along with ΔRMSEA ≤ 0.01, or ΔSRMR≤0.025 for MeI and ΔCFI≤0.005 along with ΔRMSEA≤0.01, or ΔSRMR≤0.005 for SI for unequal groups (as in our study). Conventionally, Hu and Bentler (1999) cutoff criteria have been used extensively for determining model fit. However, there are no common standards available in the literature to guide the determination of MI (Sass, 2011) and this necessitates a subjective call from researchers while making interpretations. Consequently, in addition to changes in fit-indices, we also consider the magnitude of difference in parameter estimates for successive models while making our interpretations. All the analyses were done using maximum-likelihood estimation with robust standard errors (MLR estimator) based on Sass et al.’s (2014) recommendation. The standard errors were adjusted to accommodate the nested structure of data (students nested under classrooms) by using type = COMPLEX (sandwich estimator) command in Mplus.

We had one sample (N = 6434) and two observations: at beginning of a year (pre) and at the end of a year (post). The two phases of data collection for the same constructs allowed us to perform the analysis separately for pre and post data which enhanced the robustness of the results.⁵ During the analysis, the factor loadings of two reverse coded items from MCSR (32 and 39) were found to be very low (standardized loadings < 0.2). Our findings are consistent with previous studies on Metacognitive Strategies for Self-Regulation Scale. Tock & Moxley (2018) cite several studies that show original factor structure for MCSR subscale fail (including in their own study). Their own study found a two-factor structure with reverse-coded items (MCSR 32 and MCSR 39 in our study) loading on a separate factor which they labelled as “Performance Assessment”. Since the intent of our study was to validate an existing instrument (MSLQ), we did not drop these two items and cite results including them. From the full dataset, missing responses were minuscule (<0.001%). Hence, no imputation was carried out and missing values were replaced with −999 for analysis.

4. Results

4.1. Measurement invariance across genders (Research question 1)

Hu and Bentler (1999) suggest RMSEA, CFI, and SRMR cut-offs of <0.06, >0.95, and <0.08 for good model fit, and <0.08, >0.90, and <0.10 for moderate model fit. Six scales showed a good model fit for pre-data while MCSR showed moderate fit based on HU & Bentler’s criteria (Table 1; Configural model). For the post data, four scales (IGO, SE; ELB; ORG) showed a good model fit, while MCSR, RHR, and CTHNK showed moderate fit (post-data analysis available on request. See footnote-8). Thus, according to Hu and Bentler (1999) widely used criteria, we have evidence for configural invariance of all seven scales across genders (male and female) in both pre and post data.

Table 1. Model fit statistics for pre-data

Gender							Caste
Model	CFI	ΔCFI	RMSEA	ΔRMSEA	SRMR	ΔSRMR	CFI	ΔCFI	RMSEA	ΔRMSEA	SRMR	ΔSRMR
Intrinsic Goal Orientation (IGO)
Configural	0.999		0.012		0.005		0.998		0.024		0.009
Metric	0.996	0.003	0.025	0.013	0.020	0.015	0.997	0.001	0.020	0.004	0.019	0.010
Scalar	0.992	0.004	0.029	0.004	0.023	0.003	0.996	0.001	0.018	0.002	0.019	0.000
Self-efficacy for Learning and Performance (SE)
Configural	0.980		0.046		0.021		0.976		0.051		0.024
Metric	0.981	0.001	0.042	0.004	0.023	0.002	0.975	0.001	0.047	0.004	0.033	0.009
Scalar	0.980	0.001	0.040	0.002	0.023	0.000	0.974	0.001	0.043	0.004	0.036	0.003
Rehersal (RHR)
Configural	0.997		0.036		0.009		0.996		0.043		0.011
Metric	0.995	0.002	0.035	0.001	0.022	0.013	0.993	0.003	0.038	0.005	0.028	0.017
Scalar	0.994	0.001	0.033	0.002	0.026	0.004	0.992	0.001	0.033	0.005	0.031	0.003
Elaboration (ELB)
Configural	0.992		0.034		0.014		0.992		0.035		0.015
Metric	0.991	0.001	0.033	0.001	0.022	0.008	0.992	0.000	0.030	0.005	0.022	0.007
Scalar	0.991	0.000	0.030	0.003	0.024	0.002	0.991	0.001	0.027	0.003	0.024	0.002
Organization (ORG)
Configural	0.999		0.015		0.005		0.998		0.027		0.008
Metric	1.000	0.001	0.011	0.004	0.010	0.005	0.997	0.001	0.024	0.003	0.021	0.013
Scalar	0.998	0.002	0.017	0.006	0.013	0.003	0.996	0.001	0.022	0.002	0.023	0.002
Critical Thinking (CTHNK)
Configural	0.998		0.022		0.008		0.996		0.029		0.012
Metric	0.998	0.000	0.017	0.005	0.010	0.002	0.995	0.001	0.027	0.002	0.024	0.012
Scalar	0.998	0.000	0.014	0.003	0.011	0.001	0.992	0.003	0.029	0.002	0.030	0.006
Metacognitive Strategies for Regulation (MCSR)
Configural	0.939		0.058		0.038		0.933		0.061		0.040
Metric	0.939	0.000	0.056	0.002	0.039	0.001	0.933	0.000	0.057	0.004	0.044	0.004
Scalar	0.938	0.001	0.053	0.003	0.039	0.000	0.931	0.002	0.055	0.002	0.045	0.001

Note. CFI = Comparative Fit Index; RMSEA = Root Mean Square Error of Approximation; SRMR = Standardized Root Mean Square Residual. ^a indicates change in fit statistics greater than the recommended cutoff values.

For metric invariance, the difference in the CFI, RMSEA, and SRMR values for CI and MeI models were calculated. To gauge MeI, we compared the ΔCFI, ΔRMSEA, and ΔSRMR values to the criteria suggested by Chen (2007), i.e., either ΔCFI≤0.005 and ΔRMSEA≤0.01, or ΔCFI≤0.005, and ΔSRMR≤0.025, respectively. The fit statistics did not show significant change. In pre-data, for six scales, all three criteria were met. Only in case of IGO, ΔRMSEA value was 0.013, but even for this, ΔCFI (0.003) and ΔSRMR (0.015) values were within acceptable range, thus meeting Chen’s (2007) criteria for metric invariance (Table 1; Metric model).

For post data MeI, for RHR, ΔRMSEA was 0.015 (beyond cutoff value of ≤0.010). Even for this, ΔCFI (0.000) and ΔSRMR (0.004) values were within limits. For all other sub-scales, all three differences were below cutoff values. Hence, we had evidence for metric invariance across genders for all the seven MSLQ subscales.

Finally, scalar invariance was judged by calculating the ΔCFI, ΔRMSEA, and ΔSRMR for MeI and SI models. Imposing more restrictive conditions in SI model did not cause significant changes in the goodness of fit indices and all the three differences for all seven subscales were below the threshold values for both pre (Table 1; Scalar model) and post data. Consequently, we can say that there was evidence for configural, metric, and scalar invariance across the two genders for all the seven subscales of MSLQ.

4.2. Measurement Invariance across caste groups (Research question 2)

In pre-data, the CFI, RMSEA, and SRMR values for six scales were below the cutoff, thus suggesting a good model fit While the values for MCSR suggested moderate fit (Table 1; Configural model). For post-data, IGO, SE, ELB, ORG showed good model fit. For the remaining three, CFI value for MCSR was 0.933 (0.90< CFI<0.95) and RMSEA values for RHR (0.060), CTHNK (0.072), and MCSR (0.061) were above the suggested cutoff (<0.06) for good model fit but less than the value for moderate model fit (<0.08) (Hu & Bentler, 1999). Hence, there was evidence for CI across the four caste groups for all the seven sub-scales.

In pre-data, the ΔCFI, ΔRMSEA, and ΔSRMR values met the metric invariance criteria (Table 1; Metric model) for all seven scales. In post data, ΔRMSEA values were above the threshold (0.01) for four scales: IGO (0.011), RHR (0.015), ORG (0.011), and CTHNK (0.014). Even for these, ΔCFI and ΔSRMR values were below threshold. Chen (2007) suggests either ΔCFI≤0.005 and ΔRMSEA≤0.01 or ΔCFI≤0.005 and ΔSRMR≤0.025 for establishing metric invariance. Hence, we can argue for evidence of metric invariance across the four caste groups for all the seven subscales of MSLQ.

The comparison of CFI, RMSEA, and SRMR between MeI and SI models provided evidence of scalar invariance. All the three Δ values were below the desired cutoff levels for all the scales in pre (Table 1; Scalar model) and post data. The only exceptions were CTHNK in pre-data and RHR in post-data with ΔSRMR values of 0.006 in both cases against the desired cutoff of <0.005 as suggested by Chen (2007). Even for these, the other two differences in CFI and RMSEA values were within acceptable range. Hence, based on Chen’s (2007) suggestion of ΔCFI≤0.005 along with either ΔRMSEA≤0.01 or ΔSRMR≤0.005 for establishing scalar invariance, we have evidence for scalar invariance across the four caste groups for all the seven scales.

Overall, we can argue that our data establish configural, metric, and scalar invariance across the four caste groups, as well as the two genders, for all the seven scales of MSLQ. The unconstrained factor loadings and intercepts for pre data and post data were identical for all the items, thus providing additional evidence supporting our claim for full MI across caste groups and genders.

4.3. Convergent validity for gender (Research question 3)

The MGCFA model fit statistics for gender are presented in Table 2. The chi-square values were 3315.6 and 3695.9, and relative chi-square (chi-square/degrees of freedom) values were 1.89 and 2.11 for pre and post data (p < 0.001), respectively. When gender was selected as the grouping variable, CFI, TLI, RMSEA, and SRMR values were 0.964, 0.963, 0.017, and 0.028 for pre-data, and 0.954, 0.953, 0.019, and 0.030 for post-data. All model fit indices were well within the acceptable criteria indicating a good fit between the hypothesized model and data.

Table 2. Model-fit statistics for multi-group confirmatory factor analysis (pre-data)

	Chi-sq.	df	p-value	chi-sq./df	CFI	TLI	RMSEA	SRMR
Gender	3315.6	1750	<0.001	1.895	0.964	0.963	0.017	0.028
Caste	5761.2	3572	<0.001	1.613	0.958	0.958	0.020	0.034

4.4. Convergent validity for caste groups (Research question 4)

The MGCFA model fit statistics for caste are also presented in Table 2. Chi-square values were 5761.2 and 6377.6, and relative chi-square values were 1.61 and 1.79 for pre and post data (p < 0.001), respectively. Model fit statistics were well within the suggested cut-off values. CFI, TLI, RMSEA, and SRMR values were 0.958, 0.958, 0.020, and 0.034 for pre-data, and 0.947, 0.946, 0.022, and 0.037 for post-data. Overall, the hypothesized model fitted the data well.

The minimum and maximum factor loadings (completely standardized) for all the seven subscales ranged from 0.47 to 0.75 across all caste groups and genders (full data available on request). The only exceptions were C32 and C39 for which factor loadings were <0.30. This is in line with findings of other researchers cited in Tock & Moxley (2018). The differences in the factor loadings and unconstrained intercepts between males and females, as well as among the four caste groups were small. Thus, we can argue that similar patterns are observed across the two genders and the four caste groups for all the seven sub-scales.

4.5. Correlations

Correlations for the gender groups for pre-data are available in Table 3.⁶ All the inter-item correlations for the latent factors across gender and caste groups were positive. The minimum correlations for males and females were 0.63 and 0.61, respectively, while the maximum values were 0.96 and 0.96, respectively. Across caste groups, the minimum and maximum values of inter-item correlation for pre-data were respectively 0.54 and 0.96 for General, 0.63 and 0.95 for OBC, 0.61 and 0.94 for SC, and 0.586 and 0.97 for ST.

4.6. Reliability

For both genders, the scale reliability estimates were in the range of 0.74–0.84 for pre-data and 0.76–0.84 for post data (except IGO; 0.63–0.68). For caste groups, the Cronbach alpha estimates were in the range of 0.74–0.85 except for IGO (0.64–0.72). The composite reliability (ω) values for the whole sample were similar to the alpha values in all the cases. These estimates are presented in Table 4 as alpha and omega values.

Table 3. Inter-item correlation matrix (pre-data): Gender *

	IGO	SE	RHR	ELB	ORG	CTHNK	MCSR
IGO	1.00	0.88	0.70	0.67	0.63	0.67	0.64
SE	0.84	1.00	0.88	0.81	0.76	0.78	0.73
RHR	0.76	0.88	1.00	0.86	0.81	0.77	0.77
ELB	0.71	0.80	0.88	1.00	0.96	0.90	0.86
ORG	0.61	0.73	0.78	0.94	1.00	0.92	0.84
CTHNK	0.66	0.75	0.79	0.93	0.91	1.00	0.95
MCSR	0.63	0.71	0.78	0.87	0.84	0.96	1.00

Lower diagonal: Female; Upper diagonal: Male

* All values were significant at p < 0.001

Table 4. Completely standardized parameter estimates for multigroup confirmatory factor analysis (pre-data)

	Unconstrained factor loadings						Unconstrained intercepts
Scale and item	General	SC	ST	OBC	Male	Female	General	SC	ST	OBC	Male	Female
Intrinsic Goal Orientation (IGO)
Composite reliability (ω) (Full sample) = 0.66
Alpha	0.64	0.66	0.66	0.66	0.66	0.65
1	0.57	0.60	0.58	0.57	0.58	0.57	3.00	3.24	3.03	3.19	3.14	3.32
2	0.57	0.59	0.62	0.63	0.62	0.60	2.83	3.01	3.06	3.28	3.19	3.32
3	0.62	0.61	0.63	0.62	0.62	0.62	3.19	3.25	3.26	3.36	3.28	3.55
4	0.47	0.47	0.48	0.48	0.48	0.48	2.71	2.78	2.79	2.94	2.83	3.08
Self-efficacy for Learning and Performance (SE)
Composite reliability (ω) (Full sample) = 0.83
Alpha	0.85	0.83	0.83	0.83	0.84	0.83
5	0.59	0.57	0.58	0.57	0.59	0.56	3.34	3.45	3.41	3.49	3.43	3.58
6	0.57	0.56	0.54	0.54	0.55	0.54	3.13	3.30	3.10	3.24	3.12	3.40
7	0.69	0.67	0.66	0.67	0.66	0.68	3.47	3.61	3.42	3.63	3.42	3.88
8	0.64	0.60	0.59	0.60	0.61	0.59	3.08	3.08	2.92	3.10	3.02	3.23
9	0.67	0.67	0.64	0.64	0.64	0.65	3.38	3.61	3.39	3.52	3.35	3.76
10	0.67	0.63	0.65	0.66	0.67	0.64	3.41	3.44	3.45	3.64	3.52	3.73
11	0.61	0.60	0.59	0.61	0.61	0.59	2.89	2.99	2.91	3.08	2.98	3.18
12	0.66	0.63	0.66	0.66	0.66	0.65	3.22	3.31	3.38	3.48	3.35	3.61
Rehearsal (RHR)
Composite reliability (ω) (Full sample) = 0.75
Alpha	0.77	0.76	0.77	0.76	0.76	0.75
13	0.63	0.65	0.67	0.64	0.64	0.64	3.17	3.22	3.24	3.29	3.24	3.45
14	0.65	0.67	0.67	0.66	0.67	0.65	3.32	3.37	3.31	3.44	3.42	3.54
15	0.71	0.66	0.71	0.68	0.69	0.69	3.32	3.05	3.23	3.28	3.23	3.46
16	0.69	0.69	0.66	0.66	0.66	0.67	3.27	3.23	3.04	3.22	3.15	3.41
Elaboration (ELB)
Composite reliability (ω) (Full sample) = 0.79
Alpha	0.78	0.82	0.79	0.78	0.78	0.79
17	0.62	0.63	0.60	0.60	0.60	0.61	2.89	2.73	2.78	2.87	2.81	3.02
18	0.57	0.61	0.57	0.55	0.56	0.56	2.68	2.64	2.64	2.62	2.61	2.78
19	0.58	0.62	0.60	0.61	0.60	0.60	2.51	2.47	2.55	2.66	2.61	2.75
20	0.59	0.67	0.63	0.59	0.60	0.60	2.62	2.75	2.79	2.70	2.70	2.87
21	0.67	0.72	0.68	0.68	0.68	0.67	3.23	3.18	3.24	3.32	3.29	3.45
22	0.65	0.67	0.65	0.63	0.64	0.64	3.10	2.93	3.09	3.06	3.04	3.23
Organization (ORG)
Composite reliability (ω) (Full sample) = 0.75
Alpha	0.76	0.76	0.74	0.74	0.74	0.75
23	0.66	0.67	0.65	0.65	0.64	0.66	2.75	2.85	2.83	2.93	2.88	3.08
24	0.71	0.67	0.67	0.67	0.67	0.68	3.06	2.94	3.01	3.13	3.11	3.25
25	0.64	0.65	0.61	0.62	0.63	0.62	2.68	2.76	2.68	2.81	2.82	2.89
26	0.64	0.66	0.66	0.65	0.65	0.65	2.67	2.80	2.88	2.96	2.93	3.05
Critical Thinking (CTHNK)
Composite reliability (ω) (Full sample) = 0.78
Alpha	0.78	0.78	0.77	0.78	0.77	0.78
27	0.61	0.62	0.60	0.59	0.59	0.60	2.85	2.93	2.92	2.87	2.87	3.02
28	0.64	0.63	0.62	0.64	0.63	0.64	2.91	2.94	2.95	3.04	3.00	3.14
29	0.66	0.64	0.67	0.67	0.65	0.69	2.83	2.83	3.02	3.02	2.93	3.19
30	0.66	0.66	0.66	0.66	0.64	0.67	3.02	3.09	3.18	3.15	3.09	3.33
31	0.66	0.65	0.65	0.66	0.65	0.66	3.02	3.08	3.11	3.16	3.13	3.28
Metacognitive Strategies for Regulation (MCSR)
Composite reliability (ω) (Full sample) = 0.82
Alpha	0.83	0.83	0.82	0.83	0.82	0.83
32	0.29	0.30	0.29	0.29	0.29	0.29	1.95	1.93	1.99	1.94	1.96	1.97
33	0.57	0.60	0.59	0.58	0.58	0.59	2.80	2.73	2.91	2.87	2.83	2.96
34	0.63	0.68	0.66	0.64	0.64	0.65	3.36	3.39	3.52	3.42	3.40	3.55
35	0.61	0.65	0.60	0.63	0.61	0.63	3.17	3.15	3.15	3.27	3.19	3.34
36	0.58	0.63	0.59	0.60	0.60	0.61	2.84	2.90	2.92	2.93	2.89	3.00
37	0.63	0.64	0.65	0.64	0.63	0.66	2.97	2.80	3.02	3.01	2.96	3.12
38	0.51	0.54	0.53	0.53	0.54	0.51	2.60	2.58	2.70	2.70	2.73	2.69
39	0.23	0.24	0.23	0.23	0.24	0.23	1.62	1.59	1.67	1.66	1.69	1.65
40	0.62	0.64	0.65	0.62	0.61	0.64	2.98	2.82	3.08	2.95	2.91	3.10
41	0.64	0.67	0.65	0.65	0.63	0.66	3.20	3.13	3.27	3.26	3.18	3.40
42	0.59	0.61	0.62	0.60	0.61	0.60	3.14	3.03	3.30	3.19	3.19	3.27
43	0.58	0.61	0.58	0.59	0.58	0.59	3.20	3.11	3.21	3.22	3.17	3.31

5. Discussion and conclusion

In this study, we analyzed the convergent validity of the seven sub-scales of the widely used Motivated Strategies for Learning Questionnaire (two motivation subscales: Intrinsic goal orientation [IGO] and Self-efficacy for learning and performance [SE]; and five learning strategies subscales: Rehearsal [RHR] Elaboration [ELB], Organization [ORG], Critical thinking [CTHNK], and Metacognitive strategies for self-regulation [MCSR]) for application in the Indian context to measure non-cognitive competencies of students. We analyzed the convergent validity of these MSLQ sub-scales and tested measurement invariance across the socially significant caste groups (SC, ST, OBC, and General), and across the two gender groups (male and female) in the Indian context. All the scales were found to be reliable with Cronbach Alpha and composite reliability values above 0.64 for all scales across all caste and gender categories. The analysis established the convergent validity for all the seven subscales for both the groups (caste and gender). Most scales showed a good model fit while a few scales showed moderate fit (e.g., Rehearsal and Critical Thinking in post-data, and MCSR in both pre and post data). The associations between all the scales were significantly positive and had similar values across the compared groups.

The correlations between Organization and Elaboration, and Metacognitive strategies for self-regulation and Critical Thinking were very high, i.e., above 0.95 for all caste and gender groups. Even for other scales, most correlations were relatively high (above 0.70). In their analyses, Pintrich et al. (1993) did not find such high correlations. A meta-analytic review of MSLQ by Credé and Phillips (2011) also found results similar to Pintrich et al., but Lee et al.’s (2010) Chinese adaptation and Erturan Ilker et al.’s (2014) Turkish adaptation of MSLQ found high correlations among latent factors. For example, in Erturan Ilker et al. (2014), the correlations between motivational beliefs and self-efficacy (r = 0.79), between self-regulation and motivational beliefs (r = 0.81), and self-regulation and cognitive structure use (r = 0.79) were high. In case of motivational beliefs and intrinsic value, correlation was 0.99. Similarly, in Lee et al. (2010), the correlations between intrinsic value and self-efficacy (r = 0.76), strategy use and intrinsic value (r = 0.78), self-regulation and self-efficacy (r = 0.83), and self-regulation and intrinsic value (r = 0.79) were high. Our findings are consistent with these two studies that included local language adaptations of the original instrument. In our study, this raises a question about the way the students understand the items in MSLQ in general and these two scales in particular, though this understanding is similar across caste and gender groups. Perhaps, studies in developing countries tend to find high correlations between sub-scales of MSLQ and this needs to be investigated further.

Importantly, all the seven scales were found to be invariant across caste and gender groups in the Indian state of Gujarat even when translated into the local language, i.e., Gujarati. For all these scales, full Measurement Invariance was achieved, and the factor loadings and the intercepts made similar contributions to the factor means across the four caste groups and the two gender groups. A unit increase in an observed item score of a scale was thus linked to a similar increase in the scale score across all the tested groups. Hence, one could compare the scale scores to measure non-cognitive competencies across the groups, but item-level comparisons should be avoided.

In conclusion, the present study established the validity, reliability, and the measurement invariance of MSLQ across caste and gender groups in Indian context. This provides researchers and practitioners an accessible option to use a valid and reliable instrument for students in Indian schools. The continuous measurement and monitoring of progress on such non-cognitive measures, and interventions to improve these could prove useful for policymakers and practitioners (including teachers, teacher-educators, education administrators, etc.), especially in public schools. Such interventions could not only improve the academic performance of students but also their life outcomes. Within global context, the present study contributes to a large set of studies using MSLQ and provides evidence of its validity in a developing country context. Within Indian context, the study opens the possibility of using an easily accessible instrument to understand and improve the non-cognitive development of children.

Compliance with ethical standards

This research was partially supported by the Research and Publications Committee of Indian Institute of Management Ahmedabad. All permissions for the research were arranged by the provincial government. The authors declare that they have no conflict of interest.

Written informed consent in the local language was obtained from the teachers and the selected children, and the purpose of the research was explained to the school management committee members and parents. A summary of the preliminary findings (in the local language) was shared with the schools.

Additional information

Funding

This research was partially supported by the Research and Publications Committee of Indian Institute of Management Ahmedabad. [NA].

Notes on contributors

Deepak Maun

Deepak Maun is an Assistant Professor at International Institute for Higher Education Research and Capacity Building (IIHEd), O. P. Jindal Global University, Sonipat, India. His work revolves around teacher workplace learning, teacher professional development, and critical thinking.

Kathan Dushyant Shukla

Kathan Dushyant Shukla is an Assistant Professor at Ravi J. Matthai Centre for Educational Innovation (RJMCEI), Indian Institute of Management, Ahmedabad. He conducts research on applications of structural equation modeling, multilevel modeling, latent class analysis, and school climate & peer victimization.

Vijaya Sherry Chand

Vijaya Sherry Chand is a Professor at Ravi J. Matthai Centre for Educational Innovation (RJMCEI), Indian Institute of Management, Ahmedabad. His areas of specialization include innovations in education, management in education, non-profit sector, and social entrepreneurship. He has worked extensively with state education department teachers with a focus on mainstreaming their localized innovations.

Notes

1. In the western nations’ context, race was historically a predominant mode of discrimination. Within Indian context, it is caste. Ironically, in India, caste-based subdivisions creeped into Christianity and among Muslims, the religions that did not have such divisions outside boundaries of India, as people converted to these religions, or in Sikhism that denounced any such divisions.The Indian Hindu society is divided into varnas which are further divided into castes and sub-castes. Traditionally, Brahmins were literate, performed all rituals, and had a monopoly on knowledge and learning. They taught Kshatriyas, the varna that ruled as Kings and aristocrats, as Guru (teachers). The Vaishyas were the third varna. They were workers engaged in diverse professions (including animal rearing, food production, pottery, weaving, blacksmithing, etc.). The Shudras were the fourth varna and were mostly considered untouchables. They cleaned the filth of the society and performed cremation of the dead. They were not allowed to enter homes or public spaces accessed by the other three varnas and in worse cases (the lowest sub-castes among Shudras), they could not come out during the day time till other people were awake. Another category of people, the tribals (forest dwellers), were not part of Hindu society and prayed to animistic Gods. They have also been disadvantaged due to their remote habitations and exploitation by colonial rulers.One acquired the varna, caste, and sub-caste based on birth, i.e. from their mother and father, and inter-caste mobility was not possible. Caste is closely linked to social class in Indian context. Consequently, Indian constitution made provisions to make individuals belonging to Scheduled Castes (SC), Scheduled Tribes (ST), and Other Backward Castes (OBC) eligible for affirmative action, while those not included in these three categories are usually referred to as General castes. For details about the critical role played by caste system in India, see Ambedkar (1936/1936), and Shah et al. (2006).

2. India is a highly patriarchal society where women have been discriminated against in all sections of society. The gender, along with caste, plays a significant role in defining life chances of individuals.

3. The original MSLQ was designed to freely allow administration of one or more sub-scales independently, or in combination. The subset of seven competencies was chosen for practical reasons, i.e. to limit the length of questionnaire so that maximum primary school students are able to complete it. Further, competencies that were relatively more relevant and important within Indian context, as well as those that were amenable to change within schools were chosen. See “measures” section for more details.

4. For quick reference, the abbreviated names used in the study are provided here:IGO: Intrinsic goal orientation (4-items)SE: Self-efficacy for learning and performance (8-items)RHR: Rehearsal (4-items)ELB: Elaboration (6-items)ORG: Organization (4-items)CTHNK: Critical thinking (5-items)MCSR: Metacognitive strategies for self-regulation (12-items)

5. Due to space constraints, only pre-data tables are presented. Results for post-data are available on request.

6. Only correlations for gender groups for the pre-data are presented in the table. Complete post-data (gender and caste) and pre data (all four caste groups) correlations have not been presented for brevity and only their minimum and maximum values are provided in the text. These values were not very different from correlations from gender groups (pre-data).