Understanding student success in higher education in Azerbaijan: the role of student engagement

ABSTRACT

Research on student engagement in higher education (HE) has addressed the conceptualisation, measurement and criticism of the phenomenon over the last two decades, predominantly in Western countries. The conceptualisation of student engagement has received little attention in countries of a lengthy association with the Soviet realm. This study investigated student engagement in Azerbaijani universities using the National Survey of Student Engagement (NSSE) tool, under licence granted by Indiana University in 2018, to gather data. The survey was conducted among more than 430 students at eight Azerbaijani universities to explore what universities do to engage their students in learning. After a preliminary examination of the data for reliability, only the data from urban universities comprising 266 students were considered suitable for further analysis. The data were analysed to look at descriptive statistics and to identify linkages between student activities and student outcomes in Azerbaijan through bivariate correlation and regression analysis. The study revealed that, according to the NSSE categories, a supportive student environment and the quality of interactions, combined under the campus environment theme, and student–faculty interactions were essential elements for student learning, success, satisfaction and engagement in academically challenging practices.

KEYWORDS:

• Student engagement
• student learning experiences
• perceived gains
• National Survey of Student Engagement (NSSE)
• stepwise regression analysis

Introduction

Student engagement is regarded as a predictor of student success, retention and personal and professional development (Kuh 2009; Pike, Smart, and Ethington 2012). Universities worldwide are interested in building conditions that engage students in learning experiences in and out of class. Research has shown that student engagement positively impacts grades, satisfaction, perceived learning outcomes, critical thinking and students’ professional and personal lives after graduation (Laird et al. 2014; Rocconi, Liu, and Pike 2020). However, the impact of student engagement on student outcomes is still a subject of scholarly debate, and the necessary degree of institutional input is underestimated (Baron and Corbin 2012; Brint and Cantwell 2014).

This research covers higher education (HE) in Azerbaijan, a post-Soviet republic. While student engagement has become a much-studied topic in HE elsewhere due to its high correlation with learning and personal development, it has not received attention at the policy or institutional level in Azerbaijan and has not been extensively studied there (Hasanov et al. 2021). Although interactions between students and faculty and active and collaborative learning – driving forces of student engagement – are emphasised as alternative measures of the quality provided within HE (Kuh 2003), in most Azerbaijani universities collaboration between faculty and students is limited to classroom teaching. Learning more about student engagement is thus likely to provide higher education institutions (HEIs) with data that will enable them to judge the quality of student learning mediated by a challenging curriculum, collaborative learning, a learning support system, teaching quality and interactions between students and faculty.

Azerbaijan, having regained its independence more than three decades ago, is becoming a regional powerhouse of geo-strategic significance with a rapidly developing economy, whereas the educational system is falling behind, with perpetual reforms bringing few noteworthy changes and leaving the quality of education uncertain (Guliyev 2016; Isaeva and Aliyev 2023). In Azerbaijan, universities rarely have the academic freedom to design their curricula (Isakhanli and Pashayeva 2018). Moreover, the relationships between public institutions and HEIs are irregularly based on collaboration and dialogue.

While the quality of HE in Azerbaijan is a subject of continuous debate, certain changes have taken place since the collapse of the Soviet Union. These changes came along with an excessive number of newly established state and private HEIs, bringing diversity to the ways in which the universities are managed and provide quality (Isakhanli and Pashayeva 2018). Many of the changes relating to quality assurance, internationalisation, teaching and learning and, in general, institutional governance brought about by the Bologna process were barely enough to improve the HE context, given its strong and long-lasting Soviet legacy. Eleven private and 40 public universities currently have over 200,000 students, qualifying HE for the mass stage according to Trow’s massification stage division (Trow 1973; Smolentseva, Huisman, and Froumin 2018), with 36% enrolment in 2021. Government expenditure on education has improved significantly, reaching 4.3% of the GDP in 2020, compared with 2.7% in 2019. The series of changes has also been marked by policy-level documents, such as the State Strategy for the Development of Education in the Republic of Azerbaijan approved by the President (2013), the National Qualifications Framework for Lifelong Learning of the Republic of Azerbaijan (2018), the Regulation of the Agency of Quality Assurance in Education (2019) and the Accreditation Rules for Educational Institutions, which addresses student involvement in the quality assurance process, while nonetheless overlooking the need for student engagement to be defined and measured at the national level. Thus, a separate critical policy discourse analysis is needed to shed light on how student identity and/or student engagement have been addressed in policy documents.

The HE context in Azerbaijan differs from that of countries using the National Survey of Student Engagement (NSSE) or other tools to measure student engagement. The context is challenged culturally, but resource inaccessibility, the quality of teaching and support staff, the level of infrastructural development and the readiness of staff to conceptualise student engagement make it even more challenging. Student engagement is not measured at the national or institutional level, and student participation in any decision-making is uncommon, which can be explained by an educational culture characterised by centralised decision-making, power distance and academic hegemony. One of the cultural features distinguishing Azerbaijani students from their Western counterparts is their stronger ties with their families. Family ties are embedded in a collectivistic society, which implies that families have more authority over the behaviour of children, whereas the latter have less autonomy in decision-making (Rothon, Goodwin, and Stansfeld 2012; Schlee, Mullis, and Shriner 2009; Asgarova and Tsang 2022). Research also suggests that in collectivistic cultures, people are more inclined to enjoy and experience satisfaction in interpersonal relationships (Triandis 2000). Azerbaijani students traditionally do not stay on campus and, in most cases, universities do not have dormitories on campus. These aspects may affect the way in which students engage in collaborative learning, because it is a challenge for them to exercise tight control over the time spent on commuting and preparing for the following day’s classes. However, as stated by Coates (2007), students learn better when they experience different ways of learning, for instance, through interactions with peers.

Further research would add extra value to the conceptualisation of student engagement at the individual, institutional and policy levels in developing countries. Moreover, measuring student engagement at the national level is likely to provide institutions and policymakers with extensive information on how effective these institutions are in terms of student learning, given that they are regulated and provisioned by the state. Finally, clarifying the interdependence between investment and outcomes, provided that universities use scarce public resources, would inform policymakers on how effectively finances are used (Pusser and Marginson 2013; Pike et al. 2006).

This research aims to analyse how student engagement is linked to study success and how institutional factors impact student engagement in Azerbaijani universities.

Student engagement

As stated by Trowler et al. in 2022, student engagement, at its best, is the result of mutual investments that students and institutions make in quality learning. The ability of HEIs to support student engagement by creating a culture that appreciates and promotes student engagement from various perspectives is likely to bring about multiple positive student and institutional outcomes. Drawing on recent research, the following discussions cover these elements of student engagement.

The basis and manifestation of student engagement

Student engagement has been found to be a complex phenomenon that is based on two underlying theories: Alexander Astin’s (1984) theory of involvement and Pace’s (1980) concept of quality of effort. These theories are founded on the view that students improve personally and professionally while attending HEIs, and there are two sources of input contributing to student engagement in HE. Student input, according to many definitions, comprises the time, effort and energy students devote to learning knowledge and skills, while institutional input refers to institutional resources, enhanced educational activities that support and promote student learning, and, most importantly, enriched and challenging curricula that encourage students’ learning (Kuh 2003, 2009; McCormick, Kinzie, and Gonyea 2013). Student engagement in HE occurs both in and out of class, with students taking on various roles and responsibilities relating to decision-making and involvement in different out-of-class activities (Carey 2013). Consequently, HEIs must respond with investment in curriculum development, faculty and student support development, and the promotion of enriching educational activities. Pike and Robbins (2019) argued that investments in academic and support services are positively related to the student graduation rate.

Human factors such as personality, behaviours, emotions, and cognition, as well as situational, economic and social factors contribute to student engagement, making it a ‘meta’ construct (Fredricks, Blumenfeld, and Paris 2004, 60). Kahu (2013) reported that there are precedents for student engagement, namely, students’ social belongingness, former qualifications, and personality. Many studies have emphasised the role of motivation in student engagement, looking at it through the lenses of motivational theories, such as self-determination theory, achievement goal theory, achievement motivation theory, attribution theory, self-efficacy theory and the expectancy–value theory of achievement (Eccles and Wang 2012; Zepke and Leach 2010).

However, the impact of institutional contexts in engaging students in their studies is undeniable. As Zepke (2018) noted, student engagement happens within a context. Numerous studies have reported that institutional and disciplinary variations impact student engagement in the different experiences available to them (Pike, Smart, and Ethington 2012; Umbach and Wawrzynski 2005). Students will learn more as they are engaged in more academically purposeful activities. In addition, universities providing support and conditions for students to successfully reach their learning outcomes are likely to experience subsequent student perceptions, satisfaction and achievements (Kuh 2003).

The statement by Kahu (2020, 658) that ‘student engagement is a critical element of the educational interface’ leads us to think that students’ learning and their perception of the learning environment are linked. As Lizzio, Wilson, and Simons (2002) concluded, students’ perceptions of their learning environment, supported by their motivation and expectations, determine how institutional factors influence their learning strategies. Moreover, they stated that students’ perception of their learning environment is strongly related to reaching outcomes, such as student satisfaction, academic achievement and development of transferrable skills.

Outcomes of student engagement

Research has reported a positive impact of student engagement on learning outcomes (Laird et al. 2014), grades and satisfaction (Webber, Krylow, and Zhang 2013). Acquiring skills and knowledge during studies is essential for students to feel accomplished and fundamental to their success. For many students, attaining marketable skills is a success factor in their future studies and professional life. As a result of their achievements, students feel accomplished and satisfied (Maatta and Uusiautti 2017). Whether measured by retention, grades, grade point average (GPA) or completion rate, it is essential that students’ success and their learning, along with the acquisition of the necessary skills to meet the demands of the international job market, remain the ultimate goals of HEIs.

Numerous studies have shown that interactions between students and faculty – dialogue, discussions outside of the class and the quality of interactions – lead to better student outcomes (Kuh and Hu 2016; Pascarella and Terenzini 2005; Isaeva et al. 2020). Furthermore, students feel supported and better understand their subject matter if they have friendly and supportive relationships with the faculty. To foster learning, HEIs are therefore expected to promote students’ integration into their social and academic communities as equal members from the first year of their studies (Zepke 2018; Kahu 2013).

With regard to the most critical measurable goals and the reputation of HEIs, the outcomes of student engagement are learning, students’ collaboration with each other and faculty members and, ultimately, fast graduation (Bunce, Baird, and Jones 2017; Määttä and Uusiautti 2016; Trowler 2010). Moreover, student engagement has been shown to improve the quality of education (Coates 2015). In this regard, Kuh (2003) noted that institutions that engage their students in various activities leading to valued learning outcomes can state that they are of higher quality than universities where students are less engaged.

Thus, learning experiences and improvements in well-being, satisfaction, behaviours and civic-mindedness, as well as the development of students as academic community members (Ansala, Uusiautti, and Määttä 2015; Winstone et al. 2017), are all factors that provide immense support to the larger community (Chankseliani, Qoraboyev, and Gimranova 2021).

Methodology

This study aims to describe student engagement in Azerbaijani HE by asking the following research questions:

1. What are the specific engagement factors affecting student learning and success?

2. What are the institutional factors affecting student engagement in academic learning?

Using the NSSE questionnaire under licence from Indiana University granted in 2018 for research purposes only, the survey was conducted in April/May and September/October 2018 among university students in Azerbaijan to acquire a comprehensive overview of student experiences.

The NSSE

The NSSE survey instrument was used to gather data from students in Azerbaijan. The NSSE has been produced to measure the extent to which universities provide conditions for students to engage in educational experiences (Kuh 2009). Initially designed in 2000 and updated in 2013, the instrument assesses student engagement through 10 Engagement Indicators (EIs) united under the four themes presented in Table 1.

Table 1. NSSE Engagement Indicators.

THEMES	ENGAGEMENT INDICATORS
Academic Challenge	Higher Order Learning (HO)
	Reflective and Integrative Learning (RI)
	Learning Strategies (LS)
	Quantitative Reasoning (QR)
Learning with Peers	Collaborative Learning (CL)
	Discussions with Diverse Others (DD)
Experiences with Faculty	Student Faculty Interactions (SF)
	Effective Teaching Practices (ET)
Campus Environment	Quality of Interactions (QI)
	Supportive Environment (SE)

The instrument contains items concerning quantitative reasoning, interactions among diverse populations, learning strategies and teaching practice presented in precise and consistent language (Fosnacht and Gonyea 2018). Moreover, the instrument is accurate in measuring the effectiveness of educational practices (McCormick, Kinzie, and Gonyea 2013).

The questionnaire has questions regarding the extent to which students are engaged in a variety of educationally purposeful activities, the requirements of the university, the extent to which the curriculum is challenging, the supportiveness of the university environment and the extent to which the university contributes to students’ personal development (NSSE 2022). The survey has been used in the United States, Canada and other countries worldwide, and it has served as the basis for developing similar surveys in Australia, New Zealand, the UK, China and elsewhere. Furthermore, university administrators and faculty members have used NSSE results to make significant decisions on the effectiveness of educational practices provided on campuses (Pascarella, Seifert, and Blaich 2010; Fosnacht and Gonyea 2018; McCormick, Kinzie, and Gonyea 2013).

Data collection and research participants

The questionnaire was translated into Azerbaijani with extreme accuracy using a team of experts who examined the initial translation by one of the authors, who is a native speaker of Azerbaijani and fluent in English. The author is also an expert on the area, with more than 20 years of experience working in HE, and thus knows how to present data accurately in Azerbaijan. A team of experts from the Social and Educational Sciences and Humanities and from the Graduate School examined the translation for cultural and linguistic validation (Behr 2017).

Students were accessed through the official procedures of the university that allowed the researcher to conduct a survey. A non-probability sampling method, namely, convenience sampling, was used because the survey was conducted among students who were present at each university on the day of the designated data collection and who could thus be contacted for data collection purposes (Bornstein, Jager, and Putnick 2013). As it was difficult to predict how many students would be available to participate, the goal was to get at least 80 respondents from each university. Although convenience sampling has disadvantages, such as the challenge involved in controlling sociodemographic variances, it was considered appropriate for this research because each participating university was accessible to the researchers (Bornstein, Jager, and Putnick 2013).

In all, 640 questionnaires were distributed equally among eight universities. Of these, 450 were completed (70% response rate), which is considered satisfactory (Babbie 2020). In 27 cases, students did not respond to every question and were therefore excluded from the study while entering the data. Most of the questionnaires were distributed as hard copies, as an online questionnaire was administered only at one university.

The responses to the items were based on a 4-point Likert scale, except for questions relating to the quality of interactions, where the response scale ranging from 1 to 7 was replaced with a scale ranging from 1 to 4 during the analysis. The questions about student demographics concerned factual information, such as GPA, desired highest education level, age and gender. In addition, each participant signed an informed consent form and participated in the survey voluntarily.

Validity and reliability of a self-reported survey

The NSSE was exclusively designed to assess student engagement in sound educational practices and to explore what students have gained during their university experience. The NSSE questions, formulated clearly, concerned regular and familiar activities in which students are involved at the university. The survey was conducted anonymously to avoid embarrassment and to prevent the respondents’ privacy from being threatened or violated (Kuh et al. 2001).

To measure the internal consistency of construct in the study, Cronbach’s alpha was used, where an alpha (α) value greater than .70 was taken to indicate that a construct was reliable (Hair, Ringle, and Sarstedt 2013). A reliability analysis was conducted to diagnose the NSSE instrument’s reliability across eight Azerbaijani universities in Baku, the capital, and other regions. First, a Cronbach’s alpha reliability analysis was performed to compare the NSSE EI scale scores for samples of 25, 50, 100, 200 and 433 sequentially selected students; the scale’s reliability was not affected by the number of students involved in a sample. Second, Learning Strategies, Quantitative Reasoning, Student–Faculty Interactions, Quality of Interactions and a Supportive Environment were identified as indicators that yielded similar results in different cases (Appendix 1). Finally, a reliability analysis conducted through random selection demonstrated that only two indicators – Quality of Interactions and Supportive Environment – improved as a result of an increase in the number of students.

The reliability analysis performed across the institutions showed that only two universities generated a high-level α coefficient for almost all indicators (Appendix 2). Our basic deduction was that α did not improve as we changed the number of students but improved across the universities. The results revealed that the scales generated by urban universities were more reliable than those generated by rural universities (Table 2). All alpha coefficients exceeded the accepted cut-off scores for urban universities (α > 0.6). The alpha reliability criterion of meeting 0.9 ≤ α < 0.8 was characterised as good and 0.8 ≤ α < 0.65 as acceptable (Nunnally and Bernstein 1994; Vaske, Beaman, and Sponarski 2017).

Table 2. Reliability Test by Urban and Regional Universities.

Indicator/categories	Number of items	Urban universities (266 students)	Regional universities (167 students)
HO	4	.669	.334
RI	6	.701	.703
LS	3	.749	.614
QR	3	.781	.687
CL	4	.717	.401
DD	4	.667	.682
SF	4	.781	.701
ET	5	.626	.317
QI	5	.823	.728
SE	8	.838	.793

Consequently, only urban universities were chosen for further analysis. A possible explanation for the relatively low internal consistency may be attributed to the small number of items in each scale (McCormick, Kinzie, and Gonyea 2013). Nevertheless, in this study, the reliability test revealed somewhat strong internal consistency for the NSSE construct in the case of 266 students representing urban Azerbaijani universities. The study examined how HEI students in Azerbaijan engaged in different activities identified in the NSSE. The participants comprised 266 students representing five universities in Baku. Second-, third- and fourth-year students were surveyed because junior students may have had fewer chances to integrate socially and academically into their courses and institution and because students with low social and academic engagement may have a different perception of student success (Zepke 2018).

Table 3 presents some demographic data on the students participating in the study. The survey collected information on the year of study, gender, age, major and student status. The average age of the students was 21 years. The gender distribution was 45% males and 55% females. Of these, 13% were junior students, and 87% were senior students. In terms of academic performance (out of 100), about 7% had a GPA below 70, 11% above 90, and 82% between 71 and 90.

Table 3. Summary of Participants’ Sociodemographic Profile.

Sociodemographic	Frequency	Percentage	Mean	SD
Age			21.59	1.520
18–20 years	38	14.3
20–21 years	122	66.5
23 years	51	20.2
Gender			1.55	.499
Male	120	45.1
Female	146	54.9
Academic major			2.54	1.191
Science	78	29.3
Education	44	16.5
Social sciences	67	25.2
Humanities	77	28.9
Academic year
One			3.99	.833
Two	29	10.9
Three	6	2.3
Four	169	63.5
Five	62	33.3
GPA			80.40	6.689
Below 70	19	6.8
Between 71–90	218	82.3
Above 90	29	10.9

Variables

The variables used in this study were based on the data provided by the NSSE instrument. There were four themes with 10 EIs composed of different numbers of questions.

Academic challenge

The Higher-Order Learning I (HO) ndicator was composed of four questions rated on a 4-point Likert scale (Very much = 4, Quite a bit = 3, Some = 2, Very little = 1) and focusing on how students applied, analysed, evaluated and formed facts, theories, ideas, experiences and points of view.

The Reflective and Integrative Learning (RI) indicator was composed of six questions rated on a 4-point Likert scale (Very often = 4, Often = 3, Sometimes = 2, Never = 1) and focusing on whether students combined ideas, connected their learning to societal problems, had diverse perspectives, tried to better understand someone else’s ideas and had learned something that had changed their way of understanding.

The Learning Strategies (LS) indicator contained three questions rated on a 4-point Likert scale (Very often = 4, Often = 3, Sometimes = 2, Never = 1) and focusing on whether students identified key information, reviewed their notes and summarised what they had learned.

The Quantitative Reasoning (QR) indicator had three questions rated on a 4-point Likert scale (Very often = 4, Often = 3, Sometimes = 2, Never = 1) and focusing on the frequency with which students reached their calculations, learned about real-world issues and assessed other people’s assumptions using numerical information.

With regard to the second research question, Academic Challenge was identified as the dependent variable designated as ‘Academic Learning’ because Academic Challenge involves questions about students’ experiences of academic learning activities.

Learning with peers

The Collaborative Learning indicator (CL) was composed of four questions rated on a 4-point Likert scale (Very often = 4, Often = 3, Sometimes = 2, Never = 1) and focusing on how often students helped others, explained course material, prepared for exams by discussing or working through course material and worked on course projects with others.

The Discussions with Diverse Others indicator (DD) was composed of four questions rated on a 4-point Likert scale (Very often = 4, Often = 3, Sometimes = 2, Never = 1) and focusing on how often students had discussions with people of different races and ethnicities, economic backgrounds, religious beliefs and political views.

Experiences with faculty

The Student–Faculty Interactions indicator (SF) was composed of six questions rated on a 4-point Likert scale (Very often = 4, Often = 3, Sometimes = 2, Never = 1) and focusing on how often students talked to and worked with faculty members and discussed topics, ideas, concepts and academic performance with them.

The Effective Teaching Practices indicator (ET) was composed of six questions rated on a 4-point Likert scale (Very much = 4, Quite a bit = 3, Some = 2, Very little = 1) and focusing on the extent to which students’ instructors clearly explained the course goals, taught in an organised way, used examples and illustrations to explain difficult points, provided feedback on drafts and provided prompt and detailed feedback on tests or completed assignments.

Campus environment

The Quality of Interaction (QI) indicator was composed of five questions rated on a 4-point Likert scale (Excellent = 4, Good = 3, Fair = 2, Poor = 1) and focusing on students’ interactions with other students, academic advisors, faculty, student services and administrative staff members.

The Supportive Environment (SE) indicator was composed of eight questions rated on a 4-point Likert scale (Very much = 4, Quite a bit = 3, Some = 2, Very little = 1) and focusing on how much students’ institutions emphasised spending significant time studying, providing support to help students succeed academically, using learning support systems, encouraging contact among students from different backgrounds, providing opportunities to be involved socially, supporting their overall well-being, helping to manage non-academic responsibilities, and attending campus activities and events that addressed important social, economic or political issues.

To respond to the second research question, the indicators combined under the themes Learning with Peers, Experiences with Faculty, and Campus Environment were identified as independent variables designated as ‘institutional factors’ because the requested experiences under these themes mostly focused on the students’ experiences of engagement emphasised by the institutions.

Dependent variables – student outcomes

GPA – The students self-reported their GPA on a scale of 1–100 in response to an open-ended question.

The Perceived Gains scale (PG) was developed based on ten responses provided by students to the question, ‘How much has your educational experience at this institution contributed to your knowledge, skills and personal development in the areas of writing and speaking clearly and effectively, thinking critically and analytically, analysing numerical and statistical information, acquiring job-related knowledge and skills, working effectively with others, developing or clarifying a personal code of values and ethics, understanding people of other backgrounds, solving complex real-world problems and being an informed and active citizen?’

Student Satisfaction (ST) was assessed by students’ responses to the question, ‘How would you evaluate your entire educational experience at this institution? The responses were rated on a 4-point Likert scale (Excellent = 4, Good = 3, Fair = 2, Poor = 1).

Data analysis

Based on the objectives, the study was designed to examine institutional support variables contributing to student engagement and the linkages between student engagement, Perceived Gains, Student Satisfaction and GPA. A Pearson’s correlation analysis and a stepwise regression analysis were used to examine student outcomes in terms of the dependency of GPA, ST and PG on institutional factors.

The design of the research questions in this study led us to choose the stepwise regression method because of its power to select and reveal important independent variables in predicting dependent variables (Huang and Cheng 2013). This method provides the relative advantage of avoiding collinearity and finding the best combination of independent variables in predicting dependent variables with forward selection and backward elimination (Metsämuuronen 2017).

To respond to the first research question, we looked for interdependencies between the EIs and Perceived Gains, identified as learning outcomes, and success, identified by GPA and Student Satisfaction as a factor strongly related to students’ attachment to their university. In addition, we performed correlation and regression analyses to establish the interdependencies.

To respond to the second research question about the impact of institutional factors on student engagement, we built a stepwise regression between EIs with Higher Order Learning, Reflective and Integrative Learning, Learning Strategies and Quantitative Reasoning as dependent variables and Collaborative Learning, Discussions with Diverse others, Student–Faculty Interactions, Effective Teaching Practices and Supportive Environment as independent institutional variables.

The construct validity analysis in this study was based on assumptions by Pike (2013), who referred to the construct validity framework of Messick (1989) in relation to whether the nature of NSSE allows for factor analysis and generalisability over items. Furthermore, while a factor analysis helps in analysing the interactions of a measure and external variables, it does not help in assessing ‘structural component validity’, as argued by Pike (2013, 151). Many studies have proven the validity and reliability of the NSSE instrument in measuring the construct it was designed to measure (Pike 2013). It has been proven in multiple studies that because the construct of the NSSE was designed to provide information on how colleges and universities build student engagement, it is suitable for analysing universities rather than students (McCormick and McClenney 2012; Pike 2013). Thus, the analysis of this study is primarily based on its generalisability to universities as a unit of analysis. It then looks at the correlation between the EIs and the regression analysis to examine the interdependency of GPA, Perceived Gains, Student Satisfaction and student engagement.

Findings

The correlation analysis was utilised to compare the degree and patterns of correlation between the 10 EIs and student outcomes identified as GPA, Perceived Gains and Student Satisfaction.

As Table 4 shows, higher order learning had a very significant correlation with reflective integrative learning (r(264) = .203, p < .001), learning strategies (r(264) = .352, p < .001), quantitative reasoning (r(264) = .205, p < .001), student–faculty interactions (r(264) = .263, p < .001), effective teaching practices (r(264) = .252, p < .001) and supportive environment (r(264) = .214, p < .001). Surprisingly, higher order learning did not have a correlation with collaborative learning or quality of interactions and had a negative correlation with discussions with diverse others. Reflective and integrative learning had a correlation with almost all EIs at a very significant level, but it showed no correlation with effective teaching practices or perceived gains.

Table 4. Correlations between Engagement Indicators, Student Outcomes and Satisfaction (ST).

	HO	RI	LS	QR	CL	DD	SF	ET	QI	SE	PG	GPA	ST
HO	1
RI	.203**	1
LS	.352**	.302**	1
QR	.205**	.205**	.413**	1
CL	.034	.193**	.084	.049	1
DD	-.017	.149**	.086	.271**	.009	1
SF	.263**	.427**	.253**	.393**	.176**	.221**	1
ET	.252**	.059	.157*	.227**	.009	.003	.172**	1
QI	.091	.221**	.242**	.284**	-.029	.217**	.322**	.222**	1
SE	.214**	.261**	.356**	.377**	.162**	.123*	.329**	.287**	.313**	1
PG	.253**	.073	.242**	.246**	.070	.078	.195**	.217**	.171**	.362**	1
GPA	.035	.142*	.201**	.170**	-.125*	.048	.107	-.022	.281**	.015	.168**	1
ST	.253**	.190**	.385**	.332**	.025	.123*	.146*	.236**	.381**	.427**	.309**	.118	1

**. Correlation is significant at the 0.01 level (2 – tailed).

*. Correlation is significant at the 0.05 level (2 – tailed).

While statistically significant, the associations were comparatively low. In only two cases was the correlation between learning strategies and quantitative reasoning (r(264) = .413, p < .001) and reflective and integrative learning and student–faculty interactions (r(264) = .427, p < .001) relatively high at a very significant level. It is also worth mentioning that supportive environment had a significant correlation with all other EIs. Surprisingly, though, collaborative learning showed no correlation with any other indicator except SE (r(264) = .162, p < .001) and SF (r(264) = .176, p < .001) and a negative correlation with GPA (r(264) = -.125, p < .005) at a significant level. As expected, student satisfaction had a very significant correlation with all EIs and with GPA.

The results of the stepwise regression model for predicting student GPA, perceived gains and student satisfaction are presented in Table 5. Thereafter, we discuss the most solid models.

Table 5. Stepwise Regression Predicting GPA, Perceived Gains (PG) and Student Satisfaction (ST).

	Standardised β	Partial Correlation	p	Adjusted R^2	F	p
GPA
Model 1
QI	.281	.281	< .001	.076	22.69	< .001
Model 2
QI	.247	.245	< .001	.091	14.27	< .001
LS	.141	.143	.020
Model 3
QI	.240	.240	< .001	.105	11.32	< .001
LS	.154	.156	.011
CL	-.131	-.137	.026
Perceived gains (PG)
Model 1
SE	.362	.362	< .001	.128	39.83	< .001
Model 2
SE	.323	.326	< .001	.157	25.70	< .001
HO	.184	.193	.002
Student satisfaction (ST)
Model 1
SE	.427	.427	< .001	.179	58.84	< .001
Model 2
SE	.341	.427	< .001	.244	43.81	< .001
QI	.274	.381	< .001
Model 3
SE	.269	.427	< .001	.288	36.65	< .001
QI	.241	.381	< .001
LS	.231	.385	< .001

In the stepwise regression of GPA, Model 3 was the strongest explainer (R²= .105), showing that QI (β = .24, t = 4.76, p < .001), in particular, was the best predictor of self-reported GPA. Hence, students who interacted more with their professors, faculty, student services and other administrative staff achieved higher grades. Surprisingly, however, the model predicted that collaborative learning (β = -.13, t = −2.23, p < 026) would negatively impact GPA, meaning that students preferred to prepare for classes individually and did not consider the contribution of fellow students to be significant. The results showed that if students helped other students by explaining the course material or preparing for exams by discussing them with others, they had a lower GPA.

With regard to the stepwise regression of perceived gains, Model 2 showed that PG was explained by 15.7% of supportive environment and higher order learning (R² = .157). Supportive environment (β = .323, t = 6.31, p < .001), in particular, was a predictor of students’ perceived gains, such as writing and speaking effectively, thinking critically, analysing data, and understanding and supporting others. This signifies the importance of a supportive environment for student learning and mastery of skills. Students believed that being involved socially, attending campus events and using support services contributed to their effective writing and speaking skills and their ability to think critically and analyse numerical and statistical information.

Finally, in terms of student satisfaction, the stepwise regression analysis indicated that Model 3 was quite good (R²= .288). The results showed that supportive environment is also a good predictor of Student Satisfaction (β = .269, t = 7.67, p < .001). However, student satisfaction can also be explained by the quality of interactions (β = .241, t = 4.87, p < .001) and learning strategies (β = .231, t = 4.12, p < .001). Thus, students saw that they would be more satisfied with their universities if they had quality interactions, worked hard after classes, reviewed their notes, worked on their assignments and summarised what they had learned in courses.

Table 6 presents a stepwise regression model in which student–faculty interactions, effective teaching practices, discussions with diverse others, collaborative learning, quality of interactions and supportive environment were identified as institutional factors. Here, we analysed how those factors predicted students’ Academic Challenge indicators (HO, RI, LS and QR). Finally, we discuss the most solid models generated for each indicator.

Table 6. Stepwise Regression of Academic Challenge Indicators to Institutional Factors.

	Standardized β	Partial correlation	p	Adjusted R^2 for the model	F	p
Higher order learning
Model 1
SF	.263	.263	< .001	.066	19.68	< .001
Model 2
SF	.227	.231	< .001	.107	16.79	< .001
ET	.213	.252	< .001
Reflective integrative learning
Model 1
SF	.427	.427	< .001	.179	58.72	<.001
Model 2
SF	.382	.374	< .001	.192	32.54	< .001
SE	.136	.142	.021
Learning strategies
Model 1
SE	.356	.356	< .001	.124	38.36	< .001
Model 2
SE	.306	.299	< .001	.141	22.78	< .001
SF	.153	.154	.012
Quantitative reasoning
Model 1
SF	.393	.393	< .001	.151	48.09	< .001
Model 2
SF	.301	.307	< .001	.217	23.34	< .001
SE	.278	.286	< .001
Model 3
SF	.265	.273	< .001	.245	10.74	< .001
SE	.268	.281	< .001
DD	.180	.199	.001
Model 4
SF	.254	.263	< .001	.254	4.23	< .001
SE	.238	.246	< .001
DD	.185	.206	< .001
ET	.115	.126	.041

Note: SF, ET, DD, CL, QI and SE are identified as institutional factors, and HO, RI, LS, and QR are identified as engagement indicators making up an academic challenge theme.

The stepwise regression of higher order learning in Model 2 showed that SF (β = .227, t = 4.43, p < .001) and effective teaching practices (β = .213, t = 3.60, p < .001) predicted quite a bit of higher order learning, (R²= .107). The model indicated that students who experienced positive student–faculty interactions and effective teaching practices tended to analyse the data, apply them to different situations, evaluate points of view and form new ideas.

The regression of reflective integrative learning presented in Model 2 disclosed (R²= .192) that students experiencing positive student–faculty interactions (β = .328, t = 7.66, p < .001) and a supportive environment (β = .136, t = 2.32, p < .021) tended to learn in a reflective and integrative way. The regression model of learning strategies presented in Model 2 revealed (R²= .141) that a strong supportive environment (β = .306, t = 6.19, p < .001) predicted learning strategies. The results indicate that a supportive environment among students helps them formulate and improve their learning abilities.

Finally, the regression model of quantitative reasoning in Model 4 showed that student–faculty interactions (β = .245, t = 6.93, p < .001), SE (β = .238, t = 4.83, p < .001) and discussions with diverse others (β =  .185, t = 3.40, p < .001) positively influenced quantitative reasoning (R²= .254). Therefore, there seems to be a tendency among students to engage with statistical and numerical information to make conclusions. This tendency, though, was conditioned by student–faculty interactions, Discussions with diverse others and a supportive environment associated with academic support and participation in other services and activities organised at the university. However, when effective teaching practices, such as clear explanations and organisation, use of illustrative examples and provision of formative and informative feedback, explained higher order learning to a statistically significant extent, it had a low effect on students’ quantitative reasoning (< .041).

Discussion

Research on student engagement has demonstrated that it positively impacts student outcomes, such as GPA, perceived learning gains, and student satisfaction. This study shows that if students experience quality interactions with faculty and staff, receive support and use learning strategies, they learn new skills, achieve higher marks and feel satisfied with their educational experience. This study confirms the research conducted by Zilvinskis, Masseria, and Pike (2017), in which they argued that EIs are positively related to perceived gains. In addition, however, this study found that when students were likely to get support from the campus environment through collaborative learning, that is, meeting with others to get assistance, prepare for exams and work on group projects, it had a negative impact on their GPA. A possible explanation may be that students eagerly seek quality out-of-class engagement, which they identify as repeating what they have covered and helping them to improve their learning (Carey 2013).

The quality of interactions, a supportive environment and learning strategies predicted student success in this study in that students learned better when they were supported with academic arrangements, encouraged to be active in diverse interactions and provided with social opportunities, campus activities, health and wellness. At the same time, a curriculum requiring students to identify essential information, examine real-world problems and summarise course materials contributes to student learning. Based on the study, students’ satisfaction is greatly enhanced by a deliberately structured student support system, the amount of help students receive through that system, students’ perceptions of the quality of their interactions with staff and teachers, and the approaches students use to learn. These findings are supported by studies of Pittaway and Moss (2006) and Kuh and Gonyea (2003) that support systems are important for student engagement and outcomes.

Institutional factors affect the way students are engaged in academically challenging learning. The study revealed that students’ ability to analyse ideas and experiences, evaluate information from different sources and form new ideas depends on the frequency of meaningful and substantive interaction between teachers and students. This finding confirms the earlier finding by Umbach and Wawrzynski (2005) that students are affected by academic staff’s behaviours and attitudes to a great extent. Furthermore, it is essential for students’ higher order learning that instructors offer clear explanations and organisation, use illustrative materials and provide formative and effective feedback. In addition, student–faculty interactions and a supportive environment are related to reflective and integrative learning, exemplified by the connections students make with prior knowledge, other courses and societal issues, consideration of diverse perspectives and learning strategies, such as identifying critical information to reach conclusions, reviewing notes after classes and summarising class material.

Quantitative reasoning was contingent on student–faculty interactions, a supportive environment, discussions with diverse others and effective teaching practices, which leads us to think that students’ ability to reach conclusions, examine real-world problems and evaluate the way others view reality is defined by how well they communicate with faculty and diverse others, how well the classes are taught and how well the campus environment is organised to support their learning.

However, students also apply themselves to learning so as to advance their employability and success in further education and life. Skills such as thinking critically and analytically, speaking and writing clearly and effectively, being able to analyse numerically and statistically, working effectively with others, being able to solve complex problems and being tolerant and supportive of others will enhance their employability and define their success in future workplaces and life in general. Previous research has shown that employability skills can be taught in class using various teaching methods (Pegg et al. 2012; Riebe et al. 2010). In this regard, Zilvinskis, Masseria, and Pike (2017) reported that course effort, learning strategies, writing experiences and reflective learning, identified as in-class forms of engagement, help students to achieve academic learning outcomes and that collaborative learning and interaction with faculty outside of class, identified as out-of-class forms of engagement, help them to develop practical skills. Students’ knowledge of the factors affecting the acquisition of the above-mentioned outcomes may encourage them to invest more time, energy and effort in them.

Many studies have shown that human interaction and support are essential in encouraging and motivating students to learn. This study has shown that technologically advanced modern campuses may contribute to students’ perceptions of success, but reaching success requires human interaction, dialogue and support. Although the study yielded some strong evidence, its results should be generalised with caution for several reasons. To start with, the analysed data concern only five universities in the capital city, which may not represent the entire education system, given that using the NSSE to gather data on student experiences in the context of Azerbaijani HE may have produced information that differs somewhat from that acquired in other contexts thus far. Furthermore, different results might have been obtained if the participating universities ran similar surveys among their students annually, at the least. If students were certain that their feedback could result in some changes, were more familiar with the types of questions and had experience with long surveys, they would likely give more considered responses to the enquiries. Furthermore, although we relied on self-reported data that may be questioned as to its reliability, numerous studies have shown that such data can provide accurate results. (Arico et al. 2018; Cole and Gonyea 2010).

Student engagement is a complex construct consisting of controllable and uncontrollable variables that influence it. Even so, a heavy responsibility is placed on universities to persuade students to be engaged in their learning, whether by challenges placed on students through the agency of the curriculum or professors or by out-of-class activities enriched with learning practices or by university support systems that enable students’ effective learning. Student engagement in genuine learning mediated by the curriculum, teachers and support systems is likely to decrease some negative aspects in post-Soviet Azerbaijani HE (Sadigov 2014).

Despite these limitations, this study offers important implications for university leaders and policymakers reforming HE in Azerbaijan and/or countries with similar educational context.

First, emphasising student engagement at the policy level and identifying its indicators in documentation would provide policymakers data to further assess decisions about the effectiveness of policy-level changes. It would be advisable for institutional leaders to reconsider the conditions and strategies for student engagement and to embed the development of critical employability skills into curricula, regardless of the field.

Second, the study utilised the NSSE in a country outside of its coverage to explore how students learn in a culturally different country, where HE leaders rely on centralised decision-making and a high level of authority. To change this tradition and to engage students in both in-and out-of-class activities to the extent we observe in developed countries may require a ‘shift in mindset’ (Ergun and Kondakci 2021). The basic deduction is that wherever students are in any given cultural or economic situation, they must be provided with appropriate conditions, support and interactions with other students and faculty to engage them fully in learning and to support them in building the future.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data are available upon request.

Appendices

Appendix 1

Reliability test by the number of sequentially selected student samples

Table

SCALE	25 students	50 students	100 students	200 students	433 students
HO	.642	.753	.517	.494	.587
RI	.554	.528	.678	.678	.677
LS	.810	.705	.749	.714	.709
QR	.728	.771	.708	.709	.750
CL	.584	.688	.741	.584	.648
DD	.731	.720	.564	.725	.687
SF	.713	.728	.709	.730	.750
ET	.566	.442	.682	.433	.558
QI	.778	.711	.778	.755	.792
SE	.783	.777	.798	.818	.822

Reliability test by the number of randomly selected student samples

Table

SCALE	25	50 students	100 students	200 students	300 students
HO	.510	.579	.342	.555	.603
RI	.667	.693	.708	.683	.665
LS	.618	.538	.746	.634	.707
QR	.820	.774	.766	.699	.751
CL	.705	.637	.718	.670	.645
DD	.285	.614	.618	.671	.679
SF	.793	.790	.748	.708	.759
ET	.655	.598	.385	.543	.530
QI	.769	.787	.809	.813	.786
SE	.846	.854	.787	.823	.822

Appendix 2

Reliability tests by universities

Table

	1 (62)	2 (85)	3 (31)	4 (39)	5 (49)	6 (56)	7 (60)	8 (51)
HO	.704	.701	.455	.761	.604	.379	.322	.343
RI	.818	.727	.623	.639	.718	.641	.599	.677
LS	.827	.765	.745	.718	.509	.221	.670	.688
QR	.878	.649	.796	.777	.828	.592	.725	.628
CL	.653	.768	.796	.763	.581	.485	.421	.190
DD	.788	.702	.577	.647	.547	.814	.537	.631
SF	.813	.758	.787	.720	.826	.707	.695	.609
ET	.706	.691	.613	.190	.576	.343	.187	.409
QI	.831	.766	.718	.827	.893	.769	.651	.706
SE	.844	.801	.876	.764	.841	.855	.707	.773