Differential effect of university entrance score on first-year students’ academic performance in Portugal

Abstract

The main goal of this study is to show that the association between university entrance score and first-year students’ academic performance varies randomly across courses after controlling for students’ sociodemographic, schooling trajectory and motivational variables. The sample consists of 2697 first-year students who were enrolled in 54 courses at a Portuguese public university in 2015/16. Multilevel modelling of academic performance suggests that 34% of variability in grade point average is due to differences among courses and that 80% of such variability is explained by the field of study, whether the university is the student’s first choice, and the student’s gender, age and parents’ level of education. In addition, the results corroborate that the university entrance score is the strongest predictor of first-year academic performance.

Keywords:

• Academic achievement
• first-year experience
• higher education
• multilevel model

Introduction

Portuguese higher education is organised as a binary system, with university education oriented toward the provision of solid academic training, combining the efforts and responsibilities of both teaching and research units, while polytechnic education concentrates on vocational and advanced technical training that are professionally oriented (Assembleia da República 2007). From 1971 to 2011, the number of students enrolled in higher education increased by 700%, from 49,461 to 396,268. Dias (2015) presented a detailed analysis of official statistics to investigate how this strong increase was achieved and raised the question of whether such an increase (‘massification’) led to greater equity in terms not only of access but also of success in higher education. This growth implied an increase in the heterogeneity of the student population in terms of gender, socioeconomic level, age group and past academic trajectories. There is a broad consensus on the need for universities to promote students’ success, since many do not achieve their goals or fully realise their potential during their university experience, thus wasting talent and human potential. The rates of failure and abandonment have increased in recent years in Portugal, leading to questions regarding how institutions can support student retention (Dias et al. 2011; Ferreira, Vidal, and Vieira 2014).

Access to higher education in Portugal is organised by a system of numerus clausus defined as a national public tender to give priority of admission to students with higher admission scores – according to official statistics, in 2015/16 approximately 50% of students were placed in their first option for course and institution. Portela et al. (2008) suggested that students implicitly rank courses and institutions when making their choices, reinforcing the need to quantify the effect of first-option admission on students’ individual performance in undergraduate courses. The interaction between student and university starts when the student chooses the university and course s/he intends to attend. For instance, Almeida et al. (2006, 512) explored students’ sociodemographic characteristics related to the choice of courses at the University of Minho (a public university in northern Portugal), and concluded that students from less advantaged social classes were more likely to enrol in social sciences courses and that ‘higher sociocultural status and female students had higher entrance exam scores and averaged better academic achievement at the end of their first year at the university’.

Several authors noted that students’ socioeconomic and cultural capital are associated with the preference for more prestigious courses or institutions (e.g. Tavares et al. 2008; Tavares and Ferreira 2012). According to Tavares et al. (2008), students from disadvantaged family backgrounds have stronger preferences for courses related to teacher training and management, while students from advantaged family backgrounds are more likely to choose courses in the areas of law, fine arts, health and science. Considering, for example, the health sector, the authors found that the parents of 73.2% of students who were attending a university course in medicine hold higher education degrees, while 73.0% of the students enrolled in a health care course at a polytechnic institute came from families with no higher education degrees. Such findings focus the equity debate on the transition from high school to higher education and on the rules regarding access to public higher education, since the evidence appears to support the idea of strong socioeconomic selectivity during the process of students’ admission to higher education.

Several studies suggested the relevance of a student’s high school achievement or university entrance score as predictors of academic performance in higher education (McKenzie and Schweitzer 2001; Almeida 2002; Soares et al. 2009; Johnston et al. 2016) and as predictors of academic, social and institutional adjustment (Páramo Fernández et al. 2017). Nonetheless, McKenzie and Schweitzer (2001, 29) mentioned that ‘university entry scores need to be interpreted with caution, as they explain less than half of the variance in GPA’. Vulperhorst et al. (2018) compared the predictive validity of high school scores on academic success in university between the Dutch pre-university (314 graduates) and the International Baccalaureate (n = 113 graduates) diploma. According to the authors, the most valid high school score to be used for selection procedures in higher education remains unclear, and it seems to depend on the diploma students enter with.

Although several studies have examined these issues around the world, little is known about how university/course choice mediate the influence of prior achievement on academic performance in the context of higher education in Portugal. To our knowledge, this relationship has not been analysed in the literature as a differential effect that is dependent on the course. Therefore, this work has three main objectives: (1) to quantify the effect of first-option admission (to the course or the university) on students’ academic performance, controlling for sociodemographic variables and admission score; (2) to estimate the variability of students’ academic performance at the end of the first year by the hierarchical levels of student and course and (3) to estimate the effect of students’ admission scores on their academic performance, controlling for sociodemographic and schooling trajectory variables (sex, age, socioeconomic status and failure in primary or secondary education) and quantifying the amount of the effect due to differences among students and differences across courses.

The relationship between students’ performance in high school and in college

International research shows that tests for university admission, such as the Scholastic Aptitude Test (SAT) or American College Testing (ACT), or high school grade point average (GPA), are assumed to be the best predictors of students’ academic performance in higher education (Baron and Norman 1992). Those three predictors taken together can account for 25% or more of the variance in the mean scores during the first year (Mouw and Khanna 1993; Robbins et al. 2004). Other variables related to students’ personality (O’Connor and Paunonen 2007; Poropat 2009), motivation (Eccles and Wigfield 2002; Pintrich 2004) and methods of study are also important (Diseth et al. 2010), but the meta-analysis performed by Richardson, Abraham, and Bond (2012) with all these variables confirms the higher predictive capacity of the SAT, ACT and high school GPA.

Pike and Saupe (2002) analysed data for 8674 first-time college students from 124 high schools who enrolled at a midwestern USA university between 1996 and 1999 and considered the students’ first-year GPA as a dependent variable. They also included in the analysis the ACT score as part of the university’s admission policy, high school class percentile rank, and a binary variable representing whether the student had or had not met new core course requirements implemented in 1997. Their results show a significant positive relationship between having met the new core course requirements and first-year GPA, providing evidence of the appropriateness of the new admission requirements for students to succeed in higher education. Regarding the effect of the ACT assessment composite score and high school class percentile rank on GPA, both coefficients were positive and statistically significant.

Fuller, Wilson, and Tobin (2011) explore to what extent the cross-sectional and longitudinal National Survey of Student Engagement (NSSE) scores predict a college student’s final GPA. Their models include ACT scores and high school GPA as indicators of incoming students’ precollege abilities. Based on longitudinal data modelling, the authors conclude that 27.8% of the variance explained could be attributed to the precollege academic indicators. McKenzie and Schweitzer (2001) conducted an investigation of the relationship between academic, psychosocial, cognitive and demographic factors and students’ academic performance, assessed by a student’s classification at the end of the first term in college. The authors suggested that as university entrance scores are moderately correlated with a high GPA, entry to courses with a high level of difficulty should be established at a realistic level to avoid undue problems for students with lower university entrance scores. The literature also shows that students’ performance over the first year tends to determine their performance in the following academic years (Berg and Hofman 2005; Yorke and Longden 2007).

Differentials of academic performance by area of study

Masui et al. (2014) aimed to clarify the role of study time in academic performance by examining its capacity in the prediction of course grades, beyond relevant student characteristics (e.g. prior domain knowledge, prior study delay) and evaluating the predictive role of course characteristics in study time investment. The study was based on a sample of 168 (out of 196) freshmen in business economics at Hasselt University. According to the authors, their findings emphasise the relevance for future research of examining relationships at the course level, since the influence of students’ prior knowledge and skills on their academic performance seems to depend on their match or mismatch with the course requirements. They suggested that studies on the topic should take into account the field of study because differential grading induces the mismeasurement of academic performance.

In line with this view, Beenstock and Feldman (2016) applied regression models to longitudinal data gathered between 2003 and 2008 from 1217 social sciences students (majoring in psychology, sociology, political science, international relations, statistics, economics, business studies, accounting, communications, politics, philosophy, economics and geography) at the Hebrew University of Jerusalem with the purpose of decomposing course grades into the rewards to student ability at the university and the rewards to instructor quality. The authors reported a correlation of 0.41 between a student’s high school ability and his/her performance at the university and showed that there were grade differentials that were probably established by policies determined at the departmental level. Their findings suggested that students majoring in economics obtained higher grades in non-economics courses after controlling for students’ high school ability.

Considering data from algebra courses at a large community college in California, Chingos (2016) applied regression models to the longitudinal data of 8654 students collected from 2008 to 2011 with the objective of examining to what extent student learning varies across different sections. The author found that student learning varies systematically across instructors, who appeared to influence student learning beyond their impact on course completion rates.

In general, students with higher socioeconomic status and women have higher levels of academic performance as shown by GPA (Smith and Naylor 2001; Laforge 2003; Robbins et al. 2004; Dennis, Phinney, and Chuateco 2005; Voyer and Voyer 2014), as these socioeconomic factors are present in the explanation of variance in predictors (SAT, ACT or high school GPA) as well as in the average classifications during the first year of college (Sackett et al. 2009). Regarding students’ age, in some studies, older students show a slight superiority in academic performance (Clifton 1997), but in other studies, the youngest students perform better (Farsides and Woodfield 2007). In a meta-analysis by Richardson, Abraham, and Bond (2012), the correlations between GPA and socioeconomic background, gender and age suggested that students from higher socioeconomic status, female students and older students obtained higher GPA. In spite of being statistically significant, the effect size estimates were small.

This study sought to answer the following research questions: Q1. Does the university or course choice matter for the student’s academic performance? Q2. How much is the variance of first-year students’ academic performance due to the variability across courses? Q3. Is there any evidence that students’ university entrance score is predictive of the first-year academic performance? Q4. Does the relationship between university entrance score and first-year academic performance vary across courses? Q5. How great are the differences in grading among the areas of study? Thus, the first and second research questions are linked respectively to the first and second research objectives, while the remaining research questions are related to the third objective.

We are confronted with hierarchically structured data in which students are nested within courses. According to some authors (e.g. Pike and Rocconi 2012) much of the research in higher education has failed to properly account for the hierarchical nature of the data. However, multilevel modelling has become increasingly popular, as Wells et al. (2015) showed. Multilevel models (Bryk and Raudenbush 1992; Goldstein 2003) allow us to simultaneously consider multiple levels of effects, as many as the hierarchical structure includes. To obtain empirical evidence to answer our research questions, we used two types of multilevel models: the variance component model (VCM) and the random coefficients model (RCM).

Participants, variables and procedures

The sample consists of 2697 first-year students who were enrolled in undergraduate studies at the University of Minho in the academic year 2015/16, the same year that it became a public foundation under private law (article 129 of the Legal regime for higher education institution, RJIES). The University of Minho is located in the North of Portugal and offers graduation courses and postgraduate, including masters and doctorate degrees, for around 20,000 students in all fields of study. Data used for the purpose of this paper concern the following student attributes: academic performance measured by the average score (GPA) at the end of the first year; university entrance score; first-option admission (yes/no) to the institution and the undergraduate course; gender; age; schooling trajectory assessed by the early experience of repetition (yes/no); and parents’ education as proxy for the student’s socioeconomic status. The area of the course was also included as an explanatory variable.

Most of the students (57%) were female and were enrolled in their first-choice course (59%) and first-choice higher education institution (72%). Almost half of the students (47%) were registered in a science, technology, engineering and mathematics (STEM) course; 17% in economics, management or public administration; 13% in humanities, 10% in social sciences; 8% in health (medicine or nursing) and 5% in law. The students’ ages ranged from 16 to 61 years, with an average of 18.9 (SD 3.6), and 91.9% were full-time students. The average university entrance score was 152.4 (SD 18.9). Most of the students (83.3%) declared that they had never repeated a year along their schooling trajectory, and 13.9% declared at least one repetition. The distribution of students by parents’ education shows that 34.4% of their fathers and mothers had no more than a lower secondary education, and for 15.7%, both parents had a higher education degree. The academic performance data were available for 72% of the students, which is 1940. The remaining cases (28%) were attributable mainly to student dropout and transfers to other undergraduate courses, departments or institutions.

The descriptive statistics for the subset of students without an academic score at the end of the first year showed that they are older, more likely to have repeated a year during primary or secondary education, and had a lower university entrance score than the students who composed the full sample. The university entrance score is missing for 6% of students, the majority of whom got special access to higher education (Ministério da Educação e Ciência 2014). Such students could be immigrants or older than 23. In addition, there are no more than 4% of missing data in the remaining variables, which we consider to occur completely at random (Little and Rubin 2002). Students were informed about the objectives of the study and gave their written agreement, which included the possibility of accessing their classifications (GPA) at the end of the first semester and at the end of the first year (second semester). The remaining data were collected at the moment of enrolment in the university. For statistical modelling, we decided to standardise the variables GPA and university entrance score; thus, the results are interpreted in terms of standard deviations from the mean.

Statistical modelling

We applied multilevel (Goldstein 2003) or hierarchical (Bryk and Raudenbush 1992) models, specifically the Variance Components Model (VCM) and the Random Coefficient Model (RCM), since they have several advantages when the data are hierarchically structured, as is the population in the study. That is, students are grouped in undergraduate courses, so two-level models are applied, with students (indexed by i) at level 1 and courses (indexed by j) at level 2. The advantages of this class of models include the ability to model differential effects and to improve the efficiency of the estimators, i.e. with narrower confidence intervals and shrunken estimators, which are more efficient, especially when the group size is small or there is a large within-group to between-group variance ratio (Longford 1993; Ray, McCormack, and Evans 2009).

For each student i in undergraduate course j, Y represents the first-year academic performance measured by the student’s average score (GPA) obtained at the end of the academic year, and the observed scores are $y_{ij}$, where $i=1,\dots ,n_j\text{ and }j=1,\dots ,J$, with nj representing the number of students in course j and J representing the total number of undergraduate courses offered. The event Y is assumed to follow a normal distribution with mean XB and variance $\mathrm{\Omega }$, i.e. $Y\sim N\left(XB,\mathrm{\Omega }\right)$, where XB is the additive linear predictor or deterministic component of the model; X is the matrix of independent variables described above; B is the vector of fixed parameters; and $\mathrm{\Omega }$ is the variance-covariance matrix, which includes the random parameters of the model.

The two-level VCM to address objective 1 is written as follows: (1.1) $$y_{ij}={\beta 0}_j+{\sum }_{p=1}^6\beta p{Xp}_{ij}+e_{ij}$$(1.1) (1.2) $${\beta 0}_j=\beta 0+{u0}_j$$(1.2)

Equation (1.1)(1.1) $$y_{ij}={\beta 0}_j+{\sum }_{p=1}^6\beta p{Xp}_{ij}+e_{ij}$$(1.1) is defined at level 1 and consists of the linear predictor and the level 1 random component ($e_{ij})$, and Equation (1.2)(1.2) $${\beta 0}_j=\beta 0+{u0}_j$$(1.2) is defined at level 2, with the term of intercept ($\beta 0)$ and the level 2 random component (${u0}_j)$. The fixed parameters of the model are $\beta 1,\mathrm{ }\beta 2,\mathrm{ }\beta 3,\mathrm{ }\beta 4,\mathrm{ }\beta 5,\text{ and }\beta p$, where p denotes the number of covariates. The probability distribution assumptions for the random components are $e_{ij}\sim N\left(0,{\sigma }_e^2\right)$ and ${u0}_j\sim N(0,{\sigma }_{u0}^2)$, respectively. That is, ${\sigma }_e^2$ is the random parameter at level 1 (the level 1 variance) and ${\sigma }_{u0}^2$ is the random parameter at level 2 (the level 2 variance). According to the model defined by Equations (1.1)(1.1) $$y_{ij}={\beta 0}_j+{\sum }_{p=1}^6\beta p{Xp}_{ij}+e_{ij}$$(1.1) and (1.2)(1.2) $${\beta 0}_j=\beta 0+{u0}_j$$(1.2) and by the assumptions, the intercept of the model varies randomly across programmes (level 2 units).

We used this model to analyse the relationship between academic performance and the student’s set of background covariates. Specifically, we were interested in understanding the relationship between the dependent variable and the student’s sociodemographic characteristics, such as gender, family sociodemographic and economic background, schooling trajectory assessed by the experience of retention, and score on tests for access to higher education. We also wanted to determine the amount of academic performance variance explained by the background variables and how much is due to the undergraduate programme. For this purpose, we compared the estimates of the random components obtained from the full model with those obtained from the null model. Two statistics are used for that purpose: the coefficient of determination and the variance partition coefficient (Goldstein, Browne, and Rasbash 2002).

The two-level RCM is defined by Equations (2.1)–(2.3), meaning that the linear relation between university entrance score and academic performance ($\beta 1$) varies randomly across courses: (2.1) $$y_{ij}={\beta 0}_j+{\beta 1}_j{X1}_{ij}+{\sum }_{k=2}^P\beta k{Xk}_{ij}+e_{ij}$$(2.1) (2.2) $${\beta 0}_j=\beta 0+{u0}_j$$(2.2) (2.3) $${\beta 1}_j=\beta 1+{u1}_j$$(2.3) .

The probability distribution assumptions for the random components are now $$e_{ij}\sim N(0,{\sigma }_e^2),$$ $$\left[\begin{array}{c}{u0}_j\\ {u1}_j\end{array}\right]\sim N\left(\begin{array}{cc}\left[\begin{array}{c}0\\ 0\end{array}\right],\mathrm{ }& \left[\begin{array}{cc}{\sigma }_{u0}^2& {\sigma }_{u01}^{}\\ {\sigma }_{u01}^{}& {\sigma }_{u1}^2\end{array}\right]\end{array}\right).$$

This model includes two additional random parameters for level 2. The variance ${\sigma }_{u1}^2$ quantifies the variability of $\beta 1$ across courses, and ${\sigma }_{u01}^{}$ quantifies the respective covariance between the intercept and $\beta 1$.

The empirical evidence for question Q1 (Does the university or course choice matter for the student’s academic performance?) is provided by a RCM; for question Q2 (How much is the variance of first-year students’ academic performance due to the variability across courses?) is provided by a VPC; for question Q3 (Is there any evidence that students’ university entrance score is predictive of the first-year academic performance?) either by VPC or RCM; and for questions Q4 (Does the relationship between university entrance score and first-year academic performance vary across courses?) and Q5 (How great are the differences in grading among the areas of study?) the evidence is based on a RCM.

We used the iterative generalised least squares (IGLS) algorithm as an estimation procedure implemented in MLwiN (Rasbash et al. 2014). In addition, the variance partition coefficient (Goldstein, Browne, and Rasbash 2002) was applied to quantify the amount of variance distributed by each hierarchical level.

Results

Tables 1 and 2 present the estimates of the fixed and random parameters for the VCM and RCM, respectively. Asterisks are used to identify statistically significant effects based on a critical t value of ±1.96, i.e. a level of significance of 5%, and double asterisks when the critical t value is ±1.65, i.e. a level of significance of 10%. The variance partition coefficient applied to the null model parameter estimates (Table 1) shows that the variability of GPA decomposes 34% across courses and 66% among students or within course. Such estimates allow us to provide empirical evidence that supports objective 2 and research question Q2. The column M1 in Table 1 contains the estimates for university entrance score as a unique covariate, and M2 includes sociodemographic variables (sex, age and socioeconomic status), school trajectory (failure in primary or secondary education) as well. The column M3 in Table 2 adds the random coefficient parameters keeping the M2 set of explanatory variables, and M4 includes also the covariates related to university and programme choice, and areas of study. Thus, objective 1 and research question Q1 is addressed by the results obtained from model M4 and the final model (Table 2); empirical evidence that supports the research questions underlying objective 3 is provided by all fitted models.

Table 1. Parameter estimates for variance component models.

	Null model		M1		M2
	Estimate	SE	Estimate	SE	Estimate	SE
Fixed Parameters
Constant	–0.075	0.081	0.027	0.065	0.031	0.067
University entrance score			0.554*	0.028	0.553*	0.030
Male vs. female					–0.128*	0.041
Age >19					0.169*	0.082
Parents’ education: no more than lower secondary					0.118*	0.040
Parents’ education: higher education					–0.045	0.052
Repetition at primary (yes vs. no)					0.164	0.150
Repetition at secondary (yes vs. no)					–0.009	0.062
Random Parameters
Level 2: course
Var(constant)	0.340*	0.068	0.198*	0.043	0.181*	0.040
Level 1: student
Var(constant)	0.661*	0.022	0.552*	0.019	0.542*	0.018
Number of level 2 units: course	54		53		53
Number of level 1 units: student	1940		1826		1797
Estimation procedure	IGLS		IGLS		IGLS
–2*log-likelihood:	4847.948		4223.197		4121.568

(*) Statistically significant at the level of 5%.

Table 2. Parameter estimates for random coefficients models.

	M3		M4		Final
	Estimate	SE	Estimate	SE	Estimate	SE
Fixed Parameters
Constant	0.019	0.066	–0.207	0.073	–0.275	0.062
University entrance score	0.531*	0.038	0.580*	0.046	0.567*	0.037
Male vs. female	–0.131*	0.040	–0.131*	0.040	–0.126*	0.040
Age >19	0.147**	0.082	0.131	0.084	—	—
Parents’ education: no more than lower secondary	0.122*	0.040	0.129*	0.040	0.135*	0.038
Parents’ education: higher education	–0.043	0.051	–0.028	0.052	—	—
Repetition at primary (yes vs. no)	0.179	0.149	0.126	0.147	—	—
Repetition at secondary (yes vs. no)	–0.038	0.062	–0.053	0.062	—	—
Course first choice (yes vs. no)	—	—	0.072**	0.042	0.078**	0.042
University first choice (yes vs. no)	—	—	0.139*	0.045	0.141*	0.045
Area of study: ECO vs. STEM	—	—	–0.178	0.117	—	—
Area of study: HUM vs. STEM	—	—	0.722*	0.134	—	—
Area of study: SOC vs. STEM	—	—	–0.152	0.144	—	—
Area of study: HEALTH vs. STEM	—	—	0.391*	0.197	—	—
Area of study: LAW vs. STEM	—	—	–0.795*	0.281	—	—
Area of study ECO X University entrance score	—	—	–0.036	0.102	—	—
Area of study HUM X University entrance score	—	—	–0.269*	0.107	—	—
Area of study SOC X University entrance score	—	—	–0.083	0.122	—	—
Area of study HEALTH X University entrance score	—	—	0.391	0.242	—	—
Area of study LAW X University entrance score	—	—	0.004	0.191	—	—
Area of study: HUM vs. STEM + ECO + SOC	—	—	—	—	0.791*	0.130
Area of study: HEALTH vs. STEM + ECO + SOC	—	—	—	—	0.424*	0.194
Area of study: LAW vs. STEM + ECO + SOC	—	—	—	—	–0.735*	0.249
Area of study HUM X University entrance score	—	—	—	—	–0.271	0.104
Random Parameters
Level 2: Course
Var(constant)	0.167	0.039	0.061	0.018	0.062	0.018
Cov(university entrance score/constant)	–0.037	0.016	–0.013	0.009	–0.013	0.009
Var(university entrance score)	0.028	0.013	0.019	0.011	0.021	0.011
Level 1: student
Var(constant)	0.530	0.018	0.520	0.018	0.520	0.018
Number of level 2 units: course	53		51		51
Number of level 1 units: student	1797		1746		1747
Estimation procedure	IGLS		IGLS		IGLS
–2*log-likelihood:	4101.203		3890.304		3914.561

(*) Statistically significant at the level of 5%; (**) Statistically significant at the level of 10%.

Both models produced estimates for fixed coefficient and standard error, indicating that there is a positive statistically significant relationship between GPA and university entrance score (b1 = 0.55; se =0.03). The coefficient of determination (R2) indicates that the university entrance score explains 25% of the GPA total variance, and, focussing on level 2, 42% of the variability of GPA across courses. The results for model M2 show that after controlling for sociodemographic variables and schooling trajectory, the marginal effect of university entrance score on GPA remains stable (b1 = 0.55), and the total variance explained increases to 28% (3 more percentage points than before) and increases to 47% at level 2 (5 more percentage points than before). The results also suggest that a student’s GPA is related to gender, the student’s age, and the parents’ level of education. When the influence of university entrance score on GPA is compared to the influence of socioeconomic variables, either the variance explained or the estimates for the fixed parameters indicate that the university entrance score has a stronger influence.

M3 adds to the previous findings by allowing the university entrance score coefficient to vary across courses. The change in the –2 log-likelihood value, which is also the change in deviance, has a chi-square distribution on 2 degrees of freedom under the null hypothesis that the extra two random parameters (variance and covariance) have population values of zero. The deviance is 20.365 (p value <0.001). Thus, the statistically significant random effect indicates that the relationship between GPA and university entrance score varies randomly across courses, i.e. the linear relation is stronger or weaker depending on the course.

The results for M4 suggest that the first-choice university and area of study are relevant variables in the GPA analysis. At the level of significance of 5%, our results do not show a statistically significant effect of first-choice course on GPA unless the variable first-choice university is excluded from the model. In such a situation, the effect seems to be due to the omitted variable, meaning a spurious relationship between the variables. However, the fixed parameter is statistically different from zero at the level of 10%.

Regarding the area of study, on average, students enrolled in humanities, health or law courses have a significantly different GPA than their colleagues who enrol in any STEM, economics and social sciences course. According to our estimates, grading in humanities or health is higher than in STEM courses, and in law courses grading is lower. In economics or social sciences, it is not possible to say whether, on average, grading is significantly different than in STEM courses. For that reason, in the final model, the baseline for areas of study comprises the STEM courses, economics, management, public administration and social sciences.

The estimates for the interaction terms between area of study and university entrance score suggest a statistically significant term for humanities course with negative estimate, meaning that the humanities students with higher entrance score have their GPA considerably reduced, having added the principal terms related to university entrance score and the dummy term for humanities. Such interaction represents the narrowing in the dispersion of classifications that occurs with greater incidence in the humanities courses.

The final model includes all explanatory variables that coefficients revealed to be statistically significant in models M1, M2 and M3. The coefficient of determination applied to the final model variances indicates that, on average, the model explains 42% of the GPA total variance; focussing on level 2, it explains 82% of the GPA variability across courses and 21% of the GPA variability at the student level.

The estimates obtained confirm the interpretations presented; that is, there is evidence in favour of an effect of first-option admission to the university on students’ academic performance, controlling for sociodemographic variables and admission score. Those students for whom the University of Minho was their first choice have their GPA increased by 0.141 standard deviations on average compared to their colleagues. The results do not provide enough evidence, at the level of significance of 5%, to allow us to say that the first-choice course influences students’ GPA.

According to the model, male students have their GPA reduced by 0.126 standard deviations, and students whose parents both have no more than a lower secondary education (i.e. a lower level of education) have their GPA increased by 0.135, on average, compared to their colleagues whose parents have any other level of education.

The final model replicates what Figure 1 shows for the relationship between a student’s entrance score and their GPA, that is, the differential effect between the two variables is represented by one line fitted for each course. It can be observed that the positive relationship between the variables is stronger or weaker depending on the course and, in general, for negative values of student’s entrance score the course lines dispersion is greater than for positive values. This phenomenon is quantified by the covariance between the intercept and university entrance score slope, which has a negative estimate. Consequently, the respective correlation is approximately –0.4, suggesting that at the course level, the higher the GPA intercept, the lower the coefficient between university entrance score and GPA.

Figure 1. Graph based on final model estimates.

This correlation can be observed in the graph (Figure 1). As an example, two course lines were highlighted with dashed and dotted lines. The dashed line presents a higher intercept (average GPA) and lower slope (effect of university entrance score on GPA), while the dotted line shows a lower intercept and higher slope. The slope for the dotted line course is higher than the slope for the dashed line course, suggesting a stronger effect on GPA for the dotted course than for the dashed. We need more research on this topic to explore in detail the influence of the curricular prerequisites on the slope of the lines, that is, for the explanation of the degree of the effect of the university entrance score on students’ academic success.

Discussion and conclusion

In this study, we applied VCMs and RCMs to data collected from 1940 Portuguese first-year students enrolled at a public university in 2015/16, considering first-year GPA as a dependent variable. The partitioning of the first-year GPA variance suggests that 66% is accounted for at the student level and 34% at the course level. Part of this variability is due to the differences in the mean of the classification by scientific area of study. When STEM courses were considered as a baseline, it was verified that, on average, the GPA is not different from that in courses in economics and social sciences. However, courses in the humanities and health areas have a statistically higher mean GPA than STEM courses, and law courses tend to present a statistically lower mean GPA than STEM courses. These findings appear to be in concordance with those of Masui et al. (2014), who demonstrated differential grading by field of study, and support the idea that differential grading is possibly induced by departmental norms (Beenstock and Feldman 2016).

The estimates obtained suggest that the relationship between university entrance score and GPA at the end of the first year is on average 0.567 per standard deviation and varies randomly across courses after controlling for sociodemographic variables (gender, parents’ education), area of study, and whether it was the first-choice university. The magnitude of the fixed parameter estimate is larger when compared to other marginal effects included in the linear predictor. In addition, the model explains 42% of the GPA variability and 82% at level 2; i.e. it explains 82% of the GPA variability across courses and 21% of the GPA variability at the student level. Our findings corroborate the literature on the predictive power of prior achievement. For instance, Fuller, Wilson, and Tobin (2011) reported that 27.8% of the variance explained was attributable to the precollege indicators. Moreover, our findings suggest that some courses establish stronger conditions for prior knowledge prerequisites than others, with implications for curriculum development. This evidence supports the theoretical model proposed by Alarcão (2000), granting to curriculum a status similar to that granted to the student, teacher or institution.

Our findings also suggest that those students for whom the University of Minho was not their first choice have a lower GPA. The results do not provide evidence at the level of significance of 5% to allow us to say that the first-choice course matters in their academic performance. However, according to the results obtained by Ferrão and Almeida (2018), the admission of the student to their first option programme is important for their persistence over the first year of studies. The estimates presented suggest that the odds ratio in the persistence of students who were admitted in their first option programme is 1.85. This means that the probability of persistence over the probability of dropping out, suspending or transferring is almost twice as high in the group of students who were admitted to the first option course compared to students not admitted in their first choice. Those authors add ‘the conditions of access to higher education, namely the condition underlying the entrance classification, can contribute to the student’s choice being conditioned’ (p. 674).

The statistical model for first-year academic performance shows that the students’ socioeconomic level has influence through the increased motivation of the subgroup of students whose parents have no more than a lower secondary education. This effect seems to represent the willpower of students from socioeconomically disadvantaged groups who were admitted to the university and strive to achieve good results, because good achievement is the requirement for keeping the scholarship. This finding may contribute to the literature on social equity in higher education and to the debate on the motivational component of social status (e.g. Eckland 1964; Tinto and Cullen 1973; Cantwell, Archer, and Bourke 2001) but needs to be confirmed with more cohorts of entrants.

Such results also give quantitative support to the conjecture advanced by Engrácia and Baptista (2018), according to which

children of parents with low schooling who, despite the contrary winds, can finish high school and enter higher education, will be, on average, young people with school performance and levels of resilience above average among students from disadvantaged backgrounds . Therefore, their subsequent performance in higher education is expected to be more comparable to the performance of students from favored socio-economic backgrounds. (p. 18)

To our knowledge, for the first time in Portugal, the results show the advantage in GPA that students of disadvantaged socio-cultural origin have compared to their colleagues, even after controlling for the remaining variables.

We believe that our study constitutes a meaningful contribution to the available research by using microdata from a large sample of first-year students, by considering their schooling trajectory through retrospective data, and by taking into account the hierarchical structure of the data in statistical modelling. However, care should be taken not to generalise the results. The study concerns the entering cohort of 2015/16. Had different cohorts been selected, the results might have been affected. A full investigation of different cohorts was outside the scope of the current study, but it would be very interesting to investigate the sensitivity of these results to new nationwide cohorts of students.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This research was supported in part by the project CEMAPRE - UID/MULTI/00491/2013 from the Foundation for Science and Technology, FCT/MCTES through national funds.