Differential effect of university entrance scores on graduates’ performance: the case of degree completion on time in Portugal

Abstract

Degree completion on theoretical time is a phenomenon seldom explored in the higher education literature. We applied variance components models and random coefficients models to the microdata of an entire entrant cohort of first-time, full-time undergraduate students who completed their three-year programme at a Portuguese institution during the theoretical period. The study showed that the variance partition coefficient is 0.27, considering the hierarchical structure of students nested in programmes. The differential effect of students’ university entrance scores on degree completion grade point average is stronger across programmes than across faculties, controlling for students’ sociodemographic background (gender, age and parents’ level of education), social scholarship granted, and preference regarding the institution and programme attended. The fixed effects related to the areas of study and type of institution (e.g. university or polytechnic) were also quantified. The estimates indicated that secondary school preparation is the most important predictive factor for the final grade point average of degree completion among the variables at enrolment. Moreover, differences based on gender, age, and areas of study were found.

Keywords:

• University entrance score
• degree completion
• final grade average
• multilevel model

Introduction

More than 38 million students attended the European Higher Education Area (EHEA) in 2018–2019 (European Commission/EACEA/Eurydice 2020); most of them (56.4%) were enrolled in three-year programmes. Widening the access to study programmes and improving degree completion rates have been priorities since the Bologna Declaration. A large-scale comparative study on students’ dropout and degree completion conducted by Vossensteyn et al. (2015) provides insight into the policies that 35 participating countries and higher education institutions have applied to address these phenomena and monitor the effectiveness of such policies. The literature review was based on well-established bibliographic data banks and enriched by each of the 35 participating countries’ national experts, who were asked to ‘collect information on the most recent policies and initiatives that have been implemented’ and ‘to summarize grey literature and evaluation studies (especially those published in their national language) on the factors impacting on study success’ (Hovdhaugen, Kottmann and Thomas 2015, 9). According to these authors, ‘there were no studies available about Iceland, Lithuania, Cyprus, Macedonia, Malta, Slovenia, the Czech Republic, Bulgaria, Hungary, Montenegro, Romania, Slovak Republic, Serbia, Portugal, Liechtenstein, and Luxemburg’ (9). This means that no publication was found in 46% of the participating European countries. The percentage is unsurprising as it confirms Altbach’s (2014) findings on the country’s unbalanced higher education research. If Vossensteyn and colleagues’ study had been representative of the 48 EHEA countries, the percentage could be even larger.

Nevertheless, from 2015 to date, several related articles have emerged from some of these countries, addressing the phenomena of students’ persistence (Ferrão and Almeida 2018, 2021; Casanova et al. 2018), dropout (Carreira and Lopes 2021), and access and performance (Ferrão and Almeida 2019b, 2019a; Nagy and Molontay 2021). Such studies are based on empirical evidence referring to only one higher education institution; most of them refer to the first year of studies, and none have addressed the phenomenon of degree completion.

This study aims to fill this gap by addressing the final grade point average (FGA) of degree completion considering a cohort of first-time, full-time undergraduate students who completed their three-year program at the same institution in the theoretical period.

Student success: degree completion

Different intertwined reasons motivate the study of degree completion as the most significant individual, institutional and system outcome in higher education research for policy purposes (Chalmers 2008; Hovdhaugen, Kottmann and Thomas 2015; Johnston et al. 2016). First, low completion rates represent a waste of talent for individuals, as well as a loss for families and society worldwide (Aina 2013; Meggiolaro, Giraldo and Clerici 2017; Konstantinovskiy 2017; Pitman et al. 2017; Ferrão and Almeida 2019a; Araújo, Mariano and Oliveira 2021; Nagy and Molontay 2021). Second, as the debate on issues related to students’ success has moved from being the student’s responsibility to that of the higher education institution (Tinto 2006; Tight 2020), the target of institutional commitment through support, feedback and involvement/engagement programmes should be degree completion (Vossensteyn et al. 2015; Holliman, Martin and Collie 2018) on time. Third, given that the financing of higher education has progressively shifted from the state’s responsibility to that of the students and their families (Marginson 2018; Tight 2020), this calls for a more accountable evaluation of private/public funding and demands more effective social justice policies (e.g. Pitman et al. 2017).

Regarding European countries, we found very few recent studies on degree completion. In Italy, considering the cohort of students who enrolled in a three-year bachelor’s degree in 2013–2014, the rate of degree completion on time was 30.6% (ANVUR—National Agency for the Evaluation of Universities and Research Institutes, (2018, 43); however, the degree completion rates vary according to regions and the type of upper secondary education. Within a 10-year timeframe (e.g. 2006–2007 entering cohort and degree completion up to 2016–2017), 36.9% of the students dropped out. Meggiolaro, Giraldo and Clerici (2017) applied multilevel survival models to data extracted from the University of Padova’s administrative archives and followed up 32,201 freshers in the academic years 2002–2003 to 2005–2006 in 81 first-cycle degree courses. Student data were available for a maximum period of five years and not after December 2009. The results indicated that the students’ sociodemographic factors and pre-university educational experience, as well as the selectivity, area of study and programme size (number of students) influence the profile of a successful student.

Vulperhorst et al. (2018) applied path analysis to data extracted from three cohorts of students enrolled in a university in the Netherlands between 2009 and 2011 and who graduated between 2011 and 2014. Two samples of students were considered a sample size of 377 for the Voortgezet Wetenschappelijk Onderwijs diploma and a sample size of 146 for the International Baccalaureate diploma. Due to their small sizes, other samples were discarded. Their findings suggest that ‘whether high school grade point average or core subject grades are the most valid measure of high school achievement to be used for selection procedures in higher education remains unclear, as the best predictor of academic achievement in university seems to be dependent on the students’ high school diploma’ (411).

Nagy and Molontay (2021) applied statistical models to the data of 21,547 students enrolled in the Budapest University of Technology and Economics between 2010 and 2016 to investigate the relationship between students’ university entrance scores and degree completion. Their findings suggest that prior knowledge plays an important role in degree completion and performance; that is, the current Hungarian centralised entrance score is a valid predictor, varying significantly across disciplines, and high school grades are strong predictors as well.

Regarding the Portuguese higher education, the longitudinal descriptive analyses carried out with the 2011–2012 cohort of 41,797 students enrolled in three-year programmes at public institutions showed that, four years later, 53% of students admitted to their first-choice programme and only 38% admitted to their sixth-choice programme had graduated (Engrácia & Baptista, 2018, 8). Ferrão and Almeida’s (2019a) institutional research explored how a student’s university programme choice mediates the relationship of prior achievement (university entrance score [UES]) on first-year grade point average and whether the relationship has a differential effect depending on the programme of study. Individual sociodemographic variables were included as control variables. The study found that 34% of the variability in first-year grade point average (GPA) was due to differences across programmes and that the UES was the strongest predictor of first-year GPA. It also showed the advantage in first-year GPA that students of disadvantaged sociocultural origins have over their colleagues, even after controlling for the remaining variables. Had such a research been based on empirical evidence related to nationwide students, it would be quite promising to support the European objectives of widening access and reducing socioeconomic inequality. This study also aims to fill this gap by considering the above issue from a nationwide perspective.

The present study

This study has three main objectives. First, it investigates how FGA of students receiving their first degree on due time depends on their access conditions and sociodemographic background, that is, on students’ UESs, admission to their first-choice institution programme pair, gender, age at enrolment, parents’ level of education, working situation, and if the student attends a non-local university. In Portugal, as in other countries, some students choose to move from their family’s place of residence to another place to carry out their studies. Second, it explores the role of social scholarships/financial aid in overcoming the effects of students’ socioeconomic disadvantages. Third, the study quantifies the variance decomposition of the students’ FGA by considering students nested within programmes and faculties and quantifies differentials by area of study and type of institution (university/polytechnic).

This paper expands previous studies, which investigated the relationship between the transition to higher education and students’ success. In particular, it extends the institutional research study (Ferrão and Almeida 2019a) in the following ways: by considering students’ FGA obtained at degree completion rather than the first-year GPA, and empirical evidence obtained by statistical models applied to students’ microdata with national coverage instead of just one university data warehouse. Microdata was extracted from the system information of the Directorate-General of Education and Science Statistics (DGEEC) under a specific protocol regarding privacy data protection (Cordeiro 2019), and information on students’ social scholarship was made available. Such enlargement allows us to address specific objectives: (1) estimating the effect of students’ admission scores on their FGA, controlling for sociodemographic variables, and quantifying its variability across programmes and pedagogical/scientific organic units; (2) estimating the first-option effect on bachelors’ FGA; and (3) estimating the effect of receiving or not receiving a social scholarship based on bachelors’ FGA. Therefore, the study contributes innovatively to the higher education field, specifically to the themes of students’ success (degree completion), equity (financial aid), system evaluation and resource allocation. Since Portugal is one of the European countries where the costs of funding higher education are supported primarily by taxpayers, this topic of research matters not only for public policy regarding the increase of equity but also for the efficiency of public resource allocation.

Few studies on degree completion explore the advantages of the hierarchical nature of higher education microdata with the areas of study and the structural and organisational characteristics of programmes, considering students’ prior achievement and sociodemographic background. To our knowledge, regarding the EHEA, no published nationwide study addresses degree completion by considering an entrant cohort of first-time, full-time undergraduate students who complete their three-year program at the same institution in due time and simultaneously considers students’ background and choices, institutional organisation characteristics and the area of study.

Methods

Procedure

We used microdata made available by the DGEEC under Protocol 5/2020 with the research centre CEMAPRE for data privacy protection. According to the DGEEC, the survey ‘Registo de Alunos Inscritos e Diplomados do Ensino Superior’ [Register of students enrolled in and graduated from higher education] (RAIDES) is carried out within the scope of the National Statistical System which is mandatory (DGEEC—Direção-Geral de Estatísticas da Educação e Ciência 2020). Data are collected by higher education institutions and exported in XML format to the DGEEC twice a year (January and April; December 31 and March 31 as time reference, respectively), through the ‘Plataforma de Recolha de Informação do Ensino Superior’ [Platform of Data Collection in Higher Education] (PRIES). Two types of files were already pseudonymised: a file containing the dataset of students enrolled in a given academic year and a file containing the dataset of the graduates in a given academic year. For this study, we proceeded as follows: (1) datasets for students enrolled in the academic year 2013–2014 and who graduated in 2015–2016 were paired via student code (fkentidade); and (2) those students who were not enrolled in their first year for the first time, whose access to higher education was different from the national competition, or who registered in programmes other than those of the 180 European Credit Transfer and Accumulation System (ECTS) (three-year) were excluded from the analyses.

Variables used for this study purpose concern the following student attributes: the outcome is academic performance measured by the FGA score; the list of independent variables include the UES; first-option admission to the institution and programme (yes/no); gender; age at enrolment (in years); parents’ education (maximum between mother’s and father’s education) as a proxy for students’ socioeconomic status; receiver of social scholarship (yes/no); working student (yes/no); and non-local university (yes/no). The area of study was also included, coded as dummy variables, according to the identification and codification of the formative offers (Ministério das Actividades Económicas e do Trabalho 2005).

Participants

We considered students who entered undergraduate programs of 180 ECTS by the national competition (CFORMAINGRESSO_INSCR = 10) in the 2013–2014 academic year (the share of 70.4% based on valid cases) and who obtained the respective diploma three years later. The number of students involved was 12,966, representing 43% of those who entered by the national competition. In other words, most students (57%) enrolled in a three-year programme did not receive a diploma in due time. Descriptive statistics are presented in Tables 1 and 2 for categorical and continuous variables respectively. Female students represented 65.9%, 25.1% received a social scholarship (Ação Social no Ensino Superior [ASES]), 27.1% moved from their family’s place of residence, less than 2% were declared as working students, 66.3% of the students graduated from their first preference for higher education institution and programme, and 38.1% graduated from a polytechnic institution. The distribution by parents’ education showed that 18.2% of parents obtained at maximum the second cycle of primary education, 18.8% obtained the third cycle, 29.4% the upper secondary education, and 33.6% the tertiary level diploma. The distribution by area of study (CCNAEF_1D_INSCR variable) showed the following in decreasing order: social sciences/business/law (41.3%); arts/humanities (19.6%); services (10.9%); science/mathematics/computing (9.5%); engineering/industry (8.2%); education (5%); health, social protection (4.2%); and agriculture (1.2%).

Table 1. Description of categorical variables.

Variable	Categories	Distribution(%)	Missing (%)	Variable	Categories	Distribution (%)	Missing (%)
Gender			0	Working student			0
	Female	65.9			No	98.4
	Male	34.1			Yes	1.6
Grant			0	Parents’ education			5.3
	No	74.0			None	0.3
ASES (Social scholarship)		25.1			< 2^nd cycle	17.9
	Other	0.9			3^rd cycle	18.8
Non-local university			0		secondary	29.4
	No	72.9			Tertiary	33.6
	Yes	27.1		Admission option			12.6
Area			0.1		1^st	66.3
Education		5.0			2^nd	19.2
Arts, Humanities		19.6			3^rd	8.1
Soc.Sc., Business, Law		41.3			4^th	3.6
Science, Maths, Computing		9.5			5^th	1.8
Engineering, Industry		8.2			6^th	0.9
Agriculture		1.2		Institution			0
Health, Social Protection		4.2		Type	Polytechnic	38.1
Services		10.9			University	61.9

Source: RAIDES 2013/14 paired with RAIDES 2015/16. Own calculation.

Table 2. Description of continuous variables.

Variable	Average	Standard Deviation	Median	Valid cases	Number of missing
University Entrance Score (UES)	142.34	18.11	140.50	12966	0
Final Grade Average (FGA)	14.23	1.34	14.00	12966	0
Age at enrolment	18.83	1.94	18.00	12966	0

Source: RAIDES 2013/14 paired with RAIDES 2015/16. Own calculation.

Table 2 shows: (1) the UESs had a mean 142.34 out of 200 (SD = 18.11) and median of 140.5; (2) the FGA mean was 14.23% out of 20 (SD = 1.34) and a median of 14; and (3) students’ age at enrolment had a mean of 18.83 (SD = 1.94) and a median of 18. Furthermore, the first and third quartiles of age were 18 and 19, respectively, indicating that 25% of the students were older than 19 during the first enrolment. Most students enrolled immediately after completing upper secondary education. Only 0.5% of the students were older than 30 years.

Table 3 presents the descriptive statistics (mean, standard deviation and the number of students) of UES in 2013–2014, and the FGA in 2015–2016 by the area of study. The comparison between relative frequencies in 2013–2014 and 2015–2016 shows that the relative frequency increased in 2015, except for science/mathematics/computing and engineering/industry, which decreased by 1.4% and 4.7%, respectively. Similarly, the degree completion rate varies significantly across the areas of study, from 27% in engineering/industry to 66% in health/social protection.

Table 3. Descriptive statistics of university entrance score and final grade average by areas of study.

	Entrance score 2013/14				Degree completion FGA 2015/16				Rate of degree completion
	Mean	SD	N	%	Mean	SD	N	%
Education	129.66	14.22	994	3.3	14.54	1.17	646	5.0	65
Arts, Humanities	140.52	18.03	5646	18.7	14.57	1.32	2549	19.7	45
Soc.Sc., Business, Law	139.57	18.6	12120	40.1	14.03	1.35	5352	41.3	44
Science, Maths, Computing	141.24	18.81	3307	10.9	14.59	1.46	1231	9.5	37
Engineering, Industry	134.82	15.32	3921	13.0	14.05	1.36	1069	8.2	27
Agriculture	128.55	13.07	339	1.1	13.75	1.22	152	1.2	45
Health, Social Protection	131.47	13.18	822	2.7	14.27	1.2	544	4.2	66
Services	131.52	14.3	3044	10.1	14.06	1.12	1413	10.9	46
Unknown	137.83	15.44	52	0.2	14.9	1.45	10	0.1	19
Total	137.83	17.77	30245	100	14.23	1.34	12966	100	43

For this study, UES and FGA variables were standardised. Missing data reported in Table 1 were subject to multiple imputation likelihood procedures considering students’ background variables, with the exception of missing by design.

Statistical analyses

We applied variance components models and random coefficients models as specified by Ferrão and Almeida (2019a). Such methodological choices are well-grounded in literature on higher education (Liu 2011; Steedle 2012; Kim and Lalancette 2013; Shavelson et al. 2016) and success measurement (Foley and Goldstein 2012; Leckie and Goldstein 2019). For this study, multilevel models were applied considering two hierarchical structures at two levels. Structure I considers students within programmes (level 1 unit coded as original pseudo-identifier ‘fkentidade’; level 2 unit coded as original ‘cod_cursoINSCR’) and Structure II considers students within pedagogical/scientific organic units (level 1 unit coded as original pseudo-identifier ‘fkentidade’; level 2 unit coded as original ‘cod_estabelecimentoINSCR-CUO’). Thus, there were 12,966 students nested in 479 programmes and an equal number in 191 pedagogical/scientific organic units. We used the iterative generalised least squares algorithm for parameter estimation, implemented in the MLwiN (Charlton et al. 2020). In addition, the variance partition coefficient (Goldstein, Browne, and Rasbash 2002), also known as intraclass correlation (ICC) in the sample survey literature, was applied to quantify the extent of clustering.

Results

Final grade average variance decomposition

Considering hierarchical structures I and II, the estimates of the null model and the variance component model with UES in the linear predictor are presented in Table 4. The variance partition coefficient is 0.27 for Structure I and 0.20 for Structure II. Controlling the FGA for UES, the coefficient is 0.22 for Structure I and 0.19 for Structure II. The obtained coefficients confirm the need to apply multilevel models so that the FGA varies across level two units (either programmes or pedagogical/scientific organic units) in a proportion that ranges from 0.19 to 0.27. Furthermore, the UES fixed parameter estimates showed that the UES relationship with FGA was the same (0.44) regardless of the hierarchical structure considered. However, it explained more FGA variability across programmes than pedagogical and scientific organic units. At level two, the coefficient of determination indicated that the students’ UES explains 26% of the FGA variability across programmes and 16% of the FGA variability across organic units.

Table 4. Estimates of the null model and model conditioned on university entrance score.

Variables	Structure I		Structure II
	Null model	With UES	Null model	With UES
	Coefficient (SE)	Coefficient (SE)	Coefficient (SE)	Coefficient (SE)
Fixed component
Intercept	−0.006 (0.020)	−0.001 (0.021)	−0.02 (0.034)	0.002 (0.035)
z-University entrance score	—	0.437 (0.009)	—	0.435 (0.015)
Random component
Level 1: intercept	0.735 (0.001)	0.703 (0.008)	0.806 (0.011)	0.719 (0.009)
Level 2: intercept	0.265 (0.017)	0.195 (0.016)	0.204 (0.022)	0.171 (0.024)
Total	1	0.898	1	0.890
Variance partition coefficient	0.27	0.22	0.20	0.19

Several random coefficients models were adjusted considering different sets of variables with and without second-order interactions between variables. Table 5 presents the estimates of the fixed and random parameters for the final models, that is, those statistically significant at the 5% level.

Table 5. Estimates of the FGA model parameters.

Variable	Structure I				Structure II
	Coeff.	S.E.		z stat	Coeff.	S.E.	z stat
Fixed components
Intercept	−0.478	0.114	−4.19		−0.010	0.116	−0.09
z-UES	0.648	0.088	7.36		0.623	0.084	7.42
1^st choice	0.055	0.019	2.89		0.056	0.023	2.43
ASES grant	0.001	0.022	0.05		—	—	—
Female	0.055	0.019	2.89		0.058	0.018	3.22
Age	0.022	0.005	4.4		0.020	0.005	4.00
Parents’educ
< =2^nd cycle	0.011	0.027	0.41		0.058	0.021	2.76
>2^nd cycle	—	—	—		—	—	—
Areas of study
Education	—	—	—		—	—	—
Arts, Humanities	0.269	0.072	−3.74		0.341	0.058	−5.88
Soc.Sc., Business, Law	−0.176	0.070	−2.51		−0.543	0.048	−11.31
Science, Maths, Comp	−0.063	0.105	−0.60		−0.422	0.070	−6.03
Engineering, Industry	−0.151	0.043	−3.51		−0.451	0.060	−7.52
Agriculture	—	—	—		−0.704	0.100	−7.04
Health, Social Protection	—	—	—		−0.199	0.059	−3.37
Services	—	—	—		−0.409	0.068	−6.01
Type of Institution	—	—	—
Polytechnic	—	—	—		—	—	—
University	−0.169	0.031	−5.45		−0.213	0.069	−3.09
Interactions
Age X UES	−0.012	0.004	−3.00		−0.009	0.004	−2.25
ASES grant X Parents’educ.< =2^nd cycle	−0.742	0.296	−2.77		—	—	—
1^st choice X Arts, Humanities	—	—	—		0.098	0.047	2.09
1^st choice X Sci, Maths, Computing	0.176	0.06	2.93		0.135	0.061	2.21
Random components
Level 1: intercept	0.678	0.009			0.699	0.010
Level 2: intercept	0.159	0.016			0.117	0.018
Level 2: slope z-UES	0.027	0.006			0.018	0.005
Level 2: corr(intercept, z-UES)	−0.271				−0.045
Deviance (two df)	89.81				51.75

The relationship between UES and FGA

When all variables and second-order interactions were considered, the marginal effect of UES on FGA increased to 0.648 (Structure I) and 0.623 (Structure II), with an additive negative interaction term with age (−0.009, −0.012), suggesting that older students with higher entrance scores tend to achieve slightly lower FGA degree scores than their younger peers.

The random components at level two allow the coefficient to vary across courses or organic units. The change in deviance suggests that both random components are significant. The deviance test sets that under the null hypothesis, the extra two random parameters (variance and covariance) have population values of zero. Given the chi-square distribution on two degrees of freedom at the 5% significance level, the null hypothesis is rejected and the alternative accepted, meaning that the relationship between UES and FGA varies randomly across programmes and organic units.

The association between students’ choice and FGA

The estimates suggest that students’ admission to their first preference positively influences the degree final score in several ways. Regardless of the hierarchical structure, the estimate of the fixed parameter is 0.06. Furthermore, for the areas of arts/humanities and science/mathematics/computing, the positive interaction effect suggests that the degree final score increases when students are admitted to the programmes classified in such areas in their first preference.

Differentials of academic performance by area of study, programme or scientific/pedagogical organic unit, and type of institution

The FGA varies randomly across programmes and organic units. The estimates also suggest a fixed effect for each area of study, with education as the baseline. The final model indicates arts/humanities as the areas where students achieved higher final scores after controlling for the remaining variables. The interpretation of the results for Structure II calls for additional details in the area of study classification to better understand the results regarding the organisation by faculties and departments. For example, social science, business and law programmes are usually organised in different departments and/or organic units. The results obtained for both hierarchical structures are consistent with each other. Both indicated that, on average, graduates from universities have lower FGA than graduates from polytechnic institutions (−0.169, SE = 0.031; −0.213, SE = 0.069).

The association between background, social scholarship and FGA

The results in Table 5 show that receiving a social scholarship (ASES) does not influence students’ degree completion rating. The Structure I model estimates suggest that students whose parents’ education is at the maximum second cycle of primary education are socially disadvantaged, and who had been eligible for an ASES grant finished their degree on time with a reduced FGA (−0.742; S.E. = 0.296). According to the Structure II model estimates, graduates whose parents’ education is at the maximum second cycle of primary education, on average, achieve slightly higher scores (0.058; S.E.=0.021) than those whose parents’ education is greater than the second cycle of primary education.

Female graduates achieved, on average, 0.06 points more than male graduates. The estimate of the age coefficient is positive and almost negligible (0.02; S.E.=0.005).

We found no association between FGA and other individual attributes, such as moving from family’s place of residence to conduct higher education studies or working status, having tested the main effect of second-order interactions with the remaining variables.

Discussion

In this study, we quantified the relationship of students’ UESs on degree completion grade average for three-year programmes. We considered a cohort of students who graduated on time from Portuguese higher education, equating such relationships for students’ sociodemographic background and preference regarding the institution and programme attended, and by considering the area of study and type of institution. We applied variance components models and random coefficients models to microdata, referring to a nationwide cohort of students enrolled for the first time in 2013–2014 and who graduated in 2015–16. Two hierarchical structures were considered: students nested in programmes and in pedagogical/scientific organic units or faculties.

The estimates indicate that high school preparation is the most important predictive factor for FGA among the remaining variables at enrolment, confirming previous research on degree completion (Clerici, Giraldo and Meggiolaro 2015; Meggiolaro, Giraldo and Clerici 2017; Nagy and Molontay 2021). On average, the final score increases by 0.6 standard deviations per additional standard deviation of the UES. Its magnitude is larger than the estimate obtained between UES and first-year GPA (Ferrão and Almeida 2019a), reinforcing the role of prior achievement in academic performance (Fuller, Wilson and Tobin 2011; Clerici, Giraldo and Meggiolaro 2015; Nagy and Molontay 2021), particularly to degree completion on due time.

Moreover, the relationship varies randomly across programmes and organic units, suggesting that the UES strength on FGA depends more on the program itself than on the organic unit to which the course belongs. Accordingly, estimates of the variance partition coefficient also suggest that the FGA variability due to differences among programmes is larger than the variability among organic units. The coefficient of determination carried out at level two suggests that the UES explains more FGA variability across programmes per se than across pedagogical/scientific organic units. Our findings also show that, considering the students’ sociodemographic background and social scholarship granted, the degree of completion of their first preference, and differentials by area of study and type of institution, the FGA unexplained variance is larger across programmes than across scientific/organic units. In other words, despite the relevance of prior achievement and admission conditions for degree completion, there is a great deal of room for improvement from students’ admission to degree conclusion. Drawing from the results, measures aimed at academic performance improvement should be mainly taken and leveraged in bottom-up actions focusing on programmes instead of scientific/pedagogical organic units.

Our results suggest that female students achieve higher FGA than male students, and that a negative interaction between age and UES on FGA can be observed, which may be evidence that students who had enrolled immediately after finishing high school are more likely to succeed (Clerici, Giraldo and Meggiolaro 2015).

The model with organic units as level two suggests that socially disadvantaged students have a slightly increased FGA compared to their peers, following descriptive analyses conducted by Engrácia and Baptista (2018) and in line with modelling estimates of the first-year GPA study (Ferrão and Almeida 2019a). However, the model with programme as the level two unit did not yield statistically significant results. Furthermore, it suggested that the minority group of students whose parents’ education is not higher than the second cycle of primary education and who receive a social scholarship demonstrate lower FGA compared to their peers.

Our analyses confirm the differences of FGA based on areas of study reported in the literature (Werner 2009; Clerici, Giraldo and Meggiolaro 2015; Ferrão and Almeida 2019a; Nagy and Molontay 2021). With education as the baseline, the estimates for fixed parameters depend on the hierarchical structure considered. Students who were admitted based on their first preference in a programme classified in science/mathematics/computing showed increased FGA. The result is promising for further research since this area of study has the second lowest rate of degree completion on time. FGA tends to be higher in polytechnic institutions than in universities.

The main limitation of this study is the number of programmes per organic unit that prevents the study from using a three-level model (student, programmes, organic unit), which could provide more powerful insights into undergraduate academic performance. To overcome this problem, a wider study that simultaneously accommodates several cohorts of entrant-concluding students should be planned. Another limitation of our analyses was the classification of the areas of study. The current classification aggregates areas that belong to different organisational units, which may have undermined the results presented for the model that was hierarchically structured by organic units. It would be valuable for future research to consider a more detailed classification by disaggregating the study areas.

Funding

This research was supported in part by the project CEMAPRE/REM—UIDB/05069/2020 FCT/MCTES from the Foundation for Science and Technology through national funds. RAIDES files are protected by the GDPR. The consent for their use in the data safe center in Lisbon may be obtained by any researcher (https://www.dgeec.mec.pt/np4/contactos/).

Acknowledgements

I would like to thank Dr. Joana Duarte and Dr. Catarina Afflalo for the use of DGEEC data safe centre.