Do work placements improve final year academic performance or do high-calibre students choose to do work placements?

Abstract

This study investigates whether the completion of an optional sandwich work placement enhances student performance in final year examinations. Using Propensity Score Matching, our analysis departs from the literature by controlling for self-selection. Previous studies may have overestimated the impact of sandwich work placements on performance because it might be the case that high-calibre students choose to go on placement. Our results, utilising a large student data set, indicate that self-selection is present, but the effects of a placement on student performance still have an impact. This robust finding is found to be of a remarkably similar magnitude across two UK universities.

Keywords:

• British higher education
• degree performance
• work placement
• self-selection
• propensity score matching

Introduction

In the UK the recent Wilson Review (2012) reaffirms the perceptions of educationalists and employers with regard to the importance of a placement (internship) period to the student learning experience. Yet the White Paper (2011), Higher Education: Students at the Heart of the System, from which the Wilson Review (2012) was commissioned notes that there has been a decline in the percentage of undergraduate students taking a so-called sandwich year – a year-long integrated period of work experience which is undertaken by students at many UK universities as part of their degree (often between the second and final year). In 2002/2003, 8.2% of students completed a sandwich degree, whereas in 2012/2013 the comparative figure was 5%.¹ The White Paper (2011) attributed this to employers investing fewer resources in ‘creating good placements’ and students feeling that ‘the extra year of study was not producing enough added benefit’ (41). These facts are disappointing given that placements allow employers to trial a potential employee, as well as gain a direct link to the university research that applies to their sector (Department for Business Innovation & Skills [BIS] 2012).

In an international context, Arthur and Little (2010) report that in 1999/2000, 55% of all European graduates had undertaken some form of internship period, with over 80% in Finland, Germany and the Netherlands. The apparent UK student perception that the taking of a sandwich year has little value-added has been one factor which has acted as a catalyst for pedagogic researchers to investigate a number of possibly interlinked issues, namely, does the completion of a placement year make students more employable and does it result in higher degree classifications? Although the empirical evidence to date has been somewhat disjointed with individual studies focusing upon specific courses at specific universities and subject disciplines (Mansfield 2011; Moores and Reddy 2011) and is subject to critical review (Duignan 2002), the evidence presented overwhelmingly supports the contention that the completion of a work placement year is associated with enhanced employment. Indeed a follow-up report of the Wilson Review by the Department for BIS (2012) using HESA 2009/2010 data states that the average salary of students who have completed a sandwich placement is 8% higher than those who did not 6 months after graduating.

In terms of academic performance, there is currently quite a substantive and growing body of work which suggests that the taking of a work sandwich placement (or internship) year is associated with better final year degree performance (Surridge 2008; Green 2011; Mansfield 2011). It is certainly our experience at Aston University and the Ulster University that when students return from work placements they appear to be more driven to succeed. Students attend classes more regularly, engage more with the lecturer both during seminars and in one-to-one sessions, and have more confidence to express themselves in front of their peers. At face value therefore, it would seem that students who go on sandwich placements must surely improve their academic performance relative to their counterparts who choose not to go on placement.

This study investigates this issue using student record data from two institutions in the UK. Both Aston University and the Ulster University have a history of offering degree courses that have a fully integrated work placement sandwiched between the second and final year of the degree. Indeed, the sample size we utilise for Aston is, as far as we know, larger than that in any other existing studies – 6645 students are included, all of whom had the option to go on a work placement or not. This study offers a contribution in one important dimension – it specifically controls for self-selection bias using a technique not currently adopted in the literature. As we go on to discuss below, previous studies have often found that work placements improve student performance in final year examinations. However, self-selection in this context may arise because it might be the case that ‘high-calibre'² students who go on work placements would have performed at a high standard regardless of the placement. Thus, previous studies may report upwardly biased estimates of the placement effect.

The methodology we use is called Propensity Score Matching and it is essentially a statistical matching technique that attempts to estimate the effect of a treatment, policy or other intervention – in this case, the impact of a work placement on final year degree performance. By using Propensity Score Matching techniques, we are able to construct a reliable control group of students (those who choose not to go on placement) to compare with a treatment group of students (those who choose to go on placement) to control for self-selection in order to ascertain the strength of the evidence for a placement effect. Our findings demonstrate that for both academic institutions there is evidence of self-selection. Nonetheless, we find that the impact of work sandwich placements on student performance is still positive. Remarkably, this result is robust across both institutions.

The remainder of this paper is set out as follows. The next section critically reviews the existing literature that looks at the impact of placements on student performance. We discuss in more detail here the issue of self-selection. Following this, we discuss the Propensity Score Matching methodology and show how we estimate the Average Treatment Effect of the Treated (ATET) (i.e. the effect of work placements on student performance). We then go on to discuss the data and report the results. Finally, in the conclusion we discuss the importance of our results in the context of the enhancement of employability skills in Higher Education. We summarise our findings, acknowledge certain limitations and suggest avenues for future research.

Literature review

Work placement or internship is a characteristic of contemporary higher education which is focussed towards improving the development of students’ employability and transferable skills. The National Council for Industry and Commerce characterised the sandwich principle as being founded upon ‘an interaction of academic study and practical applications such that each serves to illuminate and stimulate the other’ (Brennan and Little 1996, 4). Harmer (2009) suggests that learning and teaching would be enhanced for all stakeholders if every student experienced the real-life complexities of organisations as part of their degree. Many previous studies look at how effective such placements are in developing career-related competencies (e.g. Murakami et al. 2009). Auburn (2007) looks at the skills acquired on placement in relation to students’ final year studies once they return to complete their degree. He identifies a range of knowledge areas, skills or values acquired from the placement year which can be deployed in the final year.

Over the years there have been a number of attempts to gauge the effectiveness of work placements (e.g. Duignan 2002, 2003; Morse 2006; Lavinal, Decure, and Blois 2007; Bennett, Eagle, and Mousley 2008; HEFCE 2009). Little and Harvey (2007) investigate the student perspective. They find that the majority of undergraduates included in their study indicate that they gained personal and intellectual development and report increased levels of confidence and enhanced motivation towards study. Furthermore, Blackwell et al. (2001) seek to assess the effect of placements on both academic performance and employment rates. Their conclusion is that placements are more likely to be successful where the higher education institution consistently encourages students to reflect on their learning. This once again explores the link between university studies and placements, as well as hinting at the extra gains which effective learning brings. Ellis (2000), Huntington, Stephen, and Oldfield (1999) and Webber (2005) also stress the need for the careful management of work placement programmes, including the preparation of students prior to placement, which may relate to what marks out the ‘better’ students.

In contrast, Schaafsma (1996) and Leslie (1999) take a critical view of the claims made for the added value of placements. The former examines work placement as a site for ‘contested learnings’ and suggests that work-based trainers and educators need to make use of these learnings to ensure they add value to students’ understanding of their work placements in context. The latter also makes proposals to address his view that the benefits often attributed to work experience are not always realised. Furthermore, Bourner and Ellerker (1998) identify that the main solution to ensuring the effectiveness of the work placement is to guarantee as much integration as possible between the placement and the student's academic studies.

The impact of placements on student performance

There are a number of studies which use statistical methodologies to look at the impact of work placements on student performance. Duignan (2002) concludes that: ‘No significant difference was found between those who undertook a placement and those who did not’ (214). His survey is a statistical analysis of examination results, relating to two cohorts of business undergraduates. One of the main issues with the Duignan (2002) study is that it does not control for the essential problem in these types of studies, which is whether there is some form of student predisposition for deciding to take a placement year and if there is, how does one control for this. In addition, Duignan (2002) further acknowledges that several factors such as family illness, bereavement and income uncertainty were not captured within his data set, but clearly other factors that Duignan (2002) does not acknowledge are also important to consider such as socio-economic background and prior schooling.

Other literature suggests that a work placement does improve performance. Gomez, Lush, and Clements (2004) anecdotally state that ‘students on their optional Bioscience sandwich degrees benefit academically from a placement experience’ (373). They then go on to investigate this using multivariate regression analysis, and conclude that ‘students taking a sandwich placement exhibit improved academic performance in the final year – on average, placement students gain an advantage of 4%’ (378). Surridge (2008) also using regression analysis for a cohort of Accounting and Finance students find an average improvement of 3.6%. Furthermore, Mandilaras (2004) surveyed economics students studying on a degree with an optional placement year. The results reported suggest that participation in the placement scheme significantly increased the chances of obtaining an upper-second-class degree by 30 percentage points.

Likewise, Mansfield (2011) investigates the issue on students studying a Surveying degree over five cohorts using analysis of covariance and concludes that a statistically significant improvement in final year marks is observed for students who completed a placement year, with an average increase of 3.46 marks observed in the final year. Furthermore, Green (2011) investigates a single cohort of students studying Business Studies using both parametric and non-parametric tests and concludes that the mean mark for students who go on placement is 61.19% compared to a mean mark of 56.96% for non-placement students. Finally, Crawford and Wang (2014) find that work placements increase student performance for both UK and international students, but the impact on the former is greater than that on the latter. Interestingly, Crawford and Wang present evidence of the presence of self-selection among UK students.

Self-selection

The critical issue faced in the above studies is that they do not adequately control for self-selection, in that it is the high-calibre students who choose to do work placements such that if they had not chosen this option they would have performed well regardless. In a follow-up study, Duignan (2003) acknowledges this problem and states:

the issue of a relationship between placement and academic performance was also found to be more complex than that suggested by the underlying hypothesis: there was some evidence indicating that the opportunity for placement may lead to self-selection: those who chose to undertake placement tended to be more academically capable than their non-placement peers. (345)

In order to control for self-selection using regression analysis, previous studies (Green 2011; Crawford and Wang 2014) have integrated controls that proxy for a student's ability. These include A-level scores, A-level tariffs or prior performance in the first or second year of the degree programme studied. Although these controls go some way to addressing the self-selection issue, it is questionable as to whether they really do measure a student's underlying ability. Furthermore, by including these controls in a regression specification and at the same time including the placement dummy, endogeneity may result. Endogeneity can arise as a result of measurement error whereby two independent variables are correlated with the unobserved error term. This potentially leads to biased estimates and unreliable standard errors.

Perhaps the only paper that has attempted to specifically control for self-selection econometrically is that of Driffield, Foster, and Higson (2011) using data for Business students at Aston University. These authors use a standard Heckman (1979) model that in the first stage estimates a probability model (Probit) to determine the likelihood of a student undertaking a placement. From this stage a selection parameter is estimated (the Mills ratio – see Greene 2007) and integrated into a second stage model that estimates student performance. Driffield, Foster, and Higson (2011) find clear evidence of self-selection because the selection parameter is found to be biased. Indeed, the overall findings suggest that the impact of placements on student performance is minimal. However, there is one major issue with the Driffield, Foster, and Higson (2011) study and this is linked to the instrument used to identify the placement effect. An instrument is a variable that impacts upon the placement choice (modelled in the first stage) but does not impact upon student performance (modelled in the second stage). Driffield, Foster, and Higson (2011) choose socio-economic class, proxied by parental background, as the instrument of choice, but offer no theoretical justification as to why this is appropriate.

The current study, using data from a similar population to that of Driffield, Foster, and Higson (2011), overcomes self-selection by using Propensity Score Matching. This methodology is useful because there is no need to construct a valid instrument. All it relies upon is a matching technique that allows a justifiable comparison between those students who choose to go on placement and those students who choose not to. As far as we know, this is the first study of its kind to adopt this approach.

Methodology

In order to estimate the impact of a work placement on final year performance, we assume that there is a treatment indicator variable for each student i, that takes the value 1 if a student has been on placement (is treated), or takes the value 0 if the student has not been on placement (is not treated). We also assume that there is an observed outcome variable (the student's average mark for finals) that takes the value y _i1 if the student has been on placement, or takes the value if the student has not been on placement. A simple estimator to use is represented by Equation (1). This compares the difference in outcome (performance in finals) for the treated students (those who chose to go on placement) from that of the non-treated students (those who chose not to go on placement).(1) here, the first term on the right-hand side is the Average Treatment Effect of the Treated (ATET) and the second term is the bias. The source of this bias is most likely due to a correlation that exists between students who choose to go on placement and a number of unobserved characteristics that effect student performance. In this context this revolves around self-selection, in that it is the ‘high-calibre’ students who choose to go on placement. In other words, these students would have performed well regardless of going on a placement.

An OLS specification (reported in Table 3) that regresses final year performance on a constant and a placement dummy suffices for this simple approach and the interpretation is straightforward: it shows the percentage point difference in final year performance between students who choose to go on placement and those who do not. An approach that adopts this methodology might mistakenly come to the conclusion that work placements improve student performance when the improvement in performance might have been caused by other factors.

In order to overcome the self-selection issue, a research design that randomly assigns students to treatment (going on placement) and non-treatment (not going on placement) would eliminate the potential upward bias in the approach outlined above and provide a reliable set of estimates. In this context however, this is clearly not feasible for ethical reasons as it would mean deterring a subset of students at the outset from going on a work placement. It is therefore very important to emphasise that all students included in this study, from both Aston and Ulster, were given the option to go on a work placement.

Consequently, what we are principally interested in is constructing an estimator that takes account of the bias. This means attempting to estimate the first part of the right-hand side of Equation (1), which is the ATET. The ATET is the difference between the outcome of the treated (i.e. the final year performance of students who go on placement) and the outcome of the treated students if they had not been treated (did not go on a work placement). Clearly, this is impossible to construct because there is no way of observing the counterfactual. It therefore needs to be estimated.

A natural approach that tries to mimic a randomised experiment is to focus on controlling for confounded variables (see Rubin 1974). A confounded variable is a variable that affects at the same time the decision to participate in treatment (i.e. go on placement) and the potential outcome (i.e. final year performance). By finding groups of students who have comparable characteristics but differ in the treatment, it is possible to attribute the difference in outcome between the groups of students as being attributable purely to the treatment effect, thus mitigating the potential for bias. In order to handle this problem, a matching algorithm is needed.

The method used in this paper is called Propensity Score Matching and is commonly used in the literature concerned with labour market policy evaluation (see Sianesi 2004; Lechner and Wunsch 2009). On the basis that it is possible to observe the confounding variables x, then the probability to participate in the treatment can be estimated (the propensity score), and it is then possible to match students on the propensity score and not the vector x. Hence, the method tries to match every student who goes on placement with a student who did not go on placement, but both have very similar or even identical propensity scores. The propensity scores estimated are simply those obtained from running the following logit model as shown by the following equation:(2)

where, Placement_i is the treatment which takes the value 1 if student i has been on placement or takes the value 0 if student i has not been on placement; Stage1_i is student i’s average first year mark; Stage2_i is student i’s average second year mark; Gender _i equals 1 if student i is female and zero otherwise; is student i’s age; School _i is equal to 1 if student i was educated at a Grammar School prior to joining the university and zero otherwise; is equal to 1 if student i has a parent who is identified as working in a Higher Managerial and Professional Occupation³ and 0 otherwise; equals 1 if student i is a home fee-paying student⁴ and 0 otherwise and finally, is a standard error term.

Once the propensity scores (predicted probability for each student of choosing to go on placement) are obtained, we use nearest neighbourhood matching with replacement and controlling for common support⁵ to estimate the ATET. This gives an unbiased estimate of the impact, in percentage point terms, of a student undertaking a work placement on final year performance. The procedure is relatively straightforward. All that is needed is a comparison between the mean average final year performance of the treatment group (those who went on placement) and the mean average performance in finals of the control group (constructed from the propensity scores).⁶ Not only do we calculate the ATET for Aston University students, but as a robustness test we also calculate the ATET for students from the Ulster University. Finally, we also run models across each of Aston's five schools to determine whether the impact of a placement upon performance differs depending on discipline.

Labour market factors

One potential limitation of the foregoing analysis is concerned with whether the decision to go on placement is impacted upon by the state of the labour market. It could quite conceivably be argued that it is not the choice of the student as to whether or not to go on a placement, but instead the choice of the employer. Clearly if the state of the labour market is not conducive to students, that is, there are not enough placements available for students to go on, then the argument that students have a choice has no practical relevance. Crawford and Wang (2014, 19) state that ‘it is possible that self-selection among placement students is caused by the fluctuating demand and supply relationship in the placement market’. However, there is no evidence offered to back up this claim and no other existing studies have accounted for the impact of labour market dynamics on student placement choice. Indeed, this would be incredibly difficult to control for given the fact that the placement market is of a global nature.

In 2013–2014, Aston University advertised 6022 UK-based roles and 1297 international roles. In 2013–2014, in total aggregate, 1260 students went on work placement, of which 20.7% went overseas. Clearly, therefore, there is an excess supply of placements available to Aston students. The comparable figure for 2011–2012 (the final year of our sample) was 1203 students, notwithstanding the fact that many students arrange their own placements with an employer of their choice.

Needless to say, these figures certainly do not rule out the possibility that some ‘high-calibre’ students may have searched for a placement and were not successful. Given this possibility, and the fact that we do not have the data to account for it, the effect on our estimator is unlikely to be severe. Students of this nature would be mistakenly included in the control group of students who did not go on placement. This would have the potential to create an upward bias in the mean value of final year performance for the control group, thus narrowing the performance gap between the mean score for placement versus non-placement students.⁷ Given this fact, the placement effects we estimate would in all likelihood be higher if it is the case that some ‘high-calibre’ students are mistakenly added to the control group.

Data

This study uses student record data on graduates from Aston University (UK) and the Ulster University (Northern Ireland). It is important to note that every student in this study had a choice as to whether to go on work placement or not. The Aston data set is much more comprehensive than any other study that has looked into the effects of a work placement on final year student performance. It includes 6645 students who graduated over the period 2004–2011.⁸ The coverage of students is taken right across the University's five schools: (1) Aston Business School; (2) Engineering & Applied Sciences; (3) Languages and Social Sciences; (4) Life & Health Sciences and (5) Interdisciplinary Studies. A correlation matrix for each institution is included in Appendix and Table 1 provides detailed descriptive statistics for Aston students.

As can be seen in Table 1, 38% of the 6645 Aston graduates included in the sample decided to go on a work placement between their second and final year of study. Overall, the students averaged a mark of 54.04% during the first year of study and 57.12% from the second year in comparison to a higher average obtained during finals of 59.63%. It is important to note that the first year does not formally count to a student's final degree classification; it is, however, important because it may impact upon the quality of work placement obtained. The gender split is as expected, as is the average age on entry at 19.23 years. Overall, 76% of the sample can be classified as home fee-paying students, with just 6% of them educated in a Grammar School. In terms of socio-economic class, 6% of students have parents in the Higher Managerial and Professional class.

Table 1. Descriptive statistics for Aston university students.

Variable	Observations	Mean	SD.
Placement Dummy	6645	0.38	0.49
Stage 1 Average	6645	54.04	15.09
Stage 2 Average	6645	57.12	7.41
Final Stage Average Mark	6645	59.63	7.79
Female	6645	0.49	0.50
Home	6645	0.76	0.42
Grammar	6645	0.06	0.24
Age	6645	19.23	2.37
Higher Managerial Class	6645	0.06	0.25

 Table 2 provides mean values for each variable for each of Aston's five schools. As can be seen, the largest cohort is from Engineering and Applied Sciences with 2632 students compared to the smallest cohort from Languages and Social Sciences of 287. The number of students who decide to go on placement varies per school. For Life and Health Sciences, almost half of the students go on placement in contrast to only 23% of students from Aston Business School. However, it is important to note that students included from Aston Business School are all overseas students. This is because it is compulsory for all Home and EU students to go on placement; thus, student records for these students cannot be included in the analysis. In terms of year averages, performance improves as the students’ progress. In terms of gender, it is notable that Engineering and Applied Sciences have a greater number of males to females, but this is the reverse for all of the other schools. The average age is consistent across the sample; likewise, the proportion of students forms the highest socio-economic class.

Table 2. Descriptive statistics for Aston's schools.

Aston University Schools/Variables	Aston Business School	Engineering & Applied Sciences	Languages & Social Sciences	Life & Health Sciences	Interdisciplinary Studies
Number of Students	746	2632	287	1381	1599
Placement Dummy	0.23	0.34	0.29	0.48	0.45
Stage 1 Average	54.98	54.1	55.99	49.28	57.23
Stage 2 Average	57.37	55.71	58.86	58.74	57.59
Final Stage Average	57.11	58.61	62.79	61.03	60.67
Female	0.54	0.24	0.7	0.8	0.57
Home	0.00	0.81	0.83	0.93	0.85
Grammar	0.00	0.04	0.11	0.09	0.06
Age	19.65	19.29	19.47	19.28	18.82
Higher Managerial Class	0.02	0.06	0.08	0.08	0.06

The data set for the Ulster University comprises 737 Business School students who graduated in either 2009 (409 students) or 2010 (329 students). Although considerably smaller than the Aston data set, it is still large relative to other studies that have considered this issue.

With regard to the Ulster University, Table 3 shows that 66% of the 737 graduates included in the sample went on work placement in between their second and final year of study. This is significantly higher than the Aston cohort. The students’ average first year performance stood at 58.76%, slightly higher than the second year but lower than the final year performance average of 61.16%. All of Ulster's students are classified as home fee-paying students and it is notable that the amount of females significantly outnumbers males. The age on entry of 18.48 years is as expected, whereas the proportion of students who went to Grammar School is significantly higher than the Aston cohort at 36%. In terms of socio-economic status, the Ulster data mirrors the Aston cohort, with 7% of students having a parent from the highest Managerial and Professional class.

Table 3. Descriptive statistics for the Ulster University students.

Variable	Observations	Mean	SD
Placement Dummy	737	0.66	0.47
Stage 1 Average	737	58.76	7.72
Stage 2 Average	737	57.93	7.02
Final Stage Average Mark	737	61.16	7.04
Female	737	0.69	0.46
Grammar	737	0.36	0.48
Age	737	18.48	0.94
Higher Managerial Class	737	0.07	0.26

Empirical results and analysis

The most logical first step in the empirical analysis that deals with analysing the impact of a work placement on student performance is to run the simple estimator as stated in Equation (2). This turns out to be a regression of the placement dummy on the final year average mark. Essentially, it provides an estimate that is akin to running a simple t-statistic comparing the means of students who choose to go on a work placement with that of those who choose not to. The results of this exercise are reported in Table 4 for both the Aston and Ulster samples. As can be seen, the average final year mark for students who choose not to go on work placement (the constant in the regression) is 57.59 and 57.44, respectively, for Aston and Ulster graduates, whilst the average mark for those students who did choose to go on work placement is 5.308 and 5.635 percentage points higher for Aston and Ulster graduates, respectively, and this is significant at the 1% level. Table 4 portrays the difference between those who did and who did not go on work placement, without taking into consideration any other variables that might affect the probability of going on work placement and at the same time might impact upon final year performance – for this is potentially a source of significant upward bias in the reported estimates.

Table 4. Simple estimator of the placement effect on mean final marks.

University	Observations	Variable	Coefficient	Std error	p-value	R²
Aston	6645	Placement	5.308	0.182	0.000	0.110
		Constant	57.594	0.119	0.000
Ulster	737	Placement	5.635	0.559	0.000	0.144
		Constant	57.440	0.494	0.000

To correct for the potentially biased estimates due to self-selection, we use Propensity Score Matching. This allows us to tease out the causal effect of going on work placement on final year degree performance – in other words, the ATET. In order to implement this technique, we begin by running a Logistic regression that estimates the probability of a specific student going on work placement. The conditional Logit distribution estimator is shown in the following equation:(3)

where are the explanatory variables outlined above in Equation (2) and α and are parameters that need to be estimated. The error terms are assumed to be distributed according to a standard logistic distribution. The results from the Logistic regression are presented in Table 5 for both the Aston sample and the Ulster sample. The χ² test indicates that all of the explanatory variables are statistically significant at the 1% level for both samples.

Table 5. Logistic regression.

Dependent Variable: Placement	Aston			Ulster
Explanatory Variables	Coefficient	std. error	p-value	Coefficient	std. Error	p-value
Stage 1 Average	−0.00111	0.00197	0.574	−0.0269	0.0164	0.101
Stage 2 Average	0.0579	0.0040	0.000	0.0885	0.0181	0.000
Female	0.0485	0.0537	0.366	0.162	0.189	0.392
Home^a	0.767	0.0716	0.000
Grammar	0.110	0.1070	0.303	0.111	0.173	0.522
Age	−0.174	0.0223	0.000	1.129	0.209	0.000
Higher Managerial Class	0.281	0.1050	0.007	0.368	0.303	0.224
Number of Observations	6645			737
Log Likelihood	−4421.367			−401.108
LR χ²	600.462			141.899
Prob > χ²	0.000			0.000

^aAll of Ulster's students are home fee-paying students.

For logistic models the interpretation of the coefficient estimates is straightforward. The estimate is the expected change in the log odds of being on placement for a unit increase in the corresponding predictor variable. By taking the exponential of the coefficient, it is possible to calculate the change in odds in the multiplicative scale for a unit increase in the corresponding predictor variable holding the other variables at their means. An example will make this clearer. The coefficient estimate for home fee-paying students using the Aston sample is 0.767 and is statistically significant at the 1% level (p-value = .001); qualitatively the positive sign attributed to the estimate suggests that home fee-paying students have a greater probability of going on a work placement than Overseas students. By taking the exponential of the coefficient, we can be more precise and say that the odds of a British student going on a work placement over the odds of an overseas student going on a work placement is exp(.767) = 2.15. In terms of percentage change, we can therefore say that the odds for British students going on placement are 115% higher than the odds for overseas students.

In terms of the other coefficient estimates, they tend to be as one would expect. On a qualitative basis, students with a higher Stage 2 Average are more likely to go on placement – this is indicative of self-selection in that high-calibre students choose the work placement option. This is true not only for Aston but also for Ulster students and the reported estimates are both highly statistically significant. The estimates for the Stage 1 Average are statistically insignificant. Interestingly, females are more likely to go on a work placement at both institutions, but the percentage difference in odds is relatively mild. It would appear that prior schooling at a Grammar School has no impact on the probability of choosing to go on a work placement, whereas students with parents from a higher socio-economic class are more likely to choose to go on a work placement at Aston. Finally, the only coefficient that varies across institutions is that of the age variable, which is negative for Aston and positive for Ulster. On face value the latter is surprising as you would expect older students to be more cautious of doing a work placement because they are more likely to have had experience in the labour market – this is what the coefficient for the Aston cohort suggests. Perhaps the reason why the coefficient is different for Ulster is that there is little heterogeneity in the Ulster cohort – the majority of students are aged 18–19 years on entry; there are thus no, what might be termed, ‘mature’ students in the Ulster sample which can be defined as students older than 21 years.

Having estimated the Logistic regression, we can now compute the propensity score (probability) of a student choosing to go on work placement for every student. This allows us to implement the nearest neighbourhood matching algorithm so that we can calculate the ATET. The matching procedure iterates as follows: (1) finds for each student in the treatment group (students who choose to go on placement) a student from the control group (a student who has chosen not to go on placement) with an identical or very similar propensity score; (2) takes the difference between their final year marks; (3) repeats this for all of the individuals in the treatment group (those students who have chosen to go on placement) and (4) calculates a weighted average of the difference in marks for all of the student pairs identified. The results for the ATET are shown in Table 6 for the Aston and Ulster samples using matching with replacement and imposing common support.

Table 6. ATET of students choosing to go on placement.

University	Treated	Controls	ATET	Std error	t-stat
Aston	62.9022	60.3303	2.5719	0.2764	9.3
Ulster	63.0757	59.24651	3.8292	1.0150	3.77

The results are fascinating. It would appear that Aston students who go on a work placement improve their final year average performance in finals by 2.6 percentage points. This is highly statistically significant at the 1% level. Remarkably, the result is even stronger for students at the Ulster University. Students who go on a work placement improve their final year performance in finals by nearly 3.8 percentage points. Again, the effect is highly statistically significant.

It is important to compare these results with the results for the simple estimator reported above. As can be seen, the coefficient estimates for the ATET are lower than the simple estimates reported in Table 4. This indicates that there is a self-selection issue in that high-calibre students do indeed choose to go on work placements and thus estimates reported in the literature may have an upward bias. Nevertheless, the results do still demonstrate a positive impact of placements on finals performance in contrast to Driffield, Foster, and Higson (2011).

As an additional robustness check we also implement the analysis separately for each of Aston's five schools to determine whether the ATET is different across disciplines.⁹ It is important to make clear that in this analysis, the students are now only matched with other students from their own school. As far as we know, no other study has looked at heterogeneity across disciplines at the same institution. The striking results are reported in Table 7. First of all we again report evidence of self-selection. The simple estimators in column 2 are higher than the ATET for each school reported in column 5. As can be seen, the ATET varies significantly across Aston's five schools. The impact of a work placement on finals performance for Aston Business School students is equal to 3.47 percentage points. Interestingly, this estimate is quite close to the ATET estimate for Ulster students of 3.8% – all of whom studied Business (see Table 4).

Table 7. ATET of students choosing to go on placement across Aston's five schools.

Aston University Schools	Simple Estimator	Treated	Controls	ATET	Std error	t-stat
Aston Business School	4.5990	60.6669	57.2004	3.4664	0.7601	4.56
Engineering & Applied Sciences	7.3421	63.4527	59.1449	4.30771	0.5780	7.45
Languages & Social Sciences	2.7032	64.7001	63.2618	1.43823	1.0271	1.40
Life & Health Sciences	2.6449	62.2565	60.6243	1.63220	0.4528	3.60
Interdisciplinary Studies	4.2553	63.0054	60.8333	2.17208	0.4810	4.52

There is also a large impact (4.30%) of undertaking a placement on final year performance for students studying in Engineering & Applied Sciences. Some of the large programmes offered to students in this school include Bachelor's degrees in Computing Science, Logistics, Chemical Engineering and Mechanical Engineering. Perhaps it is not surprising that placements improve student performance on degrees that have greater links to industry. In contrast however, the ATET for the other schools – Interdisciplinary studies, Languages & Social Sciences and Life & Health Sciences – are all far lower, indicating only a minor and positive effect of a placement on student performance. Large programmes offered in these schools include Bachelor's degrees in Biomedical Science, English Language, Human Psychology, Politics and Sociology. These results are particularly important because they give an explanation as to why there is so much variation reported in the literature. The results may indicate that it is not just placements per se that improve final year performance, but how well the placement is aligned (Biggs and Tang 2007) with a student's academic discipline. Indeed alignment might mean better quality placements improve student performance.

Conclusion and implications

The results of this paper have shown clearly that there is evidence of self-selection which gives credence to the argument that past studies that analyse the impact of work sandwich placements on student performance might well be upwardly biased. Nevertheless, the effect of taking an integrated sandwich work placement appears still to have a positive and significant impact on final year academic performance. We report estimates in the range of 2–4% across both institutions. These results are somewhat comparable to other estimates reported in the literature (Gomez, Lush, and Clements 2004; Surridge 2008; Mansfield 2011). Our analysis also examines the impact of a work placement across different disciplines by calculating the treatment effect for each of Aston University's five schools. It would appear that Business School students and Engineering students appear to reap the benefits of a work placement relative to students from the other schools. This suggests that the impact of work placements on student performance is perhaps more complex. It is certainly possible that work placements that are aligned to academic disciplines are more likely to lead to superior performance. Certainly additional research in this area is needed to determine whether this interpretation of our results is correct. Indeed the current literature provides little qualitative evidence based on student perceptions. In addition, there seems to be no articles that discuss the quality of a work placement or the type of work placement students undertake. Further research in this area is needed in order to gauge whether these factors may impact upon student performance.

The policy implications resulting from this study on the UK are clearly supportive of the recommendations of the Wilson review (2012, 40) that:

Ideally, every full-time undergraduate student should have the opportunity to experience a structured, university-approved undergraduate internship during their period of study.

In addition, specifically in the context of Northern Ireland, which does have an independent regulatory framework, the results are entirely consistent with the recommendation from ‘Graduating to success’ (2012, 18) that:

The Department expects the institutions to ensure that all learners have the opportunity to undertake a period of work placement whilst undertaking a higher education course.

Future course design, irrespective of academic discipline, should consider the substantial and growing empirical evidence that the completion of a placement year confers an advantage to students with regard to final year degree performance.

Acknowledgements

We would like to thank participants at the 2012 Higher Education Academy Annual Conference, Agelos Delis for useful comments on earlier drafts, Etta Parkes Head of Placements at Aston University and three anonymous reviewers who provided invaluable comments.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. These figures are obtained directly from the Higher Education Statistics Agency (HESA). They are defined as the total proportion of students (Home, EU and Overseas) who had a sandwich placement during part of their degree. The definition of a sandwich placement is a student who has undertaken periods of study, tuition or work experience amounting to an average of at least 21 hours per week for a minimum of 24 weeks.

2. Defining what we mean by a ‘high-calibre' student could be open to much interpretation and controversy. Our experience suggests that students who obtain grades above 60% or in terms of the UK degree classification system obtain an Upper-Second Class Honours Degree almost always demonstrate higher levels of learning and are more successful in obtaining a graduate level position once they enter the labour market.

3. We acknowledge that some students categorised as zero may not have filled out this information on their Universities and Colleges Admissions Service (UCAS) form. We have to assume therefore that missing data are essentially random.

4. This variable includes only UK students.

5. The Common Support assumption says that each individual must have a positive, but smaller than one probability of participating in the treatment: $.

6. Strictly speaking one might argue that there are potentially two important methodological weaknesses in our analysis.

Firstly, it could be argued that the two groups of students (treated and non-treated) are not comparable because they do not all participate in the same degree programmes or sit exactly the same examinations. However, we argue that although academic programmes differ in terms of knowledge, the underlying skills learnt on a degree programme are similar regardless of the academic discipline. These skills include critical thinking, communicating effectively both in writing and orally, analysing quantitative data and integrating knowledge from a variety of sources and fields. We argue that because of these common skills, amongst others not listed, we are justified in comparing students from different academic disciplines. Furthermore, by running the models separately for each school we do partially control for heterogeneity across disciplines.

Secondly, if there is grade inflation across the sample, the results may be upwardly biased. However, descriptive statistics of our data reveal that it is not present in the Aston or Ulster samples. The average final year mark was equal to approximately 59% for each separate year in the Aston sample and was equal to approximately 61% in the Ulster sample. Further statistics showing the moments of each sample are available on request.

7. Indeed it would also bias downwards the placement parameter estimate for a simple regression that does not account for self-selection.

8. There were 231 graduates in 2004; 590 in 2005; 691 in 2006; 918 in 2007; 944 in 2008; 1035 in 2009; 1143 in 2010 and 1093 in 2011.

9. We do not report the results of the Logit models used to calculate the propensity scores for students from each school separately as it would overburden the reader. The results are similar to those reported in Table 4. These results are available on request.

Appendix – Correlation matrices for Aston and Ulster

Correlation Matrix for Aston

Table

Variable	1	2	3	4	5	6	7	8	9
1. Placement Dummy	1
2. Stage 1 Average	0.079	1
3. Stage 2 Average	0.1926	0.3745	1
4. Final Stage Average Mark	0.331	0.2631	0.6508	1
5. Female	0.0453	−0.0377	0.122	0.0685	1
6. Home	0.1678	−0.022	−0.0121	0.1335	0.0076	1
7. Grammar	0.0492	0.0297	0.0151	0.038	0.0436	0.1314	1
8. Age	−0.1514	−0.1294	−0.0226	−0.0904	−0.0434	−0.1939	−0.0849	1
9. Higher Managerial Class	0.0563	0.0037	0.0322	0.0482	0.0077	0.0647	0.0492	−0.0518	1

Correlation Matrix for Ulster

Table

Variable	1	2	3	4	5	6	7	8
1. Placement Dummy	1
2. Stage 1 Average	0.0651	1
3. Stage 2 Average	0.1957	0.7001	1
4. Final Stage Average Mark	0.3793	0.4951	0.632	1
5. Female	0.0515	0.0958	0.1839	0.2124	1
6. Grammar	0.0153	0.0556	0.0049	0.0031	0.0425	1
7. Age	0.3344	−0.0474	0.0162	0.0742	−0.072	−0.0337	1
8. Higher Managerial Class	0.0181	0.0094	0.0023	−0.0191	−0.0659	0.0438	−0.0563	1