Evaluating Employability Skills: Employer and Student Perceptions

Abstract

Graduate employability is a key issue for Higher Education. In this two-part study student employability skills have been evaluated from the perspective of sandwich students and graduates in biomolecular science, and their employers. A strong correlation was found between employer and sandwich student/graduate perceptions of the relative priorities amongst employability skills. Skills such as enthusiasm, dependability and team-working scored higher than subject knowledge skills, whilst commercial awareness, negotiation and networking were given lowest priority. Furthermore, the lowest ranked skills were those that sandwich students/graduates were assessed to be least proficient in. Overall skills of new graduate employees were rated less highly by their employers than by the graduates themselves. In the second part of the study an employability skills profile was compiled and distributed to biomolecular science students at levels 1, 2 and 3, as part of personal development planning. Level 3 students rated themselves more highly than level 1 and level 2 students in subject knowledge, most core skills and personal qualities, except tolerance to stress. Implications of this study, including the value of student self-assessment of their skills and utility of the profile to underpin personal development planning and inform graduate recruitment processes, are discussed and recommendations made.

Keywords:

• Employability skills
• Employer
• Sandwich student
• Graduate
• Self-assessment

Introduction

There are various definitions of ‘employability’, the one adopted here is that of Yorke (2006):

“a set of achievements — skills, understandings and personal attributes — that make graduates more likely to gain employment and be successful in their chosen occupations, which benefits themselves, the workforce, the community and the economy.”

Employability is a critical issue for both government and Higher Education Institutions (HEIs). With the expansion in HE and recent economic downturn, there is intense competition for jobs in the graduate employment market. The Summer Graduate Recruitment Survey 2009 (Association of Graduate Recruiters, 2009) indicated that on average there were 48 applications per graduate vacancy.

One of the key reasons why many students invest in university education is to improve their employment prospects. However, whilst achievement of good academic qualifications is highly valued, it no longer appears sufficient to secure employment (Yorke, 2006). Additionally, employers expect students to have well developed employability skills, so that they can make an immediate contribution to the workplace when recruited (e.g. Confederation of British Industry, 2008). Thus, whilst some employers screen job applications on the basis of degree classification, such achievements are much less important at the short-listing stage. Moreover, in some cases employers initially use criteria other than the honours degree to assess applications; often requiring applicants to undertake a series of skills activities and psychometric tests, and to produce a personality profile (Graduate Prospects, 2009). Some employers adopt such practices for purposes of ensuring equal opportunities, of widening access and of recognising a more diverse group of potential employees. It is believed that graduates with good employability skills may otherwise be missed because they have not attained good academic qualifications (Denholm, 2004; Morley et al., 2006; Morley and Aynsley, 2007). Furthermore, it is increasingly recognised that the honours degree classification system may not be reliable (Yorke, 2007) and there has been concern about grade inflation (Baty, 2007). The different regulations and practices pertaining to degree outcomes in different universities could undermine the fairness and comparability of the classification system across different institutions (Lowe, 2007; Yorke et al., 2007). However, the external examiner system should help to safeguard standards (e.g. Jackson, 2005; Quality Assurance Agency for Higher Education, 2006), although this has been questioned (House of Commons Innovation, Universities, Science and Skills Committee Report, 2009).

The current trend of placing increased emphasis on graduate/key skills therefore dictates that the HE curriculum incorporates opportunities to develop such skills in conjunction with subject-specific skills and knowledge. This should enhance applicants’ potential for success in the recruitment process by producing ‘business ready’ graduates, able to make a dynamic start and rapidly adapt to change. To this end, different academic programmes in different universities are adopting various strategies by, for example, offering work experience, work-related learning and employability skills modules, and ‘ready for work’ events, as well as involving employers in course design and delivery. In many cases, with employability skills already embedded in the curriculum, universities employ a range of initiatives to make them more explicit to students (Cranmer, 2006).

In Biomolecular Sciences (BML) at Liverpool John Moores University (LJMU) an evidence-based employability skills profile is being used, as a component of personal development planning (PDP), to raise awareness of graduate skills and to provide the opportunity for students to relate to, reflect upon, assess and manage their own skills. This paper describes the procedures used to identify and prioritise the specific employability skills required by potential employers of bioscientists and to compile and implement the employability skills profile.

Various surveys have been conducted (e.g. Archer and Davison, 2008; Bunt et al., 2005; Canny, 2004; National Employers Skills Survey 2007; Stephens and Hamblin, 2006) to identify the desired skills required for graduate employment and considerable agreement has been found amongst different employers (irrespective of the nature of the employment) on the skills valued most. However, it is important to ascertain whether students have similar views to employers. Accordingly, our study focussed on the perceptions not only of employers, but also of undergraduate sandwich students and of graduates, in order to recognise opportunities in the curriculum for developing and enhancing student employability skills. Moreover, the employers selected here had specific links with our institution and our BML programmes.

The main aims of the research were:

• to survey the main stakeholders viz sandwich students, graduates and employers to determine their views on the nature and relative importance of different employability skills, and on the preparedness of students and graduates for employment;

• to compile a skills inventory (based on the needs of employers) and incorporate into an employability skills profile for distribution to undergraduate bioscience students (to facilitate students in identifying their own strengths and weaknesses); and

• to utilise data from the student self-assessed skills profile to inform the curriculum and delivery of modules.

Methods

The study was undertaken in two phases. In phase 1 sandwich students (i.e. students undertaking a full year of employment on a placement between levels 2 and 3) and graduates, and their employers were surveyed by questionnaire to determine their perceptions of the relative importance of different graduate skills and to assess the preparedness of sandwich students and graduates for employment. The students and graduates were chosen on the basis that they had experience of bioscience-related employment in e.g. the NHS, biotechnology industries, the Forensic Science Service and research institutes. They were representative of all the main programmes in Biomolecular Sciences in the School, including Biochemistry, Biomedical Science, Biotechnology, Forensic Science and Microbiology. The employers were all involved in bioscience-related areas and employed our BML sandwich students and graduates. They had prior contact with LJMU and were familiar with the School and its programmes.

A skills inventory was compiled, based on the responses of the employer’s questionnaire, and used in phase 2 as a framework to create a self-assessed, evidence-based employability skills profile. The profile was distributed to biomolecular sciences students at each of levels 1, 2 and 3 for completion, and results analysed to determine the effectiveness of skills provision in BML.

Phase 1: Employability skills questionnaires

An ‘employability skills’ questionnaire was designed for employers, incorporating a set of skills/competencies deriving from a number of sources (e.g. Murphy, 2001; Tariq and Cochrane, 2003; Tomkins, 2004; Yorke and Knight, 2004). Skills were differentiated into three main categories: ‘personal qualities’, ‘core skills’ and ‘subject knowledge’. Employers were requested to prioritise the skills (in the ‘Priority’ column) on a 3-point scale, where 1 = low; 2 = medium; 3 = high importance and any skill deemed not to be applicable to be deleted. There was also space for other skills, identified as important to employers, to be listed. In addition, employers were requested to assess the skills proficiency of their BML sandwich students and graduates on entry to the workplace as ‘poor’, ‘average’ or ‘good’. This assessment was afterwards converted to a point scale where 1 = poor, 2 = average, 3 = good. Where a number of students/graduates had been employed over several years, employers were asked to give average scores. At the end of the questionnaire there was space for ‘free response’ comment on any matters relating to student employability. (See Appendix 1 for questionnaire.)

A similar questionnaire was designed for the sandwich students and graduates, requesting prioritisation of the graduate skills and self-assessment of their proficiency in such skills. The same scoring regimes were applied as used for the employers, so that direct comparisons could be made.

Questionnaires (including a stamped self-addressed envelope for reply) were sent by post in February 2006 to 22 BML sandwich students (those placed for 2005/2006), to 52 employed graduates (identified from the LJMU First Destination Survey dating from 2002–2006) and to 59 employers of BML sandwich students and graduates. Completed questionnaires were received by post up until the end of May 2006.

Phase 2: Employability skills profile

Employability skills profile documentation was produced by formulating a skills inventory based on the survey of employers (from phase 1) and differentiated into the same three skills categories. The inventory was incorporated into the profile, together with sections for: student self-assessment of their skills using a 5-point scale, where 1 = very poor, 2 = poor, 3 = average, 4 = good, 5 = excellent; supporting evidence (from curricular/extracurricular activities) in order to claim proficiency in the skills; and an action plan to develop/improve the skills. In this way students could identify and build on their strengths, address their weaknesses and generally manage and up-date their skills (Table 1).

In addition, there was space for students to compile their own techniques inventory, by reflecting on the techniques they had developed, and the learning outcomes they had achieved at each level of their bioscience programme. Finally students were given free space to comment on their career plans. The profile was used to support PDP.

Table 1 Main features of employability skills profile

Skills Inventory	Student Self-assessment of Skills	Evidence	Action Plan
Techniques Developed		Learning Outcomes Achieved
Career Plans

The document was disseminated to all students via a core, skills-based module at levels 1, 2 and 3, in the academic year 2006/2007; completion was not compulsory. A hard copy of the profile was contained in the module handbook distributed at the start of each module. Additionally, it was available on the LJMU virtual learning environment, Blackboard™, to ensure all students had an opportunity to engage with the profile. Completed hard copies of the profiles were collected at the end of the respective modules viz: the end of semester 1 for the level 2 module; the end of semester 2 for the level 1 and level 3 modules. Instruction for completion of the profile was provided by the module leaders and a set of support materials made available on Blackboard™. Students were offered feedback on the completed profile from the academic staff who looked for indications of strengths and weaknesses based on the evidence provided.

Compilation of support materials

A set of support materials for the employability skills profile was produced (listed below) to assist students in understanding the meaning and significance of the skills, in recognising links between skills and the curriculum, and in how to complete the profile. Materials were uploaded to an ‘Employability Skills’ Organisation on Blackboard™, where students could access them.

• Employability skills document online

• Glossary of skills terms with descriptors

• Guidance notes on self-assessment and collecting evidence

• Sample profile entries

• Reflective questions for interviews

• External links to relevant employability sites

• Discussion board for comments on initiative

Analysis of data

Quantitative data were analysed using the Minitab ® Statistical Software Package (version 15, 2007; Minitab Inc. State College, PA).

For prioritisation of the graduate skills, sandwich students and graduates were treated as one group, and employers as another group. The overall mean values and standard deviation (SD) of the scores for each skill were determined for each of these two groups. Skills were ranked in descending order (standard competition order) by the mean value and SD.

To determine the correlation between the prioritisation of skills data for the two groups (viz employers and students/graduates) the overall mean values of the employers’ scores and of the student/graduate scores for each skill were expressed as a scatter plot and Pearson’s correlation coefficient derived.

For assessment of sandwich student and graduate performance in the skills on entering the workplace a Student’s two-sample t-test was used to compare the perceptions of sandwich students and sandwich student employers, and of graduates and graduate employers using the overall mean performance scores made by each of the four groups of assessors. In addition, the percentage of each group of assessors (sandwich students, sandwich student employers, graduates and graduate employers) assessing the skills proficiency of the sandwich students or graduates as ‘average’ or ‘good’ was calculated. Skills that students and graduates were judged to be less proficient in were identified as those that fewer than 80% of assessors had assessed as ‘average’ or better.

In the second part of the study students at each level assessed their own employability skills and the overall mean scores for personal qualities, core skills and subject knowledge at the different levels were compared using a one-way analysis of variance (ANOVA).

Unless otherwise stated, the standard criterion of statistical significance (p<0.05) was applied.

Results

Phase 1: Employability skills questionnaires

The employability questionnaire was sent to 59 employers of BML sandwich students and/or BML graduates, and 29 responses were received, representing a response rate of 49%. Of the respondents, nine had employed both BML sandwich students and BML graduates. The employers were from bioscience-related sectors including a number of biotechnology/pharmaceutical industries, the NHS, the Forensic Science Service and some Academic Research Institutes.

Of the 22 sandwich students and 52 graduates who were sent the student version of the employability questionnaire there were 17 and 16 responses respectively, representing response rates of 77% and 31%.The proportion of these respondents was 51.5% sandwich students and 48.5% graduates.

The respondents were from a cross-section of the School’s programmes and included both part-time and full-time students. Given the relatively small numbers of these respondents, (where applicable) data from both groups were combined,

Tables 3 and 4 present the graduate skills, as prioritised by employers and sandwich students/graduates respectively, on the basis of overall mean values of the assessors scores for each skill (from analysis of the questionnaires). Skills were placed in descending rank order. Both employers and sandwich students/graduates ranked personal qualities very highly: six of the first eight priorities for employers and four of the first eight for sandwich students/graduates. The five lowest scoring (averaging <2.0) skills were the same for both groups.

Table 3 Prioritising Skills: Employer’s View

Skills were scored on a scale of 1 = low; 2 = medium; 3 = high importance. Mean scores from the employers were used to rank the skills from the most (no.1) to the least (no.36) important. , personal qualities. Skills below red bar () averaged <2.0.

Table 4 Prioritising Skills: Sandwich students / Graduates View

Skills were scored on a scale of 1 = low; 2 = medium; 3 = high importance. The data from the sandwich students and the graduates were pooled, and the mean scores used to rank the skills from the most (no.1) to the least (no.36) important. , personal qualities. Skills below red bar () averaged <2.0.

A scatter plot of student/graduate and employer mean scores for the importance of the various skills showed a strong positive linear correlation. The correlation coefficient was 0.902, indicating that there was good agreement (Figure 1), and this was strongly statistically significant (p<0.001). The most important skills (mean scores >2) and the least important (mean scores 1–<2) were generally the same for both groups. There were some outliers (taken as those data points furthest from the regression line) with employers rating creativity, work ethic, integrity and written communication more highly than the students, whereas students/graduates rated leadership, professional development, decisiveness and technical ability more highly than the employers.

Figure 1 Scatter plot showing correlation between skills priorities of employers and students / graduates.

Mean scores of the prioritised skills of sandwich students and graduates plotted against those of employers, with a correlation coefficient of +0.902 (P<0.001). Outliers:

, creativity;

, work ethic;

, written communication;

, integrity; (considered more important by employers);

, decisiveness;

, professional development;

, leadership;

, technical (considered more important by sandwich student/graduates).

Overall there was good agreement between the BML sandwich students/graduates and the employers about the relative importance of the employability skills, although a few interesting differences were found.

In addition to the skills listed on the questionnaire, another attribute ‘personal image’ was identified by a number of employers as important. Accordingly, this was added to the skills inventory used in the employability skills profile.

To assess the proficiency of our sandwich students and of our graduates, their employability skills on entering the workplace were scored as ‘poor’, ‘average’ or ‘good’ and converted to a point-scale of 1, 2 or 3 respectively. Overall mean scores of proficiency in all the skills made by each group of assessors (namely sandwich students and sandwich student employers, assessing sandwich students; graduates and graduate employers assessing graduates) were then compared using a Student’s t-test. By comparing sandwich student self-assessment with the assessment made by employers of sandwich students only a very small difference (student opinion 0.04 points higher than that for employers) was found and this was non-significant (p=0.167). A similar test, comparing graduates with employers of graduates showed a clearer difference (student opinion 0.33 points higher than that for employers) and this was strongly significant (p<0.001). Thus, unlike the sandwich students, the graduates rated themselves more highly than did their employers.

Table 5 shows the percentage of assessors in each group (sandwich students, sandwich student employers, graduates and graduate employers) scoring the skills proficiency of either the sandwich students or graduates (as applicable) as ‘average’ or ‘good’. Proficiency in all skills, with the exception of commercial awareness, creativity, leadership, negotiation and networking, was judged as ‘average’ or better by ≥80% of assessors across all groups. Interestingly, the lower scoring skills, based on <80% of assessors in one or more groups scoring the skills as ‘average’ or ‘good’, corresponded to those that had been rated of lower priority (refer to Tables 3 and 4). Moreover, commercial awareness, the least valued skill, was assessed as ‘average’ or ‘good’ by less than 80% of assessors in all four groups.

Table 5 Assessment of skills proficiency of sandwich students and graduates.

‘Percentage of assessors (StudentS, sandwich student [n=17]; EmployerS, sandwich student employer [n=28]) assessing the skills proficiency of sandwich students as ‘average or ‘good’. • Percentage of assessors (Graduate, [n=16]; EmployerG, graduate employer [n=10]) assessing the skills proficiency of graduates as ‘average’ or ‘good’. , skills graded ‘average’ or ‘good’ by < 80% of assessors of one or more groups.

Phase 2 Employability Skills Profile

In the second part of this study undergraduate BML students (of academic year 2006–2007) were able to assess their skills on a 5-point scale as part of an employability skills profile, distributed to students at all levels through skills-based modules. At level 1 there were 67 returns from 180 distributed (response rate 37%); at level 2, 31 returns from 195 (response rate 16%); and at level 3, 40 returns from 143 (response rate 28%). When the overall mean scores for self-assessment of skills proficiency were compared between levels 1, 2 and 3 no statistical difference was found (p=0.321). Although there seemed to be a trend for level 3 students to rate their personal qualities (except tolerance to stress), most core skills and subject knowledge more highly than students at levels 1 and 2.

Discussion

This study has identified skills and qualities needed by our graduates to enhance their employability. Employability skills identified by employers of LJMU biomolecular scientists were in line with those reported by other researchers surveying different employers (Archer and Davison, 2008; Brennan et al., 2001; Herrmann, 2009; Institute of Directors, 2007). The majority (86%) of the skills identified had mean scores of 2 or above, indicating that they are of medium or high value. Interestingly, enthusiasm and willingness to learn, which the employers valued most, were highlighted in an Industry and Parliament Trust’s survey back in 1997 (Clarke, 1997). Employers ranked a number of the personal attributes and core skills more highly than technical and subject-specific skills; a finding that is supported by other surveys (e.g. Archer and Davison, 2008; Cotton, 2001; Felstead et al., 2007). This confirms the view that employers, even in highly technical scientific jobs, generally value certain generic skills and traits above specialist occupational skills and knowledge. This is not to ignore the importance of subject knowledge, but to emphasise the added value of effective personal qualities and core skills. Moreover, it should be recognised, particularly with changing economic conditions and the global challenges currently facing many employers, that other skills, such as entrepreneurial response and an ability for multi-cultural working, are likely to emerge as important attributes for new graduate recruits (Archer and Davison, 2008; Herrmann, 2009). Such skills would be included for prioritisation in any future survey.

Commercial/business awareness, the core skill valued least in this survey, was also one that students and graduates felt most deficient in; a finding consistent with that of other surveys (e.g. Brown et al., 2005; Gilworth and Thambar, 2006). The importance placed on commercial awareness appears however, to be quite variable among employability skills surveys in general. The skill seems to be more highly valued in business and management graduates (e.g. Employability and Placement Unit, the University of Wolverhampton Business School). Although Kay Wardle, managing director of a life science recruitment company, also listed commercial awareness as an important transferable skill (Biosciences Federation Education Colloquium, 2005). Clearly for students to make an early contribution to the world of work and the economy, some commercial understanding might be considered beneficial. However, since this skill is often likely to be job-oriented, it is probably more effectively developed once in the workplace.

It was gratifying to find that there was strong convergence of student/graduate and employer perceptions of the relative importance of the different employability skills. Interestingly however, students rated technical skills more highly than the employers. At a time when laboratory work is being reduced by limitations on resources in many HEIs (Hughes, 2006), it appears that employers do not value technical competence as highly as certain personal skills and may prefer to train graduate recruits in-house, at least in some specific aspects of laboratory work.

In addition to prioritising skills in order of perceived importance, employers evaluated student performance on entering the workplace and students assessed themselves, so that the preparedness of students for employment could be gauged.

Generally the opinions of the employers were favourable and there was close agreement with those of the sandwich students. Agreement was less good with the graduates, who tended to overrate themselves. However, the sandwich students represented a somewhat biased sample, since, due to the current competition for sandwich placements, they were effectively a self-selected group of higher achievers. The findings were, nonetheless, in line with those of, for example, Boud and Falchikov (1989) and Longhurst and Norton (1997) that high achieving students generally make more realistic self-assessments. Although the accuracy of self-assessment is problematic, with some students/graduates inflating their abilities and others underestimating them, the process can be effective in fostering reflection on one’s own learning and development. Furthermore, it can provide a useful springboard for determining personal improvement goals (Pintrich and Schunk, 1996).

Self-assessment was also used in the second part of the study where the employability skills profile documentation was distributed to undergraduates at levels 1, 2 and 3, with the primary role of supporting PDP. Students were encouraged to analyse their strengths and weaknesses, and to find evidence to support their claims. There was no significant difference between the assessment of skills made by level 1, 2 or 3 students, although level 3 students tended to score most of their skills more highly than the students at other levels. The accuracy of the assessments was not explicitly judged here, but feedback was given on the basis of evidence presented. It was, however, clear (both from anecdotal evidence and from results of a student semi-structured feedback questionnaire [data not shown]) that students found the self-assessment process and the concept of evidence-supported claims, difficult. This was so, despite the guidance given by staff and the availability of supporting materials. The former in particular is, perhaps, not unexpected given the paucity of self-assessment tasks undertaken by our undergraduates. Accordingly, a recommendation from this study would be to incorporate other opportunities into the curriculum for students to practise self-assessment, especially as this is pivotal to PDP. More emphasis appears to have been placed on the process, rather than the physical collection of evidence, by those involved with instigating sector policy on PDP (Cooper, 2006). Students therefore need to be encouraged to reflect critically and self-evaluate effectively in order to have accurate perceptions of their own abilities, performance and achievements, which could then be applied throughout their working lives.

Whilst the 5-point scale used for self-assessment is a balanced scale, the scoring relied on individual interpretation of the categories and there appeared to be a central tendency bias. A forced-choice rating scale, for example a 4-point scale, avoiding the response of ‘average’, or an increased number of categories may improve reliability and validity of the assessments, particularly when making comparisons between different groups (Dawes, 2008; Friedman and Amoo, 1999).

Response rates for completion of the employability skills profile by students at all levels were below 50%. However, since participation was voluntary and no credits were awarded for submission of the profile, a high response rate was not anticipated. Whilst time was specifically allocated to completing the profile at level 1, as a component of the Tutorials Module, this was not the case at levels 2 or 3. Thus there may be a need to set some time aside at these levels as well. However, evidence demonstrating the ‘value-added’ of the profile as an element of PDP and the effectiveness of the PDP process itself may be needed to convince students of the potential advantages of participating (see Cooper, 2006).

Conclusions and Recommendations

Sandwich students and graduates, and their employers appear to be largely in accord with respect to the priorities amongst employability skills, all ranking certain personal qualities above subject-specific skills.

The evidence-based employability skills profile produced here has provided a personal development planning resource that is being used by students at all levels of BML programmes. It has offered a supportive framework for reflection and self-evaluation, enabling students to identify their own strengths and weaknesses, and make provision for self-development and improvement. Furthermore, it should prove useful for production of personal statements, CVs and preparation for job/course interviews. By introducing the profile at level 1 in degree programmes, students have been encouraged to forward plan for employment and embark on decision-making processes from an early stage. In this way they can further develop skills over the course of their programme. The profile has permitted a more holistic approach to learning and development, drawing on both curricular and extracurricular activities, with the aim of promoting life-long learning. Whilst the reliability of self-assessment is uncertain, it is allowing students greater ownership of their skills, in turn empowering them to take control of their own development, consistent with the concepts of PDP. Self-assessment is itself a skill that needs to be developed, but further guidance, more explicit feedback and more practice may be needed to exploit the process fully.

This study has highlighted the importance of aligning employability with academic values by making explicit links between the curriculum and employability. In addition, it has identified challenges that need to be overcome if more students are to embrace the potential benefits of engaging fully with the skills profile, as part of their personal development planning.

Acknowledgements and Dedication

This work was supported by a HEFCE-funded Learning and Teaching Award. Dr Phil Rowe is thanked for his guidance with the statistical analysis. Grateful thanks also to all students, graduates and employers who completed the questionnaires.

Notes

This paper is dedicated to the memory of Katherine Zuzel, who died suddenly in December 2008 during its preparation. Katherine was a committed teacher, with a particular interest in supporting students in the development of their employability skills. Her experiences as a Senior Lecturer, Programme Leader and Sandwich Placement Tutor at LJMU were invaluable to this research.