An institute-wide framework for assessed group work: development and initial implementation in an Irish Higher Education Institution

ABSTRACT

Despite the academic, practical and social benefits, the assessment of group work brings challenges . S In response to feedback from students and staff, a framework and set of guidelines around assessed group work were developed by Dundalk Institute of Technology’s Learning and Teaching sub-committee of Academic Council. This article describes the development, implementation, and inititial evaluation of the framework and its impact. The mixed-method evaluation involved multiple stakeholders -student representatives, academic staff and key informants The evaluation suggests the framework is having a positive impact on both the staff and student experience. Predictably, the greatest impact is on awareness although there is also evidence of impact on lecturers’ practice. The impact at programme level is limited but encouraging early indications exist. Nonetheless, the evaluation demonstrates the need for ongoing support in interpreting and implementing the framework

KEYWORDS:

• Assessment
• group work
• student experience
• higher education
• programme assessment

Introduction

Group learning is widely recognised as beneficial (Hassanien, 2007; Lee, Ahonen, Navarette, Frisch, 2015) and consequently group projects and other forms of group work are widely and increasingly used as part of the assessment process in higher education. There are many academic, social and practical benefits (e.g. Hammar Chiriac, 2014; Lee et al., 2015; Noonan, 2013). These include development of critical skills, independence, enhanced motivation and the development of important skills such as collaboration, negotiation and leadership as well as other aspects of social learning that emulate work environments (Cassidy, 2006; Fearon, McLaughlin, & Yoke Eng, 2012; Glover, Sue Law, & Youngman, 2002; Mutch, 1998). Group work based projects also offer practical advantages in terms of efficient use of lecturer time with large classes. Despite these considerable advantages, there remain many challenges. Perhaps the greatest of these are students’ perceptions of (un)fairness (Caple & Bogle, 2013; D’Arcy, Geoghegan, Gibson, Hines, & MacAnaney, 2016; Macfarlane, 2016; Noonan, 2013; Carvalho, 2013; Smith & Rogers, 2014). Other significant challenges include its assessment and the demands placed on students’ time (Caple and Bogle, 2013; Lee et al., 2015). Certainly, the evidence indicates that group work does not automatically benefit students; if it is to be successful, it needs to be facilitated and students need preparation and guidance (e.g. Noonan, 2013). Group work that is not well planned and supported can impede learning, create a difficult social environment and cause students to experience stress and distress. Perhaps more than any other form of assessment, group work highlights the ethical issues inherent in assessment (Noonan, 2013).

The assessment of group work is particularly challenging in terms of fairness from the students’ perspective. Assessing only the final product or outcomemay lead ‘ … to inevitable inequities in the allocation of grades where the efforts of hard working students may be missed and students who “free ride” are unjustly rewarded’ (Caple & Bogle, 2013, p. 200). Including assessment of the process can go some way to addressing this, although measuring individual contributions may be difficult and time consuming for lecturers. Student peer review increasingly is incorporated into assessment of the group work process and there is evidence that, if well planned, it can reduce free-riding (Brooks & Ammon, 2003; Porr, 2016). However, students often perceive peer assessment as procedurally unjust (Wilson, Diao, & Huang, 2015). As Wilson et al. (2015) point out, combining peer learning and peer assessment asks a lot of students, particularly given the competitive atmosphere common in higher education. It is clear students need support to develop and practice group work skills; for example, peer assessment should be introduced early in a programme if it is to be successful (Brooks & Ammons, 2003). D’Arcy and colleagues (2016) reported that a combination of a considered approach and a transparent effort to be just was more likely to be perceived as fair. While there is no simple solution, certainly the evidence suggests that interventions at the programme and/or institutional level are more likely to be effective.

This article discusses the development, implementation and initial evaluation of a framework and guidelines for assessed group work within an Irish Institute of Technology.¹ This work was initiated in response to concerns around student experiences of assessed group work. Student feedback from course evaluations, programme boards and local data from the Irish Survey of Student Engagement (ISSE) indicated assessed group work consistently emerged as both a very positive and very negative dimension of our students’ experiences. Furthermore, feedback from Student Services indicated that, for some students, difficulties with group work, particularly inter-personal tensions, were having a negative impact. These concerns reflect the challenges identified in the literature. In response, the Academic Council tasked its Learning and Teaching subcommittee to review the issue and make recommendations. A working group was established to do this. This group reviewed the literature and consulted extensively with both students and lecturers. This led to the development of a framework and guidelines for assessed group work aimed at programme teams and lecturers. These are not prescriptive but rather offer a structure for thinking about the purpose and nature of assessed group work, within both programmes and modules and for making choices that will enhance the student experience. In this article, we discuss the development, implementation and initial evaluation of this framework.

Development of the framework for assessed group work

The internal process commenced with informal discussion and reflections on our diverse experiences of assessed group work. This was followed by a review of the literature and the identification of the relevant stakeholders: student representatives, academic staff and Student Services.

Student (‘class’) representatives are elected by each year group on every programme offered. These representatives are members of the relevant programme boards² and this allows them to represent the views of their class groups and inform the management of the programme. Given their role, student representatives have a good understanding of the range of views and experiences of the students they represent, and they are well placed to provide feedback on programme level issues. They are also an important source of information and advocacy for their fellow students. Therefore, we considered their engagement essential to understanding and responding to the challenges of assessed group work.

The consultation process involved:

• Three cross-school engagement sessions with student representatives from across the institute,

• An online survey of academic staff (n = 52) reviewing current practice,

• Drop-in consultation sessions for academic staff, these were held in coffee areas in all main buildings,

• An informal interview with a student counsellor.

The consultation identified many examples of excellent practice and positive experiences. It was noteworthy that the positive examples identified by staff and students were clearly structured, monitored by lecturers and perceived by both as contributing to the development of key skills. The consultation also identified a number of concerns. The (in)visibility of assessed group work emerged as a key issue. Often assessed group work projects were identified simply as ‘projects’ or ‘presentations’ on assessment schedules. This meant that programme teams were not always aware of the volume of group work that any one class group was involved with. A minority of student representatives reported being involved in 5–6 pieces of assessed group work in a single semester, often with different group members. Whilst staff often highlighted the importance of group work with regard to transferable skills and career development (such as leadership, management, conflict resolution etc.), students were particularly concerned with group selection, logistics and workload. Discussions with students consistently focused primarily on group work organisation, monitoring and assessment approaches, rarely mentioning the potential skill development.

Ultimately, it was evident that there was no one ‘perfect’ solution for group work and the difficulty in generating a proven set of clear rules or processes emerged. Therefore, the goal of this initial engagement was refocused to generate a guiding framework on group work that could advise and assist both programme staff and students working together to plan, monitor, assess and enjoy group work, harnessing the potential it can bring to learning and skill development. The main areas and themes identified from the analysis where guidance was needed were: (i) programme development, (ii) module design and (iii) the assessment and (iv) monitoring of group work. Addressing group work at the programme level emerged as particularly important and the framework also was informed by the work of the National Forum for the Enhancement of Learning and Teaching (NFETL) on programme level assessment. A case study (NFETL, 2017) based on the development this framework is included as an example of leading change in the NFETL’s resource on programme approaches to assessment .³

Assessed group work: a framework and guidelines

Following consultation with relevant stakeholders, including academics and students, and incorporating current literature on group work, ‘Assessed group-work, a framework and guidelines’ (DkIT, 2016) was presented to, and approved by Academic Council. It was designed as a non-prescriptive framework for thinking about assessed group work. It begins with the premise that group work does not automatically benefit students, but needs to be well planned, structured and supported. The framework includes a set of guiding questions to be considered by those involved with programme and module development and delivery. At each level of development, a structured, hierarchical approach to considering the need for group work is presented; where and when group work will happen; how, and by whom, it will be managed, monitored, assessed and supported. As there is no single ‘best’ approach to forming groups, managing the process and assessing, the framework does not seek to provide definitive answers. Instead, the framework helps to guide the decision making at each stage of the process: Programme Development; Module Planning; Assessing and Monitoring.

Figure 1 presents the process of developing, monitoring and assessing group work as a flowchart of questions at the various stages. At each stage of the process, the appropriate stakeholders, documentation and resources are identified. Stakeholders include the academic staff developing the programme, academics who are delivering a module, and the students; documentation includes programme and module learning outcomes, assessment schedules and rubrics; resources may include supporting collaborative technologies, learning contracts, FAQs (‘Frequently Asked Questions’), timetables and room availability. Within the body of the framework, a discussion and related action points provide further guidance for each stage of the process. This guidance encourages those involved in assessed group work to fully consider the choices available and the impact of their decisions when introducing and developing group work activities.

Figure 1. Planning, implementing and assessing group work.

Below, we summarise each section of the framework document.

‘Programme development’

This section considers, at a programme level, the appropriateness of group work, the number of group work assessments and scaffolding of group work skills. Graduate attributes and programme learning outcomes should be consulted to determine the necessity and benefits of group work to the students. Taking a programmatic approach and scaffolding the development of group working skills over the course of a student’s studies supports both skill development and assessment literacy. To improve visibility of group work, assessment schedules should explicitly identify group work assessments and be discussed by programme boards, which include academic staff and student representatives, and consider whether the amount of group work is ‘reasonable’. Notably, the framework does not specify what constitutes ‘reasonable’, as this may depend on the programme or discipline.

Any required training and supports for staff and students should also considered at this level, looking at the capacity of the programme to successfully implement group work activities. An important element of this training aims to increase staff and student assessment literacy in relation to group work. While staff should be familiar with sound assessment practice, students will need to become familiar with assessment terminology and with standards of quality in assessment (Taylor, 2009; Webb, 2002). This is supported by the Institute’s Assessment and Learning Policy, which includes building capacity among staff and students as a principle (DkIT, 2017). This section concludes with a set of action items that should ensure group work is considered systematically as part of the Institute’s regular activities at programme level, through programme boards, assessment schedules, student induction and as part of the 5-yearly Programmatic Review process.

‘Module planning’

This section considers the module learning outcomes, assessments and learning activities, and considers group allocation, training and support. Some elements at this stage of group work were identified for particular focus in the literature and through stakeholder consultation. These aspects of group work have the potential to cause student dissatisfaction, including group formation and “free-riding''. The advantages and disadvantages of student self-selection and lecturer-formed groups are outlined to guide the decision. ‘Free-riding’, where some group members do not contribute or engage fully with the group work activity, is identified as an issue by academic staff, and highlighted in the literature. A number of strategies are presented to help encourage participation by all members of the group, along with a link to further case studies from Perry (2008). The aim is to improve the implementation of group work activities by considering these potential barriers to successful group work, in advance, at the module planning stage. The strategies can be used at various stages of the group work activity, and may impact on milestones or resources, for example using learner contracts, peer assessment, logs or reflective journals.

‘Assessing’

This section emphasises the need to differentiate between the process and product in the assessment (Dijstra, Latijnhouwers, Norbart & Tio, 2016). The framework asks lecturers to clarify whether they are assessing process, product or both. If the process is being assessed, the deliverables and rubric should reflect this. The framework suggests opportunities to make the process more visible, including the use of collaborative technologies such as discussion forums, or other methods of managing, recording and monitoring group interactions and individual activities. This is covered in more detail at the ‘Monitoring’ section of the framework.

Grading is considered in detail, and the advantages and disadvantages of the different approaches. For example, the guidelines suggest that a single mark, while it may be appropriate in some circumstances, is likely to be perceived as unfair (e.g. Caple & Bogle, 2013). Individualised grades or a weighted combination of marks based on member contribution or individual roles are more likely to be perceived as fair. The implications of introducing peer- and self-assessment are discussed in the guidelines, along with the need for training. Formative peer evaluation, lecturer feedback and reflections on learning may be important especially where the objective is to learn about the process of working in groups.

‘Monitoring’

This section considers the level of visibility needed on the group work activity, and how issues are identified and resolved. Technologies and processes to support organising and managing groups should be considered at this stage. The prior experience of the students and the availability of resources to support the group work activities will affect the positive outcome of this phase. The guidelines outline a three-phase approach to monitoring: before, during and after the group work activity. For example, discussing expectations and agreeing a mechanism for resolving problems before the group work begins; supporting students in allocating tasks, reviewing progress regularly and intervening, when necessary, to resolve issues as the group work is on-going; finally, providing timely feedback and the opportunity to reflect on the group work activity afterwards is also considered.

Initial implementation of the framework

Once approved by Academic Council, the framework was published on the Institute’s website and circulated to all staff via email. We recognised that the implementation would need further support. A series of seminars was offered to academic staff throughout semester 2 of 2016/17. These included external speakers and internal examples of good practice. While some of these were well attended, others were not. The framework was also presented to all Programme Directors (PDs) as part of a programmatic assessment event at the end of the academic year. This was to stimulate consideration of group work at a programme level. This event was well attended and positively evaluated.

It was important also to support students, particularly given the recognised need for scaffolding group work skills. The Student Learning and Development Centre (SLDC) developed a series of tutorials for students on how to work most effectively in a group, and these are now part of the Centre’s suite of tutorials offered to students every semester (sessions on conflict resolution with groups are also delivered upon request). These tutorials have been most effective when delivered to classes or groups in collaboration with lecturers, and with full attendance by all group members. However, they have not been well attended as stand-alone offerings, suggesting that support for students needs to be integrated into their modules, and as importantly, explicitly in their programmes.

Updates were, and continue to be, provided to student class representatives at several stages. The Students’ Union provides training and support for class representatives and organises regular meetings. The Assistant Registrar and Head of Learning and Teaching address these meetings several times a year to provide information and seek feedback on a range of issues and initiatives. The framework for assessed group work was highlighted in a number of these and representatives were encouraged to discuss group work at their programme boards. These meetings will continue to provide a basis for communicating to students about the framework and a platform for discussing its implementation.

Evaluating the impact of the framework

In order to assess the initial impact of the group work framework, we decided to engage with stakeholders to evaluate their awareness and experience of the framework, identify any changes made and capture areas requiring further support. Specifically, the evaluation sought to:

1. Assess awareness of the framework among staff and students.

2. Identify whether the framework has had any impact on the student experience of assessed group work.

3. Identify whether the framework has had an impact on the practice of individual lecturers.

4. Identify whether the framework has had an impact at the level of academic programmes.

5. Explore ways to improve and better support the implementation of the framework.

Methodology

The evaluation was grounded in pragmaticism (see Shannon-Baker, 2016). A concurrent mixed methods approach (Creswell & Plano Clark, 2007) was used to gather evidence from stakeholders within Dundalk Institute of Technology: academic staff, students and key respondents from the Registrar’s office and Student Services (see Figure 2). The mixed methods approach facilitated triangulation allowing for a deeper understanding of the outcomes. It also provided the flexibility to allow us to meaningfully engage with a wide range of stakeholders in a short period of time. Ethical approval was obtained from the Institute’s Research Ethics Committee (IREC) prior to commencing the study (see Appendices A, Band C for questions posed in each evaluation approach).

Figure 2. An overview of the stakeholders engaged throughout the evaluation.

Academic staff

An email invitation to participate in an anonymous online survey was sent to all academic staff in December 2017. The email contained a link to a 13 item SurveyMonkey questionnaire (see Appendix A). The closed and open questions probed awareness of the framework, the impact of the framework on respondents’ teaching practice, their perceptions of impact at the programme level and further support needed. No questions were mandatory. A final open-ended question asked for any additional comments or feedback.

There were 69 responses, a response rate of approximately 24%. We deliberately chose not to gather information about respondents’ departments and programmes to encourage open responses, ensure anonymity and avoid any perception that we were attempting to ‘police’ the implementation of the framework. The only information sought was about the respondents’ roles and whether they used assessed group work.

Student representatives

All student class representatives were invited to participate in a focus group to discuss the impact of the framework. Reminders were sent by a student representative. Seven representatives volunteered to participate in a lunchtime focus group and four of these turned-up on the day, representing 3 of the 4 academic schools. A sandwich lunch was provided.

The focus group was facilitated by a member of the working group who is not involved in teaching undergraduates. The questions asked participants about the issues around assessed group work reported to them by the students they represent; the discussion of group work at programme boards and identification of group projects in assessment schedules and their awareness and perceptions of the framework (see Appendix B). The focus group lasted 45 minutes. It was audio recorded and transcribed verbatim.

Key informants

The key informant technique originates in social anthropology but is increasingly widely applied in health and social research. A key informant is someone whose ‘ … formal role should expose them to the kind of information being sought by the researcher’ (Marshall, 1996, p. 92). We identified several key informants who could provide valuable information about the impact of the framework: the Assistant Registrar, the Students’ Union President, the Student Counsellors and Student Learning and Development Centre tutors. All agreed to participate

In an evaluation like this, the use of key informants provides significant advantages in terms of the range and depth of information that can be gathered relatively quickly. However, more importantly for this evaluation, these key informants could provide information at college-wide level, beyond individual experiences or specific programmes. All had valuable insight into important dimensions of impact.

The interviews were either individual (Assistant Registrar, Student Union (SU) President) or in pairs (Student Counsellors, learning development tutors) and lasted between 20 and 40 minutes. Informants were asked about issues with group work they encountered, any changes they had perceived over the past year, their awareness of the framework of their perceptions of its impact (see Appendix C). They were also asked for any further suggestions. They were not audio recorded; detailed notes were taken, a summary prepared and agreed with the informant.

Findings

In this section, the outcomes obtained from each stakeholder group are presented to illustrate the key elements being identified by each group. Triangulation of these elements did identify common aspects and recommendation areas, each of which will be detailed later in the article.

Staff survey

There were 69 responses; 68 lecturers and one Head of Department. The lecturers included 14 Programme Directors and 10 Stage (Year) Convenors.

Awareness: Table 1 shows that while a majority of the sample was aware of the guidelines, slightly fewer than half reported that they had read them.

Table 1. Awareness of the framework.

	Yes	No	Not sure
Aware of the framework	72% (n = 50)	15% (n = 10)	13% (n = 9)
Read the framework	48% (n = 33)	45% (n = 31)	7% (n = 5)

Impact on own practice : Respondents were asked: ‘Have these guidelines had any impact on your teaching practice?’ For example, in the design and implementation of your group work projects?’ A little under half indicated that they had, with 7 (10%) responding ‘Yes, definitely’ and 25 (36%) responding ‘yes, to some extent’. A further 26 (38%) responded ‘No, I’m not familiar with it’, 9 (13%) didn’t use group work and 2 responded ‘No, it’s not useful’.

Impact on programmes: A number of questions addressed the consideration of assessed group work by Programme Boards and teaching teams. A majority indicated that the level of assessed group work was considered by their programme boards, with 16 (23.5%) responding ‘yes, always’ and 26 (38%) responding ‘sometimes’. However, 11 (16%) responded ‘only if a problem is reported’ and 15 (22%) responded ‘no’. Respondents were also asked if the use of assessed group work at a programme level had been discussed within their programme teams during the past year. Responses indicated that this kind of strategic, programme level discussion was less common than discussions of the amount of group work. Just over half agreed that it happened for all (9, 13%) or some (26, 38%) programmes, while 43.5% (30) responded ‘no’ and the remainder were unsure. A majority (81%) reported that assessed group work was explicitly considered on some or all assessment schedules although only 40% felt that the framework had had an impact on this.

It is important to bear in mind that only minor changes can be made to validated programmes. However, all programmes offered by the institute are scheduled to be reviewed within the forthcoming academic year, as part of the 5-yearly Programmatic Review. Over half the respondents indicated that they were aware of plans to review assessed group work as part of this process on all (22%) or some (35%) of the programmes they were involved with.

Certain questions within the survey allowed participants to provide open responses. There were 42 responses to Q12, ‘Is there any support the institute could provide that would assist you with the implementation of the group work framework in programmes or in your practice?’ The single most common response (n = 13) was that nothing was needed and/or the guidelines were sufficient. The most frequently mentioned suggestions were for information/training (n = 10) and sharing practice/resources (n = 8):

‘Targeted support for group-work at discipline or departmental level would be useful’

‘A central resource where there are examples of best practice in running group work would be useful. I would find it useful if all staff throughout the institute had a forum to share practical ideas and have these ideas available centrally … ’

Other suggestions included time (n = 5) and management support (n = 5), these were linked in some cases.

‘Directors encouraging shared assessments between modules by providing the time and support for lecturers to develop these assessments.’

‘Time is needed for this, it takes time to read policies, guidelines, consider how to implement, pilot, evaluate, monitor, assess, mentor, guide etc. I like this guideline document as it is brief and easy to read.

A small number of responses (3) asked for further guidance

‘Further guidance in relation to attendance and absenteeism – what happens when students are absent for periods of time during group work projects.’

The final question allowed staff to present comments and feedback on the framework and guidelines for assessed group work. Excluding those that referred to the response to the previous question or responded ‘nothing’, left 13 (of 21). Seven of these were positive evaluations and 3 contained specific comments regarding the balance between group work and individual assessments, particularly in the final year.

‘Very clear, concise and can be easily applied to almost all contexts when developing modules and programmes with elements of assessed group work. Takes account of different perspectives and on going issues for both students and lecturers. Good guidance to support best practice’

‘I just had a quick scan of the guidelines and they certainly give a good overview of the area. They contain some good ideas, some of which I am trying and more of which I need to consider.’

‘Group work should not form part of the final year of any programme – a student’s final award should not be impacted upon or influenced by the performance of another student’

Four comments indicated that the respondents had not engaged with the framework, although most of these did intend to.

In summary, the survey suggests a high level of awareness of the framework among academic staff responses. Perhaps unsurprisingly, the number who reported having read the framework is considerably lower. Nonetheless it seems fair to say that awareness of the framework guidelines is developing amongst academic staff and it is particularly encouraging that almost half the sample reported an impact on their teaching practice.

There is less evidence of impact on programmes. Group work is being discussed in programme board meetings and it is being identified on assessment schedules thus promoting visibility. However only a minority of respondents perceived any impact of the framework at this level and there is little evidence here of strategic discussion at the programme level.

Student representatives

Generally, awareness of the framework was low and students were not able to say much about the impact. While two were somewhat familiar with the framework, only one recalled it having been discussed with them. All felt that it would be good for students to be more familiar with and involved with the guidelines. None had perceived any changes in assessed group work over the past year.

The student experience: There was a consensus that group work was an important part of the student experience and it was considered valuable, particularly in terms of employability skills.

‘The reason why we do it in college is to reflect real life situations … it’s a requirement in my opinion’

Echoing student feedback in the initial consultations, it was also evident that the volume of group work varied, and experiences were diverse. Some reported highly structured experiences, monitored by lecturers (including an example of using peer assessment) while others reported that lecturers were very ‘hands-off’. However, there was agreement that experiences are better when the lecturer has some monitoring role.

It was clear that group work remains an issue for many students with ‘free-riding’ and interpersonal difficulties being the main concerns. Notably students perceived effort on the part of fellow group membersas more important than perceived contribution

‘‘You can’t fault somebody on their ability, it’s always effort like … … … it’s a trusting in each other as well”

The programme level: The students reported that group work was explicitly identified as such on their assessment schedules. While group work was often raised as an issue with individual lecturers, these students rarely raised group work issues at programme boards, despite the fact that the volume and balance of group work across a stage and/or programme was identified as a concern in some cases. All discussed the need for students to be trained and supported; where this had happened, it had been very successful. All agreed that structure and support are needed, including peer support.

‘Something where students can be taught the team work skills, like good practice and bad practice … . You’re giving them the skills they can then apply … a lot of times when there is an issue the students don’t really know what to do … .’

‘Not just an hour in first year … we learn from our peers, there’s some things you’ve said today that I’m going to take forward … we need students’ opinions of what works, what doesn’t work as well …’

In summary, while the student representatives felt the framework was positive, they did not perceive much direct impact. Awareness of the framework among students was low.

Key informant interviews

From the interactions with key informants, the data were triangulated with common themes emerging, namely group formation, monitoring progress, programme impact and the growing institutional group work focus. Below, these themes are presented, all focusing on the impact on the student experience.

Impact on the student experience: Assessed group work continues to be a source of both positive and negative feedback at programme boards, however the key informant interviews provide some evidence of impact on the student experience. Both the counsellors and the SU President felt that there was a growing awareness among students and staff that the institute is actively engaged in efforts to improve students’ experiences of assessed group work. The SU president’s perception was that there had been an improvement in the experience of group work and this was attributed to a more positive conversation occurring about how to improve group work across programmes. The learning development tutors found that assessed group work was a significant concern for learners, compared to other issues. Perhaps most encouragingly, the counsellors reported that the issue was much less ‘on their radar’ this year, and that when it did come up, the issues were more likely to be interpersonal rather than frustration about the volume of group work or how it is being managed.

Impact on the Programme Level: The Assistant Registrar reviewed the annual programme monitoring reports (Programme Board reports) that had been submitted at the time of interview. These reported on academic year 2016/17. This evidence indicated that assessed group work remained a source of both positive and negative feedback from students. However, it was noted that there was greater evidence of analysis of group work at programme boards than in previous years. There was little evidence of explicit changes to modules or programmes in response to the framework, however, as noted earlier, it is difficult to make changes to validated programmes.

Other key informants stressed the need for assessed work to be considered and co-ordinated at the programme level. The SU President drew attention to the balance between group and individual work in the final year and recommended that programme teams consider less group work at this stage to allow students to demonstrate their individual strengths and abilities. The counsellors highlighted the need for consistency within programmes.

All informants provided significant feedback on the management of group work and the balance across a programme, particularly in the final year. The issues raised are addressed in the framework and the advice and suggestions generally supported and reflected the framework. The counsellors and learning development tutors also highlighted the need to adequately support students with learning agreements or other specific issues.

In summary, the evidence from key respondents indicates some success in raising awareness of assessed group work and starting a conversation about it. Feedback from the counsellors suggested improvements in some aspects of the student experience but it is clear that group work remains an issue. While there is little evidence of impact on modules and programmes this is unsurprising given the short time scale and the fact that all programmes are due for review next academic year. Nonetheless it is encouraging to see evidence of greater analysis at a programme level.

Discussion

This initial evaluation has demonstrated that, even within a short time frame, the framework has had some impact. This has been greatest in terms of raising awareness and visibility and there is also evidence of impact on lecturers’ own practice. While there has been some impact on the student experience and at the programme level, this is more limited. Although it targets the three inter-dependent levels identified by

Hénard and Roseveare (2012), as essential to support quality teaching – the institution, the programme and the individual- sustained support is needed to ensure a lasting impact.

Feedback from all stakeholders suggested that the growing dialogue within the Institute around group work, and ongoing work to improve practice through the framework document itself, as well as the initial implementation activities such as workshops, have led to a more positive view of group work. Staff and students seem to welcome the focus or ‘spotlight’ on the issue and are supportive of all efforts to improve the student experience of this form of assessment. Within the institute, as elsewhere (e.g. NFETL, 2017), there is increasing emphasis on programme approaches to assessment and group work has provided a useful focus here. The process of evaluation also helped to continue the conversation and increase awareness of both the framework and issues around group work.

The framework has been generally well received by academic staff and it is particularly encouraging that almost half the sample reported an impact on their teaching practice. Of course, caution is needed as it is likely that those who have actively engaged with the framework are over-represented in the sample of respondents; the levels of awareness and impact reported here are almost certainly over-estimates. Nonetheless, even within a relatively short time frame, the framework has had a direct impact on the work of some lecturers. It seems likely that meeting the demand for further training and resource sharing would increase this impact.

While individual lecturers each have a significant influence on group work at the module level, the framework emphasises the need to consider group work at the level of the programme and the feedback obtained from all stakeholders reinforced this. However, the evidence indicates that the framework has had a more limited impact at the programme level. Group work is being discussed in programme board meetings and it is being identified on assessment schedules thus promoting visibility, however the findings point to inconsistency across programmes in this and there is clearly scope for improvement. Only a minority of respondents perceived any impact of the framework on their programmes and there is little evidence of strategic discussion at the programme level. In many ways this is not surprising at such an early stage. As noted earlier, there are limited opportunities to make changes to validated programmes. All of our programmes are due to be revised the academic year following the evaluation as part of the five-yearly ‘programmatic review’ process and the impact of the framework should be clearer then. Over half the lecturers who responded to the survey reported that they were aware of plans to review group work as part of programmatic review on some or all of their programmes. Nonetheless, the fact that a significant minority are not suggests that further support may be needed to implement the framework consistently.

The impact on individual lecturers and programmes appears somewhat uneven, however, as we did not gather information about programmes or departments we cannot identify any patterns in this. While this can be seen a disadvantage, we feel strongly that it was important in framing the evaluation as a positive opportunity to explore the issues rather than a test of compliance. However, programmes and departments vary considerably in both the nature and volume of their group work. We would expect very little impact on programmes that use little group work. Equally, as we found in the initial consultation, there are many examples of well-established excellent practice at both individual and programme levels and again we would not expect to see a significant impact in these cases. It is also important to acknowledge that, while the survey provided a useful overview, it did not allow us to explore the questions in any depth. Further work is needed to explore engagement with the framework at a programme level.

The evidence from the SU President and Student Services indicated some possible improvements in the student experience at a macro level. This may indicate that some success in encouraging lecturers to take a more hands-on approach to managing and monitoring the process, particularly combined with the evidence that some lecturers have made changes to their practice, but further research would be needed to explore this. Certainly, it is clear that assessed group work remains an issue for students. Student awareness of the framework appears low and the student representatives did not perceive any real impact of the framework. Of course, it is difficult to generalise from such a small sample, however triangulating with the additional feedback from the key informants does support this. In many ways this is predictable as, despite the extensive consultation with students as part of its development, the framework was aimed mainly at academic staff. The benefits of engaging with students as partners in assessment are widely recognised (e.g. Deeley & Bovill, 2017) and it is clear from this evaluation that there is scope for much more meaningful engagement with students on an ongoing basis about this issue. This will be particularly important in informing programmatic review.

The process of conducting this evaluation also provided useful feedback on the framework. This will help to guide further improvements and demonstrates the need to continue working on this to improve the student experience and promote the development of key skills. The framework was generally positively received and evaluated by all groups of stakeholders. The fact that the framework asks staff to reflect on and discuss their group work approaches, rather than advocating a specific, prescribed method, was seen as an important aspect of its success. Evidence indicates that policies must be relevant to practitioners own contexts if they are to translate into quality practice (Mårtensson, Roxa, & Stensaker, 2014). It was striking that stakeholders reiterated and reinforced many of the elements of the framework. In particular, they emphasised the need to consider group work at a programme level and to plan how group work skills are proactively developed or scaffolded over the course of the programme. The need to balance group work and individual work, particularly in the final year, emerged very strongly as a concern for all. While this is implicit in taking a programme approach, the evidence suggests that this issue should be addressed more explicitly in the framework. Inclusivity, while implicit, would also benefit from more explicit consideration, particularly given evidence around the role of group work in intercultural education (Mittelmeier, Rienties, Tempelaar, & Whitelock, 2018; Moore & Hampton, 2015). Some staff also asked for specific guidance on difficult issues, particularly fails and repeat assessments. While the framework was not designed to provide procedural guidance, there does seem to be a need to explore these issues in more detail.

Conclusion and recommendations

Lessons learned from this evaluation suggest that it is important to include representatives from different stakeholders across the institute to more fully understand the issues and challenges involved and the different group work scenarios in different disciplines. This helped to avoid a ‘one size fits all’ solution. It also facilitated a sharing of group work techniques and tools between schools and departments that had not been previously there.

The framework encourages all programme development teams to consider the volume and appropriateness of group work at a programme level. However, at the module level, staff still have the flexibility to design group work assessments appropriate for different discipline areas, class sizes, and so on. Ongoing training is needed to support both staff and students if the principles suggested by the framework are to meaningfully influence practice in the long term Programmatic approaches to assessment continue to be an institute priority and this is reflected in ongoing efforts to build capacity amongst students and staff. The forthcoming programmatic reviews provide the key opportunity to reflect on and systematically review assessment, including group work, at the programme level and this should help to embed the framework.

This evaluation has demonstrated the need to engage more actively with students on this issue and we will be actively seeking ways to work in partnership with students to develop this framework further. We also need to continue to support staff in planning, managing and assessing group work, however the responses suggest that training may need to be targeted. We are in the process of developing, with the Institute’s Centre for Excellence in Learning and Teaching (CELT), a repository of group work resources that can be freely shared. The evaluation also has provided another opportunity to identify examples of good practice and we are exploring ways to develop these as case studies.

In conclusion, the development and implementation of this framework provided an opportunity to focus attention on a specific aspect of assessment at an institute level and start a conversation about programme level assessment. The evaluation suggests that the framework is beginning to have a positive impact on both the staff and student experience of group work. While the greatest impact is on awareness this is to be expected in the short term and there are encouraging early indications of wider impact. Nonetheless, the evaluation demonstrates the need for ongoing support in interpreting and implementing the framework, particularly at a programme level.

Acknowledgments

The authors wish to thank all staff and students in Dundalk Institute of Technology (DkIT) for their significant contributions to the initial research for creating the Framework and their subsequent evaluation of its implementation. They also wish to acknowledge the support of DkIT and the National Forum for the Enhancement of Teaching & Learning in Higher Education for their support and the dissemination of the initial work.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. Higher Education in Ireland is provided by Universities, Institutes of Technology and private colleges. The fourteen Institutes of Technology have an applied and practical focus.

2. The committee that manages an academic programme.

3. See https://www.teachingandlearning.ie/resource/case-study-d-a-framework-for-assessed-group-work-development-and-initial-implementation-of-an-institutionwide-approach-dkit-2/.

Appendix A

Staff survey questions

1. About you. Please tick all that apply

• I am a lecturer

• I am a stage convenor

• I am a Programme Director

• I am a Head of Department/Section

2. Are you aware, or have you heard, of the DkIT Framework and guidelines for assessed group work?

• Yes

• No

• Unsure

3. Have you read this Framework and Guidelines document? (It is available: https://www.dkit.ie/system/files/groupwork_framework_guidelines_2016.pdf)

• Yes

• No

• Unsure

• Any comments

4. Do you use assessed group work in your teaching currently? Please select all that apply

• I currently use it

• I have used it in the past

• I intend to use it in the future

• Any comments?

5. Have these guidelines had any impact on your teaching practice? For example, in the design and implementation of your group work projects?

• Yes, definitely

• Yes, to some extent

• No, it’s not useful

• No, I’m not familiar with it

• I don’t use assessed group work

• Please explain

6. Is the level of assessed group work considered at programme boards for the programmes you are involved with?

• Yes, always

• Sometimes

• Only if a problem is reported

• No

7. Are group assessments explicitly identified on the continuous assessment (CA) schedules for the programmes you are involved with?

• Yes, on all programmes

• Yes, on some programmes

• No, not on any programmes

• Unsure

8. In your opinion, has the framework for assessed group work had any effect on whether/how it is considered at the programme boards you are involved with?

• Yes, on all programmes I’m involved with

• Yes, on some programmes I’m involved with

• No, not on any programmes I’m involved with

• Not sure

• Any comments

9. Have you discussed the use of assessed group work at a programme level with your Programme Team(s) during the last year?

• Yes, for all programmes I’m involved with

• Yes, for some programmes I’m involved with

• No

• Not sure

10. In your opinion, have these guidelines for assessed group work had any impact on the programme/s you are involved with?

• Yes, a good deal

• Some

• Very little

• None

• Unsure

• Please explain

11. Are you aware of plans to review assessed group work on the programmes you are involved with as part of the next programmatic review?

• Yes, for all programmes I’m involved with

• Yes, for some programmes I’m involved with

• No

• Not sure

• Other (please specify)

12. Is there any support the institute could provide that would assist you with the implementation of the group work framework in programmes or in your practice?

13. We would welcome any comments or feedback you have about the framework and guidelines for assessed group work.

Appendix B.

Student focus group questions

1. Is group work an important issue for the class group that you represent?

2. How does group work affect the student experience?

3. Have you raised the issue of group work at Programme Boards you are involved with over the past year?

4. Are concerns about grading/volume/(un)fairness dominating discussions – or – are recommendations for improved group work format ever mentioned?

5. What specific issues do your fellow students inform you about when they come to you regarding group work?

6. Are there any positive elements of group work you have encountered in a particular module that you would like to see implemented in other modules?

7. Has group work been explicitly identified on CA Schedules for programmes you have received with over the past year?

8. Were you aware that, last year, DkIT published staff guidelines for assessed group work?

9. Have group work processes employed by lecturers improved your classmates’ experience of group work over the past year?

Appendix C.

Interview questions

1. In your role (as X), what issues do you come across in relation to assessed group work?

2. How would you describe the student experience of assessed group work here at DkIT?

3. Do you think there have been any changes in the past year or so? (Prompt for explanation)

4. Are you familiar with the Framework and Guidelines for Assessed Group work?

5. Do you think this has had any impact? (prompt: on the student experience? On staff?)

6. How do you think that the student experience of assessed group work could be improved?

7. Is there any other feedback you would like to give on assessed group work and/or the framework?