What is the structure of a Challenge Based Learning project? A shortitudinal trajectory analysis of student process behaviours in an interdisciplinary engineering course

ABSTRACT

Challenge Based Learning (CBL) is an educational approach that has gained popularity in response to the need for authentic learning environments. While the CBL literature is predominantly focused on cases of pedagogical implementations, the actual processes by which students develop CBL projects remain under-investigated. This shortitudinal study seeks to examine the phases of CBL project development and associated process behaviours at group level, as they unfold. The participants are 6 interdisciplinary student groups totalling 22 students enrolled in a first-year course on ethics and data analytics. Data was collected weekly throughout 10 weeks via reflective diaries (n = 15 students) and observation of course sessions (n = 22) and is complemented by interviews after course completion (n = 15). The data was subject to a thematic trajectory analysis. The study identifies 7 distinct phases in the temporal structure of a CBL project: gaining client know-how, articulating a problem, mapping the problem context, setting the aim, proposing an action path, testing and evaluating it, and implementing the solution. The article concludes with recommendations for further research into CBL project development processes, which may support the growing adoption of real-life interdisciplinary projects in engineering education.

KEYWORDS:

• Challenge Based Learning
• interdisciplinarity
• sociotechnical engineering education
• shortitudinal study
• thematic trajectory analysis
• group projects

1. Introduction

Engineering practice takes place in ill-structured interdisciplinary contexts that offer little clarity or specificity as to the nature of the tasks involved (Wolff, 2018). Engineering design is open-ended with respect to the problem, its solution, and the process by which it is addressed (Lammi, Denson, and Asunda 2018). Engineering projects require not only problem-solving but also problem-setting. Nevertheless, the undergraduate engineering curricula prioritises solving well-structured problems, focused on providing the right answer via a predefined heuristic (Martin, Conlon, and Bowe 2021a). Well-structured problems require different cognitive skills than those needed to engage with ill-structured design contexts (Isaac 2021; Dringenberg, Guanes, and Leonard 2021). Engineering practice gives rise to ‘wicked problems’ (Lönngren & van Poeck, 2020), which cannot be solved solely through rational­istic or deductive means (Martin, Conlon, and Bowe 2019; Douglas et al. 2012).

Educational research recognises the importance of active learning environments that prepare students for cooperative and work-based learning through wicked problems of an interdisciplinary nature (Gutiérrez Ortiz, Fitzpatrick, and Byrne 2021; Tormey et al. 2021). In engineering education, efforts are made in this direction through experiential approaches such as community service learning (Lucena, Schneider, and Leydens 2010), humanitarian engineering (Nieusma and Riley 2010; Mazzurco and Murzi 2017), eco-design projects or competitions (Leal et al. 2020), Project Based Learning (Kolmos et al. 2023), or more recently, Challenge Based Learning (Bombaerts et al. 2021; Membrillo-Hernández et al. 2019).

Challenge Based Learning, or CBL for short, is an educational approach that grew in popularity in the last decade in response to the need to expose engineering students to more authentic learning environments (Leijon et al. 2022). As such, CBL strives to build on the aim of Project Based Learning of connecting students’ disciplinary knowledge with professional practice by bringing key features of the engineering workplace into the classroom ( Sukackė et al. 2022). In CBL, the focus is on the

‘identification, analysis and design of a solution to a sociotechnical problem. The learning experience is typically multidisciplinary, involves different stakeholder perspectives and aims to find a collaboratively developed solution, which is environmentally, socially and economically sustainable.’ (Malmqvist, Rådberg, and Lundqvist 2015; italics added)

Thus, students have an authentic exposure to the type of problems and multi-stakeholder needs, expectations and interactions they may encounter in engineering practice (Martin, Conlon, and Bowe 2021a).

Although there are currently several CBL frameworks positing its phases (Nichols, Cator, and Torres 2016; Ambrosi and Hermsen 2023), the structure of CBL projects and associated processes remains under-investigated empirically or is adopted tacitly (Perna, Recke, and Nichols 2023, 16). As such, despite the growing literature depicting descriptive cases of pedagogical implementations of CBL (Gallagher and Savage 2023, 1141), there is still little known about the actual processes by which students develop CBL projects (Jimarkon 2022). Examining students’ process of developing real-life interdisciplinary projects via CBL becomes crucial given that learning experiences and outcomes are a central concern of the academic field of higher education (Liu, Li, and Hossain 2022). The focus on student learning in CBL that emerged in the last decade (Berland et al. 2013; Kohn Rådberg et al. 2020) would benefit from anchoring such processes temporally, to capture the dynamic nature of this phenomena. The study responds to this gap by aiming to identify the main phases of project development in CBL and their characteristic student process behaviours.

2. Background

2.1. Characteristics of Challenge Based Learning projects

CBL is considered particularly suitable for STEM education (Gallagher and Savage 2023, 1145). In fact, the literature review by Leijon et al. (2022, 611) found that CBL is most prominent in engineering education of all higher education disciplines. This can be due to the characteristics of the projects that students develop in CBL, which strive to resemble those encountered in the workplace. A CBL project can be defined as an educational challenge based on a real-life problem proposed in an academic setting by a stakeholder, such as a company, government, NGO, research group, or community members. CBL fosters an open-ended learning environment via the interaction between students and university partners for the development of sociotechnical projects linked with the partners’ activity (Kohn Rådberg et al. 2020).

A CBL project has the following characteristics: it is authentic, ill-structured and open-ended, ambiguous, collaborative, sociotechnical, interactive, embedded and embodied in the university ecosystem. All these features make CBL a particularly complex pedagogy, and it is important to understand the intricacies of how students develop projects to better support them.

2.1.1. Authentic

A CBL project is authentic. Authenticity refers to the projects’ resemblance or connection to the activities of real-world professionals (van den Beemt, van de Watering, and Bots 2023). Such projects encourage students to leverage the technology used in their daily lives to solve real-world problems (Nichols and Cator 2008, 1). They often require students to analyse, design, develop, and execute solutions for real-life problems, similar to those they may encounter in the workplace (Pérez-Sánchez, Chavarro-Miranda, and Riano-Cruz 2023).

2.1.2. Ill-structured and open-ended

CBL projects put forward a problem area that needs to be narrowed before being addressed, rather than a problem to be solved. The projects have an ill-structured and open-ended nature that requires students to be not only problem solvers, but also problem definers and problem framers. A CBL project is not based on a clear problem and envisioned solution, allowing varying solution paths (Brophy et al. 2008).

2.1.3. Ambiguous

The problem area of a CBL project can be ambiguous, as the specific issues to be addressed are initially unclear. The ambiguity of CBL projects seeks to make learners open to a vast range of possibilities and curious about the world around them, encouraging the deployment of divergent and lateral thinking (Nichols 2023).

2.1.4. Collaborative

CBL facilitates the acquisition of skills through collaborative effort when developing a project (Sternad 2015, 252). Students work not only with peers, but also with teachers, experts and external stakeholders (Nichols and Cator 2008, 1). External stakeholders take the role of a ‘real client’ to whom students can relate (Morselli and Orzes 2023), a ‘co-creation partner’ in the development of the solution for a problem linked to their line of activity (Bombaerts et al. 2021), or a ‘training partner’ who offers instructions to students in the development of on-site solutions, is present throughout the process and assesses the deliverables alongside teachers (Membrillo-Hernández et al. 2019).

2.1.5. Sociotechnical

A core component of CBL is the focus on addressing a problem area with the knowledge and methods of distinct disciplines (Mesutoglu and Bayram-Jacobs 2022). Most often, CBL thrives when it is based on ‘collaboration across difference’ (Cruger 2018, 100). This means that CBL requires the integration of skills and perspectives from different areas. In the case of CBL, interdisciplinarity manifests as a sociotechnical endeavour, bridging scientific and technical disciplines with non-technical disciplines such as ethics, communication, or leadership (Malmqvist, Rådberg, and Lundqvist 2015; Membrillo-Hernández et al. 2019; Hart and Randall 2005). Grand societal challenges are a prime example of sociotechnical projects tackled via CBL (Lara-Prieto et al. 2019).

2.1.6. Embedded and embodied

As Membrillo-Hernández et al. (2019) point out, irrespective of the degree of the involvement of a stakeholder in CBL projects, the thematic focus is nonetheless strongly linked with the features, needs or problems of the ecosystem in which the university is situated and its actors. CBL projects are embedded in the features of the university ecosystem. Furthermore, real-life issues are revealed through this interaction, which go beyond the bounded classroom setting. There is a potential of positive impact on other people, and this impact can encourage students to embody specific personal or professional behaviours, such as responsibility or activism (Cruger 2018).

2.1.7. Interactive

CBL involves stakeholders from academia, industry, or other societal actors (Kohn Rådberg et al. 2020). A CBL project involves some sort of interaction between students and the university ecosystem. This interaction contributes to the dynamic character ofCBL projects, which lead to a rich learning experience and foster growth and change for students and stakeholders alike.

2.2. CBL frameworks and structure

Currently, there are several CBL frameworks articulating a structure for the phases of project development (Table 1). According to Perna, Recke, and Nichols (2023), the most popular framework is the one developed for Apple by Nichols, Cator, and Torres (2016). It consists of three phases - Engage, Investigate, and Act -, each divided into several sub-phases. The framework has been adopted in various institutional settings across the world, such as Dublin City University (O’Riordan and Gormley 2021) or Hamburg University of Technology (n.d.), and is at the basis of the TEC21 Model of Technological University of Monterrey (Gutiérrez-Martínez et al. 2021) and the Challenge Based Learning Framework of the European Consortium of Innovative Universities (Ambrosi and Hermsen 2023) as well as the corresponding CBL ECIU Toolkit developed by University of Twente (Imanbayeva 2021). Under the name of Challenge Based Instruction, researchers from Vanderbilt, Northwestern-Harvard and MIT Engineering Research Center (VaNTH ERC) developed the STAR Legacy Cycle which has 6 phases (Cordray, Harris, and Klein 2009). All the different CBL frameworks mention the student process behaviours expected for each phase.

Table 1. CBL frameworks with phases and associated process behaviours.

Framework	Phases	Student process behaviours
Apple (Nichols, Cator, and Torres 2016)	1. Engage	Defining the problem Questioning personal interests and the community’s needs Identifying a compelling and actionable challenge statement.
	2. Investigate	Generating and prioritising questions to address the challenge Identifying and using resources or activities to answer the questions Analysing the accumulated data and synthesising it into themes Communicating the proposed solution
	3. Act	Developing solution concepts Developing prototypes, experiments and tests Implementing the solution Measuring outcomes Reflecting on the process Determining the impact on the challenge Refining the solution Communicating the solution
Tec 21 (Gutiérrez-Martínez et al. 2021).	1. Theoretical background	Reviewing and researching the theoretical background
	2. Initial challenge situation	Exploring the challenge situation Defining with the teacher and external stakeholder the objectives, variables, sample size and deadlines for the challenge
	3. Established challenge situation	Understanding, analysing and identifying key elements of the challenge Identifying the process threats and opportunity areas
	4. Guiding activities	Setting a roadmap to address the challenge
	5. Proposed solutions	Generating and analysing proposed solutions
	6. Implementation	Implementing the solution
	7. Conclusion	Evaluating the solution
	8. Continuous improvement	Synthesising findings Communicating findings to the external stakeholder Reflecting on the process
ECIU Challenge-based Learning Framework (Ambrosi and Hermsen 2023) and the CBL ECIU Teamcher Toolkit (Imanbayeva 2021).	1. Engage: Big Idea & Essential Questions	Phrasing essential questions Identifying stakeholders
	2. Engage: Actionable Challenge	Articulating a challenge statement Developing an action plan Identifying personal tasks and roles
	3. Engage: Finalising Actionable Challenge	Asking feedback from the external stakeholder Refining and finalising the challenge definition Setting personal learning objectives
	4. Investigate: Guiding Questions, Activities and Resources	Researching resources, methods and tools related to the challenge Asking stakeholders questions about the challenge
	5. Investigate: Synthesis	Analysing the accumulated data and synthesising it into themes Communicating findings
	6. Act: Solution Concepts and Development	Engaging in design thinking Defining solution requirements Brainstorming possible solutions and solution designs Evaluating different solutions Choosing a final solution Communicating the solution
	7. Act: Solution Implementation	Implementing the solution
	8. Act: Evaluation and sharing	Evaluating the results Deriving conclusions from the process and from the content Communicating the results
STAR Legacy Cycle (Cordray, Harris, and Klein 2009).	1. Challenge Scenario	Discerning which information or activities are relevant to the task
	2. Generate ideas	Eliciting prior knowledge in connection to the challenge
	3. Multiple perspectives	Accessing resources and experts to obtain knowledge Asking questions
	4. Research and revise	Doing field and lab/in-class tests Researching the challenge
	5. Test your mettle	Self-assessing research Self-evaluating learning needs Overcoming misconceptions and gaps in knowledge
	6. Go public	Communicating the solution

3. Setup of a Challenge Based Learning course on ethics and data analytics

The course E3Challenge2 is a first-year course on ethics and data analytics offered by TU Eindhoven. In the academic year 2020/21, 45 students enrolled in the course, representing 10 study programmes: Mechanical Engineering (9), Computer Science (8), Electrical Engineering (6), Chemistry Engineering (6), Industrial Design (4), Applied Physics (3), Automotive Engineering (3), Sustainable Innovation (2), Psychology & Technology (2) and Industrial Engineering (1).

The course was delivered online due to restrictions imposed by the COVID pandemic. The CBL format fostered the conditions for students to make decisions that allow for multiple possible solutions to interdisciplinary problem areas involving stakeholders active in the university’s ecosystem, who took the role of ‘clients’. Students were grouped in interdisciplinary teams of four-five and asked to develop a project connected to the activity of the ‘client’ in a problem area related to smart-mobility, smart-health, or smart-energy (Table 2). The projects unfolded in parallel over 10 weeks and were integrated into an end assignment that required students to develop a solution for the client to a problem they identified.

Table 2. Real-life interdisciplinary project areas for the CBL course E3Challenge2.

Challenge	Description
Smart mobility	A student team that is part of a European large-scale pilot project investigating the value and potential benefits of adding autonomous vehicles to a worldwide 5G network. The team works on enabling the control of a car from a remote station using an internet connection and feedback from various sensors on the car. Some of the car’s sensors that are necessary for autonomous control cease working and a user must take control of the car from a remote location.
Smart health	The project is a collaboration between the university, a medical centre and a company. It applies the sciences of data learning and biomedical simulations to an existing diabetes gaming platform. The collaboration aims to develop a data-driven, personalised serious game that empowers individuals with diabetes to manage their disease.
Smart energy	The student team collaborates with the university on providing insights into how the campus can be more sustainable. The team aims to develop a model to simulate the main campus buildings, which will allow users to add and configure various technologies such as solar panels, wind turbines, and charging points.

The course gathered an interdisciplinary team comprising fourteen teaching staff and three stakeholder groups taking the role of clients. The roles of the teaching staff were distributed as follows: one ethics lecturer, three coaches, five expert generalists, and nine teaching assistants (master students in ethics or data-analytics).

There were five types of learning activities dedicated to students, in addition to two weekly peer-to-peer meetings for the teaching team, as seen in Table 3.

Table 3. Course organisation of an interdisciplinary CBL course in ethics and data analytics.

Course activity	Role	Members of the course team
weekly 2-hour ethics sessions	Delivering ethical content via lectures Practicing ethical reflection on topics relevant to the case and assignments	Ethics lecturer
weekly 1-hour support sessions	Assisting groups in the development of the project and giving process feedback	Teaching Assistants in data analytics and ethics
weekly 30-minute coaching sessions	Discussing weekly reflections on group progress and processes for developing the project Delivering process-related feedback and guidance for individual or groupwork	Coaches (including the course coordinator and the data analytics lecturer)
weekly 1-hour expert meetings	Answering questions and giving expert feedback on technical and ethical aspects related to the project	Technical, ethics, educational experts and sometimes clients
three client meetings	Meeting 1 (week 1): introducing the case Meeting 2 (mid-course): delivering feedback on the tentative solution plan Meeting 3 (week 9): Reacting to and evaluating the solution implemented	Stakeholders Lecturers Coaches (including the course coordinator) Teaching Assistants
two weekly 1-hour peer-to-peer reflective meetings for the teaching team	Meeting a: supporting and guiding the work of teaching assistants Meeting b: discussing notable aspects or challenges occurring during the week and planning ahead	Meeting a: coaches, ethics lecturer, teaching assistants Meeting b: lecturer, experts, coaches, researchers

4. Methodology

The study reported in this article is part of a broader project investigating student experience in CBL (see Bombaerts et al. 2022; Martin and Bombaerts 2022; Doulougeri et al. 2022). The studyexplores the research question: what are the phases for developing an interdisciplinary CBL project and their characteristic processes, as they unfold throughout the course? To address this question, we conducted a thematic trajectory analysis of data collected via shortitudinal qualitative methods.

4.1. Shortitudinal trajectory study design in an engineering education setting

The study is a temporal investigation into CBL project development in a first-year interdisciplinary engineering course on Ethics and Data analytics. As it follows a student cohort at regular intervals across a short period, throughout a 10-week course, it can be classified as a shortitudinal study. This term was coined by Saldaña (2003, 34–5): unlike longitudinal studies, which follow participants over long periods with a relatively lengthy time lag between measurements, shortitudinal studies unfold over a shorter period and with a relatively shorter time lag between measurements. Considering the optimal time lag between measurements, although no research on this matter has been conducted in educational settings, research in other disciplines involving human participants such as psychology has indicated that ‘far shorter time intervals than those frequently found in the literature are justifiable’ (Dormann and Griffin 2015, 489). This has recently led to calls for the adoption of more shortitudinal studies (Dormann and Griffin 2015), which are considered especially relevant when it comes to capturing dynamic phenomena (Griep et al. 2021). While in the last five years, there was an increase in the number of shortitudinal studies in medical and social sciences, the adoption of this approach in education remained scarce. In fact, a search conducted on Google Scholar identified only one such study, focused on mathematics education at pre-university level (Putwain and Wood 2022). Hereby, we report on the design and application of a shortitudinal study in engineering education, hoping to offer guidance to other teachers and researchers looking to understand dynamic phenomena in their classrooms or educational activities.

For this study, we use a qualitative trajectory approach, which is particularly suited for shortitudinal research (Join-Lambert, Boddy, and Thomson 2020; Spencer et al. 2021). It takes into account temporality, thus facilitating a deep understanding of individuals’ progress, processes, and experiences over time (Grossoehme and Lipstein 2016). In engineering education research, trajectory approaches have been frequently used as part of longitudinal quantitative studies with a large participant sample (see Sheppard et al. 2004; Miller et al. 2019; Perkins et al. 2021). Nevertheless, findings from medical research suggest that qualitative trajectory methods can also provide meaningful insights, especially when examining lived experiences, including identifying the characteristics of specific interactions and the reason behind individuals’ choices and decision-making (Grossoehme and Lipstein 2016). This prompts us to adopt a qualitative trajectory approach in our study given its fit with the dynamic phenomenon investigated and the small sample size confined to students enrolled in one course. We note that qualitative trajectory approaches in the context of shortitudinal studies are not represented in engineering education research, which may be an important omission given their potential to generate hypotheses for future research and contribute to improvements (Grossoehme and Lipstein 2016, 1).

Following the guidelines provided by Grossoehme and Lipstein (2016, 2), a trajectory study is designed according to the following principles: (i) the research focus is on how processes or experiences change over time; (ii) the sample maintains the same participant cohort; (iii) the level of analysis is individual people or groups, (iv) the analysis is conducted when data collection is complete, and (v) the analytical approach is informed by the research question and used consistently throughout the study. In addition, as a shortitudinal study, the design involved (vi) data collection at regular intervals, (vii) during a short period.

The methods used for this shortitudinal trajectory study comprise document analysis of student reflections (submitted weekly as course components), participant observation (of weekly support sessions and coaching sessions throughout the course), and interviews (conducted at the end of the course, after the last graded deliverable was submitted). The protocol for data collection is described in Table 4.

Table 4. Protocol for shortitudinal qualitative data collection.

Collection method	Focus of data collection	Data description	Collection timelag
Reflections	Process behaviours	How is project progress described or discussed What individual and group processes are mentioned What strategies are mentioned to overcome challenges or make progress	Once weekly, for 10 weeks
	Process experiences	What individual and group challenges are mentioned What individual and group feelings are mentioned What learning experiences are mentioned and what contributed to these
	Temporal markers	When are behaviours and experiences occurring When are past behaviours and experiences described to have occurred
Observations	Verbal process behaviours and interactions	Who speaks to whom How the division of labour is set and what tasks are distributed What and whose suggestions are followed up How is project progress discussed What questions are asked and what are the answers received What feelings are mentioned	Twice weekly (during support sessions and coaching sessions), for 10 weeks
	Physical process behaviours	What people do and who does what Technical behaviours for developing the project Ethics-related behaviours for developing the project
	Process experiences	What challenges and difficulties are mentioned What people’s behaviours may indicate about their emotions
	Temporal markers	When are behaviours and experiences occurring When are past behaviours and experiences described to have occurred
Interviews	Process behaviours	What technical behaviours for developing the project are mentioned What ethics-related behaviours for developing the project are mentioned How is the step-by-step process fordeveloping the project described What project values and stakeholders are mentioned How is the integration of ethics and technical aspects described How the process for deciding the solution is described Mentions of changes in project development	Once, after the end of the course
	Process experiences	What were the most difficult aspects when developing project How was the experience working on a real-life project How was ethics experienced for the project How important was ethics and when did awareness of the importance of ethics occur Ethics issues identified during the process How was the interaction with the client perceived and the helpfulness of their feedback What resources or external support were used to develop the project
	Personal views	Satisfaction with the project developed What course sessions and support were the most useful Views about the role of ethics in engineering now and at the beginning of the course
	Temporal markers	When are behaviours and experiences described to have occurred

The rationale for using several methods at regular collection moments was to better capture the dynamic data purporting to students’ progress in developing their projects. The use of multimethods allows to triangulate findings between students’ self-reported process behaviours for developing the CBL project, as rendered in the reflective course components, and the researcher’ interpretation of these processes, captured via observation. This contributes to higher accuracy of the data analysis for enhanced quality control.

4.2. Document analysis of reflective diaries

Shortitudinal trajectory studies are particularly suited for collecting data using written materials generated by the participants, such as diaries (Liebenberg, Scholtz, and De Beer 2022). In engineering education, reflections have become popular and are deployed as reflection diaries or journals, portfolio reflections, end-of-course meta-learning, peer-assisted learning sessions, online reflections, or team reflections (Kavanagh and O’Moore 2008).

Reflection diaries enable students to collect evidence of their learning and of individual and group processes (Christy and Lima 1998). Asking students to explain their reasoning and processes regularly via reflection diaries facilitates making connections between learning activities, goals, and processes (Wallin and Adawi 2018). As such, these documents can measure growth and progress in a specific area or discipline (Spencer et al. 2021). As Spencer et al. (2021) note, diaries fit the purpose of a trajectory investigation as they can render both the ‘down’ and ‘across’ of qualitative data by bringing both the richness of data captured in the moment and recording change over time.

For this study, the document analysis used diaries comprising 10 weekly reflections submitted as a course assignment. These were prompted reflections following the DEAL Model for critical reflection developed by Ash and Clayton (2004) and the ‘What? So What? Now What?’ reflective framework developed by Rolfe, Freshwater, and Jasper (2001), whereby students considered their weekly progress and process behaviours, what they learned from this experience, and their plans for next week.

4.3. Participant observation

Participant observation aims to closely examine the activity within a community, via note-taking and asking questions to uncover the meaning behind the behaviours observed (Guest, Namey, and Mitchell 2013). For a shortitudinal study, Saldaña (2003, 34–35) considers apt the strategy of gathering data by following an event during its total period, from beginning to end.

For this study, the first author collected behavioural data via observations at two points every week: during the support sessions and during coaching sessions, when student groups discussed the reflections with their coach. The observations were complemented by the second author’s internal perspective articulated during the weekly peer-to-peer sessions, bringing greater depth to the analysis (Fleming 2018). From the four stances of participant observation theorised in Gold’s (1958) seminal work, the first author embodied the observer as participant. She was not a member of the groups of students and teachers under observation, but her participation in the course activities was known to them and allowed. According to this stance, the researcher’s main role is to collect data, which is used to provide an enhanced understanding of the group's activities and improve it. This stance is considered the most ethical towards the participants (Kawulich 2005).

The observer followed the major guidelines for conducting participant observation, such as being upfront about conducting the study, seeking informed consent, noting comments and occurring interactions (Kawulich 2005). The observation process followed a pre-defined template based on Mack et al. (2005), which was adapted to fit the study aim. This method enabled the first author to examine at regular weekly intervals students’ process behaviours, decision-making and project progress from session to session and juxtapose it to the weekly reflections. The observations were used to identify the number of distinct phases of developing a CBL project which were confirmed later by interviews.

As the course took place remotely due to the pandemic, approximately 90 h of observations were conducted online. The data analysis resulting from participant observation relied on live note-taking and retroactively viewing the recordings of the course sessions. This ensured a close and continuous observation of students’ processes in their own learning environment and in the temporal progression of the course, which was double-checked after the course ended.

4.4. Interviews

Interviews were conducted in the final week of the course when the contact moments with teachers ended. The format was semi-structured (Adams 2015, 493). The authors agreed on the formulation of the questions, which were partly informed by processes and experiences mentioned in the reflections and observed during the course. The questions aimed to generate rich descriptions of students’ experiences of developing a CBL project and their interpretation of these experiences. All interviews were conducted by the same person (first author) to ensure a consistent application of the interviewing technique and protocol, towards an enhanced process reliability.

4.5. Thematic trajectory analysis

Qualitative analysis methods are considered to have lagged behind their quantitative counterparts when it comes to exploiting the temporal nature of data (Spencer et al. 2021). Thematic trajectory analysis has emerged as a response to this deficit.

Grossoehme and Lipstein (2016, 3–4) recommend that a thematic trajectory analysis should be constructed as a X-Y matrix that identifies themes (on one axis) and temporal units of analysis (on the other axis). The focus of the analysis is on how the data in the thematic groupings changed or did not change over time. Grossoehme and Lipstein (2016) draw attention that as the coding analysis progresses, new conceptual groupings may be needed as new, time-related concepts emerge. In the case of this study, the final time-related concepts that emerged during the coding received the name ‘project phases’ to signify distinct sets of process behaviours.

The data was analysed with the software Atlas.ti, which allows to identify code co-occurrence:

First, the data gathered from each individual participant was analysed, and the behaviours mentioned or observed received unique codes related to process behaviours as well as codes marking the temporal occurrence (example in Table 5). The coding of behaviours was based on guidance provided by Saldaña (2021). The temporal codes marked the week in which the process behaviour was mentioned or observed.

Table 5. Coding examples of process behaviours.

Code	Definition	Usage	Exclusion	Example	Occurrence
Evaluating tests for ethical faults	Process behaviour referring to students’ ethical evaluation of tests.	When students evaluate the tests for ethical faults.	When students mention an evaluation of the tests that doesn’t consider ethics.	‘For the test, we would first start with a base solution, like a data-based solution and then we would evaluate that solution to see what potential ethical breaches could be’.	The week when students evaluated the tests for ethical faults or mentioned evaluating tests for ethical faults.
		When students mention that or how they evaluated the tests for ethical faults.	When students mention an ethical evaluation of the final solution
Agreeing on the deliverable	Process behaviour referring to students agreeing on a deliverable.	When students express agreement on the deliverable to be pursued.	When students discuss potential deliverables, without agreeing on one.	‘We were pretty much straightforward and we were all agreeing on the deliverable. There wasn't any discussion because we said the same and that was it’.	The week when students agreed on a deliverable.
			When students agree on other outputs (i.e. subdeliverables, process tasks a.o.).

Second, emerging time-related concepts were identified, which were named ‘project phases’. These were coding categories that grouped several process behaviours (as subcodes) based on their temporal occurrence. To subsume process behaviours under specific coding categories, we were guided by the hierarchical coding structure of phases and associated student process behaviours posited by existing CBL frameworks (per Table 1). To identify distinct process behaviours for the development of the project, behaviour codes repeating for three weeks or more were excluded. This was the case with codes that indicated project management behaviours occurring every week (i.e. meeting with the team members, moderating meetings, note-taking a.o). With a similar aim of focusing on processes related to project development, we excluded other behaviours indicating team management (i.e. team member introductions, addressing freeriding, solving team conflicts, conversations about personal matters a.o.).

Third, given that the CBL projects were developed in groups, the individual process behaviour codes were organised by student group. We used Atlas.ti to identify the co-occurrence of temporal codes (i.e. the ten weeks of course) and process behaviour codes for each group. We then analysed the trajectory of each group considering when process behaviours occurred, from project phase to phase, throughout the course.

Fourth, we developed with Atlas.ti two Sankey diagrams to confirm the identification and temporal unfolding of project phases, based on the coded process behaviours and temporal units (represented by the 10 weeks of the course) as described above. Sankey diagrams contain nodes (representing codes) and arcs that flow from source nodes to target nodes (marking co-occurrence of codes). The size of each node and width of each arc are indicative of the number of codes. These features make Sankey diagrams a powerful visualisation tool to depict change and trajectories, having been used in medical research to illustrate pacient trajectories (Lamer et al. 2020). Sankey diagrams can be interpreted based on the direction of flow and width of the arcs, which point to the frequency by which specific codes co-occur. In Figure 1, the arcs flow from source nodes (representing each occurrence of process behaviour codes) to target nodes (project phases). In Figure 2, the arcs flow from source nodes (project phases) to target nodes (temporal units).

Figure 1. Occuring CBL process behaviours for each project phase (generated with Atlas.ti).

Figure 2. Temporal occurrence of CBL project phases (generated with Atlas.ti).

For each student group, we identified the range of project behaviours coded. This is captured in Figure 3, which was generated with SankeyMATIC to render for all groups the co-occurrence of process behaviours with project phases and of project phases with course weeks. As such, the Sankey diagram offers a visual representation of the temporal trajectory of CBL project development, by highlighting the group process behaviours for each project phase and the weeks when they occur (Figure 3). For example, 4 groups evaluated their tests for ethical faults between weeks 6-8, during the test and evaluation phase.

Figure 3. Occurrence of group process behaviours (n = 6 groups) for each project phase and week (generated with Sankeymatic).

The temporal phases identified were discussed with the second author after the end of the course to better identify how they overlap with his own situated experience and interactions with the students. Lincoln and Guba (1985, 314) consider member checks as ‘the most crucial technique for establishing credibility’ in qualitative research.

4.6. Participants

All 45 students enrolled in the course were invited to participate in the study. They received an information sheet and consent form. Students could opt separately for data to be collected via participant observation, analysis of reflections, and interviews. The participants are students from 6 groups: 22 students were subject to participant observation and 15 students were subject to interviews and analysis of the reflective assignment submitted for the course (Table 6). Participation was based on informed consent and followed the guidelines of the Ethical Review Board (as extension to ERB2021ESOE6). The data was anonymised and each participant received a pseudonym.

Table 6. Overview of participants and data collection methods.

Group/participant	Participant observation	Reflections	Interviews
Group 1
Rohit	V	V	V
Aart	V	V	V
Luc	V
Group 2
Thijs	V	V	V
Jean	V	V	V
Krish	V	V	V
Ceyda	V
Group 3
Amir	V	V	V
Willem	V	V	V
Marinus	V	V	V
Bas	V
Group 4
Mila	V	V	V
Marton	V	V	V
Mihai	V
Group 5
Lukas	V	V	V
Maribel	V	V	V
Nic	V
Gabriel	V
Group 6
Sam	V	V	V
Tess	V	V	V
Bram	V	V	V
Kiran	V

4.7. Positionality

Given the intense data collection process, it is important to reflect on our positionality in relation to the study subject, the participants, and the research-project context (Rowe 2014).

Author 1 is a philosopher and educational researcher focusing on engineering ethics, who designed one interdisciplinary CBL course (different from the one reported in this study). She researched the study without direct involvement in the course. Author 2 is a philosopher with a physics background. He coordinates a 20 ECTS programme aiming to raise engineering students’ awareness of the user, society, and entrepreneurship aspects of technological innovation. He was involved as coordinator and coach in the course reported in the article and contributed insider perspectives to the analysis.

In our CBL teaching and research, we are motivated by the belief that students’ exposure to real-life sociotechnical issues arising in the design, development, operation, or use of engineering artefacts can provide a meaningful educational experience. We sought to understand how engineering students develop CBL projects and to use this understanding in our activity, by improving the organisation of interdisciplinary CBL courses centred on real-life sociotechnical projects and the support offered to students.

5. Findings

The analysis identified seven phases for CBL project development, which have distinct areas of student focus and process behaviours. Via a Sankey diagram we captured the unfolding of group project phases by week and their associated process behaviours (Figure 3).

As the course progressed, the focus of the students’ activity and their process behaviours changed. Table 7 renders the temporal trajectory of CBL project development, with the group process behaviours for each project phase and when they occur. We proceed by describing each phase of CBL project development and its associated process behaviours.

Table 7. CBL project phases and associated process behaviours.

Week	W1	W1-W2-W3	W3-W4	W3-W4-W5	W4-W5-W6	W6-W7-W8	W8-W9-W10
Project phase	CLIENT	PROBLEM	CONTEXT	AIM	PATH	TEST AND EVALUATION	SOLUTION
Project focus	Familiarising with the problem area Obtaining direct info about the client’s activity, technical equipment and profile Asking questions about the project process Discussing course expectations Researching the client	Brainstorming the problem area Brainstorming client needs and risks Discussing deliverable expectations Linking the problem area with real-life issues Linking the problem area with the profile and activity of the client Assigning project roles Familiarising with the data and data analysis	Clarifying deliverable expectations Identifying stakeholders Brainstorming stakeholder needs, values and risks	Narrowing client and stakeholder needs and risks Researching the problem area Identifying ethical issues, dilemmas and conflict of interests and values Agreeing on the deliverable	Prioritising process tasks to meet the aim Assigning specialised process tasks Generating solution paths Researching solution paths Engaging in technical tasks Connecting ethics to technical tasks	Testing solution paths Setting values as evaluation criteria Evaluating tests for ethical faults Revisiting stakeholder risks, values, needs and preferences Addressing ethical issues, dilemmas and conflict of interest Researching, optimising and refining tests Clarifying client needs	Adapting tests to client needs and characteristics Synthesising, structuring and writing up data from tests Integrating ethical and technical aspects into the final solution Justifying the solution in terms of value creation and usefulness Communicating the solution

5.1. Phase 1: client

In the first week, students were introduced to the course and received the brief to develop a project of their choice for a client, set in a broad problem area linked with the client’s activity. Students also had the opportunity to meet with the client.

This is a period marked by general confusion not only about the clients’ expectations about the deliverable, but also the overall expectations of the course. Several students commented on the vague nature of the case. Amir found that ‘the case was really vague at the beginning. […] In the first couple of meetings it was really hard for me and for other group members to understand the case’. Similarly, for Rohit ‘the first week was very vague’, while Marton recounts that the project ‘was not very clear from the beginning, but as we went along, it improved’.

Students engaged in a series of process behaviours to gain clarity and advance their understanding of what is expected of them. These included asking questions about the project process during the TA meetings. Thijs noted that

‘a problem we as a team ran into at the start of this project was the unclarity in what actions we could take to make progress for our project. We gained appropriate information by asking specific questions we had prepared beforehand with our team in our very first meeting with the TA’.

Gaining client know-how was crucial, with groups either questioning clients about their activity and equipment during the initial meeting or researching it independently. As Willem recalled, ‘in the beginning, the challenge was a lack of information, and we tried to get a clear idea of what the client exactly wants’. For Marton’s group, ‘our approach was looking at the vision of the company and its future rollouts on the bigger markets’. Other groups resorted to researching the client, by looking at company websites and gathering online information about their activity. As groups interacted with each other during the support sessions, students from group 6 who didn’t research the client later acknowledged the importance of such research and preparing questions about the project process as a takeaway for future projects.

Students familiarised themselves with the project’s problem area by asking questions to the client and teachers. These questions were primarily focused on the technical specifications of the problem area and the features of the field the Client works in (i.e. electricity grids, medical devices). The information was used to develop an understanding of the client's needs that can serve as a basis for tackling the project.

5.2. Phase 2: problem

Between weeks one and three, student groups focused on identifying a problem to work on. Group 4 was the only group that jumped straight to stating a problem from week 1, which they later considered ‘a false start’ (Marton). The second week was marked by intense brainstorming processes of potential problems and mapping clients’ needs and risks. This meant that groups had to ‘make assumptions’ (Jules) or ‘formulate a preliminary solution’ (Willem). There was a high focus on generating ideas through teamwork. Lukas found that ‘teamwork was helpful for coming up with original ideas more frequently, […] bouncing ideas off each other, […] and developing the idea so well’.

Other process behaviours included listing their client's needs and asking questions about the problem area or the course expectations, this time directed at teachers. According to observations, this phase recorded the highest number of questions addressed during meetings or as messages to the teaching team, which decreased between the third and eighth week (before the final project submission). The bulk of questions targeted expectations about the delivery of the project as a graded course component or requested additional information about the project and its boundaries.

To identify a key problem, the groups started to explore the data, using data as a prompt to discuss how to link the problem area with real-life issues and with the needs of the client. Three groups also requested additional data from the client to get a clearer picture of the problem area. In this phase, the technical overrode the ethical when identifying the problem. According to Jean, ‘at first, when we started the project, it was very technical’. In the first weeks, a problem was deemed relevant or important primarily from a technical perspective. According to Sam, ‘in the beginning, our reference frame was focusing on sufficiency, efficiency and getting the job done without thinking of ethics’. Only group 4 included ethical considerations in their process of looking at the data and identifying client needs. According to Marton, his group considered

‘both ethical and technical potential roadblocks when imagining our client’s path towards the big open market. Some examples are customers not being comfortable with remote driving, technical limitations to the vision, and operators unable to perform appropriately when they are needed most’.

As students’ understanding increased, they attempted to narrow down a problem in the broad and open space of the project’s problem area. Marton talked ‘about the process, first we were very confused about the actual course. So once that was a bit out of the way, we could start actually finding a problem’. In the coaching session, Maribel also recalls how in week 2 her group used the experience of the first week to ‘brainstorm ideas and formulate a problem statement’.

The end of this phase is marked by assigning project roles within their groups. Strategies mentioned during the coaching sessions included dividing tasks, setting goals for each task and linking each with a responsible person.

5.3. Phase 3: context

The third phase occurred between weeks three and four of the course, with context behaviours more prominent in the third week. This phase had a heavy emphasis on mapping the context of the problem area, using visualisation and mind-mapping tools. Jean gives a detailed description of this process, which for his group started in week 2:

‘I learned to create concept maps and stakeholder diagrams […] using the Miro software. Ceyda introduced this app to us as she is already quite versed with it and walked our group through the basics. Besides, watching the teacher’s videos and reading his notes about concept mapping further helped me create my first concept map related to stakeholder relations. […] The whole process from installing the Miro board to completing the first concept map was more formative than achieving the result. This is because my group used Miro for much more than just the concept maps: we discussed new ethics ideas, pasted them as sticky notes on a virtual whiteboard, rearranged them, and linked them’.

In this phase, ethical aspects started to appear on the radar of most groups, even if underdeveloped or not explicitly identified as such. The ethical focus manifested through process behaviours aimed at generating direct and indirect stakeholders, their needs, values, potential risks and the scope of intended consequences. According to Rohit, his group ‘made a concept map of the values and discussed transparency’, stressing that ‘it also extends to our stakeholders’. Jean recounted how at this phase, ‘for the stakeholders’ values we made assumptions, for example, that the government wanted transparency and value the environment and sustainable development’.

5.4. Phase 4: aim

Throughout weeks three-five, process behaviours focused on agreeing on the deliverable and stating a clear and explicit aim for the project. After the first weeks were dominated by vagueness and uncertainty, students now gained more clarity about their end goal. For Willem’s group, ‘the goal of this project became relatively clear’ in week 4. In the coaching session, Tess recounts how the clarity about the direction of the project led to a shift in focus to defining her group’s goals and questions for the client meeting. Sam recalls a turning point at this phase, noting that

‘after the fourth week we did find that goal, and the way got clear how we could get there. This was a great experience because we came from troubled waters to a calm situation where we […] had our faces all in the same direction’.

During this phase, groups spent more time researching the problem area and the client. When setting the aim, groups considered how the project may help the client. This focus is best summed up by Mila, who describes that ‘when we were doing stuff, we were always thinking if it will have a purpose with our client’.

The research conducted by students prompted further questions about clients’ needs and led to brainstorming that required clarification from teachers and experts. Marton’s group mentioned during the coaching session of week 4 how they prepared questions about the ‘general vision of the client’, which helped ‘shape their research’ of the problem area. For two groups (group 2 and 4), the process of brainstorming and questioning led to revisiting the needs of the client and restating the problem. Marton confirmed that ‘once we've gotten a clearer understanding of what the course was about, what we were supposed to do, we thought again at the problem statement’. In week 5, Thijs noted asking themselves ‘why’ questions for each problem they mapped to narrow down and identify ‘the root cause of the problem’ the Client was facing. He later described questions as his group’s ‘only tool’, which allowed to dig into the project’s problem area ‘like archeologists’.

The information gathered via self-directed research and the advice of the teaching team prompted further processes related to narrowing the needs and risks for the client and stakeholders previously identified. Some groups struggled with this process due to lack of information. Tess recounts that ‘it was really difficult considering his [n.m. client’s] needs, when you don't have a clue exactly. We tried websites, we also tried to ask during the sessions’.

During this phase, the focus on ethics in the context of projects involving energy transitions or data sharing starts to naturally deepen. Thij’s group

‘wanted to map the whole process of our client and where they faced difficulties, and then we would link those difficulties to an underlying problem. And that is when we discovered that ethics was really relevant in the context of data sharing and then we really wanted to find out how we could help the client with this ethical dilemma’.

For four groups, the focus on ethics translated into identifying ethical dilemmas and conflicts of interest between stakeholders. Jean gives the example of how his group balanced that

‘campus owners wanted to minimize costs and provide less data because it was just easier for them to have less sensors and cheaper, while the Client might want a more precise model, and the more sensors they have the better’.

The deepened interest in ethics brought many questions. Students mentioned having little to no prior experience with ethics, and their questions were related to identifying stakeholders and values, the specifics of ethical theories, or understanding intended outcomes and potential constraints when setting a project aim relevant to the client and the challenge. Rohit admits that ‘it actually took us quite a while to understand the ethics of it [n.m. the project], because it wasn’t obvious’. When setting the aim, Mila also recalls that her group

‘were a bit lost at the beginning and then we started coming up with something on the practical side. Then in week three or four […] we just looked at the data and tried to be critical about it and then came up with the ethics questions’.

The culmination of this phase was the group agreement on the deliverable. According to Lora, in week 4, ‘we were pretty much straight forward and we were all agreeing on the deliverable. There wasn't any discussion because we said the same and that was it’.

5.5. Phase 5: path

This phase occurred mid-course, as early as week four for one group, and in weeks five and six for the other groups. Student process behaviours concentrated around the formulation of an acceptable and defensible action path to reach the project aim.

All groups generated and researched several action paths that would later make the subject of testing. During support sessions, Marton recounts that his group ‘thought of some experiments for each of the problems’ to ‘examine the scope of the obstacle’, while Mila describes how her group researched sensors for testing.

Groups now gained more clarity and awareness of the technical and ethical input needed. They prioritised tasks and assigned specialised tasks to address these separately based on individual strengths and disciplinary backgrounds. Thijs recalls that setting an aim contributed to enhanced clarity in week 5, which was useful to ‘put things in perspective and prioritize actions’. If the statement of problems and aims were described as the outcome of joint decision-making, this phase saw a division of roles within groups, with some students working on technical aspects and others on ethics. Describing this process, Tess states that

‘We had divisions between the group members. Some people were from faculties that had more knowledge about coding and some people were better with the ethics assignment part. So we made a really equal division between tasks for my group, over the whole challenge’.

Lukas notes that he was ‘very involved with the ethics and I did that out of my own volition, but other group members in my team were the opposite. So they did the data, they love doing the data’. Willem also notes that ‘we all worked on different parts of the solution’. His teammate Amir confirms that ‘me and Willem were often relating to the ethics part, and Bas was just on the technical aspect of coding. But he was really skilled at that’. In Rohit’s group, ‘the stakeholder part was done by the Industrial Design student and the Sustainable Innovation Student in our group’, while he worked on the technical part.

Groups focused on the technical data analysis. When it came to ethics, the project processes focused on connecting ethics to technical tasks and identifying ways to create value for the client. In this phase, the potential for value creation acted as a motivator. This is best captured by Thijs, for whom ‘value creation is something I truly care about, which leads me to be motivated and ambitious’.

5.6. Phase 6: test and evaluation

Between weeks six and eight, students focused on testing different solution pasts, as well as researching, optimising and refining the tests. This phase still sees differences in the division of process tasks, mostly based on individual strengths. Maribel remarks that

‘since our colleagues, some of them, were better in technical things than others, they would often discuss more about this. Then, for example, I or another team member would discuss the ethical values that we will have and how we could apply them to our technical tests’.

Lukas opposes the individual focus of the process in this project phase to the collective one during initial phases, noting that

‘we came up with the original solution together […] and then I think everybody equally contributed to finding specific answers to subsets of the problem. For example, Maribel came up with the idea to add a health coach during the tests to have easier access to immediate healthcare if something happens, and then I came up with the idea to slowly increment the total length of the test’.

While the tests had a strong technical component, as week 8 approached, the groups started researching values to understand how to integrate these into their technical work. During testing, several groups mentioned they had ‘a breakthrough’ or ‘eye-opening moments’, while others still encountered several difficulties connecting ethics to the technical specifications of the project. For example, Amir mentioned the challenge of ‘trying to integrate ethics with the data analysis part or trying to relate it to the project’ because ‘we didn't know how’. Eventually, his group ‘did include it a bit. For every technical point, we can mention the ethical thing about it. But we didn't integrate it until we saw that it's directly affecting the risk assessment during testing’.

Ethics gained the most prominence in how students approached the project since the course started, which also contributed to an awareness of its significance. Rohit mentions how until week 7, values seemed ‘just a collection of words’, and that ‘the research process was eye-opening because I understood the values that we had selected’. As such, the groups became preoccupied with evaluating the tests for ethical faults and then refining these to meet stakeholders’ needs, preferences and values or diminish their risks. Jean admits that ‘at first, when we started the project, it was very technical since we were just designing what exactly we wanted out of the test and only with the testing we thought about all the ethical implications’. His teammate Krish describes how this changed for their group during testing,

‘whenever we talked about a technical aspect, the functionality that was added, we usually always saw it from a perspective of how it would affect the stakeholders, and then based on how it would affect the stakeholders, we sort of saw what laws were already there and what would need to be changed to accommodate for it’.

Mila has a similar experience, remarking that

‘you usually don't think about ethics when you're doing a project because we are not used to think about ethics when doing projects. But from my point of view, following the test, I think it was important to discuss whether it is a just requirement or discrimination, because of the long term goal and consequences. We were just looking at the test and tried to be critical about it, tried to find what is wrong and is that question really wrong or is that our subjective opinion?’

Aart also described his focus on ethics in this phase, noting that

‘I played around with the data and discovered that when you manipulated data just a bit, you can figure out when a building is empty, when for instance people have left, or when the building is getting occupied again. So I thought about privacy and the risks related to having such an open data set, because when the Client uses the model data set, it's getting more and more open. And I thought that was really interesting, to combine the findings of the data analysis test with the ethics and then seeing what are the possible ethical consequences’.

In this phase, groups set values as an evaluation criterion for the tests. The most mentioned values were safety, privacy, sustainability, transparency and equality. Lukas recalls how his group ‘took all possible measures to ensure that the tests would be useful and guarantee our ethical solution’. Following the test evaluation, ethical criteria and values are described by four student groups to overtake the technical criteria. According to Maribel,

‘The most important is the safety of remote drivers, so they can both perform effectively, efficiently, and safely. If you would design a test, it would obviously have to pass an ethical board, so you would have to really make sure that the safety of the remote drivers is ensured’.

Three groups changed their initial plan to ensure that their project would implement an ethical solution. Testing exposed ethical weaknesses in the proposed solution, disrupting the linear project development.As such, students went to redefine their project’s problem and aimnd values to better meet the ethical criteria and values they agreed to consider. This process is best summarised by Jean, who notes that

‘the problems we identified at the end weren't the ones at the beginning. […] At the beginning of the project, we didn't know half of what we knew at the end. Because at the beginning we thought we would build concrete data analytics, like a code, and then try to justify our code with the ethics’.

Similarly, Lukas recalls that

‘afterwards when we were doing our individual research into the solutions, we did find some things that we wanted to change and then we reflected back to our original solution and then with the ethics to back it up we changed it and made it better founded’.

According to Tess, the testing phase ‘made you think and rethink your own opinion’, such that ‘half-way through, after testing the k-clustering, we completely scrapped it and diverted to decision trees, purely because we came together and said, we don't think this could be the most ethical solution’. Tess goes on to add that although ‘it came a bit late, made this decision even though we already worked on one thing really hard. It didn't stand in our way, because we were we have to think at the ethics as well’. Her teammate Sam reflects on this moment, considering that

‘a major step for us was when we decided to do the decision tree and let go of what we normally would have done to be as accurate and precise as possible, which needs k-means clustering. That was the moment we said ‘no, this is a boundary because this course is not about data, this course is about combining data and ethics’. And that is why we accepted it’.

Lukas recalls his group had a similar experience:

‘We started by thinking of the things we actually wanted to test, and after, we found that there were quite a bit of problems arising from an ethical standpoint. […] At first, we came up with a solution that was not really ethically founded, and afterwards we find some things we wanted to change. Then we reflected back to our original solution to back it up with ethics, we changed it, and made it better’.

Students used the tests to further clarify their client’s needs, for example by talking ‘more in-depth with the Client’ (Jean). The phase culminates with groups agreeing on what solution to implement and how.

5.7. Phase 7: solution

The final project phase coincided with the last three weeks of the course. Process behaviours focused on implementing a sociotechnical solution to the broad problem area of the challenge brought by the client. During this phase, all groups strived to combine the technical and ethical input from the tests in their report and presentation to the client. Several strategies for sociotechnical integration were employed: undertaking a risk analysis, justifying and explaining the data analysis strategy via value statements and stakeholder mapping, writing both an ethics section and a technical section for the report and explaining how these are interrelated.

All groups adapted the tests to client needs and characteristics, based on their client’s feedback. For example, Maribel describes the focus of this phase as ‘just being able to adapt our project to our clients’ liking’. She notes that

‘towards the end the client revealed that they were actually working on a smaller scale, with less budget. So we had to adapt everything. For example, we wanted to input a health coach or health supervisor as our ethical solution and seeing they have a much smaller scale, this wouldn't be needed’.

The groups justified the solution in terms of value creation, stakeholder impact and usefulness for the client. According to Jean, ‘we had a lot of values because of the stakeholder map. We tried to consider all of them and come up with a solution that wouldn't compromise one stakeholder or the other’. Maribel is ‘pretty sure that the clients will be using parts of our information and ideas to their product’.

Once groups were satisfied with their proposed solution, they synthesised, structured and wrote up data from tests. To achieve sociotechnical integration, team collaboration intensified to merge the individual technical and ethical contributions of the team members. Developing a sociotechnical project nevertheless proved challenging. For Tess’s group, this was due to the ‘opposite importances’ assigned to ethics and technical aspects when it came to data storage and transparency, which ‘collided a bit’. According to Tess, ‘understanding the ethics, that's step one, and then adding it to your solution is another whole step’.

The final phase brings retrospective clarity to students about several project aspects that were deemed confusing or of little relevance at the start of the course. In the final week of the course, students report peak awareness of the importance of ethics and strategies for integrating societal aspects in future work. This concern is reflected in the emphasis on presenting an ethically-focused solution. Amir ‘didn't think that ethics was related to our solution until the end, when I realized it was deeply related’, while according to Thijs, ‘that's when it [n.m. ethics] was important, it happened near the end of the course’. A similar realisation occurred for Sam, whose group ‘at the beginning didn't expect that much to happen with ethics, but at the end I think our group was most engaged with ethics and we really liked doing the ethics. It was a major part for us’.

6. Discussion

6.1. CBL project structure

The analysis of the CBL project structure found that the process unfolds in a non-uniform and non-linear manner (Figure 4).

Figure 4. CBL project phases.

The non-uniform character of CBL project development reflects in the identification of seven phases corresponding to specific temporal periods, each exhibiting a distinct set of process behaviours (Figure 3; Table 7). Student groups first focused on obtaining knowledge about the client (phase 1), then generated a problem statement to address (phase 2), mapped the problem context, including considerations about other stakeholders and how they are affected by the problem (phase 3), identified the aim to be pursued (phase 4), proposed an acceptable action path for solving the problem (phase 5), conducted preliminary tests to evaluate the data and outputs (phase 6), before implementing the solution (phase 7). Furthermore, the phases also alternated between varying emphasis on group and individual work. While joint group project processes were more marked in the initial phases, phases 5 and 6 were characterised by division of tasks based on students’ strengths and affinities with either ethics or technical processes.

The non-linear character of CBL project development is revealed through changes in the articulation of the problem and its aim upon further research of the problem area or testing and evaluating the solution path. The iterative aspect of the project development does not mean that students go through all process behaviours undertook in a previous phase, but that they revisit some of the core decisions or assumptions that made the focus of that phase. Thus, the project problem and aim underwent several iterations, as students developed their technical analysis and increasingly considered ethical criteria, values, stakeholder needs, potential risks and client constraints, either as an outcome of the feedback received from their teachers, the clients or other groups, or due to their own additional research. The phase of solution testing and implementation was especially marked by revisions of previous work, coinciding with a more advanced data analysis and students’ growing appreciation of the relevance of ethics and societal concerns for their project and engineering practice.

The study identified several similarities with the phases posited by existing CBL frameworks. The project includes a phase characterised by a focus on defining the problem, corresponding to process behaviours in the Engage phase posited by Nichols, Cator, and Torres (2016) and exploring the challenge situation, as in the Tec 21 model described by Gutiérrez-Martínez et al. (2021). The final phase displays similarities to the Act phase (Nichols, Cator, and Torres 2016) and the implement phase (Gutiérrez-Martínez et al. 2021), through the focus on implementing an agreed solution. It also includes process behaviours related to communicating the solution, as the last phase of going public of the STAR Legacy Cycle (Cordray, Harris, and Klein 2009) and the Act phase (Nichols, Cator, and Torres 2016). The study also found similarities to existing frameworks at intermediary project phases, via the focus on process behaviours related to setting a roadmap to address the challenge (Gutiérrez-Martínez et al. 2021), researching resources, methods and tools related to the challenge (Ambrosi and Hermsen 2023), generating questions (Nichols, Cator, and Torres 2016), asking questions and accessing resources and experts to gain knowledge (Cordray, Harris, and Klein 2009), and generating and analysing proposed solutions (Gutiérrez-Martínez et al. 2021).

Nevertheless, there are also important differences in the identification of specific process behaviours and the timing when they occur. While the Apple framework puts forward three broad phases, where the ACT phase includes both developing prototypes, experiments and tests and implementing the solution (Nichols, Cator, and Torres 2016), our study identified differences in the trajectory of project development processes that sees the need to distinguish between solution testing and implementation as distinct project phases. Our study also found that process behaviours for testing and evaluating the solution occurred prior to its implementation, unlike the Tec 21 model where they were posited after the solution implementation (Gutiérrez-Martínez et al. 2021).

Unlike the current frameworks, given the educational collaboration with a real-life partner, our study identified the focus on process behaviours related to understanding the client, their activity and equipment as the first phase in developing the project. Other frameworks mentioned as a first phase process behaviours that were shown in our study to come in the later weeks and project development phases, such as those in the Engage phase of identifying a compelling and actionable challenge statement, defining the problem, or questioning personal interests and the community’s needs (Nichols, Cator, and Torres 2016), reviewing and researching the theoretical background (Gutiérrez-Martínez et al. 2021), discerning which information or activities are relevant to the task (Cordray, Harris, and Klein 2009), and identifying stakeholders and phrasing essential questions (Ambrosi and Hermsen 2023).

6.2. Group trajectory

Although all groups went through the same phases of CBL project development, nevertheless each had their own pace (Table 8). Some groups (i.e. groups 1 and 3) moved at a slower pace and reached some phases (such as the Aim, Path and Solution) later than others. One group (group 4) rushed straight to the Problem phase from week 1. Some groups spent several weeks in the same phase (i.e. groups 2-6 spent three weeks in the Solution phase). This shows the importance of following each group as the course unfolds, to capture their processes and how these processes are influenced by the individual experiences of group members, the successful strategies employed, and the challenges encountered.

Table 8. Group trajectory by project phase and course week

Week Group	WEEK 1	WEEK 2	WEEK 3	WEEK 4	WEEK 5	WEEK 6	WEEK 7	WEEK 8	WEEK 9	WEEK 10
Group 1	Client	Problem	Context	Context	Aim	Path	Testing & Evaluation	Testing & Evaluation	Solution	Solution
Group 2	Client	Problem	Context	Aim	Aim Path	Testing & Evaluation	Testing & Evaluation	Testing & Evaluation Solution	Solution	Solution
Group 3	Client	Problem	Problem	Context	Aim	Path	Testing & Evaluation	Solution	Solution	Solution
Group 4	Client Problem	Problem	Context Aim	Aim Path	Path	Testing & Evaluation	Testing & Evaluation	Testing & Evaluation Solution	Solution	Solution
Group 5	Client	Problem	Context Aim	Aim Path	Path	Testing & Evaluation	Testing & Evaluation	Solution	Solution	Solution
Group 6	Client	Problem	Context	Aim	Path	Path Testing & Evaluation	Testing & Evaluation	Solution	Solution	Solution

The course fostered interaction between groups through joint sessions. For example, the support sessions brought together all groups working for the same client. During these weekly sessions, groups shared with each other and with the teaching assistants their progress to date and the strategies employed. They also received feedback from the teaching assistants and other groups. In some instances, this high level of interaction and cooperation prompted groups that were struggling or lagging behind to adopt and adapt process strategies that were deemed effective by their colleagues. As such, group trajectories in CBL project development may depend on the level of interaction or isolation set by teachers.

6.3. Limitations

Given that ‘the Act phase [n.m. of CBL frameworks] is very discipline-specific’ (Ambrosi and Hermsen 2023), the primary limitation purports to the application of the study in one interdisciplinary CBL course. Monodisciplinary CBL projects or technico-scientific CBL projects may exhibit different process behaviours.

While some CBL frameworks include process behaviours related to reflection as a last phase (see Nichols, Cator, and Torres 2016; Gutiérrez-Martínez et al. 2021), reflections targeting the experience in the project and the learning that occurred were incorporated throughout the course via dedicated weekly assignments, as a continuous process behaviour, rather than as a distinct project phase. This is testament to the importance placed by the teaching team on reflection for sense-making and having it integrated in the course, given that ‘unfortunately, students tend not to engage in these processes naturally’ (Ambrose et al. 2010, 7) and require explicit and structured prompting (Ryan and Ryan 2013). As such, for other courses, reflective process behaviours may occur (or not) at a specific phase based on when and whether students are prompted to reflect.

Process behaviours for developing a CBL project may also be impacted by the level of openness or structure of the course, as well as the level of interaction and cooperation between student groups. As this course had a fairly open structure, students had the autonomy to self-direct their project development processes, with a few course moments steering them towards interactions with the client (during the client meeting) or submitting the final project (at the end of the course). The course also had a high level of student interaction and cooperation, facilitated by joint support sessions. Student groups shared among them their progress and gave feedback on strategies for developing the project undertaken by other groups. Although the findings are valuable for better understanding key phases of CBL project development and associated process behaviours, we recommend further studies set in courses with different disciplinary areas and degrees of structure or cooperation.

6.4. Recommendations

Shortitudinal research may suit the temporal boundness of engineering course work, providing insights related to students developmental trajectories over a relatively short period. To refine the adoption of this methodology in engineering education research, we recommend experimenting with various qualitative and quantitative data collection methods and timelag between measurements, as well as expanding the focus from behaviours (which may be individual or collective) to personal characteristics and learning gains.

To deepen the insights into how students develop CBL projects, we recommend further research identifying students’ subjective experiences throughout the different phases. Given that the current literature on CBL often lacks in-depth explorations of student perspectives, future research could examine the challenges and critical incidents experienced by students at each project phase, the strategies employed to overcome these and accompanying emotions.

Furthermore, there is a need for complementary research examining scaffolding mechanisms in CBL. The distinct character of students’ engagement with each project phase highlights the need for targeted and time-specific course support and interventions to better account for the varying experiences. While the importance of scaffolding in educational contexts is acknowledged, there is a paucity of research focused on the specific scaffolding mechanisms employed at the different phases of CBL project development. Future research is needed to explore how teachers can provide support and guidance throughout the different CBL project phases, as well as problematising what is understood by adequate support and identifying the different roles needed in a teaching team to provide such support. As institutions and courses differ in terms of available resources, research on student support should consider aspects related to the effective use of resources and scaling up CBL to larger classes. Understanding effective scaffolding practices can contribute to the development of instructional strategies that optimise student engagement and learning outcomes.

Building on the temporal process behaviours identified in the study, research is also needed to identify how to tailor continuous assessment to the different project phases. This research involves exploring innovative and contextually relevant assessment methods for CBL that capture the varied student project processes linked with each phase. It includes a focus on formative assessments informing ongoing instructional strategies and summative assessments for a holistic evaluation of project outcomes. The development of phase-specific assessment frameworks can contribute to enhanced learning experiences.

7. Conclusion

Engineering education sees an increasing interest in switching from hypothetical scenarios to more immersive projects set in real or realistic settings (Martin, Conlon, and Bowe 2021a). Projects with real-life components and an open-ended problem area can provide students with more complete exposure to the features of professional environments (Bairaktarova and Woodcock 2017), thus enhancing their preparedness to enter the workforce (Caratozzolo et al. 2021), the awareness of their sociotechnical responsibilities (Martin, Conlon, and Bowe 2021b), and the development of transversal skills such as complex problem-solving, leadership, ethics or communication (Félix-Herrán, Rendon-Nava, and Jalil 2019; Gonzalez-Hernandez et al., 2020). In recent years, such projects have become the mark of CBL (Bombaerts et al. 2021; Martin et al. 2023).

The study responded to the need to understand the prospects of CBL and its implementation.

It is based on an interdisciplinary CBL course on Ethics and Data Analytics that saw students developing a project rooted in a problem area from the activity of the university’s stakeholders. The study examined the processes by which interdisciplinary student groups developed a CBL project and grouped them into distinct temporal phases. The paper contributes to the limited body of evidence on naturalistic project processes in educational contexts involving real-life partners (see as an example Corple et al. 2020).

The paper makes three important contributions to the engineering education community:

First, at the methodology level, the paper presented the design of a shortitudinal qualitative study to capture the dynamic trajectory of data related to students’ project development behaviours throughout an engineering CBL course. Although in the last 5 years shortitudinal research approaches started to emerge in medical and social sciences, the description of their implementation in an engineering education research setting is currently missing. The paper thus provides the first example of the design of a shortitudinal trajectory study in an engineering education setting, which may guide other teachers or researchers looking to examine student processes in the classroom.

Second, considering the findings of the study, the paper contributes to the growing literature on the implementation of CBL by identifying the phases of a CBL project and their specific student process behaviours, as they unfolded throughout a 10-week course. Understanding such processes in-vivo is an important complement to the project structure posited by existing CBL frameworks, such as the framework developed for Apple by Nichols, Cator, and Torres (2016) or the STAR Legacy Cycle (Cordray, Harris, and Klein 2009). According to our study, the temporal structure of a CBL project consists of 7 non-linear and distinct phases: gaining client know-how, articulating a problem, mapping the problem context, setting the aim, proposing an action path, testing and evaluating it, and implementing the solution. Each phase exhibits distinct student process behaviours and focus.

Third, the paper puts forward several recommendations for further research building on the existing findings to advance the implementation of CBL and support student learning. The study presented in this paper will be continued with a temporal investigation of the challenges experienced by students during CBL project development, the strategies for overcoming them, students' emotional trajectory, and learning gains.

Acknowledgements

Author credit is distributed as follows: Writing: DM; editing: DM & GB; course setup: GB; setup of data collection (diaries and observations): GB; setup of data collection (interviews): DM & GB; methodological conceptualisation: DM; data collection and analysis: DM; data visualisation: DM; funding acquisition: GB. The authors want to thank the members of the teaching team: Adam Watkins (course set-up, data analytics teacher and coach), Jeff White (ethics teacher), Regina Luttge (expert) and Rick Lange (expert and coach), the enthusiastic team of teaching assistants (Theodor Chirvăsuţă, Diana Frances, Waldo Heugebaert, Ruben Latuy, Ilja van Oort, Fleur Petit, Sander Poot, Marijn Stijvers, Milind Tripati), the educational partners representing the Diagame project and the student teams RED and MOBIX. Acknowledgement for Karolina Doulougeri, Jan Vermunt, Gunter Bombaerts, and Michael Bots for the diary set-up as a course assignment.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by TUe Innovation Fund.

Notes on contributors

Gunter Bombaerts

Gunter Bombaerts is an Assistant Professor in Ethics of Technology at Eindhoven University of Technology. He focuses on socio-technical energy-, sustainability- and AI-systems, using several theories from system thinking over Buddhist ethics to Foucauldian analysis. He is also the coordinator of the User-Society-Enterprise program at TU Eindhoven. He is involved in and publishes about education innovations projects on challenge-based learning and ethics. He is active in the SEFI Ethics SIG.