Involving students in engineering course design: a combined approach based on constructive alignment and multi-criteria decision-making

Figures & data

Figure 1. An overview of the proposed approach combining CAT and MCDM with a post-design implementation.

Figure 2. The focus points in designing the ILOs.

Table 1. Analysis of the identified ILOs in terms of clarity and levels in Bloom’s taxonomy.

ILOs of the course	In which levels are these in Bloom’s taxonomy
ILO1: Describe the fundamentals of data science, its applications (AI/ML), data-driven modeling, and big data analytics.	Level 1; Remembering - recall facts and basic concepts Level 2; Understanding - Explain ideas or concepts
ILO2: Apply the basics of well-known libraries of the toolboxes for data scientists.	Level 3; Applying - use information in new situations Level 4; Analysing - draw connections among ideas Level 5; Evaluating - justify a stand or decision
ILO3: Describe the steps of the data mining process.	Level 1; Remembering - recall facts and basic concepts Level 2; Understanding - Explain ideas or concepts
ILO4: Describe and apply visualisation techniques with respect to the data mining process.	From level 2 to level 5; identify, apply, and select
ILO5: Describe and perform data pre-processing methods to ensure a multi-dimensional measure of data quality.	From level 2 to level 6; identify, plan, design, and perform
ILO6: Explain and interpret utilisation of data and applicability of AI/ML algorithms for improving production systems and product development.	Level 2 and level 3; identify, list, describe, and interpret
ILO7: Interpret and discuss state-of-the-art knowledge from scientific papers related to data science in mechanical engineering.	Level 2 and level 3; identify, list, describe, and interpret
ILO8: Implement commonly used AI/ML algorithms, analyse their performance, and discuss their application using industrial applications from the product realisation life cycle.	From level 3 to level 6; apply, analyse, evaluate, and create – (produce new or original work)
ILO9: Critically analyse and argue key ethical principles and potential impacts of AI on people and society and evaluate social and human requirements of systems and scenarios.	From Level 1 to level 4; list, communicate, and argue

Figure 3. Master programme’s distribution of the students taking the course.

Table 2. The weights of the ILOs’ criteria.

The ILOs criteria	Aggregated values by using GMM	Normalised value
Theory-based knowledge	3.107	0.186
Practical-based knowledge	5	0.299
Generic academic skills	4	0.239
Key transferable skills	4.642	0.277

Figure 4. The ranking results of the TOPSIS method for prioritisation of ILOs of the course.

Table 3. A summary of the designed TLAs for each course module.

Module no	Lectures	Guest lectures	Workshops	Software seminars	Examination project work	Presentation	Self-paced hands-on exercises
1	√	√	√
2	√	√	√	√			√
3	√	√		√	√	√ (Literature seminar)	√
4					√	√ (Project work)

Table 4. Summary of focus group interview results on students’ perceptions of co-creating course design.

Interview questions	Key points from the responses of the focus group
1. How would you describe the co-creation process between teachers and students for the course?	‘Good initiative with actual involvement with students.’ ‘Identification of important aspects and contribution of personal input’ ‘Foster active participation and innovative thinking.’ ‘Helps understand course goals.’ ‘Encourages students to take ownership of their learning.’ ‘Ensures the course is relevant, meaningful, and effective in reaching learning outcomes.’
2. From your perspective, was the co-creation process effective in practice?	‘Yes, definitely.’ ‘I would say it was really helpful for us, and it will be helpful for the upcoming students as well, I think.’ ‘From my perspective, based on factors such as the willingness of teachers to collaborate, engagement, and participation of students, the co-creation process was effective and well-implemented to meet the learning objectives of the course.’
3. In your opinion, was the co-creation process worthwhile overall?	‘Co-creation process was worthwhile despite the required time and procedures.’ ‘The process should be executed properly to achieve the course learning objectives effectively.’ ‘It helped me understand what I should know and get from the course, and how to interact with the tutor.’ ‘The process is a fundamental need for students to prepare for the upcoming course.’ ‘The co-creation process was worthwhile for course design and implementation.’ ‘Students are empowered to collaborate and engage closely.’

Table 5. Summary of focus group interview results on co-creation process experience.

Interview questions	Key points from the responses of the focus group
1. How would you describe your contributions to the co-creation of the course?	‘The co-creation process was a new and positive experience for me. However, the initial stage turned out to be beneficial for me, as it provided me with valuable insights.’ ‘By working in this way, I discovered new learning opportunities and had the chance to engage with the teachers.’ ‘It helped me understand how things operate, and I was able to provide my input to make the process more effective.’ ‘My contributions to the co-creation of the course helped me to achieve the intended learning outcome.’
2. What are your personal experiences of participating in the co-creation process of the course?	‘It was really a nice thing to bring up. It was really impressive for me, at least.’ ‘From my personal experience, I believe that greater student involvement, coupled with pre-designed strategies, can further enhance the effectiveness of the course. This applies to different milestones throughout the course.’ ‘In my experience, the process contributes to the success of the learning experience for the student.’
3. How would you describe the satisfaction with your involvement in the co-creation process?	‘If I were to rate my satisfaction on a scale of one to five, I would give it 4.’ ‘While it was a positive experience, there is still room for improvement, and upcoming courses can also benefit from this. Continuous improvement is a parallel process, and there is always potential for future enhancements.’ ‘In my opinion, satisfaction with this process can be stated as being based on the goals and expectations of the parties participating, as well as their own experiences and views.’
4. Is there anything that you would like to add that we have not addressed in the interview?	‘I suggest that you should elaborate more on time frame. The time frame is also crucial to achieving the course objectives, and it should be emphasised.’ ‘Another point to mention is the importance of involving student representatives with as many students as possible. However, this can be challenging at times. It would be beneficial to have some measures to assess the representative’s level of involvement with the students. This would provide a way to objectively or subjectively measure the level of input that can be expected from the representative.’ ‘As a practical suggestion, prior to these types of activities, it may be helpful to organise brainstorming sessions.’

Table

Please evaluate the ILOs criteria according to their importance for the course by using the linguistic scale below this matrix.
Criteria	Student representative 1	Student representative 2	Student representative 3
Theory-based knowledge
Practical-based knowledge
Generic academic skills
Key transferable skills
Linguistic expression	Corresponding evaluation score
Very low (VL)	1
Low (L)	2
Medium (M)	3
High	4
Very high (VH)	5

Table

To what extent do you think that the identified ILOs are essential in terms of each criterion, such as theory-based knowledge (1), practical-based knowledge (2), generic academic skills (3), and key transferable skills (4)? Please use the linguistic scale given in below this matrix.
The identified ILOs of the course	Student representative 1				Student representative 2				Student representative 3
	1	2	3	4	1	2	3	4	1	2	3	4
ILO1
ILO2
ILO3
ILO4
ILO5
ILO6
ILO7
ILO8
ILO9
Linguistic expression	Corresponding evaluation score
Very low (VL)	1
Low (L)	2
Medium (M)	3
High	4
Very high (VH)	5