Distance and presence in interdisciplinary online learning. A challenge-based learning course on sustainable cities of the future

ABSTRACT

Addressing complex sustainability issues in higher education requires the combination and integration of various disciplines, perspectives and approaches. Challenge-Based Learning (CBL) can support interdisciplinary collaboration on sustainability issues. It requires students to actively explore, discuss, reflect on and integrate information and methods from various disciplines. Online learning could enhance interdisciplinary collaboration since it is associated with greater geographical and educational flexibility and accessibility. Applying an active learning approach such as CBL in an online setting is believed to support interdisciplinary learning and collaboration. We present a case study that took place in a 10-week online interdisciplinary, inter-university undergraduate course on sustainability education. Our research is based on well-known online learning theories “Transactional distance” and “Community of Inquiry” (CoI). The aim of this study was to investigate how transactional distance, presence and (online) interdisciplinary learning are perceived by students. 23 undergraduate students from three universities were enrolled in the course. Quantitative survey data (N = 13) and qualitative data from student reflection papers and interviews (N = 15) were collected. Students perceived low levels of transactional distance and high levels of presence. Unexpectedly, a small increase in perceived distance between students was measured which could be explained by reported limitations of the course design. Students valued the open, interactive and creative character of the course and the online format was not perceived as hindering. Students reflected on interdisciplinary competences that they developed during the course. This study is a first step towards future national as well as international interdisciplinary, inter-university educational collaboration on sustainability issues.

KEYWORDS:

• Challenge-based learning (CBL)
• student-perceptions
• online interdisciplinary learning
• transactional distance
• presence
• sustainability challenges

Introduction

The earth is facing serious and urgent sustainability challenges (Rockström et al. 2009; United Nations 2015; IPCC 2018). Addressing these complex sustainable development challenges efficiently in higher education efficiently requires the combination and integration of various disciplines, perspectives and approaches, including combining theory and practice (Bootsma et al. 2014; Wiek et al. 2014; Brudermann et al. 2017). A broad spectrum of transversal skills such as collaboration, critical thinking, persistence and problem solving are needed to succeed in a world faced by global challenges (Monterrey 2015; Portuguez Castro and Gómez Zermeño 2020; Gallagher and Savage 2020). To contribute to sustainable development and addressing global challenges interdisciplinary collaboration skills are crucial for students to acquire (Tress et al. 2005; Bootsma et al. 2014; Brudermann et al. 2017; Uthrapathi Shakila et al. 2021).

Interdisciplinary skills are not only crucial but also complex requiring boundary crossing thinking and skills such as awareness of and reflection on disciplines, changing perspectives, connecting and integrating different disciplines (Spelt et al. 2009). Interdisciplinary thinking and collaboration can comprise a number of complex skills that can be further divided into subskills e.g. disciplinary knowledge, higher-order cognitive skills and communication skills (Spelt et al. 2009). Given the complex character of interdisciplinarity, it is the teacher’s responsibility to facilitate and stimulate students in their development of interdisciplinary thinking skills as well as during interdisciplinary collaboration.

Despite the necessity of interdisciplinarity in higher (sustainability) education, a systematic review study by Spelt et al. (2009) stressed the limited and explorative character of empirical research on interdisciplinary learning in higher education. Based on the literature, Spelt et al. (2009) created a list with essential conditions and subskills for interdisciplinary thinking related to students’ personal characteristics, the learning environment (e.g. curriculum, pedagogy, teachers and assessment) as well as the learning process (Spelt et al. 2009).

Somewhat in line with this study yet more focused on competences, a literature review by Lattuca et al. (2012) identified eight dimensions of interdisciplinary competence: (1) awareness of disciplinarity, (2) appreciation of disciplinary perspectives, (3) appreciation of non-disciplinary perspectives, recognition of disciplinary limitations, (5) interdisciplinary evaluation, (6) ability to find common ground, (7) reflexivity and (8) integrative skills.

Interdisciplinary learning focused on authentic and real-life learning content and experiences can be supported through different course designs and teaching approaches. Challenge-Based Learning (CBL) is one of those approaches enabling interdisciplinary learning. As the name implies, Challenge-Based Learning (CBL) involves collaboration on real-life challenges. It is an active, student-centred instructional approach with promising pedagogical benefits. CBL is said to increase student learning, motivation, collaboration, integration of prior knowledge and skills in a multidisciplinary setting as well as fostering the acquisition transversal skills, e.g. communication, collaboration, decision making and critical thinking, developing values and knowledge are gained through collaborative work and experimentation (Malmqvist et al. 2015; Gallagher and Savage 2020; Membrillo-Hernández and García-García 2020; Portuguez Castro and Gómez Zermeño 2020; Kohn Rådberg et al. 2020).

With CBL complex real-life challenges can be approached in the classroom and in interdisciplinary teaching and learning settings (Nichols et al. 2016; Bohm et al., 2020; Barynienė et al. 2022). Literature on CBL also shows that it can positively influence students’ interest in sustainable development since it requires them to actively engage with a real-life challenge (Portuguez Castro and Gómez Zermeño 2020).

To provide even more contrasting perspectives and thus stimulate boundary-crossing, higher education institutes increasingly join forces to address global issues in the form of interdisciplinary courses which are co-developed and co-taught (Brudermann et al. 2017; Uthrapathi Shakila et al. 2021). Three Dutch universities (Utrecht, Eindhoven and Wageningen University) launched a strategic alliance in 2020 to offer more interdisciplinary courses and to work on societal challenges. One of these courses, the “Inter-University Sustainability Challenge” course (IUSC) focuses on sustainable cities of the future. Cities have a major impact on sustainability and the environment, but also offer unique settings that are highly challenging and at the same time offer great opportunities for change (General for Environment 2010) A challenge-based learning approach has been applied in this interdisciplinary, inter-university, online course. The use of online learning tools can be useful to foster (interdisciplinary) collaboration among universities on a (inter-)national level since it makes the physical presence of students and teachers obsolete.

For the educational design of this online, interdisciplinary CBL course, two theoretical concepts were key: transactional distance and Community of Inquiry (CoI). Transactional distance (TD) can be described as the psychological and communication distance between teachers and learners (Moore 2013). TD is relevant in face-to-face as well as online education. Teachers as well as students can perceive TD which then can lead to misunderstanding, lack of motivation, and decreased learning. The greater the perceived transactional distance the more difficult it can be for teachers to support the students’ learning process. Online teaching, depending on the course design, might increase distance and thus decrease students’ learning and collaboration. Despite the broad recognition of the relevance of the concept, empirical studies on student perceptions of transactional distance are thin (Lebeck 2017).

The “Community of Inquiry” (CoI) framework is a widely known framework guiding research on and explaining the design of online learning. relevant for the educational design (Garrison et al. 1999). In their framework they describe three essential elements needed for successful learning: cognitive, social and teaching presence (Garrison et al. 1999). Without presence, students will not be able to achieve and maintain meaningful inquiry especially in an online learning environment (Joo et al. 2011).

From our perspective, both concepts (transactional distance as well as community of inquiry) are highly relevant for (online) interdisciplinary CBL courses. Interdisciplinary education requires several conditions such as (inter-) disciplinary knowledge, communication, active learning and collaboration (Spelt et al. 2009) which are in line with the theory of transactional distance and the community of inquiry framework. Interdisciplinary collaboration increases the diversity in a course which makes it even more essential for teachers to support interaction, communication and collaboration. Additionally, online courses deal with an increased physical distance between students and teachers which is expected to have an influence on perceived transactional distance and presence (CoI).

Investigating student perceptions of transactional distance and the three presences (cognitive, social and teaching) can inform teachers and course designers about possible barriers during interdisciplinary (online) learning. Both concepts provide insight into student perceptions regarding the communication, exploration and integration of disciplines and ideas which are reflected in the cognitive presence and autonomy. Increased interdisciplinary communication and interdisciplinary group cohesion are reflected by social presence and dialogue. The facilitation and design of a CBL course to foster interdisciplinary thinking and collaboration is reflected by teaching presence and course flexibility.

Three research questions, all focusing on student perceptions on different levels, were guiding this case study. First, student perceptions regarding collaboration and interaction in an online, interdisciplinary CBL course were studied. Interdisciplinary collaboration requires active student participation and intensive group work over a period of several weeks (10 weeks in this course). Collaboration can be influenced through the online setting and benefit if possible barriers are considered. To investigate the influence of the course design and setting on collaboration, student perceptions of transactional distance and presence were studied and used as indicators for the success of online interdisciplinary collaboration. Next to student perceptions on interdisciplinary collaboration, we were interested which interdisciplinary competencies students were able to develop in an online course setting. Therefore, the second research question zooms in on interdisciplinary competencies students perceived to have developed during interdisciplinary teamwork. Lastly, to get an overall impression of online, interdisciplinary CBL courses, the third research question provides a broader insight on student perceptions of such a learning approach.

RQ1:

How do students enrolled in an online interdisciplinary, challenge-based sustainability courseperceive transactional distance and presence?

RQ2:

Which interdisciplinary competencies do students perceive to have developed in an online, interdisciplinary CBL course on sustainable cities?

RQ3:

Which strengths and weaknesses do students perceive of an online interdisciplinary CBL course on sustainable cities?

The motivation behind this case study is twofold. First the obtained insights can inform course designers how to strengthen the design of future online interdisciplinary and/or inter-university education. Second, this case study expands the current literature on (online) CBL (Barynienė et al. 2022) by investigating student perceptions of online, interdisciplinary collaboration in the context of an online inter-university CBL course.

Prior to this study, we expected that the weekly interactive, synchronous online lectures and tutorials would lead to low perception of transactional distance and high presence. An interactive Virtual Classroom at Utrecht University was specifically chosen for this course setup enabling students to learn online interacting with peers and teachers in an engaging and innovative way. In the VC the teacher stands in front of several large screens. The screens are equipped with cameras, enabling students different personalized, views as well as possibility for personal interaction through polls and quizzes. Content wise the course offered students flexibility and autonomy, e.g. challenge approach, focus, data collection, integration of their disciplinary knowledge and skills. During the course, each student team was supported by a coach. All this together lead us to expect low perceptions of transactional distance, especially between students and high social and cognitive presence.

The course deliverables (written group paper, a visualization of the challenge solution and a reflection paper) required students to apply, reflect and integrate their various disciplines and required students to apply interdisciplinary skills.

Based on Garrison and Cleveland-Innes (2005) who state that cognitive presence can be achieved in study programs that require higher-order thinking including debate and critical thinking, we assumed that by applying a CBL approach, students would perceive high cognitive presence. CBL requires student-centred inquiry, critical thinking, interdisciplinary collaboration and integration which should result in high cognitive presence.

In the next section, we unpack the theory on transactional distance and presence, and explore the relation with online challenge-based learning. Next, we introduce the case, the IUSC course and describe the methodology.

Transactional distance and presence in online challenge-based learning

Online learning is a double-edged sword. On the one hand, it can facilitate and support collaboration on a distance (Moore 2013; Bolliger and Halupa 2018), enabling inter-(national) collaboration across higher education institutions on global sustainability issues. For instance, synchronous online learning enables teachers to interact with students in real time, for instance during live lectures, coaching, and Q&A sessions. It enables various forms of (social) interaction, discussions, and the provision of instant (formative) feedback without requiring students and teachers to travel. Additionally, asynchronous learning tools enable flexible and cost-efficient student-teacher and student-student interaction through communication via email, chat, discussion boards, collaborative writing, literature research, and document sharing. And of course: online education and learning was and is the lifebuoy in the Covid-9 pandemic in most parts of the western world (Dhawan 2020).

On the other hand, an online learning setting can be perceived as blocking communication, building trust and thus collaboration in general (Haythornthwaite 2006; Bolliger and Halupa 2018). This is why the transactional distance theory (Moore 2013) and the CoI framework (Garrison and Arbaugh 2007) are of great importance in the context of an online, interdisciplinary CBL course. Both theories are widely known and used in online learning design. Being aware of these theories, teachers can influence the online learning and collaboration processes of their students.

Transactional distance

Moore (2013) states that three related variables influence transactional distance: dialogue, structure and learner autonomy. Dialogue refers to constructive interaction between students and teachers leading to learning. The more dialogue the less transactional distance. The flexibility of the course structure influences what, how and when students learn. Lastly, the more autonomy students have of their own learning, the less transactional distance they will perceive. Learner autonomy increases as students can independently work on assignments (Nwankwo 2013).

Teachers can influence these three variables and thus transactional distance through their course design and the use of technology. Depending on the use, online tools can influence how much and how often students receive feedback (dialogue), they can add flexibility to the learning process (structure) and as such, increase students’ learning autonomy (Nwankwo 2013).

The theory has been used as a theoretical framework to study student experiences (Chen 2001; Stein et al. 2005; McBrien et al. 2009) however, empirical studies on student perceptions of transactional distance are thin on the ground. One example is a study by Chen (2001) who studied the impact of several variables on students’ perceived transactional distance. The study revealed that students who frequently participated in online discussions perceived less learner-learner transactional distance. In a recent study, Bolliger and Halupa (2018) explored student perceptions of transactional distance, engagement and outcomes in three higher education online courses. Their results showed moderately low levels of perceived transactional distance and a moderate correlation between transactional distance and student engagement. Distance is a complex phenomenon, but the transactional distance framework will help us to get insight into students’ perceptions of distance between them and their teachers, peers and the course content.

Presence

A second key concept used to understand the effectiveness of online CBL in terms of interdisciplinary collaboration is presence. The “Community of Inquiry” (CoI) framework by Garrison et al. (1999) is a widely known framework stating that successful learning occurs in a community and requires three essential elements: cognitive, social and teaching presence. Cognitive presence is the basis for all learning and a crucial element in critical thinking which can be achieved through e.g. puzzlement, exploration and integration (Garrison et al. 1999). Social presence is about enjoyable interaction and students’ awareness about the “realness” of their peers (Garrison et al. 1999). It can be achieved through e.g. open communication and group cohesion. Lastly, teaching presence entails all facilitating aspects of an educational experience such as the course design and thus supports aspects related to cognitive and social presence (Garrison et al. 1999).

Despite the large quantity of studies on the CoI framework and the three presences (Garrison and Cleveland-Innes 2005; Shea et al. 2005; Akyol and Garrison 2011; Kyei-Blankson et al. 2019) it is yet unknown how the different types of presence are perceived by students in an interdisciplinary online course with a CBL approach.

The case: the inter-university sustainability challenge (IUSC)

In the IUSC bachelor students interested in sustainability worked on a complex, real-life challenge in an interdisciplinary and inter-university context and visualized their interdisciplinary, sustainable solution.

The overall learning objectives of this course were to become familiar with the Sustainability Development Goals (SDGs) from an interdisciplinary perspective and to develop an innovative and scientifically sound solution for one of the three challenges (1) Air Quality, (2) Energy Transition or (3) Urban Agriculture) using the interdisciplinary knowledge and skills within their team.

The course content reflects and integrates knowledge from all three universities and aimed for theoretical knowledge about the SDGs, interdisciplinary collaboration, critical thinking, systems thinking and design based research. A total of six teachers were involved in the course, two from each university. This interdisciplinary teaching team developed the course and played a role as teacher and/or coach. Weekly online lectures, workshops and (on demand) coaching sessions, resulted in an average of 5 weekly contact hours spread across three days a week for a 10-week period.

Due to the online setting, several educational technologies were integrated to support dialogue, flexibility and cognitive, social as well as teaching presence. Brightspace was used as a platform for the course environment. Lectures were held in an advanced Virtual Classroom (VC) at Utrecht University. The VC is a physical space designed for interactive remote teaching. The classroom is equipped with six large screens, each showing six students with whom the teacher can interact in a natural way (see Figure 1). For each group of six students, a camera is installed below their screen, following the teacher in the classroom, and enabling eye contact with the students. Remotely, students can see the teacher standing in the VC from a front-view or classroom view (backroom camera, Figure 1). The latter enables students to see their peers on the six big screens. The VC uses the weConnect digital platform, which has similar functions as MS Teams or ZOOM including screen sharing, polls & quizzes, breakout rooms and hand-raising.

Figure 1. Classroom view from the background camera in the virtual classroom.

Workshops (e.g. on systems thinking, academic writing) and coaching sessions were provided through MS Teams. Additionally, the course made use of simulation software Tygron EN Geodesign platform (https://www.tygron.com/en/) and Minecraft (https://www.minecraft.net/nl-nl) enabling students to visualize their challenge solutions as “artefacts”.

At the beginning of the course the teaching staff created interdisciplinary and inter-university student teams of 4 students taking into account students topic preferences. After the team formation, students started exploring their challenge topic by developing concept maps and writing their research proposal.

The course design supported interdisciplinary thinking and collaboration in various ways which is in line with the essential conditions and subskills mentioned by Spelt et al. (2009) as well as the eight dimensions of 2012). With the applied pedagogy (challenge-based learning) the course achieved interdisciplinarity, active learning and collaboration which are necessary conditions for interdisciplinarity in higher education (Spelt et al. 2009). The curriculum itself focused on broad sustainability topics and therefore provided disciplinary as well as interdisciplinary knowledge.

Already at the start of the course, the student teams were asked to reflect on their own and each other’s disciplines, strengths and limitations. During the course, bi-weekly reflection assignments required student to reflect on their disciplinary awareness, knowledge and interdisciplinary team collaboration. These also served as input for the weekly coaching sessions who could support students in the integration of the various disciplines.

Learning activities such as a group paper and a group visualization of the challenge solutions, explicitly required students to integrate all their disciplines in explaining the contribution of each discipline. In the group paper, students applied (inter)disciplinary knowledge and skills and described the relevance of their own discipline and the value of integration and interdisciplinary collaboration for the specific sustainability challenge. Via the online visualizations using Tygron or Minecraft students were asked to show their interdisciplinary process and solution in a creative way. An interdisciplinary teaching team co-developed and regularly co-taught lectures in an interactive online learning environment.

Being an elective course open to 2^nd, 3^rd and 4^th year Bachelor students, the course attracted students who were highly motivated, curious about sustainability, open and respectful to new ideas perspectives and students from other universities and backgrounds which also are necessary conditions for interdisciplinary higher education (Spelt et al. 2009).

Method

Through a case study methodology, quantitative as well as qualitative data were collected. A quantitative approach was used for the first research question, regarding transactional distance and presence. A qualitative approach was used for the other research questions regarding interdisciplinary competencies and the strengths and weaknesses of an online interdisciplinary, interuniversity course.

Participants

Due to the technical capacity of the Virtual Classroom, a maximum of 30 students could enrol in the online course. A total of 23 undergraduates from the three universities of which 15 students agreed to participate in our study. An overview of students’ study backgrounds is shown in Table 1.

Table 1. Overview of students’ (N = 13) study background.

Bachelor program	N	University
Sustainable Innovation	4	Eindhoven
Philosophy Politics and Economics	1	Utrecht
Human Geography and Spatial Planning	1	Utrecht
Architecture, Urbanism and Building Sciences	1	Eindhoven
Environmental Science	1	Wageningen
Industrial Design	1	Eindhoven
Educational Sciences	1	UU
Science and Innovation Management	1	UU
Chemical Engineering and Chemistry	1	Eindhoven
Nutrition and Health	1	Wageningen

By the end of the course, 13 out of the 15 students completed the survey. In this sample, 8 female and 5 male students were included. Their ages ranged from 18 to 24 years (M = 20.62; SD = 1.61). Most participants had Dutch nationality (N = 9). Other occurring nationalities were Austrian (N = 1), South African (N = 1), Hungarian (N = 1), and Romanian ( = 1). This study aimed to represent students from different universities and student teams, however participation took place on a voluntary basis. In the end, the highest number of participants was enrolled at Eindhoven University (N = 7), followed by Utrecht University (N = 4) and Wageningen University (N = 2).

Instruments

Being aware of the literature on CoI and transactional distance, this study applied two validated instruments (Arbaugh et al. 2008; Paul et al. 2015). To collect quantitative data on students’ perceptions (1) the Revised Scale of Transactional Distance (RSTD) by Paul et al. (2015) and (2) the community of inquiry instrument developed by Arbaugh et al. (2008) were used. Items of both questionnaires were slightly adapted in terms of wording for a better fit to the current study context.

The Revised Scale of Transactional Distance (RSTD) scale (Paul et al. 2015) is a revised version of Zhang’s (2003) original scale of transactional distance. The scale was used to measure students’ perceived transactional distance on three interaction levels: student-student, student-teacher and student-content. It contains 12 Likert-scale items with scores ranging from 1 (strongly disagree) to 5 (strongly agree) with a high score meaning less transactional distance which is preferable.

The community of inquiry instrument (Arbaugh et al. 2008) was developed to operationalize Garrison’s et al. (2000) community of inquiry (CoI) framework. It contains 34 Likert-scale items from 0 (strongly disagree) to 5 (strongly agree). To match it with the RSTD questionnaire a Likert-scale from 1 to 5 was applied in the study. The questionnaire was used to measure students’ perceived presence on three levels: teaching, social and cognitive presence. Unlike the RSTD scale, a high score on the presence scale refers to high presence which is preferable.

Qualitative data on student perceptions of interdisciplinary competencies and strengths and weaknesses were collected through the analysis of student course reflection papers and a semi-structured interview. The eight dimensions of interdisciplinary competence as listed by Lattuca et al. (2012) were used as the basis for the coding of competencies.

Data collection procedure

Upon ethical approval of Utrecht University, data was collected on several moments during the course period from February to April 2021. Students were invited via email to fill in the online questionnaires and to sign up for an interview. Questionnaire data were collected and stored with Qualtrics, a survey software used by Utrecht University. Participation was anonymous, voluntary and rewarded with a gift voucher. In the third course week, a pre-test survey investigating students’ prior knowledge and experience with the course topic and online learning tools was sent out per email. After seven weeks, an online questionnaire containing the RSTD and Presence scale was sent to the students. At the end of the course, a 1-hour lasting student interview took place, and the course reflection papers were handed in and analysed.

Data analysis

Of the 23 enrolled students, a total of 13 students filled in the questionnaires. Due to the small course size, quantitative analysis focused on frequencies and descriptive statistics.

The course reflection papers from 15 students were analysed in a qualitative way using content analysis. Interdisciplinary competences students perceived to have developed during the course (research question 2) were coded by the eight dimensions of 2012) using a deductive coding approach. An inductive coding approach was applied to analyse the perceived strengths and weakness of an online, interdisciplinary, inter-university CBL course (research question 3). Additionally, 4 students responded to the interview invitation.

The coding process for the content analyses was done following a strict procedure. To start with, 4 papers were independently coded by three authors, followed by a discussion. As a result, the first version of the codebook was created. Two of the three authors continued to code the remaining 11 papers (see Figure 2). After independently coding the papers, the coders checked for similarities and differences. Coding differences were resolved through discussion. In cases where no agreement could be reached the third coder got involved. During this process the codebook further developed. The final version can be found in appendix A (Appendix A is the Codebook used for the qualitative analysis of the student’s reflection papers).

Figure 2. Coding process.

Quantitative results

First students were asked about their prior experience with the applied pedagogies and software. A total of 13 students reported being experienced with MS Teams and interdisciplinary learning (see Table 2). They were less experienced with the concept of challenge-based learning. Brightspace, a digital learning environment used by Wageningen University was for obvious reasons less familiar to students from Eindhoven and Utrecht University.

Table 2. Means and Standard Deviations for student experience (N = 13).

	M	SD
MS Teams	3.7	0.48
Interdisciplinary learning	3.5	1.33
Challenge-based learning	2.9	0.76
Minecraft	1.9	1.26
Brightspace	1.6	1.12
Virtual Classroom	1.4	0.88
Tygron	1.0	0.00

Note: Range from 1 = no experience, 2 = novice, 3 = intermediate, 4 = advanced, 5 = expert. M = mean; SD = standard deviation.

Students’ prior knowledge of main course topics

At the beginning of the course, students indicated their prior knowledge level of the main course topics (Table 3). They were most knowledgeable about sustainable development goals (SDGs) and energy transition.

Table 3. Means and standard deviations for students’ prior knowledge.

	M	SD
SDGs	3.2	1.30
Energy Transition	3.1	1.12
Sustainable cities	2.6	1.26
Urban Agriculture	2.5	1.05
Air Quality	2.2	0.59

Note: Range from 1 = no experience, 2 = novice, 3 = intermediate, 4 = advanced, 5 = expert. M = mean; SD = standard deviation.

Research question 1: student perceptions of transactional distance

By the end of the course students had a moderately low perception of transactional distance which is reflected in high scores on the Transactional Distance scale shown in Table 4. In this scale, a 5-point Likert-scale was used: 1 (=strongly disagree) to 5 (=strongly agree). Note that a high score in Table 4 means low transactional distance which is desirable since it stands for a feeling of connectedness between students, students and teachers and students and course content.

Table 4. Means and standard deviations for items on the transactional distance scale (N = 13) after 7 weeks.

	Transactional Distance Student Teacher (TDST)
Item	M	SD
1. During lectures and workshops the teacher pays no attention to me. [R]	3.85	0.99
2. I receive prompt feedback from the teacher on my performance.	3.08	1.11
3. The teacher was helpful to me.	3.92	0.49
4. The teacher can be turned to when I need help in the course.	4.54	0.52
	Transactional Distance Student Content (TDSC)
Item	M	SD
5. This course emphasized synthesizing and organizing ideas, information, or experiences into new, more complex interpretations and relationships.	3.77	1.17
6. This course emphasized using and reflecting on existing knowledge for creating new sustainable solutions.	4.23	0.59
7. This course emphasized applying theories and concepts to practical problems or in new situations.	3.54	0.88
	Transactional Distance Student Student (TDSS)
Item	M	SD
8. I get along well with my team members.	4.00	0.71
9. I feel valued by my team members in this online course.	3.77	1.17
10. My team members in this online course value my ideas and opinions very highly.	3.54	1.05
11. My team members respect me in this online course.	4.38	0.87
12. My team members are supportive of my ability to make my own decisions.	4.23	0.83
Overall scores	46.85	4.47

Note: Scale ranges from 1 (strongly disagree) to 5 (strongly agree), R = recoded negatively worded item.

The total possible minimum and maximum scores on this scale were 12 (indicating high transactional distance) and 60 (indicating low transactional distance). The overall mean score for the Transactional Distance Scale at the end of the course was 46.85 (SD = 4.47).

On the TDST subscale, the majority of the students agreed with item 3 and 4. Teachers could be turned to for help (53,8% strongly agreed and 462% somewhat agreed). 846% agreed with item 3 and felt that the teachers were helpful. Only 385% (somewhat/strongly) agreed with item 2 which referred to teachers providing prompt feedback. On the TDSC subscale, students had the highest agreement with item 6 referring to the need to reflect on their prior knowledge for creating new sustainable solutions. 615% somewhat agreed and 308% strongly agreed. On the TDSS subscale, most of the students somewhat agreed or strongly agreed with item 8 (76,9%), while 923% somewhat agreed or strongly agreed with item 11 as well as with item 12. These results indicate that students got along well with their team members, that they felt respected and supported by them.

Research question 1: student perceptions of social, cognitive and teaching presence

Students showed high levels of presence (Table 5). Again, a 5-point Likert-scale was used 1 (= strongly disagree) to 5 (= strongly agree), however, in contrast to the transactional distance scale, a low score in Table 5 implies low perceived presence which is to be avoided in educational settings. The total possible minimum and maximum scores on the presence scale were 34 (indicating low presence) and 170 (indicating high presence). At the end of the course, the overall mean score for the presence scale was 119.62 (SD = 16.92).

Table 5. Means and standard deviations for items on the presence scale (N = 13) after 7 weeks.

Item	Teaching Presence
	M	SD
1. The teacher clearly communicated important course topics.	3.23	0.93
2.The teacher clearly communicated important course goals.	2.38	1.12
3.The teacher provided clear instructions on how to participate in course learning activities.	3.46	1.05
4.The teacher clearly communicated important due dates/time frames for learning activities.	2.92	1.44
5. The teacher was helpful in identifying various disciplinary approaches and views on course topics that helped me to learn.	2.62	1.12
6. The teacher was helpful in guiding the class towards understanding course topics in a way that helped me clarify my thinking.	2.85	1.14
7. The teacher helped to keep students engaged and participating in productive dialogue.	3.69	1.03
8. The teacher helped keep the students on task in a way that helped me to learn.	3.31	1.11
9. The teacher encouraged students to explore new concepts in this course.	3.77	0.83
10. Teacher actions reinforced the development of a sense of community among students.	3.31	1.03
11. The teacher helped to focus discussion on relevant issues in a way that helped me to learn.	3.38	1.33
12. The teacher provided feedback that helped me understand my strengths and weaknesses relative to the course’s goals and objectives.	3.15	0.99
13. The teacher provided feedback in a timely fashion.	4.23	0.83
	Social Presence
Item	M	SD
14. Getting to know other students gave me a sense of belonging in the course.	3.62	1.45
15. I was able to form impressions of some students.	4.31	0.63
16. Online communication is an excellent medium for social interaction.	2.23	0.83
17. I felt comfortable communicating in this online course.	3.69	1.03
18. I felt comfortable participating in the course discussions.	3.54	1.05
19. I felt comfortable interacting with other students.	4.31	0.95
20. I felt comfortable disagreeing with other students while still maintaining a sense of trust.	3.77	1.17
21. I felt that my point of view was acknowledged by other students.	3.92	1.19
22. Online discussions help me to develop a sense of collaboration.	3.69	1.03
	Cognitive Presence
Item	M	SD
23. Sustainability challenges posed increased my interest in course issues.	3.77	1.01
24. Course activities increased my curiosity.	3.62	1.19
25. I felt motivated to explore content related questions.	3.31	1.18
26. I utilized a variety of information sources to explore problems posed in this course.	4.00	0.71
27. Brainstorming and finding relevant information helped me resolve content related questions.	3.92	0.86
28. Online discussions were valuable in helping me appreciate different perspectives.	3.77	1.01
29. Combining new information helped me answer questions raised in course activities.	3.69	1.11
30. Learning activities helped me construct explanations/solutions.	3.08	1.12
31. Reflection on course content and discussions helped me understand fundamental concepts in this class.	3.62	0.87
32. I can describe ways to test and apply the knowledge created in this course.	3.38	0.77
33. I have developed solutions to sustainability challenges that can be applied in practice.	3.77	1.24
34. I can apply the knowledge created in this course to my education or other non-class related activities.	4.31	0.48
Overall scores	119.62	16.92

Note: Scale ranges from 1 (strongly disagree) to 5 (strongly agree); a high score implies high perceptions of presence which is desirable.

On the teaching presence subscale, the majority of the students somewhat agreed or strongly agreed with item 13 (92,3%) “The teacher provided feedback in a timely fashion”. Mixed responses were seen for item 7 and 9. 616% of the students (somewhat/strongly) agreed with item 7 “The teacher helped to keep students engaged and participating in productive dialogue” and half (53,9%) somewhat/strongly agreed with item 9 “The teacher encouraged students to explore new concepts in the course”.

Items least agreed with were related to course design aspects. Only 154% somewhat/strongly agreed with item 2 (69,2%) “The teacher clearly communicated important course goals” and had mixed feelings about item 5 (30,8% somewhat agreed) “The teacher was helpful in identifying various disciplinary approaches and views on course topics that helped me to learn”.

On the social presence subscale, the majority of the students somewhat agreed or strongly agreed with item 15 (92,3%) “I was able to form impressions of some students”, item 19 (84,6%) “I felt comfortable interacting with other students”, and item 21 (84,6%) “I felt that my point of view was acknowledged by other students”. Regarding online communication, 692% of students somewhat or strongly disagreed that online communication is an excellent medium for social interaction (item 16) whilst 308% had a neutral perspective or somewhat agreed.

All items of the cognitive presence subscale received a mean score above 3.00. The items with the highest student agreement were item 34 with 692% somewhat agree and 308% strongly agree and item 26 with 769% somewhat agree and 154% strongly agree. Students reported that they were able to apply the knowledge gained in this course to their education or other non-class related activities (item 34) and that they utilized a variety of information sources to explore problems posed in this course (item 26).

Qualitative findings

This section reports qualitative findings regarding the development of interdisciplinary competence (second research question) as well as perceived course strengths and weaknesses (third research question). Students’ reflection papers which were one of the graded course deliverables were coded and analysed. In the papers students were asked to describe (1) their contribution to the challenge, (2) the interdisciplinary team collaboration and (3) to reflect on the course. They were not specifically asked to reflect on interdisciplinary competences nor were they specifically asked to write about the strengths and weaknesses of the course.

Research question 2: interdisciplinary competences students perceive to have developed

In total 15 student reflection papers were analysed to investigate interdisciplinary competences students perceived to have developed during the course. A total of 14 students wrote about interdisciplinary competences in their reflection paper. The eight dimensions of interdisciplinary competence as described by2012) were held against the student reflection papers. Table 6 shows the eight dimensions of interdisciplinary competence used by Lattuca et al. (2012) in descending order of times being mentioned.

Table 6. Types of interdisciplinary competences mentioned in 14 student papers.

Type of interdisciplinary competence N = 14; R = 38	Typical Quotes from student reflection papers
Appreciation of disciplinary perspectives N = 9; R = 15	“First, you see that there are different approaches to problems. Often in other courses you work with people form the same study so you share paradigms and theoretical knowledge”. [ID5] “The course also helped me to look at problems from a different perspective, especially when the problem involves sustainability. My study is very much focused on sustainability, and we create a quite clear focus on such problems, but this course taught me to look at it from different angles in order to find a better solution”. [ID13]
Integrative skill N = 8; R = 10	“This course taught me the importance of including different disciplines, aspects into a project like economic, social and environmental aspects in order to come up with an inclusive solution”. [ID18] “By integrating the different disciplines we got a more inclusive design then when you do it with just one discipline. Everyone has a different view on the project which is very interesting”. [ID2]
Reflectivity N = 4; R = 5	“Through this course I did thus learn that design thinking can be experienced differently within another discipline”. [ID7] “The course has showed me that language when it comes to certain concepts can be an obstacle in interdisciplinary collaboration as it sometimes occurred group members failed to communicate the ideas they had, possibly due to a misunderstanding in used terminology”. [ID14]
Awareness of disciplinarity N = 3; R = 5	“Sometimes our methods were quite different. I am really used to having one hypothesis and then doing literature research. The system approach we took now is very different from what I would usually do”. [ID2]
Recognition of disciplinary limitations N = 2; R = 2	“You learn how you can use your knowledge in such multidisciplinary teams. Where your knowledge starts and where it ends and how you can use it. For example, I think I learned that I am quite a generalist with knowledge about different aspects, but that I lack some knowledge on a deeper level for certain aspects”. [ID5]
Ability to find common ground N = 1; R = 1	“We resolved this by really listening to each other and playing by our strengths”. [ID2]
Appreciation of non-disciplinary perspectives N = 0; R = 0	-

Note: N= number of student papers the competence was mentioned in; R= total number of times the competence was mentioned.

As shown in Table 6, the majority of students (N = 9) reported to have appreciated the different disciplinary perspectives in their team when working on a sustainability challenge. In this course they were asked to not only work with but also integrate different perspectives. Most students (N = 8) have described the importance and benefits of integrating different perspectives when working on a challenge solutions. Students reflected on their interdisciplinary project work (N = 4) and a few (N = 2) wrote about limitations of their own discipline and/or working in a monodisciplinary way. Since no external stakeholders were involved in this course, students did not write about non-disciplinary perspectives.

Research question 3: strengths and weaknesses students perceived of an online interdisciplinary, interuniversity CBL course

The same reflection papers were used to analyse the perceived strengths and weaknesses of the course. To get a better understanding of the strengths and weaknesses we clustered them into two categories (1) “level” and (2) “relation”.

The level indicates where the perceived strength/weakness originated:

1. individual student,

2. student team or

3. course.

“Relation” indicates where the strength/weakness relates to:

1. online learning,

2. interdisciplinary learning,

3. CBL,

4. inter-university learning and parallel courses,

5. this course design and (6) Covid/other.

As shown in Table 7, an equal number of students reported course strengths as well as weaknesses. Most strengths were related to the CBL approach and most weaknesses were related to the course design. An overview of all reported strengths and weaknesses can be found in Appendix B (Appendix B shows an overview of all reported strengths and weaknesses reported in the reflection papers).

Table 7. Number of strengths and weaknesses reported in 15 student reflection papers.

Related to …	Strength	Weakness
Online Learning	N= 3; R= 3	N= 5; R= 7
Interdisciplinary learning	N= 10; R= 22	N= 9; R= 14
CBL	N= 15; R= 59	N= 13; R= 39
Course Design	N= 13; R= 35	N= 15; R= 83
Inter-university	N= 5; R= 5	N= 6; R= 9
Covid/Other	-	N= 1; R= 2

Note: N= number of student papers a strength/weakness was mentioned in; R= total number of times a strength/weakness was mentioned.

Online learning

Strengths related to online learning were the acquisition of new knowledge, teaching and lectures. On an individual level, students felt that they have gained new knowledge and skills by using programs such as Tygron or Minecraft. Both programs could be used to visualize their challenge solution (artefacts). They also acquired experience with video making when preparing their artefact presentation. On a course level, the synchronous online lectures were received very well. The majority of students (N = 7) perceived the Virtual Classroom teaching as refreshing and more interactive compared to the other courses which due to COVID-19 have been given online via MSTeams. The results of the focus groups confirm the positive impressions of the Virtual Classroom. Additionally, the diverse, interdisciplinary teaching team and guest lectures were highly appreciated and perceived as valuable.

Weaknesses of online learning were mostly related to low motivation and energy levels. A high number of students (N = 9) struggled with this during the course caused by COVID-19 restrictions at that time and thus being forced to stay at home and follow online education for all their courses. Some students also mentioned that online collaboration and connection is harder to achieve compared to a face-to-face setting.

Interdisciplinary learning

The open and creative process, acquiring new knowledge and methods, applying prior knowledge, collaborative learning and student-team constellation were strengths related to interdisciplinary learning. Working in an interdisciplinary student team, students learn with and from each other. By integrating the different disciplines represented in their team, students could benefit from each other’s knowledge and skills. On an individual level, students learned how to collaborate with other disciplines. During the interdisciplinary collaboration students got the space to come up with creative ideas. Additionally, they were able to apply knowledge and skills from previous courses and benefitted from the prior knowledge of their peers. The interdisciplinary student-team constellation was highly valued by students since it gave them the means to develop a more complete solution.

Despite the previously mentioned benefits of interdisciplinary collaboration, students perceived struggles with it. Communication and project management within an interdisciplinary team can be more complicated and time intensive. Misunderstandings and explaining different understandings of the same concepts to each other was perceived as “slowing down the process”.

Challenge-based learning

Similar to online and interdisciplinary learning, strengths of CBL were related to the open and creative processes, applying prior knowledge, the student-team constellation and the teachers. Additionally, students valued acquisition of collaboration skills that were required in this CBL approach and the related personal development they went through. Almost all students (N = 13) reported the open, creative process as a strength of this course.

“The course of Inter-University Sustainability challenge has been an out of the box thinking period”. [ID16]

Students got the space to roam free, try out different ideas and self-direct their working progress. They were encouraged to find creative solutions and making use of the interdisciplinary knowledge available to them. By collaborating in teams, students (N = 11) reported to have further developed their collaboration skills such as good communication and planning. They had to divide their tasks, manage their time and build good connections with their teammates. Working in a self-directed and interdisciplinary way on an open complex challenge gave students the possibility to gain new knowledge and skills not only from the interdisciplinary teaching team but also from each other. The support students got from coaches and teachers was valued since it gave them focus and direction when needed.

“The knowledge and skills I gained was of multiple uses and disciplines. One of the most important insights are related to group work but also, I improved my analytical and writing skills”. [ID11]

Although the majority of comments related to CBL were strengths (N = 59), students still perceived a high number of weaknesses (N = 39). Among the weaknesses students mentioned difficulties in interdisciplinary collaboration, low motivation and energy, difficulties of collaborative writing and dissatisfaction with their team constellation.

As is often the case with teamwork, not all students (N = 10) were satisfied with their teammates. Difficulties with task division, speaking up, connecting online, low quality input, low motivation of peers as well as poor communication with their team were reported. Collaborative writing was perceived as challenging (N = 4). Students reported that their reports lacked critical reflection, structure and that it was hard to combine different writing styles. A lack of interdisciplinarity within the teams was also perceived as a weakness when participating in a course that is based on a CBL approach.

Inter-university learning/parallel courses

Students reported that learning in an inter-university course was beneficial for their personal development and collaboration skills. They learned from the collaboration with different perspectives and appreciated the inter-university set-up of the course.

Mentioned weaknesses were related to parallel courses students were taking. No weaknesses related to the inter-university character were mentioned. Deadlines of parallel courses which overlapped with this course deadlines lead to personal time management issues which caused extra stress and lower motivation especially in the last course weeks.

Course design

Strengths related to the course design were as previously mentioned, the open creative process, gaining new knowledge and skills, personal development, being able to apply prior knowledge and the student-team constellation. Additionally, students valued the course set-up, the lecture content and felt highly motivated. Students felt well-informed about tasks, deadlines and enjoyed this course experience. The topics covered in the lectures were described as insightful, new, interesting and complementary to previous lectures they had in other courses. They were excited to collaborate in a team and work on a challenge.

“The setup and organization of the course was one of the best throughout my entire educational experience. We were extremely well-informed about weekly tasks, and happenings, which is uncommon in my experience. The fact that there were so many teachers involved, both as mentors and as guest lecturers, was incredibly impressive and valuable”. [ID17]

From all the mentioned weaknesses, the most were related to the course design. At the beginning of the course, students could choose between three challenge topics. Regardless of the challenge topics, all students received the same lectures which were mandatory. This resulted in some students (N = 7) being dissatisfied with the course set-up and the relevance of the lecture content.

“I would say the lectures could be connected better to the research of the students”. [ID16]

A majority of students (N = 13) perceived unclarities about the course deadlines, requirements, amount of deliverables and were unsatisfied with the weightages of the deliverables. Being unfamiliar with the Tygron program, students underestimated the time investment it required and needed extra support. Due to the number and difficulty of assignments, students felt a high workload in this course.

“None of us had experience with Tygron so it was a challenge”. [ID4]

Additionally, some student teams lacked interdisciplinarity caused by student drop out and an overrepresentation of certain disciplines/study backgrounds within a team. Some students (N = 8) struggled with the self-directed learning required in CBL. They found themselves struggling with deciding what to do and how to do it, got lost in the vagueness and had difficulties creating a clear strategy.

“Personally sometimes I got lost in the structure of our project. We all had different ideas and in the end we wanted to implement so much that our project got quite extensive”. [ID2]

COVID-19 restrictions/other

One mentioned weakness was specifically related to covid.

“Throughout this quarter I have struggled with staying focussed, Covid has not allowed me to be in a productive environment for most of my time”. [ID6]

Conclusion & discussion

The world is facing many urgent sustainability challenges which can only be addressed in an interdisciplinary way. Several universities aim to provide students with interdisciplinary collaboration skills by offering learning opportunities, e.g. in Challenge based learning and interdisciplinary online courses. However, the effectiveness of these course designs really depends on the perceived transactional distance and presence in these courses. Literature for these settings, and especially for the student perspective, are underdeveloped. In this research we focus on the case of the “Inter-University Sustainability Challenge” course, an online CBL based course that focusses on “Sustainable cities of the future”. The course focusses on inter-disciplinary and inter-university learning. This research set out to study 1) student perceptions of transactional distance and presence, 2) perception of acquired interdisciplinary competences and 3) perceived strengths and weakness of online, interdisciplinary, inter-university CBL courses. Upfront our starting hypotheses were that students would perceive low transactional distance and high presence. The results of this study show that students had low perceptions of transactional distance (TD), low distance is desirable, and high perceptions of presence. The most reported strengths and weaknesses were related to CBL approach and the course design. Additionally, almost all students reported to have used and developed interdisciplinary skills during this online course. The two most used ID competencies were appreciation of disciplinary perspectives and integrative skill.

This case study confirms that low levels of transactional distance and high levels of presence can be achieved in an online, interdisciplinary, inter-university course. This study supports evidence from a previous study on transactional distance and engagement in online education by Bolliger and Halupa (2018) who reported moderately low levels of transactional distance with a mean score of 47.74 (SD = 6.94) in 629 participating students.

The reported weaknesses regarding the course organization, the set-up and teacher communication are expected to have a negative impact on the perceived transactional distance and presence in this course. Since this course was a pilot run, improvements regarding the course organization and set-up are to be expected. At the beginning of the course, students did not have many deadlines and they were still exploring how to tackle their challenge. However, by the end of the course, when the online questionnaire was filled out and students delivered their course reflection papers, they had overcome several obstacles regarding teamwork, collaborative writing and assignments. Although students valued interdisciplinary collaboration and perceived to have gained collaboration skills, some struggled to meet the deadlines and to work efficiently.

Perceptions are fluid, can change over time and are difficult to measure during a 10-week online course that includes lectures from a diverse teaching team and a variety of learning activities. Despite the small sample size, the findings are promising and support the further development and evaluation of online CBL courses with a focus on interdisciplinary collaboration.

It could be argued that CBL inherently supports high presence, high dialogue and collaboration, high course structure flexibility and high learner autonomy which are aspects that are expected to result in low perceptions of transactional distance. Yet, successful interdisciplinary collaboration within CBL still requires teachers and course design support. (Spelt et al. 2009). Interestingly, students did not report online learning as a major factor influencing their perceptions and learning. However, it can be expected that students’ overall perception of this course got affected since all their courses were provided online. Compared to other university courses that had to promptly switch to online education due to COVID-19 regulations, this course was intended to be provided in an online format pre-COVID-19. Therefore, this course was one of the many online courses students were following.

While preliminary, these findings suggest that CBL is appreciated by many students when working on real-life challenges e.g. sustainable cities. The findings also suggest that CBL and interdisciplinary learning in an online setting is feasible and fruitful. It enables students to learn in a creative, self-directed way in which the development of collaboration and interdisciplinary skills i.e. integration of various perspectives and disciplines is possible. This is in line with related research on CBL in an online learning setting (Colombari et al. 2021; Barynienė et al. 2022).

The fact that this was the first course run and that the course was co-developed by teachers from three universities should be accounted for. It is expected that mentioned course design weaknesses are improved in the second course run where the study will be repeated. We are interested in follow-up studies with a bigger and more diverse student group. Regarding the findings on interdisciplinary competences, it would be interesting to measure student competences next to collecting student perceptions. Additionally, future research could include and compare teacher and student perceptions of transactional distance and presence.

Acknowledgments

This work was supported by the incentive programme for open online education, executed by SURF and financed by the ministry of Education of the Netherlands, under Grant number OO20-24.The development of the IUSC course was funded by the alliance between Eindhoven University of Technology, Wageningen University & Research, Utrecht University and the University Medical Center Utrecht.

Disclosure statement

No potential conflict of interest was reported by the authors.

Data availability statement

The data that support the findings of this study are available on request from the corresponding author, J.Kasch. The data are not publicly available due to them containing information that could compromise the privacy of research participants.

Additional information

Funding

The work was supported by the Ministerie van Onderwijs, Cultuur en Wetenschap .

Appendix

Quantitative question answered by survey questions

RQ1: How do students enrolled in an online interdisciplinary, challenge-based sustainability course perceive transactional distance and presence?

Qualitative Research Questions; related to the Personal Reflection Papers + Interviews

RQ2: Which interdisciplinary competencies do students perceive to have developed in an online, interdisciplinary CBL course on sustainable cities?

RQ3: Which strengths and weaknesses do students perceive of an online interdisciplinary CBL course on sustainable cities?

Codes Research Question 2

Interdisciplinary Competences Lattuca et al., 2012

Table

Step 1

Read the personal reflection paper with research question 2 in mind. After you have coded/highlighted text relevant to RQ2, you will continue with research question 3. Paper Lattuca et al. (2012) will be used for deductive coding the interdisciplinary (ID) competences. Highlight text that relates to interdisciplinary skills in green and add as a comment which of Lattuca’s interdisciplinary competence dimensions it is. Text will only be coded as a competency if the student is writing that (s)he acquired/developed this interdisciplinary competency through/during this course.

Table

1	Awareness of disciplinarity
2	Appreciation of disciplinary perspectives
3	Appreciation of non-disciplinary perspectives
4	Recognition of disciplinary limitations
5	Interdisciplinary evaluation
6	Ability to find common ground
7	Reflexivity
8	Integrative skill

Codes Research Question 3

Strengths & Weakness

Table

Step 2

Strengths and weaknesses will be coded in an inductive way (emerging from the students’ texts). Read the personal reflection paper with research question 3 in mind. Each strength and weakness will receive a code indicating the “type” a code for the “level” and a code for “relation”.

Perceived type strengths

Table

	Types of strengths	Code description
	Open creative & critical process	Includes text units that focus on the CBL approach, creativity and critical thinking during the learning process.
	Collaboration skills: communication & planning	Text units focusing on collaboration skills students acquired/developed due to collaboration.
	Teachers & Teaching	Includes teaching quality, coach support, diversity of teaching team etc.
	Applying prior knowledge & skills	Prior skills and knowledge students were able to apply in this course
	New knowledge & methods	New (interdisciplinary) knowledge and methods students learned from peers and/or the course content/lectures/assignments. Also includes online tools such as Tyrgon, Minecraft or video making tools.
	Collaborative Learning	Text units that highlight the benefits of collaboration. Does not focus on skills but collaborative learning as such.
	Lecture content	Strengths related to the content that was shared and discussed during lectures.
	Personal Development	Focuses on the personal growth a student has gained due to this course.
	Motivation & Energy	Text units that indicate high motivation and energy gained through this course.
	Student-team constellation	Includes text units about the interdisciplinarity of student teams and student drop out within their teams.
	Course set-up & organization	Text units focusing on all aspects that are related to the course design in terms of organization, deadlines, grading, teacher communication.

Perceived type weakness

Table

	Type of weakness	Code description
	Open process	Includes text units that focus on the CBL approach, the openness, feelings of getting lost, a lack of critical thinking during the learning process.
	Collaboration skills	Text units focusing on the lack of collaboration skills of peers within a student-team. For example poor communication, not showing up for team meetings, task division etc.
	Collaborative writing	Challenges related to collaborative writing within the student team.
	Time management	Personal time management issues that are caused by students themselves or factors outside of the course.
	Student-Team constellation	Includes text units about the interdisciplinarity of student teams and student drop out within their teams.
	Assignment clarity & difficulty	Text units that focus on course assignments and improvements that could be made regarding the clarity of requirements and the difficulty.
	Overlap prior knowledge	Students writing about too much overlap between their existing knowledge and the topics covered in this course.
	Lecture content & quality	Text units focusing on the topics that were (not) covered in the lectures and their quality in terms of how interesting/appealing they were.
	Motivation/energy level	Text units that indicate high motivation and energy gained through this course.
	Interdisciplinary collaboration	Collaboration challenges caused due to interdisciplinary differences.
	Course set-up, communication & organization	Text units focusing on all aspects that are related to the course design in terms of organization, deadlines, grading, teacher communication.

Code for the ‘Level’

Table

Student perceives this strength/weakness originating at a …
Individual level	Purely personal aspects such as illness, stress, COVID-19, parallel courses, personal shortcomings etc.
Team level	Affects the student team and was created/caused by the team itself
Course level	Affects all students of the course. It was created/caused by the course design/teachers.

Code for the ‘Relation’

The relation indicates to what a strength/weakness is related to. This can be one or several of the 5 options:

Table

Strength/weakness related to
(1)CBL approach	CBL learning in general; in general team work
(2)Online learning	Learning in an online setting: online lectures, online collaboration etc.
(3)Interdisciplinary learning	Learning between students from different disciplines
(4)Inter-university learning/working; parallel-course	Learning between students from different universities or courses
(5)This course design	Everything that relates to the practical aspects e.g. lectures, grading, assignments, schedule; group size
(6)Covid/Restriction/Other

B. Qualitative Results

Strengths

Table

Type of strength N = 15; R = 114	Typical Quotes	Summary of all quotes
Open, creative & critical process N = 13; R = 25	“the course of Inter-University Sustainability challenge has been an out of the box thinking period”. [ID16] “I got to experience and work with new perspectives. For me, this was the main added value of this course”. [ID 7]	Students were invited to be creative and innovative when working on a challenge solution. Looking at a challenge from different perspectives and being required to integrate different disciplines, students learned to critically think about the choices they made. The CBL approach gave students space to roam free and try out different things. Working in interdisciplinary teams on an interdisciplinary challenge required out of the box thinking.
Collaboration skills: communication, planning N = 11; R = 24	“During the course, I learned that I can recover from the chaos that I felt, but only with the help of my team members. Communication is key and your team members might be struggling with the same things you are”. [ID12] “I learned about teamwork and working with students from other universities”. [ID17]	Working in (interdisciplinary) groups, students learned about the importance of open communication, friendly discussions and mutual understanding. They learned to divide their tasks, manage their time and build good connections with their team mates.
New knowledge & methods N = 10; R = 18	“The knowledge and skills I gained was of multiple uses and disciplines. One of the most important insights are related to group work but also, I improved my analytical and writing skills”. [ID11] “I have learned a lot about operating in a multidisciplinary setting and the research methods of students doing another bachelor than me”. [ID14]	Working in interdisciplinary teams and getting lectures from an interdisciplinary teaching team enabled students to broaden their knowledge and certain sustainability topics but also provided them with new information research methods and applications
Applying prior knowledge & skills N = 8; R = 10	“I really enjoy creating, therefore I was the one for our group to edit the video. It had been a while since I edited a video, but it was a great opportunity to get back into it”. [ID12] “The knowledge from my psychology elective courses helped me to understand and predict consumer behaviour and potential changes which was relevant for the long-term solution proposed in our project”. [ID4]	According to the students, this course gave them the opportunity to make use of their prior knowledge regarding several aspect e.g. CBL, teamwork, interdisciplinary learning, video editing skills and data analysis
Personal Development N = 7; R = 8	“My biggest achievement while attending this course is personal growth as a team worker and as an individual researcher”. [ID11] “I was forced to work quickly and efficiently, which is definitely an important skill to learn and improve”. [ID17]	Students reported personal growth in several areas such as: being a team player, flexible, working efficiently, communication and writing skills.
Teachers & Teaching N = 7; R = 7	“I am glad we had the tutor meetings on Tuesday afternoons since those typically gave us some direction on where to go next”. [ID4] “The Virtual Classroom is an amazing tool, and it was always refreshing to participate in the lectures in that environment, rather than the mundane Microsoft Teams call I am growing tired of”. [ID17]	Lectures were given in the Virtual Classroom at [anonymized] which students experienced as interactive, refreshing and high quality. They enjoyed the enthusiastic teachers and felt motivated.
Lecture content N = 5; R = 6	“the materials of the course are complementary to my study. Furthermore, the subjects given in the course were interesting and very diverse”. [ID3] “It really re-ignited my love for projects around the topic of sustainability”. [ID12]	Interactive lectures provided students with diverse, new and interesting information. Students enjoyed the topics and lectures given.
Collaborative learning N = 4; R = 4	“thoroughly enjoyed collaborating with people from all different disciplines. I had a different look towards this report compared to any other report I had ever written”. [ID12] “However, I personally learned a lot about education and child psychology from this member through discussions and debates about the topic”. [ID17]	Learning from others; learning with others helped
Student-team constellation N = 4; R = 4	“I also got to know other students, normally this is not really my strength as I tend to be a bit shy and do not really make steps to meet new people”. [ID5] “Our group was a nice mix of different disciplines, which was essential in reaching our final product, as the system we focused on had both social, technical, political and economic aspects”. [ID8]	Students enjoyed the different disciplines in their team and saw the importance of interdisciplinary collaboration.
Course set-up & organization N = 2; R = 6	“We were extremely well-informed about weekly tasks, and happenings, which is uncommon in my experience. The fact that there were so many teachers involved, both as mentors and as guest lecturers, was incredibly impressive and valuable”. [ID17] “It is very great however that you took the many deadlines we have into account and that you offered flexibility in terms of offering options for postponement”. [ID18]	The course was well organized and provided students with the needed flexibility regarding deadlines. Students felt well informed about tasks and deadlines and enjoyed the diversity of the teaching team.
Motivation & Energy N = 2; R = 2	“The energy then increased being in a new team and dealing with a new project”. [ID4] “I was excited of the fact this was a new course and I never followed a challenge based course”. [ID18]	Students felt excited to work in a team/project

Weaknesses

Table

Type of weakness N = 15; R = 164	Typical Quotes	Summary of all quotes
Course set-up, communication & organization N = 13; R = 41	“Some aspects of the course were not thought through enough. These aspects are planning of the course, lectures, and requirements. I see the planning of the course to be imperfect because our team was always in an environment of time limitations”. [ID11] “An improvement I would recommend for this course, entails the before-mentioned clarity. A schedule including all aspects of the course would have helped me in the first place, but a clearer course guide including the assignment descriptions (and rubrics) would have helped even more”. [ID7]	Communication regarding the course structure was perceived as unclear. Students did not know what was expected from them, were unsatisfied with the weightage and deadlines of the deliverables.
Assignment clarity & difficulty N = 12; R = 17	“What the course can improve is that at the beginning of the course it was not clear what the reports should look like”. [ID18] “Tygron was a bit hard to do and did not get the results that I wanted”, [ID3]	Unclarities about the group assignment were reported. Some requirements were perceived as unclear and vague but also irrelevant.
Collaboration skills N = 10; R = 24	“Since other students are from other disciplines, one cannot always help the other to do their part. Missing both knowledge of their expertise and insight into these students’ perspective created misunderstandings on both ends. For me this was the most difficult part on the inter-disciplinary aspect of the course”. [ID7] “Our group did not function optimally in terms of work division and communication. However, this can hardly be attributed to the interdisciplinary nature of the course, but rather our personalities and work ethic”. [ID17]	Student collaboration was not always easy. Students experienced difficulties with task division, speaking up, connecting online, low quality input, low motivation of peers as well as poor communication with their team.
Motivation & energy level N = 9; R = 14	“Throughout this quarter I have struggled with staying focussed, Covid has not allowed me to be in a productive environment for most of my time”. [ID6] “I had several other deadlines and exams. Because I was very busy, my energy levels were lower than ever before in the course”. [ID16]	Low(er) motivation and energy levels were experienced due to deadlines in parallel courses. Additionally, students struggled with following complete online education for months.
Open process N = 8; R = 19	“Personally sometimes I got lost in the structure of our project. We all had different ideas and in the end we wanted to implement so much that our project got quite extensive”. [ID2] “While we clung to the idea for weeks and wrote paragraphs about this aspect of the project, we soon realized that it was not truly fitting to our solution, and we were forcefully trying to include it in order to check the interdisciplinarity box, rather than realistically and logically assessing the situation”. [ID17]	The student-centred, open process was experienced as challenging. Students found themselves struggling with deciding what to do and how to do it. They got lost in the vagueness and had difficulties creating a clear strategy.
Student-team constellation N = 7; R = 18	“the benefits of interdisciplinary might not work at such a small scale with such little students (merely 5, with 4 from the TU), if not all students keep a constant critical look on the research conducted by the other students”. [ID10] “Working with three students with the exact same background, especially Sustainable Innovation, in a course titled Sustainability Challenges, might have been overwhelming and discouraging for those members”. [ID17]	The lack of interdisciplinarity, in some student teams was perceived as disappointing and difficult to manage. This was caused by student drop out and an overrepresentation of students with the same study background.
Lecture content & quality N = 7; R = 10	“As for the lectures, I found many lectures to be irrelevant to our specific topic and the fact that they were compulsory was confusing and disrupting to the process of group work”. [ID11] “I would say the lectures could be connected better to the research of the students”. [ID16]	The added value and connectedness to the research topics of some lectures was questioned.
Time management N = 6; R = 7	“creating the artefact in Minecraft, although it was easy it took more time than I had previously expected”. [ID6] “The last week was very challenging due to the fact that 3 courses had their deadlines at the same moment”. [ID16]	Students experienced difficulties managing their tasks within this course in combination with obligations in parallel courses or due to personal reasons.
Overlap prior knowledge N = 5; R = 6	“I often found myself unengaged during lectures as I wasn’t truly intellectually stimulated due to my pre-existing experience with many of the topics”. [ID17] “I had difficulties keeping up with the lectures. This was mainly because almost every topic was already discussed in my study at least one time, and some even more”. [ID13]	Students with a sustainability background were familiar with parts of the course lectures. This overlap was experienced as boring.
Collaborative writing N = 4; R = 4	“I think the one of the most difficult parts of the course was ensuring that all of the work that the group had created had a clear narrative in our written assignments. Mixing various writing styles was a drawback of working together, it took more time than expected to review, clarify, edit and accept the texts from all people involved”. [ID16]	Writing a paper with an interdisciplinary group of students was perceived challenging. Students struggled to write in a structured way and to handle different writing styles.
Interdisciplinary collaboration N = 3; R = 4	“Before realizing what my role was, I found myself struggling with finding my way to contribute”. [ID7] “certain challenges to this such as some terminology and theories being unfamiliar to me and having to catch up with that”. [ID4]	Collaborating in an interdisciplinary team requires students to understand each other’s perspectives. Students perceived it challenging to find a common ground and to find ways to integrate their own knowledge and skills in the team process.