Conceptualising variety in challenge-based learning in higher education: the CBL-compass

ABSTRACT

Increasingly higher education programs are made learner centred and flexible to face societal changes. Challenge-based learning (CBL) is an educational concept shaping these open and flexible programs. This article aims to articulate a framework for analysing CBL characteristics within and between study components in academic curricula. It contributes to a detailed conceptualisation of CBL and clarity on what CBL implementations consist of. The dimensions and indicators of the framework reflect points of attention for research and evaluation of CBL design and implementation. We argue for variety in CBL characteristics between study components or curricula. Furthermore, we point out how this conceptualisation of CBL opens for research into designing and teaching for multiple domains, and how it contributes to an identification of commonly agreed characteristics of CBL. Recent CBL projects are referenced as an illustration of the approach. The detailed conceptualisation informs debate and development in a nascent field of research.

KEYWORDS:

• Challenge-based learning
• higher education
• educational innovation
• conceptualisation
• definition

1. The need for conceptualising challenge-based learning

Today's global challenges, such as climate change, energy renewal, biodiversity, healthcare, or migration, are complex, and often open-ended and ill-defined (Gómez Puente, Van Eijck, and Jochems 2013). Some challenges are even called ‘wicked’ (Lönngren 2019) because every aspect appears to be related to everything else. These challenges go beyond the traditional tasks and responsibilities of professionals in fields such as engineering, healthcare or design (Vojak, Price, and Griffin 2010). In response, many universities make their educational programs learner centred and flexible to face the challenges demanded by a changing and uncertain world (Gallagher and Savage 2020).

Higher education institutions’ efforts towards open and flexible curricula or study components can be found under a variety of labels such as challenge-based education (e.g. Charosky et al. 2018; Pisoni and Gijlers 2021), challenge-based instruction (e.g. Quweider and Khan 2016; Roselli and Brophy 2006), or challenge-driven education (Högfeldt et al. 2019; Magnell and Högfeldt 2015). The definitions behind this variety of labels and purposes (see also Gallagher and Savage 2020; Leijon et al. 2021) share how challenges are seen as self-directed work scenarios in which students engage (Johnson et al. 2009; Gaskins et al. 2015). The goal of these challenges is to learn to define and address the problem and to learn what it takes to work towards a solution, rather than to solve the problem itself. The final deliverable can be tangible or a proposal for a solution to the challenge (Membrillo-Hernández and García-García 2020). The idea is to implement these challenges in engaging approaches to teaching and learning that encourage students to collaborate and develop deeper subject knowledge and share their experience (Nichols and Cator 2008).

Definitions of CBL trace back to a pilot study by the Apple company. This pilot aimed to make education more motivating and relevant to students (Johnson et al. 2009; Nichols and Cator 2008). Malmqvist, Kohn Rådberg, and Lundqvist (2015) translated the Apple approach to higher engineering education, with a focus on learning as a collaborative multidisciplinary experience, taking place in an international context, with the aim to find a sustainable solution. Thus, Malmqvist, Rådberg, and Lundqvist (2015) propose a wider scope of grand sociotechnical problems. In general, existing research presents descriptive case studies of CBL as an educational intervention based on either of these definitions (Leijon et al. 2021). The preference for descriptive case studies is understandable for a field in its infancy that is trying to define CBL (Gallagher and Savage 2020). However, the lack of a conceptualisation of CBL in terms of dimensions and measurable indicators, potentially leads to definitional muddying.

Current case studies most often include CBL as an approach to supplement existing structures, rather than as embedded curriculum practice (Gallagher and Savage 2020). However, if universities intend to use CBL as a concept to make their educational programs open, flexible and learner centred (Gallagher and Savage 2020; Membrillo-Hernández et al. 2019), a developmental perspective is needed. This perspective aims to scaffold learning with a series of challenges, which implies a variety in CBL characteristics across study components, ranging from small-scale to full-fledged versions of challenges and their implementation. Hence, we need a conceptualisation of CBL that allows for discussing and researching variety in implementations. Existing literature shows a limited understanding of this variety in CBL characteristics, and how it affects research and educational development.

Taken together, we are in need of clear definitions of characteristics representing CBL, and a specification that allows for measuring these characteristics. Therefore, this article aims to articulate a detailed framework for analysing CBL characteristics within and between study components in an academic curriculum. Instead of focusing on theoretical differences and diversity in descriptions of CBL, the present study searches for all-embracing commonalities of CBL in education. These commonalities are brought together in a framework that allows for variety in CBL characteristics between study components or curricula. This framework can serve as a methodological approach, and contributes to an identification of commonly agreed characteristics of CBL aiming to provide clarity to practitioners and researchers on what CBL implementations consist of (Leijon et al. 2021).

Conceptualising CBL is needed for not only descriptive but also explanatory research, for example on mechanisms supporting efficacy and success of CBL implementations. Despite its promise for education, evidence for CBL is still scarce, and mostly limited to benefits for students. Reported benefits include industry networking, technical skills, application of skills in a real-world environment, teamwork, problem-solving skills, a deeper understanding of knowledge, and innovative thinking ability (Gallagher and Savage 2020).

The prevalence for CBL in engineering education offers a starting point to find common ground, and to subsequently formulate dimensions and indicators in ways useful for other domains. Currently, CBL approaches within the literature, though few, included students from medicine, law, and marketing (Eraña-Rojas et al. 2019). It is also in engineering education that CBL is studied as embedded curriculum practice (Malmqvist, Kohn Rådberg, and Lundqvist 2015; Membrillo-Hernández et al. 2019; Doulougeri et al. 2022), rather than as a novel pedagogical approach to supplement existing structures. This reinforces the idea of a wider scope and variety in CBL characteristics, which contributes to a conceptual basis in flexibility (Gallagher and Savage 2020), needed to inform debate and development in a field of research that is still in its infancy.

2. Current conceptualisations of CBL

Conceptualising CBL can be problematic, because it is often perceived as an intervention or teaching method (Leijon et al. 2021; Johnson et al. 2009). In our perception, CBL as an educational concept represents views on what is worth learning and how students should acquire that learning (cf. Thomas 2001). It underscores a complex set of educational practices that ask for a specific organisation. These practices include vision and support, but above all teaching methods, which in turn can be defined as the principles and activities used by teachers to enable student learning.

Understanding the current conceptualisations of CBL gives an idea about how our conceptualisation compares with them. Understanding prior definitions of key characteristics of our conceptualisation also helps us to decide whether we plan to challenge those definitions or rely on them for our own work. Recently two literature reviews aimed to conceptualise CBL by covering the whole field of empirical studies. These two review studies are important landmarks. However, their conclusions also show the need for a more thorough conceptualisation in terms of dimensions and indicators, to advance both CBL research and practice.

Gallagher and Savage (2020) conclude from their review that CBL is perceived as a flexible approach that frames learning with challenges using multidisciplinary actors, technology-enhanced learning, multi-stakeholder collaboration and an authentic, real-world focus. They continue to conclude that a lack of definitional clarity coupled with variety in approaches and frameworks presents problems for educators and researchers. They foresee problems in both implementing CBL and establishing the efficacy of CBL due to a lack of consistency of reported results. Their review resulted in a preliminary conceptual framework summarising the following key defining features of CBL: global themes, real-world challenges, collaboration, technology, flexibility, multi-disciplinarity and discipline specificity, creativity and innovation, and challenge definition.

Although we acknowledge the importance of this conceptual framework, it indicates also the lack of attention in existing research for amongst others the role of stakeholders, self-directed learning, assessment, or support (Van den Beemt, Van de Watering, and Bots 2021). Furthermore, the key features appear complex for both educational design and research. Therefore, we are in need of dimensions to specify each key characteristic, and indicators to operationalise the characteristics.

Leijon et al. (2021) use their literature review to express a critical stance towards CBL research. They conclude that neither in the initial Apple paper nor in the guideline from 2016, CBL is explicitly theoretically grounded. Still, many studies in their review used the Apple definition as a starting point, while only a few studies showed a more critical understanding of CBL. This critical discussion appears missing especially when CBL is reduced to a model for pedagogic intervention. However, when this discussion is included, it shows how CBL invites a holistic and critical understanding of knowledge production and learning processes. Leijon et al. (2021) continue to conclude that a critical analytic approach towards learning was marginally present in the majority of articles, which can be explained by the disciplinary dominance of engineering with less focus on learning theories compared to educational science.

Conclusions for research that can be derived from these two conceptualisations are that we are in need of instruments that support exploring CBL and, amongst others, student perception, praxis, and evidence on learning. This in turn requires a next step in describing dimensions and indicators of CBL. Conclusions for educational practice are the suggestion of a curriculum approach and critical discussion, and valuing variety and flexibility in CBL, yet with a common grounding within and across disciplines.

3. Conceptualising CBL in terms of variety

To conceptualise CBL in terms of variety, we propose a framework in two parts: a high-level conceptual framework, and for each concept a set of accompanying dimensions and indicators (see Figure 1 for an overview of dimensions, and Table 1 for a detailed listing of all dimensions and indicators). This conceptual framework builds on a basic why-how-what approach (Sinek 2009), which supports thinking about educational strategies from the ground-up. The high-level concepts allow to identify educational processes at the three levels of vision, teaching and learning, and support (Van den Akker 2003; see also Van den Beemt et al. 2020). They also allow research focussing on one or combinations of concepts. The dimensions and indicators together form the basis for an educational perspective on CBL. Our argument is not that all characteristics are fully present in every project or course. Rather, we expect a variety of designs and perceptions of CBL to be found in current and future study components.

Figure 1. Dimensions of challenge-based learning.

Table 1. CBL-compass: dimensions and indicators.

Vision
Real-life open-ended challenges	The extent to which challenges are real-life and authentic	theoretical/abstract to real-life
	The extent to which challenges are open-ended	pre-defined to open-ended
	The extent to which challenges are complex	one-dimensional to complex
	The extent to which challenges are interdisciplinary	mono-, multi-, interdisciplinary
Global themes	The extent to which challenges focus on transforming business-as-usual practices and raising awareness and trust among actors	no focus to full focus
	The extent to which challenges focus on short-term societal impact or long-term societal impact	no focus to full focus
Involvement of stakeholders	The extent to which challenges have a challenge owner from (1) academia, or from (2) industry, government, or culture	internal to external
	The extent to which challenges require collaboration with external stakeholders	no collaboration to full collaboration
Teaching and learning
T-shaped professionals	The extent to which learning activities create a rigorous treatment of fundamental knowledge and skills	not implemented to fully implemented
	The extent to which challenges stimulate the combination of deep understanding and a broader view	not implemented to fully implemented
	The extent to which learning activities enable critical thinking	not implemented to fully implemented
	The extent to which learning activities enable creative thinking	not implemented to fully implemented
	The extent to which learning activities allow problem formulating and designing	not implemented to fully implemented
Self-directed learning	The extent to which materials and learning activities support contextualised acquisition and application of knowledge and skills	not implemented to fully implemented
	The extent to which learning activities support the development of meta-cognitive skills and self-regulatory abilities	not implemented to fully implemented
	The extent to which learning activities enable ownership and self-directed learning	not implemented to fully implemented
	The extent to which learning activities enable dealing with uncertainty	not implemented to fully implemented
Assessment	The extent to which the assessment is balanced between product (assessment of learning) and process (assessment for learning).	imbalanced to fully balanced
	The extent to which the assessment is balanced between individual and team learning	imbalanced to fully balanced
	The extent to which the assessment is balanced between (in)formative and summative assessment	imbalanced to fully balanced
Teaching	The extent to which coaching supports scaffolding of students’ learning	not implemented to fully implemented
	The extent to which teachers find a balance between openness and scaffolding	not implemented to fully implemented
	The extent to which teachers can act as coaches, and co-learners, and co-creators	not implemented to fully implemented
Interdisciplinarity	The extent to which challenges require interdisciplinary teamwork	not implemented to fully implemented
	The extent to which challenges support combinations of individual and teamwork	not implemented to fully implemented
	The extent to which learning activities support the development of interdisciplinary professional skills	not implemented to fully implemented
Collaborative learning	The extent to which challenges stimulate cycles of divergent and convergent reasoning	not implemented to fully implemented
	The extent to which learning activities enable peer learning	not implemented to fully implemented
Learning Technology	The extent to which learning activities are facilitated by innovative use of educational technologies	not implemented to fully implemented
	The extent to which learning analytics are applied	not implemented to fully implemented
Support
Facilities	The extent to which facilities offer required materials	no support to full support
	The extent to which facilities offer required spaces	no support to full support
	The extent to which facilities offer required tools, including ICT	no support to full support
Teacher support	The extent to which support structures offer course design and pedagogical support for teachers	no support to full support
	The extent to which support structures guide teachers in developing coaching skills and other teaching skills required in a CBL context	no support to full support

3.1. Vision

Vision serves as a foundation for the implementation of CBL by describing the basic motivations and goals governing an educational program. The initial definition of CBL (Malmqvist, Kohn Rådberg, and Lundqvist 2015) and preliminary conceptual model (Gallagher and Savage 2020) emphasise these basic motivations in terms of types of challenges and types of themes. Kohn-Rådberg et al. (2020) while comparing CBL with traditional engineering and problem-based learning (PBL), bring focus to the involvement of stakeholders, including external partners.

3.1.1. Real-life open-ended challenges

CBL focusses on relevant real-life, authentic, open-ended challenges to trigger learning. These challenges can be mono- and interdisciplinary, originating from various sources (Malmqvist, Kohn Rådberg, and Lundqvist 2015). Authentic here refers to resembling or being derived from the activities of real-world professionals (see also Baloian et al. 2006) to allow also for challenges that could emerge in the future. Open-ended assignments are common in fields such as engineering education because engineering design is open-ended with respect to both the solution and the process (Lammi, Denson, and Asunda 2018), however, examples are also found in medicine (Brauner et al. 2007), literature (Coby 2016) and language studies (Egbert, Herman, and Lee 2015).

Open-ended challenges allow students to discover both a problem and a solution, allowing varying solution paths (Brophy et al. 2008). These varying solution paths refer to complexity as an indicator of challenges (see also Cennamo et al. 2011). Complexity arises when something is impossible to analyse with simple frameworks (Munda 2000), which in turn can be understood as a call for bringing together multiple fields of expertise and epistemologies (Redshaw and Frampton 2014). If experts from these fields succeed in some level of integration among those fields, it counts as interdisciplinary (Huutoniemi et al. 2010; Klein 2010). Interdisciplinarity thus requires methodological or conceptual synthesis with the aim of deepening knowledge and skills (English 2016; Van den Beemt et al. 2020). Variety in CBL would allow a minimum characterisation of theoretical, pre-structured, one-dimensional, and mono-disciplinary challenges.

3.1.2. Global themes

Thematic content areas addressed in CBL are predominantly rooted in themes of global importance, such as sustainability (Gallagher and Savage 2020). In that respect, CBL is value-driven, with a focus on transformative value and integrative value (Larsson and Holmberg 2018; see also Kohn Rådberg et al. 2020). Transformative value is perceived as outcomes that challenge business-as-usual practices understood as unsustainable. Integrative value can be described as awareness raised and trust built when a diverse group of actors, disciplines, and perspectives are brought together in dialogue to explore a common issue. Both types of value can have either a short-term or long-term societal impact, of which students need to be aware (Larsson and Holmberg 2018). That is not to say that long-term impact should be preferred. Indeed, challenges that combine short-term societal impact with high urgency are not necessarily children of a lesser god. Global themes respond to the need for students to have skills and knowledge contributing to a global mindset (Sternad 2015). This dimension, ranging from no focus to full focus on global themes, allows to formulate questions about how challenge themes that impact sociotechnical problems relevant to students can be a motivating factor.

3.1.3. Involvement of stakeholders

The involvement of stakeholders is considered an important distinction between CBL and traditional learning and partly also PBL (Kohn Rådberg et al. 2020). CBL engages students by involving stakeholders from academia, industry, or the societal context (Kohn Rådberg et al. 2020). A distinction can be made between (1) university developed challenges, reflecting little collaboration with external stakeholders and (2) challenges brought and actively supported by stakeholders (Membrillo-Hernández et al. 2019). This distinction supports variety in the scope and complexity of challenges.

Measuring the dimensions under vision, contributes to answering questions such as the relation between indicator scores and richer learning experiences, or how the integration of disciplines can be shaped in challenges and learning goals, or efficacy of involving stakeholders in the assessment of student work.

3.2. Teaching and learning

Teaching puts vision into action, with learning as a mutually enforcing parallel process. Teaching and learning processes depend on conditions and resources being in place that facilitate their development and operation (see also ‘Support’ below). Dimensions and indicators under teaching and learning are ordered according to steps in course design: content, learning objectives, assessment, teaching and learning activities (Fink 2003). From an educational research perspective teaching and learning can be considered key in conceptualising CBL. However, concluding from the literature reviews by Gallagher and Savage (2020) and Leijon et al. (2021), existing research pays little attention to teachers and what they actually do in CBL or to student learning. This is especially in engineering education research an often-overlooked aspect (see also Van den Beemt et al. 2020). At the same time, teachers appear in need of competencies for coaching and scaffolding of students (Van den Beemt and MacLeod 2021; Pepin and Kock 2021).

3.2.1. T-shaped professionals

When dealing with real-life open-ended challenges, disciplinary boundaries become unclear, and asks for individuals with a depth and breadth of expertise (Conley et al. 2017). The T-shaped professional model (Gardner 2017) combines in-depth disciplinary expertise with the ability to work with a broad range of people and situations (Gero 2014). T-shape metacognitive abilities have long been emphasised in engineering education but can be found in many areas (Demirkan and Spohrer 2018). CBL challenges educators to present learning activities that contribute to an in-depth disciplinary expertise, by creating a rigorous treatment of fundamentals (Kohn Rådberg et al. 2020), which represents ‘content’ in the course design process. Furthermore, innovation and creativity are considered important aspects in many CBL cases (Gallagher and Savage 2020). This can be operationalised in creative thinking (Sternberg 2003) and critical thinking (Bailin 2002). Creative thinking can be considered as thinking that is novel and produces valuable ideas (Sternberg 2003). Critical thinking, argued to be most important for sustainability (Rieckmann 2012), contextualises these ideas, by examining what constellation of resources is required in particular contexts in response to particular challenges and what the range of application is for those resources (Bailin 2002). Finally, CBL is characterised by a combination of problem formulating and designing, which implies working in an iterative cyclical way, involving both analysis and synthesis (Malmqvist, Kohn Rådberg, and Lundqvist 2015). The set of indicators under T-shaped professionals, including a rigorous treatment of knowledge, the combination of deep understanding and broader view, critical thinking, creative thinking, and problem formulating and design, serve as input for learning objectives in the course design. Furthermore, these indicators prompt research such as effective combinations of these indicators given types of challenges, and types of students.

3.2.2. Self-directed learning

The definition of challenges as self-directed work scenarios (Johnson et al. 2009) in which students collaboratively engage, urges self-directed learning (SDL) as a dimension in the conceptualisation. Existing literature knows multiple definitions of SDL, sharing how students assume responsibility to control their learning objectives and means, with the aim to meet personal goals or perceived demands of their context (Morris 2019). Some definitions perceive this as a highly individually directed process, opting for individual or collaborative learning if students believe it to be conducive to their learning efforts (e.g. Brookfield 2009). In the context of CBL, we would rather emphasise students assessing their learning needs, securing resources, planning and conducting learning activities, and assessing the activity results (Brockett and Hiemstra 1991 as cited in Khiat 2017).

Morris (2019), while building on Sawatsky et al. (2017) proposed four characteristics of SDL, which can be translated to the CBL context. First, there is the cognitive aspect, namely how knowledge is construed. CBL encourages students to both acquire and apply knowledge and skills that are needed to work on a specific challenge, which makes their learning contextualised (e.g. Edson 2017). The materials and learning activities will be different for each student, thus enhancing student participation in conceiving and defining their own pathway in learning, also known as ‘learning trajectories’ (Pepin and Kock 2019). Defining your own pathway involves the management of learning tasks – the second characteristic – also known as meta-cognitive skills that in turn foster deep learning (cf. Novak 2002). These meta-cognitive skills are closely related to self-regulatory abilities and hence to personality characteristics of students (e.g. Morosanova 2013), which is the third characteristic.

CBL is also active learning (Arrambide-Leal et al. 2019) that allows students to construct a network of knowledge and take ownership (agency) of their own learning process, including the freedom to choose within a broader challenge the specific problem they want to focus on (Hernández-de-Menéndez et al. 2019). Active learning is perceived as an approach that creates student engagement with learning materials through interactions such as reading, watching, listening, writing, analysing, experimenting, and thinking (Kalinga and Tenhunen 2018; Nascimento et al. 2019). To organise these interactions, students have to show agency and ownership, which can be defined as the personal responsibility in identifying learning gaps and setting learning goals. However, the fourth characteristic emphasises how the possibility and likeliness for learners to show ownership and undertake SDL is influenced by the context. This can be considered a call for challenges and learning activities to facilitate ownership and SDL. Further studies on SDL in CBL could emphasise understanding the student's context.

In our view, one characteristic is missing from existing conceptualisations of SDL, namely dealing with uncertainty. Agency, ownership and SDL also include an entrepreneurial mindset. This mindset finds ways to deal with uncertainty (Maya et al. 2017) and open-endedness because the outcomes of challenges are unclear, and because defining and addressing the problem, and learning what it takes to work towards a solution, in sum defining your pathway in learning, might fail. Scholars emphasise how in this process SDL urges students to look for expertise: a resource person (Schugurensky 2000) or peers (Brookfield 2009). Feedback given in this process is considered an essential element of supporting the facilitation of SDL (Morris 2019). SDL thus can be considered a learning objective, which is assessed for example through feedback, self-reports and reflection on learning activities.

Criticisms of SDL address the quality of learning results. Brookfield (2009) argues that learners may claim to have developed skills or knowledge but be unable accurately to estimate their true level of accomplishment. The gap in comprehensive studies that examine the effectiveness of SDL represents another important research topic. SDL as a dimension shares ‘context’ as an important aspect with T-shaped professionals. Self-directed learning thus supports research questions about the efficacy of learning processes in CBL, and about processes involved in contextualised acquisition and application of knowledge and skills, including the role of peer feedback and co-regulation.

3.2.3. Assessment

Case studies on educational innovations in domains including STEM show relatively infrequent attention to assessment (Van den Beemt et al. 2020; Richter and Paretti 2009). However, generating constructive alignment between learning goals and assessment procedures raises significant challenges, especially when students from different disciplines collaborate (Borrego and Cutler 2010; Valencia et al. 2020). Gallagher and Savage (2020) show how CBL research that follows a framework approach generally uses both summative and formative assessments, and assessment of individual and team involvement. We perceive this as that CBL assessment can be characterised by a balance between traditionally separated forms of assessment, which fits trends towards a holistic view on assessment that combines assessment strategies (see also Van der Vleuten, Heeneman, and Schuwirth 2017).

Because CBL evenly values the process of working towards a solution, it should stimulate forms of assessment balanced between product-focused assessment and process-focused assessment. In product-focused assessment the deliverable represents what is learnt in terms of content knowledge and understanding, and the mastery of real-world skills (Nichols, Cator, and Torres 2016). Process focused assessment evaluates whether the knowledge and skills have been obtained, also known as assessment for learning, which includes feedback loops and meta-cognition (William 2011). The balance between these two stands for the extent to which intended learning behaviour becomes visible in both product and process (Magnell and Högfeldt 2015), known as ‘assessment as learning’ (Van der Vleuten, Sluijsmans, and Joosten-ten Brinke 2017). Focussing on the balance between forms of assessment allows for research on efficacy of CBL aspects such as team progress, interdisciplinarity, and advanced knowledge and skills, which can be evaluated during regular checkpoints with teams and individuals (Nichols, Cator, and Torres 2016). Because little is known about this balance and the different aspects of CBL assessment, it should be part of future research.

3.2.4. Teaching

CBL involves adaptive teacher and expert guidance of the construction of knowledge by students. Given the open-ended and ill-defined character of challenges, educators act most often as a coach rather than an instructor. Research shows a possible underestimation by curriculum designers of the level of support students need in interdisciplinary contexts, including CBL (Soares et al. 2013). Students need scaffolding towards content (also known as clear signposting), towards active learning (Johnson et al. 2009; Piironen et al. 2009; Binder et al. 2017), and towards expertise (Brookfield 2009; Morris 2019). Yet, given the level of open-endedness and complexity of challenges, teachers are suggested to find a balance between openness and scaffolding. It appears that this balance is easier to be found when teachers act as coaches and co-learners and co-creators (cf. Balasubramanian and Wilson 2007; Botha and Herselman 2016). Brookfield (2009), in the context of SDL, proposes several roles for teachers, including advising students on skills and knowledge that might be of their greatest benefit, on the possible range of learning resources, on the design of a learning plan, on grouping DSL activities, and on teamwork. The author also adds direct instruction and evaluation. The indicators under teaching, including the set of teacher roles appears currently underrepresented in CBL research (see also Gallagher and Savage 2020; Leijon et al. 2021). It allows research on effective pedagogies and required professional development of teachers.

3.2.5. Interdisciplinarity

Interdisciplinarity, as a teaching and learning activity in course design, relates to teaching and assessment in acknowledging the balance between individual and team as a key aspect (Kohn Rådberg et al. 2020). Interdisciplinary CBL facilitates students from different (sub-)disciplines to learn to work in a team. Their interdisciplinary interactions can be seen as attempts to integrate heterogeneous knowledge bases and knowledge-making practices (Krohn 2010). Interdisciplinarity thus requires some level of integration between fields of expertise (Huutoniemi et al. 2010; Klein 2010). Individuals in interdisciplinary teams learn from others’ perspectives and produce work in an integrative process that would not have been possible in a mono-disciplinary setting (McNair et al. 2011). The result, at least in theory, is that participants emerge from such interactions speaking ‘one language’ (Van den Beemt et al. 2020). Bringing together disciplines and epistemological frameworks has been proven to strengthen CBL and contributes to the conceptual basis in flexibility, including combinations of CBL with design-based learning, or research-based learning (Gallagher and Savage 2020).

3.2.6. Collaborative learning

The preliminary conceptual framework of Gallagher and Savage (2020) includes the dimension ‘collaboration’, which as a learning activity involves students collaborating with other students, and with external stakeholders or experts. For our conceptualisation, we made a distinction between external stakeholders (see vision – stakeholders above), and student teams. Given its character, CBL implies working in an iterative cyclical way in teams (Jensen, Utriainen, and Steinert 2018; Baloian et al. 2006). These cycles consist of divergent and convergent reasoning bringing students closer to possible solutions to the challenge. Divergent reasoning includes a variety of perspectives and solutions, while convergent reasoning brings focus and priority to this variety. Ideally, these cycles are discussed and evaluated in groups, which in turn enables room for peer feedback and support. This dimension supports research on quality and aspects of group learning, such as co-regulation and shared-regulation, and formulating shared learning goals and a team learning agenda (Vrieling et al. 2016; Huijben et al. 2021).

3.2.7. Learning technology

Because the nature of CBL presumes extensive access to technology (Johnson and Adams 2011), technology-rich learning environments lend themselves to support learning aspects of CBL such as active learning, deep learning, social learning, and learning analytics (Johnson et al. 2009; Gallagher and Savage 2020). Bocconi, Kampylis, and Punie (2012) consider learning technology the core of creative classrooms and creative thinking. Especially for engineering education, learning technology plays a key role in learning processes, for example with simulators and virtual labs, and is also often a product of this learning (Martin et al. 2019). From the literature, it can be concluded that CBL projects are technology ‘infused’. Technology serves communication, dissemination, access to information, collaboration, and support (Gallagher & Savage). Increasingly, support is based on learning analytics and evaluative dashboards (Ifenthaler and Gibson 2019).

3.3. Support

CBL entails active learning, and requires explicitly support in terms of facilities, more than traditional education. Central elements to active learning are physical spaces, technology, including online access and lab equipment, interactions and dialogue, together leading to, amongst others, an enhanced conceptual understanding and improved student performance (Hernández-de-Menéndez et al. 2019). Support for these facilities, and resources for developing educator competences are an often-overlooked aspect of educational innovations, including CBL. This type of support is essential for reaching desired quality standards in teaching and learning, and appear a challenge, especially in innovations in engineering education (Van den Beemt et al. 2020). Conceptualising support thus helps in answering research questions about educational quality.

3.3.1. Facilities

CBL involves the facilitation of learning and teaching in terms of resources that students perceive as required, spaces such as classrooms or laboratories, and tools including ICT (Gardner et al. 2014; Rashid 2015; Lantada, Bayo, and Sevillano 2014). Especially the combination and alignment of physical and online facilities are reported as important by stakeholders (Mielikäinen 2021).

3.3.2. Teacher support

CBL involves support for teachers and tutors, not only on the design of challenges and related learning activities but also in dealing with uncertainty (Membrillo-Hernández and García-García 2020). Especially the shift from content expert to being both expert and coach could lead to resistance among teachers, which needs to be addressed with schooling and ongoing support.

3.4. Must have indicators

Because the approach of measuring the level of implementation implies a variety of CBL within a curriculum, a minimum requirement is needed for study components to be called ‘CBL’. This minimum requirement includes the smallest number of ‘must have’ indicators and the smallest score on certain indicators, before we can speak of CBL as an educational concept. However, defining CBL solely by this minimum requirement renders a bleak version of this otherwise rich educational concept.

Engaging students in ‘real-life challenges’ to trigger learning is considered a core characteristic of CBL (e.g. Arrambide-Leal et al. 2019). Some studies add that challenges are ‘authentic’, meaning that they are derived from activities of professionals (Baloian et al. 2006) and closely related to students’ interests and development (Van den Beemt and MacLeod 2021):

• The extent to which challenges are real-life and authentic (dimension: Vision – Real-life open-ended challenges)

Furthermore, from the CBL literature, two more indicators emerged as ‘must haves’ for CBL implementations and research (Kohn Rådberg et al. 2020; Malmqvist, Kohn Rådberg, and Lundqvist 2015; Membrillo-Hernández et al. 2019). These two indicators also emerged as essential for the local colour of CBL at our university (see the illustrations below):

• The extent to which learning activities create a rigorous treatment of fundamental engineering knowledge and skills (dimension: Teaching and Learning – T-shaped professionals)

• The extent to which challenges stimulate the combination of deep understanding and broader view (dimension: Teaching and Learning – T-shaped professionals)

The large or full implementation of these three indicators in study components distinguishes CBL from regular education.

4. Putting CBL on the research agenda

Our framework allows asking what happens with the motivations for CBL, effective teaching and learning activities, and required support structures for specific study components or curricula. The aim of this exercise would be to translate the concept CBL to practice, thus helping curriculum designers or educators in developing their courses and teaching, and in formulating support requirements. The framework also shows how CBL builds on for instance approaches such as Problem-based learning (PBL) or Project-based learning (PjBL) (Kohn Rådberg et al. 2020). At the core of CBL is a strong need to action, which leads to exploring through the lens of multiple disciplines a range of topics from diverse fields, allowing to discover links between content areas that might not be evident (Kalinga and Tenhunen 2018). Where for example PBL focuses on designing a product solution, as a team effort to address customer needs, CBL widens the scope to the social context (see also Membrillo-Hernández et al. 2019 for a detailed comparison). This context encourages both problem formulating and designing, both team and individual efforts to propose value-driven deliverables (Kohn Rådberg et al. 2020). In that sense, CBL can be considered an educational evolution, rather than a revolution. However, this evolutionary character might hinder conceptualisations of CBL, because of the risk of educators and researchers drawing on their perception of PBL when working on CBL.

The framework gives a justification for research questions such as: How do students learn in CBL contexts? What is the efficacy of CBL teaching? What type of challenges are suitable for developing specific domain knowledge? Moreover, the framework allows from different educational contexts how knowledge and skills are developed differently in activities with a variety in vision and teaching and learning. It raises questions about scaffolding students and required teacher competences. And it invites research on learning designs aimed at this student support.

We illustrate the feasibility of such research questions with an extensive educational innovation initiative focused on large-scale development, implementation, and evaluation of CBL at a Dutch university of technology. Research on this initiative contributes to an understanding of ‘what works’ in a specific educational context. Our conceptual framework serves as a basis for a research agenda that both monitors and guides all experiments in the initiative. The illustration below is based on a translation of the framework into an instrument, labelled ‘CBL-compass’. The CBL-compass is an online instrument that includes all indicators of our framework, which draw on four-point Likert-scale items (Not implemented – 1; To some extent – 2; To a large extent – 3; Fully implemented – 4) indicating evidence of the characteristics. The resulting scores are visualised in a radar graph. The instrument is filled in for separate courses by the responsible teacher together with an educational researcher or teacher supporter in a dialogue session. During the dialogue dimensions and indicators from the CBL-compass serve as prompts for reflection on course design and implementation. The outcomes, presented in a radar chart, are meant as a visualisation of the current situation, rather than a value judgement on the level of CBL implementation in the course.

Of the more than 40 CBL experiments running at our university, we highlight three that represent different levels of CBL implementation. The first course focused on technology forecasting and was offered to students in innovation sciences (see also Figure 2). The varying scores on indicators under ‘real-life and open-ended challenges’ reflect how the assignments are rather theoretical and structured. Variety in scores on dimensions under ‘vision’ trigger questions about considerations for CBL and about the purposiveness of implementing specific aspects. The indicators under ‘teaching and learning’ appear rather well implemented in this example, apart from self-directed learning. This caused the responsible teachers to reflect that their aims were high, however, in their perception students were often not able to reach the intended levels. Although the course was focused on interdisciplinary work, the indicators for collaboration scored rather low, especially regarding peer review. The framework thus allows for questions about the relation between different aspects of collaboration, including communication with team members and external stakeholders, and characteristics of interdisciplinarity.

Figure 2. Radar chart for course #1.

The second course was part of a learning line that integrates different disciplines and epistemologies (see Figure 3). In this challenge-based learning line students develop hands-on experience on how to combine physics-inspired quantitative approaches and psychology to quantify, model and nudge social systems. The responsible teacher reported to take pride in the full implementation of most indicators in our framework. Still, although this and other teachers reported to go to large extents in scaffolding students, they in general did not consider themselves as co-learners or co-creators of solutions. It is a prompt for research on educators’ competences and considerations for limiting their role as coach.

Figure 3. Radar chart for course #2.

The third example was a course in mechanical engineering that showed overall low scores on CBL implementation (Figure 4). These scores were considered a trigger to discuss ways to increase (not improve) the implementation of CBL characteristics. This course exemplifies how ‘Rigorous treatment of discipline knowledge’ received in general high scores. Teachers most often reported it as a ‘must have’ for CBL implementation. Scores on the dimension ‘Assessment’ were influenced by the perceived level of balance on all three indicators. Teachers in this and other courses explained how they perceive their score as an encouragement to bring more balance to assessing process and product, individual and teamwork, and formative and summative assessment. The indicators under support provoked strong responses by teachers. They responded either highly positive about each of these dimensions, or highly negative. Teachers explained their response being related to perceived support on a university level, either in terms of materials or in terms of pedagogical support. In general, educators expressed a developmental perspective, with low scoring indicators in the framework as starting points for future work.

Figure 4. Radar chart for course #3.

Researchers could use the CBL-compass to systematically evaluate the variety of CBL implementation across study components. The question behind each combination of values for CBL characteristics would be ‘what do students gain from this specific CBL approach?’ Furthermore, a related question is ‘how do specific combinations of characteristics affect learning patterns?’ (see also Vermunt and Donche 2017), or in other words: ‘which learning mechanisms need to be activated with CBL?’. Further research could detail distinctive CBL characteristics of courses, which scored highly on some of the indicators, identifying patterns in these indicators.

5. Conclusion

The aim of this article was to articulate a detailed framework for analysing the variety of CBL characteristics within and between study components in an academic curriculum. The framework contributes to a more detailed conceptualisation of CBL and clarity to practitioners and researchers on what CBL implementations consist of. We illustrated the framework's use with CBL experiments in engineering education. To this end, the framework was translated into an online instrument, labelled CBL-compass. Our illustration showed that not all characteristics are fully present in every project or course. We approached CBL as embedded curriculum practice, which reinforces variety in CBL implementations over study components, rather than a full-fledged version including full marks on all indicators. This variety resonates with the necessity to adapt to student development over time. In the process, a minimum set of CBL characteristics were distinguished for study components to be called CBL.

Although we based the framework on literature reviews and seminal works in the field, some aspects might need further consideration in future work. For example, regarding assessment, it could be discussed how self- and peer-assessment, or the role of feedback are addressed in the current indicators or subsequent operationalisation. Furthermore, the current framework invites to explore questions about aspects of self-regulation, including co-regulation and shared-regulation. Future research could also include conceptualisation and operationalisation of professional and personal identity related to CBL. What types of students flourish under CBL, and how does working on challenges contribute to students’ identity development? Finally, because the framework serves analysing a variety of CBL, future research should focus on inter-institutional comparisons of study components or curricula.

The indicators presented in our framework under the label of support offer starting points for research on the costs and scalability of large-scale implementation of CBL as an educational concept. Implementing CBL, with high scores on most indicators only to individual courses, bears the risk of high costs associated with small scale education in teams. This in turn could cause CBL to be available only to a few students, while the aim would be the availability for many or all students. This, of course under the assumption that the benefits of CBL are valuable for all students.

Our conceptualisation does not only apply to engineering education or STEM programs, because CBL at its core is a multidisciplinary pedagogy. However, implementations of CBL in non-STEM higher level courses appear to be a significant gap in the research, which urges an exploration of multiple disciplines in the design, analysis and evaluation of CBL (Gallagher and Savage 2020). This in turn would invite a holistic and critical understanding of knowledge production and learning processes in higher education (Leijon et al. 2021). Because of its granulation, our framework might prove useful for evaluating any form of CBL in other domains as well. It shows how CBL can be moulded to fit different disciplines, curricula, or assessment types. However, although when combined with other theories, CBL opens for analytic depth and critical reflection (Leijon et al. 2021), the flexible methodological approach could lead to definitional muddying rather than a conceptual basis in flexibility.

Using the CBL-compass presented in this paper in conjunction with for instance design principles would broaden the evaluation of CBL implementation and thus strengthen CBL as an educational concept (Doulougeri et al. 2022). The dimensions and indicators of the CBL-compass are fundamental characteristics of CBL. Using the indicators as measurements for implementations serve the visualisation of an institution's local colour of CBL.

Keypoints

This article provides a detailed conceptualisation of CBL in higher education, builds on literature reviews on CBL, provides a framework, labelled CBL-compass that serves as an instrument to support analysis and implementation of CBL in study components, and illustrates the framework with three examples from a large-scale university innovation program.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Antoine van den Beemt

Antoine van den Beemt is an associate professor with the Eindhoven School of Education, working as teacher educator and researcher in the domain of STEM-teacher professional development. His current research focuses on Challenge-Based Learning, and innovative pedagogical approaches to blended and online learning.

Gerard van de Watering

Gerard van de Watering, PhD is an educationalist and working as a strategic education policy advisor at Eindhoven University of Technology. His portfolio consists of assessment, quality assurance, BI, and educational research.

Michael Bots

Michael Bots is Program Manager of Challenge-based Learning and policy officer on education in the staff office of the Executive Board of Eindhoven University of Technology (TU/e) in the Netherlands. He has a background in educational science and change management and is specialized in managing large-scale learning innovations in higher education. His current focus is on managing the development of challenge-based learning at TU/e and the redesign of bachelor and master programs.