A scoping review of experimental evidence on face-to-face components of blended learning in higher education

ABSTRACT

The practice of combining digital and face-to-face elements into blended learning courses is becoming the new normal in higher education and offers a promising learning format. While studies on the effects of blended learning have so far focused mostly on the online components of the blends, the success of blended learning also rests on the quality of the integrated face-to-face activities. This scoping review examines evidence from 59 experimental studies conducted in higher education settings to explore what makes face-to-face components of blended learning efficacious. The focus is on pedagogical intentions rather than on quantifying the balance between online and face-to-face activities. The results indicate which face-to-face activities support the pedagogical objectives of higher-order processing, social interaction, and engagement. The review identifies current gaps in blended learning research and calls for richer characterizations of face-to-face activities in blended learning to support the development of finely tuned interventions and guide practice.

KEYWORDS:

• Blended learning
• face-to-face activities
• improving classroom teaching
• higher education
• teaching/learning strategies

Introduction

Technological advances are increasing the potential of online modes of instruction (Arbaugh et al. 2009, Arbaugh et al. 2010; Drysdale et al. 2013; Means et al. 2010); a development most recently witnessed in higher education as millions of students and teachers have continued their classes online during the COVID-19 pandemic. Conversely, however, this delivery of education online has also brought home the significant value of face-to-face meetings in the learning experience. Technology is certain to keep playing a major role in the delivery of learning, and the ongoing transition to blended learning is the ‘new normal’ for many higher education institutions and is bound to accelerate in the coming years (Dziuban et al. 2018; Porter et al. 2014; Bruggeman et al. 2021).

While there is no universal definition of blended learning, it is increasingly defined as the integration of online and traditional face-to-face class activities in a planned, pedagogically valuable manner with a portion of face-to-face time replaced by online activity (Dziuban et al. 2016, 8). Comparisons of learning outcomes between face-to-face, fully online and blended learning are not unequivocal but in most cases blended learning has shown better results than fully online or fully face-to-face learning (Bernard et al. 2014; Boelens, De Wever, and Voet 2017; Means et al. 2013; Spanjers et al. 2015; Strelan, Osborn, and Palmer 2020), the argument being that blended learning combines ‘the best of both worlds’ (Arbaugh 2014, 788).

While the online dimension of blended learning is an essential means of exploring new opportunities and challenges (Rasheed, Kamsin, and Abdullah 2020; Beckman et al. 2021), a unilateral focus on online components leaves a crucial and often-cited benefit of blended learning unexamined: a shift to lecture-based activities online frees up class time to engage students in high-quality face-to-face interaction in-class. When studies on blended learning address face-to-face activities, these activities are often mentioned as opportunities for deep learning, discussion, and engagement (Akçayır and Akçayır 2018; Fleck 2012; Halverson et al. 2014; O’Flaherty and Phillips 2015; Vaughan 2007), but less frequently scrutinized for their instructional effect or contribution to learning outcomes (Kjærgaard 2017). In other words, researchers need to explore how general pedagogical competencies in higher education, such as content delivery, communication, and assessment (Yürekli Kaynardağ 2019), require recasting to match the opportunities and limitations of new blended formats.

As indicated above, research on blended learning is already a large and rapidly expanding research field. It is within this existing body of knowledge that we point our attention specifically to the face-to-face side of blended learning in higher education. The purpose is to contribute new insights about the face-to-face dimension of blended learning, thereby adding to knowledge from previous reviews that have focused on the online dimension of blended learning (see for example Er et al. 2015; Mohamed and Lamia 2018; Rasheed, Kamsin, and Abdullah 2020; Al-Samarraie and Saeed 2018; Alamri, Watson, and Watson 2021). We do this through a scoping review, outlining the face-to-face activities that have been studied and evaluating the available evidence.

Recognizing that the dominant focus in the literature is on the technology-side of blended learning, the scoping review allows us to map and assess the relevant literature, with the goal to stimulate and guide future inquiry and research on the face-to-face dimension of blended learning (Arksey and O’Malley 2005). The next section situates our scoping review in the context of relevant trends in the blended learning literature and presents the research questions that are the focus of the review.

Trends in blended learning research

While most educators and students are enjoying or looking forward to returning to campus after a long period of fully online teaching, much effort has been put into designing online activities that have shown their value for learning and will be a continued part of educational offerings in the future, often in combination with face-to-face activities. Based on the latest experiences with online learning, it does seem that technology-enabled modes of instruction can support and provide new opportunities for learning. However, the role of technology is not to simply take over the face-to-face social interaction, intense debates, discussions and dialogs previously taking place on campus (Asarta and Schmidt 2015; Phillips and Trainor 2014). Instead, technology is intended to complement rather than substitute the campus experience in blended formats (Garrison and Kanuka 2004; Gomis-Porqueras, Meinecke, and Rodrigues-Neto 2011).

This increased use of technology also supports pre-covid challenges for universities, in particular recent years’ increase in the number of students in higher education and the accompanying challenge for creating dialogue and interaction in the classroom. In large-size classes, opportunities for dialogue and collaboration are limited and students often become passive receivers of knowledge rather than active contributors to the learning process. From a learning perspective, this is not optimal and has become a significant challenge to the quality of learning (Northey et al. 2015; Snowball 2014).

While the potential for enhancement of learning in blended formats is broad, three dominant categories are presented in the literature: efficiency, transformation and enhancement of education (Graham 2009; Kirkwood and Price 2014). Efficiency in terms of providing more cost and time effective, sustainable and scalable delivery, such as opportunities of reducing seat-time for students in-class to create a more flexible environment or to reduce costs (Parsad, Lewis, and Tice 2008). Transformation in terms of introducing new activities or processes or new forms of delivery. This includes the potential of blended learning to involve industry partners more actively in student learning by engaging with teachers and students in a new triad instead of the traditional teacher-student dyad (Duffy and Ney 2015) or how new blended learning delivery formats provide more flexible learning environments for a diverse group of learners (Holley and Haynes 2003). The necessary shift to blended learning during the pandemic is an example of transformation as the need for reducing the number of students present at campus triggered transformation of fully face-to-face teaching to new blended formats, as well as fully online.

Finally, enhancement of education and student learning outcome is the most dominant theme in the literature. Blended learning provides opportunities for introducing new and innovative activities and often involves spending extra time and effort on rethinking the instructional design (Kaleta, Skibba, and Joosten 2007). In the flipped classroom model the focus is specifically on changing the instructional design to support pedagogical aims of more active and student-centered learning by shifting presentation of content online whereby in-class time can be spent on interactive learning activities (Akçayır and Akçayır 2018; O’Flaherty and Phillips 2015; Bishop and Verleger 2013). Moreover, blended learning can provide students with more control over the time, place, and pace of their learning as they can watch videos and access materials when and where they choose, and they can revisit content if needed. Finally, blended learning provides support for organizing and scaffolding students’ learning activities outside of class, helping students to structure their learning and avoid overload (Chandra and Watters 2012).

Although several reviews have investigated the effects of blended learning on student learning (Lo, Hew, and Chen 2017; O’Flaherty and Phillips 2015; Spanjers et al. 2015; Strelan, Osborn, and Palmer 2020; van Alten et al. 2019; Bishop and Verleger 2013), and while several reviews have addressed challenges and provided suggestions for successful implementation (Boelens, De Wever, and Voet 2017; Drysdale et al. 2013; McGee and Reis 2012; Anthony et al. 2022), none have focused specifically on the role of face-to-face activities in blended learning. More often the online activities and opportunities have been in focus (see for example Mohamed and Lamia 2018; Rasheed, Kamsin, and Abdullah 2020). Seeking to contribute to the overall agenda of determining how to best reap the pedagogical potential of blended learning in higher education (McGee and Reis 2012), the aim of this scoping review is to better understand:

1. Which face-to-face activities are used in the studies published, and what are their main characteristics?

2. What empirical evidence do the reviewed studies report about the effects of face-to-face activities on student learning?

Materials and methods

Sampling method

First, we searched the following databases ERIC, PsycInfo, Scopus, Web of Science (core collection), Business Source Complete, and ACM in May 2019. To capture experimental and quasi-experimental studies of blended learning in higher education, we formulated a detailed search string, the specifics of which appear in Appendix A. We regard experimental studies with random assignment as the gold standard methodology but also include quasi-experimental studies with a treatment and control group. In all the databases used, we searched academic journals only. A total of 1,567 articles were ultimately retrieved, 519 duplicates of which were removed. To avoid overlooking studies with no explicit mention of the higher education setting, we did an additional search in Scopus in December 2019. In this search, we targeted eight educational journals particularly likely to feature experimental studies of blended learning in higher education: Academy of Management Learning and Education, Management Learning, Studies in Higher Education, Internet & Higher Education, Computers & Education, Journal of Applied Research in Higher Education, and Active Learning in Higher Education. The extra search produced only one additional study.

Paper selection

As is standard practice in scoping reviews (Arksey and O’Malley 2005), we honed the exclusion criteria on the basis of increasing familiarity with the literature, before sorting the full body of identified studies using the defined criteria. Thus, we examined 1,049 articles according to the following exclusion criteria: (a) articles that the search string was not intended to capture, that is, studies not concerning higher education, experiments, and blended learning and not published in an academic journal; (b) non-English language articles; (c) extra-curricular courses such as English as a foreign language, Microsoft Excel courses, etc.¹; and (d) articles not accessible online. As this review specifically focuses on the face-to-face component of blended learning, we added the following two exclusion criteria: (e) no face-to-face component in the treatment conditions (i.e. online vs. online, as well as blended vs. online) and (f) same face-to-face component in all experimental conditions, that is, an online component was added without changes to the face-to-face component. Based on the above criteria, our sample then consisted of 114 articles that were imported to the qualitative data analysis software NVivo, and we carefully read through the full text of each article. To ensure a minimum level of quality and comparability, two additional exclusion criteria were added: (g) fewer than 20 participants in each experimental condition, and (h) no outcome measure of learning. Using these additional exclusion criteria, we ended up with a final sample of 59 studies. To provide an overview, Figure 1 presents our selection process in a visual outline that matches the standard commonly used in the PRISMA guidelines (Moher et al. 2009).

Figure 1. Overview of the search protocol.

Categorization and coding of articles

The next step for our scoping review was the charting of data from the selected articles. Following Arkey and O’Malley (2005, 28), this stage involved recording of ‘key items of information obtained from the primary research reports’ on the basis of both interpretation and synthesis. During this full-text analysis, the sample of 59 studies were first sorted into three broad categories based on the aspirations reported in the articles about pedagogical objectives of the included face-to-face activities. These categories are higher-order processing, engagement, and social interaction. If no objective was explicitly stated, all three authors discussed the pedagogical objective discerned from the activities described in the given article and placed it in the dominant category.

To analyze the articles in detail and provide an overview of the articles in each category, they were coded by the three authors in NVivo according to: (1) face-to-face activities in both experimental conditions, (2) online activities in both conditions, and (3) the results reported. To establish a consistent coding scheme at the beginning of the coding process, we coded the articles while physically co-located. Each author presented their coding to the others and any uncertainties or inconsistencies were clarified. The coding scheme was adjusted to also include the number of observations in each category. After coding all articles, the authors exchanged results and discussed all articles where the coding had not been straightforward. If there were any doubts or uncertainties about the coding, the article was also coded by the other authors and any discrepancies or ambiguities were discussed until consensus was reached.

In addition, we assessed the quality of the studies based on the method applied and divided them into three groups: (1) experiments with random assignment; (2) non-equivalent groups showing balance on or controlling for prior knowledge; and (3) non-equivalent groups with no controls or balance test.

Results

Overview of the included studies

The 59 studies included in this review contain 64 experimental contrasts, as some have more than two experimental conditions. The studies cover 11 academic disciplines across five continents. Most of these studies have been conducted in the natural sciences. Thirty-one percent of the studies were conducted in the discipline of medicine while 16% in teacher education. Most of the evidence has been produced in the United States (39%), Asia (20%), and Europe (19%), while the sample contains only one study from South America and none from Africa. Moreover, most of the studies have been published in the last six years, with only one study published before 2014. This shows a rapidly growing interest in quantitative evidence on blended learning.

Blended learning is a broadly defined concept, as evinced in the wide variety of treatment condition designs. That said, most of the studies examine the effects of shifting from a lecture-based to a flipped classroom (78%), a shift that adds both an online component (videos) and changes the face-to-face component (from knowledge transmission to active learning strategies).

Regarding research methodology, 70% of the studies follow a nonequivalent group approach utilizing either two adjacent cohorts with different teaching formats or two different groups within the same year, without random assignment. Thirty percent of the evidence was obtained from randomized controlled trials. The number of study participants varies from 45 to 1,100 (with a median of 116). Moreover, most studies report a course-long intervention, with courses varying from 4 weeks to 8 months.

The fact that most studies rely on nonequivalent groups makes it important to adjust for pre-existing differences between groups. Sixty percent of the studies utilizing this approach recognize this and include some control variables, the most common of which is to condition the experiment on prior achievement shown from either pre-tests or data from previous exams. A few studies also include age and gender as control variables, and even fewer look at differential effects among subgroups.

Although we here focus on learning outcomes, most of the studies have more than one outcome measure (median of 2). Such other outcomes include motivation, satisfaction, engagement, and self-regulation, among others. Outcomes are generally measured shortly after the intervention, and only three studies include long-term outcomes (measured after the final exam). As such, this review concerns only short-term outcomes, as longer-term outcomes are generally absent from the literature.

R1: Which face-to-face activities are used in the studies published, and what are their main characteristics?

As mentioned earlier, the 59 studies fell into three broad categories concerning the pedagogical objectives which the articles reported as rationales relevant to the included face-to-face interventions: (1) higher-order processing, (2) engagement, and (3) social interaction. The face-to-face activities in each category are described next. Tables 1–3 below outline the main features of the individual studies included in the review and thus provide a detailed overview of both the activities used as well as the direction of the reported effects.

Table 1. Overview of studies in the category of higher-order processing.

Study	Overall change of modality	Activities in the treatment condition		Participants	Method	Outcome
		Face-to-face	Online
Bergfjord & Heggernes (2016)	Lectures = > BL	Theory recap, assignments, discussion	Lecture videos	100	Group 3	↑ Exam scores
Calimeris & Sauer (2015)	Lectures = >BL	Quizzes, Q&A, experiments,	Lecture videos	66	Group 1	↑ Exam scores
Cheng et al. (2017)	Lectures = > BL	Analyzing and discussing news articles, television shows, mind- and concept maps, worksheets	Lecture videos w. embedded quizzes	111	Group 2	↑ Exam scores
El-Miedany et al. (2019)	Lectures = > BL	Interactive activities, Q&A, role-play	Lecture videos	55	Group 3	↑Clinical skills
Hmelo-Silver et al. (2009)	Lectures = > BL	Group work, discussion, student presentations, PBL	Video cases, writing observations, discussion, problem sets	70	Group 2	↑Posttest
Ilic et al. (2015)	Didactive learning = >BL	Lectures	Lecture videos and mobile learning when interacting with patients.	147	Group 1	↔Clinical skills
Jensen et al. (2015)	BL = >BL w. concept application	Active concept application: Group work, apply concepts to novel situations, individualized feedback, discussion	Content attainment: Online homework assignments.	108	Group 2	↔Exam scores
Karaoğlan Yilmaz et al. (2017)	Lectures = >Structured BL	Class attendance required, assignment submissions with no extra time completion time, Q&A, lecturing	Lecture videos	78	Group 2	↑Post test
*Karaoğlan Yilmaz et al. (2017)	Flexible BL = >Structured BL	Class attendance required, assignment submissions with no extra time completion time, Q&A, lecturing	Lecture videos	81	Group 2	↑Post test
Kazanidis et al. (2019)	Lectures  = > BL	Interactive lab-based workshops, Q&A, hands-on excercises, guided readings, project-work	Lecture videos, discussion forum	128	Group 2	↑Post test
Kiviniemi (2014)	Lectures (60%) and active learning (40%) = >BL	80% group work and discussions, lecturing when necessary	Lecture videos	66	Group 2	↑ Exam scores
Kurt (2017)	Lectures = >BL	Brief review of lecture, practice based interactive tasks.	Lecture videos and quiz	62	Group 3	↑Exam scores
Lax et al. (2017)	Lectures = >Partial BL	Group work on worksheets, aid from instructors, questions answered using personal response system	Lecture videos	171	Group 3	↔Exam scores
Lewis et al. (2018)	Lectures = > BL	Case analysis, feedback from instructors, plenary discussion	Lecture videos	136	Group 2	↔Exam scores
*Lichvar et al. (2016)	BL = >BL w. virtual case	Virtual case. Q&A about case, explanations and elaboration from instructors, structured debriefing on case	Lecture videos	109	Group 3	↑Exam scores
Lombardini et al. (2018)	Lectures = >partital BL	Lecturing to student questions (one third of the time), clickers, exercises, discussions, optional group-work mostly outside class.	Lecture videos	303	Group 2	↑ Exam scores
Lombardini et al. (2018)	Lectures = >BL	Lecturing to student questions (one fifth of the time), clickers, exercises, discussions, mandatory group-work both inside and outside class.	Lecture videos	274	Group 2	↔Exam scores
Luna and Winters (2017)	Lectures = >BL	Active learning: discussions, assignments, activities	Online assignments, Quizzes	124	Group 2	↔Exam scores, ↑Post test
Oh et al. (2017)	Lectures = >BL	Discussions	Lecture videos	110	Group 3	↑Post test
Şengel (2016)	Teacher lead problem solving = > BL	Group discussions and working on problems in groups and individually.	Lecture videos	96	Group 2	↑Post test
Son et al. (2016)	Team-based labs->BL (virtual) labs	Group exercises in physical lab, lectures, discussion of online exercises, report writing, lectures	Individual excercises in virtual lab simulations	748	Group 3	↑Exam scores
Thai, Thi, and Wever (2017)	Lectures w. guiding questions = >BL lectures	Lectures	Guiding questions, delayed feedback	44	Group 2	↑Posttest
Thai et al. (2017)	Lectures w. guiding questions = > BL	Guiding questions, immediate feedback	Lecture videos	45	Group 2	↑Posttest
Thai et al. (2017)	BL lectures = >BL	Guiding questions, immediate feedback	Lecture videos	45	Group 2	↔ Posttest
Turan and Goktas (2016)	Lectures = >BL	Recall and practical applications (both individually and in groups)	Lecture videos	116	Group 2	↑Post test
Yang & Newman (2019)	Lectures = >BL	Mini-project lab exercises including delivering demonstrations and writing reports, seminar discussions	Lecture videos	121	Group 2	↑Post test

Notes: *denotes experiments that hold the online condition constant between treatment and control. Group 1: RCTs, Group 2: Non-equivalent groups adjusting for prior differences, Group 3: Non-equivalent groups with no adjustments.

Table 2. Overview of studies in the category of engagement.

Study	Overall change of modality	Activities in the treatment condition		Participants	Method	Outcome
		Face-to-face	Online
Aguilar-Rodríguez et al (2018)	Case studies = > BL	Face-to-face group sessions, debate, role-play, case-study	Various online activities	129	Group 1	↑ Posttest
Albert & Beatty (2014)	Lectures  = > BL	Concept application activities: application questions, video cases with application questions, video on key concepts, multimedia material	Lecture videos	975	Group 3	↑ Exam scores
Alsancak et al. (2018)	Lectures w. online assignments  = > BL	Q&A, discussion, higher-order learning-oriented activities	Lecture videos	66	Group 3	↑ Posttest
Anderson & Brennan (2015)	Lectures  = > BL	Short spell of lecture, class discussion, group work	Lecture videos	312	Group 1	↑ Exam scores
Baepler et al (2014)	Lectures  = > BL w. reduced F2F time	Problem-sets in smaller groups, answer clicker questions, spot explanations of key-concepts, short demonstrations	Lecture videos	1100	Group 2	↔ Exam scores
Blissitt (2016)	Lectures  = > BL	Questions and student-centered activities	Required online activities (completed assignment is required for students to come to class)	56	Group 2	↔ Posttest
Cagande & Jugar (2018)	Lectures  = > BL	In-class activities, instant feedback from teacher	Lecture videos	155	Group 1	↑ Posttest
*Chen & Hwang (2019)	BL  = > BL w. instant response system	Answering individually and in groups via instant response system (‘Kahoot!’), information search, group discussion, teacher feedback	Lecture videos	85	Group 2	↑ Posttest
Deepak et al (2019)	Lectures  = > BL	Group and team-based exercises, various activities including Jigsaw, POGIL, peer-led team-based learning, critical thinking exercises and case-based discussions	Lecture videos	150	Group 3	↑ Exam scores
Elmaadaway (2018)	Lectures  = > BL	Assignment of tasks dependent on previous online work, individual work, presentations	Watch video clips and write down comments	58	Group 1	↑Posttest
*Entezari and Javdan (2016)	BL  = > BL w. active learning	One-minutte paper, group work, discussion, partial outline, personal response card	Lecture videos	66	Group 3	↑ Exam scores
Goh & Ong (2019)	Lectures  = > BL	Group discussion, group presentation using analogies, work finding answers to set question, student presentations of answers	Lecture videos	137	Group 2	↑ Exam scores
He et al. (2019)	Lectures = > BL	Group presentation, in-deep case discussions, group-based exercises and problem-solving	Lecture videos, quizzes	137	Group 1	↑ Posttest
*Maciejewski, 2016	BL  = > BL w. active learning	Lecturing, multiple choice questions via clickers, discussion of answers in pairs, instructor guiding on difficult questions.	Lecture videos, quizzes	690	Group 3	↑ Exam scores

Notes: *denotes experiments that hold the online condition constant between treatment and control. Group 1: RCTs, Group 2: Non-equivalent groups adjusting for prior differences, Group 3: Non-equivalent groups with no adjustments.

Table 3. Overview of studies in the category of social interaction.

Study	Overall change of modality	Activities in the treatment condition		Participants	Method	Outcome
		Face-to-face	Online
Anderson Jr et al (2017)	Lectures  = > BL	Group and individual tests, brief lectures, instructor modeling and examples, discussion, case work, guided note-taking, problem sets, simulation, think-pair-share	Lecture videos	78	Group 1	↑ Exam scores
Almodaires et al. (2019)	Lectures = > BL	Group discussion, in-class activities	Lecture videos, quizzes	195	Group 3	↑ Exam scores
Asikoy & Ozdamli (2016)	Lectures  = > BL	Presentation of news, group work, discussions, Q&A, simulations, problem-solving	Lecture videos, quizzes	66	Group 2	↑ Posttest
Blair et al (2016)	Lectures w. groupwork  = > BL	Q&A about videos, group work	Lecture videos	113	Group 3	↔ Exam scores
Blazquez et al (2019)	Lectures = > BL	Discussion of questions, group work	Lecture videos	110	Group 1	↑ Exam scores
Bonnes et al (2017)	Lectures w. individual application = > BL	Collaborative learning, problem-solving, self-reflection, small-group debate, group project	Lecture videos	143	Group 3	↑ Posttest
Cabi (2018)	Lectures  = > BL	Group presentation of assignment, feedback	Lecture videos	59	Group 1	↔ Posttest
Choi & Lee (2018)	Lecture w. application  = > BL	Interactive learning activities: discussion, teacher presentation, teacher feedback on student products	Review e-book	79	Group 1	↑ Posttest
Colon et al. (2017)	Lectures = > BL	Collaborative work, Problem solving, Activities for solving doubts about the topic studied	Working on course materials online	152	Group 3	↑ Posttest
*Eryilmaz and Cigdemoglu (2019)	BL  = > cooperative BL	Group presentations, group simulation practice, group posters	Lecture videos	57	Group 1	↔ Exam scores
Ficano (2019)	Lectures = > BL	Short lecture review, group based problem-solving	Lecture videos, quizzes	130	Group 2	↔ Exam scores
Foldnes (2016)	Lectures = > BL	Individual content application, Q&A, ad hoc team-based learning	Lecture videos	1569	Group 1	↔ Posttest
Foldnes (2016)	Lectures = > BL	Structured team-based learning	Lecture videos	235	Group 1	↑ Posttest
Hava and Gelibolu (2018)	Lectures  = > BL	Excercises, discussion, report writing on Google Docs	Lecture videos	59	Group 2	↑ Posttest
*Hwang & Chen (2019)	BL  = > BL w. promotion of collective efficacy	Collaborative problem-solving with interactive response system (IRS), group work and – competition, presentations with feedback	Lecture videos	72	Group 2	↑ Posttest
Kennedy et al. (2015)	Lectures  = > BL	Group work, Q&A, worksheets, groupwork on worksheet	Lecture videos	173	Group 2	↔ Exam scores
Prescott et al. (2016)	Lectures  = > BL	Case-based learning (30%), laboratory skills (15%), team-based learning (10%), discussion (5%)	Lecture videos (40%)	253	Group 3	↑ Exam scores
Rui et al. (2017)	Lectures  = > BL	Q&A, lecturing, group analysis of cases, plenary discussion of cases	Lecture videos	181	Group 1	↑ Clinical skills
Sadiq et al. (2018)	Lectures  = > BL	Problem-solving and discussion in groups	Online readings, online solution guides, access to own solution through gadget	150	Group 1	↑ Posttest
Shattuck (2016)	Lectures  = > partital BL	Problem-solving and discussion in groups	Interactive online videos. Review sessions on video	54	Group 3	↑ Related exam questions
Wozny et al. (2018)	Lectures = > BL	Discussion based on pre-class answers to comprehension questions, small group work, mini-lectures	Lecture videos, graded comprehension questions	137	Group 1	↔ Exam scores
Yong et al. (2015)	Lectures w. active learning  = > BL	Answering questions about the video, independent and group work on homework problems, Q&A	Lecture videos	196	Group 2	↔ Posttest
Yough et al. (2019)	Lectures = > Partial BL	Activities for knowledge and comprehension, case studies	Lecture videos, quizzes	263	Group 2	↑ Posttest
Ziegelmeier & Topaz (2015)	Lectures  = > BL	Question and answers, solving problem sets individually, student presenting solutions, group-based problem-solving and lab work, teacher circulating guiding and clarifying questions	Lecture videos, quizzes	45	Group 3	↔ Posttest

Notes: *denotes experiements that hold the online condition constant between treatment and control. Group 1: RCTs, Group 2: Non-equivalent groups adjusting for prior differences, Group 3: Non-equivalent groups with no adjustments.

Higher-order processing

The category of higher-order processing comprises 23 studies containing a total of 27 experimental contrasts. These studies stress the benefits of face-to-face blended learning activities in terms of a deepening understanding and application of content. For example, Bergfjord and Heggernes (2016, 2) summarize their intervention as follows: ‘A significant proportion of the direct instruction was moved out of the classroom in this trial, and time in the classroom was spent using concepts to solve assignment problems.’ Similarly, the intervention in the study by Kurt (2017, 214) was aimed to provide students ‘the opportunity to grapple with real-world problems and apply theory to practice.’ Table 1 presents details of face-to-face activities included in the studies that aim to enhance higher-order processing. As evident from Table 1, the studies include various combinations of different activities but activities that stand out as most frequently applied are discussion, group work and Q&A.

Bloom’s well-known taxonomy (Bloom 1956; Krathwohl 2002) is useful in explicating how these studies utilize the face-to-face modality to move beyond knowledge of terminology, facts, classifications, and theories. This group of studies collectively aspires to reach the higher levels of Bloom’s taxonomy, especially those of comprehension and application. Although the teaching interventions vary, two main approaches stand out. First, the face-to-face activities catering to the objective of comprehension tend to offer students time to ask questions, discuss, and receive tailored teacher elaboration about the learning content they have often encountered online in preparation for class. Second, the face-to face modality centered on the objective of application provides opportunities for practicing the material concerning cases, in-lab settings, or in-real-life practicum settings.

Interventions fostered comprehension through activities such as the discussion of assignment results (Bergfjord and Heggernes 2016), Q&A sessions with the instructor (El-Miedany et al. 2019), group work on worksheets (Lax, Morris, and Kolber 2017), questions answered by means of a personal response system (Lax, Morris, and Kolber 2017), and lecturing tailored to respond to student questions posted on an online discussion board (Lombardini, Lakkala, and Muukkonen 2018).

Application, on the other hand, was practiced through mini-projects (Yang 2012), role play (El-Miedany et al. 2019), independent group work scaffolded by the instructor and the online learning material (Hmelo-Silver et al. 2009), interactive lab-based workshops, hands-on exercises, and project work (Kazanidis et al. 2019), case analysis (Lewis, Chen, and Relan 2018), problem-solving (Sengel 2014), and mini-project exercises in lab sessions (Yang 2012).

Several of the interventions catered to both application and comprehension by combining different types of activities. For example, Lichvar et al. (2016) combined case work and Q&A sessions with instructors, who would explain, elaborate on, and debrief the case. Another example is the intervention presented in Son et al. (2016), where the lecturing and discussion of online exercises were integrated with group exercises in a physical lab and with report writing. In the case of Calimeris and Sauer’s (2015) study, quizzes, Q&As, and the creation of mind maps and concept maps were combined with experiments and the analysis and discussion of news articles and video clips.

The face-to-face activities in this category of higher-order processing were integrated with online components including video lectures in almost all the reviewed studies, whereas alternative online formats, which did not include video lectures, relied on video-based cases (Hmelo-Silver et al. 2009), virtual lab simulations (Son et al. 2016) and online homework assignments (Jensen, Kummer, and Godoy 2015). Most of the interventions used multiple online elements, thus also including online activities such as case challenges (e.g. Hmelo-Silver et al. 2009), forums (e.g. Kazanidis et al. 2019), quizzes (e.g. Kurt 2017), and virtual patient case simulations (e.g. Lichvar et al. 2016).

Engagement

The growing numbers of students in higher education has been described as making student engagement increasingly problematic, and significant attention has therefore been devoted to identifying factors associated with higher levels of engagement (Kahn 2014, 1005). Keeping students engaged has also been posed as a major challenge in technology-enabled learning (Morgan-Thomas and Dudau 2019; Henrie, Halverson, and Graham 2015). The category of engagement contains 14 studies that focus on the pedagogical objective of engaging and motivating students. Details of the face-to-face activities included in the studies addressing student engagement are presented in Table 2. Again, some of the most frequently used activities are group work and discussions, but many different activities are applied.

A common denominator in this category of studies is the assumption that student-centered activities are conducive to student engagement. As Alsancak Sirakaya and Ozdemir (2018, 76) state: ‘the teacher-centered approach has given way to the student-centered approach which requires students to take responsibility for their own learning, to actively participate in learning processes, to control their own learning processes and create the environment where students play a central role.’ Active learning is therefore a focus in all the studies as ‘learners must be active in the learning process and work to construct their own knowledge. When actively learning, course content becomes meaningful and learners develop a deep understanding of the course concepts that are crucial to their success’ (Blissitt 2016, 228). Moreover, all but the studies by Cagande and Jugar (2018) use a combination of individual and collective activities, a design intended to ensure that students participate actively and that opportunities for engagement are plentiful.

Another common denominator of these studies is a reluctance to spend classroom time on lecturing, which is described as lacking ‘mechanisms for ensuring intellectual engagement to the topic’ (Deepak Nallaswamy, Subha, and Asha 2019, 572) and ‘where students sit back passively in an attempt to absorb the information transmitted by their instructors’ (Entezari and Javdan 2016, 221). Instead, knowledge transmission is substantially reduced or shifted online to free up time for other types of classroom activities where students are active participants rather than passive receivers (see e.g. Anderson and Brennan 2015).

The activities applied in the studies cover a broad spectrum of problem-solving tasks and case-studies (see e.g. Baepler, Walker, and Driessen 2014; Deepak Nallaswamy, Subha, and Asha 2019; He et al. 2019), group presentations (see e.g. Elmaadaway 2018; Goh and Ong 2019), and group discussions (see e.g. Aguilar-Rodríguez et al. 2019). Moreover, they include various activities that require students to be active, for example, by participating in role plays (Aguilar-Rodríguez et al. 2019) or in technology-supported activities like response systems (Baepler, Walker, and Driessen 2014; Maciejewski 2016; Chen and Hwang 2019) and cooperative and individualized learning systems (Deepak Nallaswamy, Subha, and Asha 2019). Moreover, instant feedback is understood to support engagement, as reported by Chen and Hwang (2019).

To encourage engagement, teachers are to design, structure, and instruct the many face-to-face activities as well as to integrate these activities with the tasks completed by students online. As noted by Albert and Beatty (2014, 420), ‘the role of the educator is paramount to transform the education process from content centered and teacher centered to learning centered and student centered.’ This means that the teacher is responsible for integrating the online and the face-to-face components and should ‘redesign the curriculum so that the videos watched prior to class are integrated into each class with active learning pedagogies’ (Albert and Beatty 2014, 422). The studies describe students’ role in the classroom as active, as evinced in the wide array of engaging activities carried out in the face-to-face component and thus far from from the traditional lecturing often so prevalent in higher education classrooms. Indeed, students are supposed to take responsibility for their own learning, as described by Alsancak Sirakaya and Ozdemir (2018) and Elmaadaway (2018).

The integration of online components with the face-to-face activities in this category included videos in every study but that by Aguilar-Rodríguez et al. (2019). Often the videos were complemented with a set of questions to be answered in quizzes, discussion forums or homework to be brought to class. These questions are meant to prepare students to actively engage in the face-to-face sessions, which can take the form of in-class discussions, interaction with the teacher, or feedback sessions: ‘A flipped classroom model fundamentally changes the lecture-centered mode of instruction to one that is more learning-centered where the instructor focuses on using class time to improve understanding that the student has attained from watching prerecorded video material and completing assigned readings’ (Albert and Beatty 2014, 419).

Social interaction

The third category, social interaction, includes 23 studies that test the value of interaction as a facilitator for learning in face-to-face blended learning. Articles in this category generally assume that interaction facilitates learning (Akçayır and Akçayır 2018; Bergmann and Sams 2012), and thus maintain that a substantial part of classroom time should be spent on teacher-student interaction or student collaboration (Johnson and Johnson 2009). Student-centered learning is mentioned as a key driver for more social interaction in face-to-face activities. Details of the face-to-face activities included in the studies addressing social interaction are presented in Table 3. As in the two other categories group work and discussions stand out as frequently used activities.

The studies in the social interaction category cover a wide range of activities, including answering questions about preparation tasks such as e-readings (Choi and Lee 2018) or videos watched before class (Blair, Maharaj, and Primus 2016; Yong, Levy, and Lape 2015), group-based problem-solving (Ziegelmeier and Topaz 2015; Shattuck 2016; Sadiq et al. 2018; Bonnes et al. 2017; Colon, Galiano, and Colmenero-Ruiz 2017; Hwang and Chen 2019), and case work (Prescott Jr. et al. 2016).

Student discussions and debates are applied in many of the studies, for example, to discuss solutions to tasks done before class (Blázquez et al. 2019) or assignments completed in-class (Asiksoy and Özdamli 2016). Feedback from teacher to students (Yong, Levy, and Lape 2015; Wozny, Balser, and Ives 2018) as well as between student peers (Cabi 2018; Foldnes 2016) is also integrated into the social interaction studies. Finally, Hava and Gelibolu (2018) increase social interaction through collaborative report writing, and Eryilmaz and Cigdemoglu (2019) do the same in a collaborative hands-on exercise based on the creation of posters in groups. To further incentivize students in the learning process, Hwang and Chen (2019) include intergroup competitions for student engagement.

Although social interaction is central to the learning in all these studies, some combine it with individual activities. In Ziegelmeier and Topaz (2015), for example, students solve problems individually before group work. Other researchers are particularly interested in exploring the differences between individual and collaborative models. This is true of Eryilmaz and Cigdemoglu (2019), who compare individual flipped learning with cooperative flipped learning.

Social interaction is perceived as not only a means for learning but also an important learning in itself, one that prepares students for future work life. Blair, Maharaj, and Primus (2016) state that, by engaging in cooperative learning in the in-class group activities, students prepare for ‘the real world of work where teamwork is essential’ (1469). This aspiration to teach students collaborative skills is also mentioned by Colon, Galiano, and Colmenero-Ruiz (2017), who aim to teach students how to ‘share responsibilities and divisions of tasks, to facilitate correction and the addressing of ideas, while promoting respect, tolerance and open-mindedness towards others’ (132).

The instructor primarily acts as a guide on the side, facilitating group work or problem-solving activities (Ziegelmeier and Topaz 2015; Bonnes et al. 2017; Ficano 2019), providing clarifications, including mini-lectures (Wozny, Balser, and Ives 2018; Anderson et al. 2017), as well as answering questions (see for example Yong, Levy, and Lape 2015; Rui et al. 2017; Blair, Maharaj, and Primus 2016). This guiding or facilitating instructor role is reflected in teachers’ ambition for students to teach each other. As described by Kennedy et al. (2015), students can receive help from both their peers and their instructor and have an opportunity to explain material to their peers (Kennedy et al. 2015, 893). On a similar note, Anderson Jr. et al. (2017) report that peers might even convey the meaning of difficult concepts more effectively than an instructor can.

Compared to the studies in the higher-order processing and in the engagement categories, the face-to-face interventions focusing on social interaction were combined with a narrower range of online components. Most notably, 10 out of the 23 studies exclusively use video lectures as the online component. For the remaining 13 studies, videos form the backbone of the online component, supplemented, for example, with self-test questions for students to answer (e.g. Eryilmaz and Cigdemoglu 2019), quizzes (e.g. Yough et al. 2019), discussion forums (e.g. Ziegelmeier and Topaz 2015), and preparation questions (e.g. Asiksoy and Özdamli 2016).

R2: What empirical evidence do the reviewed studies report about the effects of face-to-face activities on student learning?

Forty-six out of the 64 studied experimental contrasts were reported to yield significant positive changes in learning measures. In the higher-order processing category, 19 of the 27 experimental contrasts produced significant positive changes, in the engagement category 12 out of 14, and in the social interaction category 15 out of 23. Notably, none of the reviewed studies found a negative effect of their interventions on learning outcome. As already mentioned, most of the included studies contrast a lecture-based approach with active blended learning approaches, and of these, 74% found positive effects and none found negative effects. Nine studies had an active learning approach in the control condition, six of which found positive effects. Finally, another nine studies compared two blended learning conditions, with six finding positive effects. However, the varied nature of the interventions and research designs necessitates a careful parsing of the reviewed evidence.

The great majority of the reviewed studies measured effects on complete interventions rather than specific activities. As shown in Appendix B, the studies introduced variation both in the online and in the face-to-face dimension, thus limiting the scope for attribution of effects on learning outcomes specifically to the nature of the face-to-face activities. Similarly, it was difficult to dissect contributions of specific face-to-face activities because the studies tended to introduce multiple new face-to-face activities within the same intervention.

Although we limited our assessment of the effects to student learning, there remain some differences in the outcome measures used. Roughly half of the studies used a content-specific post-test, while the other half used a final exam. Looking at differences in the effects reported between these two groups, we see that 80% of the effects are positive for studies using post-tests, while only 63% are positive for studies using the exam as an outcome measure. Despite this difference in measures, the positive effect on student learning, which was reported in most of the included studies, suggests that the change to active blended learning strategies is valuable to student learning. Although none of the studies reported negative effects on student learning outcomes, the 18 studies that found non-significant effects are a reminder that shifting the learning experience to blended learning does not necessarily produce positive effects.

As mentioned, most research designs focus on comparing lectures with blended learning and thus evaluate the effect of both the online and face-to-face components. However, to know more about what works in the face-to-face component, researchers need to compare the two different forms of blends and vary only the face-to-face component. Accordingly, we now look more closely at the seven studies that did compare two different face-to-face conditions (Chen and Hwang 2019; Entezari and Javdan 2016; Hwang and Chen 2019; Karaoğlan Yilmaz, Öztürk, and Yilmaz 2017; Lichvar et al. 2016; Maciejewski 2016). Among the seven studies changing only face-to-face activities, two focused on investigating whether the positive effects of blended learning can be attributed to active learning strategies. Entezari and Javdan (2016) and Maciejewski (2016) both compared active and lecture-based blended learning, finding that active blended learning has positive effects on academic achievement.

The remaining five studies compared two different active learning strategies within a blended learning setting. Structuring the blended learning format by setting mandatory deadlines and attendance, for example Karaoğlan Yilmaz et al. (2017), was found to positively affect academic achievement, as did using virtual patient cases as opposed to written cases (Lichvar et al. 2016). Chen and Hwang (2019), Hwang and Chen (2019), and Eryilmaz and Cigdemoglu (2019) all investigated the effects of cooperative blended learning. Hwang and Chen (2019) and Chen and Hwang (2019) both investigated the effects of combining interactive response systems with cooperative learning and found positive effects compared to cooperative learning not using interactive response systems. Focusing solely on the effect of cooperative blended learning compared to individual blended learning, Eryilmaz and Cigdemoglu (2019) found no differential effect between the two conditions. In addition to these seven studies, Foldnes (2016) compared two cohorts in consecutive years and found that cooperative blended learning compared to lectures had a positive effect on learning when activities were organized around team-based learning but insignificant effects when group work was ad hoc.

All in all, discussions and group work are the most predominant face-to-face activities emphasizing the social nature of face-to-face activities. However, very few studies have rigorously investigated the effectiveness of different face-to-face activities in a blended learning environment and more research is needed on how to optimize the time students and teachers physically spend together in a future where many classroom learning activities are likely to be shifted online.

Discussion

Below, we discuss the findings from the review and the implications for further research as well as practice. Consistent with other recent reviews of blended and flipped learning, we find that effects on learning outcomes are notably positive across the reviewed studies (Akçayır and Akçayır 2018; O’Flaherty and Phillips 2015), yet findings at the level of specific face-to-face activities are not possible to rigorously assess from the current state of this research. Accordingly, the evidence in our view can serve first and foremost to inspire further experimental studies by researchers and educators.

When looking across the reviewed studies, several face-to-face activities recur such as discussions and group work. However, no apparent pattern can be inferred to learning outcomes as multiple examples of face-to-face activities are associated with both significant and non-significant effects. For example, in the group of studies focusing on higher-order processing, discussion activities were part of the intervention in eight of the studies that found significant positive changes in learning outcome. However, discussions were also used in four studies that reported insignificant effects. A possible key contribution to these seemingly inconsistent results is the variation in ways to measure the dependent variable – learning outcome. As Dziuban et al. (2018) have noted, impact of blended learning designs can only be assessed indirectly through ‘measures such as success, grades, results of assessment protocols, and student testimony about their learning experiences’ (12). As already highlighted, some studies used final exam scores, while others used in-course tests or devised instruments specifically designed to measure the impact of interventions, which means that timing and methods of measurement differed greatly between studies. Another possible contribution is the fact that discussions can be implemented in various forms and most studies provide very little information about the implementation of the face-to-face activities. Accordingly, future research should pay attention to accurately describe the face-to-face activities used as well as the outcome measures used.

Another issue concerns the design around the independent variables – the face-to-face activities. As described in the previous section, only seven studies experimented specifically with the face-to-face condition, which considerably limits the inferences that can be drawn about the contribution of the particular face-to-face activities introduced. This is further accentuated by limited details about the independent variable, that is, the face-to-face activities, thus making comparison or replication difficult. While the assessment of learning outcomes (the dependent variable) is described quite thoroughly in the included studies, more information on the design of the interventions would create a valuable contribution. As Boelens et al. (2017) conclude, future studies should take care to describe the specific blended learning designs along with ‘the rationale for selecting particular online or face-to-face activities’ (12). Such reporting would help not only to better understand the intervention effects but also for formulation of new hypotheses based on a more rigorous variation between treatment and control groups, which would allow tests of specific activities as opposed to a combination of them. Thus, to know more about what works in the precious time students and teachers spend together, future research needs to focus on designing rigorous research studies that allow for inference on the activities of interest holding other factors (such as the online activities) constant.

The final area on which the knowledge base could be expanded, based on the findings from the review, concerns how hypotheses about the mechanisms assumed to account for changes in learning outcomes are explicated and tested. Attention to such underlying mechanisms is important because the specificities of different versions of the same activity type can vary greatly across a range of parameters. For example, the use and nature of group discussions often vary between courses. Thus, rather than attempting to control for such differences, we echo the findings of O’Flaherty and Phillips (2015) and propose that attention instead be focused on the pedagogical mechanisms assumed to be at play. This, we expect, would also make the outcomes of this research even more relevant to educators, as it would become clearer which pedagogical principles to follow in the design of activities.

Implications

Next, we focus on the implications from the study in the broader perspective of previous research on blended learning. Our research has focused specifically on the face-to-face dimension to complement research on the online dimension (see e.g. Rasheed, Kamsin, and Abdullah 2020; Beckman et al. 2021). While separating the two dimensions contributes to explicate the strengths and weaknesses of each, in practice the distinction made between them provides a somewhat artificial division as this binary is blurred in today’s networked learning institutions ‘where the context increasingly allows simultaneous engagement with networks of communities and sources of information beyond the physical walls of the university’ as pointed out by Gourlay (2012, 208). Future research should specifically address the integration of the dimensions, for example, by focusing on the learning objectives and related activities before turning to the mode of delivery. Such an activity-led design process may also need to focus more sharply on the change management aspects of implementing blended learning. This is becoming an important topic for research, as the use of technology in teaching has now become the standard in higher education but in many cases has become so without a clear pedagogical strategy as it was introduced during an emergency (Bond et al. 2021). Learning from the experiences of what works in the online as well as the face-to-face dimensions of learning should now become a strategic priority to make the most from the arduous situation.

Another implication of prioritizing the face-to-face aspects of blended learning concerns the need to revisit opportunities and challenges considering recent experiences from online learning during COVID-19 induced lockdown (Cohen, Nørgård, and Mor 2020). In many instances, the accelerated use of technology to deliver learning has led to an acceptance of ‘good-enough’ solutions, a pragmatic guiding principle used in our own institution, which nevertheless needs rethinking towards more ambitious alternatives, including the design of face-to-face elements. While practical considerations have, for good reasons, driven the transition, the time is right to explore blended learning in relation to pedagogical insights about higher education teaching and learning and thus identify ‘optimal blends’ for student learning, as previously called for (see, for example, Arbaugh 2014). To this end, it could be relevant to explore the value of different pedagogical frameworks for good instruction, for example the framework of Communities of Inquiry (Garrison, Anderson, and Archer 2000), which is already widely used in blended learning research (see for example, Akyol, Garrison, and Ozden 2009; Kim et al. 2014; Vaughan, Cleveland-Innes, and Garrison 2013).

Finally, attention to the quality of the face-to-face dimension of blended learning brings out implications related to the role of educators and institutions in implementing blended learning. Most research into blended learning takes the perspective of students, and relatively little research focuses on professional development or instructors, despite their crucial role in the design and facilitation of blended learning (Bruggeman et al. 2021; Smith and Hill 2019). Studies that do explore teachers’ roles have found that some instructors feel ‘squeezed between the ‘top-down’ demands of their institution and ‘bottom-up’ expectations of undergraduate students regarding the use of web-based technologies’ (Benson and Kolsaker 2015, 324), and they feel inadequately equipped regarding pedagogical or instructional skills to take on the new role. This is also noted by Anthony et al. (2021), who find that teachers’ intentions to use blended learning may not align with their actual usage. Attention to differences in teaching skills, roles and means of communication is also important when teachers take a team-teaching approach to blended learning (McKenzie et al. 2022). Further research is needed to explore the role of the teachers in adopting blended learning but also the role of institutions more broadly in supporting the implementation from the managerial and administrative side as suggested by Bokolo et al, (2020). For example, it has been observed that blended learning’s recasting of face-to-face activities puts demands on the physical layout of higher education institutions, pushing towards new use of existing buildings to create interactive learning commons, spaces for cooperation with industry, or student-led learning spaces (Bebbington 2021, 161).

Conclusion

Based on this scoping review, there appears to be little systematic evidence of how teachers make best use of time in the classroom. The review highlighted a variety of face-to-face activities related to three predominant pedagogical objectives, including higher-order processing, social interaction, and engagement. Numerous configurations of face-to-face activities were found to be associated with positive effects on student learning, and no studies reported negative effects. Our findings show that group work and discussions are the most common face-to-face activities regardless of the pedagogical objective at hand. Yet, while results are largely positive and no negative effects are reported in the included studies, the reviewed literature does not allow conclusions at the level of specific activities. The main contribution of the assessment of the literature offered in this review is therefore the establishment of a foundation for further systematic research into the topic of the face-to-face dimension of blended learning in higher education to complement the extensive research on the online dimension.

Turning the attention to the face-to-face aspect of blended learning offers considerable research opportunities for expanding current knowledge and this review has made visible the need to carefully contrast and compare variations of face-to-face activities in studies on blended learning, while importantly keeping online aspects constant to allow for robust inference. Tested activities need to be clearly described, preferably with a theoretical rationale about the mechanisms underpinning expected benefits to student outcomes. Moreover, as this area of research grows, an important focus also becomes systematic inquiry into the ways activities can best integrate face-to-face interaction with technology.

Although online learning has proved its worth in the later years, it has also become clear how important it is for students to interact face-to-face with other students as well as teachers for learning in higher education. Blended learning has the potential to combine the two modes to create new learning opportunities for students. While much attention has been paid to online activities, there has been less focus on the face-to-face dimension of blended learning and there is a danger that face-to-face activities are not prioritized in designing for better student learning, missing out on important learning opportunities.

Limitations

There are several issues to bear in mind when interpreting the current findings. The research in this article represents a systematic effort to include all available studies on the effects of face-to-face activities in experimental and quasi-experimental studies on higher education blended learning. Accordingly, we chose a rather broad search strategy including six databases. However, this might include some journals that are more oriented towards practitioners rather than scientific journals and thus, despite our eligibility criteria, include studies with less methodological rigor. Therefore, the review findings should be viewed against the limitations of the method. While we strove to minimize selection bias, some articles may have been omitted from the sample due to failures in locating sources or an unintentional but still unjustified exclusion during the screening process. Moreover, some researcher interpretation is always required in the coding and categorization of articles, and we cannot be sure whether this led to any relevant omissions in the sample.

One should also be aware that the studies reviewed tended to report positive effects from changes to the face-to-face activities, but this tendency might, at least in part, be due to a publication bias favoring studies that find positive effects (Franco, Malhotra, and Simonovits 2014). As such, we can only encourage researchers to publish studies showing no or negative effects, which will create the best possible knowledge base for design decisions regarding blended learning.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 These courses varied considerably in length and depth and did not have a pedagogical objective but rather focused on enabling students to participate in other courses.