Pharmacy students’ perceptions and attitudes toward face-to-face vs. virtual team-based learning (TBL) in the didactic curriculum: A mixed-methods study

ABSTRACT

Introduction

Virtual TBL is an online adaptation of the team-based learning (TBL) instructional strategy, emphasizing collaborative learning and problem-solving. The emergency shift to virtual TBL during the COVID-19 pandemic presented unique challenges. This study aims to 1) compare overall pharmacy students’ perceptions and attitudes toward face-to-face (FTF) TBL vs. virtual TBL in the didactic curriculum and stratify their perceptions and attitudes by various students’ characteristics; 2) evaluate students’ perceptions of the strengths and weaknesses of virtual TBL.

Methods

This mixed-methods, pre-post, cross-sectional study utilized an anonymous survey to collect the data. Pharmacy students completed a survey to compare their perceptions and attitudes toward learning, class experience, learning outcomes achieved, and satisfaction with FTF TBL vs. virtual TBL using a 5-point Likert-type scale. Additionally, the survey included two open-ended questions to gather students’ perceptions of the strengths and weaknesses of virtual TBL. Quantitative survey data were analyzed using the Wilcoxon matched-pairs signed rank exact test, while qualitative survey data were analyzed using thematic analysis.

Results

A total of 117 students (response rate of 59.4%) completed the study survey. Pharmacy students perceived FTF TBL to be superior to virtual TBL in their attitudes toward learning, class experience, learning outcomes achieved, and overall satisfaction across various students’ characteristics. While the students identified some unique strengths of using virtual TBL, they also highlighted several weaknesses of using this learning modality compared to FTF TBL.

Conclusions

Pharmacy students perceived FTF TBL to be superior to virtual TBL across various students’ characteristics. These findings can be helpful to pharmacy programs considering the implementation of virtual TBL in their didactic curricula. Future research should explore whether a purposefully designed virtual TBL environment, as opposed to the pandemic-driven emergency TBL planning, can influence students’ perceptions and attitudes toward virtual TBL.

KEYWORDS:

• team-based learning
• TBL
• virtual
• online
• face-to-face
• pharmacy

Introduction

Team-based learning (TBL) is an active and collaborative pedagogical approach that promotes student engagement and the application of conceptual knowledge. It involves preclass preparation, in-class assessment, instructor feedback, and team-based activities [1] The standard TBL framework includes student preclass preparation guided by well-developed learning objectives from instructors, a readiness assurance process including individual assurance test (iRAT) and team readiness assurance test (tRAT), immediate instructor clarification, team applications of critical concepts, appeals, and student peer evaluations. In recent years, TBL has been increasingly adopted in pharmacy curricula, as it has demonstrated benefits such as enhancing critical thinking, teamwork, interpersonal communication, and clinical reasoning skills [2–6].

During the COVID-19 pandemic, academic institutions worldwide, including pharmacy schools, experienced an unprecedented shift from traditional face-to-face (FTF) to virtual remote learning. This emergency change presented unique challenges in course delivery and raised questions about how students perceive and engage with online vs. FTF active learning. These challenges were particularly significant when using TBL as it requires substantial interaction between students within the same team, between different teams, and between students and instructors [2].

Some of the common obstacles experienced during the emergency transition to remote learning included inadequate technological infrastructure, difficulties in engaging and motivating students, the need to adapt teaching methods and instructional materials for effective online delivery, ensuring fair and reliable assessment methods, addressing academic dishonesty in online environments, and maintaining sufficient social interaction and collaboration among students and educators [7–10]. While virtual TBL attempts to preserve the key steps involved in FTF TBL, the virtual delivery of TBL may have an impact on student preparedness and engagement, ultimately influencing their overall TBL learning experience and outcomes.

A recent randomized-controlled study examined students’ perceptions of online case-based learning (CBL) vs. FTF CBL in an undergraduate physiotherapy course [11]. The study revealed that students reported dissatisfaction and perceived a decreased depth of learning with online CBL compared to the FTF CBL environment [11] Another study focused on engineering education and compared the quality of teamwork online vs. FTF, highlighting that team projects suffered from fewer interactions and deliberations when shifted to an online mode [12]. In a hybrid pharmacokinetics course, Franklin et al. (2016) compared students’ perceptions of FTF TBL vs. online TBL [13]. The study included one online TBL group and two FTF TBL groups based on geographic location [13]. The findings indicated that the online TBL cohort perceived lower levels of interdependence and had less favorable perceptions of the team experience compared to the FTF TBL cohorts [13]. Another study by DeMasi et al. (2019) revealed that students in an undergraduate immunology course preferred FTF TBL over virtual TBL [14]. Interestingly, a recent study by Silva et al. (2022) reported favorable perceptions of TBL as a learning pedagogy among students from various disciplines regardless of the delivery mode [15].

A significant limitation of the published literature in this area is its narrow focus on specific modules or courses within programs rather than comprehensively evaluating students’ perceptions and attitudes toward FTF TBL vs. online TBL. Furthermore, there are few studies comparing FTF TBL to virtual TBL in medical and pharmacy education, and none have explored differences in students’ perceptions and attitudes based on students’ various characteristics. Therefore, it is crucial to assess how the emergency shift from FTF TBL to virtual TBL has impacted pharmacy students’ TBL experience and to evaluate their perceptions of the strengths and weaknesses of virtual TBL. Gaining a deeper understanding of pharmacy students’ perceptions and attitudes toward virtual TBL will provide insight into the potential challenges they experienced, assisting pharmacy programs and educators in addressing these challenges in a post-COVID-19 pandemic context.

The objectives of this study were to 1) compare overall pharmacy students’ perceptions and attitudes toward FTF TBL vs. virtual TBL in the didactic curriculum overall and stratify their perceptions and attitudes by various students’ characteristics; 2) evaluate students’ perceptions of the strengths and weaknesses of virtual TBL. To our knowledge, this is the first study to specifically compare pharmacy students’ perceptions and attitudes toward FTF TBL vs. virtual TBL in the pharmacy didactic curriculum while also comparing the students’ perceptions and attitudes toward the two learning modalities stratified by various students’ characteristics.

Materials and methods

Study design and population

This study employed a mixed-methods, retrospective, pre-post, cross-sectional survey design. An anonymous survey was distributed to first-, second-, and third-year professional pharmacy students (P1-P3) at The University of Texas at Tyler Fisch College of Pharmacy from July to August 2021. The inclusion of students from the P1-P3 years in pharmacy school aimed to encompass various topics in the didactic curriculum, teaching and assessment styles, and students’ learning levels. All P1-P3 students had completed at least one semester of didactic courses delivered using TBL, which is the predominant teaching method in most didactic courses at The University of Texas at Tyler Fisch College of Pharmacy. A total of 197 students were eligible to participate in this study: P1 students (n = 46), P2 students (n = 76), and P3 students (n = 75). The survey was developed and distributed via email using Qualtrics® XM (Qualtrics LLC, Provo, UT). The survey was based on students’ FTF TBL experience before the COVID-19 pandemic compared to their virtual TBL experience during the pandemic in pharmacy didactic courses.

During the COVID-19 pandemic, Zoom^© audiovisual teleconferencing (Zoom Video Communications, San Jose, CA) was utilized as the platform for class meetings, discussions, and collaborations for all didactic courses taught to P1-P3 students. The virtual TBL approach consisted of several components. First, individual readiness assurance tests (iRATs) and team readiness assurance tests (tRATs), were administered remotely using ExamSoft® (ExamSoft, Inc., Dallas, TX) or Canvas^© (Instructure, Inc., Salt Lake City, UT). Following the readiness assurance tests, students were assigned to Zoom^© breakout rooms based on their team assignments, where they engaged in discussions regarding the tRAT and collaborated on team applications. The breakout rooms feature in Zoom^© allowed instructors to divide students into separate teams or bring them back together as needed. Students could share their screens with team members to facilitate collaborative work on tRATs and team application exercises. Instructors could also join breakout rooms as facilitators to oversee and engage with the teams. To maintain academic integrity during various virtual course activities, third-party online testing and proctoring platforms, including ExamMonitor™ (ExamSoft, Inc., Dallas, TX) and ProctorU (Meazure Learning, Inc., Birmingham, AL) were utilized.

Survey development

To evaluate students’ perceptions and attitudes toward FTF TBL vs. virtual TBL, a survey was developed since there are no standardized, validated instruments available for this specific purpose. The survey items were created based on the expertise of the study investigators in FTF TBL and virtual TBL, as well as the existing literature that evaluated students’ perceptions and attitudes toward FTF TBL and virtual learning in general [13,16–22],

The survey consisted of 27 items organized into five parts. Part one focused on capturing students’ attitudes toward learning with FTF TBL vs. virtual TBL (eight items). Part two aimed to evaluate students’ class experience with FTF TBL vs. virtual TBL (eight items). Part three assessed students’ perceptions of the learning outcomes achieved with FTF TBL vs. virtual TBL (five items). Part four evaluated students’ satisfaction with FTF TBL vs. virtual TBL (six items). Finally, part five included two open-ended questions for the students to share their perceptions of the strengths and weaknesses of virtual TBL. Each survey item in the first four parts of the survey was presented to the students twice, with students asked to rate their agreement on each item for FTF TBL vs. virtual TBL using a 5-point Likert-type scale (1=strongly disagree to 5=strongly agree). A schematic diagram of the survey design and study participants can be found in Figure 1.

Figure 1. Survey design and study participants.

To ensure face validity, the survey underwent faculty review and was pretested by fourth-year pharmacy students at The University of Texas at Tyler Fisch College of Pharmacy. This process aimed to assess readability and gather feedback on how effectively the survey measures students’ perceptions and attitudes toward FTF TBL vs. virtual TBL. The data collected from this pilot testing were used iteratively to refine the survey items, layout, length, and flow until a consensus was reached. Content validity was established through a comprehensive literature review, ensuring that the survey items represented all the relevant aspects of the constructs being measured. To assess the reliability of the survey and its first four parts, Cronbach’s alpha was used as the index of internal consistency. This index provides an indication of the overall reliability of the survey. The survey is available upon request.

Data analysis

To minimize the risk of observer bias, this study employed mixed methods of data collection for the same observations. Quantitative measures were obtained using a 5-point Likert-type rating scale, while qualitative data were gathered through open-ended questions. Multiple investigators independently performed the quantitative and qualitative data analysis and interpretation, following a predetermined analysis plan developed before examining the data. This approach aimed to ensure consistency and reduce bias. None of the study investigators directly interacted with the participants to discuss the study, thereby avoiding any potential influence on the results.

Descriptive statistics were used to summarize student demographics and other various characteristics. Continuous data were summarized using mean and standard deviation, and categorical data were summarized using frequencies and percentages.

Regarding the quantitative survey data, the students’ ratings of the survey items were evaluated for normality using the Shapiro-Wilk test. The results indicated that the students’ ratings were not normally distributed (p < 0.05). Consequently, the nonparametric Wilcoxon matched-pairs signed rank exact test was employed to compare the medians of students’ ratings for FTF TBL vs. virtual TBL for each survey item. The students’ perceptions and attitudes toward FTF TBL vs. virtual TBL were compared for the overall study sample, as well as stratified by students’ professional school year (corresponding to the number of semesters taught using FTF TBL), cumulative GPA, age, sex, race, highest level of education, and previous virtual learning experience. The central tendency and dispersion of the survey data were represented using the median with interquartile range (IQR).

For the qualitative survey data, two researchers (OS, SP) independently reviewed all responses to the open-ended questions. They then conducted a thematic analysis by inductively identifying the main themes raised by the participating students regarding their perceptions of the strengths and weaknesses of virtual TBL. The researchers categorized the themes following the recognized steps for this type of analysis [23,24]. This method allows the themes to emerge from the data rather than imposing preconceived themes before data collection and analysis [23]. The two researchers compared and discussed the identified themes until reaching a consensus. In cases of disagreement, a third study investigator discussed and resolved the discrepancies until all investigators agreed on the reported themes. Each student’s response might have included more than one theme. The identified themes and representative quotes for each theme were reported.

All statistical tests were two-sided and performed at an a priori significance level of 0.05 using Stata® version 18.0 (StataCorp LLC, College Station, TX). This study was reviewed and deemed exempt by The University of Texas at Tyler Institutional Review Board (Protocol number FY2021–108).

Results

A total of 117 students completed the study survey, resulting in a response rate of 59.4%. The demographic information and other relevant characteristics of the pharmacy students who participated in the study can be found in Table 1.

Table 1. Characteristics of pharmacy students who participated in the survey (N = 117).

Characteristic	No. (%)^a
Professional school year (number of semesters using FTF and virtual TBL)
P1 year (one semester of FTF TBL, one semester of virtual TBL)	21 (17.9)
P2 year (three semesters of FTF TBL, one semester of virtual TBL)	39 (33.3)
P3 year (five semesters of FTF TBL, one semester of virtual TBL)	57 (48.7)
Cumulative GPA
<3.00	29 (24.8)
3.00–3.49	52 (44.4)
3.50–4.00	36 (30.8)
Age (years)
18–24	50 (42.7)
25–34	57 (48.7)
≥35	10 (8.5)
Sex
Male	35 (29.9)
Female	81 (69.2)
Other	1 (0.9)
Race
White/Caucasian	47 (40.2)
Black/African American	32 (27.4)
Other minorities^b	35 (29.9)
Prefer not to answer	3 (2.6)
Highest level of education
High school	27 (23.1)
Associate degree	36 (30.8)
Bachelor’s degree or higher	54 (46.2)
Previous virtual learning experience
None	14 (12.0)
One course	12 (10.3)
Two or more courses	91 (77.8)

FTF = face-to-face; P1 = first-year pharmacy student; P2 = second-year pharmacy student; P3 = third-year pharmacy student.

^aPercentages may not add up to 100% due to rounding.

^bOther minorities included students who selected Asian/Asian American, American Indian/Alaska Native, or ‘Other’ race.

The overall Cronbach’s alpha for all survey items (27 items: Q1-Q27) evaluating students’ perceptions and attitudes toward FTF TBL vs. virtual TBL, was 0.94 and 0.93, respectively. The Cronbach’s alpha values were also calculated for each learning modality across the first four parts of the survey. Part one examined Students’ attitudes toward learning with FTF TBL (eight items: Q1-Q8, α = 0.75) and with virtual TBL (eight items: Q1-Q8, α = 0.71). Part two assessed students’ class experience with FTF TBL (eight items: Q9-Q16, α = 0.86) and with virtual TBL (eight items: Q9-Q16, α = 0.85). Part three focused on students’ learning outcomes achieved with FTF TBL (five items: Q17–Q21, α = 0.86) and with virtual TBL (five items: Q17–Q21, α = 0.86). Finally, part four examined students’ satisfaction with FTF TBL (six items: Q22–Q27, α = 0.82) and with virtual TBL (six items: Q22–Q27, α = 0.79). The reliability of the overall survey and its four parts were considered acceptable, with Cronbach’s alpha values equal to or above 0.7 [25].

Quantitative data (survey parts 1–4)

Overall students’ perceptions of FTF TBL vs. virtual TBL

Students perceived FTF TBL to be superior to virtual TBL in 24 of the 27 survey items. They rated both teaching modalities similarly in the remaining three items. In the first part of the survey, which focused on attitudes toward learning (Q1-Q8), students perceived FTF TBL to be superior to virtual TBL in several aspects. They found FTF TBL to be more effective in terms of their individual and team members’ preparedness for learning (p < 0.001), requiring less self-discipline during courses (p = 0.021), keeping them motivated to learn (p < 0.001), feeling accountable for their team’s learning (p < 0.001), feeling supported by their team (p < 0.001), and feeling less stressed during class activities (p < 0.001). However, there was no significant difference in students’ ratings regarding feeling accountable for their individual learning with FTF TBL compared to virtual TBL (p = 0.398).

Moving on to the second part of the survey, which explored students’ class experience (Q9-Q16), students perceived FTF TBL to be superior to virtual TBL in several areas. They felt more comfortable asking questions (p = 0.041), engaging in team discussions (p < 0.001), collaborating on team applications (p < 0.001), having other team members engaged in discussions (p < 0.001), interacting with instructors (p < 0.001), being less distracted in class (p < 0.001), and learning better (p < 0.001). However, students rated their comfort level in expressing opinions in class similarly with both teaching modalities (p = 0.054).

The third part of the survey focused on students’ perceptions of the achieved learning outcomes (Q17-Q21). Here, students perceived FTF TBL as superior to virtual TBL in all aspects evaluated. They found it easier to remember what they learned (p < 0.001), improve their understanding of concepts during tRAT discussions (p < 0.001), apply their knowledge to solve application exercises (p < 0.001), develop teamwork skills (p < 0.001), and feel prepared for exams (p < 0.001).

In the fourth part of the survey, which assessed students’ satisfaction with FTF TBL compared to virtual TBL (Q22-Q27), students perceived FTF TBL to be superior in several areas. They felt that their course workload was more manageable (p < 0.001), communication with instructors was easier (p < 0.001), they had less struggle with time management during class activities (p = 0.004), and they were overall more satisfied with the learning experience (p < 0.001) with FTF TBL. However, there was no significant difference in students’ ratings of the two learning modalities regarding the use of technology as a facilitator of learning (p = 0.168).

Overall, the survey results indicate that students generally favored FTF TBL over virtual TBL across multiple dimensions, including attitudes toward learning, class experience, learning outcomes, and satisfaction. Table 2 provides a comprehensive comparison of pharmacy students’ perceptions and attitudes toward FTF TBL vs. virtual TBL.

Table 2. Comparison of overall pharmacy students’ attitudes and perceptions of FTF TBL vs. virtual TBL (N = 117).

No.	Survey item^a	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b
	Attitudes toward learning with FTF TBL vs. virtual TBL
Q1	I felt prepared for learning.	4 (4–5)	3 (2–4)	<0.001
Q2	My team members were prepared for learning.	4 (3–4)	3 (2–4)	<0.001
Q3	Courses required self-discipline.	4 (4–5)	5 (4–5)	0.021
Q4	I was motivated to learn.	5 (4–5)	3 (2–4)	<0.001
Q5	I was accountable for my individual learning.	4 (4–5)	4 (4–5)	0.398
Q6	I was accountable for my team’s learning.	4 (3–5)	3 (2–4)	<0.001
Q7	I felt supported by my team.	4 (4–5)	3 (2–4)	<0.001
Q8	I felt stressed when participating in class.	3 (2–4)	4 (2–4)	<0.001
	Class experience with FTF TBL vs. virtual TBL
Q9	I felt comfortable asking questions.	4 (3–4)	4 (3–4)	0.041c
Q10	I felt comfortable expressing my opinion.	4 (3–4)	4 (3–4)	0.054
Q11	I was engaged in team discussions.	4 (4–5)	4 (3–4)	<0.001
Q12	My collaboration on team applications was effective.	4 (4–5)	4 (2–4)	<0.001
Q13	My team members were engaged in team discussions.	4 (4–5)	3 (2–4)	<0.001
Q14	My interaction with instructors was effective.	4 (4–5)	3 (2–4)	<0.001
Q15	I was distracted during class.	2 (2–3)	4 (3–5)	<0.001
Q16	I learned better.	4 (4–5)	2 (1–3)	<0.001
	Learning outcomes achieved with FTF TBL vs. virtual TBL
Q17	I easily remembered what I learned.	4 (3–5)	3 (2–4)	<0.001
Q18	The tRAT discussions allowed me to improve understanding of concepts.	4 (4–5)	3 (2–4)	<0.001
Q19	It was easy to apply knowledge to solve application exercises.	4 (4–5)	3 (2–4)	<0.001
Q20	It helped develop my teamwork skills.	4 (4–5)	3 (2–4)	<0.001
Q21	I felt prepared for exams.	4 (4–5)	3 (2–4)	<0.001
	Satisfaction with FTF TBL vs. virtual TBL
Q22	My overall course workload was manageable.	4 (3–5)	3 (2–4)	<0.001
Q23	It was easy to communicate with my instructors inside the classroom.	4 (4–5)	3 (2–4)	<0.001
Q24	It was easy to communicate with my instructors outside the classroom.	4 (3–5)	4 (2–4)	<0.001
Q25	I struggled with time management while working on class activities.	3 (2–4)	3 (2–4)	0.004d
Q26	The use of technology was a facilitator to my learning.	4 (3–5)	4 (3–5)	0.168
Q27	Overall, I am satisfied with the learning experience.	4 (4–5)	3 (2–4)	<0.001

FTF = face-to-face; TBL = team-based learning; tRAT = team readiness assurance test.

^aStudent perceptions of survey items were measured on a five-point Likert scale: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree.

^aEach item was presented to study participants twice, once for FTF TBL and a second time for virtual TBL to compare their perceptions with the two learning experiences.

^bP values were calculated based on Wilcoxon matched pairs signed-rank exact test comparing students’ ratings of FTF TBL vs. virtual TBL for each survey item.

^bBold indicates p-values below the significance threshold.

^cIndicates higher rank sum for FTF TBL when the medians are equal.

^dIndicates higher rank sum for virtual TBL when the medians are equal.

Students’ perceptions of FTF TBL vs. virtual TBL stratified by professional school year (corresponding to the number of semesters learning using FTF TBL)

When stratified by students’ professional school year (corresponding to the number of semesters learning using FTF TBL), the perceptions of students varied. Among P1 students, FTF TBL was perceived to be superior to virtual TBL in five out of the 27 survey items. For P2 and P3 students, FTF TBL was perceived as superior in 21 and 22 out of the 27 survey items, respectively. Importantly, regardless of their professional year in pharmacy school, none of the students perceived virtual TBL to be superior to FTF TBL in any of the survey items. A detailed breakdown of the stratification of students’ perceptions and attitudes toward FTF TBL vs. virtual TBL by their professional school year in school and the number of semesters learning using FTF TBL can be found in Table 3.

Table 3. Comparison of pharmacy students’ attitudes and perceptions of FTF TBL vs. virtual TBL stratified by professional school year/number of semesters learning using FTF TBL (N = 117).

		P1 year One semester of FTF TBL One semester of virtual TBL			P2 year Three semesters of FTF TBL One semester of virtual TBL			P3 year Five semesters of FTF TBL One semester of virtual TBL
		FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b
	N (%)	21 (17.9)			39 (33.3)			57 (48.7)
No.	Survey item^a
	Attitudes toward learning with FTF TBL vs. virtual TBL
Q1	I felt prepared for learning.	4 (4–4)	4 (3–4)	0.971	4 (4–4)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q2	My team members were prepared for learning.	4 (3–4)	3 (3–4)	0.551	4 (3–4)	2 (2–4)	<0.001	4 (3–4)	2 (2–3)	<0.001
Q3	Courses required self-discipline.	4 (4–5)	5 (4–5)	0.336	4 (4–5)	5 (4–5)	0.012	4 (4–5)	5 (4–5)	0.439
Q4	I was motivated to learn.	4 (4–5)	4 (3–4)	0.054	4 (4–5)	2 (2–4)	<0.001	5 (4–5)	3 (2–4)	<0.001
Q5	I was accountable for my individual learning.	4 (4–5)	4 (4–5)	1.0	4 (4–5)	5 (4–5)	0.699	4 (4–5)	4 (4–5)	0.33
Q6	I was accountable for my team’s learning.	4 (4–4)	4 (4–4)	1.0	4 (3–5)	3 (2–4)	0.015	4 (4–5)	3 (2–4)	<0.001
Q7	I felt supported by my team.	4 (4–5)	4 (3–4)	<0.001c	4 (4–5)	2 (1–4)	<0.001	4 (4–5)	3 (2–3)	<0.001
Q8	I felt stressed when participating in class.	3 (2–4)	4 (2–4)	0.386	3 (2–4)	4 (2–5)	0.027	3 (2–4)	4 (3–4)	0.013
	Class experience with FTF TBL vs. virtual TBL
Q9	I felt comfortable asking questions.	4 (3–4)	4 (3–5)	0.556	4 (3–5)	3 (2–4)	0.016	4 (3–4)	4 (3–4)	0.217
Q10	I felt comfortable expressing my opinion.	4 (3–4)	4 (3–5)	0.755	4 (3–4)	3 (2–4)	0.127	4 (4–5)	4 (3–4)	0.091
Q11	I was engaged in team discussions.	4 (4–5)	4 (4–5)	0.202	4 (4–5)	4 (2–4)	<0.001c	5 (4–5)	4 (3–4)	<0.001
Q12	My collaboration on team applications was effective.	4 (4–5)	4 (3–4)	0.202	4 (4–5)	3 (2–4)	<0.001	5 (4–5)	4 (2–4)	<0.001
Q13	My team members were engaged in team discussions.	4 (4–4)	4 (3–4)	0.045c	4 (4–5)	2 (1–3)	<0.001	4 (4–5)	2 (2–4)	<0.001
Q14	My interaction with instructors was effective.	4 (4–5)	4 (3–4)	0.203	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q15	I was distracted during class.	2 (2–3)	3 (3–4)	0.014	2 (1–3)	4 (3–5)	<0.001	2 (2–3)	4 (3–5)	<0.001
Q16	I learned better.	4 (3–5)	3 (2–3)	0.041	4 (3–5)	2 (1–4)	<0.001	5 (4–5)	2 (1–3)	<0.001
	Learning outcomes achieved with FTF TBL vs. virtual TBL
Q17	I easily remembered what I learned.	4 (3–4)	3 (3–4)	0.253	4 (3–4)	2 (2–3)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q18	The tRAT discussions allowed me to improve understanding of concepts.	4 (4–5)	4 (3–4)	0.031c	4 (4–4)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q19	It was easy to apply knowledge to solve application exercises.	4 (3–4)	3 (3–4)	0.234	4 (3–4)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q20	It helped develop my teamwork skills.	4 (4–5)	3 (2–4)	<0.001	4 (3–5)	3 (2–4)	<0.001	5 (4–5)	3 (2–4)	<0.001
Q21	I felt prepared for exams.	4 (3–4)	4 (3–4)	0.824	4 (3–4)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
	Satisfaction with FTF TBL vs. virtual TBL
Q22	My overall course workload was manageable.	4 (3–4)	4 (3–5)	0.694	4 (3–4)	4 (3–4)	0.857	4 (4–5)	3 (2–4)	<0.001
Q23	It was easy to communicate with my instructors inside the classroom.	4 (3–5)	4 (3–5)	0.317	4 (4–5)	3 (2–4)	<0.001	5 (4–5)	3 (2–4)	<0.001
Q24	It was easy to communicate with my instructors outside the classroom.	4 (3–4)	4 (3–5)	0.866	4 (3–4)	4 (3–4)	0.119	4 (4–5)	3 (2–4)	<0.001
Q25	I struggled with time management while working on class activities.	3 (2–3)	2 (2–4)	0.657	3 (2–4)	3 (2–4)	0.433	2 (2–3)	4 (3–4)	<0.001
Q26	The use of technology was a facilitator to my learning.	4 (3–5)	4 (4–5)	0.063	4 (3–4)	4 (3–5)	0.614	4 (3–5)	4 (3–5)	0.695
Q27	Overall, I am satisfied with the learning experience.	4 (3–5)	4 (2–5)	0.303	4 (3–4)	3 (2–4)	0.011	4 (4–5)	3 (2–4)	<0.001

FTF = face-to-face; TBL = team-based learning; tRAT = team readiness assurance test.

^aStudent perceptions of survey items were measured on a five-point Likert scale: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree.

^aEach item was presented to study participants twice, once for FTF TBL and a second time for virtual TBL to compare their perceptions with the two learning experiences.

^bP values were calculated based on Wilcoxon matched pairs signed-rank exact test comparing students’ ratings of FTF TBL vs. virtual TBL for each survey item.

^bBold indicates p-values below the significance threshold.

^cIndicates higher rank sum for FTF TBL when the medians are equal.

^dIndicates higher rank sum for virtual TBL when the medians are equal.

Students’ perceptions of FTF TBL vs. virtual TBL stratified by cumulative GPA

Students’ perceptions and attitudes toward FTF TBL and virtual TBL varied based on students’ cumulative GPAs in 10 out of the 27 survey items analyzed. Specifically, students with GPAs of 3.00–3.49 or 3.50–4.00 perceived FTF TBL to be superior to virtual TBL in nine of those 10 survey items, compared to students with GPAs below 3.00 who rated both teaching modalities similarly. For the remaining survey item related to feeling stressed during class participation (Table 4: Q8), students with GPAs below 3.00 or GPAs 3.00–3.49 perceived FTF TBL to be superior to virtual TBL (p = 0.035 and p = 0.024, respectively). However, students with GPAs of 3.50–4.00 did not exhibit a significant difference in their ratings of the two teaching modalities for this item (p = 0.161). Importantly, none of the students perceived virtual TBL to be superior to FTF TBL in any of the survey items, irrespective of their GPA category. Table 4 provides a more comprehensive analysis of students’ perceptions and attitudes toward FTF TBL and virtual TBL stratified by their cumulative GPA.

Table 4. Comparison of pharmacy students’ attitudes and perceptions of FTF TBL vs. virtual TBL stratified by cumulative GPA (N = 117).

		GPA <3.00			GPA 3.00–3.49			GPA 3.50–4.00
		FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b
	N (%)	29 (24.8)			52 (44.4)			36 (30.8)
No.	Survey item^a
	Attitudes toward learning with FTF TBL vs. virtual TBL
Q1	I felt prepared for learning.	4 (4–4)	3 (2–4)	0.021	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q2	My team members were prepared for learning.	4 (4–5)	3 (2–4)	0.002	4 (3–4)	3 (2–4)	<0.001	4 (3–4)	2 (2–3)	<0.001
Q3	Courses required self-discipline.	4 (4–5)	5 (4–5)	0.238	4 (4–5)	5 (4–5)	0.042	4 (4–5)	5 (4–5)	0.507
Q4	I was motivated to learn.	4 (4–5)	3 (2–4)	<0.001	5 (4–5)	3 (2–4)	<0.001	5 (4–5)	3 (2–4)	<0.001
Q5	I was accountable for my individual learning.	4 (4–5)	4 (4–5)	0.46	4 (4–5)	4 (4–5)	0.707	5 (4–5)	5 (4–5)	0.212
Q6	I was accountable for my team’s learning.	4 (3–5)	3 (2–4)	0.007	4 (3–5)	4 (2.5–4)	0.009c	4 (3–5)	4 (2–4)	0.021c
Q7	I felt supported by my team.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	2 (1–4)	<0.001
Q8	I felt stressed when participating in class.	3 (2–4)	4 (2–4)	0.035	3 (2–4)	4 (2.5–4)	0.024	3 (2–4)	4 (2–4)	0.161
	Class experience with FTF TBL vs. virtual TBL
Q9	I felt comfortable asking questions.	4 (3–4)	4 (3–4)	0.882	4 (3–4)	4 (3–5)	0.247	4 (4–5)	4 (2–4)	0.038c
Q10	I felt comfortable expressing my opinion.	4 (3–4)	4 (3–4)	0.631	4 (3–4.5)	4 (3–5)	0.489	4 (4–5)	3.5 (2–4)	0.003
Q11	I was engaged in team discussions.	4 (4–5)	4 (4–5)	0.237	5 (4–5)	4 (2.5–4)	<0.001	5 (4–5)	3.5 (2–4)	<0.001
Q12	My collaboration on team applications was effective.	4 (4–5)	4 (3–4)	0.018c	4 (4–5)	3.5 (2–4)	<0.001	4 (4–5)	4 (3–4)	<0.001c
Q13	My team members were engaged in team discussions.	4 (4–4)	3 (2–3)	0.045	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	2 (1–3.5)	<0.001
Q14	My interaction with instructors was effective.	4 (4–5)	3 (2–4)	0.203	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q15	I was distracted during class.	2 (2–3)	4 (3–4)	0.008	2 (2–3)	4 (3–5)	<0.001	2 (2–3)	4 (3–5)	<0.001
Q16	I learned better.	4 (3–5)	3 (2–4)	0.005	5 (4–5)	3 (2–4)	<0.001	5 (4–5)	2 (1–3)	<0.001
	Learning outcomes achieved with FTF TBL vs. virtual TBL
Q17	I easily remembered what I learned.	4 (3–4)	3 (2–4)	0.073	4 (4–5)	3 (2–4)	<0.001	4 (3–5)	3 (2–3)	<0.001
Q18	The tRAT discussions allowed me to improve understanding of concepts.	4 (4–5)	4 (3–4)	0.003c	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q19	It was easy to apply knowledge to solve application exercises.	4 (3–4)	3 (2–4)	0.002	4 (3–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–3.5)	<0.001
Q20	It helped develop my teamwork skills.	4 (4–5)	3 (2–4)	0.005	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q21	I felt prepared for exams.	4 (3–4)	3 (2–4)	0.035	4 (3–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
	Satisfaction with FTF TBL vs. virtual TBL
Q22	My overall course workload was manageable.	4 (3–4)	4 (2–4)	0.365	4 (3–5)	3 (2–4)	0.018	4 (3–4.5)	3 (2–4)	0.007
Q23	It was easy to communicate with my instructors inside the classroom.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q24	It was easy to communicate with my instructors outside the classroom.	4 (4–5)	4 (2–4)	0.007c	4 (3–4.5)	3 (2–4)	0.046	4 (4–5)	4 (3–4)	0.004c
Q25	I struggled with time management while working on class activities.	3 (2–4)	3 (2–4)	0.438	3 (2–4)	3 (2–4)	0.251	2 (2–3)	4 (2.5–4)	0.004
Q26	The use of technology was a facilitator to my learning.	4 (3–5)	4 (4–5)	0.473	4 (3–5)	4 (3–5)	0.309	4 (3–4)	4 (3–5)	0.743
Q27	Overall, I am satisfied with the learning experience.	4 (4–4)	4 (2–4)	0.053	4 (3.5–5)	3 (2–4)	0.001	4 (4–5)	2 (1.5–4)	<0.001

FTF = face-to-face; TBL = team-based learning; tRAT = team readiness assurance test.

^aStudent perceptions of survey items were measured on a five-point Likert scale: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree.

^aEach item was presented to study participants twice, once for FTF TBL and a second time for virtual TBL to compare their perceptions with the two learning experiences.

^bP values were calculated based on Wilcoxon matched pairs signed-rank exact test comparing students’ ratings of FTF TBL vs. virtual TBL for each survey item.

^bBold indicates p-values below the significance threshold.

^cIndicates higher rank sum for FTF TBL when the medians are equal.

^dIndicates higher rank sum for virtual TBL when the medians are equal.

Students’ perceptions of FTF TBL vs. virtual TBL stratified by age

Variations in perceptions and attitudes toward FTF TBL vs. virtual TBL were observed across different student age groups. Students aged 18–24 years or 25–34 years perceived FTF TBL to be superior to virtual TBL in 13 out of 27 survey items. However, students aged 35 years or older rated both teaching modalities similarly for those items. Overall, students in all age groups perceived FTF TBL to be superior to virtual TBL in 13 survey items while rating both modalities similarly in one item. A detailed analysis of students’ perceptions and attitudes toward FTF TBL vs. virtual TBL by age is provided in Table 5.

Table 5. Comparison of pharmacy students’ attitudes and perceptions of FTF TBL vs. virtual TBL stratified by age (N = 117).

		18–24 years old			25–34 years old			≥35 years old
		FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b
	N (%)	50 (42.7)			57 (48.7)			10 (8.5)
No.	Survey item^a
	Attitudes toward learning with FTF TBL vs. virtual TBL
Q1	I felt prepared for learning.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	4 (3–4)	0.002c	4 (4–5)	2.5 (2–4)	0.031
Q2	My team members were prepared for learning.	4 (3–4)	3 (2–4)	<0.001	4 (3–5)	2 (2–4)	<0.001	4 (3–4)	2.5 (1–3)	0.063
Q3	Courses required self-discipline.	4 (4–5)	4.5 (4–5)	0.699	4 (4–5)	5 (4–5)	0.002	4.5 (4–5)	5 (4–5)	0.5
Q4	I was motivated to learn.	5 (4–5)	2.5 (2–4)	<0.001	4 (4–5)	4 (2–4)	<0.001c	5 (4–5)	4 (3–5)	0.031
Q5	I was accountable for my individual learning.	4 (4–5)	4 (4–5)	0.04c	4 (4–5)	5 (4–5)	0.486	4.5 (4–5)	5 (4–5)	1.0
Q6	I was accountable for my team’s learning.	4 (3–5)	3 (2–4)	<0.001	4 (3–5)	4 (2–4)	0.03c	4 (4–5)	3.5 (2–5)	0.125
Q7	I felt supported by my team.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001	4 (4–4)	2 (1–4)	0.031
Q8	I felt stressed when participating in class.	2.5 (2–3)	4 (2–4)	0.003	3 (3–4)	4 (2–4)	0.247	3 (2–4)	4 (4–5)	0.094
	Class experience with FTF TBL vs. virtual TBL
Q9	I felt comfortable asking questions.	4 (3–4)	4 (2–4)	0.046c	4 (3–4)	4 (3–4)	0.754	4 (3–5)	3 (2–5)	0.438
Q10	I felt comfortable expressing my opinion.	4 (3–4)	4 (2–4)	0.032c	4 (3–5)	4 (3–4)	0.829	4 (3–5)	3.5 (2–5)	0.527
Q11	I was engaged in team discussions.	4.5 (4–5)	4 (2–4)	<0.001	4 (4–5)	4 (3–4)	<0.001c	5 (4–5)	4 (3–4)	0.117
Q12	My collaboration on team applications was effective.	4 (4–5)	4 (3–4)	<0.001c	4 (4–5)	4 (3–4)	<0.001c	4.5 (4–5)	3 (2–4)	0.047
Q13	My team members were engaged in team discussions.	4 (4–5)	3 (2–4)	<0.001	4 (3–5)	3 (2–4)	<0.001	4 (4–4)	2 (1–3)	0.004
Q14	My interaction with instructors was effective.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001	4.5 (4–5)	3 (2–4)	0.031
Q15	I was distracted during class.	2 (2–3)	4 (3–5)	<0.001	2 (2–3)	4 (3–4)	<0.001	2 (1–3)	4 (4–5)	0.039
Q16	I learned better.	4.5 (4–5)	2 (1–3)	<0.001	4 (3–5)	3 (2–4)	<0.001	5 (4–5)	1.5 (1–3)	0.016
	Learning outcomes achieved with FTF TBL vs. virtual TBL
Q17	I easily remembered what I learned.	4 (4–5)	3 (2–3)	<0.001	4 (3–5)	3 (2–4)	<0.001	4 (3–4)	3 (2–3)	0.098
Q18	The tRAT discussions allowed me to improve understanding of concepts.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	2.5 (1–5)	0.016
Q19	It was easy to apply knowledge to solve application exercises.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001	4 (4–4)	2 (2–3)	0.008
Q20	It helped develop my teamwork skills.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001	5 (4–5)	2.5 (2–5)	0.031
Q21	I felt prepared for exams.	4 (4–5)	3 (2–4)	<0.001	4 (3–5)	3 (2–4)	<0.001	4 (4–4)	3.5 (2–4)	0.086
	Satisfaction with FTF TBL vs. virtual TBL
Q22	My overall course workload was manageable.	4 (3–5)	3 (2–4)	<0.001	4 (3–4)	4 (3–4)	0.184	4 (3–4)	3 (3–4)	0.469
Q23	It was easy to communicate with my instructors inside the classroom.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	2 (2–4)	0.031
Q24	It was easy to communicate with my instructors outside the classroom.	4 (4–5)	3.5 (2–4)	0.002	4 (3–5)	3 (3–4)	0.02	4 (4–5)	4 (2–4)	0.094
Q25	I struggled with time management while working on class activities.	2 (2–3)	3 (2–4)	0.011	3 (2–4)	3 (2–4)	0.084	3 (2–4)	4 (1–4)	1.0
Q26	The use of technology was a facilitator to my learning.	4 (3–4)	4 (3–5)	0.254	4 (3–5)	4 (4–5)	0.108	4 (3–5)	3.5 (2–5)	0.188
Q27	Overall, I am satisfied with the learning experience.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	0.004	4 (4–5)	1.5 (1–3)	0.004

FTF = face-to-face; TBL = team-based learning; tRAT = team readiness assurance test.

^aStudent perceptions of survey items were measured on a five-point Likert scale: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree.

^aEach item was presented to study participants twice, once for FTF TBL and a second time for virtual TBL to compare their perceptions with the two learning experiences.

^bP values were calculated based on Wilcoxon matched pairs signed-rank exact test comparing students’ ratings of FTF TBL vs. virtual TBL for each survey item.

^bBold indicates p-values below the significance threshold.

^cIndicates higher rank sum for FTF TBL when the medians are equal.

^dIndicates higher rank sum for virtual TBL when the medians are equal.

Students’ perceptions of FTF TBL vs. virtual TBL stratified by sex

There were some variations in the ratings of FTF TBL and virtual TBL between male and female students. Female students perceived FTF TBL to be superior to virtual TBL in five survey items, whereas male students rated both modalities similarly for those items. However, male students perceived FTF TBL to be superior to virtual TBL in one survey item, while female students rated this item similarly for the two teaching modalities. Neither male nor female students favored virtual TBL over FTF TBL in any survey items. The perceptions and attitudes of students toward FTF TBL vs. virtual TBL based on their sex are summarized in Table 6.

Table 6. Comparison of pharmacy students’ attitudes and perceptions of FTF TBL vs. virtual TBL stratified by sex (N = 116).

		Male			Female
		FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b
	N (%)	35 (30.2)			81 (69.8)
No.	Survey item^a
	Attitudes toward learning with FTF TBL vs. virtual TBL
Q1	I felt prepared for learning.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q2	My team members were prepared for learning.	4 (3–4)	2 (2–3)	<0.001	4 (3–4)	3 (2–4)	<0.001
Q3	Courses required self-discipline.	4 (4–5)	5 (4–5)	0.06	4 (4–5)	5 (4–5)	0.139
Q4	I was motivated to learn.	5 (4–5)	3 (2–4)	<0.001	5 (4–5)	3 (2–4)	<0.001
Q5	I was accountable for my individual learning.	4 (4–5)	4 (4–5)	0.619	4 (4–5)	4 (4–5)	0.518
Q6	I was accountable for my team’s learning.	4 (3–5)	4 (2–4)	0.091	4 (3–5)	3 (2–4)	<0.001
Q7	I felt supported by my team.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q8	I felt stressed when participating in class.	3 (2–4)	3 (2–4)	0.099	3 (2–4)	4 (2–4)	0.003
	Class experience with FTF TBL vs. virtual TBL
Q9	I felt comfortable asking questions. FTF	4 (4–4)	4 (3–4)	0.095	4 (3–5)	4 (3–5)	0.201
Q10	I felt comfortable expressing my opinion.	4 (3–5)	3 (3–4)	0.014	4 (3–4)	4 (3–5)	0.545
Q11	I was engaged in team discussions.	5 (4–5)	4 (2–4)	<0.001	4 (4–5)	4 (3–4)	<0.001c
Q12	My collaboration on team applications was effective.	4 (4–5)	4 (2–4)	<0.001c	4 (4–5)	4 (3–4)	<0.001c
Q13	My team members were engaged in team discussions.	4 (4–5)	2 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q14	My interaction with instructors was effective.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q15	I was distracted during class.	2 (2–3)	4 (3–5)	<0.001	2 (2–3)	4 (3–5)	<0.001
Q16	I learned better.	5 (4–5)	2 (1–4)	<0.001	4 (4–5)	3 (1–3)	<0.001
	Learning outcomes achieved with FTF TBL vs. virtual TBL
Q17	I easily remembered what I learned.	4 (3–5)	3 (2–4)	<0.001	4 (3–5)	3 (2–4)	<0.001
Q18	The tRAT discussions allowed me to improve understanding of concepts.	4 (4–5)	3 (2–3)	<0.001	4 (4–5)	4 (2–4)	<0.001c
Q19	It was easy to apply knowledge to solve application exercises.	4 (4–4)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q20	It helped develop my teamwork skills.	5 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q21	I felt prepared for exams.	4 (4–4)	3 (2–4)	0.002	4 (4–5)	3 (2–4)	<0.001
	Satisfaction with FTF TBL vs. virtual TBL
Q22	My overall course workload was manageable.	4 (3–4)	4 (3–4)	0.248	4 (3–5)	3 (2–4)	<0.001
Q23	It was easy to communicate with my instructors inside the classroom.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q24	It was easy to communicate with my instructors outside the classroom.	4 (3–4)	4 (3–4)	0.323	4 (4–5)	3 (2–4)	<0.001
Q25	I struggled with time management while working on class activities.	3 (2–3)	3 (2–4)	0.168	3 (2–4)	3 (2–4)	0.013d
Q26	The use of technology was a facilitator to my learning.	4 (3–4)	4 (4–5)	0.205	4 (3–5)	4 (3–5)	0.456
Q27	Overall, I am satisfied with the learning experience.	4 (3–5)	3 (2–4)	0.012	4 (4–5)	3 (2–4)	<0.001

FTF = face-to-face; TBL = team-based learning; tRAT = team readiness assurance test.

^aStudent perceptions of survey items were measured on a five-point Likert scale: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree.

^aEach item was presented to study participants twice, once for FTF TBL and a second time for virtual TBL to compare their perceptions with the two learning experiences.

^bP values were calculated based on Wilcoxon matched pairs signed-rank exact test comparing students’ ratings of FTF TBL vs. virtual TBL for each survey item.

^cIndicates higher rank sum for FTF TBL when the medians are equal.

^dIndicates higher rank sum for virtual TBL when the medians are equal.

Students’ perceptions of FTF TBL vs. virtual TBL stratified by race

White/Caucasian or other minority students perceived FTF TBL to be superior to virtual TBL compared to Black/African American students, who did not show a significant difference in their ratings for 13 out of 27 survey items. However, Black/African American students rated the use of technology as a facilitator for learning higher for virtual TBL compared to FTF TBL (p = 0.023). A comprehensive analysis of students’ perceptions and attitudes toward FTF TBL vs. virtual TBL stratified by their race is provided in Table 7.

Table 7. Comparison of pharmacy students’ attitudes and perceptions of FTF TBL vs. virtual TBL stratified by race (N = 114).

		White or Caucasian			Black or African American			Other minorities^e
		FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b
	N (%)	47 (41.2)			32 (28.1)			35 (30.7%)
No.	Survey item^a
	Attitudes toward learning with FTF TBL vs. virtual TBL
Q1	I felt prepared for learning.	4 (4–5)	3 (2–4)	<0.001	4 (3–4)	3 (3–4)	0.4	4 (4–5)	3 (2–4)	<0.001
Q2	My team members were prepared for learning.	4 (3–4)	2 (1–3)	<0.001	4 (3–4)	3 (2–4)	0.024	4 (3–5)	3 (2–4)	<0.001
Q3	Courses required self-discipline.	4 (4–5)	5 (4–5)	0.907	4 (4–5)	5 (4–5)	0.017	4 (4–5)	5 (4–5)	0.076
Q4	I was motivated to learn.	5 (4–5)	3 (2–4)	<0.001	4 (3–5)	3 (3–4)	0.02	5 (4–5)	3 (2–4)	<0.001
Q5	I was accountable for my individual learning.	4 (4–5)	5 (4–5)	0.547	4 (4–5)	5 (4–5)	0.521	5 (4–5)	4 (4–5)	0.068
Q6	I was accountable for my team’s learning.	4 (3–5)	3 (2–4)	0.002	4 (3–4.5)	4 (3–4)	0.205	4 (4–5)	4 (3–5)	0.003c
Q7	I felt supported by my team.	4 (4–5)	2 (1–4)	<0.001	4 (4–5)	3.5 (2.5–4)	0.03	5 (4–5)	3 (2–4)	<0.001
Q8	I felt stressed when participating in class.	3 (2–4)	4 (2–5)	0.06	3 (2–4)	3 (2–4)	0.37	3 (2–4)	4 (3–4)	<0.001
	Class experience with FTF TBL vs. virtual TBL
Q9	I felt comfortable asking questions.	4 (4–4)	4 (2–4)	0.038	4 (3–4.5)	4 (3–5)	0.906	4 (4–4)	4 (3–4)	0.198
Q10	I felt comfortable expressing my opinion.	4 (3–4)	4 (2–4)	0.076	4 (3–4)	4 (3–4.5)	0.492	4 (4–5)	4 (3–4)	0.008c
Q11	I was engaged in team discussions.	5 (4–5)	3 (2–4)	<0.001	4 (4–5)	4 (4–5)	0.481	5 (4–5)	4 (3–4)	<0.001
Q12	My collaboration on team applications was effective.	4 (4–5)	4 (2–4)	<0.001c	4 (4–5)	4 (3–5)	0.042c	4 (4–5)	3 (3–4)	<0.001
Q13	My team members were engaged in team discussions.	4 (4–4)	2 (1–3)	<0.001	4 (3–5)	3 (3–4)	0.009	4 (4–5)	3 (2–4)	<0.001
Q14	My interaction with instructors was effective.	4 (4–5)	3 (2–4)	<0.001	4 (3.5–5)	3 (2–4)	0.002	4 (4–5)	3 (2–4)	<0.001
Q15	I was distracted during class.	2 (2–3)	4 (3–5)	<0.001	2.5 (2–4)	3 (2.5–4)	0.048	2 (2–3)	4 (3–4)	<0.001
Q16	I learned better.	5 (4–5)	2 (1–3)	<0.001	4 (3–5)	3 (3–4)	0.205	5 (4–5)	2 (2–3)	<0.001
	Learning outcomes achieved with FTF TBL vs. virtual TBL
Q17	I easily remembered what I learned.	4 (3–5)	2 (2–3)	<0.001	4 (3–4.5)	3 (3–4)	0.252	4 (4–5)	3 (2–4)	<0.001
Q18	The tRAT discussions allowed me to improve understanding of concepts.	4 (4–5)	2 (1–4)	<0.001	4 (4–5)	4 (3–5)	0.023c	4 (4–5)	3 (2–4)	<0.001
Q19	It was easy to apply knowledge to solve application exercises.	4 (4–5)	3 (2–4)	<0.001	4 (3–5)	3 (2.5–4)	0.015	4 (4–5)	3 (2–4)	<0.001
Q20	It helped develop my teamwork skills.	4 (4–5)	3 (1–4)	<0.001	4 (4–5)	4 (2–4)	0.041c	4 (4–5)	3 (2–4)	<0.001
Q21	I felt prepared for exams.	4 (4–5)	3 (2–4)	<0.001	4 (3–5)	3 (3–4)	0.148	4 (4–5)	3 (2–4)	<0.001
	Satisfaction with FTF TBL vs. virtual TBL
Q22	My overall course workload was manageable.	4 (3–4)	3 (2–4)	0.032	4 (3–5)	4 (2.5–4)	0.321	4 (3–5)	3 (2–4)	0.001
Q23	It was easy to communicate with my instructors inside the classroom.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	0.001	4 (4–5)	3 (2–4)	<0.001
Q24	It was easy to communicate with my instructors outside the classroom.	4 (3–5)	4 (3–4)	0.006c	4 (3–5)	4 (3–4)	0.135	4 (4–5)	3 (2–4)	0.002
Q25	I struggled with time management while working on class activities.	2 (2–3)	3 (2–4)	0.251	3 (2–4)	3 (2–4)	0.228	2 (2–4)	4 (3–5)	0.001
Q26	The use of technology was a facilitator to my learning.	4 (3–4)	4 (3–5)	0.931	4 (3–5)	4 (4–5)	0.023d	4 (3–5)	4 (3–5)	0.922
Q27	Overall, I am satisfied with the learning experience.	4 (4–5)	2 (1–4)	<0.001	4 (3–5)	3.5 (2–4.5)	0.327	4 (4–5)	3 (2–4)	<0.001

FTF = face-to-face; TBL = team-based learning; tRAT = team readiness assurance test.

^aStudent perceptions of survey items were measured on a five-point Likert scale: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree.

^aEach item was presented to study participants twice, once for FTF TBL and a second time for virtual TBL to compare their perceptions with the two learning experiences.

^bP values were calculated based on Wilcoxon matched pairs signed-rank exact test comparing students’ ratings of FTF TBL vs. virtual TBL for each survey item.

^bBold indicates p-values below the significance threshold.

^cIndicates higher rank sum for FTF TBL when the medians are equal.

^dIndicates higher rank sum for virtual TBL when the medians are equal.

^eOther minorities included students who selected Asian/Asian American, American Indian/Alaska Native, or ‘Other’ race.

Students’ perceptions of FTF TBL vs. virtual TBL stratified by highest level of education

Variations in students’ perceptions and attitudes toward FTF TBL and virtual TBL were observed based on their highest levels of education in six out of 27 survey items. Students with an associate degree or a bachelor’s degree or higher perceived FTF TBL to be superior to virtual TBL in five of those six items, while students with a high school degree rated both teaching modalities similarly for those items. However, students with a high school degree perceived FTF TBL to be superior to virtual TBL in one item, while students with an associate degree or those with a bachelor’s degree or higher rated both modalities similarly for this item. None of the students with different levels of education perceived virtual TBL to be superior to FTF TBL in any survey items. Detailed analysis of students’ perceptions and attitudes toward FTF TBL vs. virtual TBL by their highest level of education is presented in Table 8.

Table 8. Comparison of pharmacy students’ attitudes and perceptions of FTF TBL vs. virtual TBL stratified by highest level of education completed (N = 117).

		High school			Associate degree			Bachelor’s degree or higher
		FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b
	N (%)	27 (23.1)			36 (30.8)			54 (46.2)
No.	Survey item^a
	Attitudes toward learning with FTF TBL vs. virtual TBL
Q1	I felt prepared for learning.	4 (3–5)	4 (2–4)	0.058	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q2	My team members were prepared for learning.	4 (3–4)	3 (2–4)	0.018	4 (3–4)	3 (2–4)	<0.001	4 (3–4)	2 (2–3)	<0.001
Q3	Courses required self-discipline.	5 (4–5)	5 (4–5)	1.0	4 (4–5)	5 (4–5)	0.342	4 (4–5)	5 (4–5)	0.024
Q4	I was motivated to learn.	5 (4–5)	3 (2–4)	0.001	4 (4–5)	3 (2–4)	<0.001	5 (4–5)	3 (2–4)	<0.001
Q5	I was accountable for my individual learning.	5 (4–5)	4 (4–5)	0.027	4 (4–5)	4 (4–5)	0.768	4 (4–5)	5 (4–5)	0.884
Q6	I was accountable for my team’s learning.	4 (4–5)	3 (2–5)	0.003	4 (3.5–4)	3 (2.5–4)	0.003	4 (3–5)	4 (2–4)	0.049c
Q7	I felt supported by my team.	4 (4–5)	3 (2–4)	0.001	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q8	I felt stressed when participating in class.	3 (2–4)	4 (2–4)	0.248	3 (2–4)	4 (3–5)	0.002	3 (2–4)	4 (2–4)	0.112
	Class experience with FTF TBL vs. virtual TBL
Q9	I felt comfortable asking questions. FTF	4 (3–4)	4 (3–4)	0.833	4 (3.5–5)	4 (3–5)	0.174	4 (4–4)	4 (2–4)	0.095
Q10	I felt comfortable expressing my opinion.	4 (3–4)	4 (3–4)	0.812	4 (4–5)	4 (3–5)	0.203	4 (3–4)	3 (3–4)	0.115
Q11	I was engaged in team discussions.	4 (4–5)	4 (2–4)	0.007c	4.5 (4–5)	4 (3–4)	<0.001	5 (4–5)	4 (2–4)	<0.001
Q12	My collaboration on team applications was effective.	4 (4–5)	4 (3–5)	0.017c	4 (4–5)	4 (2–4)	<0.001c	4 (4–5)	4 (2–4)	<0.001c
Q13	My team members were engaged in team discussions.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	2.5 (2–4)	<0.001	4 (4–5)	3 (2–3)	<0.001
Q14	My interaction with instructors was effective.	4 (4–5)	3 (3–4)	0.002	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q15	I was distracted during class.	2 (2–3)	3 (3–4)	0.002	2 (2–3)	4 (3–5)	<0.001	2 (2–3)	4 (3–5)	<0.001
Q16	I learned better.	4 (3–5)	3 (2–3)	0.002	5 (4–5)	2 (1–3)	<0.001	5 (4–5)	3 (1–3)	<0.001
	Learning outcomes achieved with FTF TBL vs. virtual TBL
Q17	I easily remembered what I learned.	4 (3–5)	3 (2–4)	0.016	4 (4–5)	3 (2–4)	<0.001	4 (3–5)	3 (2–4)	<0.001
Q18	The tRAT discussions allowed me to improve understanding of concepts.	4 (3–4)	4 (2–4)	0.003c	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q19	It was easy to apply knowledge to solve application exercises.	4 (3–4)	3 (2–4)	0.004	4 (4–5)	3 (2–4)	<0.001	4 (3–5)	3 (2–4)	<0.001
Q20	It helped develop my teamwork skills.	4 (4–5)	3 (2–4)	0.004	4 (4–5)	2 (2–4)	<0.001	5 (4–5)	3 (2–4)	<0.001
Q21	I felt prepared for exams.	4 (3–5)	3 (3–4)	0.011	4 (4–4)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
	Satisfaction with FTF TBL vs. virtual TBL
Q22	My overall course workload was manageable.	4 (3–5)	4 (3–4)	0.486	4 (3–4)	3.5 (2–4)	0.041	4 (3–5)	3 (2–4)	0.002
Q23	It was easy to communicate with my instructors inside the classroom.	4 (4–5)	4 (3–4)	0.011c	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q24	It was easy to communicate with my instructors outside the classroom.	4 (4–5)	4 (3–4)	0.056	4 (3–4.5)	3 (2–4)	0.079	4 (4–5)	3 (2–4)	<0.001
Q25	I struggled with time management while working on class activities.	3 (2–4)	3 (2–4)	0.51	2.5 (2–4)	4 (2–5)	0.116	3 (2–3)	3 (2–4)	0.015d
Q26	The use of technology was a facilitator to my learning.	4 (3–5)	4 (3–5)	0.281	4 (3–5)	4 (3–5)	0.383	4 (3–4)	4 (3–5)	0.626
Q27	Overall, I am satisfied with the learning experience.	4 (3–5)	3 (2–4)	0.014	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001

FTF = face-to-face; TBL = team-based learning; tRAT = team readiness assurance test.

^aStudent perceptions of survey items were measured on a five-point Likert scale: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree.

^aEach item was presented to study participants twice, once for FTF TBL and a second time for virtual TBL to compare their perceptions with the two learning experiences.

^bP values were calculated based on Wilcoxon matched pairs signed-rank exact test comparing students’ ratings of FTF TBL vs. virtual TBL for each survey item.

^bBold indicates p-values below the significance threshold.

^cIndicates higher rank sum for FTF TBL when the medians are equal.

^dIndicates higher rank sum for virtual TBL when the medians are equal.

Students’ perceptions of FTF TBL vs. virtual TBL stratified by previous virtual learning experience

When stratified by previous virtual learning experience, students who had previously experienced at least one virtual course perceived FTF TBL to be superior to virtual TBL in 15 out of 27 survey items. However, students with no previous virtual learning experience and those who had previously experienced two or more virtual courses perceived FTF TBL to be superior to virtual TBL in two survey items, while students who previously experienced one virtual course rated both modalities similarly for those items. Regardless of their previous virtual learning experience, none of the students perceived virtual TBL to be superior to FTF TBL in any survey items. Detailed ratings of students’ perceptions and attitudes toward FTF TBL vs. virtual TBL stratified by their previous virtual learning experience can be found in Table 9.

Table 9. Comparison of pharmacy students’ attitudes and perceptions of FTF TBL vs. virtual TBL stratified by previous virtual learning experience (N = 117).

		None			One course			Two or more courses
		FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b
	N (%)	14 (12)			12 (10.3)			91 (77.8)
No.	Survey item^a
	Attitudes toward learning with FTF TBL vs. virtual TBL
Q1	I felt prepared for learning.	4 (3–4)	3 (2–4)	0.137	5 (4–5)	3.5 (2.5–4)	0.065	4 (4–5)	3 (2–4)	<0.001
Q2	My team members were prepared for learning.	4 (4–4)	2.5 (2–4)	0.023	4 (3.5–5)	3 (2–4)	0.078	4 (3–4)	2 (2–4)	<0.001
Q3	Courses required self-discipline.	4 (4–5)	5 (4–5)	0.453	4 (4–5)	5 (4–5)	0.438	4 (4–5)	5 (4–5)	0.051
Q4	I was motivated to learn.	4 (4–5)	3 (2–4)	0.018	5 (4–5)	3 (1.5–4)	0.01	5 (4–5)	3 (2–4)	<0.001
Q5	I was accountable for my individual learning.	4 (4–5)	4.5 (4–5)	0.859	4 (4–5)	4 (4–4.5)	0.375	4 (4–5)	5 (4–5)	0.651
Q6	I was accountable for my team’s learning.	4 (3–4)	4 (3–4)	0.188	4 (3.5–5)	3.5 (2.5–4)	0.164	4 (3–5)	3 (2–4)	<0.001
Q7	I felt supported by my team.	4 (4–5)	2.5 (2–4)	0.008	4.5 (4–5)	2 (2–4)	0.002	4 (4–5)	3 (2–4)	<0.001
Q8	I felt stressed when participating in class.	3 (2–4)	3.5 (3–4)	0.31	3 (2–4)	4 (3.5–4.5)	0.109	3 (2–4)	4 (2–4)	0.007
	Class experience with FTF TBL vs. virtual TBL
Q9	I felt comfortable asking questions. FTF	4 (3–4)	4 (2–4)	0.125	4 (3–4)	4 (3–4)	0.918	4 (3–5)	4 (3–4)	0.076
Q10	I felt comfortable expressing my opinion.	4 (3–4)	3 (3–4)	0.313	4 (2.5–4)	4 (3–4)	0.453	4 (3–5)	4 (3–5)	0.039c
Q11	I was engaged in team discussions.	4.5 (4–5)	4 (3–5)	0.063	5 (4–5)	4 (2.5–4.5)	0.078	4 (4–5)	4 (3–4)	<0.001c
Q12	My collaboration on team applications was effective.	4 (4–5)	4 (3–4)	0.25	4 (4–5)	3.5 (2.5–4)	0.031	4 (4–5)	4 (2–4)	<0.001c
Q13	My team members were engaged in team discussions.	4 (4–4)	3 (1–4)	0.004	4 (4–5)	3 (2–4)	0.02	4 (4–5)	3 (2–4)	<0.001
Q14	My interaction with instructors was effective.	4 (4–5)	3 (2–4)	0.023	4 (4–4.5)	3 (2–4)	0.051	4 (4–5)	3 (2–4)	<0.001
Q15	I was distracted during class.	2 (2–3)	4 (3–4)	0.051	2 (1.5–3)	4 (3–5)	0.014	2 (2–3)	4 (3–5)	<0.001
Q16	I learned better.	4 (3–5)	3 (2–4)	0.008	5 (3.5–5)	2 (1.5–3)	0.016	4 (4–5)	2 (1–3)	<0.001
	Learning outcomes achieved with FTF TBL vs. virtual TBL
Q17	I easily remembered what I learned.	4 (3–4)	3 (2–4)	0.016	4 (3.5–4)	3 (2–3.5)	0.141	4 (3–5)	3 (2–4)	<0.001
Q18	The tRAT discussions allowed me to improve understanding of concepts.	4 (3–5)	3 (2–4)	0.016	4 (4–4.5)	3 (2–4)	0.016	4 (4–5)	3 (2–4)	<0.001
Q19	It was easy to apply knowledge to solve application exercises.	3 (3–4)	3 (2–4)	0.172	4 (3.5–4)	3 (2–4)	0.105	4 (4–5)	3 (2–4)	<0.001
Q20	It helped develop my teamwork skills.	4.5 (4–5)	3 (2–4)	0.004	4.5 (4–5)	2.5 (2–4)	0.028	4 (4–5)	3 (2–4)	<0.001
Q21	I felt prepared for exams.	4 (3–4)	3 (3–4)	0.125	4 (3.5–5)	2 (2–4)	0.049	4 (4–5)	3 (2–4)	<0.001
	Satisfaction with FTF TBL vs. virtual TBL
Q22	My overall course workload was manageable.	4 (3–4)	4 (3–4)	1.0	4 (3–4.5)	3.5 (2–4.5)	0.219	4 (3–5)	3 (2–4)	<0.001
Q23	It was easy to communicate with my instructors inside the classroom.	4 (4–5)	2.5 (2–4)	0.065	4 (3.5–4.5)	3 (3–4)	0.242	4 (4–5)	3 (2–4)	<0.001
Q24	It was easy to communicate with my instructors outside the classroom.	4 (3–5)	4 (2–4)	0.375	4 (3–4)	3.5 (2–4)	0.352	4 (3–5)	3 (3–4)	<0.001
Q25	I struggled with time management while working on class activities.	3 (2–4)	3 (3–4)	0.602	2.5 (2–4)	4 (3–4.5)	0.078	3 (2–4)	3 (2–4)	0.02d
Q26	The use of technology was a facilitator to my learning.	4 (3–5)	4.5 (4–5)	0.797	4 (3–4.5)	4 (3–5)	0.75	4 (3–5)	4 (3–5)	0.228
Q27	Overall, I am satisfied with the learning experience.	4 (4–4)	4 (2–5)	0.295	4 (4–5)	2 (2–4)	0.018	4 (3–5)	3 (2–4)	<0.001

FTF = face-to-face; TBL = team-based learning; tRAT = team readiness assurance test.

^aStudent perceptions of survey items were measured on a five-point Likert scale: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree.

^aEach item was presented to study participants twice, once for FTF TBL and a second time for virtual TBL to compare their perceptions with the two learning experiences.

^bP values were calculated based on Wilcoxon matched pairs signed-rank exact test comparing students’ ratings of FTF TBL vs. virtual TBL for each survey item.

^bBold indicates p-values below the significance threshold.

^cIndicates higher rank sum for FTF TBL when the medians are equal.

^dIndicates higher rank sum for virtual TBL when the medians are equal.

Qualitative data (survey part 5)

Strengths of virtual TBL

Although students generally perceived FTF TBL as superior to virtual TBL in all survey items, their responses to open-ended questions revealed some unique strengths of virtual TBL. Many students commented that ‘virtual TBL offers the time flexibility … and allows us to structure our own individual schedules to manage whatever we have going on or move around study time as needed’ and that with virtual learning, they ‘have better time management and feel more productive’ allowing them to learn at their own pace. Students also expressed that virtual TBL saved them time and money by not commuting to class, as several students commented that ‘virtual learning allowed me more time to study because I did not have to spend so much time driving back and forth … ’. Additionally, the students highlighted the convenience of learning from locations other than the classroom with virtual TBL, the ‘ability to work with their team in a more private environment during virtual TBL’, and ‘allowing more students to be present in class even when they had things going on compared to FTF TBL’. Some students’ comments indicated that virtual TBL taught them how to collaborate with teams remotely and be more comfortable communicating using technology. They stated that virtual TBL ‘gave them the chance to work with a completely virtual group of individuals, conversations flowed, and interactions could be maintained … ’. However, it is worth noting that these strengths contradicted the students’ ratings favoring FTF TBL in engagement in team discussions (Table 2: Q11, Q13) and the effectiveness of collaboration with their teams on application exercises (Table 2: Q12) compared to virtual TBL.

A few students also commented that virtual TBL allowed them to be ‘able to fully concentrate … ’ and ‘have more private discussions without interference from other teams’. However, the students rated virtual TBL higher than FTF TBL in being distracted during class (Table 2: Q15). The students highlighted that virtual TBL encouraged them to practice self-discipline as ‘self-discipline development skills are integral in being successful in virtual TBL … ’. This theme was evident as the students rated virtual TBL higher than FTF TBL in requiring self-discipline (Table 2: Q3). The themes that emerged from students’ perceptions of the strengths of virtual TBL compared to FTF TBL, along with representative student quotes, are presented in Table 10.

Table 10. Students’ perceptions of virtual TBL strengths (N = 55).

Theme	Representative Quotes	No. (%)
Allows students the flexibility to learn at their own pace	‘Virtual TBL allowed students to prepare earlier for classes and create a schedule for studying. From the student perspective we were able to attack preclass material faster so when virtual class started we dived straight into TBL applications versus face to face where it seemed more of a crank-up method.’ ‘Virtual TBL offers the time flexibility that otherwise we may not have and allows us to structure our own individual schedules in order to manage whatever we have going on or move around study time as needed.’ ‘I was able to use down time to work on whatever I needed to work on … I felt it was a much more efficient use of my time.’ ‘With virtual learning, I have better time management and I feel more productive.’	23 (42)
Saves time and money from not commuting to class	‘Virtual TBL gave me much more time. It saved me a trip to school and time to get ready. I think having some courses virtual is a great idea’ ‘It makes pharmacy school more affordable by saving money on parking, commuting back and forth to school, and student housing.’ ‘The only strength I can think of is time. I feel as virtual classes gave you more time to study as it minimizes the time of getting dressed up and having to drive to school – especially for those who commute.’ ‘Virtual learning allowed me MORE TIME to study because I did not have to spend so much time driving back and forth. I was able to gain more rest and energy with virtual learning in order to prepare and study for classes.’	16 (29)
Convenience to learn from locations other than the classroom	‘I liked the convenience and ability to work with my team in a more private environment during virtual TBL’ ‘Virtual TBL allowed for more students to be present in class even when they had things going on compared to FTF TBL.’ ‘Being in the comfort of your own space of your choosing was very beneficial, no outside factors contributing to your study time.’ ‘It is much more convenient to be able to stay home and attend class virtual.’	12 (22)
Learning how to collaborate with teams remotely	‘It allows us to be flexible and be more technological savvy, especially if we would ever need to have long-distance conferences in our careers.’ ‘When given the chance to work with a completely virtual group of individuals, conversations flowed, and interactions could be maintained. As well, we could share google docs and seamlessly dictate and make changes to applications.’	9 (16)
More comfortable communicating using technology	‘Flexibility. I felt more comfortable unmuting myself and asking a question or commenting because it doesn’t feel like I am speaking in front of 75+ people. I asked far more questions that I would in person.’ ‘Individuals who aren’t comfortable with public speaking were able to engage more and answer questions or discuss by typing in Zoom chat or use audio’ ‘It was nice to be able to talk with the instructor in a private chat, it can cause anxiety in some students to have to always ask a question in front of everyone.’	8 (15)
Less distraction from other teams in class	‘I did not have the distractions such as classmates talking during lecture to distract me. I was able to fully concentrate on lecture’ ‘I felt like we were able to have more private discussions without interference from other teams.’	6 (11)
Encourages students to practice self-discipline	‘Self-discipline development skills are integral in being successful in virtual-TBL. It is easier to be on your phone during an virtual course, but dedicated students will remain on task.’	2 (4)

TBL = team-based learning; FTF = face-to-face.

Weaknesses of virtual TBL

Students also identified several weaknesses associated with virtual TBL. One prominent concern was the lack of accountability and engagement from team members. The students expressed that ‘having to work in teams through Zoom and not face to face made it harder to keep the whole team involved and actively participating on applications/group assignments’ and ‘virtual TBL felt less engaging than face-to-face’ and that they were not ‘as motivated to study and prepare for class as with face-to-face TBL’. These themes were supported by the quantitative survey data showing that the students rated FTF TBL higher than virtual TBL in feeling that their team members were prepared for learning (Table 2: Q2), that they were accountable for their team’s learning (Table 2: Q7), and that they or their team members were engaged in team discussions (Table 2: Q11, Q13).

Technical challenges or difficulty using technology was another issue that students raised as they commented that ‘it was a bit of a struggle when it came to finding the best way to be on a video call together … ’ and they also ‘had technical issues that would occur fairly often’. Students mentioned that ‘being out of the classroom setting makes it easier to get distracted by things going on in the home’ and that ‘virtual TBL required high levels of self-discipline’. This was evident in the survey ratings as the students felt more distracted with virtual TBL (Table 2: Q15) and perceived it to require more self-discipline than FTF TBL (Table 2: Q3). Moreover, students highlighted decreased interpersonal communication between team members and difficulties in communicating with instructors as they commented, ‘there was zero team chemistry … ’ during virtual TBL and ‘I didn’t feel connected to my team, and I didn’t even bother learning their names because I never needed to like in face-to-face’. This concurs with the students rating virtual TBL lower than FTF TBL in the effectiveness of interaction with the instructors (Table 2: Q14), developing their teamwork skills (Table 2: Q20), and communicating with the instructors inside the classroom (Table 2: Q23).

Some students also perceived virtual TBL as increasing their workload. They commented that ‘professors took virtual courses = more coursework and assignments since they were under the impression we had more free time’. The students supported this by rating FTF TBL higher than virtual TBL in having a manageable course workload (Table 2: Q22). A few students perceived instructors to be less prepared or passionate during virtual TBL sessions, as some commented, ‘I felt like the professors as a whole were less invested in the process during virtual learning’. Furthermore, the students mentioned that there were ‘high opportunities for cheating to occur during virtual TBL, despite measures taken’ and that classes were less structured than FTF TBL. Table 11 lists the themes that emerged from students’ perceptions of the weaknesses of virtual compared to FTF TBL, along with representative student quotes.

Table 11. Students’ perceptions of virtual TBL weaknesses (N = 60).

Theme	Representative Quotes	No. (%)
Lack of accountability or engagement from team members	‘Having to work in teams through Zoom and not face to face made it harder to keep the whole team involved and actively participating on applications/group assignments.’ ‘Virtual TBL is not an effective method for students to learn. It is unrealistic to expect that all students in an virtual setting will make meaningful contributions to the team applications. Typically, what happens is that only a few team members would participate in the team applications.’ ‘It allowed students to remain silent through the duration of the class and not be held accountable for their contribution. As you could see, generally only one person per group was comfortable even talking in class, which could have been mediated through an in person setting.’ ‘Lack of participation from others. Holding a group accountable for assignments and tasks is difficult when you are not sitting next to each other.’ ‘With virtual learning some students may not have prepared appropriately. Additionally, there was less accountability since we couldn’t see them in person.’	34 (57)
Technical challenges or difficulty using technology	‘The only weakness that I can think of with virtual TBL is that sometimes technology was not working. It would just set us back on time dedicated to the lecture.’ ‘It was a bit of a struggle when it came to finding the best way to be on a video call together. We also had technical issues that would occur fairly often.’ ‘Technology problems also made virtual learning an issue (professor not sure how to set things up/team members not able to use technology). It made it more difficult to truly learn and make the most of class time.’ ‘Technology issues wasted valuable class time.’	12 (20)
More distractions during class time	‘Being out of the classroom setting makes it easier to get distracted by things going on in the home.’ ‘ … it is very easy to get distracted or walk away from the computer.’ ‘No matter how hard I tried to find a quiet place to learn, there were always distractions.’	9 (15)
More self-discipline required to remain focused during class	‘Self-discipline was a must in staying on top of courses.’ ‘Virtual TBL may have been too convenient and required high levels of self-discipline.’ ‘I have to put more time and effort into my own learning.’	8 (13)
Decreased interpersonal communication between team members	“ … lack of human connection. In the classroom, we are able to see our classmates’ body language and we can just freely speak out. Virtual learning requires unmuting yourself and the spotlight would be entirely on you. There’s also uncertainty with reactions that go on behind the screen. I understand that for people with public speaking issues, it requires an extra step, making it harder to gain experience and confidence as face-to-face would give.” ‘There was zero team chemistry during virtual semesters because off class work topic chatting was impossible. I didn’t feel connected to my team, and I didn’t even bother learning their names because I never needed to like in face-to-face.’ ‘Virtual TBL doesn’t strengthen communication skills or work ethic for people that already do not like to talk or contribute to team activities.’	8 (13)
Difficulty communicating with instructors	‘Many professors do not seem to care to put in the effort to learn the technology and be able to check our messages. Often, we are answering them in the form of messages or raised hands (so as to prevent interrupting them), but they do not care to check messages until the end or say can’t figure out how to leave the messages tab open. This ultimately limits the ability of the virtual student to participate fully in the course, even though I never felt I could not simply speak up if I needed to have a point reiterated.’ ‘There was a lack of communication on some tasks that needed to be completed, unanswered questions, and when we were able to ask questions via Zoom or email professors acted as if it was our fault that we did not know…or that we were a bother… a lot of miscommunications.’	7 (12)
Increased students’ workload due to perceived increase in students’ free time by professors	‘For some reason, professors took virtual courses = more coursework and assignments since they were under the impression we had more free time.’ ‘It seemed like professors would give more work to students when we were virtual which took up more of our time.’	6 (10)
Instructors were less prepared/passionate during class	‘I felt like the professors as a whole were less invested in the process during virtual learning.’ ‘Poor team interactions and professor enthusiasm were some of the greatest weaknesses of virtual TBL. Professors read straight from PowerPoints which brought the momentum of learning down. We are in a graduate program, we can read a PowerPoint by ourselves. There is no need for a class time if the class is going to be led in that fashion.’	5 (8)
Students were more likely to engage in academic dishonesty	‘The security of RATs and exams is practically non-existent virtual. Being monitored via webcam is not adequate to ensure testing integrity.’ ‘High opportunities for cheating to occur during virtual TBL, despite measures taken.’	5 (8)
Students are less individually motivated/engaged in class	‘Virtual TBL sometimes felt less engaging than face-to-face’ ‘I was not as motivated to study and prepare for class as I was with face-to-face TBL.’	4 (7)
Classes were less structured	‘There was a lot of down time in some classes where we were just in the breakout rooms finished with discussions or assignments waiting to go back to the main group.’	3 (5)

TBL = team-based learning; RATs = readiness assurance tests.

Discussion

The COVID-19 pandemic has necessitated a rapid shift in educational delivery methods, with pharmacy schools transitioning from FTF classroom instruction to virtual learning. This shift to emergency remote teaching has had a profound impact on how students learn, particularly in the context of TBL, which relies heavily on collaborative interactions between students and instructors. Researchers and scholars have conducted several studies to gain insights and learn from this unprecedented transition.

To the best of our knowledge, this is the first study specifically comparing pharmacy students’ perceptions and attitudes toward FTF TBL vs. virtual TBL in the pharmacy didactic curriculum. We employed a pre-post design, allowing us to assess students’ perceptions and attitudes toward both learning modalities stratified by students’ professional school year, number of semesters learning with FTF TBL, cumulative GPA, age, sex, race, highest level of education, and previous virtual learning experience. It is worth noting that most existing studies in the literature have only compared students’ perceptions and attitudes toward FTF TBL vs. remote learning without specifically exploring the virtual TBL pedagogy [26–30].

In contrast to studies that have evaluated perceptions based on a single course, our study considered students’ overall experience with FTF TBL vs. virtual TBL throughout the pharmacy curriculum [13, 22, 31]. This comprehensive approach provides a broader perspective on students’ perceptions and attitudes. Furthermore, our analysis delved into the multifaceted aspects of both FTF TBL vs. virtual TBL, drawing from an extensive literature review, the expertise of the study investigators with several years of experience in TBL, and valuable input from students as TBL learners.

Overall students’ perceptions of FTF TBL vs. virtual TBL

Overall, the students in this study perceived FTF TBL to be superior to virtual TBL in most survey items. Regardless of the students’ stratification criteria, none of the students perceived virtual TBL to be superior to FTF TBL in any of the survey items, except for Black/African American students who rated technology as a facilitator for learning higher for virtual TBL compared to FTF TBL. These findings are consistent with a study by Atwa et al. (2022), which evaluated the experiences and preferences of medical faculty and students in various learning modalities during the COVID-19 pandemic [32]. The study found that faculty and students perceived FTF learning and blended learning to be superior to online learning in all their survey statements [32]. Similarly, Ahmed et al. (2021) found that while some students showed interest in online learning, most preferred traditional FTF lectures [33]. On the other hand, a recent study by Frederick et al. (2023) demonstrated that pharmacy students perceived blended learning positively for delivering a pharmacy course on management, leadership, and economics compared to FTF learning [34]. Bączek et al. (2020) did not find a difference between medical students’ opinions toward FTF and online learning in terms of increasing knowledge [35]. However, students in their study perceived online learning as less effective in enhancing skills and social competence, and they reported being less active during online classes [35]. In a comparative study by Silva et al. (2022) across multiple scientific disciplines (psychology, communication sciences, biological sciences, and engineering), students equally accepted FTF TBL and virtual TBL, with similar results observed regardless of the scientific discipline [15]. These findings warrant further investigation to explain the discrepancies between studies, particularly considering the absence of TBL in their methodology. It would also be interesting to explore pharmacy and other healthcare professional students’ perceptions of blended TBL compared to FTF TBL and virtual TBL and the factors influencing students’ learning outcomes and satisfaction with the learning experience.

The survey data indicated that students felt more prepared to learn with FTF TBL than with virtual TBL. Similarly, students overall felt that their team members were more prepared to learn with FTF TBL than with virtual TBL. Factors such as technological proficiency, communication and collaboration skills, time management and self-regulation, interpersonal and intercultural competence, and motivation and engagement contribute to students’ preparedness for virtual TBL.

Proficiency in using technology and online tools is crucial for effective participation in virtual TBL, encompassing familiarity with video conferencing platforms, collaborative document editing, file sharing, and online communication tools. Additionally, the students in this study perceived instructors’ occasional lack of proficiency in using technology as an obstacle that hindered students’ ability to collaborate and learn effectively during virtual TBL classes. Venkatesh et al. (2019) emphasized the significance of accessible and reliable technology, prompt technical support, and user-friendly platforms in contributing to medical students’ satisfaction and engagement with a blended integrated e-learning experience [36]. Addressing these factors through appropriate training and practice can significantly mitigate the issues reported in virtual TBL sessions.

Virtual TBL was perceived by the students in this study to require more self-discipline and cause more stress among students compared to FTF TBL. Effective time management and self-discipline skills are essential for students engaging in virtual TBL. Students must set goals, prioritize tasks, and manage their time effectively to meet deadlines and contribute to their team’s progress. Moreover, students felt more motivated to learn, accountable for their individual and team learning, supported by their team, and engaged in team discussions with FTF TBL than with virtual TBL. Motivation plays a crucial role in virtual TBL, as motivated and engaged students are more likely to actively participate, contribute ideas, and take ownership of their learning in the virtual team environment [37].

In the thematic analysis, students identified several weaknesses and strengths of virtual TBL. The most prominent weakness reported was the need for increased accountability and engagement from team members during team exercises. Deep discussions, problem-solving, and learning from peers are essential aspects of TBL that require active engagement from team members. Students expressed the desire for more support from their team members in virtual TBL. The reduced personal communication in virtual TBL may contribute to decreased individual accountability, teamwork, and care among team members. Additionally, students reported a lack of interpersonal communication in virtual TBL. FTF interactions were perceived as fostering robust team dynamics, interpersonal connections, social interactions, and engagement among team members. Developing interpersonal and intercultural competence is crucial for students to navigate cultural differences, build relationships, and work effectively in diverse team settings [38,39].

Distraction was a common concern reported by students during virtual TBL. Effective communication and collaboration are essential in virtual TBL, requiring students to express ideas clearly, actively listen to team members, and engage in constructive discussions and feedback exchanges [40]. Distractions from external factors such as pets or family, as well as engagement in electronic activities unrelated to the class, can hinder students’ ability to retain information and participate fully in virtual TBL [41,42]. However, some students reported being less distracted in virtual TBL when provided with an isolated and controlled environment, enabling them to focus on the team’s activities without interference from other teams or conversations within the classroom.

Students also perceived a lack of academic integrity as a disadvantage of virtual TBL. In virtual TBL settings, academic dishonesty can manifest through sharing answers, copying from peers, unauthorized use of resources, or outsourcing assignments. The absence of physical monitoring by instructors in virtual environments makes it challenging to ensure an environment free from cheating.

Despite the weaknesses identified, students acknowledged several strengths of virtual TBL. Virtual TBL was perceived by the students to facilitate the development of time management and self-discipline skills. Students recognized the flexibility in terms of time and location offered by virtual TBL, saving time and money for those who no longer needed to commute to attend classes in person. Students also highlighted the opportunity to practice collaborating in a virtual environment, which can benefit their future careers as virtual meetings and working environments become increasingly prevalent. Additionally, some students reported a reduced sense of stress and anxiety when communicating virtually, as it provided a level of anonymity and decreased individual attention.

Students’ perceptions of FTF TBL vs. virtual TBL stratified by professional school year (corresponding to the number of semesters learning using FTF TBL)

When stratified by professional school year and the number of semesters of FTF TBL experience, differences in students’ perceptions and attitudes toward FTF TBL and virtual TBL were observed. Students in the P2 and P3 years perceived FTF TBL to be superior to virtual TBL in most survey items compared to P1 students, who rated the two teaching modalities similarly in most survey items.

One possible explanation for these differences is that P1 students have had limited exposure to FTF TBL, with only one semester of experience in both FTF and virtual TBL. In contrast, P2 students have experienced three semesters of FTF TBL and one semester of virtual TBL, while P3 students have had five semesters of FTF TBL and one semester of virtual TBL at the time of the survey. Therefore, the increased familiarity and positive aspects of FTF TBL for P2 and P3 students may have made it more challenging for them to adapt to the newer virtual TBL experience compared to P1 students.

Furthermore, the curriculum structure may have influenced students’ perceptions. The P1 curriculum primarily consists of courses from the biomedical, pharmaceutical, and social/administrative/behavioral sciences, as outlined in the Accreditation Council for Pharmacy Education (ACPE) Standards 2016 [43]. In contrast, the P2 and P3 curricula mainly consist of pharmacotherapy courses from the clinical sciences [43]. These pharmacotherapy courses focus on evidence-based clinical decision-making, therapeutic treatment planning, and medication therapy management for patients with specific diseases and conditions [43]. They require extensive case discussions within and between different teams. As a result, P2 and P3 students may have perceived FTF TBL to facilitate more effective teamwork, immediate information sharing, and in-person interactions among team members compared to virtual TBL.

Interestingly, a study by Shawaqfeh et al. (2020) observed that students in the more advanced years of the pharmacy program had more favorable perceptions and attitudes toward online learning compared to FTF learning during the COVID-19 pandemic [44]. Specifically, fourth-year (P4) students favored online learning the most, while P2 students favored it the least [44]. The authors explained this observation by highlighting that the P2 year is when students start and finish the largest number of therapeutic courses, which aligns with our findings considering the different allocation of pharmacotherapy/therapeutics courses in other curricula [44]. It is important to note that the study by Shawaqfeh et al. (2020) did not utilize TBL, and online learning in their study included a combination of recorded lectures and live online sessions, discussions, and presentations [44].

Overall, these results suggest that students’ perceptions and attitudes toward FTF TBL and virtual TBL are influenced by their professional school year, the number of semesters of FTF TBL experience, and the specific curriculum structure. The findings highlight the importance of considering these factors when implementing and assessing different teaching modalities in pharmacy education.

Students’ perceptions of FTF TBL vs. virtual TBL stratified by cumulative GPA

Students’ perceptions of FTF TBL and virtual TBL varied based on their cumulative GPAs. In general, students with higher GPAs (3.00–3.49 and 3.50–4.00) showed a stronger preference for FTF TBL compared to students with lower GPAs (less than 3.00). This suggests that academic performance may influence students’ perceptions of the two teaching modalities.

Students with higher GPAs typically demonstrate strong academic performance and may positively perceive both FTF TBL and virtual TBL. However, they may prefer FTF TBL due to its structured format, interactive nature, and the opportunity for direct engagement with instructors and peers. On the other hand, students with lower GPAs may appreciate the anonymity and reduced social pressure provided by virtual TBL. To support these findings, previous research by Nguyen et al. (2015) examined student preferences for online vs. FTF courses [45]. Their study revealed that students with higher GPAs tended to prefer FTF courses, possibly due to the perceived benefits of in-person interactions and immediate feedback [45]. Conversely, Ortiz-Rodriguez et al. (2010) found that students with higher GPAs slightly favored online courses, potentially due to their self-regulatory skills and ability to manage independent learning [46].

In addition, two other studies investigated the relationship between GPA and student engagement in online learning environments [47,48]. Both studies indicated that students with higher GPAs reported higher levels of engagement, active participation, collaboration, motivation, and satisfaction with online courses [47,48]. Although these studies provide valuable insights into student preferences based on GPA, they do not specifically focus on TBL.

Overall, the results suggest that students’ perceptions of FTF TBL and virtual TBL are influenced by their cumulative GPAs, with students with higher GPAs generally favoring FTF TBL. However, more research is required to explore this relationship specifically within the context of TBL and to provide a deeper understanding of why these differences exist.

Students’ perceptions of FTF TBL vs. virtual TBL stratified by age

In examining the students’ perceptions of FTF TBL vs. virtual TBL, stratified by age, notable differences were observed. The younger students (18–24 years old or 25–34 years old) expressed a clear preference for FTF TBL over virtual TBL in nearly half of the survey items when compared to the older students 35 years and older. This finding aligns with previous research conducted by Torres et al. (2017), which revealed that senior students (≥30 years old) reported higher levels of engagement and satisfaction with online learning compared to their younger counterparts [49]. The authors suggested that this discrepancy could be attributed to the greater self-regulation and motivation exhibited by older students [49].

Furthermore, a study by Ertmer and Ottenbreit-Leftwich (2010) indicated that older students generally hold more positive attitudes toward technology integration in learning environments when compared to younger students [50]. This suggests that the older cohort may be more receptive to virtual TBL as a mode of instruction. Supporting this notion, Tabata and Johnsrud (2008) conducted research on student perceptions of online learning and found that older students displayed more positive attitudes toward online learning compared to their younger counterparts [51]. They emphasized that senior students particularly valued the flexibility, convenience, and self-directed nature of online courses [51].

Contrasting viewpoints were presented in a study by Nistor et al. (2010), which revealed that older students tended to prefer FTF learning due to its inherent social interaction and immediate feedback, while younger students appreciated the flexibility offered by online learning [52]. While these studies provide partial explanations for the observed perceptions of FTF TBL and virtual TBL in our study, it is important to note that they did not specifically compare the two modalities in our study.

Overall, the results suggest that age plays a significant role in shaping students’ preferences for FTF TBL vs. virtual TBL. Younger students tend to favor the FTF approach, while older students may exhibit a greater acceptance of virtual TBL due to their higher levels of self-regulation, positive attitudes toward technology, and appreciation for flexibility in learning environments. However, the specific comparison between FTF TBL and virtual TBL warrants further investigation in future studies.

Students’ perceptions of FTF TBL vs. virtual TBL stratified by sex

Female students consistently perceived FTF TBL to be superior to virtual TBL across several survey items. They reported higher levels of accountability for their team’s learning, reduced stress levels, a more manageable course workload, ease of communication with instructors, and fewer difficulties with time management while working on class activities with FTF TBL. In contrast, male students did not show a preference between the two learning modalities for these survey items. However, male students did perceive FTF TBL to be superior to virtual TBL when it came to feeling comfortable expressing their opinions.

Interestingly, our findings align with previous research conducted by Venkatesh et al. (2019), who reported that male medical students expressed higher overall learner satisfaction with blended integrated e-learning compared to their female counterparts [36]. The authors attributed these differences, in part, to male students’ significantly higher levels of computer self-efficacy and female students’ preference for a sense of community in their learning environment [36]. Additionally, Silva et al. (2022) found that female students tend to favor collaborative learning, while male students exhibit more competitive and individualistic tendencies [15]. These gender differences may be further accentuated in TBL, which requires active engagement and collaboration within and between different teams.

Our study supports these previous findings, as female students perceived FTF TBL as offering greater accountability for team members, improved communication with instructors, a less stressful learning environment, and better time management during class activities. It is important to acknowledge that the majority of participants in our study were females (69.2%), which may have introduced some bias into the results. Therefore, further investigation is warranted to better understand the influence of gender differences on students’ acceptance of FTF TBL vs. virtual TBL, as well as the underlying factors contributing to these variations.

Students’ perceptions of FTF TBL vs. virtual TBL stratified by race

The perceptions of FTF TBL and virtual TBL, stratified by race, revealed interesting findings. Black/African American students rated FTF TBL and virtual TBL similarly in most survey items, whereas White/Caucasian or other minority students perceived FTF TBL to be superior to virtual TBL in the majority of items. Notably, there is limited existing literature exploring race and ethnicity differences in students’ preference for FTF vs. virtual learning in general, and no studies have specifically compared these modalities using TBL. Ke and Kwak (2013) highlighted the potential cultural issues that may arise in an online learning environment, stemming from dominant cultural values and the possibility of miscommunication among students due to cultural differences [53]. Furthermore, it is plausible that the differences observed in students’ perceptions within our study were influenced by reduced racial bias, as the absence of FTF interaction may diminish the impact of students’ social identity. Future studies should consider incorporating diverse experiences and racial and ethnic differences when examining learning preferences within and across these groups.

Students’ perceptions of FTF TBL vs. virtual TBL stratified by highest level of education

A few survey items revealed differences in students’ perceptions of FTF TBL vs. virtual TBL based on students’ highest level of education. Students with higher degrees, such as bachelor’s or graduate degrees, may have had more exposure to FTF learning throughout their academic journey. This increased familiarity with traditional classroom settings and FTF interactions could potentially contribute to a preference for FTF TBL. Furthermore, higher degree programs often emphasize collaboration and networking opportunities among students and faculty, which may further reinforce the preference for FTF learning.

However, it is important to recognize that this does not necessarily indicate a negative perception of virtual learning among students with higher degrees. Many advanced degree programs now incorporate online components, and individuals pursuing higher education have become more adaptable to virtual learning as they recognize its benefits. This adaptability might explain the fewer variations in perceptions of the two learning modalities based on students’ highest degree in this study. It is worth noting that there is a research gap in this area, and further studies are warranted to explore the relationship between students’ level of education and their perceptions and attitudes toward FTF TBL vs. virtual TBL.

Students’ perceptions of FTF TBL vs. virtual TBL stratified by previous virtual learning experience

The majority of students in this study (77.8%) had previous experience with two or more online courses. These students consistently perceived FTF TBL to be superior to online TBL across almost all survey items. On the other hand, students with little to no previous online learning experience showed a preference for FTF TBL over virtual TBL in only a few survey items. A study conducted by Almaghaslah et al. (2018) investigated the relationship between pharmacy students’ prior experience with online courses and their preference for FTF learning vs. online learning in general [54,55]. In that study, 97% of the students had previous experience with at least one online course, with the majority expressing a preference for FTF lectures over online lectures [54]. Additionally, 17% of the students had no preference for either teaching modality [54]. There is a gap in the literature in this area, and additional research is warranted to evaluate whether previous virtual learning experiences can influence students’ perceptions and attitudes toward FTF TBL vs. virtual TBL.

Key takeaways

The purpose of this study was to investigate students’ perceptions and attitudes toward FTF TBL vs. virtual TBL as instructional modalities in pharmacy education, specifically in the context of emergency remote teaching. The findings contribute to the existing knowledge by shedding light on the challenges and opportunities associated with remote teaching and learning within the TBL framework. Institutions aiming to optimize instructional strategies during similar crises or considering the integration of virtual TBL into their curricula should prioritize investment in robust and scalable technology infrastructure. This includes ensuring reliable internet connectivity, utilizing learning management systems, video conferencing, and providing adequate technical support.

The effective implementation of virtual TBL necessitates instructors to adapt their pedagogical approaches by employing more engaging and interactive teaching methods that foster student participation and enhance learning outcomes. Sustaining student engagement in virtual learning environments is critical and can be achieved through clear instructions, regular communication channels, and personalized feedback [8,9]. The experience of emergency remote teaching has underscored the need for faculty to receive sufficient training and ongoing support to navigate virtual teaching effectively. Professional development programs and resources focusing on virtual TBL pedagogy, technology integration, and instructional design can assist instructors in delivering high-quality online instruction [8,9].

It is important to acknowledge that remote learning can have a negative impact on students’ social and emotional well-being. Therefore, institutions should prioritize providing social support, fostering a sense of belonging, and integrating mental health resources into virtual learning environments. Continuous evaluation and improvement of virtual teaching practices are essential for enhancing the quality and effectiveness of remote teaching. Gathering feedback from students, instructors, and stakeholders can inform necessary adjustments to instructional design, technological infrastructure, and support services.

It is worth noting that perceptions and attitudes toward FTF TBL and virtual TBL may vary depending on students’ personality types, course characteristics, contexts, learning styles, motivation, and the evolving nature of virtual learning environments [9]. Factors such as the nature of the course, subject matter, instructor facilitation, and the advancements and quality of technology infrastructure can influence student perceptions and attitudes toward virtual TBL [56].

The pharmacy students in our study expressed lower satisfaction with virtual TBL compared to FTF TBL. This is concerning, considering the increasing prevalence of online pharmacy education. Many pharmacy schools currently offer or are in the process of establishing online or blended programs that may incorporate virtual TBL. Institutions and educators play a vital role in addressing students’ concerns and creating engaging, inclusive, and well-designed learning experiences regardless of the modality employed. Future studies should evaluate whether there are differences in students’ perceptions and attitudes toward a more carefully and intentionally constructed virtual TBL environment, as opposed to the contingency TBL planning that occurred due to the COVID-19 pandemic.

Limitations

This study has several limitations that should be acknowledged. First, the sample consisted of pharmacy students from a single institution, which limits the generalizability of our findings to other populations. Additionally, the specific demographics of the study participants may introduce potential biases and restrict the broader applicability of the results.

Second, the study focused solely on students’ perceptions and attitudes toward FTF TBL vs. virtual TBL without directly assessing objective learning outcomes or the long-term impact of these modalities on students’ academic performance or professional development. Future studies could incorporate measures of learning outcomes and follow-up assessments to provide a more comprehensive understanding of the comparative effectiveness of FTF TBL vs. virtual TBL.

Third, it is important to consider that students’ perceptions of the quality of instruction and engagement in each learning modality can be influenced by various factors. These factors include the specific teaching methodologies employed, the level of student-teacher interaction, and the overall course design. As FTF TBL allows for direct, real-time engagement with instructors, students’ perceptions of virtual TBL may depend on the instructional quality, engagement, and the instructor’s effectiveness in leveraging technology and virtual tools in their teaching. These variations among different instructors could have introduced confounding that may have influenced the study findings.

Fourth, it is important to acknowledge that variations in student and instructor mastery levels of using technology may have influenced students’ learning experiences and, consequently, their perceptions and attitudes toward virtual TBL. Some students reported difficulties using technology as a weakness of virtual TBL, while others felt more comfortable communicating using technology during virtual TBL sessions. These differences in technological proficiency could have impacted students’ overall learning experiences and their perceptions of virtual TBL.

Fifth, it is important to consider that students’ perceptions and attitudes may change over time as they become more familiar with a specific format or as technology and virtual TBL environments evolve. If the study were conducted over a more extended period after students and instructors had overcome the initial learning curve associated with virtual TBL, their perceptions and attitudes might have differed.

Sixth, that students’ perceptions and attitudes toward virtual TBL compared to FTF TBL, particularly in areas such as preparedness, self-discipline, motivation, stress, distraction, and accountability, may have been influenced by the collateral effects of COVID-19 on students’ well-being. The unique circumstances of the pandemic could have impacted students’ experiences and perceptions of both modalities.

Seventh, there is the possibility of student recall bias since the FTF TBL sessions occurred before the COVID-19 pandemic. Students’ perceptions and attitudes toward FTF TBL may have been more favorable compared to the ‘more recent’ frustrations associated with the shift to virtual TBL.

Despite these limitations, this study provides valuable insights into students’ perceptions and attitudes toward FTF TBL vs. virtual TBL. Further research addressing these limitations would contribute to a more comprehensive understanding of the comparative effectiveness and implications of different instructional modalities in the context of pharmacy education.

Conclusions

The results of this study indicate that pharmacy students consistently rated FTF TBL higher than virtual TBL across multiple dimensions, including attitudes toward learning, class experience, achieving learning outcomes, and overall satisfaction. While virtual TBL exhibited certain unique strengths, it was also associated with several weaknesses when compared to FTF TBL. These weaknesses, as identified by the students, include challenges related to technology, reduced interaction and engagement, and potential limitations in effectively achieving learning outcomes.

Furthermore, the study revealed that students’ perceptions and attitudes toward FTF TBL vs. virtual TBL varied depending on various factors, including their professional school year, the number of semesters they had experienced FTF TBL, cumulative GPA, age, sex, race, highest level of education, and previous virtual learning experience. These findings underscore the importance of considering students’ individual differences and characteristics when implementing virtual TBL in pharmacy education.

The findings of this study provide valuable insights for pharmacy programs and educators who are currently utilizing or considering the use of virtual TBL in their didactic curricula. By understanding the challenges and limitations identified by students, educators can make informed decisions and implement strategies to address these issues, thereby enhancing the virtual TBL experience.

Future research should evaluate whether a more carefully and intentionally constructed virtual TBL environment, designed apart from the contingencies imposed by the COVID-19 pandemic, could potentially alter students’ perceptions and attitudes toward virtual TBL. By exploring specific interventions or improvements to refine the virtual TBL environment, future studies can ascertain whether these modifications can mitigate the identified weaknesses and enhance the overall effectiveness of virtual TBL as a learning modality.

CRediT authorship contribution statement

Osama A. Shoair: Conceptualization, Methodology, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization, Supervision, Project administration, Funding acquisition. Winter J. Smith: Methodology, Investigation, Resources, Writing – review & editing, Funding acquisition. May H. Abdel Aziz: Methodology, Resources, Writing – review & editing. Michael A. Veronin: Methodology, Resources, Writing – review & editing. Joseph S. Glavy: Conceptualization, Methodology, Writing – review & editing. Shelby J. Pirtle: Methodology, Formal analysis, Writing – original draft, Writing – review & editing, Visualization.

Acknowledgments

The authors would like to acknowledge Ms. Heidi Childers, Administrative Assistant III at The University of Texas at Tyler Fisch College of Pharmacy, for her help with the pretesting phase of the study survey.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by the 2021 American Association of Colleges of Pharmacy (AACP) Scholarship of Teaching and Learning (SOTL) Grant