Pharmacy students’ perceptions and attitudes toward face-to-face vs. virtual team-based learning (TBL) in the didactic curriculum: A mixed-methods study

Figures & data

Figure 1. Survey design and study participants.

Table 1. Characteristics of pharmacy students who participated in the survey (N = 117).

Characteristic	No. (%)^a
Professional school year (number of semesters using FTF and virtual TBL)
P1 year (one semester of FTF TBL, one semester of virtual TBL)	21 (17.9)
P2 year (three semesters of FTF TBL, one semester of virtual TBL)	39 (33.3)
P3 year (five semesters of FTF TBL, one semester of virtual TBL)	57 (48.7)
Cumulative GPA
<3.00	29 (24.8)
3.00–3.49	52 (44.4)
3.50–4.00	36 (30.8)
Age (years)
18–24	50 (42.7)
25–34	57 (48.7)
≥35	10 (8.5)
Sex
Male	35 (29.9)
Female	81 (69.2)
Other	1 (0.9)
Race
White/Caucasian	47 (40.2)
Black/African American	32 (27.4)
Other minorities^b	35 (29.9)
Prefer not to answer	3 (2.6)
Highest level of education
High school	27 (23.1)
Associate degree	36 (30.8)
Bachelor’s degree or higher	54 (46.2)
Previous virtual learning experience
None	14 (12.0)
One course	12 (10.3)
Two or more courses	91 (77.8)

FTF = face-to-face; P1 = first-year pharmacy student; P2 = second-year pharmacy student; P3 = third-year pharmacy student.

^aPercentages may not add up to 100% due to rounding.

^bOther minorities included students who selected Asian/Asian American, American Indian/Alaska Native, or ‘Other’ race.

Table 2. Comparison of overall pharmacy students’ attitudes and perceptions of FTF TBL vs. virtual TBL (N = 117).

No.	Survey item^a	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b
	Attitudes toward learning with FTF TBL vs. virtual TBL
Q1	I felt prepared for learning.	4 (4–5)	3 (2–4)	<0.001
Q2	My team members were prepared for learning.	4 (3–4)	3 (2–4)	<0.001
Q3	Courses required self-discipline.	4 (4–5)	5 (4–5)	0.021
Q4	I was motivated to learn.	5 (4–5)	3 (2–4)	<0.001
Q5	I was accountable for my individual learning.	4 (4–5)	4 (4–5)	0.398
Q6	I was accountable for my team’s learning.	4 (3–5)	3 (2–4)	<0.001
Q7	I felt supported by my team.	4 (4–5)	3 (2–4)	<0.001
Q8	I felt stressed when participating in class.	3 (2–4)	4 (2–4)	<0.001
	Class experience with FTF TBL vs. virtual TBL
Q9	I felt comfortable asking questions.	4 (3–4)	4 (3–4)	0.041c
Q10	I felt comfortable expressing my opinion.	4 (3–4)	4 (3–4)	0.054
Q11	I was engaged in team discussions.	4 (4–5)	4 (3–4)	<0.001
Q12	My collaboration on team applications was effective.	4 (4–5)	4 (2–4)	<0.001
Q13	My team members were engaged in team discussions.	4 (4–5)	3 (2–4)	<0.001
Q14	My interaction with instructors was effective.	4 (4–5)	3 (2–4)	<0.001
Q15	I was distracted during class.	2 (2–3)	4 (3–5)	<0.001
Q16	I learned better.	4 (4–5)	2 (1–3)	<0.001
	Learning outcomes achieved with FTF TBL vs. virtual TBL
Q17	I easily remembered what I learned.	4 (3–5)	3 (2–4)	<0.001
Q18	The tRAT discussions allowed me to improve understanding of concepts.	4 (4–5)	3 (2–4)	<0.001
Q19	It was easy to apply knowledge to solve application exercises.	4 (4–5)	3 (2–4)	<0.001
Q20	It helped develop my teamwork skills.	4 (4–5)	3 (2–4)	<0.001
Q21	I felt prepared for exams.	4 (4–5)	3 (2–4)	<0.001
	Satisfaction with FTF TBL vs. virtual TBL
Q22	My overall course workload was manageable.	4 (3–5)	3 (2–4)	<0.001
Q23	It was easy to communicate with my instructors inside the classroom.	4 (4–5)	3 (2–4)	<0.001
Q24	It was easy to communicate with my instructors outside the classroom.	4 (3–5)	4 (2–4)	<0.001
Q25	I struggled with time management while working on class activities.	3 (2–4)	3 (2–4)	0.004d
Q26	The use of technology was a facilitator to my learning.	4 (3–5)	4 (3–5)	0.168
Q27	Overall, I am satisfied with the learning experience.	4 (4–5)	3 (2–4)	<0.001

FTF = face-to-face; TBL = team-based learning; tRAT = team readiness assurance test.

^aStudent perceptions of survey items were measured on a five-point Likert scale: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree.

^aEach item was presented to study participants twice, once for FTF TBL and a second time for virtual TBL to compare their perceptions with the two learning experiences.

^bP values were calculated based on Wilcoxon matched pairs signed-rank exact test comparing students’ ratings of FTF TBL vs. virtual TBL for each survey item.

^bBold indicates p-values below the significance threshold.

^cIndicates higher rank sum for FTF TBL when the medians are equal.

^dIndicates higher rank sum for virtual TBL when the medians are equal.

Table 3. Comparison of pharmacy students’ attitudes and perceptions of FTF TBL vs. virtual TBL stratified by professional school year/number of semesters learning using FTF TBL (N = 117).

		P1 year One semester of FTF TBL One semester of virtual TBL			P2 year Three semesters of FTF TBL One semester of virtual TBL			P3 year Five semesters of FTF TBL One semester of virtual TBL
		FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b
	N (%)	21 (17.9)			39 (33.3)			57 (48.7)
No.	Survey item^a
	Attitudes toward learning with FTF TBL vs. virtual TBL
Q1	I felt prepared for learning.	4 (4–4)	4 (3–4)	0.971	4 (4–4)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q2	My team members were prepared for learning.	4 (3–4)	3 (3–4)	0.551	4 (3–4)	2 (2–4)	<0.001	4 (3–4)	2 (2–3)	<0.001
Q3	Courses required self-discipline.	4 (4–5)	5 (4–5)	0.336	4 (4–5)	5 (4–5)	0.012	4 (4–5)	5 (4–5)	0.439
Q4	I was motivated to learn.	4 (4–5)	4 (3–4)	0.054	4 (4–5)	2 (2–4)	<0.001	5 (4–5)	3 (2–4)	<0.001
Q5	I was accountable for my individual learning.	4 (4–5)	4 (4–5)	1.0	4 (4–5)	5 (4–5)	0.699	4 (4–5)	4 (4–5)	0.33
Q6	I was accountable for my team’s learning.	4 (4–4)	4 (4–4)	1.0	4 (3–5)	3 (2–4)	0.015	4 (4–5)	3 (2–4)	<0.001
Q7	I felt supported by my team.	4 (4–5)	4 (3–4)	<0.001c	4 (4–5)	2 (1–4)	<0.001	4 (4–5)	3 (2–3)	<0.001
Q8	I felt stressed when participating in class.	3 (2–4)	4 (2–4)	0.386	3 (2–4)	4 (2–5)	0.027	3 (2–4)	4 (3–4)	0.013
	Class experience with FTF TBL vs. virtual TBL
Q9	I felt comfortable asking questions.	4 (3–4)	4 (3–5)	0.556	4 (3–5)	3 (2–4)	0.016	4 (3–4)	4 (3–4)	0.217
Q10	I felt comfortable expressing my opinion.	4 (3–4)	4 (3–5)	0.755	4 (3–4)	3 (2–4)	0.127	4 (4–5)	4 (3–4)	0.091
Q11	I was engaged in team discussions.	4 (4–5)	4 (4–5)	0.202	4 (4–5)	4 (2–4)	<0.001c	5 (4–5)	4 (3–4)	<0.001
Q12	My collaboration on team applications was effective.	4 (4–5)	4 (3–4)	0.202	4 (4–5)	3 (2–4)	<0.001	5 (4–5)	4 (2–4)	<0.001
Q13	My team members were engaged in team discussions.	4 (4–4)	4 (3–4)	0.045c	4 (4–5)	2 (1–3)	<0.001	4 (4–5)	2 (2–4)	<0.001
Q14	My interaction with instructors was effective.	4 (4–5)	4 (3–4)	0.203	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q15	I was distracted during class.	2 (2–3)	3 (3–4)	0.014	2 (1–3)	4 (3–5)	<0.001	2 (2–3)	4 (3–5)	<0.001
Q16	I learned better.	4 (3–5)	3 (2–3)	0.041	4 (3–5)	2 (1–4)	<0.001	5 (4–5)	2 (1–3)	<0.001
	Learning outcomes achieved with FTF TBL vs. virtual TBL
Q17	I easily remembered what I learned.	4 (3–4)	3 (3–4)	0.253	4 (3–4)	2 (2–3)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q18	The tRAT discussions allowed me to improve understanding of concepts.	4 (4–5)	4 (3–4)	0.031c	4 (4–4)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q19	It was easy to apply knowledge to solve application exercises.	4 (3–4)	3 (3–4)	0.234	4 (3–4)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q20	It helped develop my teamwork skills.	4 (4–5)	3 (2–4)	<0.001	4 (3–5)	3 (2–4)	<0.001	5 (4–5)	3 (2–4)	<0.001
Q21	I felt prepared for exams.	4 (3–4)	4 (3–4)	0.824	4 (3–4)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
	Satisfaction with FTF TBL vs. virtual TBL
Q22	My overall course workload was manageable.	4 (3–4)	4 (3–5)	0.694	4 (3–4)	4 (3–4)	0.857	4 (4–5)	3 (2–4)	<0.001
Q23	It was easy to communicate with my instructors inside the classroom.	4 (3–5)	4 (3–5)	0.317	4 (4–5)	3 (2–4)	<0.001	5 (4–5)	3 (2–4)	<0.001
Q24	It was easy to communicate with my instructors outside the classroom.	4 (3–4)	4 (3–5)	0.866	4 (3–4)	4 (3–4)	0.119	4 (4–5)	3 (2–4)	<0.001
Q25	I struggled with time management while working on class activities.	3 (2–3)	2 (2–4)	0.657	3 (2–4)	3 (2–4)	0.433	2 (2–3)	4 (3–4)	<0.001
Q26	The use of technology was a facilitator to my learning.	4 (3–5)	4 (4–5)	0.063	4 (3–4)	4 (3–5)	0.614	4 (3–5)	4 (3–5)	0.695
Q27	Overall, I am satisfied with the learning experience.	4 (3–5)	4 (2–5)	0.303	4 (3–4)	3 (2–4)	0.011	4 (4–5)	3 (2–4)	<0.001

FTF = face-to-face; TBL = team-based learning; tRAT = team readiness assurance test.

^aStudent perceptions of survey items were measured on a five-point Likert scale: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree.

^aEach item was presented to study participants twice, once for FTF TBL and a second time for virtual TBL to compare their perceptions with the two learning experiences.

^bP values were calculated based on Wilcoxon matched pairs signed-rank exact test comparing students’ ratings of FTF TBL vs. virtual TBL for each survey item.

^bBold indicates p-values below the significance threshold.

^cIndicates higher rank sum for FTF TBL when the medians are equal.

^dIndicates higher rank sum for virtual TBL when the medians are equal.

Table 4. Comparison of pharmacy students’ attitudes and perceptions of FTF TBL vs. virtual TBL stratified by cumulative GPA (N = 117).

		GPA <3.00			GPA 3.00–3.49			GPA 3.50–4.00
		FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b
	N (%)	29 (24.8)			52 (44.4)			36 (30.8)
No.	Survey item^a
	Attitudes toward learning with FTF TBL vs. virtual TBL
Q1	I felt prepared for learning.	4 (4–4)	3 (2–4)	0.021	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q2	My team members were prepared for learning.	4 (4–5)	3 (2–4)	0.002	4 (3–4)	3 (2–4)	<0.001	4 (3–4)	2 (2–3)	<0.001
Q3	Courses required self-discipline.	4 (4–5)	5 (4–5)	0.238	4 (4–5)	5 (4–5)	0.042	4 (4–5)	5 (4–5)	0.507
Q4	I was motivated to learn.	4 (4–5)	3 (2–4)	<0.001	5 (4–5)	3 (2–4)	<0.001	5 (4–5)	3 (2–4)	<0.001
Q5	I was accountable for my individual learning.	4 (4–5)	4 (4–5)	0.46	4 (4–5)	4 (4–5)	0.707	5 (4–5)	5 (4–5)	0.212
Q6	I was accountable for my team’s learning.	4 (3–5)	3 (2–4)	0.007	4 (3–5)	4 (2.5–4)	0.009c	4 (3–5)	4 (2–4)	0.021c
Q7	I felt supported by my team.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	2 (1–4)	<0.001
Q8	I felt stressed when participating in class.	3 (2–4)	4 (2–4)	0.035	3 (2–4)	4 (2.5–4)	0.024	3 (2–4)	4 (2–4)	0.161
	Class experience with FTF TBL vs. virtual TBL
Q9	I felt comfortable asking questions.	4 (3–4)	4 (3–4)	0.882	4 (3–4)	4 (3–5)	0.247	4 (4–5)	4 (2–4)	0.038c
Q10	I felt comfortable expressing my opinion.	4 (3–4)	4 (3–4)	0.631	4 (3–4.5)	4 (3–5)	0.489	4 (4–5)	3.5 (2–4)	0.003
Q11	I was engaged in team discussions.	4 (4–5)	4 (4–5)	0.237	5 (4–5)	4 (2.5–4)	<0.001	5 (4–5)	3.5 (2–4)	<0.001
Q12	My collaboration on team applications was effective.	4 (4–5)	4 (3–4)	0.018c	4 (4–5)	3.5 (2–4)	<0.001	4 (4–5)	4 (3–4)	<0.001c
Q13	My team members were engaged in team discussions.	4 (4–4)	3 (2–3)	0.045	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	2 (1–3.5)	<0.001
Q14	My interaction with instructors was effective.	4 (4–5)	3 (2–4)	0.203	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q15	I was distracted during class.	2 (2–3)	4 (3–4)	0.008	2 (2–3)	4 (3–5)	<0.001	2 (2–3)	4 (3–5)	<0.001
Q16	I learned better.	4 (3–5)	3 (2–4)	0.005	5 (4–5)	3 (2–4)	<0.001	5 (4–5)	2 (1–3)	<0.001
	Learning outcomes achieved with FTF TBL vs. virtual TBL
Q17	I easily remembered what I learned.	4 (3–4)	3 (2–4)	0.073	4 (4–5)	3 (2–4)	<0.001	4 (3–5)	3 (2–3)	<0.001
Q18	The tRAT discussions allowed me to improve understanding of concepts.	4 (4–5)	4 (3–4)	0.003c	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q19	It was easy to apply knowledge to solve application exercises.	4 (3–4)	3 (2–4)	0.002	4 (3–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–3.5)	<0.001
Q20	It helped develop my teamwork skills.	4 (4–5)	3 (2–4)	0.005	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q21	I felt prepared for exams.	4 (3–4)	3 (2–4)	0.035	4 (3–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
	Satisfaction with FTF TBL vs. virtual TBL
Q22	My overall course workload was manageable.	4 (3–4)	4 (2–4)	0.365	4 (3–5)	3 (2–4)	0.018	4 (3–4.5)	3 (2–4)	0.007
Q23	It was easy to communicate with my instructors inside the classroom.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q24	It was easy to communicate with my instructors outside the classroom.	4 (4–5)	4 (2–4)	0.007c	4 (3–4.5)	3 (2–4)	0.046	4 (4–5)	4 (3–4)	0.004c
Q25	I struggled with time management while working on class activities.	3 (2–4)	3 (2–4)	0.438	3 (2–4)	3 (2–4)	0.251	2 (2–3)	4 (2.5–4)	0.004
Q26	The use of technology was a facilitator to my learning.	4 (3–5)	4 (4–5)	0.473	4 (3–5)	4 (3–5)	0.309	4 (3–4)	4 (3–5)	0.743
Q27	Overall, I am satisfied with the learning experience.	4 (4–4)	4 (2–4)	0.053	4 (3.5–5)	3 (2–4)	0.001	4 (4–5)	2 (1.5–4)	<0.001

FTF = face-to-face; TBL = team-based learning; tRAT = team readiness assurance test.

^aStudent perceptions of survey items were measured on a five-point Likert scale: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree.

^aEach item was presented to study participants twice, once for FTF TBL and a second time for virtual TBL to compare their perceptions with the two learning experiences.

^bP values were calculated based on Wilcoxon matched pairs signed-rank exact test comparing students’ ratings of FTF TBL vs. virtual TBL for each survey item.

^bBold indicates p-values below the significance threshold.

^cIndicates higher rank sum for FTF TBL when the medians are equal.

^dIndicates higher rank sum for virtual TBL when the medians are equal.

Table 5. Comparison of pharmacy students’ attitudes and perceptions of FTF TBL vs. virtual TBL stratified by age (N = 117).

		18–24 years old			25–34 years old			≥35 years old
		FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b
	N (%)	50 (42.7)			57 (48.7)			10 (8.5)
No.	Survey item^a
	Attitudes toward learning with FTF TBL vs. virtual TBL
Q1	I felt prepared for learning.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	4 (3–4)	0.002c	4 (4–5)	2.5 (2–4)	0.031
Q2	My team members were prepared for learning.	4 (3–4)	3 (2–4)	<0.001	4 (3–5)	2 (2–4)	<0.001	4 (3–4)	2.5 (1–3)	0.063
Q3	Courses required self-discipline.	4 (4–5)	4.5 (4–5)	0.699	4 (4–5)	5 (4–5)	0.002	4.5 (4–5)	5 (4–5)	0.5
Q4	I was motivated to learn.	5 (4–5)	2.5 (2–4)	<0.001	4 (4–5)	4 (2–4)	<0.001c	5 (4–5)	4 (3–5)	0.031
Q5	I was accountable for my individual learning.	4 (4–5)	4 (4–5)	0.04c	4 (4–5)	5 (4–5)	0.486	4.5 (4–5)	5 (4–5)	1.0
Q6	I was accountable for my team’s learning.	4 (3–5)	3 (2–4)	<0.001	4 (3–5)	4 (2–4)	0.03c	4 (4–5)	3.5 (2–5)	0.125
Q7	I felt supported by my team.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001	4 (4–4)	2 (1–4)	0.031
Q8	I felt stressed when participating in class.	2.5 (2–3)	4 (2–4)	0.003	3 (3–4)	4 (2–4)	0.247	3 (2–4)	4 (4–5)	0.094
	Class experience with FTF TBL vs. virtual TBL
Q9	I felt comfortable asking questions.	4 (3–4)	4 (2–4)	0.046c	4 (3–4)	4 (3–4)	0.754	4 (3–5)	3 (2–5)	0.438
Q10	I felt comfortable expressing my opinion.	4 (3–4)	4 (2–4)	0.032c	4 (3–5)	4 (3–4)	0.829	4 (3–5)	3.5 (2–5)	0.527
Q11	I was engaged in team discussions.	4.5 (4–5)	4 (2–4)	<0.001	4 (4–5)	4 (3–4)	<0.001c	5 (4–5)	4 (3–4)	0.117
Q12	My collaboration on team applications was effective.	4 (4–5)	4 (3–4)	<0.001c	4 (4–5)	4 (3–4)	<0.001c	4.5 (4–5)	3 (2–4)	0.047
Q13	My team members were engaged in team discussions.	4 (4–5)	3 (2–4)	<0.001	4 (3–5)	3 (2–4)	<0.001	4 (4–4)	2 (1–3)	0.004
Q14	My interaction with instructors was effective.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001	4.5 (4–5)	3 (2–4)	0.031
Q15	I was distracted during class.	2 (2–3)	4 (3–5)	<0.001	2 (2–3)	4 (3–4)	<0.001	2 (1–3)	4 (4–5)	0.039
Q16	I learned better.	4.5 (4–5)	2 (1–3)	<0.001	4 (3–5)	3 (2–4)	<0.001	5 (4–5)	1.5 (1–3)	0.016
	Learning outcomes achieved with FTF TBL vs. virtual TBL
Q17	I easily remembered what I learned.	4 (4–5)	3 (2–3)	<0.001	4 (3–5)	3 (2–4)	<0.001	4 (3–4)	3 (2–3)	0.098
Q18	The tRAT discussions allowed me to improve understanding of concepts.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	2.5 (1–5)	0.016
Q19	It was easy to apply knowledge to solve application exercises.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001	4 (4–4)	2 (2–3)	0.008
Q20	It helped develop my teamwork skills.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001	5 (4–5)	2.5 (2–5)	0.031
Q21	I felt prepared for exams.	4 (4–5)	3 (2–4)	<0.001	4 (3–5)	3 (2–4)	<0.001	4 (4–4)	3.5 (2–4)	0.086
	Satisfaction with FTF TBL vs. virtual TBL
Q22	My overall course workload was manageable.	4 (3–5)	3 (2–4)	<0.001	4 (3–4)	4 (3–4)	0.184	4 (3–4)	3 (3–4)	0.469
Q23	It was easy to communicate with my instructors inside the classroom.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	2 (2–4)	0.031
Q24	It was easy to communicate with my instructors outside the classroom.	4 (4–5)	3.5 (2–4)	0.002	4 (3–5)	3 (3–4)	0.02	4 (4–5)	4 (2–4)	0.094
Q25	I struggled with time management while working on class activities.	2 (2–3)	3 (2–4)	0.011	3 (2–4)	3 (2–4)	0.084	3 (2–4)	4 (1–4)	1.0
Q26	The use of technology was a facilitator to my learning.	4 (3–4)	4 (3–5)	0.254	4 (3–5)	4 (4–5)	0.108	4 (3–5)	3.5 (2–5)	0.188
Q27	Overall, I am satisfied with the learning experience.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	0.004	4 (4–5)	1.5 (1–3)	0.004

FTF = face-to-face; TBL = team-based learning; tRAT = team readiness assurance test.

^aStudent perceptions of survey items were measured on a five-point Likert scale: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree.

^aEach item was presented to study participants twice, once for FTF TBL and a second time for virtual TBL to compare their perceptions with the two learning experiences.

^bP values were calculated based on Wilcoxon matched pairs signed-rank exact test comparing students’ ratings of FTF TBL vs. virtual TBL for each survey item.

^bBold indicates p-values below the significance threshold.

^cIndicates higher rank sum for FTF TBL when the medians are equal.

^dIndicates higher rank sum for virtual TBL when the medians are equal.

Table 6. Comparison of pharmacy students’ attitudes and perceptions of FTF TBL vs. virtual TBL stratified by sex (N = 116).

		Male			Female
		FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b
	N (%)	35 (30.2)			81 (69.8)
No.	Survey item^a
	Attitudes toward learning with FTF TBL vs. virtual TBL
Q1	I felt prepared for learning.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q2	My team members were prepared for learning.	4 (3–4)	2 (2–3)	<0.001	4 (3–4)	3 (2–4)	<0.001
Q3	Courses required self-discipline.	4 (4–5)	5 (4–5)	0.06	4 (4–5)	5 (4–5)	0.139
Q4	I was motivated to learn.	5 (4–5)	3 (2–4)	<0.001	5 (4–5)	3 (2–4)	<0.001
Q5	I was accountable for my individual learning.	4 (4–5)	4 (4–5)	0.619	4 (4–5)	4 (4–5)	0.518
Q6	I was accountable for my team’s learning.	4 (3–5)	4 (2–4)	0.091	4 (3–5)	3 (2–4)	<0.001
Q7	I felt supported by my team.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q8	I felt stressed when participating in class.	3 (2–4)	3 (2–4)	0.099	3 (2–4)	4 (2–4)	0.003
	Class experience with FTF TBL vs. virtual TBL
Q9	I felt comfortable asking questions. FTF	4 (4–4)	4 (3–4)	0.095	4 (3–5)	4 (3–5)	0.201
Q10	I felt comfortable expressing my opinion.	4 (3–5)	3 (3–4)	0.014	4 (3–4)	4 (3–5)	0.545
Q11	I was engaged in team discussions.	5 (4–5)	4 (2–4)	<0.001	4 (4–5)	4 (3–4)	<0.001c
Q12	My collaboration on team applications was effective.	4 (4–5)	4 (2–4)	<0.001c	4 (4–5)	4 (3–4)	<0.001c
Q13	My team members were engaged in team discussions.	4 (4–5)	2 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q14	My interaction with instructors was effective.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q15	I was distracted during class.	2 (2–3)	4 (3–5)	<0.001	2 (2–3)	4 (3–5)	<0.001
Q16	I learned better.	5 (4–5)	2 (1–4)	<0.001	4 (4–5)	3 (1–3)	<0.001
	Learning outcomes achieved with FTF TBL vs. virtual TBL
Q17	I easily remembered what I learned.	4 (3–5)	3 (2–4)	<0.001	4 (3–5)	3 (2–4)	<0.001
Q18	The tRAT discussions allowed me to improve understanding of concepts.	4 (4–5)	3 (2–3)	<0.001	4 (4–5)	4 (2–4)	<0.001c
Q19	It was easy to apply knowledge to solve application exercises.	4 (4–4)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q20	It helped develop my teamwork skills.	5 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q21	I felt prepared for exams.	4 (4–4)	3 (2–4)	0.002	4 (4–5)	3 (2–4)	<0.001
	Satisfaction with FTF TBL vs. virtual TBL
Q22	My overall course workload was manageable.	4 (3–4)	4 (3–4)	0.248	4 (3–5)	3 (2–4)	<0.001
Q23	It was easy to communicate with my instructors inside the classroom.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q24	It was easy to communicate with my instructors outside the classroom.	4 (3–4)	4 (3–4)	0.323	4 (4–5)	3 (2–4)	<0.001
Q25	I struggled with time management while working on class activities.	3 (2–3)	3 (2–4)	0.168	3 (2–4)	3 (2–4)	0.013d
Q26	The use of technology was a facilitator to my learning.	4 (3–4)	4 (4–5)	0.205	4 (3–5)	4 (3–5)	0.456
Q27	Overall, I am satisfied with the learning experience.	4 (3–5)	3 (2–4)	0.012	4 (4–5)	3 (2–4)	<0.001

FTF = face-to-face; TBL = team-based learning; tRAT = team readiness assurance test.

^aStudent perceptions of survey items were measured on a five-point Likert scale: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree.

^aEach item was presented to study participants twice, once for FTF TBL and a second time for virtual TBL to compare their perceptions with the two learning experiences.

^bP values were calculated based on Wilcoxon matched pairs signed-rank exact test comparing students’ ratings of FTF TBL vs. virtual TBL for each survey item.

^cIndicates higher rank sum for FTF TBL when the medians are equal.

^dIndicates higher rank sum for virtual TBL when the medians are equal.

Table 7. Comparison of pharmacy students’ attitudes and perceptions of FTF TBL vs. virtual TBL stratified by race (N = 114).

		White or Caucasian			Black or African American			Other minorities^e
		FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b
	N (%)	47 (41.2)			32 (28.1)			35 (30.7%)
No.	Survey item^a
	Attitudes toward learning with FTF TBL vs. virtual TBL
Q1	I felt prepared for learning.	4 (4–5)	3 (2–4)	<0.001	4 (3–4)	3 (3–4)	0.4	4 (4–5)	3 (2–4)	<0.001
Q2	My team members were prepared for learning.	4 (3–4)	2 (1–3)	<0.001	4 (3–4)	3 (2–4)	0.024	4 (3–5)	3 (2–4)	<0.001
Q3	Courses required self-discipline.	4 (4–5)	5 (4–5)	0.907	4 (4–5)	5 (4–5)	0.017	4 (4–5)	5 (4–5)	0.076
Q4	I was motivated to learn.	5 (4–5)	3 (2–4)	<0.001	4 (3–5)	3 (3–4)	0.02	5 (4–5)	3 (2–4)	<0.001
Q5	I was accountable for my individual learning.	4 (4–5)	5 (4–5)	0.547	4 (4–5)	5 (4–5)	0.521	5 (4–5)	4 (4–5)	0.068
Q6	I was accountable for my team’s learning.	4 (3–5)	3 (2–4)	0.002	4 (3–4.5)	4 (3–4)	0.205	4 (4–5)	4 (3–5)	0.003c
Q7	I felt supported by my team.	4 (4–5)	2 (1–4)	<0.001	4 (4–5)	3.5 (2.5–4)	0.03	5 (4–5)	3 (2–4)	<0.001
Q8	I felt stressed when participating in class.	3 (2–4)	4 (2–5)	0.06	3 (2–4)	3 (2–4)	0.37	3 (2–4)	4 (3–4)	<0.001
	Class experience with FTF TBL vs. virtual TBL
Q9	I felt comfortable asking questions.	4 (4–4)	4 (2–4)	0.038	4 (3–4.5)	4 (3–5)	0.906	4 (4–4)	4 (3–4)	0.198
Q10	I felt comfortable expressing my opinion.	4 (3–4)	4 (2–4)	0.076	4 (3–4)	4 (3–4.5)	0.492	4 (4–5)	4 (3–4)	0.008c
Q11	I was engaged in team discussions.	5 (4–5)	3 (2–4)	<0.001	4 (4–5)	4 (4–5)	0.481	5 (4–5)	4 (3–4)	<0.001
Q12	My collaboration on team applications was effective.	4 (4–5)	4 (2–4)	<0.001c	4 (4–5)	4 (3–5)	0.042c	4 (4–5)	3 (3–4)	<0.001
Q13	My team members were engaged in team discussions.	4 (4–4)	2 (1–3)	<0.001	4 (3–5)	3 (3–4)	0.009	4 (4–5)	3 (2–4)	<0.001
Q14	My interaction with instructors was effective.	4 (4–5)	3 (2–4)	<0.001	4 (3.5–5)	3 (2–4)	0.002	4 (4–5)	3 (2–4)	<0.001
Q15	I was distracted during class.	2 (2–3)	4 (3–5)	<0.001	2.5 (2–4)	3 (2.5–4)	0.048	2 (2–3)	4 (3–4)	<0.001
Q16	I learned better.	5 (4–5)	2 (1–3)	<0.001	4 (3–5)	3 (3–4)	0.205	5 (4–5)	2 (2–3)	<0.001
	Learning outcomes achieved with FTF TBL vs. virtual TBL
Q17	I easily remembered what I learned.	4 (3–5)	2 (2–3)	<0.001	4 (3–4.5)	3 (3–4)	0.252	4 (4–5)	3 (2–4)	<0.001
Q18	The tRAT discussions allowed me to improve understanding of concepts.	4 (4–5)	2 (1–4)	<0.001	4 (4–5)	4 (3–5)	0.023c	4 (4–5)	3 (2–4)	<0.001
Q19	It was easy to apply knowledge to solve application exercises.	4 (4–5)	3 (2–4)	<0.001	4 (3–5)	3 (2.5–4)	0.015	4 (4–5)	3 (2–4)	<0.001
Q20	It helped develop my teamwork skills.	4 (4–5)	3 (1–4)	<0.001	4 (4–5)	4 (2–4)	0.041c	4 (4–5)	3 (2–4)	<0.001
Q21	I felt prepared for exams.	4 (4–5)	3 (2–4)	<0.001	4 (3–5)	3 (3–4)	0.148	4 (4–5)	3 (2–4)	<0.001
	Satisfaction with FTF TBL vs. virtual TBL
Q22	My overall course workload was manageable.	4 (3–4)	3 (2–4)	0.032	4 (3–5)	4 (2.5–4)	0.321	4 (3–5)	3 (2–4)	0.001
Q23	It was easy to communicate with my instructors inside the classroom.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	0.001	4 (4–5)	3 (2–4)	<0.001
Q24	It was easy to communicate with my instructors outside the classroom.	4 (3–5)	4 (3–4)	0.006c	4 (3–5)	4 (3–4)	0.135	4 (4–5)	3 (2–4)	0.002
Q25	I struggled with time management while working on class activities.	2 (2–3)	3 (2–4)	0.251	3 (2–4)	3 (2–4)	0.228	2 (2–4)	4 (3–5)	0.001
Q26	The use of technology was a facilitator to my learning.	4 (3–4)	4 (3–5)	0.931	4 (3–5)	4 (4–5)	0.023d	4 (3–5)	4 (3–5)	0.922
Q27	Overall, I am satisfied with the learning experience.	4 (4–5)	2 (1–4)	<0.001	4 (3–5)	3.5 (2–4.5)	0.327	4 (4–5)	3 (2–4)	<0.001

FTF = face-to-face; TBL = team-based learning; tRAT = team readiness assurance test.

^aStudent perceptions of survey items were measured on a five-point Likert scale: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree.

^aEach item was presented to study participants twice, once for FTF TBL and a second time for virtual TBL to compare their perceptions with the two learning experiences.

^bP values were calculated based on Wilcoxon matched pairs signed-rank exact test comparing students’ ratings of FTF TBL vs. virtual TBL for each survey item.

^bBold indicates p-values below the significance threshold.

^cIndicates higher rank sum for FTF TBL when the medians are equal.

^dIndicates higher rank sum for virtual TBL when the medians are equal.

^eOther minorities included students who selected Asian/Asian American, American Indian/Alaska Native, or ‘Other’ race.

Table 8. Comparison of pharmacy students’ attitudes and perceptions of FTF TBL vs. virtual TBL stratified by highest level of education completed (N = 117).

		High school			Associate degree			Bachelor’s degree or higher
		FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b
	N (%)	27 (23.1)			36 (30.8)			54 (46.2)
No.	Survey item^a
	Attitudes toward learning with FTF TBL vs. virtual TBL
Q1	I felt prepared for learning.	4 (3–5)	4 (2–4)	0.058	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q2	My team members were prepared for learning.	4 (3–4)	3 (2–4)	0.018	4 (3–4)	3 (2–4)	<0.001	4 (3–4)	2 (2–3)	<0.001
Q3	Courses required self-discipline.	5 (4–5)	5 (4–5)	1.0	4 (4–5)	5 (4–5)	0.342	4 (4–5)	5 (4–5)	0.024
Q4	I was motivated to learn.	5 (4–5)	3 (2–4)	0.001	4 (4–5)	3 (2–4)	<0.001	5 (4–5)	3 (2–4)	<0.001
Q5	I was accountable for my individual learning.	5 (4–5)	4 (4–5)	0.027	4 (4–5)	4 (4–5)	0.768	4 (4–5)	5 (4–5)	0.884
Q6	I was accountable for my team’s learning.	4 (4–5)	3 (2–5)	0.003	4 (3.5–4)	3 (2.5–4)	0.003	4 (3–5)	4 (2–4)	0.049c
Q7	I felt supported by my team.	4 (4–5)	3 (2–4)	0.001	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q8	I felt stressed when participating in class.	3 (2–4)	4 (2–4)	0.248	3 (2–4)	4 (3–5)	0.002	3 (2–4)	4 (2–4)	0.112
	Class experience with FTF TBL vs. virtual TBL
Q9	I felt comfortable asking questions. FTF	4 (3–4)	4 (3–4)	0.833	4 (3.5–5)	4 (3–5)	0.174	4 (4–4)	4 (2–4)	0.095
Q10	I felt comfortable expressing my opinion.	4 (3–4)	4 (3–4)	0.812	4 (4–5)	4 (3–5)	0.203	4 (3–4)	3 (3–4)	0.115
Q11	I was engaged in team discussions.	4 (4–5)	4 (2–4)	0.007c	4.5 (4–5)	4 (3–4)	<0.001	5 (4–5)	4 (2–4)	<0.001
Q12	My collaboration on team applications was effective.	4 (4–5)	4 (3–5)	0.017c	4 (4–5)	4 (2–4)	<0.001c	4 (4–5)	4 (2–4)	<0.001c
Q13	My team members were engaged in team discussions.	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	2.5 (2–4)	<0.001	4 (4–5)	3 (2–3)	<0.001
Q14	My interaction with instructors was effective.	4 (4–5)	3 (3–4)	0.002	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q15	I was distracted during class.	2 (2–3)	3 (3–4)	0.002	2 (2–3)	4 (3–5)	<0.001	2 (2–3)	4 (3–5)	<0.001
Q16	I learned better.	4 (3–5)	3 (2–3)	0.002	5 (4–5)	2 (1–3)	<0.001	5 (4–5)	3 (1–3)	<0.001
	Learning outcomes achieved with FTF TBL vs. virtual TBL
Q17	I easily remembered what I learned.	4 (3–5)	3 (2–4)	0.016	4 (4–5)	3 (2–4)	<0.001	4 (3–5)	3 (2–4)	<0.001
Q18	The tRAT discussions allowed me to improve understanding of concepts.	4 (3–4)	4 (2–4)	0.003c	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q19	It was easy to apply knowledge to solve application exercises.	4 (3–4)	3 (2–4)	0.004	4 (4–5)	3 (2–4)	<0.001	4 (3–5)	3 (2–4)	<0.001
Q20	It helped develop my teamwork skills.	4 (4–5)	3 (2–4)	0.004	4 (4–5)	2 (2–4)	<0.001	5 (4–5)	3 (2–4)	<0.001
Q21	I felt prepared for exams.	4 (3–5)	3 (3–4)	0.011	4 (4–4)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
	Satisfaction with FTF TBL vs. virtual TBL
Q22	My overall course workload was manageable.	4 (3–5)	4 (3–4)	0.486	4 (3–4)	3.5 (2–4)	0.041	4 (3–5)	3 (2–4)	0.002
Q23	It was easy to communicate with my instructors inside the classroom.	4 (4–5)	4 (3–4)	0.011c	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001
Q24	It was easy to communicate with my instructors outside the classroom.	4 (4–5)	4 (3–4)	0.056	4 (3–4.5)	3 (2–4)	0.079	4 (4–5)	3 (2–4)	<0.001
Q25	I struggled with time management while working on class activities.	3 (2–4)	3 (2–4)	0.51	2.5 (2–4)	4 (2–5)	0.116	3 (2–3)	3 (2–4)	0.015d
Q26	The use of technology was a facilitator to my learning.	4 (3–5)	4 (3–5)	0.281	4 (3–5)	4 (3–5)	0.383	4 (3–4)	4 (3–5)	0.626
Q27	Overall, I am satisfied with the learning experience.	4 (3–5)	3 (2–4)	0.014	4 (4–5)	3 (2–4)	<0.001	4 (4–5)	3 (2–4)	<0.001

FTF = face-to-face; TBL = team-based learning; tRAT = team readiness assurance test.

^aStudent perceptions of survey items were measured on a five-point Likert scale: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree.

^aEach item was presented to study participants twice, once for FTF TBL and a second time for virtual TBL to compare their perceptions with the two learning experiences.

^bP values were calculated based on Wilcoxon matched pairs signed-rank exact test comparing students’ ratings of FTF TBL vs. virtual TBL for each survey item.

^bBold indicates p-values below the significance threshold.

^cIndicates higher rank sum for FTF TBL when the medians are equal.

^dIndicates higher rank sum for virtual TBL when the medians are equal.

Table 9. Comparison of pharmacy students’ attitudes and perceptions of FTF TBL vs. virtual TBL stratified by previous virtual learning experience (N = 117).

		None			One course			Two or more courses
		FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b	FTF TBL Median (IQR)	Virtual TBL Median (IQR)	P value^b
	N (%)	14 (12)			12 (10.3)			91 (77.8)
No.	Survey item^a
	Attitudes toward learning with FTF TBL vs. virtual TBL
Q1	I felt prepared for learning.	4 (3–4)	3 (2–4)	0.137	5 (4–5)	3.5 (2.5–4)	0.065	4 (4–5)	3 (2–4)	<0.001
Q2	My team members were prepared for learning.	4 (4–4)	2.5 (2–4)	0.023	4 (3.5–5)	3 (2–4)	0.078	4 (3–4)	2 (2–4)	<0.001
Q3	Courses required self-discipline.	4 (4–5)	5 (4–5)	0.453	4 (4–5)	5 (4–5)	0.438	4 (4–5)	5 (4–5)	0.051
Q4	I was motivated to learn.	4 (4–5)	3 (2–4)	0.018	5 (4–5)	3 (1.5–4)	0.01	5 (4–5)	3 (2–4)	<0.001
Q5	I was accountable for my individual learning.	4 (4–5)	4.5 (4–5)	0.859	4 (4–5)	4 (4–4.5)	0.375	4 (4–5)	5 (4–5)	0.651
Q6	I was accountable for my team’s learning.	4 (3–4)	4 (3–4)	0.188	4 (3.5–5)	3.5 (2.5–4)	0.164	4 (3–5)	3 (2–4)	<0.001
Q7	I felt supported by my team.	4 (4–5)	2.5 (2–4)	0.008	4.5 (4–5)	2 (2–4)	0.002	4 (4–5)	3 (2–4)	<0.001
Q8	I felt stressed when participating in class.	3 (2–4)	3.5 (3–4)	0.31	3 (2–4)	4 (3.5–4.5)	0.109	3 (2–4)	4 (2–4)	0.007
	Class experience with FTF TBL vs. virtual TBL
Q9	I felt comfortable asking questions. FTF	4 (3–4)	4 (2–4)	0.125	4 (3–4)	4 (3–4)	0.918	4 (3–5)	4 (3–4)	0.076
Q10	I felt comfortable expressing my opinion.	4 (3–4)	3 (3–4)	0.313	4 (2.5–4)	4 (3–4)	0.453	4 (3–5)	4 (3–5)	0.039c
Q11	I was engaged in team discussions.	4.5 (4–5)	4 (3–5)	0.063	5 (4–5)	4 (2.5–4.5)	0.078	4 (4–5)	4 (3–4)	<0.001c
Q12	My collaboration on team applications was effective.	4 (4–5)	4 (3–4)	0.25	4 (4–5)	3.5 (2.5–4)	0.031	4 (4–5)	4 (2–4)	<0.001c
Q13	My team members were engaged in team discussions.	4 (4–4)	3 (1–4)	0.004	4 (4–5)	3 (2–4)	0.02	4 (4–5)	3 (2–4)	<0.001
Q14	My interaction with instructors was effective.	4 (4–5)	3 (2–4)	0.023	4 (4–4.5)	3 (2–4)	0.051	4 (4–5)	3 (2–4)	<0.001
Q15	I was distracted during class.	2 (2–3)	4 (3–4)	0.051	2 (1.5–3)	4 (3–5)	0.014	2 (2–3)	4 (3–5)	<0.001
Q16	I learned better.	4 (3–5)	3 (2–4)	0.008	5 (3.5–5)	2 (1.5–3)	0.016	4 (4–5)	2 (1–3)	<0.001
	Learning outcomes achieved with FTF TBL vs. virtual TBL
Q17	I easily remembered what I learned.	4 (3–4)	3 (2–4)	0.016	4 (3.5–4)	3 (2–3.5)	0.141	4 (3–5)	3 (2–4)	<0.001
Q18	The tRAT discussions allowed me to improve understanding of concepts.	4 (3–5)	3 (2–4)	0.016	4 (4–4.5)	3 (2–4)	0.016	4 (4–5)	3 (2–4)	<0.001
Q19	It was easy to apply knowledge to solve application exercises.	3 (3–4)	3 (2–4)	0.172	4 (3.5–4)	3 (2–4)	0.105	4 (4–5)	3 (2–4)	<0.001
Q20	It helped develop my teamwork skills.	4.5 (4–5)	3 (2–4)	0.004	4.5 (4–5)	2.5 (2–4)	0.028	4 (4–5)	3 (2–4)	<0.001
Q21	I felt prepared for exams.	4 (3–4)	3 (3–4)	0.125	4 (3.5–5)	2 (2–4)	0.049	4 (4–5)	3 (2–4)	<0.001
	Satisfaction with FTF TBL vs. virtual TBL
Q22	My overall course workload was manageable.	4 (3–4)	4 (3–4)	1.0	4 (3–4.5)	3.5 (2–4.5)	0.219	4 (3–5)	3 (2–4)	<0.001
Q23	It was easy to communicate with my instructors inside the classroom.	4 (4–5)	2.5 (2–4)	0.065	4 (3.5–4.5)	3 (3–4)	0.242	4 (4–5)	3 (2–4)	<0.001
Q24	It was easy to communicate with my instructors outside the classroom.	4 (3–5)	4 (2–4)	0.375	4 (3–4)	3.5 (2–4)	0.352	4 (3–5)	3 (3–4)	<0.001
Q25	I struggled with time management while working on class activities.	3 (2–4)	3 (3–4)	0.602	2.5 (2–4)	4 (3–4.5)	0.078	3 (2–4)	3 (2–4)	0.02d
Q26	The use of technology was a facilitator to my learning.	4 (3–5)	4.5 (4–5)	0.797	4 (3–4.5)	4 (3–5)	0.75	4 (3–5)	4 (3–5)	0.228
Q27	Overall, I am satisfied with the learning experience.	4 (4–4)	4 (2–5)	0.295	4 (4–5)	2 (2–4)	0.018	4 (3–5)	3 (2–4)	<0.001

FTF = face-to-face; TBL = team-based learning; tRAT = team readiness assurance test.

^aStudent perceptions of survey items were measured on a five-point Likert scale: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree.

^aEach item was presented to study participants twice, once for FTF TBL and a second time for virtual TBL to compare their perceptions with the two learning experiences.

^bP values were calculated based on Wilcoxon matched pairs signed-rank exact test comparing students’ ratings of FTF TBL vs. virtual TBL for each survey item.

^bBold indicates p-values below the significance threshold.

^cIndicates higher rank sum for FTF TBL when the medians are equal.

^dIndicates higher rank sum for virtual TBL when the medians are equal.

Table 10. Students’ perceptions of virtual TBL strengths (N = 55).

Theme	Representative Quotes	No. (%)
Allows students the flexibility to learn at their own pace	‘Virtual TBL allowed students to prepare earlier for classes and create a schedule for studying. From the student perspective we were able to attack preclass material faster so when virtual class started we dived straight into TBL applications versus face to face where it seemed more of a crank-up method.’ ‘Virtual TBL offers the time flexibility that otherwise we may not have and allows us to structure our own individual schedules in order to manage whatever we have going on or move around study time as needed.’ ‘I was able to use down time to work on whatever I needed to work on … I felt it was a much more efficient use of my time.’ ‘With virtual learning, I have better time management and I feel more productive.’	23 (42)
Saves time and money from not commuting to class	‘Virtual TBL gave me much more time. It saved me a trip to school and time to get ready. I think having some courses virtual is a great idea’ ‘It makes pharmacy school more affordable by saving money on parking, commuting back and forth to school, and student housing.’ ‘The only strength I can think of is time. I feel as virtual classes gave you more time to study as it minimizes the time of getting dressed up and having to drive to school – especially for those who commute.’ ‘Virtual learning allowed me MORE TIME to study because I did not have to spend so much time driving back and forth. I was able to gain more rest and energy with virtual learning in order to prepare and study for classes.’	16 (29)
Convenience to learn from locations other than the classroom	‘I liked the convenience and ability to work with my team in a more private environment during virtual TBL’ ‘Virtual TBL allowed for more students to be present in class even when they had things going on compared to FTF TBL.’ ‘Being in the comfort of your own space of your choosing was very beneficial, no outside factors contributing to your study time.’ ‘It is much more convenient to be able to stay home and attend class virtual.’	12 (22)
Learning how to collaborate with teams remotely	‘It allows us to be flexible and be more technological savvy, especially if we would ever need to have long-distance conferences in our careers.’ ‘When given the chance to work with a completely virtual group of individuals, conversations flowed, and interactions could be maintained. As well, we could share google docs and seamlessly dictate and make changes to applications.’	9 (16)
More comfortable communicating using technology	‘Flexibility. I felt more comfortable unmuting myself and asking a question or commenting because it doesn’t feel like I am speaking in front of 75+ people. I asked far more questions that I would in person.’ ‘Individuals who aren’t comfortable with public speaking were able to engage more and answer questions or discuss by typing in Zoom chat or use audio’ ‘It was nice to be able to talk with the instructor in a private chat, it can cause anxiety in some students to have to always ask a question in front of everyone.’	8 (15)
Less distraction from other teams in class	‘I did not have the distractions such as classmates talking during lecture to distract me. I was able to fully concentrate on lecture’ ‘I felt like we were able to have more private discussions without interference from other teams.’	6 (11)
Encourages students to practice self-discipline	‘Self-discipline development skills are integral in being successful in virtual-TBL. It is easier to be on your phone during an virtual course, but dedicated students will remain on task.’	2 (4)

TBL = team-based learning; FTF = face-to-face.

Table 11. Students’ perceptions of virtual TBL weaknesses (N = 60).

Theme	Representative Quotes	No. (%)
Lack of accountability or engagement from team members	‘Having to work in teams through Zoom and not face to face made it harder to keep the whole team involved and actively participating on applications/group assignments.’ ‘Virtual TBL is not an effective method for students to learn. It is unrealistic to expect that all students in an virtual setting will make meaningful contributions to the team applications. Typically, what happens is that only a few team members would participate in the team applications.’ ‘It allowed students to remain silent through the duration of the class and not be held accountable for their contribution. As you could see, generally only one person per group was comfortable even talking in class, which could have been mediated through an in person setting.’ ‘Lack of participation from others. Holding a group accountable for assignments and tasks is difficult when you are not sitting next to each other.’ ‘With virtual learning some students may not have prepared appropriately. Additionally, there was less accountability since we couldn’t see them in person.’	34 (57)
Technical challenges or difficulty using technology	‘The only weakness that I can think of with virtual TBL is that sometimes technology was not working. It would just set us back on time dedicated to the lecture.’ ‘It was a bit of a struggle when it came to finding the best way to be on a video call together. We also had technical issues that would occur fairly often.’ ‘Technology problems also made virtual learning an issue (professor not sure how to set things up/team members not able to use technology). It made it more difficult to truly learn and make the most of class time.’ ‘Technology issues wasted valuable class time.’	12 (20)
More distractions during class time	‘Being out of the classroom setting makes it easier to get distracted by things going on in the home.’ ‘ … it is very easy to get distracted or walk away from the computer.’ ‘No matter how hard I tried to find a quiet place to learn, there were always distractions.’	9 (15)
More self-discipline required to remain focused during class	‘Self-discipline was a must in staying on top of courses.’ ‘Virtual TBL may have been too convenient and required high levels of self-discipline.’ ‘I have to put more time and effort into my own learning.’	8 (13)
Decreased interpersonal communication between team members	“ … lack of human connection. In the classroom, we are able to see our classmates’ body language and we can just freely speak out. Virtual learning requires unmuting yourself and the spotlight would be entirely on you. There’s also uncertainty with reactions that go on behind the screen. I understand that for people with public speaking issues, it requires an extra step, making it harder to gain experience and confidence as face-to-face would give.” ‘There was zero team chemistry during virtual semesters because off class work topic chatting was impossible. I didn’t feel connected to my team, and I didn’t even bother learning their names because I never needed to like in face-to-face.’ ‘Virtual TBL doesn’t strengthen communication skills or work ethic for people that already do not like to talk or contribute to team activities.’	8 (13)
Difficulty communicating with instructors	‘Many professors do not seem to care to put in the effort to learn the technology and be able to check our messages. Often, we are answering them in the form of messages or raised hands (so as to prevent interrupting them), but they do not care to check messages until the end or say can’t figure out how to leave the messages tab open. This ultimately limits the ability of the virtual student to participate fully in the course, even though I never felt I could not simply speak up if I needed to have a point reiterated.’ ‘There was a lack of communication on some tasks that needed to be completed, unanswered questions, and when we were able to ask questions via Zoom or email professors acted as if it was our fault that we did not know…or that we were a bother… a lot of miscommunications.’	7 (12)
Increased students’ workload due to perceived increase in students’ free time by professors	‘For some reason, professors took virtual courses = more coursework and assignments since they were under the impression we had more free time.’ ‘It seemed like professors would give more work to students when we were virtual which took up more of our time.’	6 (10)
Instructors were less prepared/passionate during class	‘I felt like the professors as a whole were less invested in the process during virtual learning.’ ‘Poor team interactions and professor enthusiasm were some of the greatest weaknesses of virtual TBL. Professors read straight from PowerPoints which brought the momentum of learning down. We are in a graduate program, we can read a PowerPoint by ourselves. There is no need for a class time if the class is going to be led in that fashion.’	5 (8)
Students were more likely to engage in academic dishonesty	‘The security of RATs and exams is practically non-existent virtual. Being monitored via webcam is not adequate to ensure testing integrity.’ ‘High opportunities for cheating to occur during virtual TBL, despite measures taken.’	5 (8)
Students are less individually motivated/engaged in class	‘Virtual TBL sometimes felt less engaging than face-to-face’ ‘I was not as motivated to study and prepare for class as I was with face-to-face TBL.’	4 (7)
Classes were less structured	‘There was a lot of down time in some classes where we were just in the breakout rooms finished with discussions or assignments waiting to go back to the main group.’	3 (5)

TBL = team-based learning; RATs = readiness assurance tests.