Using the Jigsaw Method to Improve Resident Confidence and Medical Knowledge About Osteoporosis in Men

ABSTRACT

The jigsaw method which utilises peer-teaching in small-group settings permits a facilitator to incorporate active learning strategies into their educational toolbox. In November 2022, we evaluated the impact of the jigsaw method with groups stratified by post-graduate year (PGY) level. We assessed pre and post learner confidence and medical knowledge during a facilitator-led workshop. Participants were stratified into three groups who presented on the following subtopics about osteoporosis in men: (1) history and physical examination (PGY1s), (2) lab work and imaging studies (PGY2s) and (3) management and interventions (PGY3s). We evaluated if stratifying by PGY level impacted learner medical knowledge and confidence in November 2022. We had an 80% response rate from 16 actual participants from a total of 20 possible participants. Authors found a statistically significant increases in residents’ confidence and medical knowledge after the training for all questions, but broad application across a variety of learners is still needed. The jigsaw method showed improvement in learner confidence about osteoporosis in men and may be implemented in teams distributed by PGY levels. Distribution by PGY level allows each team to building on their current training level to educate their peers.

KEYWORDS:

• Active learning
• intermediate effect
• jigsaw method
• resident education

Introduction

The jigsaw method is an active learning strategy which uses peer-teaching in small-group settings that may help faculty to incorporate active learning strategies into their teaching toolbox. Previous research shows that the jigsaw method is effective in improving learner education when applied to mixed-learning groups on unfamiliar topics [1–4]. Active learning strategies have shown to improve learning among participants but are not widely used among medical educators [5]. Some of the perceived barriers to implementation of active learning strategies include lack of familiarity and perceived increased time for preparation [5].

Our project objective evaluated the influence of the jigsaw method for knowledge acquisition on and confidence in diagnosing and treating osteoporosis in men among learning teams of family medicine residents at the University of Texas Medical Branch (UTMB). Learning teams were grouped based on level of post-graduate training (PGY) as knowledge is generally gained as one progresses further in training [6]. The intermediate effect demonstrates that learning is not typically linear as one progresses from novice to expert [7]. Rather there is a downward slump that learners experiences as they journey through learning [7]. This is the first project to evaluate the impact of stratifying content expert groups based on post graduate training level and if it demonstrates the intermediate effect. Our hypothesis is that improvements seen by utilising the jigsaw method of learning will be sustained when splitting learning groups by PGY levels.

Main Text

The UTMB Institutional Review Board (IRB) reviewed the project and determined that it did not meet the definition of human subjects research defined by the regulations at 45 CFR 46.102 as this project involves Quality Assessment/Quality Improvement. Therefore, the project did not require IRB approval or oversight.

The lead author evaluated the jigsaw method during the dedicated didactics time, in November 2022. The lead author served as the facilitator of the two-hour workshop on osteoporosis in men. The goal of the training was to increase resident physician knowledge about osteoporosis in men, given that fewer men are impacted by osteoporosis, they have a higher mortality rate [8]. The objectives of the workshop were to (1) determine the information needed to evaluate a patient with osteoporosis; (2) illustrate the differences between normal bone mineral density, osteopenia and osteoporosis; and (3) develop a treatment plan for a man with osteoporosis. Residents completed a pre-learning assessment at the beginning of the training to assess their previous medical knowledge about and self-reported confidence in their ability to diagnose and treat osteoporosis in men. Previously published curricula informed assessment questions [9]. Resident demographics were also collected. After completion of the initial assessment, the facilitator then divided the residents into three groups: PGY1s, PGY2s and PGY3s. Each group then researched and presented on the following subtopics about osteoporosis in men: (1) history and physical examination (PGY1s), (2) lab work and imaging studies (PGY2s) and (3) management and interventions (PGY3s). Groups prepared brief presentations on their assigned subtopics for 25 minutes. The facilitator actively addressed questions during the given development time.

At the end of groundwork session, each PGY team presented to the large group. After each team’s mini-presentation, the facilitator then gave a comprehensive overview of the topic to ensure completeness of the training. At the end of the training, residents completed a post-learning assessment identical to the pre-learning assessment. A quasi-experimental, pre-test-post-test design was used to evaluate changes in confidence and knowledge among 20 residents. Medical knowledge was assessed separately from confidence to decrease the impact of perceived confidence on medical knowledge. We also assessed post-workshop confidence to see if there was an impact on confidence from having participated in the training. The assessment asked residents five confidence questions ranking from 1 (not confident) to 5 (very confident) and seven knowledge-related questions. Similar to a clinical trial, we collected pre- and post-training outcomes from the same individual. Scores of knowledge questions were averaged for each resident. Descriptive statistics were presented for each variable. Wilcoxon rank sum tests were conducted to examine whether variations in confidence before and after the training were statistically significant and paired sample t-test was used for the continuous variable (knowledge score). A 5% significance level was used for all tests and analyses were performed using STATA v16 (College Station, TX: StataCorp LLC).

Results

Sixteen family medicine residents (response rate = 80%) completed both pre- and post-assessments, and a power analysis showed that the project achieved power = 0.94 with an alpha of 0.05 and effect size = 0.319. There were 20 possible participants in attendance at the time of the lecture. Table 1 shows self-reported demographics of the participants and distribution by PGY group.

Table 1. Demographic distribution of residents participating in both pre- and post-surveys.

Total (N = 16)
Age (mean, std.)	29.9 (.62)
Female (n, %)	6 (37.5)
Year (n, %)
PGY1	4 (25.0)
PGY2	7 (43.8)
PGY3	5 (31.2)

Table 2 illustrates the average score before and after training for each PGY level. PGY1 was found to have the lowest level of confidence before training (avg = 2.50–3.50) and PGY3 has the highest level of confidence (avg = 3.40–4.20). However, PGY1 was found to have the best performance in knowledge before the training than PGY3 (3.25 > 3.00) and PGY2 has the most significant increase in knowledge from 2.86 to 6.14. We compared the changes in scores (improvement in knowledge) between each two levels of residents (p _PGY1-PGY2 = 0.800; p _PGY1-PGY3 = 0.165; p _PGY2-PGY3 = 0.163) and across three groups (p = 0.773), but no statistically significant difference was found.

Table 2. Comparing average scores by PGY level.

	PGY1 (n = 4)		PGY2 (n = 7)		PGY3 (n = 5)
Question	Pre Mean (SD)	Post Mean (SD)	Pre Mean (SD)	Post Mean (SD)	Pre Mean (SD)	Post Mean (SD)
C1	3.50 (0.29)	4.33 (0.33)	3.71 (0.18)	4.29 (0.29)	4.20 (0.02)	4.33 (0.21)
C2	3.00 (0.00)	3.37 (0.33)	3.14 (0.40)	4.29 (0.29)	4.00 (0.32)	4.33 (0.21)
C3	3.00 (0.00)	4.00 (0.00)	3.57 (0.30)	4.29 (0.29)	3.40 (0.24)	4.17 (0.31)
C4	2.75 (0.25)	4.00 (0.00)	2.86 (0.51)	4.00 (0.38)	3.40 (0.40)	4.17 (0.31)
C5	2.50 (0.50)	4.00 (0.00)	2.86 (0.51)	4.29 (0.36)	4.00 (0.32)	4.33 (0.21)
Knowledge	3.25 (0.48)	5.00 (0.58)	2.86 (0.55)	6.14 (0.34)	3.00 (0.32)	5.67 (0.49)

We found statistically significant increases in residents’ confidence and knowledge after the training for all questions (see Table 3). Among the five confidence questions, Question 5, “I am confident in my ability to refer patients appropriately for a bone health specialty evaluation”, had the largest increase from 3.13 to 4.25. Question 1, “I am confident in diagnosing osteoporosis”, achieved the next highest increase in score. The fourth question regarding their confidence in treating their own patients for osteoporosis had the lowest score (post = 4.06).

Table 3. Comparing average scores in five confidence questions and one set of seven knowledge questions.

Question	Pre Mean (SD)	Post Mean (SD)	Difference	p-Value
C1	3.81 (0.14)	4.31 (0.15)	0.50	0.0078
C2	3.38 (0.22)	4.19 (0.16)	0.81	0.0020
C3	3.38 (0.15)	4.19 (0.16)	0.81	0.0010
C4	3.00 (0.26)	4.06 (0.19)	1.06	0.0005
C5	3.13 (0.30)	4.25 (0.17)	1.13	0.0010
Knowledge	3.00 (0.27)	5.75 (0.27)	2.75	<0.001

When comparing knowledge to confidence we found no statistically significant correlation. We compared pre-class knowledge to pre-class confidence and obtained a correlation coefficient of 0.37 (SD0.42), p-value of 0.386. We also compared post-class knowledge to post-class confidence and obtained a correlation coefficient of 0.10 (SD 0.46), p-value of 0.823. Interestingly, when we compared post-class knowledge with pre-class confidence there was a very slight negative correlation − .01(SD 0.08), p-value of 0.879. Although female trainees reported lower scores than male trainees, the improvements in confidence and knowledge before and after the training were not varied by gender (p > 0.05).

Figure 1 visualises the results of Table 3. Before the training, the average scores in five confidence questions were below 3. In all cases, mean scores improved on the post-test to 4.06 and more.

Figure 1. Average scores and 95% confidence interval for each question before and after training.

Discussion

The jigsaw method showed gains in increasing learner confidence about osteoporosis in men and can be implemented in teams distributed by PGY levels. The distribution by PGY levels, which helps the learners build on their training, adds a way in which to divide groups and tailor more to where they may be during their learning. Our project demonstrates that stratifying by PGY level shows a linear progression of learning in contrast to the intermediate effect which typically shows a drop followed by a gain as learners progress in their knowledge acquisition. Since we collected pre- and post-training outcomes from the same individuals, we can confidently attribute the improvement to the intervention. Although the literature does show that women tend to rate themselves lower in self-assessments, our study did not reflect such a difference [10]. These findings are important as medical education is moving away from traditional lecture techniques in favour of active learning techniques [11]. Weaknesses of the project include that it is a single institution project, and that the jigsaw method was only evaluated during one training on one topic. We were not able to assess application in practice for workshop participants. We were also not able to assess if knowledge gains were sustained among participants as only three participants completed the 6-month follow-up assessment (response rate = 19%). Next steps include evaluating the effectiveness of this teaching method by other faculty as well as other topics and institutions. The jigsaw method showed gains in increasing learner confidence about osteoporosis in men and can be implemented in teams distributed by PGY levels. This method allows each PGY level to build on their current training level to educate their peers [6].

Author Contributions

All authors have contributed to the reported work. NB contributed to the conception, project design, acquisition of data, analysis and interpretation, drafting and revision of the manuscript. WL and KMC contributed to the data analysis, interpretation, and drafting and revision of the manuscript.

Ethics Approval

The protocol was approved by the University of Texas Medical Branch Institutional Review Board (FWA#00002729)

Acknowledgments

The authors would like to acknowledge Christen Walcher for editing and formatting the manuscript.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Data Availability Statement

The data that support the findings of this project are available from the corresponding author, NB, upon reasonable request.