Computer-assisted surgery simulations and directed practice of total knee arthroplasty: Educational benefits to the trainee

Abstract

Orthopaedic residents typically learn to perform total knee arthroplasty (TKA) through an apprenticeship-type model, which is a necessarily slow process. Surgical skills courses, using artificial bones, have been shown to improve technical and cognitive skills significantly within a couple of days. The addition of computer-assisted surgery (CAS) simulations challenges the participants to consider the same task in a different context, promoting cognitive flexibility. We designed a hands-on educational intervention for junior residents with a conventional tibiofemoral TKA station, two different tibiofemoral CAS stations, and a CAS and conventional patellar resection station, including both qualitative and quantitative analyses. Qualitatively, structured interviews before and after the course were analyzed for recurring themes. Quantitatively, subjects were evaluated on their technical skills before and after the course, and on a multiple-choice knowledge test and error detection test after the course, in comparison to senior residents who performed only the testing. Four themes emerged: confidence, awareness, deepening knowledge and changed perspectives. The residents’ attitudes to CAS changed from negative before the course to neutral or positive afterwards. The junior resident group completed 23% of tasks in the pre-course skills test and 75% of tasks on the post-test (p < 0.01), compared to 45% of tasks completed by the senior resident group. High-impact educational interventions, promoting cognitive flexibility, would benefit trainees, attending surgeons, the healthcare system and patients.

• Total knee arthroplasty
• computer-assisted surgery
• education
• simulation
• surgical skills education
• surgical training

Introduction

Total knee arthroplasty (TKA) is a challenging procedure to learn. Orthopaedic residents typically learn to perform TKA through an apprenticeship-type model, watching and executing increasingly complex parts of the procedure, with little or no opportunity to practice the required physical and cognitive skills outside the operating room (OR). This is a necessarily slow process, and the speed of acquisition of skills and cognitive understanding varies widely amongst residents. Compressing the learning curve through learning and practice outside the OR would benefit everyone: the resident, who could gain skills and understanding sooner, leading to earlier and more active participation in surgeries on live patients; the attending surgeon, who would have more valuable residents assisting and who could teach at a higher level; the healthcare system, through better use of the advisor's time, as well as potentially shorter surgery times; and the patient, by having surgeons who have maximized the use of their training time as residents, with potentially better outcomes and fewer revisions being required.

Over 500,000 TKA procedures are performed in North America every year [1]. In each case, the surgeon must integrate many competing objectives, namely proper knee alignment, soft tissue balance, central tracking of the patella within the femoral groove, stability in flexion and extension, appropriate component sizing, and optimal placement of the components, all while minimizing OR time. In addition, the surgeon and staff must routinely become familiar with new implant designs, instrumentation, surgical techniques and technologies. Surgical errors can lead to the need for revision surgery[2–5, which is more costly and has a lower success rate than primary surgery [6]. Surgical errors can also lead to postoperative pain[5–7, reduced function or range of motion [2], and premature wear of the components [2]. Furthermore, involving residents in the OR adds to the length and cost of the surgery [8], [9]. Additional training outside of the OR has the potential to reduce both of these factors.

Previous studies have demonstrated the value of a surgical skills center, where residents have the opportunity to practice the TKA procedure on artificial bones [10]. Emphasizing the cognitive aspects of the procedure further improved residents’ ability to identify common TKA errors in an error detection test [10].

Computer-assisted surgery (CAS) has been used successfully as a training tool in several studies: The use of CAS improved accuracy and shortened the learning curve for experienced surgeons performing hip resurfacing surgery [11], [12]; CAS training improved early acquisition of simple orthopaedic skills with no long-term compromise [13]; and a computer-enhanced visual model improved trainee learning for urologic procedures [14].

Many studies have demonstrated that CAS improves alignment accuracy in TKA [15], [16]. Educationally, what is most interesting is that CAS subsequently improves the accuracy of surgeons’ conventional surgery [17]. “Emerging information indicates that exposure to computer-assisted techniques for knee reconstruction… increases understanding of these procedures by both experienced and novice surgeons and improves the accuracy and reliability with which these procedures are performed using conventional, non-navigated surgical techniques” [17]. Proposed explanations for this phenomenon are that the surgeons pay more attention to specific surgical details after using CAS; perform more intraoperative measurements; have greater awareness of sources of error; learn from the computer's real-time feedback; and, in the case of trainees, gain a better understanding of the fundamental anatomical reference points and axes used.

Combining the strengths of a surgical skills center (i.e., the opportunity to practice outside the OR) with the strengths of computer-assisted surgery (i.e., the provision of real-time feedback and the opportunity to plan component placement) could improve the physical and cognitive training of orthopaedic residents. We therefore designed a course in which residents practiced the TKA procedure on artificial bones, using conventional technique and different CAS systems. The fundamental concepts underlying the course are outlined below, with specific details of the study design described in the Methods section.

Cognitive flexibility theory posits that learners understand complex content more easily through multiple representations of the same information. By repeatedly presenting complex cases in new contexts, the learner recognizes additional facets of the content [18]. In this study, the CAS systems and the manual technique each present the information differently, using different tools to accomplish the same goals. This should allow the resident to abstract their thinking by approaching the same problem in different ways, rather than simply working on technical skills through repetition of an identical situation. Gaining a more global understanding of TKA may help residents react more appropriately to new situations they encounter in the OR and help them better predict the consequences of their actions. Given that orthopaedic residents currently need to learn skills and procedures in less time due to changing work hour restrictions, it is essential to look at high-impact educational opportunities to reduce the learning curve.

There are many factors to consider when designing a surgical skills course. Although ideally each factor would be evaluated independently, with randomized groups undergoing different educational interventions, this would require a very large group of residents and evaluators, or multiple years. It has already been shown that the opportunity to practice on artificial bones rapidly improves technical skills and that allowing more opportunity for self-evaluation and instructor input improves cognitive skills as well as technical skills [10]. We propose that the next step is to combine manual practice and feedback with CAS practice plus feedback on multiple CAS systems.

The purpose of our study was to evaluate this course design by (1) investigating qualitatively how the residents’ thinking about TKA and CAS was affected by the course, determining emerging themes through structured interviews; and (2) investigating quantitatively how their technical skills, knowledge and error-detection abilities improved by the end of the course in comparison to the levels attained by senior residents. To our knowledge, qualitative analysis of an educational intervention for TKA has not been reported previously. Directly asking residents about their experiences can reveal educational benefits that have not been discovered previously through purely quantitative analyses.

Our specific research questions were as follows: (1) How does a surgical skills course that includes manual and computer-assisted surgical training affect a trainee's (simulated) operative thought process, peri-operative decision making, and self-interpretation in TKA?; (2) How does the trainee's knowledge of TKA change?; (3) Are CAS simulations a useful teaching aid for TKA?; and (4) What is the trainee's perspective regarding the clinical application of CAS before and after the course?

Methods

Subjects

The subjects comprised six junior level (PGY 1-3) orthopaedic residents. In addition, three senior level (PGY 5) orthopaedic residents performed the testing, without participating in the course, to provide a comparison group. The junior (interventional) group had significantly less TKA experience compared to the senior (comparison) group (p < 0.01) in all categories except the number of arthroplasty rotations (Table I). Notably, only one of the junior residents had performed an entire TKA, whereas all of the senior residents had done so, with a mean of 28 TKAs. None of the residents, junior or senior, had performed a TKA using CAS. Informed consent was provided to participate in the study. All participants also signed photo releases.

Table I.  Self-reported group experience for total knee arthroplasty

Number of:	Jr. resident group	Sr. resident group	P-value
Arthroplasty rotations	2.0 (1.3)	3.3 (0.6)	0.13
TKAs – observed	33.3 (16.3)	90.0 (32.8)	<0.01
TKAs – few cuts performed	11.7 (5.3)	41.7 (14.4)	<0.01
TKAs – exposure	4.7 (5.7)	45.0 (8.7)	<0.01
TKAs – all cuts	1.7 (2.9)	35.0 (17.3)	<0.01
TKAs – ligament balancing	0.5 (1.2)	28.3 (5.8)	<0.01
TKAs – whole procedure	1.2 (2.9)	28.3 (15.3)	<0.01
TKAs – with CAS	0	0	n/a
CAS – any surgery	0.3 (0.8)	7.7 (4.6)	<0.01

Educational intervention

Prior to and after the course, the junior residents were interviewed and tested, as detailed in the next two sub-sections below. The course, which was led by a fellowship-trained orthopaedic surgeon specializing in TKA (C.R.H.), ran from Friday afternoon to Sunday morning (see Figure 1 and Appendix 1). While the primary intervention was an educational and skills-related TKA course, testing played an important role in the learning process and should be included in all courses. The course design, described below, was based on an earlier course that demonstrated effective skills and cognitive teaching for TKA [10].

Figure 1. Overview of the course itinerary. The senior residents were tested as a comparison group for the quantitative scores and did not complete the course. The junior residents performed testing, observed demonstrations, and conducted TKAs at three stations. See Appendix 1 for details.

Qualitative outcomes

Standardized one-on-one interviews were conducted with each junior resident by the primary investigator (C.M.) prior to and after the course. The interview questions were trialed on four fourth-year orthopaedic residents prior to using them with the study subjects to assess their appropriateness. The interviews, which were digitally recorded for later transcription, contained questions focused around the resident's experience, thought process, and decision-making during TKA surgery (see Appendices 2 and 3). Post-course interviews were completed within 10 days of completing the course. The transcriptions were checked for accuracy and then analyzed independently by two investigators (C.A.M. and G.R.K.) using thematic analysis [19]. Any divergent findings were discussed with a third investigator (C.H.).

Quantitative outcomes

The quantitative portion of the study involved testing skills and knowledge to support the qualitative data. Pre-course testing used the Objective Structured Assessment of Technical Skills (OSATS) test, which is a previously validated skills test for TKA [10], [20]. Subjects were given 36 minutes to complete the tibiofemoral portion of the TKA procedure, including assembling the instrumentation. Each subject was marked on a grading sheet by experienced surgeons, including a task-specific checklist of items to complete as well as a global rating score (Figure 2). The residents were not given any prior instruction – their performance depended on their observations and execution of previous clinical cases. (As shown in Table I, they had observed an average of 33 cases; only one junior resident had performed a full TKA.) This testing provided a baseline for pre-/post-course comparisons and made the residents think about what they needed to learn.

Figure 2. Sample steps performed in the OSATS test for the conventional TKA procedure.

After the intervention of the educational course, post-course testing included repeating the OSATS skills test, performing an error detection test, and completing a multiple-choice knowledge test. The error detection test, which was developed to examine the cognitive abilities of the participants, included six artificial knee bone models with the prosthesis placed in an erroneous position, modeled after a previously validated error detection test [10]. The error samples included (1) incorrect posterior tibial slope; (2) internal rotation of the femoral component; (3) an anteriorly notched femur; (4) varus angulation of the tibial component; (5) asymmetric patellar resection; and (6) unbalanced flexion and extension gaps. Using visual inspection, and manipulating the implanted bone model (e.g., flexing, extending, or applying varus/valgus stress), subjects were required to report the error, state the clinical problem that it would cause, and describe how one would correct or prevent this in the OR if discovered. The error detection test was tested for validity using a fellowship-trained senior arthroplasty surgeon. The multiple-choice test included 30 questions on TKA principles, modeled after a previously developed and validated test for TKA [10]. Due to an unrecognized grammatical error, one question was discarded, making 29 questions available for analysis.

CAS systems

Two tibiofemoral (TF) CAS systems and one patellar CAS system were used in the course. The two TF CAS systems were the Orthosoft Universal Knee 2.1 software on a Sesamoid Plasty system (Zimmer CAS, Montreal, Canada)[21–23 (see Figure 3 for a sample screenshot) and the Knee Unlimited 2.1.2 software on a Kolibri system (BrainLAB, Feldkirchen, Germany) [24] (see Figure 4 for a sample screenshot). Both of these systems allow the user to plan the desired femoral and tibial component positions on a patient-specific virtual model of the knee, and then guide the user to achieve the desired cuts accurately. The patellar CAS system (Figure 5) was developed at our institution [25], [26]. With this system, the user navigates the position and orientation of the patellar sawguide compared to the desired resection line; both lines are displayed on the monitor. The desired resection line is parallel to the anterior surface of the patella, at the desired depth based on a caliper measurement of the original thickness. This system has since been developed further.

Figure 3. Sample screenshot from the Zimmer CAS system providing guidance regarding the positioning and sizing of the femoral cutting guide to achieve the desired result. (Image courtesy of Zimmer CAS.)

Figure 4. Sample screenshot from the BrainLAB CAS system showing a virtual implantation of the femoral component to determine the desired component positioning and required cuts; the dots represent digitized points on the bone surface. (Image courtesy of BrainLAB.)

Figure 5. Patellar computer-assisted surgery system developed by our group. Marker arrays were mounted on the patellar bone and the sawguide. The user then navigated the sawguide position and orientation to match the desired thickness parallel to the anterior patellar surface.

The subjects were not evaluated on their use of the CAS systems; rather, the intention was that the CAS practices, in addition to providing more practice, would augment their cognitive understanding of the procedure. Representatives from the two CAS system manufacturers were present at each of the TF CAS stations to ensure that the residents were given the best possible introduction to each CAS station, while the conventional and CAS patellar resection station was attended by one of the patellar CAS creators (C.A.). It was not part of the study design to compare the residents’ performance on the two CAS systems.

Course design

The day after testing, the subjects participated, as a group, in a 75-minute didactic lecture and demonstration by the senior surgeon running the course, followed by demonstrations at each of the TF CAS stations. The conventional and CAS demonstrations explained the steps and equipment, as well as the principles involved. This was followed by hands-on practice using artificial bone models (Sawbones, Pacific Research, Vashon, WA), with pairs of subjects rotating between stations (see Figure 6 and Appendix 1). At Station 1 the pairs were split between two stations (a and b) to allow 45 minutes at each (conventional TKA with the senior surgeon for Station 1a; patellar conventional and CAS for Station 1b). At Stations 2 and 3 (the CAS stations) the subjects each had 45 minutes of hands-on practice and 45 minutes of observing and assisting. In all cases, the same implant design and instrumentation were used (NexGen LPS-Flex, Zimmer, Warsaw, IN). Although subjects experienced three practices of the same tibiofemoral TKA task, each practice was at a different station in a different context – one as conventional surgery and two using CAS systems. The subjects also observed and assisted for two CAS practices. The TKA task consisted of making the appropriate tibial and femoral cuts on the artificial bone models and placing the trial components. In the conventional case, extramedullary instrumentation was used for the tibia and intramedullary instrumentation was used for the femur. In the CAS cases, extramedullary cutting blocks were navigated with the CAS system, which was used to pre-plan the desired size and position of the implants (Figures 3 and 4). Since the residents were in pairs at the tibiofemoral CAS stations, one performed the TKA procedure while the other assisted and observed, then they switched roles so that each resident performed a TKA with each CAS system and observed a TKA with each CAS system.

Figure 6. Course design with different stations/contexts to develop cognitive flexibility. Stations 1a and 1b were done individually (45 min each); Stations 2 and 3 were done in pairs (45 min each as primary user).

Subjects performed at least three patellar resections: two conventional (using a sawguide and their choice of freehand or a reamer), plus one navigated procedure. The patellar task consisted of measuring the patellar thickness with calipers to determine the desired resection thickness, and then resecting the patella, with the goal of achieving the desired thickness while remaining parallel to the anterior surface.

Only the junior-level residents participated in the course. The senior-level residents did not participate, but instead served as a comparison group for the skills, knowledge and error-detection testing. All participants completed a questionnaire on their experience relating to TKA.

Post-course evaluation and feedback

Structured feedback forms were provided to the subjects immediately after the course, with questions related to the testing, the CAS stations and conventional stations, and the overall experience with the course (see Table II). There were five options for each, ranging from “strongly disagree” to “neutral” to “strongly agree”.

Table II.  Average responses from structured post-course evaluations

Question	Average response
My experience on the pre-test helped my performance on the post-test.	****
The lectures and demonstrations helped my performance on the tests.	****
The hands-on experience helped my performance on the post-test.	*****
The CAS stations helped me to understand TKA concepts better.	****
The CAS stations were more helpful than the non-CAS station.	**
The CAS stations increased my understanding of alignment principles in TKA.	****
I would have learned just as much about alignment from the lectures and the demos without the CAS stations.	**
I am more comfortable in deciding the alignment and amount of tibia to be cut.	****
I am more comfortable in deciding the alignment and amount of femur to be cut.	*****
I am more comfortable in deciding how to create a balanced knee.	****
Overall the testing was a negative experience.	*
I felt intimidated on the pre-test.	****
I felt intimidated on the post-test.	**
The testing made me more aware of what I need to learn.	*****
I feel as if I have wasted a whole weekend.	*
I am glad I came to this course this weekend.	****

*Strongly disagree; ^**Disagree; ^***Neutral; ^****Agree; ^*****Strongly agree.

Statistical analysis

A statistician was consulted for statistical analysis of the quantitative data. Primary analysis of the pre-/post-course test results was done by parametric analysis using paired t-testing, with significance set at p < 0.05. Analysis of the comparison of the junior and senior groups used unpaired t-testing. Non-parametric analysis was also done to confirm the results of the parametric analysis, using Wilcoxon signed-rank testing (for pre- and post-course score data) and Mann-Whitney testing (for group comparisons). Statistical analysis was performed using SPSS software (version 17.0).

Results

Qualitative results

Thematic analysis of the interview data revealed four consistently recurring themes from the participants: confidence, awareness, deepening knowledge, and changed perspectives. The two investigators agreed upon these themes after independent analysis. Each of these themes will be presented with quotations from the interviews with the junior residents (see Appendices 2 and 3 for the questions posed).

1. Confidence. The participants consistently displayed and reflected on how their confidence had changed after the intervention. This confidence was seen in all aspects of the procedure:

“I’d say that I’m more confident, because of the course, because I’m more familiar with everything. I also think it's given me an idea of what I need to look at to be confident.” [In response to post-course Q2: “Has your confidence when performing or assisting a total knee arthroplasty changed after the course?”]

“I feel I know what's happening next, which I thought maybe I did know before, but realized that I didn’t, so now I feel more comfortable with that, as well as knowing the actual mechanics of the total knee. I feel more confident with that.” [In response to post-course Q12 “How has your confidence changed in total knee arthroplasty after the course?”]

2. Awareness. Participants had a change in awareness of their own skill and judgment after the intervention in all aspects of the TKA procedure. Feedback from the computer-assisted simulations was particularly valuable in this regard:

“For the CAS, the most valuable part was just learning how precise it could be and learning how “off” my eye actually was, so when I looked at some of the cuts that I thought they should be at, and compared it to what the computer actually said. I think that it was valuable to tell me how accurate or inaccurate my own vision was at times.” [In response to post-course Q15 “What is the most important thing you learned at each hands-on station?”]

"I have a better appreciation now of the potential downfalls or complications that can be incurred if you do not get things right in terms of the setup and your cuts, and these can be compounded and your end result can be affected." [In response to post-course Q19 “How will your approach differ the next time you are in the OR for a total knee arthroplasty procedure?”]

3. Higher and deeper knowledge levels. Subjects reported higher levels of learning after the intervention, according to the revised Bloom's taxonomy [27], which classifies learning into six levels from low- to increasingly higher-level learning – remembering, understanding, applying, analyzing, evaluating and creating – along four deepening dimensions of knowledge, i.e., factual, conceptual, procedural and meta-cognitive. Reflecting on how each cut made during the TKA task affects the alignment, rotation, soft tissue balance, etc., is an example of the highest and deepest type of learning.

While the participants’ self-reported knowledge of core principles surrounding TKA did not change, i.e., the principles remained the same, the participants were consistently able to expand upon these principles and apply this knowledge to the goals and aims of the procedure. They were also able to focus on and rely on these principles during the pressure of the skills testing. Prior to the course, their answers were hesitant and undetailed; after the course, the answers were longer, more detailed, and more confident. The following participant comments show how they were challenged into thinking through the procedure, in contrast to previous teaching where they just performed part of the procedure without having to think it through:

“I think [the course] made me much more aware of what each step entails because I think certain steps have been explained to me in the past, but I think when you are doing an elective case, it goes so smoothly and so quickly that you don’t usually realize why they are doing certain things unless it's directly explained to you.” [In response to post-course Q2 “Now that you have taken the course, has your confidence changed when you think about yourself going into either performing or assisting a total knee arthroplasty?”]

“When everything is handed to you in the OR by someone who's already put them together, knows the next step, it's really a decerebrate operation: you’re not doing anything; you’re just cutting. So now, having to think through it, put these things together, it's a lot better.” [In response to post-course Q12 “How has your knowledge or confidence changed in total knee arthroplasty after the course?”]

Deepening knowledge through cognitive flexibility was an intentional design goal for the course. During the course, the residents clearly maintained the primary focus of learning TKA across the different stations despite the changing contexts; learning was not fragmented into learning conventional TKA at Station 1, learning the Zimmer CAS system at Station 2, learning the BrainLAB system at Station 3, and so on. Based on the self-reported learning in the interviews, all participants gained a basic familiarity with three different ways of approaching TKA, including experiencing the similarities and differences between navigated and non-navigated systems, and starting to think about what they liked and disliked in the navigated systems. The subjects went well beyond learning a rote sequence of TKA steps to understanding how each step affects others, some of the implications of the decisions and cuts made, and how the course would provide a framework for future learning on an orthopaedic surgical rotation. One student articulated the additional contribution of the CAS systems with the folllowing comment:

“I think it was definitely valuable with the computer-assisted help because it gives you an idea how subtle the movements in your hands can be to create big changes in angular cuts.” [In response to post-course Q13 “What was the most valuable part of the course?” and follow-up “Would you say the hands-on part was more valuable just on its own or with the computer assisted help?”]

As further support, and to conclude this theme of deepening knowledge, another participant compared the learning experience of the educational intervention versus learning while on a junior clinical rotation:

“I think both [hands-on and demos] were helpful… Over the course of this weekend, I retained more about total knees than I did after three months of doing them on my first year rotation and I mean it's obviously a different setting – it's a real human, it's an elective case, you’re a first year resident, but I think just getting the mechanics down and having the basic understanding for the principles behind the surgery, maybe not why you’re doing it, but the principles behind it, I think they were solidified very well in that three days”. [In response to post-course Q7 “If you were to compare the demos vs. the hands-on simulation, what would you say was more helpful in triggering your thoughts or principles of total knee arthroplasty?”]

4. Changed perspectives. In the pre-course interview, the participants uniformly had a negative view of computer-assisted surgery for TKA. There was a lack of trust in the computer, suspecting that it could produce unwanted errors compared to doing it on your own. After the intervention, all the participants had moved to a neutral or positive attitude towards using CAS in their own practices. Some even went on to say that the main use of CAS would be for difficult cases, severe deformity, or revision cases. This demonstrates their change in trust and perceived value of using computer-assisted techniques. The following participant comments lend support to these findings:

“Prior to the course, I don’t think I would [use CAS]. After the course, I would if it was available and I was trained on it, I would definitely take advantage of it.” [In response to post-course Q10 “Do you think you would use computer assisted surgery in your own practice and why or why not?”]

“I think for complex deformity cases, I definitely would [use CAS] because I think it helps restore the alignment with precision”. [In response to post-course Q10 “Do you think that you would use computer assisted surgery in your own practice and why or why not?”]

Some of the residents had misconceptions of what “computer-assisted surgery” was beforehand (e.g., thinking that the preoperative plan would be fixed), and generally expected it to be complex. Afterwards, they consistently commented that the systems were easy to use, and thought that the real-time feedback would be valuable while gaining experience with the procedure. Concerns were that the surgeon might lose traditional skills, or might spend too much time getting the numbers exactly right.

Post-course evaluations and feedback

Course feedback was very positive and enthusiastic (Table II). All participants were glad they came for the weekend. When asked what the most valuable part of the course was, the most common response was the pre-test experience. When asked how often they would plan to use CAS in their practice, the range was 26–50%, mostly for complex cases. Multiple participants commented on wanting more experiences like this incorporated into their training. All commented on the course being a very positive educational experience.

Features of the CAS systems that the residents considered favorable (Q22) were having a simple, intuitive interface without a lot of extra features or steps; that it was easy for the system to detect the marker arrays (i.e., the limited line-of-sight problems); the motion-detecting interaction (e.g., the system registering a movement of the hand tool as input to the system, without the need to press a button); and being able to put the cutting block on in an approximate orientation and then adjust it into place with dials. Advantages were seen both to working with a particular implant manufacturer and to working with multiple manufacturers. Overall, they recognized that the CAS systems went through similar steps and that the main differences related to the user interface. The residents found both systems easy to use.

Quantitative results

The junior residents significantly increased their scores on the OSATS skills test (p < 0.001) and their mean global rating scores (p < 0.01) from before the course (23% and 1.8, respectively) to after the course (75% and 3.7, respectively) (Figures 7 and 8). They even surpassed the scores of the senior residents, on average (45% and 2.7), although the junior-senior differences were not significant with the numbers available (p = 0.07 to 0.21). The junior residents scored less than the senior residents on the multiple-choice knowledge test (55 ± 8% vs. 68 ± 5%, p < 0.05). While the average results on the error detection test were also less, the difference was not statistically significant (54 ± 15% vs. 70 ± 4%, p = 0.13). Non-parametric statistical analyses confirmed these results.

Figure 7. Objective Structured Assessment of Technical Skills (OSATS) task-specific checklist scores (mean ± standard deviation) for the junior residents before and after the course, compared to the senior residents, who did not participate in the course. The junior residents significantly increased their scores (p < 0.001).

Figure 8. Objective Structured Assessment of Technical Skills (OSATS) global rating scores (mean ± standard deviation) for the junior residents before and after the course, compared to the senior residents, who did not participate in the course. The junior residents significantly increased their scores (p < 0.01).

Discussion

Previous literature has demonstrated that skills simulation and cognitive assessment have a positive effect on a trainee's learning for total knee arthroplasty [10]. Our aim was to mix skills simulation (directed practice) with computer-assisted simulation and determine the qualitative and quantitative effects on the learner. The results showed a clear impact on residents’ thinking about TKA and CAS, and a change from 23% to 75% on task-specific skills after our short intervention. The quantitative results compare favorably with a previous skills course for junior residents in which they had 5-6 practices or 3-4 with expert feedback [10]. By contrast, the junior residents in our study had only one directed practice of conventional TKA (which is what they were evaluated on), with the second and third practices being done as computer-assisted simulations without expert feedback. Since the participants in the two studies started and ended at similar levels, we believe the feedback from the computer-assisted simulation had a positive effect on skills learning for TKA. A course such as this could help offset any negative effects of current work hour restrictions by adding value to the time spent by residents in the OR, enhancing their cognitive and manual skills, and subsequently allowing them to be involved at a higher level during the surgery.

One advantage of the CAS course design is that only one surgeon was required for the course itself (six surgeons were required for the pre- and post-course assessments). If a surgeon had been dedicated to each station, the amount of direct feedback would have increased; however, it is difficult for any surgeon to devote an entire weekend to the course. Instead, representatives of CAS companies provided guidance at the CAS stations. Knowledgeable users (e.g., fellows, residents or biomedical engineers familiar with the systems and with TKA) could have run the systems if they had been available, but the company representatives found it informative to see how the residents experienced the systems, especially while performing real cuts on the artificial bones rather than working with prepared models, and the representatives probably helped create the positive impressions reported by the participants. The CAS systems need to be either on site already, or brought to the course by the companies. Both companies expressed interest in participating in future courses.

The qualitative analysis brought out positive themes in relation to the participants’ learning that were reinforced by the quantitative results. Having the participants learn the TKA skill from multiple viewpoints (conventional TKA, BrainLAB CAS, Zimmer CAS, conventional and CAS patellar resections) created an environment for developing cognitive flexibility and self-assessment. This challenged the participants further, which appeared to increase the learning effect with minimal actual time and practice.

Despite the above-mentioned value of adding CAS to the course, we recommend splitting the course in two. Given the time challenge of combining the manual and CAS approaches with pre- and post-course testing in a single weekend, there was only one manual practice with expert feedback. It may be more appropriate to provide a purely manual skills course (including cognitive skills) early in the residency, followed later by a CAS course once the residents’ skills and knowledge are greater and they can take fuller advantage of the CAS learning; this will reinforce the earlier learning as well. The only reason for not doing more repetitions was to limit the duration of the course so that the participants still had an afternoon left of their weekend. There is little doubt that additional repetitions, especially with the conventional technique, would have further improved their confidence and familiarity with both the equipment and the principles of TKA, thereby further improving their OSATS scores. If this course were integrated into resident training, more repetitions are advised.

The main limitation of our study was the small number of residents participating due to on-call duties; this decreased the power in comparing the junior and senior residents, although we were able to detect significant changes in the case of the junior residents. For the qualitative analysis, six subjects were sufficient to detect recurring themes; more subjects would likely not have changed the thematic analysis. Another limitation is that we lacked a control group. We can conclude that the course was a success, but we cannot confirm which aspects of the course led to this success. However, we do feel that previous research gives us a historical control for comparison [10]. Furthermore, the senior residents acted as a comparison group, which helped overcome the lack of a control group. Our results showed that the junior residents were able to surpass the senior residents on the skills testing, despite having less than half the clinical experience. This noteworthy result suggests that this course design was an effective intervention for teaching TKA skills, and might improve future learning on the valuable surgical rotations. A final limitation is that our study only included immediate post-course testing. It would be useful to know what the longer-term learning effects are, and how this type of intervention affects clinical skills in the OR. It is difficult to evaluate long-term effects reliably due to the ongoing clinical training; however, our long-term goal is to compare residents before and after the introduction of surgical skills courses into the curriculum. We expect that the course design will evolve over time, but it is clear that any course is better than no course, and we therefore recommend that surgical skills courses be offered at all institutions.

A further improvement in the course design would be to include the error detection test before the course as well as after: doing so should not bias the post-course result since the participants would not know the answers, but it would key them into thinking in global, cognitive terms, and considering the consequences and recovery methods for common errors in TKA. One resident identified the error detection as one of the most valuable parts of the course. The OSATS skills test is an integral part of the overall educational intervention as it taught the residents right from the beginning what they did not know, providing considerable motivation for the course itself and teaching them what they needed to focus on during the course.

In conclusion, participants in our study benefited from increased confidence, changed awareness, and deepening knowledge concerning total knee arthroplasty, as well as changed perspectives regarding the advantages and use of computer navigation. With a brief intervention of skills and computer simulation, orthopaedic trainees’ skills for knee arthroplasty were significantly improved, with considerable evidence that such a course would enhance the subsequent intraoperative experience. Based on our intervention, we recommend a course that creates different contexts for the learner, challenging the student to look at the skill from different viewpoints, and thereby maximizing the learning benefit. The participants strongly agreed that the course was worthwhile, and the senior residents wished that they had had such a course.

Acknowledgements

We are grateful to the many people who made this learning experience possible. Zimmer Canada provided the instrument sets and artificial bone models; Linvatec provided the surgical saws. Lysiane Proulx from Zimmer CAS brought and attended the Zimmer CAS station; David Bean and Casey Sieber from BrainLAB attended the BrainLAB CAS station; the BrainLAB system itself was loaned to us by Zimmer Canada. Five fellowship-trained surgeons provided their time and expertise to the pre- and post-course testing: Dr. Matthew Oliver, Dr. Robert Korley, Dr. Elizabeth Pederson, Dr. Jason Werle and Dr. Robert Chan. They were supported by Chris Dusik and a group of student volunteers. We would also like to thank Barnabas Wu and Jack Fu for preparing the patellar models, and Dr. Tak Fung for his statistical advice. Many thanks to the participating orthopaedic residents for their time and enthusiasm.

Declaration of Interest: The authors gratefully acknowledge the financial support provided by the University of Calgary Orthopaedic Residency Program Research Fund, as well as the Natural Sciences and Engineering Research Council of Canada (NSERC). None of the companies that participated in the course was involved in the course design, data analysis or manuscript preparation.

Appendix 1: Course Itinerary

(Excluding setup time, morning coffee & clean-up time)

Day 1, Friday afternoon

Participants: Senior residents, junior residents, surgeon evaluators, medical student volunteers, Zimmer representatives, investigators.

Note: In a future course, the senior residents could be tested on Friday or Sunday.

Group 1 (senior residents):

1430–1530 Orientation and pre-course forms

1530–1615 OSATS Skills test (36 mins for test, 9 min to change)

1615–1700 Error detection test (6 stations: 6 min each with 1 min change)

15-min break

1715–1745 Multiple-choice written knowledge test

Group 2 (junior residents):

1515–1615 Orientation and pre-course forms

1615–1700 OSATS Skills test

Group 3 (junior residents):

1600–1700 Orientation and pre-course forms

1700–1745 OSATS Skills test

All:

1700–1815 Food and drinks for study participants and evaluators

Day 2, Saturday

Participants: Junior residents, Zimmer CAS representative, BrainLAB representatives, investigators

0830–0945 Lecture and demonstration of TKA (C.R.H.)

0945–1030 Zimmer CAS demonstration

15-min break; patellar CAS training guide distributed

1045–1130 BrainLAB demonstration

1130–1200 Lunch

1200–1330 Rotating skills stations in pairs:

 Station 1: CAS TKA Zimmer CAS (with representative)

 Station 2: CAS TKA BrainLAB (with representatives)

 Station 3: 45 min Patellar CAS and non CAS (with creators)

 45 min TKA hands-on with no CAS (with experienced surgeon)

1330–1345 15-min break

1345–1515  Rotating skills stations in pairs:

 Station 1: CAS TKA BrainLAB (with representatives)

 Station 2: 45 min Patellar CAS and non CAS (with creators)

 45 min TKA hands-on with no CAS (with experienced surgeon)

 Station 3: CAS TKA Zimmer CAS (with representative)

Day 3, Sunday

Participants: Junior residents, Zimmer CAS and BrainLAB representatives, surgeon evaluators, investigators

0800–0930 Rotating skills stations in pairs

 Station 1: 45 min Patellar CAS and non CAS (with creators)

 45 in TKA hands-on with no CAS (with experienced surgeon)

 Station 2: CAS TKA Zimmer CAS (with representative)

 Station 3: CAS TKA BrainLAB (with representatives)

0930–1000 Practice and questions

1000–1100 Multiple-choice written knowledge test

1100–1230 Post testing - OSATS Skills test, error detection test

1100–1145 Skills test for Group 2; error detection test for Group 3

1145–1230 Error detection test for Group 2; skills test for Group 3

1230–1300 Lunch break

1300–1330 Evaluation forms

1330 Course finish

Appendix 2: Pre-course interview questions

• Q1: Describe your experience in total knee arthroplasty.

• Q2: Do you do any preoperative planning when doing a total knee arthroplasty and, if so, what are your priorities?

• Q3: What affects your confidence when performing (or assisting) a total knee arthroplasty?

• Q4: What do you think about (or what are important considerations) during the preparation of the femoral cuts in a total knee arthroplasty?

• Q5: What do you think about (or what are important things to consider) during the preparation of the tibial cuts in a total knee arthroplasty?

• Q6: How do you decide whether or whether not to resurface the patella?

• Q7: What do you think about during the preparation of the patella if resurfacing?

• Q8: What are things that you would consider as principles of total knee arthroplasty? In other words, the operative goal for a total knee arthroplasty?

The interviewer informed the resident that the next 3 questions focused on computer-assisted surgery in total knee arthroplasty. They were instructed to answer based on how they envisioned it working, even if they had not seen or worked with it before.

• Q9: What advantages do you see of computer-assisted surgery?

• Q10: What disadvantages do you see of computer-assisted surgery?

• Q11: Do you think that you would use computer-assisted surgery with total knee arthroplasty in your own practice? Why or why not?

The interviewer did not use exactly the same wording in each case, but the context of the question remained unchanged.

Appendix 3: Post-course interview questions

• Q1: Do you do any preoperative planning when doing a total knee arthroplasty and, if so, what are your priorities? Has this changed after the arthroplasty course?

• Q2: What affects your confidence when performing (or assisting) a total knee arthroplasty? Is this different from before the course?

• Q3: What do you think about (or what are important considerations) during the preparation of the femoral cuts in a total knee arthroplasty?

• Q4: What do you think about during the preparation of the tibial cuts in a total knee arthroplasty?

• Q5: How do you decide whether or whether not to resurface the patella?

• Q6: What do you think about during the preparation of the patella if resurfacing?

• Q7: What are things that you would consider as principles of total knee arthroplasty? In other words, the operative goal for a total knee arthroplasty? Has this changed because of the course? Were the demos or the hands-on simulations more helpful in triggering your thoughts or principles of total knee arthroplasty?

• Q8: What advantages do you see of computer-assisted surgery?

• Q9: What disadvantages do you see of computer-assisted surgery?

• Q10: Do you think that you would use computer-assisted surgery with total knee arthroplasty in your own practice? Why or why not?

• Q11: After having exposure to CAS patella, has this changed the way you would perform your patellar resurfacing? If so, how?

• Q12: How has your knowledge or confidence changed in total knee arthroplasty after the course?

• Q13: What was the most valuable part of the course for you?

• Q14: What was the least valuable part of the course?

• Q15: What was the most important thing you learned at each demo station? That is when Dr. Hutchison did her demo and then when they did the demos with the two CAS systems.

• Q16: What was the most important thing that you learned with each of the hands-on stations, when you actually got to do the hands-on for each of the stations?

• Q17: What things got easier to understand or do as you moved around the hands-on stations?

• Q18: How would you compare this weekend course with ways in which you have been learning orthopaedic surgery so far?

• Q19: What do you still need to learn or what in particular will you be looking for the next time you scrub in for a total knee arthroplasty procedure?

• Q20: How will your approach differ the next time you are in the OR for a total knee arthroplasty procedure?

• Q21: Any comments on the sequence of the hands-on rotations – that is, would you have preferred to start somewhere else or did it work out for you?

• Q22: What features did you like or dislike about each of the CAS systems?

• Q23: Any comments on learning total knee arthroplasty on three different systems – both positive and negative?

• Q24: Any comments on the times allotted, especially for the hands-on practice?

• Q25: Any comments or other feedback about the course?