The influence of computer navigation on trainee learning in hip resurfacing arthroplasty

Abstract

Computer navigation in arthroplasty surgery is a form of concurrent augmented feedback. Motor learning theory suggests such feedback may be detrimental to learning as a result of the learner either developing a dependence on the additional feedback or being distracted from using intrinsic feedback. To determine whether computer navigation influences the learning curve of novices performing hip resurfacing arthroplasty, a systematic review and critical appraisal of the current English-language literature on the topic was conducted. There is some evidence that use of navigation by trainees facilitates more accurate placement of arthroplasty components as compared to conventional instrumentation. However, there is no evidence that training with computer navigation impairs performance in retention or transfer tests. Thus, although the published literature has significant limitations, there is no evidence that supports concerns regarding the impact of computer navigation on the learning curve of arthroplasty trainees.

• Arthroplasty
• surgical training
• orthopaedic surgery
• navigation
• computer assisted surgery

Introduction

Computer navigation in arthroplasty surgery aims to assist the surgeon in the accurate positioning of components. It has been stated that even in the hands of experienced surgeons, less than 50% of acetabular components placed in a conventional freehand fashion or with alignment guides are positioned within the safe zone of 40° ± 10° of abduction and 15° ± 10° of anteversion [1]. This is clearly an important issue, as malpositioning of components is associated with a number of complications, including dislocation, increased wear, groin pain, femoral neck fracture (resurfacing only) and femoro-acetabular impingement [2–4]. Although there is good evidence that navigation reproducibly improves the accuracy of implant positioning [5–7], there have been few studies evaluating its use by trainees.

Learning a new surgical technique involves learning new motor skills. The pathway to acquisition of these skills is known as motor learning, and has been described as a series of internal processes that are associated with practice or experience leading to relatively permanent changes in the capability for movement [8]. Individuals learn to master new motor skills by evaluating available feedback to alter future performance. Feedback may be either intrinsic (as a natural consequence of the action) or extrinsic (from an external source such as an instructor). One variant of extrinsic feedback is continuous concurrent augmented feedback. Schmidt and Wulf defined this as supplementary information presented to the learner throughout the performance of a task [9]. An example of this type of feedback is the visual information provided by computer navigation during arthroplasty surgery. This is a potentially powerful tool for learning because it guides the learner to the correct response, minimizes errors, and reinforces appropriate behavior. However, motor learning theory suggests that this type of feedback does not contribute to learning and may actually be detrimental because performance gains during practice are seldom carried over to retention tests (re-testing on the same task after an interval of time) or transfer tests (re-testing under different conditions, i.e., without concurrent feedback) [9]. It is therefore suggested that if the practiced skill is subsequently performed without the augmented feedback, the performance deteriorates. It has been hypothesized that this failure of concurrent feedback to contribute to retention of task performance is a result of the learner either developing a dependence on the additional feedback or being distracted from using intrinsic feedback [9].

In view of the huge number of factors that are currently having a negative impact on surgical training, it is important that an assessment of the influence of computer navigation on trainees be undertaken in parallel to its increasing popularity.

The aim of this study was to systematically review and critically appraise the literature to determine whether computer navigation influences the learning curve of novices performing hip resurfacing arthroplasty. The primary objectives were to determine whether components are more accurately placed when novices use computer navigation, and whether this increased accuracy is maintained when the procedure is subsequently performed in a conventional manner without navigation.

Literature review and critical appraisal of included studies

Study identification

Relevant studies were identified using the search strategy identified below. Relevance to the study was determined from the titles of the citations retrieved; abstracts were only reviewed in cases where relevance could not be determined from the title alone. Studies were included if the influence of navigation on trainees performing hip resurfacing arthroplasty was a major focus of the article. The search strategy was as follows:

1. PubMed search: “Arthroplasty, Replacement, Hip”[Mesh] OR “hip resurfacing” OR “resurfacing” AND “Surgery, Computer-Assisted” [Mesh] AND “clinical competence”[Mesh] OR “learning”[Mesh] OR “education”[Mesh]; Limits: English.

2. PubMed search: “navigation”[ti] OR “computer navigation” AND “learning”[ti]; Limits: English.

3. Review of references cited in relevant articles identified by the above search strategy for further eligible studies.

Using this search strategy, three unique, relevant studies were identified (search results are valid as of 18 July 2009). Data extraction was performed by recording the primary outcome measures reported (mean difference from intended cup abduction and anteversion angles, incidence of femoral notching, and range of error in varus/valgus neck-stem or femoral guide wire angles). A summary of the results is presented in Table I. Statistical analysis was not possible due to the heterogeneity between studies in terms of overall design, study populations and outcome measures recorded.

Table I.  Summary of data extracted from included studies.

	Cobb et al., 2007	Seyler et al., 2008	Gofton et al., 2007
Study design	Randomized, prospective	Comparative, retrospective	Randomized, prospective
Setting	Lab, Sawbones	In vivo	Lab, Sawbones
Navigation system	Acrobot	BrainLAB	BrainLAB
Implant navigated	Guide wire	Femur	Acetabulum
Immediate retention/transfer testing	Range of error of varus/valgus angulation:
Conventional instruments	23° ± 6° p < 0.002	No difference in range of error of varus/valgus or frequency of notching	No difference in mean acetabular abduction angle or anteversion
Navigation	7° ± 2°
Delayed retention testing	Not assessed	Not assessed	No difference in mean acetabular anteversion.
			Mean difference in acetabular abduction angle:
Conventional instruments			4.3° ± 1.7° p < 0.0001
Navigation			2.5° ± 1.7°

Critical appraisal of included studies

Studies included were assessed for quality and critically appraised with respect to the CONSORT statement; a summary of this appraisal is presented in Table II. The CONSORT statement is intended to improve the reporting of controlled trials, enabling readers to understand a trial's design, conduct, analysis and interpretation, and to assess the validity of its results. It consists of a checklist of 22 criteria that are considered essential to judging the reliability or relevance of the findings of a trial and reducing bias in estimating treatment effect [10]. Both authors of this review agreed on the checklist score for each of the included studies.

Table II.  Summary of CONSORT checklists for included studies. A full explanation of the checklist can be found at http://www.consort-statement.org/consort-statement

Consort checklist item	Gofton et al., 2007	Cobb et al., 2007	Seyler et al., 2008
1. Participant allocation	1	0	0
2. Scientific rationale	1	1	1
3. Eligibility and setting	1	1	1
4. Intervention detail	1	1	1
5. Objective/hypothesis	1	1	1
6. Outcomes	1	1	1
7. Sample size determination	1	1	0
8. Sequence generation	0	0	0
9. Allocation concealment	0	0	0
10. Implementation of randomization	0	0	0
11. Blinding	1	0	0
12. Statistical methodology	1	1	1
13. Participant flow	1	1	1
14. Dates of recruitment	0	0	1
15. Baseline demographics	0	0	1
16. Numbers analyzed	1	0	1
17. Outcomes and estimation	1	1	1
18. Ancillary analyses	0	0	0
19. Adverse event reporting	0	0	0
20. Interpretation of results	1	1	1
21. Generalizability	0	0	0
22. Contextual interpretation	1	1	0
Total CONSORT score (max. 22)	14	11	12

Gofton et al. reported a randomized prospective trial designed to assess the effects of computer navigation on the learning of surgical skills by trainees [11]. The objectives of the study were to investigate the hypothesis that computer navigation improves task performance at the expense of learning, but that this compromise on gaining new skills can be minimized by trainee self-assessment.

Based on a power calculation, 45 senior medical students or non-orthopaedic surgical residents without substantial exposure to hip arthroplasty were recruited to participate in the study. Participants were randomized to one of three training groups: conventional training, computer navigation training, and knowledge of results training. Although specific details of the methodology of randomization (e.g., sequence generation and allocation) were not provided, the process was described as blinded and data were presented showing no significant differences in the demographics of the three groups.

All participants viewed a video highlighting task requirements and were made aware that they would be asked to perform the task independently following training. Each participant performed 30 training attempts on a custom-designed full pelvic model (Sawbones; Pacific Research Laboratories, Vashon, WA). Acetabular cup positioning was determined by use of an optical tracking system (BrainLAB, Westchester, IL). This system relies on passive arrays (reflective markers) attached to specific bony landmarks and to a standard acetabular impactor. These markers are then detected by an infrared camera to determine pelvic and cup position, respectively.

In the conventional training group, participants were trained to position cups in a traditional freehand manner. Although the navigation software was running during the task, the trainees were blinded to this. Feedback was given at the end of each task and trainees were informed of the correct position, as determined by the navigation software, if they deviated from the ideal by more than 1° (45° abduction, 20° anteversion). The knowledge of results group differed in that trainees were given the opportunity to correct the acetabular position once the feedback had been received. In the navigation group, trainees could rely on feedback from the computer system throughout the task.

Ten minutes after the training period, each participant was asked to position the cup five times in a conventional freehand manner without feedback, and this task was subsequently repeated at 4–6 weeks after the initial training. A comparison between immediate and delayed tasks allowed for a determination of task retention or learning.

Statistical analysis of the performance of groups during the training phase and in immediate retention, transfer and delayed retention tests was conducted by independent two-way mixed analysis of variance.

The authors demonstrated that there was no difference between the groups with respect to correct determination of the acetabular version angle in immediate and delayed retention or transfer tests. However, there was a statistically significant difference between the groups with respect to determination of the acetabular abduction angle in delayed retention testing. The mean difference from the correct acetabular abduction angle for the navigated group in delayed retention testing was 2.5° ± 1.7°, which was significantly less than for the conventional group (mean acetabular abduction angle 4.31° ± 1.7°, p < 0.0001) and not significantly different to the value for the knowledge of results group (mean acetabular abduction angle 1.8° ± 0.99°, p = 0.799). However, all of these figures remain within the safe zones recommended by Lewinnek et al. [1], suggesting that although a statistically significant difference is achieved, there is no clinically relevant difference, and this reflects the authors’ conclusions that task learning was not improved or compromised by navigation.

Unfortunately, not all of the results in this study were clearly presented. The authors reported that the computer navigated group demonstrated significantly better accuracy and precision in early training (p < 0.05), but the raw data used in this statistical analysis was not presented, though graphs were included to show abduction and anteversion angle performance curves with training. From these graphs, it appears that those in the computer navigation training group showed flat performance curves, whereas the other groups showed improvement; by the final trials, all groups achieved similar accuracy in acetabular placement. Although the authors concluded that computer navigation resulted in improved early performance, there did not appear to be any outliers in any group beyond the safe zones recommended by Lewinnek et al. [1]. This suggests that navigation may not confer any particular advantage in preventing malpositioning in early training when compared to the other methods of training assessed.

Cobb et al. reported a randomized trial designed to investigate the influence of three-dimensional planning, computer navigation and conventional planning on the ability of novices to accurately pass a femoral guide wire [12]. On the basis of a sample size calculation, 20 medical students were randomized into one of three groups in the study and each performed the three tasks in rotation. In Group 1, students started with conventional instruments followed by a plan and then by navigation. In Group 2, the order was first a detailed plan, then navigation and then conventional instruments. In Group 3, the order was first navigation, then conventional instruments and then a detailed plan.

Students were provided with a single dry bone femur (Sawbones, Portland, OR) on which they performed all three tasks. The navigated and planned guide wire positions were pre-determined and offset by several millimeters from the conventional position, thus allowing students to use the same bone throughout the study. Students were informed that this was necessary to measure their ability to achieve a plan that might not look right but had been determined to be biomechanically optimal.

Computer navigation was performed with the Wayfinder system (Acrobot Co. Ltd., London, UK). After the tasks were completed, all bones were re-registered with the Acrobot system and the trajectory of each hole was measured to determine the offset from the expected position for each task. Statistical analyses were performed using the t-test.

The position of guide wire insertion (as determined by range of error in varus/valgus angulation) was significantly more accurate in the navigation group (7° ± 2°) than in the plan group (22° ± 7°) or conventional group (23° ± 6°), p < 0.002. The sequence of training did not affect accuracy of guide wire placement. The authors concluded that, when provided with an appropriate level of technology, students could achieve an expert level of accuracy when placing a femoral guide wire, and that this reflects the fact that navigation may play a major role in reducing the length of the learning curve in hip resurfacing arthroplasty. However, the authors did not comment on the fact that in Group 2, although students performed navigated and planned surgery first, the participants were no more accurate than their peers in placing a guide wire in a conventional manner. This is in effect a transfer test, and is important because it demonstrates that learning has not occurred. This is perhaps due to the fact that this study is, in fact, assessing the student's ability to follow an incorrect plan, as in reality the correct trajectory would be the same in all three scenarios. On this basis, one cannot expect this study to demonstrate learning, but it is fair to conclude that navigation appears to be useful in ensuring accurate placement of a femoral guide wire in a pre-determined position. It was not assessed whether navigation was superior to the extrinsic feedback provided by an experienced surgeon supervising a trainee.

Seyler et al. described a retrospective comparative study of radiographic positioning of the femoral component in hip resurfacing performed in 96 patients between June 2006 and December 2007 either using conventional instruments or with the aid of the VectorVision hip SR Essential 1.0 system (BrainLAB, Feldkirchen, Germany) [13].

The primary objective of the study was to assess the accuracy of the navigation system for placement of the femoral component. The secondary objective was to compare the accuracy of femoral component placement between surgeons with differing levels of experience and to determine whether navigation reduces the learning curve.

Patients were assigned to one of four groups (Group 1: computer assisted, surgeon with prior experience of >250 resurfacings; Group 2: computer assisted, resident inexperienced in resurfacing, supervised by a senior surgeon; Group 3: conventional instruments, experienced surgeon; Group 4: [from Group 3], computer assisted, surgeon with experience of 40–75 resurfacings). The basis of the allocation to each group was not stated and is clearly a confounding factor, as the resurfacing experience of surgeons using conventional instruments was unclear. In addition, there was no group containing surgeons inexperienced in resurfacing using conventional instruments.

The surgical technique and radiographic evaluation were clearly described. Outcome measures included the stem-shaft angle and the presence of notching. Two intervention-blinded independent observers performed all measurements.

There was no statistical difference between the groups that used navigation with respect to range of error in varus/valgus angulation of the stem-shaft angle (Group 1: 6° ± 1.6°; Group 2: 7° ± 2.1°; Group 4: 5° ± 1.8°; p = 0.461) or the incidence of notching (Group 1: 10%; Group 2: 5.9%; Group 4: 2%; p = 0.92). The authors concluded that inexperienced surgeons can achieve a high level of accuracy when provided with navigation, and that the learning curve is therefore reduced. However, it is difficult to justify this conclusion when there was no control group in which inexperienced surgeons used conventional instruments in supervised surgery. It would perhaps be more appropriate to simply conclude that inexperienced surgeons performing computer navigated surgery whilst under supervision are able to achieve levels of accuracy similar to those of more experienced surgeons using the same level of technology.

Discussion

The primary objectives of this review were to determine whether components are more accurately placed when novices use computer navigation, and whether this increased accuracy is maintained when the procedure is subsequently performed in a conventional manner, or if there is a detrimental effect on learning. Cobb et al. and Gofton et al. both demonstrated that component positioning was significantly more accurate when novices used computer navigation than when conventional instruments were used [11], [12]. However, the difference demonstrated by Gofton et al., although statistically significant, was clinically small and within the safe zones defined by Lewinnek et al. [1], [11]. Cobb et al. demonstrated a much larger difference that could be deemed clinically relevant; however, it is important to note that the participants were not surgeons but medical students, who were unlikely to be familiar with conventional arthroplasty instruments and whose greater inaccuracy when using this method is therefore unsurprising [12]. Seyler et al. did not study whether novices were more accurate when they used navigation as compared to conventional instruments, but did demonstrate that both novices and experienced surgeons achieved the same level of accuracy when using navigation [13]. However, it should be noted that these junior surgeons were not operating independently but under the supervision of an experienced surgeon, and as there was no control group (inexperienced surgeons performing supervised surgery with conventional instruments) the true effect of navigation on accuracy of component position cannot be determined from this study.

Both Cobb et al. and Gofton et al. reported the results of transfer tests in which participants were assessed for performance in freehand component placement after having previously performed the task with the assistance of navigation [11], [12]. Both studies showed no benefit or detriment from previous training with navigation with respect to subsequent ability to accurately place components in a freehand manner. However, it is important to note that in the study by Cobb et al., when navigation was used, the guide wire was deliberately placed in a sub-optimal position offset from the ideal conventional position to enable the Sawbones to be re-used [12]. This meant that students were unable to use intrinsic feedback gained during this task to help them perform better when subsequently asked to perform the task freehand, as they had not previously placed a guide wire in the conventional position. In any case, it is interesting that these results are unexpected on the basis of motor learning theory, which would lead us to believe that the concurrent augmented feedback provided by navigation is detrimental to learning. It is possible that this negative impact on learning was reduced by participants being frequently reminded that they would eventually be required to perform the procedure freehand and being specifically made aware of intrinsic cues that they should identify to assist in accurate placement.

Gofton et al. assessed delayed retention by getting participants to place freehand components 4–6 weeks after training [11]. They found that there was a significant difference, with those originally in the conventional group performing worse than those who had been in the detailed plan or navigation groups. However, although a statistically significant difference was demonstrated, it is within the safe zones defined by Lewinnek et al. and is unlikely to be of clinical relevance [1].

Limitations

The main limitation of this review relates to the quality of the included studies. The CONSORT checklist [10] was used as a template for critical appraisal of the included trials. Using this template, significant limitations were identified in all three included studies. A major limitation was the low external validity of the available studies. For example, trials on laboratory-based models by medical students unfamiliar with basic surgical skills do not necessarily reflect real-life training scenarios. This suggests that further well-designed studies using appropriate populations (e.g., surgeons in training) are required.

Conclusion

A systematic search strategy followed by critical appraisal and application of the CONSORT checklist [10] has demonstrated that there are significant limitations in the published literature on this topic. Despite these limitations, this review is important because, in the current era of reduced overall training hours and a move towards competency-based assessment, trainers and trainees should be aware of the impact of modern surgical technology on the learning environment. Despite concerns that concurrent augmented feedback has a detrimental effect on the learning curve [9], this review has found no evidence to support such a theory. However, it should be noted that as a result of a lack of good evidence, further study is required to establish the true effect of navigation on trainee learning. Future randomized controlled trials should consider the CONSORT criteria to aid robust study design.