Joint registry approach for identification of outlier prostheses

Abstract

Background and purpose Joint Replacement Registries play a significant role in monitoring arthroplasty outcomes by publishing data on survivorship of individual prostheses or combinations of prostheses. The difference in outcomes can be device- or non-device-related, and these factors can be analyzed separately. Although registry data indicate that most prostheses have similar outcomes, some have a higher than anticipated rate of revision when compared to all other prostheses in their class. This report outlines how the Australian Orthopaedic Association National Joint Replacement Registry (AOANJRR) has developed a method to report prostheses with a higher than expected rate of revision. These are referred to as “outlier” prostheses.

Material and methods Since 2004, the AOANJRR has developed a standardized process for identifying outliers. This is based on a 3-stage process consisting of an automated algorithm, an extensive analysis of individual prostheses or combinations by registry staff, and finally a meeting involving a panel from the Australian Orthopaedic Association Arthroplasty Society. Outlier prostheses are listed in the Annual Report as (1) identified but no longer used in Australia, (2) those that have been re-identified and that are still used, and (3) those that are being identified for the first time.

Results 78 prostheses or prosthesis combinations have been identified as being outliers using this approach (AOANJRR 2011 Annual Report). In addition, 5 conventional hip prostheses were initially identified, but after further analysis no longer met the defined criteria. 1 resurfacing hip prosthesis was initially identified, subsequently removed from the list, and then re-identified the following year when further data were available. All unicompartmental and primary total knee prostheses identified as having a higher than expected rate of revision have continued to be re-identified.

Interpretation It is important that registries use a transparent and accountable process to identify an outlier prosthesis. This paper describes the development, implementation, assessment, and impact of the approach used by the Australian Registry.

Many factors influence the outcome of joint replacement surgery. Arthroplasty registries are able to identify differences in outcome based on patient-, surgery-, or prosthesis-specific factors (Herberts 1997, Graves et al. 2004, Hallan et al. 2007, Ranstam and Robertsson 2010). The principal measure of primary joint replacement surgery is time to first revision, generally estimated using the Kaplan-Meier survival method (Dobbs 1980). This measure is an unambiguous and clear indication of a problem with the primary procedure, where both the patient and surgeon have agreed that it is serious enough to require further surgical intervention (Söderman and Herberts 2000, Robertsson 2007).

It is known that prostheses have variable outcomes and, while most perform well, some have outcomes well outside what would be regarded as acceptable. This variability in prosthesis performance highlights the need for adequate pre-market assessment and vigilant post-market surveillance. Joint replacement registries play a critical role in providing quality post-market surveillance, as well as helping to understand prosthetic use and improving patient outcomes (Herberts 2000, Kolling et al. 2007, Fevang et al. 2010, Graves 2010). Registries have also been very effective in identifying prostheses or combinations of prostheses that are outliers with respect to revision rate, when compared to others in the same class (Robertsson and Lidgren 2008, Espenhaug et al. 2009, de Steiger et al. 2011).

It is important that registries use a transparent and accountable process to identify an outlier. The AOANJRR was one of the first registries to develop a standardized process for identification of such prostheses (AOANJRR 2004 Annual Report). This process attempts to take into account the extent of difference and to determine the possible reasons for that difference. In this paper we describe the development, implementation, and assessment of that approach.

Materials and methods

The AOANJRR began a staged implementation on September 1, 1999 and has collected full national data since 2002. This registry has developed a standardized 3-stage approach to identifying prostheses that have a higher than expected rate of revision. Stage 1 has been present since the Registry commenced, stage 2 was introduced in 2003, and stage 3 in 2007.

Stage 1

The first stage is an initial screening test. It is an automated analysis that identifies prostheses where the revision rate (per 100 component years) exceeds twice that of all other prostheses in the same class, and the Poisson probability of observing that number of revisions, given the rate of the class, is statistically significant (p < 0.05). Additional criteria include that there must be at least 10 primary procedures for that prosthesis, or the proportion revised is at least 75% and there have been at least 2 revisions. In addition, if a particular class contains a prosthesis that represents more than 25% of the group, a second probability analysis is performed in stage 1. This analysis excludes the prosthesis from the overall rate and the probability is re-estimated using only the remaining prostheses. This is to avoid any bias on the revision rate that may occur by including a dominant prosthesis. This initial algorithm is based on a well-established epidemiological model identifying person-time at risk. This represents the observational experience in which disease onsets can be observed (Rothman 1998). Component years are substituted for person-years in the Registry model. Individual prostheses are identified but, specifically with primary hip replacement, a combination of prostheses may be identified. This occurs when a femoral stem and acetabular component are implanted together and the combination has a higher than expected rate of revision. Knee replacements are identified as a specific variant of the same brand if only the variant of the brand has a higher rate of revision, e.g. Genesis II Oxinium (cementless)/MBK.

Stage 2

In stage 2, Registry staff—including 3 orthopedic surgeons—review more detailed information on all prostheses identified in stage 1. An important part of stage 2 is the analysis examining the impact of potential confounders, such as age, primary diagnosis, and reason for revision, which are known to influence implant survival (Ranstam et al. 2011). This process seeks to identify patient and surgeon factors as well as device-related factors that may have contributed to the observed higher rate of revision. Prostheses may be excluded from further review for a variety of reasons, some of which may include inadequate numbers or use in complex primary situations, or if they have been combined with prostheses already known to have a higher rate of revision. Age and sex-adjusted hazard ratios are calculated using Cox regression models. If the hazard ratio of a particular prosthesis—compared to all other prostheses in the same class combined—is statistically significant, then the prosthesis or prosthesis combination progresses to stage 3. Additionally, all prostheses identified in the previous Annual Report are included in stage 2, regardless of re-identification in stage 1. The reason for this is to ensure that these previously identified prostheses undergo a complete follow-up assessment.

Stage 3

In 2007, a third stage of assessment was added, enabling senior clinicians from the Australian Orthopaedic Association Arthroplasty Society to review the detailed analyses of prostheses and combinations identified in stage 2. The panel meets with staff from the AOANJRR at a 2-day workshop to critically appraise all the information and to determine which prostheses should be identified as outliers in the Annual Report. At this stage, the expert panel may request Registry technical staff to provide further information or additional statistical analyses.

At the conclusion of stage 3, the AOANJRR then lists identified prostheses in 1 of 3 groups: (1) those that are no longer used in Australia, (2) those that have been re-identified and are still used, and (3) those that are being identified for the first time. Summary data for each prosthesis or prosthesis combination are provided in the Annual Report, and a full analysis is available in the supplementary report section on the AOANJRR website https://aoanjrr.dmac.adelaide.edu.au/annual-reports-2012.

Results

Between 2004 and 2011, the AOANJRR identified 78 prostheses or prosthesis combinations using its 3-stage approach. These included 42 conventional and 6 resurfacing hip prostheses and also 5 unicompartmental and 25 total knee prostheses. In general, once a prosthesis or prosthesis combination has been identified, it continues to be identified as an outlier in subsequent years. There have been 5 primary conventional hip prostheses or combinations that have been used in more than 150 procedures that were initially identified and subsequently after 1 year no longer satisfied the defined criteria. 1 resurfacing hip prosthesis was initially identified, subsequently removed from the list, and then re-identified the following year when further data were available. All unicompartmental and primary total knee prostheses previously identified as having a higher than epected rate of revision have been re-identified (AOANJRR 2011 Annual Report).

During preparation of the 2011 Annual Report, the AOANJRR identified 217 prostheses or prosthesis combinations in stage 1. Of these, 123 (56.6%) were analyzed in more detail in stage 2. Those that did not show a statistically significant difference in the rate of revision compared to the combination of all other prostheses in the same class were excluded. In stage 3, there were 95 (44%) prostheses or prosthesis combinations reviewed by the independent panel of orthopedic surgeons and 17 were excluded. Reasons for exclusion included identifying non-prostheses-related factors such as major differences in primary diagnosis, or where surgeon specific factors were felt to be contributing to the higher than expected revision rate.

Overall, there were 78 prostheses (36%) or prosthesis combinations identified in 2011, and 17 of these were newly identified (Table). These prostheses comprise 3.5% of all the different primary hip and knee replacements that have been recorded by the Registry. Of the prostheses identified, 37 of 78 (47%) are no longer used on the Australian market, and of those prostheses that were re-identified and were still used, 18 of 24 (75%) had had reduced use compared to the previous year. 14 combinations of acetabular cup and femoral stems have been reported that do not feature as individual prostheses, but when combined they have a higher than expected rate of revision.

Table

Prosthesis type	Identified in stage 1	Analyzed in stage 2	Reviewed in stage 3	Identified overall	Newly identified in 2011
Hips
Total conventional	150	83	56	42	13
Total resurfacing	7	6	6	5	1
Knees
Unicompartmental	5	6	6	6	1
Total knee	55	28	27	25	2
Total	217	123	95	78	17

Discussion

The approach to identifying “outlier” prostheses varies between arthroplasty registries. The Swedish Hip Arthroplasty Register publishes survivorship curves of prostheses and combinations but makes no specific comparison (Swedish Hip Arthroplasty Register - Annual Report 2010). The Norwegian Register documents the use of prostheses and publishes outcomes in peer-reviewed journals, but does not report specific survivorship curves in its annual report (Norwegian Arthroplasty Registry - Annual Report 2010). The New Zealand Joint Registry (New Zealand Registry - Annual Report 2011) publishes tables of prosthesis outcomes but does not identify outlying prostheses. The National Joint Registry for England and Wales has developed an outlier subcommittee to discuss strategy and methodology for analysis of data on each implant that has been highlighted as needing evaluation, but these have not been published as yet (National Joint Registry - Annual Report 2011).

The Swedish Knee Arthroplasty Register uses a different approach. A specific knee prosthesis is used as a reference to compare the outcome of other prostheses (Swedish Knee Arthroplasty Register Annual Report 2010). The choice of an index prosthesis requires that the prosthesis is used in numbers large enough to allow adequate comparison. At one point, the AOANJRR compared all unicompartmental knees to the most frequently implanted prosthesis, the Oxford 3. This was because at that time it was used in a large proportion (35%) of all unicompartmental prostheses (AOANJRR 2006 Annual Report). Since then, the proportion has diminished each year and it became no longer appropriate to use an approach for unicompartmental knee replacement that was different to that being used for all other classes of prostheses.

The AOANJRR chose to identify outlying prostheses within 2 years of collecting full national data, and this paper describes the development and evolution of the method over time. It is a transparent and accountable process that culminates in an independent review to determine what devices should be identified as outlier prostheses. It is important for surgeons to have current information on prosthesis outcomes, to enable them to select the best-performing devices for their patients. Registries provide an ideal form of post-market surveillance that is readily able to achieve this. Other surveillance measures such as adverse event reporting are known to have limitations (Hauser 2012, Heard et al. 2012, Willis et al. 2012, ). Most importantly, these are very dependent on what is reported and there is no provision of information on comparative performance. It is also necessary for regulatory authorities and industry to be aware of outlier prostheses as, even with internal monitoring, the real number of revisions may not be apparent (de Steiger et al. 2011, Zuckerman et al. 2011, McGee et al. 2012). Following a health technology assessment review, and in part based on registry data, the Australian government reclassified hip, knee, and shoulder replacements from Class IIB to Class III (high-risk medical devices) (Therapeutic Goods Administration Regulation Impact Statement 2012). Since 2007, the Registry approach to identification has also been incorporated into the regulatory processes in this country. Following the release of the AOANJRR Annual Report, the Therapeutic Goods Administration (TGA), which is the Australian regulatory affairs body for medicines and devices, requests further information from the industry to justify the continued use of products identified as having a higher than expected rate of revision. The response of the industry to the Registry data is then reviewed by another specialist orthopedic TGA committee, which makes recommendations about the ongoing use of the individual prostheses (TGA Reforms 2011).

The Registry analyses the rate of revision separately for acetabular and femoral components, and if there is a higher than anticipated rate, individual components are published in the Annual Report. The Registry also analyses all combinations of acetabular and femoral components. Occasionally, a combination of prostheses—only when used together—has a higher than anticipated rate of revision, and this combination is noted.

A well-performing prosthesis can also be linked to a prosthesis known to have a higher than expected revision rate, so that the combination performs less satisfactorily. The Corail/ASR combination was first reported in 2008 as having a revision rate that was more than twice its comparators even though good results had already been reported for the Corail stem (AOANJRR 2008 Annual Report). This was the first time that the ASR acetabular component, which had previously been reported with resurfacing (AOANJRR 2007 Annual Report), was associated with an increased rate of revision in conventional hip replacement. In other cases, an individual component is associated with a higher than expected revision rate no matter what prosthesis it is implanted with. If a prosthesis or combination previously identified no longer meets the criteria, it is not re-identified subsequently and this is documented in the Annual Report. Registries continually monitor changing outcomes, and it is important to note that the report reflects that particular time period.

There are both strengths and limitations to the process by which the AOANJRR identifies prostheses with higher than anticipated rates of revision. Stage 1 is effective as a screening test to flag prostheses but it does not account for changes in revision rate over time. This limitation makes it difficult to detect a difference if the higher risk of revision occurs later in the follow-up period (Hardoon et al. 2006). The introduction of stage 2 enabled further analysis to be performed on a number of variables, both device- and non-device-related. Stage 3 has proven to be valuable because it broadens the clinical perspective available to the AOANJRR. With the large number of prostheses reported to the Registry, it is difficult for the Registry surgeons to have a working knowledge of all the devices. The addition of members of the Arthroplasty Society broadens the clinical perspective. Surgeons involved in stage 3 have experience of many of the devices and add valuable input to the Registry findings. This improves the transparency and accountability of the Annual Report by ensuring peer review by the peak arthroplasty body in the country.

The Registry compares prostheses to all remaining components in their class, and therefore under-reports prostheses with a higher than expected revision rate compared to the situation where the Registry only used the better-performing prostheses as the comparator. When a prosthesis with a higher than expeected revision rate has been identified, it usually continues to be identified in subsequent reports. . After identification of the device, the usage usually declines—which may have a significant effect on its subsequent outcome, for a variety of reasons. Identification may bring the prosthesis to the attention of surgeons not performing large enough numbers to be aware that it has a higher rate of revision. They may then change their choice of prosthesis. It may also highlight patient selection issues such as resurfacing hip arthroplasty having a higher rate of revision in women, patients with smaller-diameter femoral heads, and older patients (Prosser et al. 2010). This may result in a change of indication for prosthesis use, which has been shown in the Registry (AOANJRR 2011 Annual Report)

The Registry is most effective at identifying the performance of recently introduced prostheses, but those prostheses with delayed onset of a higher rate of revision are not identified as readily. It has become evident that the approach to identification may be too broad, and it is important to perform a careful range analysis of prostheses to identify which particular type is responsible for the higher than expected rate of revision within that particular group. An early example of this process was the Preservation Unicompartmental Knee, which was first identified in 2004 (AOANJRR 2004 Annual Report). In 2006, it became apparent that only the mobile bearing component had an increased rate of revision (AOANJRR 2006 Annual Report). More recent examples of prostheses that were not identified on routine screening but that required specific sub-analysis include the LCS/Duofix knee and size issues associated with the Spectron femoral stem. The Registry will continue to develop further strategies to identify specific prostheses within a broader group, keeping in mind that reducing the numbers available for analysis may reduce statistical precision.

The Registry is aware that a single surgeon may be responsible for a prosthesis combination that has a higher rate of revision. This situation has occurred twice, and on both occasions subsequent use of the combination ceased following publication of the Annual Report.

Identification by registries of prostheses with a higher than expected rate of revision is a process that will continue to evolve and develop. This will be enhanced by international collaboration between registries, which includes the possibility of using other registries to verify or confirm outlier prostheses. In addition, systems could be established to enable data pooling, which would allow enhanced analysis to better understand the role of device-related and non-device-related factors that may contribute to the higher revision rate identified.

Conclusion

Many approaches for systematic reduction of the rate of revision have been described. Identification of prostheses with a higher than expected rate of revision is far less widely reported. Arthroplasty registries are effective in identifying outliers, and they can determine multiple factors that affect outcome—including device- and non-device-related issues. The Australian Registry has been successful in doing this and, as a result, many outlier prostheses are no longer on the market. Registries and international collaboration between registries will continue to play a major role.

RdS, SG and DD designed the research question. RdS wrote the manuscript. LM performed data extraction and, together with PR, did the statistical analysis. All the authors were responsible for editing and final approval of the paper.

The authors thank the hospitals, orthopedic surgeons, and patients whose contributions made this work possible.

There was no external source of funding for this study. The Australian government funds the AOANJRR through the Department of Health and Ageing.

No competing interests declared.