New Zealand Joint Registry data underestimates the rate of prosthetic joint infection

Abstract

Background and purpose — Recent studies have revealed deficiencies in the accuracy of data from joint registries when reoperations for prosthetic joint infections (PJIs) are reported, particularly when no components are changed. We compared the accuracy of data from the New Zealand Joint Registry (NZJR) to a multicenter audit of hospital records to establish the rate of capture for PJI reoperations. Methods — 4,009 cases undergoing total knee or hip arthroplasty performed at 3 tertiary referral hospitals over a 3-year period were audited using multiple hospital datasets and the NZJR. The number of reoperations for PJI that were performed within 2 years of the primary arthroplasty was obtained using both methods and the data were compared. Results — The NZJR reported a 2-year reoperation rate for PJI of 0.67%, as compared to 1.1% from the audit of hospital records, giving the NZJR a sensitivity of 63%. Only 4 of 11 debridement-in-situ-only procedures and 7 of 12 modular exchange procedures were captured in the NZJR. Interpretation — The national joint registry underestimated the rate of reoperation for PJI by one third. Strategies for improving the accuracy of data might include revising and clarifying the registry forms to include all reoperations for PJI and frequent validation of the registry data against other databases.

Prosthetic joint infection (PJI) is a leading cause of knee and hip arthroplasty revisions (Tande and Patel 2014). The emergence of large nationwide joint registries has improved our understanding of primary and revision arthroplasties performed for PJI. Revision rate, the main outcome used in joint registries, depends on accurate reporting of reoperations. Omissions are particularly problematic for PJIs; early PJI is often caused by contamination at the time of surgery, and registry data can often help to identify risk factors (Bongartz et al. 2008, Namba et al. 2013). Recent studies have revealed deficiencies in the reporting of secondary procedures performed for PJI (Lindgren et al. 2014, Gundtoft et al. 2015). Both of the latter studies used surrogate measures of infection to detect the true reoperation rate for PJI in nationwide cohorts. The aim of this study was to identify the true incidence of reoperations for PJI in 3 tertiary referral hospitals using published diagnostic criteria to define PJI (Osmon et al. 2013). These data could then be used to evaluate the performance of the New Zealand Joint Registry (NZJR) in reporting reoperations for PJI.

Methods

The NZJR was established in 1999, and covers arthroplasty data from all 52 New Zealand hospitals that perform arthroplasty. Participation is voluntary with funding sourced from the Ministry of Health, a compulsory implant levy and donations from the private sector. The registry is validated against the New Zealand Health Information database for public hospitals and by cross-checking with implant manufacturers’ databases for private hospitals. Discrepancies are investigated and data entry is done centrally by barcode scanning and cross-checked manual entry. Since its establishment, the registry has recorded almost 170,000 primary hip and knee arthroplasties along with 14,000 hip revision procedures and 5,500 knee revision procedures. The overall capture rate has been evaluated to be more than 95% (NZJR reports are available at www.nzoa.org.nz).

Between January 1, 2006 and December 31, 2008, the NZJR recorded 4,009 primary arthroplasties performed at 3 tertiary hospitals in Auckland, including 2,157 total knee arthroplasties (TKAs) and 1,852 total hip arthroplasties (THAs) (Table 1). These 3 hospitals have a catchment population of approximately 1.6 million. If patients had more than 1 joint replaced during this time, they were treated as 2 discrete episodes. Revision arthroplasties, bilateral arthroplasties, and unicompartmental arthroplasties were not included in the study.

Table 1. Patient demographics (n = 4,009)

Mean age (SD)	69 (11)
Sex
Male	1,684 (42%)
Female	2,325 (58%)
Hospital
Auckland Hospital	770 (19%)
Middlemore Hospital	1,744 (37%)
North Shore Hospital	1,495 (43%)
Joint
Hips	1,852 (46%)
Knees	2,157 (54%)
Indication
Osteoarthritis	3,593 (90%)
Fracture	172 (4%)
Rheumatoid arthritis	86 (2%)
Other inflammatory arthritides	27 (0.5%)
Neoplasm	57 (1%)
Others	74 (2%)

The endpoint in this study was reoperation for PJI. This was defined as PJI treated with a surgical procedure within 2 years of the primary arthroplasty. PJI cases that did not undergo surgical intervention were not included. Cases involving wound complication(s) requiring surgery were also not included if they did not meet the criteria for PJI. PJIs were identified according to the Infectious Diseases Society of America (IDSA) definition, fulfilling any one of the following criteria (Osmon et al. 2013): (1) sinus tract continuous with the prosthesis, or (2) periprosthetic purulence without any other etiology, or (3) acute inflammation on histology of a periprosthetic specimen, or (4) ≥ 2 microbiological specimens with the same organism in either periprosthetic samples or blood culture samples.

To accurately determine the rate of reoperations for PJI in the Auckland region, cases were identified by clinical coding at the 3 Auckland hospitals using both discharge summary codes and operation codes. These involved both ICD9/10 codes and theater operation codes (Appendix, see Supplementary data). Clinical coding experts at each hospital assisted in accurate identification of cases. This method had been validated against a physical database and it was found to have a ensitivity of 95% (Inacio et al. 2011). If the diagnosis was only found in 1 of the above 2 documents, physical notes were reviewed and a decision was reached by consensus between the senior authors (SY and CL). The cases were then cross-referenced with databases held separately by the infectious diseases departments at all 3 hospitals. Finally, the records of each case were manually examined and those patients with confirmed PJI who underwent reoperation within 2 years of the primary operation were entered into our study. Multiple reoperations for the same case were counted as 1 episode.

A second dataset was generated from NZJR data on secondary procedures for deep infection recorded for the same 4,009 cases, from January 1, 2006 to December 31, 2010. Deep infection is defined by the registry as infection below the deep fascia, probably involving the implant. Case records were then checked manually to confirm PJI and exclude cases where a PJI diagnosis could not be made. Reoperations to treat PJIs occurring within 2 years of the primary procedure were included in the study. Again, multiple operations for the same case were counted as a single episode. Finally, the datasets from the NZJR and from hospital records were compared.

Statistics

The cumulative incidences of reoperation due to PJI were calculated for both the NZJR data and the retrospective audit data. This was done by dividing the numbers identified by the total number of arthroplasties performed. The sensitivity and specificity values of the NZJR relative to the audit data were also calculated. Agreement between the NZJR data and audit data was calculated using Cohen’s kappa.

Results

Of the 4,009 primary arthroplasty cases (2,157 knees and 1,852 hips), the NZJR identified 27 reoperations for PJI (16 knees and 11 hips) within 2 years of the primary procedure (Table 2). In 22 cases, component exchange was performed. This gave a PJI reoperation rate of 0.67% (0.74% for knees and 0.59% for hips).

Table 2. New Zealand Joint Registry data

	Number of primary procedures
PJIs within 2 years	Hips n = 1,852	Knees n = 2,157	Total n = 4,009
NZJR data, n Percent (95% CI)	11 0.59 (0.24–0.94)	16 0.74 (0.38–1.10)	27 0.67 (0.42–0.92)
Audit data, n Percent (95% CI)	19 1.03 (0.57–1.49)	24 1.11 (0.66–1.56)	43 1.07 (0.75–1.39)

During the same time period, the audit of hospital records of our primary cohort identified 43 reoperations for PJI within 2 years, involving 19 hips and 24 knees (Table 2). All 27 reoperations identified by the registry were captured by the audit of hospital records. The overall infection rate was 1.07% (1.03% for hips and 1.11% for knees). This gave the NZJR a sensitivity of 0.63 and a specificity of 1 compared to the audit of hospital records.

When compared to the audit data, the registry failed to identify 16 cases (8 hips and 8 knees) that were reoperated for PJI within 2 years of the primary procedure. 7 were debridement-in- situ-only procedures, 2 of which eventually progressed to staged revisions (defined as exchange of non-modular components) outside the catchment time. 5 were modular component exchange procedures and 4 were staged revisions. The reasons for reoperation of 2 of the 4 patients were recorded as “acetabular loosening” and not “deep infection” in the registry (Table 3). Of these 16 missed cases, 9 occurred within 90 days of the primary procedure.

Table 3. NZJR data versus audit data

	Patients missed by NZJR	Patients captured by NZJR
Hips/knees	8/8	11/16
Debridement in situ only	7	4
Modular exchange procedure	5	7
Staged revision	4^a	15
Excisional procedure	0	1
Total	16	27

a 2 reported as acetabular loosening

Compared to the audit data, the sensitivity of the registry in identifying PJI reoperations was 63%. It managed to capture 15 of 19 staged revisions, 7 of 12 modular component exchange procedures, and 4 of 11 debridement-in-situ-only procedures. Cohen’s kappa between the registry data and audit data was 0.74.

Discussion

Registries often under-report revisions, particularly revisions for PJIs. The NZJR has an overall capture rate of greater than 95%, which compares favorably with other international registries. Despite this, we identified deficiencies in its reporting, using multiple data sources. Over a 3-year period at 3 large public hospitals involving 4,009 THA and TKA cases, the NZJR identified a PJI reoperation rate of 0.67%—as compared to the 1.1% found from hospital records. Thus, the NZJR had an accuracy of 63% when detecting reoperations for PJI.

The 2-year reoperation rate of 1.1% for PJI identified in our hospital audit is similar to those in international studies. Lindeque et al. (2014) collated data from 6 studies that followed arthroplasty patients for more than 12 months. They reported a combined deep infection rate of 1.4% for hips and knees. As approximately 90% of PJIs receive operative treatment, this equates to a reoperation rate of around 1.2–1.3%. Data from the Danish Knee Register (DKR) between 1997 and 2010 showed a 2 year reoperation rate of 0.85–1.5% for PJIs (Pederson et al. 2012), while a meta-analysis of 54,000 patients found a combined deep surgical site infection and PJI rate of 1.3% (Chen et al. 2013). This is again in keeping with our findings.

Previous studies using national joint registries have identified under-reporting of reoperations for PJIs. Between 1997 and 2003, the Finnish Knee Register (FKR) reported a 1-year reoperation rate of 0.77% for PJIs—less than the 0.89% reoperation rate identified by the Finnish Patient Register database during the same time period (Jamsen et al. 2009). In particular, modular exchange arthroplasty and excision procedures were often missed. There is evidence to suggest that national patient databases may also underestimate PJI reoperations. Gundtoft et al. (2015) followed a cohort of almost 33,000 primary THAs and used an algorithm based on laboratory results to identify reoperations for PJI. They reported a 1-year incidence of 0.86% and 5-year incidence of 1.03%. These figures are 40% higher than the PJI reoperation rate reported by both the Danish Hip Arthroplasty Register and the Danish Patient Register. Lindgren et al. (2014) captured primary total hip arthroplasty patients from the Swedish Hip Arthroplasty Register (SHAR) who were also recorded in the Swedish Prescribed Drug Register as receiving outpatient antibiotics over more than than 4 weeks. Questionnaires were sent to the patients’ primary arthroplasty provider. The authors found that the reoperation rate for PJI within 2 years of the primary procedure was 1.3%. Only 67% of these had been recorded correctly in the SHAR. When the reoperation did not involve exchange of components, the sensitivity of the SHAR dropped further to 57%. This reflects our findings, with only 4 of 11 such reoperations correctly reported to the NZJR.

There are a number of explanations for the poor capture rate. The NZJR forms for hip and knee secondary procedures do not include a column for “debridement-in-situ-only” procedures. Some surgeons may not consider these to be reportable if no components are exchanged, leading to the low capture rate for these procedures. Secondly, the acute nature in which these operations are often performed undermines the reporting process. Staged revisions are routinely performed in daylight hours, with regular theater staff who are experienced in registry reporting. Debridement procedures with or without component exchange are often performed out of hours in the acute setting. Only 7 of 12 modular exchange procedures were reported to the NZJR. This issue has also been identified by the SHAR and the FKR (Jamsen et al. 2009, Lindgren et al. 2014). In particular, the FKR captured 92% of staged revisions but only 78% of modular exchange revisions and 0% of debridement-only procedures.

As with its international counterparts, the NZJR performed best when identifying staged revisions for infection. Its capture rate of 15 of 19 compares favorably with overseas data; the SHAR reported a 74% capture rate for similar revisions due to infection. However, these figures are still lower than the generally accepted capture rate of 90–95% for registries (Soderman 2000, Arthursson et al. 2005, Karrholm 2010). Most registries are validated against national patient registers, which have also been shown to under-report reoperations for PJIs, and this may cause an overestimation of registry accuracy (Gundtoft et al. 2015). It is also possible that surgeons are more reluctant to report revisions for infection than for other causes.

Confusion regarding registry reporting requirements may also be an explanation for the poor capture rate (Lindgren et al. 2014, Witso 2015). Revisions for PJIs often occur earlier than revisions for other causes. 9 of 16 reoperations that were missed by the NZJR were performed within 90 days of the primary procedure. This suggests there is a lack of clarity regarding when a reoperation can be reported after the primary, especially if the 2 events have occurred within 90 days (Arthursson et al. 2005).

It is worth noting that over a longer follow-up period, the NZJR would have captured a larger proportion of patients treated operatively for PJIs. Many patients undergoing debridement-in- situ-only procedures will eventually require staged total component exchange procedures, which are more likely to be captured. We observed this in 2 of 7 such cases missed by the NZJR.

The strength of the present study lies in the standardized way in which PJI reoperations were identified. All 3 hospitals share a clinical records system, and all cases of PJI reoperations were confirmed by an orthopedic surgeon and an infectious diseases specialist. This result is applicable to all of New Zealand, as all centers routinely report to the NZJR. However, there may be regional differences in reporting, which might affect the generalizability of our results. All 3 hospitals in our audit were public hospitals, which is a possible source of bias—as reporting rates at private hospitals may differ. While every attempt was made to identify every reoperation through our audit, it is possible that some reoperations were missed—for example, if a patient had moved out of area. However, no such patients were identified in the national database of the NZJR. Additionally, surgeons may not suspect deep infection as the most likely cause of reoperation at the time. This is a possible cause of under-reporting by the NZJR, and it is a weakness of our study. In our series, an audit of clinical notes of all 16 cases missed by the NZJR showed clear evidence that infection was the most likely cause. It is also worth noting that 12 of the 16 cases eventually underwent component exchange or 2-stage revision procedures, giving surgeons more than 1 opportunity to correctly report to the NZJR.

Our findings suggest that registry reporting needs to improve before it can be reliably used to gauge PJIs and their risk factors. Strategies to improve completeness include frequent validation against national patient registries (Serra-Sutton et al. 2009, Kolling et al. 2007, Paxton et al. 2010, Barsoum et al. 2012). The Swedish Knee Register missed up to 20% of all revisions until routine validation was performed (Robertsson et al. 2014). This system is used as frequently as every 3 months in the Danish Knee Registry (Pederson et al. 20012). Currently the NZJR is only validated against the New Zealand Health Information Service every 3 years. Electronic form submission and validation should also be considered, as these limit errors in data reporting and can significantly reduce the resources required for validation (Paxton et al. 2010, Barsoum et al. 2012). Lastly, the NZJR reporting forms can be improved by including procedures without component exchange as an option.

Specialized databases can help reduce the number of missed cases. The Surgical Site Infection Improvement programme (SSIIP) was established in New Zealand in March 2013 (New Zealand Health Quality and Safety Commission). It tracks all hip and knee arthroplasty patients in New Zealand and reports any infections within 90 days of the indexed primary procedure. We found that reoperations during the first 90 days are poorly reported to the NZJR. If the NZJR database had been validated against the SSIIP during the study period, its capture rate would have increased to 84%. The process of sharing data between the NZJR and the SSIIP is currently under way.

A number of previous studies have used national joint registry data on deep infection as an outcome measure (Hooper et al. 2011, Namba et al. 2013), particularly when assessing risk factors for PJI. Our data suggest that a high number of PJIs are in fact missed in registry data, and this should be taken into account when interpreting these findings. However, we believe that such studies still provide useful information, and registries provide a way of achieving the large numbers required to investigate rare complications such as PJI.

In summary, we found that the NZJR data significantly underestimated the true incidence of reoperation for PJI. In particular, it had poor sensitivity when identifying modular exchange and debridement-in-situ-only procedures. Steps to improve this may include modifying registry forms to highlight the option of procedures without exchange of components, improving feedback to orthopedic surgeons, and frequent validation against other national datasets. In doing so, the registry can become a more useful tool for identifying PJIs and their associated risk factors.

Supplementary data

The Appendix is available on the Acta Orthopaedica website (www.actaorthop.org), identification number 9637.

MZ: data gathering and analysis, and preparation of manuscript. SR: data gathering and analysis. CF: data gathering. CL: data gathering and preparation of manuscript; senior author. SY: preparation of manuscript; senior author.