Identification of adverse events at an orthopedics department in Sweden

Abstract

Background and purpose Adverse events (AEs) are common in acute care hospitals, but there have been few data concerning AEs in orthopedic patients. We tested and evaluated a patient safety model (the Wim‐mera clinical risk management model) and performed a three‐stage retrospective review of records to determine the occurrence of AEs in adult orthopedic inpatients.

Methods The computerized medical and nursing records of 395 patients were included and screened for AEs using 12 criteria. Positive records were then reviewed by two senior orthopedic surgeons using a standardized protocol. An AE had to have occurred during the index admission or within the first 28 days of discharge from the Orthopedics Department. Screening of additional systems for reporting of AEs was also carried out for the same period. The number of patients suffering an AE and the number of AEs were recorded.

Results Altogether, 60 (15 %) of 395 patients checked in the screening of records experienced 65 AEs (16%) due to healthcare management. Of the 65 AEs, 34 were estimated to have a high degree of preventability. 47 of the 65 AEs occurred during the index admission and 18 within 28 days of discharge. In screening of local and nationwide reporting systems for the same patients, 4 additional AEs were identified—2 of which were previously unknown. 67 different AEs were detected by using the Wimmera model (17%)

Interpretation Using the Wimmera model with manual screening and review of records, many more AEs were detected than in all other traditional local and nationwide reporting systems used in Sweden when screening was done for the same period.

Adverse events (AEs) caused by healthcare management are common. A wide range of frequencies of AEs among hospitalized patients in acute care hospitals have been reported in several different retrospective studies, from 1.35% to 16.6% (Bren‐nan et al. 1991, Wilson et al. 1995, Schioler et al. 2001, Vincent et al. 2001, Wolff et al. 2001, Davis et al. 2002, Baker et al. 2004, Forster et al. 2004). Approximately half of these were judged to be preventable (Wilson et al. 1995, Schioler et al. 2001, Vincent et al. 2001, Davis et al. 2002, Baker et al. 2004).

In Sweden, data are available from several different reporting systems in which incidents that are possible AEs can be registered, but to our knowledge no similar Swedish studies have been published. Although international studies examining AEs in acute care hospital settings have been published (Wilson et al. 1995. Vincent et al. 2001), there are few data available for orthopedic patients specifically.

We tested and evaluated a patient safety model (the Wimmera clinical risk management model) and determined the occurrence of adverse events affecting adult patients admitted to a large metropolitan orthopedics department where both emergency and elective surgery are performed.

Patients and methods

Patients

The study was carried out at Danderyd Hospital, located in the Stockholm metropolitan area, with a 50‐bed orthopedics department. All computerized medical and nursing records of the orthopedic patients admitted for hospital care (inpatients) (Siemens record system Melior 1.5, build 208b) during August and September 2004 were included in the study.

To qualify for inclusion, an AE had to have occurred during the index admission or within 28 days of discharge from the Orthopedics Department. AEs occurring within 28 days of hospitalization were not required to result in a new admission to be included in the study; for example, a wound infection that was treated on an outpatient basis would be regarded as an AE. Symptoms already present when the patient was first admitted to hospital were excluded and were therefore not registered as AEs.

The sample constituted 15% of all 2,678 admissions to the Orthopedics Department in 2004.

The Wimmera clinical risk management model

For assessment of an AE, we used the approach presented by Wolff et al. (2001). At Wimmera Base Hospital in Australia, a systematic approach for assessment of patient safety has been developed and implemented. This approach involved manual screening of all medical records after patients had been discharged, using 8 general outcome criteria that could easily be assessed by non‐clinical staff— the intention being to identify indicators of poor patient safety. A positive result in the screening process was then reviewed by a senior member of the medical staff. AEs were discussed by a surveillance committee comprised of staff from a cross section of different clinical professions, whose task was to identify situations that should be regarded as warranting implementation of measures for patient safety. At Wimmera, other systems of reporting AEs have been implemented, for example, clinical incident reporting and general practitioner (GP) feedback on AEs. These reporting systems are intended to complement each other; they monitor the rate of AEs over time and also the effectiveness of implementation of interventions.

We have translated the Wimmera clinical risk management model into Swedish and adapted it to the Swedish healthcare system. Screening criteria relevant to orthopedic care were added (Table 1). We complemented the two‐stage review used in Wimmera with a third consensus stage. The assessment scales were the same as those used in other international retrospective record review studies (Brennan et al. 1991, Wilson et al. 1995, Wolff et al. 2001, Baker et al. 2004, Forster et al. 2004). During this first study, only details of AEs were collected and no patient interventions were done.

Table 1.  Screening criteria for identification of adverse events and their outcome

Screening criteria		Outcome stage 1 (n)	Outcome stage 3 (n)
1	Unplanned readmission as a result of healthcare management	37	15
2	Transfer from general care to intensive care	4	0
3	Unplanned transfer to another department	10	5
4	Unplanned return to operating theater within seven days	11	4
5	Complications during surgery	30	8
6	Death	4	1
7	Cardiorespiratory arrest	0	0
8	Hospital complications that developed during admission or within 28 days of discharge (e.g. myocardial infarction, stroke, or pulmonary embolism)	31	9
9	Hospital‐acquired infection or sepsis	33	21
10	Length of stay greater than 7 days	44	24
11	External cause of injury codes	10	8
12	Other undesirable outcome not covered by other criteria	46	31

Definitions

For the purposes of this study, an AE was defined as an untoward or unintended patient event caused by healthcare management. We applied a broad definition that did not require the event to cause disability or a prolonged hospital stay.

A preventable AE was defined as an error in healthcare management due to failure in following accepted practice at the level of the individual or of the system (Wilson et al. 1995, Davis et al. 2002). Healthcare management covers the actions of individual healthcare workers and also the systems and care processes used in delivering healthcare. It includes both acts of omission (for example, failure to diagnose or treat) and acts of commission (for example, incorrect diagnosis or treatment, or poor performance) (Baker et al. 2004).

Data collection

We performed a three‐stage retrospective review of computerized medical and nursing records to determine the occurrence of AEs among adult orthopedic inpatients. At Danderyd Hospital a computerized patient record system is used, which gives access to entire patient records with the exception of the anesthesia charts and medication charts.

Validation of the process

Before the actual screening process began, the screening criteria and the screening process were discussed and defined by the research group.

To validate the review process and to better achieve consistency in this process, the two orthopedic surgeons in the group discussed and agreed upon what definition of AE was to be used and how the review process should be carried out. In a trial run, they each reviewed 20 records independently and then together in a consensus process.

Screening process

In stage 1, all computerized medical and nursing records for patients admitted during August and September 2004 were screened by an experienced orthopedic screening nurse (MU) for the presence of one or more of the 12 predefined screening criteria (Table 1 and Figure). Once an incident had been found to meet at least 1 of these criteria, a judgement was made by the screening nurse as to whether it constituted a potential AE or not.

A check of traditional systems for reporting of potential AEs, other than the screening of medical and nursing records, was also carried out. This involved checking of cases reported to the Medical Responsibility Board (HSAN), to the Swedish Patient Insurance (PSR), to the Swedish system for reporting medical errors (Lex Maria), and also to the Patients’ Advisory Committee (PaN). Information from the department's own clinical incident reporting system was also screened.

During the period July 2004 through July 2005, after a patient had been discharged from the Orthopedics Department, the GP was contacted by post and requested to complete and return a standard form giving details of any potential AE that had occurred in the first 28 days after the patient's discharge. The returned forms for patients admitted to hospital during August and September 2004 were included in the screening process.

A database (Predo, constructed at Karolin‐ska University Hospital, Stockholm) in which all patient contacts with healthcare services in the greater Stockholm health service district are registered, was used for the detection of incidents that occurred within the first 28 days of discharge from the Orthopedics Department and that were possible AEs. If a potential AE had occurred, the medical record was requested from the appropriate hospital or from the GP and this was then screened manually. The screening nurse also had access to the computerized patient records from the relevant departments at Danderyd Hospital.

If a patient was admitted to Danderyd Hospital more than once or records had to be requested from outside the hospital within the first 28 days of discharge, the information from all these sources was screened but was considered to be part of the first admission. For each admission screened as positive, administrative and clinical data were collected from the patient's computerized record and also from the patient administrative data system (PASS 0.3:2), e.g. admission information, socio‐demo‐graphic data, and diagnostic classification.

After the nurse screening form (NSF) was completed for each patient record, each of the two orthopedic surgeons received a copy of all records screened as positive and the accompanying NSF for independent review.

Review process

In stage 2, the physicians reviewed all records that were found to be positive for at least 1 screening criterion at stage 1. A judgement was then made as to whether an AE had occurred. The next step was to determine whether the injury or complication was due to healthcare management. In evaluating the care given and to decide whether an AE was the result of a shortcoming in healthcare management, a 6‐point scale of causation was used (Brennan et al. 1991, Wilson et al. 1995). A score of 4 or higher was regarded as being an AE resulting from healthcare management (Brennan et al. 1991). The same scale was also used to make judgements about the preventability of an AE, irrespective of the cause. A score of 4 or more meant that the AE had been highly preventable (Table 2) (Wilson et al. 1995). The severity of the AE was graded on a 7‐point scale in which a score of 3 points or more indicated a major AE (Table 2) (Craddick and Bader 1983, Wolff and Taylor 2001).

Table 2.  Scales to judge causation, preventability, and severity of adverse events

Causation and preventability scale

1 = little or no evidence

2 = slight evidence

3 = not likely (less than 50:50 odds)

4 = more likely than not (greater than 50:50 odds)

5 = strong evidence

6 = virtually certain evidence

Severity scale

0 = minor severity

1 = minor temporary

2 = minor permanent

3 = major temporary

4 = major permanent

5 = potential major or major contributing

6 = death

A standardized protocol, review form 1 (RF 1), with the assessment scales and clinical information was completed by each reviewer. The RF 1 was the same as that used in the Wimmera study, with some modifications (Wolff and Taylor 2001).

When both reviewers had analyzed the records independently and completed an RF 1, the process moved to stage 3. The same orthopedic surgeons reviewed and analyzed all records together and subsequently completed a consensus review form (RF 2) for each event. This form was the same as the RF 1. If a patient had more than one AE, each of these was reviewed separately and given its own RF 1 and RF 2.

Evaluation of the screening process

To evaluate the nurse screening process, we selected the records of every tenth patient that had not met any of the 12 screening criteria and that had thus been deemed not to contain a potential AE in the nurse screening process. The medical and nursing records from these patients were then also screened by one of the reviewers (OM).

Statistics

Cohen's kappa was used to measure the reliability of consistency between the reviewers in stage 2. The statistics programs used to collate the results were JMP version 5.1 (SAS Institute, Cary, NC) and Excel 2003.

Ethics approval was given by the regional Ethics Committee of Stockholm (number 04–563/2).

Results

A total of 395 patients were admitted to the Department of Orthopedics, Danderyd Hospital during the study period.

There were no missing records among the hospitals records or the records requested either from other departments outside the hospital or from the GPs. We judged that there was sufficient information in the records to allow completion of all aspects of the NSF, RF 1, and RF 2.

After the initial screening of the Predo database for potential AEs that had occurred within 28 days of discharge, the screening nurse decided that 58 patient records contained a possible AE—9 from the patient records of other departments at Danderyd Hospital, 7 from the records of other hospitals, and 42 from the patient records held by general practitioners.

In 127 of the 395 patient records assessed (32%), one or more screening criteria was found to be positive and was judged to represent a possible AE at stage 1 (Figure). These 127 patient records contained 136 potential AEs.

At stage 2, the two orthopedic surgeons independently judged the number of AEs rating 4 or higher on the 6‐point causation scale to be 59 and 52, respectively. In the consensus review carried out at stage 3, 65 adverse events were considered to have occurred as a result of healthcare management (Figure). 41 of the 65 AEs were detected by more than 1 positive screening criterion (Table 1).

5 patients each had 2 AEs as found in the records and 3 patients had AEs that were identified in the traditional reporting systems (Table 3). 1 of these 3 patients had the same AE reported in 2 additional systems and it was also found in the review of records. Only 2 AEs that were not detected in the record review were found by checking the traditional reporting systems, and 4 were detected by searching the Predo database. All other AEs were detected in the computerized records system. No AEs were found in the 42 records that were requested from general practitioners. 2 potential AEs were reported from the feedback protocol from general practitioners during the study period, but neither of these was judged by the reviewers to be an AE.

Table 3.  Sources used for detecting adverse events

	Number of AEs
Medical record review	65
PSR	2:00 AM
Lex Maria	0
PaN	1:00 AM
HSAN	0
Reporting of clinical incidents	1
Feedback from GP about AE	0
PSR: the Swedish Patient Insurance
Lex Maria: Swedish system for reporting medical errors
PaN: the Patient's Advisory Committee
HSAN: the Medical Responsibility Board
GP: general practitioner
^a1 was also identified in the review of medical records.

60 (15%) of 395 patients checked in the screening of records experienced 65 AEs (16%) and the overall number was 67 (17%) AEs when the other reporting systems were included in the count (Table 3).

Of the 65 AEs found from the screening of records, 34 were estimated to have been highly preventable and the median of the preventability score was 4 (1–6). 34 of 65 AEs were considered to constitute major AEs, with a score between 3 and 6 on the severity scale. Of these 34, 29 had a severity score of 3.

The kappa value of reliability of consistency between the reviewers’ judgements at stage 2 was (1) 0.44 of whether or not an AE had occurred, (2) 0.57 of whether or not the AE had been prevented and (3) 0.62 regarding the determination of the severity of the AE.

47 of 65 AEs occurred and were detected during the index admission. 18 AEs had their trigger event during the index admission, but the actual AEs were detected within 28 days of hospitalization. 11 of the 18 AEs were detected during a new admission or were the cause of a new admission at the hospital. 7 AEs were found and treated in an outpatient setting at the hospital. 2 of these were detected during planned outpatient visits.

None of the incidents later judged to be AEs were missed in the nurse screening process.

Discussion

Using the Wimmera model, 67 AEs in 395 patients were found. This included the use of traditional local and nationwide systems of reporting AEs in Sweden. When only the traditional systems of reporting were used, 4 adverse events were found and 2 of these were not detected in the record review during the same period. The mean rate of AEs in all orthopedic inpatients in Sweden reported to the Swedish Patient Insurance is as low as 0.5% (Öhrn et al. 2006). Over half of these incidents were deemed to constitute an avoidable injury. The frequency of patient injury claims at the Orthopedics Department at Danderyd Hospital is somewhat lower.

In previous studies, the frequency of AEs has ranged between 1.35% and 16.6% (Wilson et al. 1995, Schioler et al. 2001, Vincent et al. 2001, Wolff et al. 2001, Davis et al. 2002, Baker et al. 2004, Forster et al. 2004). One explanation for this variation could be the different clinical settings. Another cause of the variation might be actual differences in the patient safety culture in different hospitals, and between departments. It is also possible that the frequency may vary depending on the identification technique. Another explanation for the variation in frequency of AEs could be differences in the definition and registration of the timing of an AE, as well as differences in the use of the causation scale (Baker et al. 2004, Zegers et al. 2007).

Few reports concerning AEs in orthopedic patients can be found in the literature—mainly because they are often included in the total surgical AE rates, which vary between 22% and 66% (Gawande et al. 1999, Kable et al. 2002, Briant et al. 2005). In a prospective study done in Gothenburg, Sweden, 30% of elective surgery patients and orthopedic patients experienced an AE postoperatively (Arvidsson et al. 1994). In two studies, the proportion of AEs was much higher for orthopedic admissions (12–14%) than for general medical admissions (7–9%) (Wilson et al. 1995, Vincent et al. 2001). These authors report a lower proportion of preventable AEs for orthopedic admissions (32–48%) than for medical admissions (73–75%). These results are similar to our findings, where half of the AEs were judged to be preventable.

In our study, three‐quarters of the AEs occurred and were also detected during the index admission. In other studies, AEs occurred in the hospital setting in 39–80% of cases (Wilson et al. 1995, Davis et al. 2002, Baker et al. 2004, Forster et al. 2004).

We found a much higher rate of AEs (17%) than in the Wimmera study (1.35%), in which nearly 50,000 patient records were screened over 8 years (Wolff et al. 2001). This difference might perhaps be explained by the fact that Wimmera Hospital is located in a rural setting, in contrast to Danderyd Hospital which is situated in a metropolitan area. The Wimmera study also included different departments and used only 8 screening criteria, and these did not require the use of any clinical judgement. In our study, both the nurses’ and the doctors’ notes in the records were assessed for the occurrence of AEs.

The number of AEs that we found was high, but it is compatible with the broad definition of an AE that we used. It is important to remember that our definition of an AE did not require that the patient should have experienced any disability or prolonged hospital stay as a result of the AE. In striving toward increased patient safety, it is valuable to identify not only AEs of a severe nature but also those that are less severe and occur more frequently. It is possible that in other situations some of the AEs we identified would have been classified as acceptable complications related to the clinical situation, rather than AEs.

Our study shows that the definition of an AE is not entirely clear‐cut. Of the 136 potential AEs identified by the screening nurse, 59 were considered to be AEs by reviewer 1 and 52 were considered to be AEs by reviewer 2 at stage 2, though they were not necessarily the same incidents. It is interesting to note that even though both reviewers received their training as orthopedic surgeons at the same institution, differences could still occur in their decisions about what incidents constituted AEs. The kappa values between the two reviewers were moderate to substantial (0.44—0.62). Although this compares favorably with the values of other similar studies, fairly low kappa values are a known methodological problem when using structured implicit review (Wilson et al. 1995, Schioler et al. 2001, Davis et al. 2002, Baker et al. 2004, Zegers et al. 2007).

The strengths of the retrospective review of medical records are that (1) it provides an effective way of estimating the incidence of AEs after surgery, (2) it is easy to plan and carry out data collection, and that (3) there are no inconveniences or interruptions to patient care (Michel et al. 2004).

In evaluating the reasons for the high frequency of adverse events in our study, it is also necessary to consider the risk of hindsight bias when screening retrospectively. This means that when looking back at a situation it will inevitably appear less complex. Another aspect of hindsight bias is that when the outcome is known to be adverse, it is easier to be more critical and detect more errors. Our study was based on a retrospective review of medical and nursing records and was therefore dependent on the availability and the quality of the information that was found in the documentation by medical and nursing staff. Lack of time and misunderstandings concerning whose responsibility it is to report an incident may reduce the quality of information regarding AEs in patient records. Another possible reason for omitting to document incidents could be the fear of litigation or disciplinary action (Vincent etal. 1999, Ödegård 2006).

Using the Wimmera model, we detected a high number of AEs. Many of them were preventable. Thus, analysis of records appears to be an important tool in improving the safety of orthopedic patients. Manual screening for AEs is, however, time consuming and is not suitable for implementation in everyday clinical practice. Instead, digital screening of patient records may be a helpful technique in the future. Such a modification of the Wimmera model would be of use both in facilitating the continuous monitoring of patient records for AEs and in detecting preventable incidents that might lead to implementation of measures to safeguard patients.

Contributions of authors

MU planned, structured and carried out the study, analyzed the data, and wrote the article. UL planned and carried out the study, and contributed to writing. OM carried out the study and contributed to writing.

We wish to thank Associate Professor Michael Soop of the Swedish National Board of Health and Welfare, who introduced us to the Wimmera clinical risk management model.

No conflict of interest declared.