Measurement tools and process indicators of patient safety culture in primary care. A mixed methods study by the LINNEAUS collaboration on patient safety in primary care

ABSTRACT

Background: There is little guidance available to healthcare practitioners about what tools they might use to assess the patient safety culture.

Objective: To identify useful tools for assessing patient safety culture in primary care organizations in Europe; to identify those aspects of performance that should be assessed when investigating the relationship between safety culture and performance in primary care.

Methods: Two consensus-based studies were carried out, in which subject matter experts and primary healthcare professionals from several EU states rated (a) the applicability to their healthcare system of several existing safety culture assessment tools and (b) the appropriateness and usefulness of a range of potential indicators of a positive patient safety culture to primary care settings. The safety culture tools were field-tested in four countries to ascertain any challenges and issues arising when used in primary care.

Results: The two existing tools that received the most favourable ratings were the Manchester patient safety framework (MaPsAF primary care version) and the Agency for healthcare research and quality survey (medical office version). Several potential safety culture process indicators were identified. The one that emerged as offering the best combination of appropriateness and usefulness related to the collection of data on adverse patient events.

Conclusion: Two tools, one quantitative and one qualitative, were identified as applicable and useful in assessing patient safety culture in primary care settings in Europe. Safety culture indicators in primary care should focus on the processes rather than the outcomes of care.

Keywords::

• patient safety culture
• process indicators
• measurement tools
• primary care
• LINNEAUS collaboration

KEY MESSAGE:

• Useful tools for assessing safety culture are the ‘Manchester patient safety framework’ and the ‘Agency for healthcare research and quality survey.’

• Useful process measures are monitoring trends, having a suggestion box, identifying practice leads and assessing safety culture.

• Safety culture indicators should focus on the processes rather than the outcomes of care.

INTRODUCTION

In recent years, the role of safety culture in both understanding how and why things go wrong has become increasingly important. When things go wrong in healthcare, politicians and policy makers frequently focus on the role of safety culture as both a cause and a solution for improvements in patient safety. Most healthcare professionals will also agree that creating a culture of openness and transparency is essential if safety culture is to be improved. However, there is very little evidence of a clear relationship between improvements in safety culture and harm reduction.

This study aimed to identify useful tools for assessing patient safety culture in primary care organizations in Europe, and to identify those aspects of performance that should be assessed when investigating the relationship between safety culture and performance in primary care.

METHODS

Definition of safety culture

For the LINNEAUS collaboration project, the following definition of safety culture was adopted: an integrated pattern of individual and group behaviour, based on shared beliefs and values, that continuously seeks to minimize patient harm that may result from the processes of delivery of care (1).This definition clearly focuses on behaviour, and the beliefs and values that underpin that behaviour and on the processes of patient care rather than on patient outcomes. This is a crucial distinction, as patient outcomes are likely to be determined by multiple factors, and so may be only distally related to the culture of a primary care organization. However, patient care processes, as manifested in the behaviour of individuals and groups, directly reflect the safety culture of that organization. Therefore, it is argued that the most relevant aspects of performance to assess should be process-related, rather than outcome-related.

General design

Three related studies were done: two consensus-based studies involving an international panel of experts and a field-test in primary care settings. Initially, comments were collected from informants from all of the LINNEAUS Euro-PC partner states, using a modified Delphi technique. Participants were chosen by LINNEAUS partners based on their interest and expertise in patient safety in primary care. They included family doctors, academics and health managers. In the assessment of tools, ratings for each instrument were obtained from a total of 15 experts from the UK, the Netherlands, Denmark, Germany, Poland and Austria (three individuals failed to complete and submit their ratings). In the selection of indicators, 31 individuals participated from the following states: Austria, Denmark, England, Germany, Greece, Poland, the Netherlands and Scotland.

Study 1: Identifying a safety culture tool for use in primary care

An inventory of patient safety instruments compiled in 2010 for the European Union Network for Patient Safety (EuNetPaS), was used as the main resource to establish what instruments existed and used in European healthcare settings (2). However, the EuNetPaS inventory primarily covered instruments used to assess patient safety culture in hospitals. Moreover, using the LINNEAUS Euro-PC definition of safety culture, some of the tools in the EuNetPaS inventory were not deemed safety culture assessment tools. Nine tools from the EuNetPaS inventory were selected as potential measures of patient safety culture.

Electronic copies of each of the nine tools were sent to three experts in each of the six European countries that were collaborating on the project. In each country, at least one person providing ratings was a primary care practitioner, and one an academic. Each participant was asked to review the tools and answer the following questions, using four-point Likert scales: How applicable is this tool to your national healthcare system? (Completely applicable (3); not at all applicable (0).) How applicable is this tool to the primary care in your national healthcare system? (Completely applicable (3); not at all applicable (0).) Two free response questions were also included: In your opinion, are any key aspects of patient safety culture in primary care missing in this tool? In your opinion, should any of the aspects of patient safety culture in primary care included in this tool be removed? The total score for each tool was calculated as the sum of all individual responses, taking into account both applicability to the national healthcare system and to primary care, giving a maximum possible score of 90.

Study 2: Identifying patient safety indicators related to culture

To identify the best patient safety indicators, an initial list of potential indicators was prepared, based on the published literature. An Internet literature search based on the terms ‘patient safety culture’ and ‘patient safety indicators’ revealed that very few studies were suitable. Consequently, a very limited number of papers suitable for inclusion were identified (3–5). From those that were, an initial set of suitable potential indicators was developed. Only those that could be relevant to primary care were included. So, for example, hand hygiene was included (because consistent use of hand hygiene protocols is a process indicator, which is relevant to safety culture), decubitus ulcer was not since its genesis and treatment was clearly based in the hospital setting. In addition, the indicator statements were worded to ensure that they focused on process rather than outcome. So for example, rather than numbers of patient-adverse events (outcome), the relevant question asked about the collecting of data on patient adverse events (process). Participants were each asked to rate a set of 26 potential indicators in terms of appropriateness, using a five-point Likert rating scale where 1 = entirely appropriate; 2 = appropriate; 3 = not sure; 4 = inappropriate; and 5 = entirely inappropriate. The ten indicators rated as least appropriate were then dropped, and the revised list was recirculated to the original participants, 26 of whom responded again, giving a response rate of 87%. The remaining 16 indicators were rated for both appropriateness and usefulness, using five-point Likert rating scales where 1 = entirely appropriate/very useful; 2 = appropriate/quite useful; 3 = not sure; 4 = inappropriate/not very useful; and 5 = entirely inappropriate/not useful at all.

Study 3: Field-testing the safety culture tools

The safety culture tools were field tested in Poland, Germany, England and Greece, to ascertain any challenges and issues arising when they are used in primary care settings. Participants in the field tests were primary care physicians. Following use of the safety culture tools they were asked how far they felt that engaging with the tool broadened/deepened their understanding of the nature of patient safety culture in primary care, how clear and helpful the instructions/guidance from the facilitator were, how easy/difficult to understand and use the materials were and how far participants thought the results would help their organization decide how to improve patient safety.

RESULTS

A safety culture tool for primary care

The two tools for assessing patient safety culture that emerged with the highest ratings were the primary care version of the Manchester patient safety framework (MaPSaF), with a total of 65 points (out of 90), and the medical office version of the survey from the Agency for healthcare research and quality, with a total of 61 points (6,7) (Table 1).

Table 1. Applicability ratings of potential measures of patient safety culture from the EuNetPaS inventory.

Measures of patient safety culture identified from the EuNetPaS inventory (numbers of papers published)	Total applicability rating (maximum possible = 90)
Manchester patient safety framework: primary care (6)	65
AHRQ hospital survey on patient safety culture (7)	61
Patient safety culture in healthcare organizations (8)	57
Teamwork and patient safety attitudes questionnaire (13)	56
Safety attitudes questionnaire (14)	51
Culture of safety questionnaire (9)	50
Checklist for assessing institutional resilience (10)	47
Safety climate assessment tool (12)	44
Hospital culture questionnaire (11)	37

EuNetPaS, European Union network for patient safety; AHRQ, Agency for healthcare research and quality.

Patient safety indicators related to culture

Table 2 lists the mean indicator rating of appropriateness in the first and second rounds, and the measure of usefulness carried out in the second round. Indicators, which had the highest mean score for appropriateness in the second round, were those where the practice-monitored trends in adverse events, gave feedback to patients experiencing adverse events, had a suggestion box, had an identified lead for patient safety/clinical risk management and assessed safety culture. In terms of usefulness, indicators that scored highly (mean score) were monitoring trends, patient feedback, a suggestion box, managing transitions to other care settings, guidelines for hand hygiene, an identified lead and assessing safety culture.

Table 2. Ratings of appropriateness of indicators for patient safety culture (first and second rounds) and usefulness (second round).

Appropriateness indicators for patient safety culture	First round Appropriateness Mean	Second round Appropriateness Mean	Usefulness Mean
The practice collects data on patient adverse events	1.87	1.36	1.44
The practice uses a formal root cause analysis tool to investigate adverse events	2.77	NA	NA
The practice looks at trends in its adverse events	2.42	1.88	1.88
The practice gives feedback to patients experiencing adverse events	1.97	1.88	2
The purpose of their medication is explained to patients	1.71	1.36	1.68
Patients are given time in the consultation to ask questions	1.77	1.52	1.52
The practice has a system that allows patients to make improvement suggestions (e.g. a suggestions box)	2.48	1.84	2.04
The practice has formal arrangements in place to safely manage the transition of patients to other care settings (hospital) and back	2.13	1.68	1.92
The practice has a system allowing staff to report risks to patients and/or staff	2.32	1.68	1.72
The practice uses a formal risk assessment process	2.84	NA	NA
The practice uses an electronic trigger tool for the surveillance of adverse drug events	2.55	NA	NA
The practice has a system for collecting data on medication error	2.71	NA	NA
The practice has a guideline for hand hygiene	2.16	1.28	2.32
The practice assesses hand hygiene, e.g. by measuring consumption of alcohol gel	3.35	NA	NA
The practice measures compliance with guidelines on the wearing of jewellery by staff	3.45	NA	NA
The topic of patient safety is regularly discussed at staff meetings	1.97	1.56	1.76
There is a person in the practice who has formal responsibility for patient safety/clinical risk management/quality	2.32	2.16	2.28
The most senior person in the practice discusses patient safety and quality with the staff	3.25	NA	NA
The practice assesses its patient safety culture on a regular basis	2.35	1.8	2
The practice measures the development of its patient safety culture	2.58	NA	NA
The practice has a formal process to ensure that all equipment is well maintained	1.68	1.08	1.64
The practice has a policy on recognizing and handling stress in staff	2.55	NA	NA
The practice regularly measures staff satisfaction	2.87	NA	NA
The practice is compliant with national regulations on the sterilizing of equipment	1.71	1.32	1.72
The practice had a formal system for handling patient complaints	2.03	1.76	1.8
The practice has a process for ensuring that its staff get regular training and updates in order to remain professionally competent	2.03	1.72	1.56

NA, candidate indicators not included in the second round have no ratings corresponding to that round.

Field-testing the safety culture tools

The wording of the MaPsaF primary care version, and the wording and scope of the AHRQ was revised to increase clarity for users in European nation states. As a result of the field test, a practical guide and a set of frequently asked questions, together with their answers, was prepared, to guide interested parties through the use of the recommended tools.

DISCUSSION

Main findings

This study has allowed for the preliminary identification and testing of two tools that may be useful for primary care practitioners to use in assessing the patient safety culture in their organization, i.e. the MaPsaF primary care version, and the AHRQ. Patient safety culture as defined here is concerned with behaviour that ‘seeks to minimize patient harm which may result from the processes of delivery of care’ and which manifests itself in intangibles such a leadership, trust and communication. We also identified a small set of 16 process indicators likely to be appropriate and useful as indicators of patient safety culture in primary care organizations in Europe.

The five indicators rated as most appropriate included those related to maintenance of equipment, guidelines for hand hygiene and compliance with regulations in sterilizing equipment. It can be argued that these indicators all related to processes readily amenable to measurement, and so relate to a culture focussing on the use of metrics to assess progress against targets. We argue that ‘the practice collects data on patient adverse events’ is the indicator rated as offering the best combination of appropriateness and usefulness.

Problems with conceptualizing patient safety culture

During all of the main rating activities, the notion of patient safety culture seemed to be difficult for healthcare professionals to conceptualize. There was sometimes a tendency to revert to thinking about the type of performance metrics more often used in relation to performance in healthcare settings, such as number of adverse events or healthcare acquired infections. Healthcare professionals are familiar and comfortable with such indicators, as they may have been required to measure them for a number of years. For example, in considering the EuNetPaS inventory against the definition of culture adopted in this study, some of the tools in the inventory were deemed not to assess safety culture. It could be argued that one good way of assessing leadership would be to assess how far ‘the most senior person in the practice discusses patient safety and quality with the staff.’ It might be speculated that the fact that this item was rated among the least appropriate from the list of potential patient safety culture indicators suggesting that among primary care professionals in Europe, thinking about patient safety culture is still at an early stage.

Strengths and limitations

The main limitations of this study are the size of the sample, the limited number of tools assessing safety culture and the pool of candidate indicators identified. Notwithstanding these issues, it is suggested that any primary healthcare professionals interested in assessing the patient safety culture in their workplace should find the suggested tools useful, and might want to consider how far their practice includes some of the potential indicators listed in Table 2.

Conclusion

This study facilitated the identification of two tools for the measurement of patient safety culture in primary care, the medical office and nursing home versions of AHRQ, and the Manchester patient safety framework (MaPSaF, primary care version). A pool of 16 candidate process indicators of a positive patient safety culture in primary care has been developed. Further research is needed to establish the feasibility of their use.

FUNDING

The research leading to these results has received funding from the European Community’s Seventh Framework Programme FP7/2008–2012 under grant agreement no. 223424.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.