2,137
Views
9
CrossRef citations to date
0
Altmetric
Original Article

Re-thinking performance assessment for primary care: Opinion of the expert panel on effective ways of investing in health

, , , , , & show all
Pages 55-61 | Received 31 Aug 2018, Accepted 05 Nov 2018, Published online: 12 Dec 2018

Abstract

Background: In 2017, the European Commission (EC) identified as a policy priority the performance assessment of primary care systems, which relates to a country’s primary care structure, services delivery and outcomes. The EC requested its Expert Panel on Effective Ways of Investing in Health (Expert Panel) to provide an opinion on ways for improving performance assessment of primary care.

Objectives: To provide an overview of domains and dimensions to be taken into consideration in assessing primary care and specific indicators to be collected and analysed to improve understanding of primary care performance.

Methods: A sub-group of the Expert Panel performed a literature review. The opinion was drafted, improved and approved through working-group discussions, consultations with the EC, the Expert Group on Health Systems Performance Assessment, and a public hearing.

Results: Drawing on the main characteristics of primary care, we propose essential elements of a primary care performance assessment system based on specific indicators. We identified ten domains with accompanying dimensions for which comparative key indicators and descriptive indicators are proposed: (1) universal and accessible care, (2) integrated, (3) person-centred, (4) comprehensive and community-oriented care, (5) provided by a team accountable for addressing a vast majority of personal health needs, (6) sustained partnership with patients and informal caregivers, (7) coordination, (8) continuity of care, (9) primary care organization, and (10) human resources.

Conclusion: The identified characteristics and criteria for development of a primary care performance assessment system provides a starting point for strengthening the coherence of assessment frameworks across countries and exchanging best practices.

Key messages

  • Performance assessments of primary care systems should cover different core primary care domains measured by comparative and descriptive indicators.

  • Indicators need to be aligned with health system objectives and allowed for the routine collection of valid and reliable data to be relevant for a health system.

Introduction

Primary care can contribute to strengthening the overall health system’s performance by providing affordable, accessible and coordinated care; and by reducing avoidable hospital admissions [Citation1]. This was recently re-emphasized in the State of Health in the EU 2017 Companion Report [Citation2]. Well structured, organized and delivered primary care can play a fundamental role in improving not only population health but also population well-being since it covers both biomedical health needs and the broader contextual or social determinants of health such as social conditions, employment and environment. As such, primary care can be an effective tool to reduce inequities in societies [Citation3–5].

Measuring performance is a prerequisite for monitoring the achievement of a health system and serves to identify opportunities for improvement. Given the complexity of primary care systems (covering its structures, services delivery processes, and outcomes), measuring its performance at national, regional or local level is challenging. Policymakers need tools and methodologies to assess the performance of primary care, to understand how primary care operates and contributes to their health system benefitting all actors in primary care. Primary care providers, including general practitioners/family physicians, require information about relevance, acceptability, quality, person-centredness, effectiveness, and sustainability of their daily care for patients. Almost all countries in Europe carry out recurrent assessments on the performance of primary care in general, or on important parts of the primary care system. However, a wide variety of indicators are used across member countries to measure performance in primary care, and primary care providers are not always part of the developmental phase. It is very important that primary care providers are part of the development of indicator sets, to increase the validity of the performance data, and its potential to contribute to quality improvement initiatives at services delivery level. In many cases, the set of indicators available to policymakers are insufficient or focused on a subset of dimensions, as shown by a recent survey among 21 EU countries [Citation6]. Most countries reported collecting descriptive information about providers, access, and patient-centredness when assessing the performance of primary care. Only half of the countries reported measuring clinical performance. Aspects such as inequalities, workload and primary care workforce satisfaction are even less frequently reported. It could, therefore, be helpful at European level if guidance could be provided to countries on tools and methodologies for adequately assessing the performance of primary care at country level. For general practitioners (GPs), these tools and methodologies could provide a welcome help in its search of professional and scientific organization for useful quality indicators. This would also increase opportunities for international comparisons, and more importantly improve the contribution of primary care to health system outcomes in each country.

In 2017, the European Commission requested its Expert Panel on Effective Ways of Investing in Health (Expert Panel), comprising 14 experts from across the EU, to provide its views on (a) domains and dimensions to be taken into consideration in assessing primary care; and (b) specific indicators to be collected and analysed to give a better understanding of the performance of primary care [Citation7]. In this study, we report the key messages from the Expert Panel’s opinion on ways of assessing performance in primary care [Citation8].

Methods

A subgroup of the Expert Panel (authors) identified tools and methodologies based on the scientific literature, classical health system performance approaches applied by international organizations (e.g. OECD, WHO), and existing primary care performance frameworks (e.g. [Citation5,Citation9]). Draft versions of the opinion were regularly discussed with the responsible technical officers of the European Commission’s Directorate General on Health and Food Safety (DG SANTE), representatives of the Expert Group on Health Systems Performance Assessment, and with the broader Expert Panel to build on their professional experiences in primary care policy, practice and research. The resulting opinion was discussed in a public hearing and improved accordingly.

Results

A comprehensive definition of primary care captured by ten core domains

Drawing on the existing literature, in 2014 the Expert Panel defined the role and goals of primary care as ‘[…] the provision of universally-accessible, integrated, person-centred, comprehensive health and community services provided by a team of professionals accountable for addressing a large majority of personal health needs. The services are delivered in a sustained partnership with patients and informal caregivers, in the context of family and community, and play a central role in the overall coordination and continuity of people’s care […]’ [Citation10].

From this definition, eight domains of primary care can be identified, i.e. (1) universal and accessible, (2) integrated, (3) person-centred, (4) comprehensive and community-oriented, (5) services provided by a team of professionals accountable for addressing a large majority of personal health needs (quality), (6) sustained partnership with patients and informal caregivers, in the context of family and community, (7) coordination and (8) continuity of people’s care. Additionally, the domains of (9) primary care organization and (10) human resources need to be added since they are key determinants of the delivery of high quality, efficient and equitable primary care services. displays some of the key dimensions of each domain. General practitioners will especially recognize that the central aspects of their professional activities: being responsive for bio-psychosocial needs of patients, building relationships and continuity with patients, coordination of chronic care, and making sure that no one is left behind, are well represented in .

Table 1. Primary care domains and dimensions classified by structure, process and outcome.

To manage these domains from a systemic perspective, it has proven useful to apply Donabedian’s general assessment framework of structure-processes-outcomes [Citation11], which applies to every health system and setting (e.g. [Citation8,Citation12]). Each of the ten domains can be classified as a primary care structure, process or outcome, as shown in . This provides the opportunity to link the primary care setting with the structures, processes and outcomes of other parts of the health system (e.g. hospitals), and thus assess primary care’s overall contribution to health system outcomes.

Comparative key indicators and additional descriptive indicators

Following the definition of primary care and the identification of the core domains and accompanying dimensions, some indicators can be developed to capture the performance of primary care. The development of these indicators should aim to link primary care providers’ actions to performance results, which in turn allows the monitoring of the achievement of health system outcomes and the identification of future policy developments and improvements.

Indicators can be split into comparative key indicators and descriptive additional indicators. Comparative key indicators are those whose score may be evaluated in comparison with a target or a benchmark (e.g. waiting time for the first visit by a physician). Descriptive (observational) indicators are those whose score provides useful information for decision makers but whose interpretation is potentially ambiguous. For example, the extent to which GPs address mental health problems will not only depend on the comprehensiveness of the approach in the consultation (person-centredness), but also on the prevalence of mental health problems in their patient population (related to e.g. socioeconomic status) and on the fact whether or not there are other primary care providers (e.g. primary care psychologists) available. Nevertheless, this data provides useful information if correctly contextualized in a specific health system and to compare progress within a country over time.

Examples of indicators along the ten domains are provided in . The comprehensive list of indicators is accessible on the Expert Panel’s website at https://ec.europa.eu/health/expert_panel/home_en [Citation9].

Table 2. Examples of comparative key indicators by domains.

Discussion

In this section, we will discuss criteria to guide the selection of indicators, approaches to collect primary care performance measurement data, key aspects of successful development and implementation process of a performance assessment system and possible implications.

Criteria to guide the selection of indicators

Regardless of who is responsible for the selection of indicators (e.g. policy makers, scientific societies of GPs/family physicians, insurers), the choice of indicators should be guided by, at least, the following criteria: (i) alignment with policy objectives (indicators are to be informative about policy objectives defined by the health system); (ii) ability to routinely collect the information, either from administrative sources or from specifically designed surveys (indicators have more meaning with a time dimension to assess progress); and (iii) validity and reliability of information (indicators need to be based on credible sources and survey instruments need to be validated, for example). For each indicator, each criterion needs to be assessed.

In addition, a performance assessment system will need to cover a ‘reasonable’ number of indicators and targets for primary care. Either an excessive or a low number of performance indicators can result in a weak correlation between performance indicators and performance itself [Citation13]. The confusion generated by many targets might disorient the actors of the organization who may then behave differently from the priority actions. Alternatively, a minimal number of targets may induce tunnel vision as a consequence of narrowing the managerial attention only to some aspects of the global performance [Citation14]. Finally, an appropriate understanding and interpretation of the data often requires additional qualitative data collection e.g. describing patients’ expectations and experiences, apart from the quantitative data, measured through indicators. When comparing outcomes of primary care providers or systems, it will be crucial to include variation in context (e.g., data on characteristics of the population and society, the health system, the social welfare system) [Citation15].

How to collect performance measurement data

Evidence-based data collection and information provision should be the basis of any performance assessment system. This includes professional evidence (based on clinical practice experiences), contextual evidence (considering characteristics of the population, society, health system) and policy evidence (based on policy priorities and impact) [Citation16].

To minimize registration workload for primary care providers, there is a need to explore the use of existing data sets to collect performance measurement data. Whenever feasible, use of administrative data is collected directly from source databases instead of explicit reporting by institutions as it will speed up the collection process and decrease the possibility of errors. When using existing data sets attention will need to be paid to issues such as regulations and privacy (particularly for the individual patient level data) as well as standardization (especially for the population level data). For instance, the increasing potential of electronic patient data could be examined, including electronic prescribing systems that allow for the analysis of safety and effectiveness issues. Electronic patient records, as used by GPs, certainly when they use appropriate coding systems (e.g. International Classification of Primary Care, second edition) can provide extremely valuable information [Citation17]. Through the adoption of new and user-friendly ICTs, it is also increasingly possible to collect evidence directly from patients, users and citizens through systematic and continuous surveys (e.g. PREMs and PROMs).

Key aspects of successful development and implementation process

The existence of a performance assessment system, even though technical and scientifically sound, does not guarantee its adoption by policymakers and other stakeholders (e.g. GPs). Also, a performance management system can lead to dysfunctional performances (also called performance paradoxes) such as perverse learning—i.e. when organizations or individuals have learned how measurement works and manipulate their performance results [Citation13].

To limit the occurrence of these paradoxes and support a successful implementation and adoption of performance evaluation systems in health, it will be important that performance assessment is primarily applied as a tool to activate a positive comparison, discussion and learning process among primary care providers based on reputation and not on penalty mechanisms [Citation18]. Transparent disclosure of performance information will likely stimulate data peer review and learning and leverages professional reputation [Citation19,Citation20]. According to Hibbard et al., [Citation21] making performance information public stimulates long-term improvements, provided that performance assessment is appropriately contextualized (e.g. through information on case-mix). It is also vital that there is a feeling of ownership and acceptance of performance assessment outcomes by key stakeholders, such as GPs and other primary care providers, healthcare managers and policymakers. This can be created by involving potential users in the development phase of performance indicators [Citation22,Citation23]. Once indicators are selected, they should be based on recent data and fed back to its users (e.g. policymakers and primary care providers) on time, to preserve its actionability (i.e. relevance to act upon). Timeliness of performance data will allow policymakers to make decisions promptly (e.g. correct poor performance or dysfunctional behaviours), and primary care providers to improve the quality of care.

Implications

It is not expected that any health system will use all identified indicators but rather will select a set that are most relevant to it, taking account of defined criteria. Of course, a compilation of possible indicators is only a first step, and we fully recognize that much more needs to be done. The challenges in many countries are considerable, with primary care being provided by a different combination of health professionals, using a variety of organizational models, in dispersed locations, and often with poorly developed systems of data collection. Moreover, the performance of the primary care system cannot be seen in isolation, given its dependence on the rest of the health system and other sectors, such as social welfare. Nonetheless, the proposed indicators do offer a means to develop and strengthen the existing dialogue among those responsible for primary care across Europe. The European Commission could encourage the development of coherent performance assessment of primary care through the voluntary cooperation between the EU Member States in the exchange of best practice. It is clear that GPs and their societies/associations have a role to play in this process, particularly in the development, implementation, and data collection phases.

Conclusion

The identified characteristics and criteria for development of a primary care performance assessment system provides a starting point for strengthening the coherence of assessment frameworks across countries and exchanging best practices.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.

References

  • Starfield B. Primary care: an increasingly important contributor to effectiveness, equity and efficiency of health services. SESPAS report 2012. Gac Sanit. 2012;26:6.
  • European Commission. State of Health in the EU: Companion Report 2017. Luxembourg: Publications Office of the European Union; 2017.
  • De Maeseneer J, Willems S, De Sutter A. Primary health care as a strategy for achieving equitable care: a literature review commissioned by the Health Systems Knowledge Network. WHO Commission on the Social Determinants of Health. Health Systems Knowledge Network. Geneva: WHO; 2007.
  • Detollenaere J, Desmarest AS, Boeckxstaens P, et al. The link between income inequality and health in Europe, adding strength dimensions of primary care to the equation. Soc Sci Med. 2018;201:103–110.
  • Kringos DS, Boerma W, van der Zee J, et al. Europe’s strong primary care systems are linked to better population health but also to higher health spending. Health Affairs. 2013;32:686–694.
  • European Commission. A new drive for primary care in Europe: Report of the Expert Group on Health Systems Performance Assessment. Luxembourg: European Commission; 2018.
  • European Commission. Expert Panel Mandates [website]. European Commission. 2018. [cited 2017 June 01]. Available from: https://ec.europa.eu/health/expert_panel/mandates_en
  • Expert Panel on effective ways of investing in Health (EXPH). Opinion on tools and methodologies for assessing the performance of primary care. Luxembourg: European Union; 2018.
  • Schafer WL, Boerma WG, Kringos DS, et al. QUALICOPC, a multi-country study evaluating quality, costs and equity in primary care. BMC Fam Pract. 2011;12:115
  • Expert Panel on effective ways of investing in Health (EXPH). Report on Definition of a frame of reference in relation to primary care with a special emphasis on financing systems and referral systems. Luxembourg: European Union; 2014.
  • Donabedian A. The quality of care. How can it be assessed? JAMA. 1988;260:1743–1748.
  • Kringos DS, Boerma WG, Bourgueil Y, et al. The European primary care monitor: structure, process and outcome indicators. BMC Fam Pract 2010;11:81.
  • Van Thiel S, Leeuw FL. The performance paradox in the public sector. Public Perform Manag. 2002;25:267–281.
  • Nuti S, Vainieri M, Vola F. Priorities and targets: supporting target setting in healthcare. Public Money Manage. 2017;37:277–284.
  • Van Weel C, Tamblyn R, Turnbull D. Variation matters and should be included in health care research for comparison of outcomes. Prim Health Care Res Dev. 2017;18:183–187.
  • McColl A, Roderick P, Gabbay J, et al. Performance indicators for primary care groups: an evidence based approach. BMJ. 1998;317:1354–1360.
  • Soler JK, Okkes I, Wood M, et al. The coming of age of ICPC: celebrating the 21st birthday of the International Classification of Primary Care. Fam Pract. 2008;25:312–317.
  • Bevan G, Evans A, Nuti S. Reputations count: why benchmarking performance is improving health care across the world. Health Econ Policy Law. 2018;1–21. DOI:10.1017/S1744133117000561
  • Brown AD, Baker GR, Closson T, et al. The journey toward high performance and excellent quality. HCQ. 2012;15:6–9.
  • Nuti S, Vola F, Bonini A, et al. Making governance work in the health care sector: evidence from a ‘natural experiment’ in Italy. Hepl. 2016;11:17–38.
  • Hibbard JH, Stockard J, Tusler M. Hospital performance reports: impact on quality, market share, and reputation. Health Aff. 2005;24:1150–1160.
  • Spurgeon P, Mazelan PM, Barwell F. Medical engagement: a crucial underpinning to organizational performance. Health Serv Manage Res. 2011;24:114–120.
  • Ham C. Lessons from the past decade for future health reforms. BMJ. 2009;339:b4372.