Limited impact on patient experience of access of a pay for performance scheme in England in the first year

Abstract

Background: Improvement of access to general practice is a priority in England. In 2006/07 an annual national survey of patient experience of access was introduced, with financial incentives to practices based on the findings of the survey among their own patients. Objectives: To describe changes in patient experience of access over the first two years of the survey and incentive scheme, and identify respondent and practice characteristics associated with patient experience of access. Design and methods: The study included 222 general practices in the east of England, which had completed the access survey in 2006/07 and 2007/08. We compared proportions of patients reporting satisfaction with different aspects of access in each year. In explanatory regression models, we investigated the associations between improvement of reported access and respondent and practice characteristics. Results: There were some small improvements in reported access between the two surveys, although satisfaction with opening hours declined marginally. The explanatory analysis showed that larger practices, a higher proportion of respondents from ethnic minority groups, and higher deprivation were associated with patient reports of worse access. These variables and practice response rates did not explain the amount of change between the two years.

Conclusions: The launch of the incentive scheme was not followed by convincing improvements in patient experience of access. Practices with deprived populations or with a high proportion of ethnic minority survey respondents are perceived as offering worse access, were not more likely to achieve improvements, and additional support should be considered to help these practices.

Key words::

• Family practice
• patient satisfaction
• health care quality
• access and evaluation

Introduction

In England in recent years, access to general practice has been the subject of considerable interest, and sometimes controversy. A succession of policy initiatives has been employed, including an advanced access scheme with targets, the inclusion of indicators on access in the pay for performance quality and outcomes framework (QOF), and from 2006/07 an annual national patient survey that provides practice specific information on patient experience of access (1–3). Improved access to general practice has been confirmed as a priority of the new coalition government (4).

Practices can take part in the annual national patient survey, and they receive payment according to their performance as reported by their patients. In the first year of the scheme, practices received payment for taking part in the scheme and additional payment linked to the response of their patients to the survey. National funding available was £36 million to provide practices with a payment simply for taking part and £72 million based on patients’ responses, the level of this payment being determined by the percentage of patients in a practice giving a positive response (5). Payment per registered patient of £1.37 was made if the highest satisfaction thresholds for all four survey indicators was reached (the thresholds for different indicators were either 80% or 90% of patients reporting satisfaction), with correspondingly lower payments for lower levels of satisfaction above a minimum threshold (between 20% and 50% satisfied, according to indicator). Thus, a practice with 10 000 patients would have received £13 700 if they had achieved the maximum satisfaction thresholds. In contrast, the quality and outcomes framework provides a larger financial incentive, up to 25% of practice income (1). The survey was completed in two consecutive years using the same survey instrument and it is possible to determine whether there has been an improvement in reported access (6). In 2009, a new survey instrument was used, the questions not being comparable with the first two years, making a comparison over three years impossible.

Access to general practice is a complex concept that may encompass the supply of services, the ability of patients to call on services as required, and the relevance of services in relation to need (7,8). With respect to simply calling on services as required, it may include not only the ability to see a doctor or nurse urgently or within a short period, but also the ability to see a doctor or nurse of the patient's choice, to book ahead to ensure an appointment on a convenient day, being able to consult a health professional at a convenient time of day, the availability of a wide range of services, the ability to readily contact the practice by telephone, and talk with a health professional by telephone or e-mail (9). The survey questionnaire used in the first two years of the scheme reflected government policy to improve the ability of patients to obtain appointments at their general practice (10), and therefore, it addressed only a limited aspect of the concept of access, restricted to elements of the ability to call on services as required. The way the appointment system is operated, the total number of appointments available, and the staffing of the telephone lines or the number of practice nurses will influence patients’ ability to call on services as required. Characteristics of the patients who respond to the survey will also influence their views on access (11,12). For example, older patients, or parents with young children, will have different preferences for access. Ethnic minority groups tend to report particularly poor experience of access (13,14).

We report a study undertaken to describe changes in patient reports of access between the first two years of the survey, and identify features of practices associated with changes in experience of access. Our hypotheses were that following the introduction of the financial incentive scheme, patient reports of access would improve between the two years of the survey, and that improvement would be greatest among practices with higher proportions of deprived or ethnic minority patients.

Methods

Access survey data

We included all practices in three local primary care trusts in the east midlands of England (Leicester City, Leicestershire County and Rutland, and Northamptonshire) that had completed the access survey in both 2006/07 and 2007/08. The 2006/07 patient access survey was administered by a market research company Ipsos MORI (15) to around 5.2 million adults aged 18 years and over, registered with 8241 practices in England, and the 2007/08 survey to 4.9 million patients registered with 8307 practices (16,17). There were 2.3 million responses (44% national response rate) in 2006/07 and 2.0 million (43%) in 2007/08. On average in 2006/07, 432 patients per practice were invited to participate in the survey questionnaire. The samples were systematic (a ‘1 in n’ basis), and for each practice were taken from either the primary care trust registration records or, in 2006/07 only in some practices, the practice computer system. The survey was primarily administered by post, but respondents were also offered online questionnaire completion and telephone completion, including the option of 10 non-English languages.

The access survey questionnaire asked five questions, one each about: being able to get an appointment within two working days, being able to book an appointment in advance (more than two days in advance), being able to get an appointment with a particular general practitioner, satisfaction with opening hours, and satisfaction with getting through to the practice on the telephone. The questions were in a yes/no closed format, and answers were requested only from patients who had used the service in the previous six months. Information was also sought about the respondents’ gender, age, ethnic group, and numbers of consultations in the past year. The data for each practice, including respondent characteristics, are available from the Department of Health web site (18).

Explanatory data

In addition to the access survey data, we sought to obtain available information about features of practices that might explain reported improvement of access. Practice characteristics included numbers of full time equivalent GPs (provided by the NHS Information Centre (19)), and whether the practice was approved as a training practice (information provided by the local training office or deanery). From publicly available quality and outcomes framework data for the years 2006/07 and 2007/08 we obtained total numbers of patients per practice (enabling calculation of numbers of patients per whole time equivalent GP), and to indicate levels of morbidity in practice populations, a factor that might influence demand for access, data on the proportions of patients recorded on practice registers as having coronary heart disease (19). Census derived health measures have been shown to correlate with practice coronary heart disease registers (20). We used the index of multiple deprivation (IMD) 2007 as an indicator of the socioeconomic status of practice populations, a higher score indicating higher socioeconomic deprivation (21).

Analyses

Access survey. We used paired t-tests to compare differences in practice access scores between years.

Explanatory analysis. We undertook regression modelling to explain differences between practices in reported patient experience in 2007/08. We used general linear models with each of the five access questions as dependent variables. Rationalized arcsine transformations (22) were applied to each dependent variable before modelling to improve normality. A problem with the traditional arcsine transformation is that the arcsines do not have an obvious relationship with the proportions that have been transformed. Utilizing the rationalized method enables them to be interpreted as if they were proportions, whilst retaining the desirable mathematical properties of the arcsine transform. The rationalized arcsine transform (or R) is a simple linear transformation of the arcsine transform, described by the following equation: R = 46.47 T – 23, where T is the traditional arcsine transformation.

The following patient characteristics were modelled: percent of respondents aged 65 years or older, gender (proportion of male respondents), proportion in any non-white ethnic group, proportion with five or more appointments with a general practitioner in the past year, and proportion of patients recorded on the practice coronary heart disease register. We also modelled practice characteristics (IMD 2007, number of patients per full time equivalent GP, whether or not a training practice, number of patients, responsible primary care trust, and response rate of practice patients to the survey). We also investigated the effect of changes in practice response rates between surveys on changes in survey findings in general linear models that in addition to change in response rate also included the variables found to explain the 2007/08 survey findings.

Results

Access survey

A total of 222 practices took part in the two annual surveys. The characteristics of the practices and responding patients taking part in the surveys are shown in Table I. In 2006/07 the mean response rate for practices in our sample was 49.5% (SD 10.8%), but this fell to 44.5% (SD 9.4%) in 2007/08. Between 2006/07 and 2007/08 there were small improvements in patients’ reports of getting an appointment within 48 hours, booking in advance, and telephone access, but there was no change in being able to see a particular doctor, and satisfaction with opening hours declined (Table II). Practices with 35% or more of patients from ethnic minority groups did not demonstrate larger improvements in patient experience.

Table I. Median and interquartile range (IQR) of practice characteristics and mean and standard deviation (SD) of characteristics of patients responding to the access surveys, 2006/07 and 2007/08.

Practices (n = 222)	2006/07 Median (IQR)	2007/08 Median (IQR)
Patients per practice	6359 (3718, 10185)	6553 (3853, 10291)
Patients per fte GP	1799 (1455, 2203)	1821 (1456, 2216)
IMD 2007^a	15.8 (9.7, 27.0)	15.8 (9.7, 27.0
Training practices (n)	43	43
Patients	Mean (SD)	Mean (SD)
Mean practice response rate (%)	49.5 (10.8)	44.5 (9.4)
% of respondents aged 65 years or older	29.6 (8.9)	28.0 (7.9)
% of respondents who were male	41.9 (5.0)	43.8 (4.8)
% of respondents reporting non–white ethnicity	14.2 (24.7)	14.2 (24.6)
Mean number of patients per practice on practice coronary heart disease registers	234.7 (162.9)	236.8 (162.7)
% of survey respondents reporting five or more appointments with a general practitioner in the past year	32.9 (8.8)	28.0 (7.5)

^aHigher scores for IMD 2007 indicate greater socioeconomic deprivation.

Table II. Patient experience of access 2006/07 and 2007/08 (mean percentages and 95% confidence intervals of percentages of patients per practice reporting positive experiences).

	All practices (n = 222)			Practices with ≥35% of patients from ethnic minority groups (n = 24 practices)
	2006/07 (95% CI)	2007/08 (95% CI)	Diff (95% CI)	2006/07 (95% CI)	2007/08 (95% CI)	Diff (95% CI)
% satisfied with phone access	83.8 (81.9, 85.6)	85.1 (83.5, 86.7)	1.31 (0.45, 2.18)	72.8 (66.4, 79.2)	75.5 (69.2, 81.8)	2.96 (0.22, 5.69)
% able to get appointment within 48 h	87.0 (85.8, 88.2)	88.3 (87.2, 89.4)	1.25 (0.44, 2.07)	82.4 (79.0, 85.8)	83.3 (79.8, 87.8)	1.12 (−1.22, 3.47)
% able book advance	68.4 (65.5, 71.2)	72.1 (69.6, 74.5)	3.69 (2.42, 4.97)	63.0 (56.2, 69.9)	67.8 (60.9, 74.7)	4.42 (−0.19, 9.03)
% able to see a particular GP	85.4 (84.1, 86.7)	85.2 (84.0, 86.5)	−0.16 (−0.93, 0.61)	73.4 (69.2, 77.7)	75.2 (70.2, 80.2)	1.21 (−2.62, 5.04)
% satisfied with opening hours	83.4 (82.4, 84.3)	81.1 (80.2, 82.0)	−2.28 (−2.8, −1.7)	75.4 (71.3, 79.6)	74.4 (70.2, 78.5)	−1.25 (−3.46, 0.96)

Explanatory models

In the regression models (Table III), undertaken to identify factors explaining differences between practices in reported access, worse experience of access in 2007/08 was associated with higher proportions of ethnic minorities (the aspects of access concerned were: being satisfied with opening hours, being able to see a particular doctor, being able to see a doctor within 48 h, and satisfaction with telephone access), increasing practice size (being satisfied with opening hours, being able to see a particular doctor, being able to see a doctor within 48 h, satisfaction with telephone access, and being able to book in advance), and greater deprivation (being able to see a particular doctor, being able to see a doctor within 48 h, satisfaction with telephone access, and being able to book in advance). There was no significant correlation between changes in practice response rates between the two surveys and changes in patient reports of access. In the general linear model analyses investigating variables associated with changes in response between years, neither change in response rate nor any of the variables that predicted the responses to the 2007/08 survey (practice size, proportion of ethnic minority respondents, deprivation) predicted the changes between years in response to any of the five survey questions.

Table III. Prediction of access scores in 2007/08 by patient and practice characteristics (general linear models, backward stepwise regression used to eliminate non-significant predictors, rationalized arcsine transformation of dependent variables).

	β (95% CI)	P	Interpretation of coefficient^a
Satisfied with opening hours
% Ethnic minorities	−0.14 (−0.18, −0.10)	<0.0001	−1.4%
Practice list size	−0.0003 (−0.0005, −0.0001)	<0.0001	−0.3%
% five or more appointments	0.24 (0.14, 0.35)	<0.0001	2.4%
r^2 0.25
% able to see a particular doctor
% Ethnic minorities	−0.1 (−0.15, −0.04)	0.0002	−1.0%
IMD 2007	−0.24 (−0.35, −0.13)	<0.0001	−2.4%
Practice list size	−0.0005 (−0.0007, −0.0002)	<0.0001	−0.5%
r^2 0.27
% able to book an appointment in advance
IMD 2007	−0.32 (−0.48, −0.16)	<0.0001	−3.2%
Practice list size	−0.0007 (−0.001, −0.0003)	0.0005	−0.7%
r^2 0.09
% satisfied with phone access
% Ethnic minorities	−0.12 (−0.19, −0.04)	0.001	−1.2%
IMD 2007	−0.29 (−0.43, −0.14)	0.0001	−2.9%
Practice list size	−0.0009 (−0.002, −0.0005)	<0.0001	−0.9%
r^2 0.26
% able to see a GP in 48 h
% Ethnic minorities	−0.06 (−0.12, −0.008)	0.03	−0.6%
IMD 2007	−0.19 (−0.31, −0.07)	0.002	−1.9%
Practice list size	−0.0005 (−0.0008, −0.0002)	<0.0001	−0.5%
r^2 0.15

For predictors which are percentages (ethnic minorities and five or more appointments) the column 3 value represents the change in the per cent of patients satisfied when there is a 10% increase in the predictor variable.

For IMD 2007, the column 3 value represents the change in the percent satisfied when there is a 10 unit increase in IMD 2007.

For Practice List size, the column 3 value represents the change in the percent satisfied when there is a 1000 persons increase in the practice list size.

^aColumn 3.

Discussion

Principal findings

There was variation between practices in patient reported access, with larger practices, practices with greater socioeconomic deprivation and practices with higher proportions of ethnic minority patients being more likely to be perceived as delivering poorer access. There were some modest improvements in access between surveys, but although some changes were statistically significant, the practical significance to patients is uncertain. There was no evidence that improvement in patients’ perceptions of access were more likely in practices with high proportions of ethnic minority or deprived patients. Although there was also variation in practice response rates, we did not find evidence that this explained the change in responses to the survey. This finding is supported by a study using the new 2008/09 survey, in which low response rate and selective non-response bias were not found to lead to unfairness in assigning questionnaire scores to practices (23). Being able to book an appointment in advance attracted lower levels of satisfaction than the other questions. This aspect of care may reflect the ability of patients to obtain personal continuity, but since satisfaction was high with being able to make an appointment with a particular doctor, the findings suggest that patients are generally content with this aspect of continuity, although they may need to actively negotiate with practices to obtain the appointments they prefer (9).

Strengths and limitations

Although our study is restricted to 222 practices in one area in England, the locality included wide socioeconomic and ethnic diversity. Nevertheless, caution is required in generalizing the findings to different areas and to other countries. The study relied on publicly available data, and therefore, the information on practices and patients was relatively limited. For example, there were no data available on how practices operated their appointment systems, nor the total number of appointments they offered each week. Furthermore, only two successive annual surveys were included. The survey questionnaire itself was designed to address an important government policy, and therefore was narrowly focused rather than accounting for the variety of ways patients frame access according to their particular needs (7,8). The response rate to the survey varied between practices and, although higher than the national rate, it was below 45% in 2007/08. The characteristics of the non-responding patients are unknown, and they may have different perceptions of access to general practice. The proportion of variance explained by the regression models was around 25% for three of the access questions, but lower for the other questions. Therefore, other factors not included in the study are influencing patient reports of access. In addition, our study design does not allow conclusions on causation. Nevertheless, despite these qualifications, the study provides information about patients’ experiences of access and initial evidence from a defined area of England that the incentive scheme was not associated with a meaningful improvement in patient reports of access.

Interpretation

The finding of this study is interesting because it suggests that the financial incentive was not particularly effective with respect to access. Other studies have shown that larger practices tend to provide worse access than smaller ones, and that ethnic minority groups and patients in more deprived areas report worse access, but there was little progress in the first years of the incentive scheme (11). Evidence from California suggests that incentive schemes can promote improved patient experience (24). Nevertheless, our findings on access are different to those on the clinical indicators of the quality and outcomes framework, when improvements took place in the first year of the scheme (1). This difference might be explained by the much larger financial incentive of the quality and outcomes framework (up to 25% of practice income) compared to the access incentive scheme; it is also possible that general practitioners regarded achievement of clinical indicators as more important than enabling patient satisfaction with access. Since a substantial increase in the financial incentive for access is unlikely, additional interventions are necessary, for example, support in re-designing appointment systems or assistance with investment in additional clinical staff. Since the circumstances and patient populations of practices vary, interventions tailored to the needs of each practice may be required.

Ethical approval

This was not required since publicly available data were used.

Acknowledgements

The authors thank Professor Mayur Lakhani for advice on access and ethnic minority groups. Dr Bankart and Professor Baker are supported by the NIHR CLAHRC for LNR. The views expressed in this paper are not necessarily those of the NIHR.

Declaration of interest: Professor Baker was lead of a national expert group on patient experience indicators for the QOF. There are no other competing interests.