Abstract
Objective. For over a decade, out-of-hours primary care in the Netherlands has been provided by general practitioner (GP) cooperatives. In the past years, quality improvements have been made and patients have become acquainted with the service. This may have increased patient satisfaction. The objective of this study was to examine changes in patient satisfaction with GP cooperatives over time. Design. Longitudinal observational study. A validated patient satisfaction questionnaire was distributed in 2003–2004 (T1) and 2007–2008 (T2). Items were rated on a scale from 0 to 10 (1 = very bad; 10 = excellent). Setting. Eight GP cooperatives in the Netherlands. Subjects. Stratified sample of 9600 patients. Response was 55% at T1 (n = 2634) and 51% at T2 (n = 2462). Main outcome measures. Expectations met; satisfaction with triage nurses, GPs, and organization. Results. For most patients the care received at the GP cooperative met their expectations (T1: 86.1% and T2: 88.4%). Patients were satisfied with the triage nurses (overall grade T1: 7.73 and T2: 7.99), GPs (T1: 8.04 and T2: 8.25), and organization (overall grade T1: 7.60 and T2: 7.78). Satisfaction with triage nurses showed the largest increase over time. The quality and effectiveness of advice or treatment were given relatively low grades. Of all organizational aspects, the lowest grades were given for waiting times and information about the cooperative. Conclusion. In general, patients were initially satisfied with GP cooperatives and satisfaction had even increased four years later. However, there is room for improvement in the content of the advice, waiting times, and information supply. More research is needed into satisfaction of specific patient groups.
Patient experiences are an important element of quality of care. This study showed that a few years after the onset of GP cooperatives in the Netherlands, patients were satisfied with this out-of-hours primary care service. On most aspects, satisfaction had even increased four years later. However, there is still room for improvement in the content of the advice, waiting times, and information supply.
Introduction
Around the year 2000, Dutch general practitioners (GPs) started reorganizing out-of-hours primary care from small rotation groups into large-scale GP cooperatives. In these cooperatives, 50 to 250 GPs take care of populations ranging from 100 000 to 500 000 citizens [Citation1,Citation2] (see Box I for more characteristics). Other Western countries have also made this shift, for instance Denmark and the United Kingdom [Citation3–5]. Reasons for this reorganization include the low personal commitment of GPs to be on call, the increasing workload with many non-urgent requests for help, and the shortage of GPs in some countries [Citation1–3,Citation6–8].
In order to provide patient-centred care, GP cooperatives should include patient experiences in their quality policies. Previous studies have shown that patients were satisfied with Dutch GP cooperatives a few years after their onset [Citation9,Citation10]. In the past years, GP cooperatives evaluated their organization on various areas, including guideline adherence [Citation11], accessibility [Citation12], quality of telephone triage [Citation13–15], and patient safety [Citation16] and they are continuously improving quality of care based on these evaluations. In addition, patients and professionals have become acquainted with GP cooperatives and the service they provide. These developments may have had positive effects on patient satisfaction, but it is not known whether patient satisfaction has changed over time. We examined changes in patient satisfaction with GP cooperatives in the Netherlands over time. In addition, we examined which patient and contact factors could predict dissatisfaction.
Material and methods
Study design and setting
We conducted a longitudinal observational study into patient satisfaction with GP cooperatives providing out-of-hours primary care. The study was performed in a convenience sample of eight GP cooperatives, located across the Netherlands. Using a patient questionnaire, patient satisfaction was measured in two time periods: 2003–2004 (T1) and 2007–2008 (T2) in different patient groups. At T1, the GP cooperatives had existed for one to four years, while T2 was four years later. A stratified sample of 9600 patients was invited to participate in the study. Stratification was based on type of contact: the questionnaire was sent to an equal number of patients who had received telephone advice, a health centre consultation, or a home visit.
Questionnaire
A previously developed and validated questionnaire was used to measure patient satisfaction [Citation17]. The questionnaire consisted of background items, an item on care expectations, and items on patient satisfaction. The items on patient satisfaction were divided into three sections: telephone triage nurse, GP, and organization. These sections consisted of different items measuring various aspects of the contact and one item measuring overall satisfaction. The sections regarding the triage nurse and GP consisted of items on communicative and medical skills. Patients were asked to rate each item on a scale from 1 to 10 (grade 1 = very bad; grade 10 = excellent). For patients who received telephone advice only, the items on satisfaction with the GP were not scored.
Data collection procedure
For each cooperative and at both T1 and T2, 600 patients received the questionnaire equally spread out over three contact types: telephone advice (200), health centre consultation (200), and home visit (200). The questionnaire was sent to the patient's home between two and four days after the GP cooperative contact, including a return envelope. In the case of non-response, a reminder was sent after 10 days [Citation17] Administrative contacts and deceased patients were excluded.
Statistical analysis
Mean scores, standard deviations (SD), and 95% confidence intervals (95% CI) were calculated for each item. In addition, the percentage of negative evaluations (grades ≤ 6) was calculated, with corresponding 95% CIs.
Differences in patient and contact characteristics between T1 and T2 were analysed with a t-test (for gender) and chi-squared tests (for age group and contact time). To compare grades at T1 and T2, Mann–Whitney U-tests were used, with differences being considered statistically significant at p < 0.05. Patients with more than 50% of all grades ≤ 6 were indicated as dissatisfied patients. Three multivariate logistic regression analyses were performed to examine whether age, sex, contact time, and type of contact could predict patient dissatisfaction, using all items at T1 and T2. Data were analysed using SPSS 16.0.
Box 1. Features of general practitioner (GP) cooperatives in the Netherlands [Citation1].
Results
Patient characteristics
The response rate at T1 was 55% (n = 2634) and at T2 51% (n = 2462). The response rate for patients with telephone advice was 49% (n = 1565), for health centre consultations 52% (n = 1667), and for home visits 58% (n = 1864). The majority of contacts took place in the evening (T1: 46.9% and T2: 44.8%). About half of the respondents were female (T1: 52.6% and T2: 51.7%). Respondents at T2 were more often above the age of 65 ().
Table I. Patient and contact characteristics.
Patient expectations
For most patients the care received at the GP cooperative met their expectations: T1: 86.1% and T2: 88.4% (not in table). The percentage of mismatches of expectation and perceived care (e.g. patients who expected to receive a health centre consultation but who received a telephone consultation) was 13.9% at T1 and 11.6% at T2.
Satisfaction with triage nurse
Satisfaction with the telephone triage nurse significantly increased on all aspects between T1 and T2. The triage nurse received an overall grade of 7.73 at T1 and 7.99 at T2. At both T1 and T2, patients gave the highest grades on the communication skills “Taking patient seriously” (T1: 7.86 and T2: 8.12) and “Taking time to talk” (T1: 7.86 and T2: 8.07), whereas the medical items “Effectiveness of advice” (T1: 7.13 and T2: 7.47) and “Quality of advice” (T1: 7.32 and T2: 7.67) were given the lowest grades.
The percentage of scores ≤ 6 per item ranged from 11.3% to 28.8% at T1 and from 10.4% to 23.2% at T2. Between T1 and T2 the percentage of scores ≤ 6 significantly decreased on the aspects “Professionalism”, “Taking patient seriously”, “Understanding problem”, “Reassurance”, “Feasibility of advice”, and “Effectiveness of advice” ().
Table II. Satisfaction with the triage nurse: mean score and percentage of negative evaluations on T1 and T2.
Satisfaction with GP
There was a significant increase in satisfaction with the GP on all items. Patients gave the GP an overall grade of 8.04 at T1 and 8.25 at T2. The GP received the highest grades on the communication skills “Taking patient seriously” (T1: 8.16 and T2: 8.29) and “Friendliness” (T1: 8.12 and T2: 8.27) and the lowest grades on the medical skill “Effectiveness of advice or treatment” (T1: 7.69 and T2: 7.93). The percentage of scores ≤ 6 ranged from 8.1 % to 17.2% at T1 and from 7.0% to 15.4% at T2. There were no significant differences in the percentage of scores ≤ 6 between T1 and T2 ().
Table III. Satisfaction with the GP: mean score and percentage of negative evaluations on T1 and T2.
Satisfaction with organization
There was a significant increase in satisfaction on the items “Information about GP cooperative”, “Time between contact and consultation”, and the overall grade and a slight decrease in “Accessibility by telephone” and “Time in waiting room”. Patients gave the organization an overall grade of 7.60 at T1 and 7.78 at T2. They gave the highest grades on “Accessibility by telephone” (T1:7.82 and T2:7.78). The lowest grades were given on “Information about GP cooperative” (T1:7.12 and T2:7.36) and “Time in waiting room” (T1:7.42 and T2:7.27). The percentage of scores ≤ 6 ranged from 13.8% to 28.1% at T1 and from 15.3% to 24.3% at T2. Between T1 and T2, the percentage of scores ≤ 6 significantly decreased on “Information about GP cooperative” and the overall grade ().
Table IV. Satisfaction with the organization: mean score and percentage of negative evaluations on T1 and T2.
Predictors of dissatisfied patients
Of all patients, 11.6% (n = 569) expressed their dissatisfaction on more than half of the items regarding the triage nurse, 14.6% (n = 712) regarding the organization, and 8.2% (n = 278) regarding the GP.
Patients older than 65 years were less often dissatisfied with the triage nurse, GP, and organization (OR = 0.55, OR = 0.58, and OR = 0.66 respectively). Parents of patients under the age of five were less often dissatisfied with the triage nurse (OR = 0.68). Male patients were less often dissatisfied with the GP (OR = 0.69). Patients who had received telephone advice were more often dissatisfied with the triage nurse (OR = 1.78) and organization (OR = 2.05). Patients with home visits were more often dissatisfied with organizational aspects (OR = 1.61; ).
Table V. Predictors of dissatisfaction with telephone nurse, GP, and organization.
Discussion
Principal findings and interpretation
A few years after the onset of GP cooperatives, patients expressed satisfaction with the triage nurses, GPs, and organization of the cooperatives, and on almost all aspects satisfaction even increased four years later.
There were two negative trends: satisfaction with the time in the waiting room and accessibility by telephone showed a slight, but non-significant, decrease between the two measurements.
The lowest scoring items were the time in the waiting room and information about the cooperative. The highest grades were given for the communicative skills of the triage nurses and GPs, whereas medical skills – the quality and effectiveness of the advice and treatment – received lower scores.
Patients who received telephone advice were more often dissatisfied with the triage nurse and the organization than patients who had a consultation at the cooperative. This might be related to the nurses’ gate-keeping function on the phone, as Dutch patients access the GP cooperative mostly via a regional telephone number (see also Box I). The triage nurses can provide self-care advice, even if the patient was expecting a GP consultation, Previous research has shown that a mismatch between expectations and actual care is associated with a negative evaluation [Citation9].
There were fewer dissatisfied respondents among (parents of) young children, the elderly, and male patients. As there were more elderly patients at T2, this could have contributed to the increase in satisfaction scores.
The aim of our study was to examine patient satisfaction with GP cooperatives over the years. We did not compare the results with the situation before the reform of out-of-hours care. Patients could have been even more satisfied in the former system of small rotation groups, but the reform was mainly introduced to reduce the workload of GPs and to manage the shortage of GPs [Citation1]. The considerably high and even increasing satisfaction scores in our study seem to show that patients have accepted the reform. We cannot conclude that satisfaction is optimal, as it is unknown what maximum level is achievable.
Over the past years, patients have been informed about the working routines of GP cooperatives and they have got used to the changed situation. This may have positively influenced patients’ opinions, especially regarding triage nurses. Initially, there were concerns about the role of triage nurses [Citation18]. Patients were anxious about their ability to describe symptoms over the telephone, or to understand and follow up the advice received [Citation19]. In the course of the years, patients have become acquainted with the function of the triage nurse, and the positive experiences may have resulted in an increased appreciation. Moreover, over time, the clinical experience of triage nurses has increased and implementation of certified education for triage nurses started in 2007. During T2, implementation of this programme was still going on, increasing the number of certified nurses. On the other hand, satisfaction is a subjective measure: satisfaction may increase if expectations decrease due to social or other changes.
Strengths and weaknesses
A large number of patients participated in this multicentre longitudinal study, contributing to the representativeness of the results. The response rates were similar to response rates in other patient survey studies in this health care sector [Citation5,Citation9,Citation10, Citation20–22]. However, we did not perform a non-response analysis. Therefore we do not know if patients who responded to the questionnaire differ from patients who did not. An extensive non-response analysis in a previous study, which partly used the same questionnaire, did not reveal any important differences between the response and non-response groups. Also, no relation was found between the response rate of the participating GP cooperatives and the satisfaction scores [Citation17]. We do not have information on the group of non-responders in our study. Compared with the total group of patients contacting a GP cooperative, reported for one region in the Netherlands [Citation23], there were more elderly people in our study (about 30% in our study versus 19%) and fewer children (about 20% versus 28%). The gender distributions were almost the same (about 47% male patients in our study versus 43%).
Although most of our results were statistically significant, the actual differences in grades between the two measurements were small: sometimes only one or two tenths of a point on a 10-point scale. This reduces the clinical relevance of our findings. However, because there was an increase in scores on almost all aspects at the second measurement (T2), we believe we have found evidence that patient satisfaction has improved over time.
Furthermore, the timing of the first measurement (T1) varied from one to four years after the inception of the GP cooperative. It is unclear how this influenced the results. A study by Christensen and Olesen showed that satisfaction decreased just after the onset of GP cooperatives, but it began to increase again as people got used to the service [Citation4]. Perhaps the differences between T1 and T2 in our study would have been even larger if all GP cooperatives had had their first measurement only one year after the start of the cooperative.
Finally, the questionnaire contained items on a broad range of topics relevant to GP cooperatives. Our study showed high satisfaction on these themes. Nevertheless, patients may be less satisfied with factors and conditions which have not been asked about in the questionnaire.
Comparison with other studies
One study, by Christensen and Olesen, also reported longitudinal results on patient satisfaction with GP cooperatives [Citation4]. One year after the onset of GP cooperatives in Denmark, 73% of the respondents were (partly) satisfied. Similar to our results, they found an increase in satisfaction three years later (81% (partly) satisfied). Four years after the onset of GP cooperatives in Denmark, 86% of the patients received the type of service they had expected and these findings are comparable to our results [Citation4].
Other studies, using cross-sectional study designs, have also shown comparable results. In a study by Giesen et al., conducted in 26 GP cooperatives in the Netherlands, which included the T1 data of telephone consultation patients from our study, the percentage of negative evaluations (scores ≤ 6) ranged from 12% to 33% for aspects concerning triage nurses and from 19% to 35% for aspects concerning organization [Citation9]. Moreover, Van Uden et al. reported an overall satisfaction rate of 76% for telephone advice and 80% for GP consultation or home visit in Dutch GP cooperatives [Citation19]. In the United Kingdom, various studies have shown that patients are satisfied with the GP cooperative, especially with regard to health centre consultations and home visits by GPs [Citation5,Citation21,Citation24].
Implications for future research and clinical practice
The results of our study indicate that in general patients are satisfied with the GP cooperative, but there is room for improvement concerning the medical skills of the professionals (i.e. the quality and effectiveness of the advice and treatment), as these skills received lower scores than communicative skills. Future studies should examine in more detail which elements of the medical skills can be improved, and education of professionals should focus on these skills. Moreover, the waiting time at the GP cooperative is an important issue to tackle in quality improvement initiatives, as patients gave low absolute scores on this item and the scores even decreased over the years. Finally, professionals should be trained to make expectations of care more explicit during the first contact with the GP cooperative, to improve patient satisfaction.
The second measurement in our study was made about five years ago. In recent years, additional developments have taken place in out-of-hours care and GP cooperatives, such as increasing collaboration with hospital emergency departments with a shared entrance, increasing education and certification of triage nurses, continuing implementation of computerized triage systems to support decisions, and use of supervising telephone GPs. We recommend further measurements of patient satisfaction with GP cooperatives, to obtain up-to-date results which take recent developments into account and to examine whether satisfaction still increases, stabilizes, or decreases.
Finally, more research is needed into the satisfaction of specific patient groups, such as the elderly, chronically ill, and non-natives, preferably using both quantitative and qualitative research methods. In this way, the effect of specific measures can be evaluated and optimal patient-centred care can be achieved.
Declaration of interest
The authors report no conflict of interest. The authors alone are responsible for the content and writing of the paper.
This work was financially supported by the participating GP cooperatives.
References
- Giesen P, Smits M, Huibers L, Grol R, Wensing M. Quality of after-hours primary care: As narrative review of the Dutch solution. Ann Intern Med 2011;155:108–113.
- Grol R, Giesen P, Van Uden C. After-hours care in the United Kingdom, Denmark and the Netherlands: New models. Health Aff 2006;25:1733–7.
- Hallam L. Primary medical care outside normal working hours: Review of published work. BMJ 1994;308:249–53.
- Christensen MB, Olesen F. Out of hours service in Denmark: Evaluation five years after reform. BMJ 1998;316:1502–5.
- Shipman C, Payne F, Hooper R, Dale J. Patient satisfaction with out-of-hours services: How do GP co-operatives compare with deputizing and practice-based arrangements?J Public Health Med 2000;2:149–54.
- Leibowitz R, Day S, Dunt D. A systematic review of the effect of different models of after-hours primary medical care services on clinical outcome, medical workload, and patient and GP satisfaction. Fam Pract 2003;20:311–17.
- Salisbury C. Evaluation of a general practice out of hours cooperative: A questionnaire survey of general practitioners. BMJ 1997;314:1598–9.
- Heany D, Gorman M, Porter M. Self-recorded stress levels for general practitioners before and after forming an out-of-hours primary care centre. Br J Gen Pract 1998;48:1077–8.
- Giesen P, Moll van Charante E, Mokkink H, Bindels P, Van den Bosch W, Grol R. Patients evaluate accessibility and nurse telephone consultations in out-of-hours GP care: Determinants of a negative evaluation. Pat Educ Couns 2007;65:131–6.
- Van Uden CJT, Ament AJHA, Hobma SO, Zwietering PJ, Crebolder HFJM. Patient satisfaction with out-of-hours primary care in the Netherlands. BMC Health Serv Res 2005;5:6.
- Willekens M, Giesen P, Plat E, Mokkink H, Burgers J, Grol R. Quality of after-hours primary care in the Netherlands: Adherence to national guidelines. BMJ Qual Saf 2011;20:223–7.
- Giesen P, Van Lin N, Mokkink H, Van den Bosch W, Grol R. General practice cooperatives: Long waiting times for home visits due to long distances?BMC Health Serv Res 2007;7:19.
- Giesen P, Ferwerda R, Tijsen R, Mokkink H, Drijver R, Van den Bosch W, . Safety of telephone triage in general practitioner cooperatives: Do triage nurses correctly estimate urgency?Qual Saf Health Care 2007;16:181–4.
- Hildebrandt DE, Westfall JM, Smith PC. After-hours telephone triage affects patient safety. J Fam Pract 2003;52: 222–7.
- Derkx HP, Rethans JJ, Muijtjens AM, Maiburg BH, Winkens R, Van Rooij HG, . Quality of clinical aspects of call handling at Dutch out of hours centres: Cross sectional national study. BMJ 2008; 337: 676–9.
- Smits M, Huibers L, Kerssemeijer B, De Feijter E, Wensing M, Giesen P. Patient safety in out-of-hours primary care: A review of patient records. BMC Health Serv Res 2010;10:335.
- Moll van Charante E, Giesen P, Mokkink H, Oort F, Grol R, Klazinga N. Patient satisfaction with large-scale out-of-hours primary health care in the Netherlands: Development of a postal questionnaire. Fam Pract 2006;23:437–43.
- Huibers, Smits M, Renaud V, Giesen P, Wensing M. Safety of telephone triage in out-of-hours care: A systematic review. Scand J Prim Health Care 2011;29:198–209.
- Payne F, Shipman C, Dale J. Patients’ experiences of receiving telephone advice from a GP co-operative. Fam Pract 2001;18:156–60.
- Salisbury C. Postal survey of patients’ satisfaction with a general practice out of hours cooperative. BMJ 1997;314: 1594.
- Campbell J, Roland M, Richards S, Dickens A, Greco M, Bower P. Users’ reports and evaluations of out-of-hours health care and the UK national quality requirements: A cross sectional study. Br J Gen Pract 2009;59:8–15.
- Garratt AM, Danielsen K, Hunskaar S. Patient satisfaction questionnaires for primary care out-of-hours services: A systematic review. Br J Gen Pract 2007;57:741–7.
- Giesen P, Franssen E, Mokkink H, van den Bosch W, van Vugt A, Grol R. Patients either contacting a general practice cooperative or accident and emergency department out of hours: A comparison. Emerg Med J 2006;23:731–4.
- Carr-Baines S, Nightingale AL, Ballard K. Patients’ experiences and satisfaction with out-of-hours GP home visiting provided by a GP cooperative. Fam Pract 2011;28:88–92.