Academic detailing in oral healthcare – results of the ADVOCATE Field Studies

Abstract

Background: Academic detailing (AD) is a defined form of educational outreach that can be used to influence decision making and reduce unwarranted variation in healthcare delivery. This paper describes the results of the proof of concept phase of the ADVOCATE Field Studies. This study evaluated the feasibility, acceptability and usefulness of AD reinforced with feedback data, to promote prevention-oriented, patient-centred and evidence-based oral healthcare delivery by general dental practitioners (GDPs).

Methods: In the Field Studies, six groups of GDPs (n = 39) were recruited in The Netherlands, Germany and Denmark. Each group had four meetings reinforced with feedback data for open discussions on dental practice and healthcare delivery. Conventional and directed content analysis was used to analyze the qualitative data collected from focus group interviews, debriefing interviews, field notes and evaluation forms.

Results: A total of nine themes were identified. Seven themes related to the process of the Field Studies and covered experiences, barriers and facilitators to AD group meetings, data collection and the use of an electronic dashboard for data presentation and storage. Two themes related to the outcomes of the study, describing how GDPs perceived they made changes to their clinical practice as a result of the Field Studies.

Conclusions: The ADVOCATE Field Studies approach offers a novel way of collecting and providing feedback to care providers which has the potential to reduce variation oral healthcare delivery. AD plus feedback data is a useful, feasible approach which creates awareness and gives insight into care delivery processes. Some logistic and technical barriers to adoption were identified, which if resolved would further improve the approach and likely increase the acceptability amongst GDPs.

Keywords:

• Academic detailing
• general dental practitioner
• oral healthcare
• quality improvement

Introduction

Reducing variation is a key in optimizing oral healthcare delivery. ‘Optimal quality care’ refers to care that is accessible, reliable, efficient and based on the best available evidence, and incorporating individual patient preferences [1,2]. Even though all stakeholders aim for optimal quality care, actual care may vary in many aspects including safety, effectiveness, equity and the individualization of care using the values and preferences of patients. Unwarranted variation in care delivery should concern oral health professionals, as it may indicate wasteful, ineffective practices and the possibility that care is not optimally serving the needs of the patient. This may be due to, for example, mistaken or limited individual professional knowledge, attitudes or skills, or disparate organizational performance [3].

However, some variation in healthcare delivery should be expected, given that differences in patient characteristics and preferences will occur naturally in different populations. Therefore, determining from feedback data alone whether variation in healthcare delivery is warranted or unwarranted is often fraught with difficulties. Data-driven normative judgements about the provided care should usually be resisted, given that many clinical situations involve decision making that is preference-sensitive. Giving and receiving feedback data about variation in healthcare delivery requires creation of a receptive learning environment because feedback data are poised to generate denial, discomfort and feelings of blame. If clinicians can self-identify areas in which change in oral healthcare delivery may be needed, providing positive, constructive feedback data in those areas could create an environment in which behaviour change may be internally recognized, accepted and subsequently acted upon [4].

Awareness of the importance of variation in healthcare delivery increased following Wennberg et al. reporting in 1988 on regional variation in healthcare [5]. Variation in healthcare delivery exists on national, regional and local levels and is driven by societal, organizational, cultural and individual factors. It is no longer a question of whether variation in healthcare delivery exists, but more a question on how to define, identify and, if appropriate, reduce the variation in healthcare delivery [6]. A review evaluating different implementation strategies for changing physician practice to reduce variation in healthcare delivery showed that active and multifaceted approaches, such as academic detailing (AD), lead to greater effects than traditional passive approaches, such as dissemination of information [7]. AD is a defined form of educational outreach which involves face-to-face education of healthcare practitioners by other trained healthcare professionals, often peers. AD has most commonly been used to explore and improve prescribing by doctors. Even though similar challenges concerning variation in healthcare delivery exist in dentistry, multifaceted approaches involving AD have not been explored for changing practice in dentistry.

In 2015, the ADVOCATE (Added Value for Oral care) project commenced. ADVOCATE is an EU Horizon 2020 project that aims to optimize delivery of oral healthcare in order to improve the wellbeing of the European population. As part of the ADVOCATE project, Field Studies have been conducted. The Field Studies are a proof of concept study aimed at evaluating whether AD, reinforced with feedback data, can be used to intrinsically motivate general dental practitioners (GDPs) towards more evidence-based, patient-centred and preventive oral healthcare. Using qualitative methods, the Field Studies evaluate the feasibility, acceptability and usefulness of AD reinforced with feedback data from the GDPs’ perspective, and whether this approach motivated GDPs to change their clinical practice. This paper presents the results of the Field Studies evaluation.

Methods

Design

The design of the Field Studies has been described in full detail elsewhere [8]. Local groups of GDPs were brought together to discuss variations and similarities in their provided oral healthcare and to reflect on optimizing their oral healthcare delivery. Figure 1 shows the overall approach used for the Field Studies. The groups of GDPs, called ‘Academic Detailing Groups’ (ADGs), were moderated by a Steward. Stewards were purposefully recruited by the ADVOCATE research team. They were dentists in active clinical practice with good interpersonal skills and prior experience as evidence-based educators of their peers. They received additional training in the methods of AD from the research team according to the principles defined by Soumerai and Avorn [9].

Figure 1. The aim, intervention and outcomes of the Field Studies.

Feedback data on GDPs’ healthcare delivery and oral health outcomes were used to inform and stimulate the discussions in the ADGs. Data were obtained from claims data and patient self-reported data. The patient self-reported data were collected through an online questionnaire administered in dental practice using a tablet (the questionnaire application) (Figure 2). The range of topics on which data was collected was based on an earlier study which defined measures of oral healthcare that were considered important, relevant and useful by GDPs, patients, health insurers and policymakers [10]. Claims data were obtained on a regional level from the health insurers or health authorities in each participating country. The acquisition of these claims data is described in an earlier publication [11].

The Field Studies were conducted in Denmark, Germany, and The Netherlands and ran from January 2017 until June 2018. Six ADG groups (total GDPs = 39) were recruited – two groups in each country. A convenience sample of GDPs were recruited by the Stewards from within their own and extended network. As earlier research showed that a sample between 5 and 8 participants per group would be ideal for qualitative research [12], the Stewards were asked to attempt to recruit a sample of 6–8 GDPs per group. The ADGs came together for four meetings over a period of 13 months; a set-up meeting and ADG meetings 1, 2 and 3. In the set-up meeting, GDPs were informed about the Field Studies and were provided with the resources to collect the patient self-reported data in their dental practice. Patient self-reported data were collected twice for a period of two months: after the set-up meeting and three months before meeting 3. Figure 3 shows the timeline of the ADG meetings and data collection periods.

Figure 2. Screenshot of the questionnaire application used to collect patient self-reported data.

Aggregated summaries of anonymized feedback data were made available for Stewards and every participating GDP in an online database, referred to as the ‘Dashboard’ (Figure 4). In ADG meetings 1–3, moderated, open, non-judgmental and confidential discussions on healthcare delivery took place using the dashboard to stimulate discussions. During each of the ADG meetings, GDPs and the Steward discussed a selection of feedback data. The initial selection of the feedback data was initially made by the Stewards and discussed before each meeting with the ADVOCATE research team (preparatory meetings) and finally discussed in the ADG meetings. During the discussion on the differences and similarities of the selected feedback data, GDPs were encouraged to reflect on their motivations for decisions made in current clinical practice, to discuss any underlying evidence, and to identify action points for improvement of their own clinical practice. Figure 5 shows the attendance of GDPs at each ADG per country.

Figure 3. Timeline; ADG meetings and data collection periods of the Field Studies.

Figure 4. Screenshot of the feedback data presented in the dashboard.

Figure 5. Attendance of ADG meetings in Denmark, Germany, and The Netherlands.

Evaluation data

The approach to evaluate the Field Studies was pre-specified and has been described in the design paper [8]. In brief, demographic background information of GDPs was collected by means of a questionnaire during the set-up meeting. The primary data used to assess the feasibility, acceptability and usefulness of the Field Studies approach, and to document GDPs’ perceived changes in their motivation and healthcare delivery, were collected by semi-structured focus groups interviews with GDPs during the last ADG meetings (meeting 3). Secondary data sources were notes from Stewards made during the ADG meetings, and debriefing telephone interviews of the research team with Stewards and evaluation forms completed by GDPs collected after the ADG meetings.

Six focus group interviews with the GDPs were conducted in June 2018 at the end of meeting 3. An interview guide consisting of open-ended questions was used [8]. Questions were centred on experiences with and perceived usefulness of the AD approach and the feedback data, actual changes made to clinical practice, or precursors of change such as reported changes in attitudes or approaches to healthcare delivery.

The focus group interviews were facilitated by a local researcher with experience in conducting focus group interviews recruited through the ADVOCATE research team. Interviewers were otherwise not involved with the ADGs and they did not have any relationship with the GDPs. The focus group interviews were conducted in a quiet room at the local dental faculties and were held in the local language of the GDPs. Prior to the focus group interview, a meeting took place involving the research team and each interviewer to standardize procedures according to the interview guide. The interview guide was available in the local language as well as in English. The focus group interviews lasted approximately 50 min. Interviewers made notes during the focus group interviews. All sessions were audiotaped and transcribed verbatim. All transcripts were translated to English for analysis by an external professional translation company and checked by the interviewers.

Data analysis

The software programme MaxQDA was used for analysis of the qualitative data collected. Data analysis of the focus group interviews was undertaken using a conventional content analysis approach [13]. Identified categories were data-driven and not preconceived categories. Initially, the focus group transcripts were read and re-read by one researcher (FB) to get an overall understanding of the data. Thereafter, segments of data were coded by identifying persistent and recurrent words or phrases. The codes were then grouped according to themes, which allowed the identification of sub-themes. The identification of themes and sub-themes resulted in the initial coding framework. As a sense-check, all authors assessed the framework on comprehensiveness. Directed content analysis was used to analyze the additional information retrieved from debriefings, notes and evaluation forms [13]. The initial coding framework of themes and sub-themes retrieved from conventional content analysis was used as the basis for coding the additional information. When segments of data were identified that did not fit in the coding framework, new codes were added to the framework. Where segments of data were determined to represent a topic that was not previously determined, a new theme emerged. Finally, the codes were checked for duplicates and whether they were categorized correctly. The analysis of the qualitative data was conducted by one author (FB), and concurrently discussed with a second author (DD).

Results

Demographic information for the 39 GDPs who participated is shown in Table 1. Three GDPs dropped out of the study because of personal reasons. A total of 26 GDPs was able to attend the focus group interviews. Seven GDPs could not attend because of logistical reasons.

Table 1. Characteristics of GDPs (n = 39) participating in the Field Studies.

Characteristic	Mean (range)
Age (years)	44 (28–70)
	n (%)
Gender
Male	20 (51)
Female	19 (49)
Denmark
Group 1	10 (59)
Group 2	7 (41)
Germany
Group 1	7 (58)
Group 2	5 (42)
The Netherlands
Group 1	6 (50)
Group 2	6 (50)
Special interest
Endodontics	22 (56)
Periodontics	20 (51)
Implantology	16 (41)
Paediatrics	7 (18)
Gnathology	5 (13)
Maxillofacial surgery	3 (8)
Orthodontics	2 (5)
Special needs dental care	2 (5)
Practice
Group	30 (77)
Solo	8 (20)
Both	1 (3)
Number of dentists in dental practice
1	7 (18)
2	8 (21)
3	6 (15)
4	0 (0)
5	7 (18)
6	2 (5)
Missing	9 (23)
Years since graduation
1–9 years	15 (38)
10–19 years	10 (26)
20–29 years	7 (18)
30–39 years	6 (15)
More than 40 years	1 (3)
Days per week working in practice
2 days	1 (3)
3 days	6 (15)
4 days	9 (23)
5 days	21 (54)
Missing	2 (5)

Conventional content analysis of the focus group interviews resulted in the identification of seven themes; ADG meetings, patient questionnaire application, claims data, dashboard, overall opinion of the Field Studies, perceived results and GDPs’ views on oral healthcare. Directed content analysis of the additional data from the evaluation forms, field notes and debriefings defined two additional themes; recruitment of GDPs and communication with GDPs. The total of nine themes could be grouped into two broad categories: (1) the process and (2) the outcomes of the Field Studies’ approach. A description for each theme is provided below. Within each theme, barriers and facilitators related to the Field Studies were identified, and are presented in Table 2. Quotes corresponding to each theme are shown in Table 3. Additional descriptive results from the evaluation forms are presented in Table 4.

Table 2. Barriers and facilitators in the ADVOCATE Field Studies.

Themes	Subthemes	Barriers	Facilitators
Recruitment of GDPs		GDPs not having enough time GDPs not seeing the added benefit	Stewards having a pre-existing network of GDPs in place
Communication with GDPs		Lack of response from GDPs to open ended questions in emails from Stewards	Email contact between Stewards and GDPs, which includes clear and directed communication Phone calls and visits to the GDPs’ practice when support was needed
ADG meetings	Steward		Stewards being committed, well prepared and organized moderators Having a neutral and positive environment for open discussions Stewards summarizing discussion points Stewards clarifying the feedback data Stewards aiding in the interpretation of the feedback data
	Logistics	Non-response or late response from GDPs to emails Busy schedules of GDPs Missing a meeting creating a feeling of lagging behind Absent and late GDPs disturbing the meeting	Having preparatory meetings with the research team
	Experiences	Feedback-data as a stand-alone resource is not sufficient to understand, discuss and reflect Not having included new feedback data in meeting 2	Feedback data and AD motivated and stimulated discussion AD meetings are essential to identify action points and make sense of the data Feedback data allows cross-country comparisons and comparison with different localities
Patient questionnaire application	Questionnaire	Long questionnaire Questionnaire including unclear and difficult questions Questionnaire not being available in different languages for the patients	Patients not minding the length of the questionnaire Patients liking to provide feedback The questionnaire being anonymous
	Data collection	Two months being too short to collect data Not enough effort taken to collect data Receptionists or assistants having time constraints to collect data Patients requiring detailed explanations from receptionists or assistants Receptionists or assistants not feeling responsible for data collection Receptionists or assistants not giving priority to data collection The questionnaire not having the option to leave questions blank The questionnaire not having the option to be paused Inappropriate instances for asking patients to fill in the questionnaire Technical difficulties with login Uncomfortable feeling of GDPS having the tablets in the waiting room Use of tablets introducing bias in selecting patients to complete the questionnaire	Establishing data-collection into the practices’ daily routine
			An external person facilitating the data collection in the practice Motivated receptionists or assistants Incentivizing data collection for the receptionist or assistant Having a digital questionnaire available outside of the practice Electronic questionnaire meant not having a lot of paper laying around
	Feedback data	Misinterpretation of questions by patients Small and selective number of collected questionnaires	Feedback data stimulating discussion Feedback data not being a tool to measure quality performance and give normative judgements
Claims data		Claims data being aggregated from regions within countries Data not being attractively and intuitively presented in the dashboard Having different oral healthcare systems across countries	Having claims data on practice or GDP level Claims data allowing cross-country comparisons
Dashboard	Usage	Not being able to see changes made to clinical practice in the feedback data Not having enough free time to look at the dashboard Having difficulties logging in to the dashboard GDPs not considering it their role to look at the dashboard	Dashboard allowing identification of points for improving care delivery
	Functionality	Dashboard providing unclear and not intuitive access to the data Receiving maintenance emails from the host of the dashboard	Dashboard presenting simple and summarized data
Overall opinion on the Field Studies		Duration of the study being too long Technical problems with the patient-app questionnaire and dashboard Unclarity amongst the GDPs regarding the aim of the study	Field studies not using normative benchmarking Field studies providing the opportunity to understand the clinical practice of colleagues Field studies not using normative patient-derived feedback Awareness that it is a proof of concept study and hurdles can be expected Having included a selective sample of GDPs who were already open to new approaches, information and changes

Table 3. Quotes from the focus groups and forms.

Process evaluation Field Study
Theme 1: recruitment of GDPs
Quote 1	‘The people that did not participate gave as main reason that they already have a high work and administrative load, so they do not want to take more work on board.’
Quote 2	‘People that did not want to participate were mainly concerned about the project taking too much time.’
Quote 3	‘The question I got from them is ‘What do I get out from it?’
Theme 2: communication with GDPs
Quote 4	‘Occasionally we arranged a visit to the practice for setting up data collection and to help with logging in onto the iPads.’
Quote 5	‘I tried to keep the emails to a minimum and when I sent out an email to the GDPs I tried to set up the email so that no reply was required from them; so more one-way email.’
Theme 3: ADG meetings
Steward
Quote 6	‘One couldn’t just go off at a tangent – it was well managed. I was particularly impressed with Mr. X’s commitment, especially if we had problems accessing the data. And when I couldn’t attend one of the sessions due to illness, we made a telephone appointment and we looked at the data on the computer and assessed it over the phone. We spoke for an hour.’
Quote 7	‘As dental care personnel we are not used to meeting people who have a positive attitude to us, we are normally met (and I’ve said before) with suspicion. All legislation around us is based on mistrust, where you suspected to have the most evil intentions, and it was really nice to have one like Mr. X, who was neutral and positive.’
Quote 8	‘With themes and sub-themes identified the entire coding framework was established and was sense-checked and checked for comprehensiveness by all authors, before being used as the framework for subsequently directed analysis of the additional data material.’
Logistics
Quote 9	‘Had to do several doodles. Because of holidays it took for some a long time to get back to me and in the meantime the others would have planned things on the selected date.’
Quote 10	‘As for myself, I had to cancel once, which was a shame, because you need to follow up on what has been talked about last time, otherwise you do not know what was agreed upon, but of course this is your own fault.’
Experiences
Quote 11	‘Then that also helps for later. So yes, I think that … There are no big differences, but you do actually look in a mirror and think: hey, I am already doing this. And that’s nice to get back, because you get an empowerment and therefore you do it a bit more often and it sticks. But, then there also things when you start thinking: hey, but here I’m not doing anything. And this is also nice to ask yourself. This is also nice to tell. In that sense it has added something.’
Quote 12	‘Both aspects: You have someone who explains this more in depth, but more importantly you do it together with colleagues and you talk about it which you otherwise would not do in that way.’
Quote 13	‘I feel that we have all been very open and honest about what has been good and bad at home, where we come from, I think we did not have any barriers. We could talk about things, which you are not so proud of, or not good at. It has been easy and safe (a confident feeling)’
Quote 14	‘the positive thing is that you get to talk to other people and particularly about attitudes to prophylaxis, a theme there has been a lot of emphasis on, and get some tips from other dentists Next, it is like attending a course, to come here and learn how other people solve our problems.’
Theme 4: App
Questionnaire
Quote 15	‘I started every time to say to the patients that nobody is going to recognise that you filled in the questionnaire, it is completely anonymous. I cannot see what you have written, I think it was also important for them that even I cannot see it, and that their answers are pooled with others, and, well, then it was OK.’
Quote 16	‘Yes, it was probably better to differentiate the questions instead of having a whole bunch of questions, for example, take 10 questions about periodontitis or gingivitis, and then 10 questions about something else …. We should not give all questions to all patients or groups.’
Quote 17	‘An English questionnaire would be convenient for those that do not speak Dutch very well.’
Data collection
Quote 18	‘Mostly the low number was due to staff problems. They find it difficult to get the staff to do this for them. The staff is not capable or not willing to participate.’
Quote 19	‘The staff in practices with high numbers of questionnaires collected were already used to collecting this sort of information.’
Quote 20	‘Last time we had the goal to improve data collection. One of the problems was that dentists with many children in their practice started the questionnaire, but half way they got distracted by the children. In the meantime the iPad had locked itself and the questionnaire had stopped.’
Quote 21	‘For example, at the physiotherapist, quality research is also annually. Then they look at patient satisfaction and everything. And then you just get a link by e-mail. So, they ask it in advance: would you like to…? And then they ask your e-mail address and then they send it.’
Quote 22	‘I also found it inappropriate after, for example, an extraction. To ask for it. I thought: they want to go home, and then to push those questionnaires under their noses, I thought: that’s going too far.’
Quote 23	‘The technique is too modern, our older patients had problems with this.’
Quote 24	‘It is not about increasing the absolute numbers of complete questionnaires, but to break a little into the bias, that only good-looking, young men get the iPad, because the assistant or receptionist can sit next to them and help fill in, to give a stupid argument.’
Quote 25	‘For the patients it went very well, surprisingly, even with the older patients. Some of them had problems in the beginning, perhaps they couldn’t proceed because the key was blocked by another finger. But when it works, they cope surprisingly well. But if it doesn’t work, the staff need to help them.’
Feedback data
Quote 26	‘It is definitely interesting to get the feedback from the questions, you get information on your own clinic and your own actions and this is my primary interest.’
Quote 27	‘I really think that’s the disadvantage of this research. That’s it, it’s comparing apples and oranges. Yes you’re asking something but is it really like that or is that patient not intelligent enough? Does he not understand, is he intelligent enough, did he not receive this information?’
Quote 28	‘I only use the mirror data so that I don’t end up being an outlier in comparison to my peers, and to prevent having to explain myself to the insurer.’
Theme 5: claims data
Quote 29	‘Yes, I don’t think it says anything about all practices. There are plenty of practices that just declare something, anything.’
Quote 30	‘And it interests me, not only in my area, but throughout Europe. What’s happening here, what’s happening in Germany, what’s happening compared to other countries?’
Theme 6: dashboard
Usage
Quote 31	‘So, that is really nice to get back. You become aware that it is also important to take some time to explain what you actually do with that patient. Yes, I really liked that.’
Quote 32	‘I'm a little disappointed because I think I had done much more to explain the things which my patients surprisingly did not understand, but my results remained the same.’
Quote 33	‘I simply haven’t got the time. Or I have the time, but I haven’t used it, because other things were more important to me- My task is really more – not to use that dashboard and so on, but to manage that the iPad works and is used and data is collected.’
Functionality
Quote 34	‘Personally I thought it was much too complex. You would have to sit there for hours to understand what can be compared and how to do it. God, I was already happy when I was logged on and it looked as if these were my numbers.’
Quote 35	‘Your e-mailbox will be spammed dead.’
Theme 7: overall opinion of the Field studies
Quote 36	‘For me, the question is, what is the aim of this European study? Are those who are implementing the study interested in me, with my feedback and where I stand compared to other dentists? Or is it about something else?’
Quote 37	‘I think they don’t quite understand the purpose of the research.’
Quote 38	‘I think it's … how do you say it? Because, I think we were already somewhat aware of how important this actually is, that is why we said yes to participate. And I don’t know if every colleague would be open to this approach.’
Theme 8: perceived results
Quote 39	‘I think I have made some changes in my behaviour in some areas, but it cannot be seen back in my figures and results’
Quote 40	‘I always ask: are you healthy? Do you take medication? Do you have allergies? But apparently that was not enough to make people realise that you are asking whether there are medical changes. So you literally have to ask: are there medical changes since your last visit? No. Are you completely healthy? Yes. Nothing changed? And yet those numbers are more or less the same. They still don’t always realise that you are asking for the medical condition.’
Quote 41	‘And they need to have the feeling that it was their own idea, and if it’s an expensive toothbrush, it’s the manufacturers responsibility. It has to do with self-responsibility, because if something doesn’t work, they like to push the blame onto someone.’
Quote 42	‘It also helps you. Because sometimes it is difficult within a group practice to say something to the other. But when there is a result from the app, then you have an opening to really talk about it.’
Theme 9: GDPs views on oral healthcare
Quote 43	‘Look, a patient in the practice is basically satisfied otherwise they would leave. But actually you just want to know when are you satisfied? And if you are not satisfied where can I improve? What matters to them?’
Quote 44	‘But they can’t check dental things and judge if it’s actually good.’

Table 4. Average evaluation ADG-meetings by GDPs.

Experiences GDPs	n/N (%)
The objectives of the Field Study were clearly defined
Strongly agree/agree	103/114 (90)
Neutral	8/114 (7)
Strongly disagree/disagree	3/114 (3)
The moderator was well prepared
Strongly agree/agree	111/113 (98)
Neutral	1/113 (1)
Strongly disagree/disagree	1/113 (1)
The moderator was well organized
Strongly agree/agree	109/113 (96)
Neutral	3/113 (3)
Strongly disagree/disagree	1/113 (1)
The Dashboard was a helpful tool during the meeting 1.
Strongly agree/agree	54/74 (73)
Neutral	15/74 (20)
Strongly disagree/disagree	5/74 (7)
This group meeting 1 experience will be useful in my work.
Strongly agree/agree	57/67 (85)
Neutral	8/67 (12)
Strongly disagree/disagree	2/67 (3)
The time allotted for the set-up meeting was sufficient.
Strongly agree/agree	96/113 (85)
Neutral	14/113 (12)
Strongly disagree/disagree	3/113 (3)

The process of the Field Studies

Theme 1: recruitment of GDPs

Stewards had different experiences with recruiting GDPs for the Field Studies. Some Stewards found it relatively easy to recruit GDPs within their network, whereas for others it took a lot of effort to explain what participation involved and to obtain agreement from GDPs to participate. The main reasons for GDPs not participating was not having enough time and not seeing an added benefit from participating (Table 3; Quotes 1–3).

Theme 2: communication with GDPs

Stewards mainly kept in touch with the GDPs via email. Occasionally a phone call or a visit to the GDPs’ practice was arranged when support was considered helpful, particularly with regard to collecting patient feedback data (Quote 4). Clear and directed communication in emails between the Stewards and GDPs was experienced by the Stewards to work better than more open questions – for example, asking the GDPs for suggestions or their opinion (Quote 5).

Theme 3: ADG meetings

Steward

GDPs were generally very positive about the role of the Stewards. They found them committed, well prepared and organized (Quote 6). GDPs appreciated the way the Stewards provided a neutral and positive environment for open discussion (Quote 7). The GDPs considered the Stewards’ role in the Field Studies approach to be essential, particularly through summarising discussion points, clarifying and interpreting the feedback data, and navigating the dashboard (Quote 8).

Logistics

Stewards experienced the preparatory meetings with the research team as useful, especially in order to decide what feedback data from the dashboard to discuss in the ADG meeting. Overall, the Stewards had little difficulty in arranging a place, date and time for the ADG meetings. Occasionally, more effort was needed to plan the meetings; during holiday seasons or when GDPs responded late to the emails with suggested dates (Quote 9). Attendance of GDPs varied across the different groups; illness, running late in dental practice and having other commitments were main reasons for non-attendance. It was considered disruptive when group members were absent or late. GDPs that missed a meeting felt they lagged behind (Quote 10).

Experiences

The feedback data and having conversations on healthcare delivery were perceived as motivating and interesting. The GDPs felt the meetings triggered them to reflect on decisions underlying their oral healthcare delivery (Quote 11). The GDPs agreed that feedback data provided to them as a stand-alone resource would not be sufficient to understand, discuss and reflect on the data (Quote 12). GDPs felt that the roles of the Steward as data interpreter and moderator of group discussions were essential to make sense of the data and to identify action points for change in clinical practice. The GDPs liked to compare themselves with similar practices from other localities. All GDPs were engaged in the discussions. The ADG meetings provided an open and safe place to learn and have constructive discussions about their healthcare delivery, relative to peers (Quotes 13 and 14). Meeting 2 was perceived by some GDPs as less useful because there were no new feedback data.

Theme 4: patient questionnaire application

Questionnaire

Overall, both Stewards and GDPs thought that the questionnaire was too long and took about ten minutes to be filled in by the patients. A few GDPs also reported that some of the questions were unclear, difficult to complete, and would benefit from some revision. However, GDPs did mention that the patients who completed the questionnaire did not mind the length. This was also confirmed by the patients in the questionnaire; the majority (91%) stated the time it took to fill in the questionnaire was reasonable and the majority of patients (93%) reported to find the questionnaire easy to understand. GDPs reported that patients liked to provide their dentist with feedback and were curious to see the results. The questionnaire being anonymous was an important factor for the patients (Quote 15). GDPs recommended to shorten and simplify the questionnaire so that it requires less time to complete. It was suggested to have the option of GDPs being able to make a selection of the questions (Quote 16). Some GDPs would also have liked the questionnaire to have been available in other languages (Quote 17).

Data collection

Many GDPs encountered challenges with regard to the collection of patient self-reported data in their dental practice. Most GDPs considered two months were too short to collect data from their patients. However, some of the GDPs felt they could have collected more data if more effort had been put into it. Getting data collection established into the practices’ daily routine was difficult for most GDPs. GDPs preferred to have someone outside of the clinical dental team to help with data collection in the practice. GDPs reported that the assistant or receptionist found it a lot of work to hand out the questionnaire tablets to the patients due to time constraints, especially when some patients required more detailed explanations (Quote 18). Other stated reasons for limited data collection was that the assistant or receptionist did not take responsibility or did not perceive it as a priority task. During the ADG meetings, it was reflected that the GDPs with the most data collected were the practices where the assistant or receptionist was actively involved (Quote 19). The GDPs recommended providing an incentive for the assistant or other means of motivation to improve data collection.

Another factor that could have hindered data collection was the fact that because of the design of the app, patients could not leave any questions blank or save questions for later; all questions had to be completed before data was submitted. Sometimes patients would be disturbed by their accompanying children or would be called to the GDP while completing the questionnaire, resulting in loss of data (Quote 20). Also, many patients asked whether they could fill in the questionnaire at home, yet the questionnaire – as currently designed – could only be completed at the practice on the tablet. Patients and GDPs would have liked the option of a paper or a link via e-mail to fill in at a later time (Quote 21). Some GDPs mentioned that there were instances where they felt it was inappropriate to ask the patient to fill in the questionnaire for example after the extraction of a tooth (Quote 22).

GDPs and Stewards questioned the use of tablets for data collection. Technical problems and difficulty with the login were perceived as annoying and time-consuming. Also, GDPs felt uncomfortable having the tablets in the waiting room, fearing they could be stolen. Having the tablet mounted to a pillar in the waiting room could have provided a possible solution. Furthermore, the GDPs felt that by using an electronic device for data collection could have introduced bias; older and patients with limited literacy had more difficulties filling in the questionnaire and the dental team – rightly or wrongly – would sometimes make a judgment and not ask some individual patients to fill in the questionnaire (Quotes 23 and 24). Despite these reported barriers, some GDPs reported to be surprised how well the majority of the patients coped with the devices (Quote 25). They also saw it as an advantage that an electronic questionnaire avoided having a lot of paper circulating.

Feedback data

Once the feedback data were collected and compiled, GDPs found the patient self-reported data interesting and useful (Quote 26). They considered it a good stimulator for discussions about clinical practice. It was reassuring for the GDPs to find few apparent large differences when they compared their data with other GDPs. However, the GDPs discussed the validity and representativeness of the data, recognizing that the number of collected questionnaires per dentist were often small and selective, and some questions might have been misinterpreted by the patients (Quote 27). Some GDPs considered the data to be unreliable since the data did not always match with the GDPs’ expectations; they were interested to know which patients provided which information. All GDPs agreed that no hard conclusions about healthcare delivery could be drawn from this data. GDPs felt that with larger numbers of participating patients the data would have had more meaning to them. They agreed that the feedback data were useful as tool to reflect on healthcare delivery and to stimulate discussions among peers, but not for normative purposes. Some GDPs indicated they used the feedback data as an indication of whether their clinical practice was similar to their peers, while not making any conclusions about the delivered healthcare (Quote 28).

Theme 5: claims data

Mainly, GDPs and Stewards found the claims data unclear and irrelevant because it was based on aggregated data from regions within countries (Quote 29). GDPs would find claims data as feedback data more interesting if they were aggregated at a practice or individual GDP level. The claims data as it was presented were seen by GDPs as being not sufficiently granular for the discussions held in the groups. Summarising and presenting the data in a more attractive and more intuitive way would make it more relevant for the GDPs. GDPs did find cross-country comparisons interesting (Quote 30), with the caveat that data should be viewed with caution because it originated from different oral healthcare systems.

Theme 6: dashboard

Usage

Despite the caveats described in Themes 4 and 5, the GDPs generally found it interesting and useful to see their own data in the dashboard and see the compiled patient self-reported data (Quote 31). Some of the GDPs were disappointed they could not see changes they made in their healthcare delivery over time in the dashboard (Quote 32). It was the Stewards who analyzed the data in the dashboard. The majority of GDPs did not access their dashboard outside the ADG meetings, despite them having access to the dashboard. The main reason given by the GDPs was not having enough time or difficulties with logging in. Furthermore, some GDPs did not consider it their role to look at the dashboard outside the ADG meetings (Quote 33).

Functionality

The dashboard was seen by the GDPs as being complex, unclear and not intuitive (Quote 34). The GDPs would have preferred simpler, summarized data. It was suggested to have the information from the dashboard in a user-friendly app. GDPs found the mails they received from the host of the dashboard that announced maintenance or updates to the database very frustrating (Quote 35).

Theme 7: overall opinion of the Field Studies

The GDPs found that the Field Studies provided a way to get an understanding of the clinical practice of colleagues, an opportunity which otherwise they would have been denied. According to the GDPs, the strength of the approach was that feedback data were collected from their patients and yet were not used to form normative judgements. GDPs had previous negative experiences with claims data being used for benchmarking performance by insurance companies or health authorities (Quote 36). The GDPs also indicated that they appreciated receiving feedback on their healthcare delivery from a peer visiting their dental practice.

Some GDPs felt that participating in a project that took a year was considered too long. The Field Studies did not run very smoothly according to the GDPs, because of the technical problems with the patient-app questionnaire and dashboard. They did recognize that this was mainly because it was a proof of concept study and therefore this could be expected. Furthermore, some GDPs reported in the focus groups that the aim of the study was unclear to them until the end of the study (Quote 37). This is in contradiction with the results from the evaluation forms were the majority of GDPs (set-up meeting; 95%, meeting 1; 87%, meeting 2; 96% and meeting 3; 80%) agreed or strongly agreed with the objective being clearly defined, as shown in Table 4. The GDPs considered that the participating GDPs were a selective sample who were already open to new approaches, information and changes (Quote 38). The GDPs anticipated that many of their colleagues would not voluntarily participate in similar projects.

The outcome of the Field Studies

Theme 8: perceived results

The GDPs initially stated that the Field Studies did not change their clinical practice. However, they then stated that the feedback data had provided them with new insights that patients often have an imperfect understanding or recollection of the healthcare they receive. For example, many patients reported that the dentists had not asked for their current medical history, while GDPs were certain that they did. This made the GDPs aware of the importance of better communication, or the potential effects of partial or incomplete recall. Based on this insight, GDPs reported they had made changes in their communication with patients (Quote 39). Most of the GDPs indicated that as a result they now try to ask questions differently, for example, by clearly asking for the patient’s preferences and by explaining their actions more explicitly to the patients (Quote 40). This is also reflected in the action points formed during the ADG meetings as shown in Table 5.

Table 5. Action points derived during the ADG meetings.

Action points	Corresponding measure
Ask patients which toothpaste they use.	Use of fluoride toothpaste
Discuss care options with the patients.	Shared decision making
Provide more prevention to patients.	Fluoride treatment in the last 24 months
Ask patients what they think about the appearance of their teeth.	Satisfied with appearance of teeth
Tell patients that their soft tissues are checked.	Oral mucosa screening (cancer screening)

Some of the GDPs also mentioned that in addition to understanding and recall, adherence to, for example, a preventive recommendation may potentially be facilitated by improved communication (Quote 41). Furthermore, GDPs recognized that better communication requires active effort and energy, and it does not provide an immediate, explicit reward. Some GDPs indicated that they would like to receive support in improving their communication skills, and this would be appropriate for the other oral healthcare workers in the dental team. The influence of the patients’ individual situation and preferences on decisions regarding healthcare delivery was discussed during the ADG meetings, and how to incorporate this in decision making and conversations with the patient.

Overall, the feedback data were considered an eye-opener for the GDPs. They thought the patient self-reported feedback data would be very useful to improve discussions about the quality of healthcare during team meetings in the dental practice. Especially in group practices where people might have difficulty in providing feedback directly to colleagues, GDPs saw feedback data from the patients as a useful, independent facilitator of discussions (Quote 42).

Theme 9: GDPs’ views on oral healthcare

GDPs found it very important that their patients are satisfied with the healthcare provided. GDPs recognized that dentistry is more than dental treatments, and that patient involvement and patient-centred healthcare are becoming increasingly important aspects of their work (Quote 43). However, some GDPs did mention that patient satisfaction does not necessarily mean that good healthcare is provided. They consider the patients are sometimes fallible when making judgements that are normative in nature (Quote 44).

Discussion

Group discussions using AD reinforced with feedback data are hypothesized to stimulate healthcare providers to calibrate their care delivery, leading to reduction of unwarranted variation. The goal of this proof of concept study was to initially explore whether this approach is a useful, feasible and acceptable way of intrinsically motivating GDPs towards a more evidence-based, patient-centred and preventive oral healthcare.

The results of this study suggest that GDPs become intrinsically motivated to change practice when they are engaged by AD. This is in line with earlier studies showing AD stimulates intrinsic motivation to improve care delivery [14]. The evaluation of the Field Studies found that GDPs became stimulated to improve communication with their patients. The Field Studies approach facilitated the identification of topics within oral healthcare were where mutual understanding of GDPs and patients is potentially not optimal. This can enable GDPs to adapt their patient communication and potentially tailor their healthcare delivery to individual patients.

However, the timeframe of the Field Studies and the nature of the available data precluded detection of quantitative changes in communicational aspects of oral healthcare delivery – for example, changes in the frequency of performing of preference-sensitive procedures. Significant reduction in variation of delivered care was not part of the design of this pilot study, was not measured and was beyond the scope of this study. However, participating GDPs did see potential for the Field Studies approach to aid in reducing variation in oral healthcare delivery. Furthermore, the two data collection periods were too short to provide evidence for causal and temporal relationships of the Field Studies approach and changes in care delivery. Future research involving larger numbers of GDPs over longer periods of time is necessary to measure the effect of the Field Studies approach on variation in care.

The focus on prevention during the ADG meetings was closely linked to communication, patient understanding and a realization of the difficulties in communicating the importance of prevention. This points to the need for preventive interventions based on the individual patient. Evidence-based healthcare may have been difficult for GDPs and Stewards to relate to in the ADG meetings, this reflected by a lack of evidence-based healthcare specific actions points formulated (Table 5). Evidence-based healthcare might not have been in the foreground during the ADG discussions, because of the current availability of evidence based on randomized controlled trials (RCTs) in dentistry [15]. RCTs and other outcomes oriented studies in dentistry that have evaluated clinically relevant interventions are relatively few in number [16]. In addition, studies have shown that the awareness of evidence-based practice amongst GDPs is low [17–19].

GDPs experienced the Field Studies approach as useful, providing them with the opportunity to reflect and learn about healthcare delivery and create awareness on variation in healthcare and patient needs. The GDPs regarded being aware of similarities in challenges in providing healthcare and discussing attitudes to practice between peers as comforting and reassuring. The approach provides the opportunity to function as a calibration tool; GDPs used the feedback data and the ADG meetings to see where they stand in their healthcare delivery compared to their peers. This is in line with the findings from Meyer and Singh [4] who stated that preventable harms by underuse and overuse of diagnostic tests and other resources can be resolved by developing well-calibrated healthcare providers. Calibration facilitates foundational knowledge on decision making (aligning competing needs and demands) and gives confidence to the healthcare provider [4].

This study shows that overall it is feasible to provide AD reinforced with feedback data in a group setting to GDPs in Denmark, Germany and The Netherlands. Stewards succeeded in moderating three meetings with GDPs and guided them through the dashboard and the interpretation of the feedback data. However, a number of barriers have been identified that need to be overcome in order to implement the Field Studies approach on a larger scale. It is likely that recruitment of GDPs to participate in AD should be stimulated by incorporating the approach in already existing systems or protocols, or by incentivizing participation. Barriers relating to the data collection in the dental practice should be reduced – for example by involving the entire oral healthcare team in the set-up meetings and data collection, instead of expecting the GDPs to successfully cascade this to their in-practice colleagues. Technical problems should be resolved and improvements made to the deployment and use of both the tablet-based patient questionnaire application and the dashboard, for example by obtaining patient self-reported data via a variety of other means, such as the patient’s own smart phone. Finally, the role of the Stewards was considered essential to initially interpret the data in the dashboard and to organize and facilitate discussions. If AD is undertaken on a wider basis, adequate capacity and resources will be needed for initial training and ongoing support to the Stewards.

Several limitations regarding the Field Studies should be considered. The use of a small convenience sample of GDPs might have resulted in a very selective sample GDPs. This limits the ability to generalize the findings. The purpose of using qualitative methods in this study was to evaluate the Field Studies approach, to gain a deeper understanding of barriers and facilitators for implementing the approach on a larger scale in dentistry. However, the data should not be considered as observational data about whether actual changes were made to GDPs’ dental practice. Furthermore, a limitation of the Field Studies approach is that no information was available on the response rate of patients to the questionnaire. Non-participation may be due to a fear of breach of confidentiality, a negative attitude towards oral healthcare and surveys in general [20]. This might have introduced selection bias amongst the patients filling in the questionnaire. Subsequent research should attempt to collect data on non-response. Furthermore, claims data were only available on a macro level, providing limited insight into the healthcare provided. Also, patient self-reported feedback data were limited since, for example, an interrupted time series comparison was not possible. If the same patients fill in the questionnaire at baseline and follow-up, the reliability of the data would be improved.

In conclusion, the Field Studies approach is feasible, useful and acceptable, when appropriate adjustments have been made to mitigate reduce the barriers to implementation. The approach establishes a novel and useful way of collecting and providing feedback data within dentistry. One of the important lessons learned was that by participating in AD, the GDPs became more reflective about their own practice and consultation skills, how they compared to their peers, and they gained insights into how they were perceived by their patients. This may be seen as an important first step to raise awareness about and give more meaning to variation in healthcare delivery.

More information is required on the scalability and reproducibility of the Field studies approach before implementation strategies can be developed. However, if those further small-scale studies confirm these results, and when appropriate adjustments have been made to mitigate the barriers to implementation, the development and evaluation of large scale AD approaches using patient-derived feedback data would be appropriate.

Ethical approval

This study was approved by the VU medical ethical committee in The Netherlands, the Heidelberg Ethics Committee in Germany and the Copenhagen Videnskabsetiske Komiteer in Denmark and the Danish Data Protection Agency.

Author contributions

F. Baâdoudi, MSc (PhD-student), contributed to design, data interpretation, drafted and critically revised the manuscript; D. Duijster, PhD (assistant professor), contributed to design, data interpretation and critically revised the manuscript; N. Maskrey, MD (research consultant), contributed to conception, design, data interpretation and critically revised the manuscript; F. M. Ali, BSc (research assistant), contributed to data collection and critically revised the manuscript; K. Rosing, PhD (Postdoc researcher), contributed to data interpretation and critically revised the manuscript; G. J. M. G. van der Heijden, PhD (Professor), contributed to conception, design and critically revised the manuscript; All authors gave final approval and agree to be accountable for all aspects of the work. F. Baadoudi and D. Duijster both have completed a course in qualitative methods in health research.

Acknowledgements

We would like to thank the contributors to the ADVOCATE project: the ADVOCATE Scientific Advisory Board – Stephen Birch, Martin Chalkley, Roger Ellwood, Ekatarina Fabrikant, Jeffery Fellows, Christopher Fox, Frank Fox, Dympna Kavanagh, John Lavis, Roger Matthews, Mariano Sanz, Paula Vassalo and Sandra White; the ADVOCATE General Assembly – Renske van der Kaaden, Lisa Bøge Christensen, Gail Douglas, Kenneth Eaton, Gerard Gavin, Jochem Walker, Stefan Listl, Gabor Nagy, Karen O’Hanlon, Andrew Taylor, Helen Whelton, Noel Woods; the ADVOCATE Ethics Advisory Board – Mary Donnelly, Eckert Feifel, Jon Fistein, Evert-Ben van Veen and Agnes Zana; the ADVOCATE project coordinator, Maria Tobin; and the co-workers of the ADVOCATE project. We specially thank the GDPs and Stewards participating in this study. We would also like to thank Ana-Lena Trescher, Olivier Kalmus and Lisa Bøge Christensen for their help in data collection and transcription of Focus groups and Olivier Kalmus in liaising with the local ethics committee in Germany.

Disclosure statement

No potential conflict of interest was reported by the author(s). All authors have no personal, commercial, political or interests to declare.

Additional information

Funding

All authors received funding from a European Union Research Grant (ADVOCATE) under Horizon 2020; [Grant Agreement No. 635183]. http://advocateoralhealth.com