Establishing a European research agenda on ‘gut feelings’ in general practice. A qualitative study using the nominal group technique

Abstract

Objective: Although ‘gut feelings’ are perceived as playing a substantial role in the diagnostic reasoning of the general practitioner (GP), there is little evidence about their diagnostic and prognostic value. Consensus on both types of ‘gut feelings’ (a ‘sense of alarm’, a ‘sense of reassurance’) has enabled us to operationalize the concept. As a next step we wanted to identify research questions that are considered relevant to validate the concept of ‘gut feelings’ and to estimate its usefulness for daily practice and medical education. Moreover, we were interested in the study designs considered appropriate to study these research questions. Methods: The nominal group technique (NGT) is a qualitative research method of judgmental decision-making involving four phases: generating ideas, recording them, evaluation and prioritization. Dutch and Belgian academics whose subject is general practice (n = 18), attended one of three meetings during which NGT was used to produce a ‘research agenda’ on ‘gut feelings’. Results: NGT yielded ten research questions and nine corresponding appropriate designs on four topics, i.e. the diagnostic value of ‘gut feelings’, the validation of its determinants, the opportunities for integrating ‘gut feelings’ in medical education and a rest group. The study designs respectively included recording and follow-up of ‘gut feelings’, video recording of consultations with stimulated recall using simulated and real patients respectively, analysing trainees’ consultation stories and videos, linguistic analyses, and vignette studies. Furthermore, two experimental designs were proposed.

Conclusion: A European research agenda on ‘gut feelings’ in general practice has been established and could be used in collaborative research.

Key words::

• ‘Gut feelings’
• general practice
• research agenda
• nominal group technique

View correction statement:

Erratum

Introduction

General practitioners (GPs) sometimes base clinical decisions on a feeling of sudden heightened awareness or alarm, which sometimes emerges during a consultation: ‘There's something wrong with this patient but I don't know exactly what. I have to do something because a delay can be harmful’. An e-mail survey among GPs in all countries involved in the European General Practitioners Research Network (EGPRN), asking them whether they recognized this ‘sense of alarm’, showed that it is a well-known and common phenomenon in general practice all over Europe (1). Primary care research into the diagnostic value of signs and symptoms for serious infections in children has identified the physician's or parents' feeling that ‘something is wrong’ as the most important diagnostic item (2). Focus group research into these ‘gut feelings’ in general practice has shown that many GPs in the Netherlands trust their ‘gut feelings’, which act as a compass, steering GPs through busy office hours and enabling them to handle complex problems (3).

Thus, although ‘gut-feelings’ are generally recognized as a diagnostic tool, particularly in situations of uncertainty which are characteristic for general practice, research into its validity seems a neglected area. It is a poorly examined tool. We do not know how physician's ‘gut feelings’ arise and how often ‘gut feelings’ lead to a correct diagnosis. A first step of answering these questions was the description of two types of ‘gut feeling’, i.e. a ‘sense of alarm’ and a ‘sense of reassurance’ (see Table I) (4). These descriptions may enable researchers to operationalize the concept of ‘gut feelings’, making them in some way measurable.

Table I. Consensus on ‘gut feelings’: ‘alarm’ and ‘reassurance’ (4).

A ‘sense of alarm’

• A ‘sense of alarm’ means that a GP perceives an uneasy feeling as he/she is concerned about a possible adverse outcome.

• A ‘sense of alarm’ implies that a GP worries about a patient's health status, even though he/she has found no specific indications yet; it is a sense of ‘there's something wrong here’.

• A ‘sense of alarm’ activates the diagnostic process by stimulating a GP to formulate and weigh up working hypotheses that might involve a serious outcome.

• A ‘sense of alarm’ means that, if possible, the GP needs to initiate specific management to prevent serious health problems

• A ‘sense of alarm’ will decrease as the diagnosis and the right management become clearer.

A ‘sense of reassurance’

• A ‘sense of reassurance’ means that a GP feels secure about the further management and course of a patient's problem, even though he/she may not be certain about the diagnosis: everything fits in.

• The ‘sense of reassurance’ and the ‘sense of alarm’ constitute a dynamic element in a GP's diagnostic process.

However, before initiating further research we need to know which questions are most relevant for daily practice and medical education and—perhaps more important and more difficult—which research designs are appropriate and feasible. Academic experts of general practice, appointed for educational or research tasks, could help us in defining and prioritising the questions with their corresponding appropriate designs. This article describes the development of a European research agenda, including research questions and corresponding appropriate designs, that aims to validate the concept of ‘gut feelings’ and to estimate its value for daily practice and medical education.

Methods

Nominal group technique

We used a qualitative research method, i.e. the nominal group technique (NGT), because the study of this topic is conceptually complex and intricate and there are no guiding examples of research in the literature. The technique enables researchers to gather information from relevant experts (5,6). It facilitates creative problem solving by means of judgmental decision making in situations where routine answers are inadequate (7–9). This means that the judgments of experts on the topic are integrated, in our case to establish a research agenda. NGT involves four phases: generating ideas, recording them, evaluation and a group decision phase.

Selection of experts

We purposively sampled well-known opinion leaders and experts on general practice in the Netherlands and Belgium, who were working at universities in educational or research programmes about general practice. We approached 30 colleagues by telephone, 27 of whom were willing to cooperate and received written information about the goal of the meeting and the procedure. These 27 colleagues were familiar with our research subject, since they had also been involved in an earlier study that aimed to achieve consensus on definitions of ‘gut feelings’ (4). We invited them to attend one of the three regionally organized meetings, and 18 accepted our invitation (six, five and seven per meeting, respectively). Reasons to decline included prior engagements and illness. No financial compensation was given.

NGT sessions

The NGT sessions were chaired by experienced and independent moderators assisted by one of the authors (MVdW, PVR, GJD) using flip-charts. We developed a scenario for the meetings in advance to ensure that all phases of NGT would be completed. In the first phase, that of ‘generating ideas’, the moderator explained the procedure and asked the participants to write down in silence, what they regarded as the main research questions relating to ‘gut feelings’, as well as corresponding appropriate designs for such research. Stimulated by the written information they had received some of the experts had already formed specific ideas in their mind.

In the second phase, ‘recording’, the members of the group were engaged in a round-robin feedback session to record concisely each idea. Research questions with corresponding designs were noted and numbered on flip-charts (six to eight in each session).

In the third phase, ‘evaluation’, each recorded idea was clarified and evaluated by discussion, in which those present proposed and weighed the arguments for and against the proposed questions with corresponding designs.

The purpose of the fourth and final phase, ‘prioritization’, was to aggregate the judgments of individual members, in order to determine the relative importance of the research questions with the corresponding designs. In this phase, the experts voted individually to prioritize the ideas with the corresponding designs, and their votes were used to arrive at a group decision. Each member selected five research questions and the corresponding designs and wrote the numbers down on separate cards, starting with the most important idea and ending with the least important one. The numbers on the cards were noted on a flip-chart and an overall prioritized list was drawn up. Afterwards, a brief discussion was held to evaluate the procedure and the outcome: no objections were made.

Analysis

After each meeting, the exact order of the research questions of the group was calculated with the Kruskal-Wallis test (because of the ordinal distribution). However, the results hardly differed from those of the qualitative assessment at the end of each session (see Table II). After the three meetings we compared all research questions and corresponding designs and categorized them. The results of the three groups were not entirely comparable since each group produced a number of unique ideas. Therefore, ranking of all research questions with corresponding designs across three groups was impossible.

Table II. Rating of all research questions on ‘gut feelings’ and appropriate study designs by three groups of experts.

	Ratings of group A		Ratings of group B		Ratings of group C
Rating	Aim	Design	Aim	Design	Aim	Design
1	Validation of determinants and cues Experience, context	Recording; (A) Intervention (F)	Diagnostic value Prevalence, relevance, test properties.	Recording, follow-up (A),	Diagnostic value Prevalence, relevance, test properties.	Recording, follow-up (A)
2	Diagnostic value Prevalence, relevance, test properties	Recording, follow-up; (A) Reflection after consultation (B)	Validation of determinants and cues Inter-doctor variation	(–)	Validation of determinants and cues Experience, context	Video recording; (C) Video stimulated recall (C)
3	Other Finding cut-off points	Manipulation in vignette studies (D)	Other Biological variables	Measuring heart rate and skin resistance (I)	Setting as a determinant	Recording, surgery versus out-of-office; Acute versus chronic (A)
4	Validation of determinants and cues Experience	Analysis of cases after consultation (C)	Validation of determinants and cues Setting as a determinant	Recording, surgery versus out-of-office; Acute versus chronic (A)	Other determinants	Case vignettes study (D)
5	Other Inter-doctor variation	Stimulated recall after observing video tapes (C)	Other Non-verbal elements	(–)	Other Doctor–patient differences	(–)
6			Validation of determinants and cues Expertise	Comparing experts and trainees; (E) Video observation (C)	Influence on guideline compliance	(–)
7			Other Semantic differences in interpretation	Linguistic analyses (H)	Validation of determinants and cues	Conversation analysis (H)
8		Medical education		Trainees telling their stories. Studying videos (G)

Legend: for (A)-(I) see Table IV.

Results

The three groups produced 20 research questions with corresponding appropriate designs. Although there was considerable overlap in the ideas, the second session produced five new questions and six new or modified designs compared to the first one and even the third session resulted in three new research questions and two new or modified designs.

After the three meetings we compared all research questions and corresponding designs. We distinguished ten main research questions, which were related to four topics: the diagnostic value of ‘gut feelings’; the validation of determinants of ‘gut feelings’, such as contextual information and the GP's experience; the possibilities for including ‘gut feelings’ in medical education; and a rest group (see Table III). Nine corresponding designs were formulated (see Table IV, A–I).

Table III. Results: research questions on ‘gut feelings’.

Diagnostic value (outcome)

• What is the prevalence of the ‘sense of alarm’ and the ‘sense of reassurance’?

• What are the diagnostic relevance and benefits of ‘gut feelings’? Is it possible to calculate their test properties? Are there differences in the prevalence of ‘gut feelings’ and the outcome between normal surgery hour services and services during out-of-office hours? Are there differences between GPs?

Validation of determinants and cues (process)

• What do specific professional-related determinants such as work-experience contribute to ‘gut feelings’? Are there differences between GPs? What is the significance of the contextual information? Which cues can be found by studying determinants?

Medical education (teaching)

• Can the concept of ‘gut feelings’ be used in medical education? Can ‘gut feelings’ be taught to young doctors?

Rest group: other research questions

• Are there measurable biological parameters of ‘gut feelings’ that would allow researchers to find more evidence?

• What minimum level of a priori disease probability is needed to experience a ‘sense of alarm’?

• What are the semantic differences and agreements between GPs in the interpretation of terms about ‘gut feelings’?

• Are there differences in the prevalence of gut feelings between GPs and their patients? Do patients' ‘gut feelings’ influence the diagnostic reasoning of GPs?

• Can ‘gut feelings’ explain the sometimes low compliance of GPs with guidelines?

• Which non-verbal elements related to ‘gut feelings’ do GPs observe?

Table IV. Results: Study designs for research on ‘gut feelings’.

A. Recording ‘gut feelings’, determinants, expected outcome (ICPC), in surgery and out of office hours. Follow-up after three months. Problem: which reference standard?

B. Stimulated recall after surgery: selection of patients on the basis of a strong sense of alarm; reflection on determinants afterwards.

C. Stimulated recall by experienced GPs and trainees after encounters with simulation patients. Same design but after observing video tapes with real patients.

D. Studies based on vignettes.

E. Experimental study with an intervention in the form of a short course to make participants aware of gut feelings and their significance involving students, inexperienced GPs and experienced GPs.

F. Experimental study with an intervention in the form of reflection after each consultation; recording ‘gut feelings’ and determinants.

G. Giving trainees an opportunity to tell their consultation stories and analyse the role of ‘gut feelings’. Studying videotapes with real patients.

H. Conversation analysis or linguistic analysis of transcripts of consultations.

I. Measuring heart rate and skin resistance during office hours with simulation patients or during observations of video-taped consultations.

In Table II, the ranking of research questions and designs is presented. In all groups research into ‘diagnostic value’ and ‘validation of determinants and clues’ scored high.

Discussion

With the help of NGT we were able to establish a research agenda on ‘gut feelings’ in general practice. We listed ten main research questions and nine corresponding designs. Overall, the highest priority was given to research questions about the prevalence of ‘gut feelings’ and its diagnostic accuracy (see Table II). ‘Gut feelings’ can function as a compass in uncertain and complex situations which are a characteristic part of a doctor's life, but do they contribute to correct decisions (3,10,11)? Although studying the test properties of this diagnostic tool is justified, recording it will not be very easy, since the ‘sense of alarm’ does not occur very frequently during surgery hours. This problem could be overcome by using vignette and video tape studies, but they lack the contextual information of daily practice, which is important for recognising pattern discrepancies. Therefore, in the future, we will still need recording in real practice situations.

All groups also mentioned the validation of determinants of ‘gut feelings’ to be an important research topic. Knowledge of the contributions of the most important determinants can create possibilities to use them in medical education and postgraduate training (see Table III). Although it seems important to study how students and trainees can be trained to develop, recognize and use ‘gut feelings’ while avoiding pitfalls, research into education of ‘gut feelings’ was not highly ranked. Another domain of research that had no high priority in our expert groups was the issue of ‘a sense of alarm’ as expressed by patients or their caregivers, and how these can influence the doctor's decision-making process.

It is remarkable that a question about the theoretical background of ‘gut feelings’ was lacking. Nevertheless, we need to understand how they arise and function in GPs’ diagnostic reasoning, next to medical problem-solving and medical decision-making. Literature on diagnostic reasoning and on psychological decision-making theories should be reviewed to find theoretical explanations of the role of ‘gut feelings’ in general practice.

NGT has rarely been used as a method to generate ideas for study designs, as a search in PubMed showed (12–14). Nevertheless, it seemed to be an efficient technique to gather specific ideas about difficult research questions and appropriate designs. The benefit of NGT is that all experts get equal opportunities to participate and to influence the decisions since it reduces the conforming common influence that tends to occur in face-to-face group meetings (6,7). Using consecutive groups seems worthwhile, since each new group also contributed new ideas.

Implications

Our results allow researchers to start high-quality studies into the diagnostic validity of ‘gut feelings’, the validation of its determinants, and the possibilities for integrating it in medical education. Support by the European General Practice Research Network (EGPRN) has allowed establishing an international expert group on ‘gut feelings’ in general practice (Cogita ProDiaman, http://www.gutfeelingsingeneralpractice.eu). This is a suitable platform to stimulate, prepare and coordinate international research projects on the diagnostic role of ‘gut feelings’ in general practice.

Conclusion

Using NGT, a European research agenda on ‘gut feelings’ has been established. Highest priority should be given to validation of the concept of ‘gut feelings’ and to estimating its diagnostic value for daily practice.

Acknowledgements

The authors are most grateful to the 18 participants for their contributions to the research agenda.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.