What are the most important non-academic attributes of good doctors? A Delphi survey of clinicians

Abstract

Background: Although a multiplicity of qualities and behaviours considered essential in a good doctor are identified in the profession's guidance documents, there is no consensus as to their relative importance, or indeed, agreement as to the core qualities that should be, or could be, feasibly assessed in the limited time of the typical medical school interview.

Aim: The aim of the study was to identify the most important generic attributes of good doctors, which can inform the content of the undergraduate medical student selection processes.

Method: The study used a Delphi survey to systematically gather the opinion of a panel of experts from a range of medical specialties as to the most important core attributes of good doctors. Additionally, a snapshot of opinion was obtained from the attendees of workshops held at a medical school educational conference.

Results: Common core attributes of a good doctor were identified across a number of medical specialties.

Conclusions: Consensus among clinicians from disparate specialties can be reached as to the most important generic attributes of good doctors and can be used to inform the choice of personal qualities and behaviours examined during undergraduate medical student selection process.

Introduction

A recommendation of the Tooke inquiry was that the medical profession work to reach a consensus about the correct and essential attributes of a future doctor (Tooke 2008). To this end, the Medical Schools Council issued a consensus statement that clearly defined the role of a doctor and described a number of abiding attributes required in doctors (MSC 2008). The attributes described derive from the outcomes expected of graduates, as outlined in Tomorrow's Doctors (GMC 2003), the qualities and behaviours needed in a doctor, as set out in Good Medical Practice (GMC 2006) and the attributes identified in the Guiding Principles for the Admission of Medical Students (MSC 2006). The latter of these reports identifies the following core attributes:

recognition that patient care is the primary concern of a doctor. Honesty, empathy, integrity, and an ability to recognise one's own limitations and those of others … good communication and listening skills, an ability to make decisions under pressure and remain calm … to cope with stress and have an understanding of professional issues such as teamwork and respect the contribution of other professions … Curiosity, creativity, initiative, flexibility and leadership (MSC 2006).

This list is by no means exhaustive. Indeed, Albanese et al. reported that the ‘literature identifies up to 87 different personal qualities relevant to the practice of medicine, and selecting the most salient of these that can be practically measured is a challenging task’ (Albanese et al. 2003). In practice, each medical school selects medical students ‘according to a profile of qualities considered by their admissions committee to be important in a medical student and future doctor (Powis 1998).

At this time there is no general agreement among UK medical schools about what the most important non-academic attributes of a good doctor are, and also whether there are generic attributes common to all medical specialties (Parry et al. 2006; Patterson & Ferguson 2007). However, if consensus can be reached on generic non-cognitive attributes required across all stages of the medical career and common to all specialties, these could be used to ‘guide the design of selection criteria used to recruit to undergraduate medical courses at the outset of training’ (Patterson & Ferguson 2007). Such agreement would go some way towards reducing the number of qualities and behaviours to the key ones that could manageably be assessed in the time allotted to the typical medical school interview.

Table

Practice points

There is no consensus as to what the most important non-cognitive attributes are. Common core attributes of good doctors can be identified across a number of specialties. Consensus among a sample of clinicians from disparate specialties can be reached as to the most important generic attributes of good doctors.

Aim

The purpose of this study was to find the consensus opinion among a sample of clinicians from a range of specialties as to the most important generic attributes of good doctors. The study will produce a ranked and rated list of personal qualities and behaviours that practising doctors consider the most important in a good doctor that can inform the content of the undergraduate medical student selection process.

Methods

The Delphi method

The Delphi survey technique is an anonymous, structured and iterative process carried out over a series of questionnaire rounds to systematically gather and aggregate the opinion of a panel of experts with the aim of reaching a consensus (Hsu 2007).

Sample size and selection of the expert panel

There is no agreement in literature about expert panel sample size. Experts in a particular field are purposefully selected to bring their expertise to bear on specialist issues based on their knowledge and practical engagement with the issue under investigation (Akins et al. 2005). This study's expert panel were purposefully recruited from trained clinicians who are active and experienced medical educators at the Peninsula Medical School. A letter explaining the aims of the study and the considerable commitment in time required resulted in a sample of 10 participants, nine males and one female. Although the range of specialties of those who agreed to participate is limited, the sample included a paediatrician, a pathologist, a haematologist, an immunologist, a surgeon and five practising general physicians (GPs).

The Delphi survey

In round one, the panel were sent a questionnaire containing a list of 20 attributes considered to be those of a good doctor derived by the authors from the following documents: Tomorrow's Doctors (GMC 2003), Guiding Principles for the Admission of Medical Students (MSC 2006), Good Medical Practice (GMC 2006) and Medical students: professional behaviour and fitness to practise (GMC 2007). Although in some cases the terminology used to describe a particular attribute was common to each guidance document, in many cases, a different terminology was used to describe the same underlying concept (e.g. probity, honesty, trustworthiness and acting with integrity). Thus, a seeming multiplicity of terms was distilled into a list of 20 attributes. The decision to provide a pre-selected list of attributes taken from key documents rather than enable participants to generate items relevant to their specialism themselves was made pragmatically. The generation of items to include in the survey would have considerably increased the number of survey rounds and commitment required of participants. Difficulty of recruitment and high attrition rates are common as the number of survey rounds increases, such as when potential participants are requested to spend several pre-survey rounds attempting to reach agreement as to what to include (Hardy et al. 2004). Additionally, the attributes considered important in a good doctor and outlined in the key guideline documents were selected by experts in the medical profession from a diversity of specialties. Thus, the pre-selected items used in this survey are no more than a tractable distillation of expert opinion on a wide range of attributes considered important in a good doctor, ready for rating and ranking to enable practical use in informing the student selection process.

Each participant was asked to rate the importance of each individual attribute on a five-point Likert scale: 1 (not at all important), 2 (moderately important), 3 (important), 4 (very important) and 5 (always important). Participants were also asked to rank each attribute in order of importance from 1 through to 20, with 1 as the most important and 20 as the least important.

In round two, participants received their individual round one questionnaire containing their rating and ranking of the attributes, along with the aggregate group rating and ranking. The panel was invited to rate and rank the attributes again and those who disagreed with the aggregate group judgement on any particular attribute, or attributes, were invited to give a brief reason for their disagreement.

In the third and final round, the process was repeated. However, participants also received a summary of the brief explanations given by those who disagreed with the aggregate group judgement for particular attributes. They were asked to rate and rank the attributes in the light of the emerging aggregate group rating and ranking and the qualitative feedback from those who dissented from the group aggregate.

Additionally, 48 attendees of two workshops held at a medical school education conference were invited to rate and rank the 20 attributes of a good doctor as they appeared in the Delphi survey questionnaire. The workshop survey sample not only included clinicians from a much wider range of specialties than those who made up the Delphi panel, but also included scientists, administrators and support staff working in a medical school (Workshop 1, n = 32, and Workshop 2, n = 16). Whilst the rating and ranking of attributes by the Delphi panel was a three-round anonymous and iterative process, the surveys of workshop attendees were much less formal and involved an element of discussion and debate between individuals.

Data analysis

There is no general agreement over the level of pre-determined measures of consensus. The criterion of agreement about the rating of each attribute used in this study is based on the use of simple summary statistics and the assumption that the scale upon which the panel express their opinion is measured at the interval level. Thus, the mean, median, range and standard deviation (SD) for the rating of each item was calculated at each round of the survey. The mean, as a measure of central tendency, is taken to represent the panel's group opinion, and the SD, as a measure of spread, the amount of disagreement within the panel (Greatorex & Dexter 2000; Holey et al. 2007).

Similarly, at each round, consensus in ranking was determined using the mean to evidence convergence of views by a movement towards central tendency, and by use of the strength of agreement indicated by comparison of the SD (strength of aggregate judgement) and the range (with larger ranges indicating outlier views). We do however acknowledge that the assumption of an interval scale for Likert-type categories and the use of the mean and SD in the calculation of central tendency is controversial (Jamieson 2004); however, it has long been accepted that the validity of parametric statistics ‘is often affected very little by gross departures from these assumptions’ (Pell 2005).

Ethical considerations

Ethical approval was sought for this anonymous survey from the Peninsula College of Medicine and Dentistry Ethics Committee, and it was decided that as the research did not raise any protection issues on human subjects, ethical approval was not required.

Results of the Delphi survey

Rating

In the final round (Table 1), ten attributes were rated at five on the five-point Likert scale by the panel and thus considered always important in a good doctor. These ten attributes also comprised the top ten aggregate rankings of the panel in the final round, and all attributes rated below five appeared in the bottom ten of the rankings, evidencing the level of consistency of judgement between the rating and ranking exercises.

Table 1.  Delphi panel's final round rating and ranking of the attributes of a ‘good’ doctor, rated on a five-point Likert scale (ranked 1 through to 20)

Core attributes of a good doctor	Delphi panel rated	Delphi panel ranked	WS 1 ranked	WS 2 ranked
Recognition that patient care is the primary concern of a doctor	5	1	1	2
Probity (being honest, trustworthy and acting with integrity)	5	2	2	1
Good communication and listening skills	5	3	4	3
Recognition of one's own limits and those of others	5	4	3	4
Pro-social attitude (has empathy and is non-judgemental)	5	5	5	8
Ability to cope with ambiguity, change, complexity and uncertainty	5	6	6	14
Commitment to lifelong learning, competence and performance development	5	7	12	7
Compassion	5	8	10	6
Motivation and commitment	5	9	9	5
Ability to be a team player	5	10	11	13
Ability to make decisions under pressure	4	11	7	11
Reflective manner	4	12	14	15
Flexibility	4	13	18	16
Initiative	4	14	15	9
Awareness of ethical issues in professional medical behaviour	4	15	8	10
Curiosity	3	16	17	18
Creativity	3	17	20	20
Insight into medicine	3	18	13	12
Potential for leadership	4	19	16	17
Objectivity in assessing the performance of others	4	20	19	19

Notes: 1, Not at all important; 2, Moderately important; 3, Important; 4, Very important; and 5, Always important. Additionally, the ranking by the two workshop attendees is shown as WS 1 and WS 2.

There was remarkable stability in the ratings in respect of the top 10 attributes across the three stages of the survey, indicating that clinicians, albeit from a limited range of specialties, appear to hold stable judgements when rating the most important attributes required of a good doctor (Table 2). The strength of agreement between raters is commonly reported as a kappa statistic which calculates the degree of actual agreement in classification over that which would be expected by chance and is scored as a number between 0 and 1, with perfect agreement equating to 1 and chance agreement to 0 (Cohen 1968). Thus, kappa has the advantage of providing more information than a straightforward calculation of the raw proportions of agreement. When more than two raters assign ratings to the ordered categorical data, as in this study, a weighted multi-rater kappa statistic is used to assess the magnitude of agreement. Weighted kappa takes into account the ordinal nature of the data and assigns less weight to agreement as categories are further apart (Viera & Garrett 2005). Weighted multiple-rater kappa estimates, calculated to assess the degree of agreement between raters in respect of the 20 items at each of the three rounds, showed agreement to be fair at round one, and moderate to good at rounds three and four (weighted kappa = 0.3222, 0.38072 and 0.4193, respectively) (Landis & Koch 1977, calculations of weighted kappa are based on the methods outlined in O’Connell & Dobson 1984). The increase in kappa across the three rounds reflects the emerging consensus across the group in respect of the top 10 items, whilst the magnitude of the kappa at round three evidences the disagreement, particularly about the importance of the remaining attributes.

Table 2.  The Delphi panel's rating of the top 10 attributes of a ‘good’ doctor over the three rounds of the Delphi survey on a five-point Likert scale

Core attributes of a good doctor		Round 1	Round 2	Round 3
Recognition that patient care is the primary concern of a doctor	Median	5	5	5
	Range	2–5	1–5	1–5
	Mean	4.6	4.6	4.6
	SD	0.97	1.26	1.26
	Rated	5	5	5
Probity (being honest, trustworthy and acting with integrity)	Median	5	5	5
	Range	3–5	4–5	4–5
	Mean	4.7	4.8	4.9
	SD	0.67	0.42	0.32
	Rated	5	5	5
Good communication and listening skills	Median	4.5	5	5
	Range	4–5	4–5	4–5
	Mean	4.5	4.7	4.8
	SD	0.53	0.48	0.42
	Rated	4	5	5
Recognition of one's own limits and those of others	Median	4	5	5
	Range	3–5	4–5	4–5
	Mean	4.3	4.6	4.6
	SD	0.67	0.52	0.52
	Rated	4	5	5
Pro-social attitude (has empathy and is non-judgemental)	Median	5	5	5
	Range	2–5	3–5	3–5
	Mean	4.3	4.5	4.5
	SD	1.05	0.85	0.85
	Rated	4	5	5
Ability to cope with ambiguity, change, complexity and uncertainty	Median	4	4	5
	Range	3–5	4–5	4–5
	Mean	4.1	4.4	4.6
	SD	0.57	0.52	0.52
	Rated	4	4	5
Commitment to lifelong learning, competence and performance development	Median	4	5	5
	Range	4–5	4–5	4–5
	Mean	4.4	4.6	4.6
	SD	0.52	0.52	0.52
	Rated	4	5	5
Compassion	Median	4	4.5	5
	Range	3–5	2–5	3–5
	Mean	4.0	4.2	4.4
	SD	0.94	1.03	0.84
	Rated	4	4	5
Motivation and commitment	Median	4	4.5	5
	Range	4–5	4–5	4–5
	Mean	4.3	4.5	4.6
	SD	0.67	0.53	0.53
	Rated	4	4	5
Ability to be a team player	Median	4	4	5
	Range	3–5	3–5	4–5
	Mean	3.9	4.3	4.8
	SD	0.74	0.67	0.42
	Rated	4	4	5

Note: 1, Not at all important; 2, Moderately important; 3, Important; 4, Very important; and 5, Always important.

Stability in the ranking of the top 10 attributes at each of the three rounds evidenced consistency of opinions across the stages of the survey (Table 3).

Table 3.  The Delphi panel's ranking of the top 10 attributes of a ‘good’ doctor over the three rounds of the Delphi survey, ranked 1 through to 20, with 1 the most important

Core attributes of a good doctor		Round 1	Round 2	Round 3
Recognition that patient care is the primary concern of a doctor	Median	1	1	1
	Range	1–20	1–20	1–20
	Mean	3.5	3	2.81
	SD	5.89	5.98	5.70
	Rank	1	1	1
Probity (being honest, trustworthy and acting with integrity)	Median	2	2	2
	Range	1–9	1–20	1–10
	Mean	2.8	4	3
	SD	2.48	5.73	2.56
	Rank	2	2	2
Good communication and listening skills	Median	4.5	4	4
	Range	2–10	2–8	2–8
	Mean	5.2	4	4.18
	SD	3.08	1.68	1.83
	Rank	3	3	3
Recognition of one's own limits and those of others	Median	5	4	4
	Range	2–10	2–9	2–8
	Mean	5.7	3.9	4.09
	SD	2.98	2.02	1.92
	Rank	4	4	4
Pro-social attitude (has empathy and is non-judgemental)	Median	5	7	5
	Range	2–19	3–16	3–16
	Mean	8.2	7.2	4.18
	SD	6.14	4.32	4.18
	Rank	5	7	5
Ability to cope with ambiguity, change, complexity and uncertainty	Median	6.5	6	6
	Range	4–11	4–16	4–9
	Mean	7.4	7.8	6.45
	SD	2.63	3.45	2.67
	Rank	8	5	6
Commitment to lifelong learning, competence and performance development	Median	7	7	7
	Range	3–14	2–14	4–14
	Mean	7.7	7.1	7.63
	SD	3.34	3.07	2.54
	Rank	6	6	7
Compassion	Median	7	8	7
	Range	1–18	3–18	4–18
	Mean	8.1	8.5	8.63
	SD	5.17	4.08	4.29
	Rank	7	8	8
Motivation and commitment	Median	8.5	9	9
	Range	2–14	3–19	2–13
	Mean	8.1	9.2	8.36
	SD	3.54	4.10	3.23
	Rank	11	9	9
Ability to be a team player	Median	11	10	11
	Range	4–15	5–13	5–13
	Mean	10.3	9.6	9.90
	SD	3.47	2.41	2.42
	Rank	9	10	10

Furthermore, the additional information provided to the panel in the form of brief summaries of reasons for dissent from the emerging group consensus does not appear to have had a dramatic effect on the stability of group opinion in respect of the ranking of the majority of the attributes.

In Table 4, the reasons given for dissent from the emerging consensus are summarised (experts, when given an opportunity to comment at round two with the knowledge of round one, aggregate group rankings). Out of the 20 attributes, recognition that patient care is the primary concern of a doctor, probity, curiosity, potential for leadership, insight into medicine, pro-social attitude, reflective manner and awareness of ethical issues in professional medical behaviour were all the subjects of expressed dissent from the group consensus. But only the attributes such as pro-social attitude, reflective manner and awareness of ethical issues in professional and medical behaviour changed the ranked position after round two.

Table 4.  Reasons given by Delphi panel members for disagreement with the emerging consensus

Attribute	Reasons for disagreement
Recognition that patient care is the primary concern of a doctor	I think a good doctor is concerned with patient care needs AND with the health needs of the community (who are not necessarily patients). A good doctor takes in to account both … recognises that these needs often conflict and manages the conflict
	At times, doctors put their own/colleagues/family/team/needs above their concern for their patient. This can be appropriate, even when there is no associated benefit to the patient
Probity (being honest, trustworthy and acting with integrity)	The lower rating of importance is in recognition of the fact that in order to prioritise being the patient's advocate, doctors may not always be completely honest in dealing with other agencies
Curiosity	Rated and ranked highly because I’m not sure it's possible to be a good communicator, listener, learner without being curious, and I don’t think that curiosity is something you can really learn … being curious about yourself, about patients, colleagues and your knowledge is essential if you are to be a great doctor
Potential for leadership	Clinical practice is all about leadership – doctors will often need to be leaders, facilitators and guarantors of teams
	Whilst I think doctors should have leadership potential, they may also function very effectively as team participants contributing other high level expertise and skills to support team function. This is a changing area – 20 years ago, my ranking would have been different. I’ve rated leadership potential closely with the team player function – but perhaps the latter should have been higher up the priority list
Insight into medicine	Cannot believe the group rated this so low – if you want to practice clinical medicine, you need to have insight into it, not just surface knowledge and understanding
Pro-social attitude (has empathy and is non-judgemental)	Medicine needs to accommodate individuals with a range of perspectives and views, which are dearly held. Medicine and our patients benefit from the diversity of views, opinions and beliefs, which we bring. We need students who recognise their values, have thought them through, and who are also open to see where these values might change as they ‘mature’ in their understanding of the role of the doctor in modern society
Reflective manner	Reflective manner is essential and much more vital than commitment to lifelong learning
	I see reflective manner as essential to maintaining a high standard in many of the other attributes
Awareness of ethical issues in professional medical behaviour	Rated this lower than the group because compliance with guidelines, etc. will ensure conformity without awareness – but does depend a lot on which speciality practised and how wide you draw the definition of issues

Whether the brief reasoned arguments for dissent had an effect on group ranking is a matter of speculation. However, pro-social attitude was elevated two places (7 to 5) despite the argument in the feedback to the panel that clearly supports the view that medicine needs to accommodate individuals with a range of perspectives. The attribute reflective manner was raised three places (15 to 12), perhaps in the light of the argument that reflective manner is essential for lifelong learning and maintaining a high standard in many of the other attributes. Awareness of ethical issues in professional and medical behaviour fell three places (12 to 15), possibly influenced by the feedback to the panel, which argued that compliance with guidelines will ensure conformity without awareness.

Results of the workshops

A snapshot of opinion was obtained from the attendees of two workshops held at a medical school conference. Although like is not being compared to like, it can nevertheless be seen in Table 1 that workshop attendees and the Delphi panel of experts ranked the same seven attributes in the ‘top ten’ of most important attributes of a good doctor and rated these attributes as either very important or always important (workshop ratings are not included in the table). Stark disagreement about particular rankings is evident in relation to the attribute, ability to cope with ambiguity, change, complexity and uncertainty, which was ranked sixth most important by both the Delphi expert panel and those in Workshop 1, but fourteenth most important by those who attended Workshop 2. Disparity in opinion about the importance ranking of the attributes, commitment to lifelong learning, awareness of ethical issues in professional medical behaviour, ability to be a team player and ability to make decisions under pressure is also evidenced among the three samples. But overall there is much agreement, especially in respect of the ‘top ten’ of attributes. Given the wider range of medical specialties among the workshop sample, the findings indicate that clinicians from diverse specialties and those from the wider population of individuals who work in medical education appear to hold common judgements as to the most important generic attributes required of a ‘good’ doctor.

Discussion and conclusions

It is widely accepted that it takes more than academic ability to make a good doctor (Harris & Owen 2007; Patterson & Ferguson 2007), and medical schools have been encouraged for some time to select students for broader characteristics and develop selection procedures that assess non-cognitive attributes (GMC 1993; World Summit on Medical Education Communique 1994; Greengross 1997; Core Committee, Institute for International Medical Education 2002). Nevertheless, as Reiter commented ‘little effort has been spent assessing the relative value of dozens of characteristics that have been identified’ (Reiter & Eva 2005). A generalisable aspect and value of this study is its demonstration of the utility of the Delphi survey method to enable individual institutions to identify the non-cognitive attributes considered most salient by their faculty, stakeholders and community. However, as Albanese points out, ‘it would be of substantial help to have a nationally defined set as a starting point’ (Albanese et al. 2003). Our starting point was the UK medical profession's key guidance documents.

The study produced a rated and ranked list of non-academic attributes of a good doctor derived from the medical profession's regulatory documents (Tomorrow's Doctors, GMC (2003); Good Medical Practice, GMC; (2006); Guiding Principles for the Admissions of Students, Medical Schools Council (2006); and Medical Students: professional behaviour and fitness to practise, GMC (2007). There was a remarkable stability across the rounds of the Delphi survey in opinion as to the top 10 attributes considered always important in good doctors. The findings indicate that clinicians from several different specialities seem to hold common opinions as to the most important core generic attributes of a good doctor.

It was reassuring that eight of the attributes ranked in the top 10 and rated as ‘always important’ in a good doctor by the Delphi panel were among those qualities and behaviours described as ‘overarching outcomes’ or behaviours students ‘must demonstrate’, by the time they graduate, as set out in the new version of Tomorrow's Doctors (GMC 2009).

Limitations to our findings are that it involved a single institution and a panel of experts from a restricted range of specialties. It is clear that the medical profession still has more fundamental work to do to establish an agreed set of essential attributes of a good doctor. Future research is recommended that surveys the opinions of experts from a much larger sample derived from a wider range of medical specialties and a greater number of institutions.

We recognise that our study included the imposition of a pre-conceived set of attributes on the panellists by not allowing them to contribute to the choice of items to be rated and ranked. However, extensive use of Delphi expert panel surveys by Keeney et al. in health care research has led them to conclude that the provision of pre-existing information, such as a list of attributes, for rating and ranking in the first round of the Delphi may be more effective and efficient in producing useful results (Keeney et al. 2006).

The influence on Delphi reliability of feedback in the form of comments on the reasons for dissent is questioned by some as to whether the increase in agreement is due to constructive feedback helping the ‘experts who are out of line with the consensus to refine their judgements, or whether such experts have just conformed to the majority view’ (Greatorex & Dexter 2000). It is impossible to know for certain but our view is that these dissenting voices provide valuable insights into the issue under scrutiny that otherwise would not be revealed by the statistically derived consensus.

It could be argued that a further limitation of the study is the questionable face validity of the rating categories and whether a rater actually discerns a difference, for instance, between ‘moderately important’ and ‘important’ on the five-point Likert scale used. As Crewson points out, agreement ‘is likely to be underestimated and not generalisable when rating categories have questionable face validity’ (Crewson 2005). Face validity is however concerned with the use of existing measures which previous researchers have shown to produce valid and reliable results. Thus, our choice of the five-point Likert scale anchored by ‘not at all important’ and ‘always important’ was informed by a practical application of the Delphi technique in nursing research (Hardy et al. 2004).

This study has contributed to the discourse on the rating and ranking of the non-academic attributes of good doctors (Reiter & Eva 2005). However, it is recognised that there remain thorny issues to resolve, such as the stability of attributes over the life course, whether some attributes can be taught or acquired during medical education and differences of opinion as to the meanings ascribed to attributes and how they can be reliably and validly assessed. Longitudinal research also needs to be undertaken to ascertain whether the selection of students with key non-academic attributes do indeed produce doctors with the defined personal qualities and behaviours set out in Tomorrow's Doctor (GMC 2009).

In conclusion, this study reports the consensus opinion of doctors from a range of specialties on the most important non-academic attributes of good doctors. The results can be used to inform the choice of personal qualities and behaviours examined during undergraduate medical student selection process.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.