Two-dimensional structure of the MAAS-Global rating list for consultation skills of doctors

Abstract

Background: The MAAS-Global (MG) is widely used to assess doctor–patient communication skills. Reliability and validity have been investigated, but little is known about its dimensionality. Assuming physicians tend to adopt certain styles or preferences in their communication with patients, a multi-dimensional structure of the MG can be hypothesized.

Aim: This study investigates the dimensional structure of the MG and explores the validity of this structure by studying the relationship between potential MG sub-scales and general practice speciality trainees’ personal characteristics.

Methods: Communication skills of 68 first-year trainees in a two-station objective structured clinical examination were assessed. Exploratory factor analysis was conducted on the resulting MG item-scores. With t-tests and correlational analysis, the relationship between MG scores and trainees’ personal characteristics was examined.

Results: Two well-interpretable factors were found, representing patient-oriented and task-oriented communication skills. Being born in the Netherlands and empathy were positively associated with overall communication skills. Prior communication skills training was exclusively related to task-oriented communication skills. Empathy was associated with patient-oriented, but not with task-oriented communication skills.

Conclusion: The two-dimensional structure of the MG may be valuable in gaining a better understanding of factors influencing the acquisition of communication skills. This may be used to optimize teaching methods in communication skills training.

Introduction

The quality of doctor–patient communication influences medical outcomes and emotional well-being of patients as well as the accuracy and efficiency of the diagnostic process (Stewart 1995; Carney et al. 1999; Little et al. 2001; Jackson 2005; Slatore et al. 2010). Therefore, extensive attention is paid to the acquisition of doctor–patient communication skills during undergraduate and again in general practice (GP) speciality training, a field of medicine in which good communication skills are considered a core competency (Allen et al. 2002). GP trainees in our experience vary in both their doctor–patient communication skills at the start of their training and their ability to improve these skills during their GP training. Exploring the nature and origin of these differences and taking them into account could possibly make an educational programme in doctor–patient communication more effective.

A commonly used instrument to assess doctor–patient communication skills is the MAAS-Global (MG) rating list (MG) (van Thiel et al. 1991). Research has shown this to be one of the better communication skills assessment tools (Boon & Stewart 1998). Its validity and reliability have been supported in several studies (van Thiel et al. 1991; van Dalen et al. 1998; van Nuland et al. 2007). The MG is generally used as a one-dimensional instrument but, as far as known, only one study by van Nuland (van Nuland et al. 2007) has been published to support this assumption. In this study, however, factor loadings were relatively low (between 0.14 and 0.69) and therefore not per se convincing with respect to the uni-dimensionality of the MG. In fact, multi-dimensionality could be a plausible alternative, considering that the MG is expected to cover all relevant aspects of doctor–patient communication and taking into account the empirical evidence that physician communication skills can be clustered in patterns based on personal preferences (Hall et al. 1987; Schirmer et al. 2005). For example, a GP who is educated in a rather authoritative educational setting may tend not to express some of the more patient-oriented behaviours listed in rating scales like the MG, in order to meet the standard of his role models (Dorgan et al. 2009).

Table

Practice points

The concept of a dichotomy in doctor–patient communication skills (patient-centred versus task-related skills) is sustained in the dimensional structure of the MG rating list for consultation skills. Empathy as a personal characteristic is related to patient-oriented doctor–patient communication skills but does not relate to task-related skills. Having had prior communication skills training seems to have more effect on task-related skills than on patient-oriented skills in first-year GP trainees.

Conceptually, doctor–patient communication skills can be broadly divided into behaviour aiming at eliciting socio-emotional aspects and behaviour aiming at the task of exchanging medical information. The latter is expected to be specifically trained through medical education (Roter & Larson 2002) while the former, besides through formal training, is also learned by informal learning in daily communication. It is known that trainability of communication skills varies (Aspegren & Lønberg-Madsen 2005). Socio-emotional and affective behaviours are probably more difficult to acquire and more related to personality traits than medically oriented behaviour (Libert et al. 2007). Bearing this in mind, unravelling the dimensionality of the MG could be of interest from an educational perspective. First, it could help transform the programmes most GP training centres use into programmes tailored to individual trainees’ actual needs, taking into account trainees’ starting skills in doctor–patient communication and their ability to acquire and develop these skills (Hobma et al. 2006). Second, it could lead to more understanding and knowledge of personal preferences and styles of GP trainees.

Previous studies suggest a relationship between global communication skills and personal characteristics such as gender, age, country of birth, prior communication skills training, and empathy (Liddell & Koritsas 2004; Roter & Hall 2004; Stratton et al. 2005; Laidlaw et al. 2006). What will happen to these relationships when compared to the expected dimensions of the MG? The outcomes could help us predict the development in doctor–patient communication skills of trainees and possibly inform us on their ability to benefit from educational programmes.

The objective of this study is to investigate the dimensional structure of the MG and explore the validity of this structure by studying the relationship between the MG sub-scales and personal characteristics of GP trainees.

Method

Participants

All 71 GP trainees who newly enrolled their GP training in 2007 at the Academic Medical Center (AMC) of the University of Amsterdam were asked to participate in this study.

Procedure

As part of a larger research programme, the trainees participated in a half-a-day test session that was organized at the very start of their educational curriculum, prior to their first communication training. For this sub-study, trainees participated in a video-taped two-station objective structured clinical examination (OSCE) with standardized patients. The duration of the consultation was limited to 7.5 min. Four different simulation patients were used for each OSCE. The first OSCE dealt with a very anxious patient with a minor skin disease. The second OSCE was a female, 24-year-old Jehovah's Witness, concerned about having caught a sexually transmitted disease. In both OSCEs, the focus was on communication.

In addition, a questionnaire was administered to assess relevant personal characteristics. All participating trainees gave written informed consent to use their data for research purposes. OSCEs were rated by an experienced female GP staff member and a male psychologist, both trained in using the MG and not involved in training the trainees included in this study.

Instruments

The OSCEs were rated with the MG rating list (van Thiel et al. 1991; van Thiel et al. 2000). The validity and reliability of the MG were found to be satisfactory in several studies (van Dalen et al. 1998; Ram et al. 1999a, b; van Nuland et al. 2007). The MG includes 13 items on doctor–patient communication referring to either phase-specific skills (e.g. Opening and consultation evaluation; n = 7) or general skills (e.g. Dealing with emotions and giving summaries; n = 6). The items are scored on a seven-point Likert scale, ranging from 0 (‘not present’) to 6 (‘excellent’) and anchored to the score of 6 (‘excellent’). Each item has two to four sub-items that reflect the aspects that have to be considered. For example, when scoring the item Empathy, the rater has to take into account the degree to which he or she assesses the trainee as (1) being concerned, inviting and sincerely empathic in intonation, gesture and eye contact and (2) expressing empathy in brief verbal responses. For each consultation all items must be completed except for the items ‘Physical examination’ and ‘Follow-up consultation’, which may not be applicable. Since in our study these two items were not applicable, only 11 items were scored. An overall consultation score was obtained by averaging the individual item-scores. Each consultation was rated by the two trained raters.

The Davis (1983) Interpersonal Reactivity Index (IRI) was used to assess the empathic skills of the participants. The IRI is a validated questionnaire developed to measure empathy on four different dimensions. The four sub-scales are: Perspective taking (PT), Fantasy (FS), Empathic concern (EC) and Personal distress (PD). In earlier research (Stratton 2005), a relationship between the IRI and clinical skills was found. The sub-scales EC and FS in particular were related to communication skills. A questionnaire was administered concerning age, sex, prior communication skills training and country of birth.

Data analysis

Pearson correlation coefficients were used to evaluate inter-rater reliability and the relation between MG scores and personal characteristics. Paired and independent t-tests were used to explore group differences in MG scores. Internal consistency and dimensional structure of the MG scale were evaluated using the average item-scores of the two consultations. Internal consistency was calculated using Cronbach's α. A principal component analysis with a direct oblimin rotation (δ = 0) was conducted to establish the dimensionality of the MG (Guadagnoli & Velicer 1988).

The number of components extracted was determined by visual inspection of the scree plot and the eigenvalues of the components (>1). After direct oblimin rotation, each component was transformed into a scale by including the items that loaded more than 0.50 on that component. The direct oblimin rotation was chosen to allow for the anticipated correlation between factors. All tests were two-sided and statistically significant findings were so at 95% confidence. SPSS 15.0® was used to analyse the data.

Results

Response

Sixty-eight out of the 71 first-year GP trainees at the AMC of the University of Amsterdam took part in the study, on a voluntary basis. Two trainees could not attend the test session due to family matters. One did not give written consent to use the collected data. The participants were 53 women and 15 men. Their average age was 30.5 year (standard deviation (SD): 3.6, range: 25–43 year). Nine trainees were born outside the Netherlands.

Reliability

The mean inter-rater reliability of the overall scores were 0.46 and 0.52, respectively. These values were consistent with outcomes of van Nuland's (2007) research. Internal consistency of the scale (consisting of item-scores averaged over two consultations and two raters) was good, with a Cronbach's α of 0.86.

MG scores

Figure 1 shows item-scores and overall scores averaged over the two consultations. The resulting pattern is consistent with results from previous studies (Hobma et al. 2006; Kramer et al. 2004), including the relatively low scores on the items Clarification of the reason for encounter, Evaluation of the consultation, Summarizing and Dealing with emotions.

Figure 1. Average MG scores, per item and overall (N = 68).

The KMO measure of sampling adequacy was good (0.777) and the Bartlett test of sphericity was significant, χ² (54) = 415.321, p < 0.0005, indicating that the data were suitable for factor analysis. Three principal components with initial eigenvalues greater than 1 (4.7, 2.1 and 1.04, respectively) were extracted that explained 72% of the total variance. Because the third eigenvalue was only slightly greater than 1 and visual inspection of the scree plot clearly indicated a two-factor solution (accounting for 62% of the variance), the two-factor solution was further analysed.

After direct oblimin rotation, the rotated solution revealed a simple structure (see the pattern matrix in Table 1). All items loaded strongly on only one factor, except Evaluation of the consultation, which did not load onto a factor. Results were checked against the structure matrix. The correlation between the two factors was 0.27. Inspection of the content of the items with the highest loadings on each factor suggests that the first factor can be interpreted as ‘patient- or affect-oriented’ aspects of communication skills and the second as ‘medically or task-orientated’ aspects of these skills.

Table 1.  Pattern matrix of principal components analysis of MG items, after direct oblimin rotation (N = 68)

	Factor
Item	1	2
Exploration of expectations and feelings	0.89	0.03
Dealing with emotions	0.80	0.06
Empathy	0.76	0.18
Opening	0.76	0.03
Clarification of the reason for encounter	0.74	0.10
Summarizing	0.60	−0.17
Communication about the diagnosis and hypothesis	−0.21	0.89
Structuring	0.07	0.85
Communication about the management plan	−0.05	0.83
Providing information	0.27	0.75
Evaluation of the consultation	0.15	0.42

Note: Loadings greater than 0.50 are given in bold face.

To further investigate this structure, two sub-scales were created, which we named MG patient and MG task, consisting of the items with loadings greater than 0.50 on factor 1 and factor 2, respectively. Internal consistency of these sub-scales was good, both having a Cronbach's α of 0.87.

Relationship between MG score and trainees’ individual characteristics

Results are presented in Table 2. The averaged score for MG task (M = 3.12, SD = 0.50) was higher than for MG patient (M = 2.38, SD = 0.61), t (68) = 9.32, p < 0.0005, implying that trainees showed better task-oriented than patient-oriented communication skills.

Table 2.  MG scores and SDs as a function of individual characteristics, including t- and p-values (N = 68)

	MG scale
	MG total score			MG patient			MG task
Personal characteristics	M (SD)	t	p	M (SD)	t	p	M (SD)	t	p
Sex		1.18	0.24		1.31	0.19		0.20	0.84
Female (n = 53)	2.55 (0.44)			2.43 (0.59)			3.12 (0.47)
Male (n = 15)	2.39 (0.51)			2.20 (0.69)			3.09 (0.60)
Age		1.98	0.06		1.64	0.11		1.95	0.06
Below 30 (n = 41)	2.61 (0.32)			2.49 (0.47)			3.21 (0.42)
30 or older (n = 27)	2.37 (0.59)			2.22 (0.77)			2.97 (0.58)
Country of birth		3.88	<0.0005		3.40	0.001		2.55	0.01
Netherlands (n = 59)	2.59 (0.42)			2.47 (0.56)			3.17 (0.47)
Elsewhere (n = 9)	2.01 (0.37)			1.78 (0.60)			2.74 (0.60)
Prior communication skills training		1.83	0.07		1.28	0.21		1.99	0.05
Yes (n = 61)	2.55 (0.44)			2.41 (0.60)			3.15 (0.47)
No (n = 7)	2.22 (0.51)			2.10 (0.69)			2.77 (0.67)

Note: Significant differences are indicated in bold face.

No group differences in communication skills were found with respect to sex and age, although for age, the differences almost reached significance. Trainees who were born in the Netherlands performed better than trainees born elsewhere, on both scales measuring communication skills. Finally, trainees with prior communication skills training showed better task-oriented communication skills than trainees without prior training.

In Table 3, the relation between communication skills (MG scores) and empathy (IRI scores) is shown. More empathic trainees, as measured with the EC and PD scales, showed better communication skills on the original MG. Patient-oriented communication skills were positively associated with three dimensions of empathy (FS, EC and PD) while task-oriented communication skills showed no association with any dimension of empathy at all.

Table 3.  Pearson correlations (r) between empathy (IRI scales) and communication skills (MG scales), including p-values (N = 68)

	MG scale
	MG total score		MG patient		MG task
IRI scales	r	p	r	p	r	p
PT	0.05	0.69	0.09	0.46	−0.08	0.50
FS	0.21	0.09	0.24	0.04	0.05	0.71
EC	0.26	0.03	0.30	0.013	0.05	0.70
PD	0.25	0.04	0.26	0.03	0.13	0.30

Note: Significant correlations are given in bold face.

Discussion

The objective of this study was to investigate the dimensional structure of the MG and explore the validity of this structure by studying the relationship between the MG sub-scales and personal characteristics of GP trainees. The findings of this study suggest that the MG may be considered as an instrument with a two-dimensional structure. We found a clear factor structure upon which the items of the MG had strong loadings. The consistency of the sub-scales was satisfactory. Explorative research supported the validity of this structure.

On the basis of their items, the two sub-scales can be characterized as two aspects of doctor–patient communication skills: patient- (or affect-) oriented and task- (or doctor-) oriented communication skills. Patient-oriented communication skills can be defined as interviewing skills that invite the patient to elaborate his needs and preferences and aiming at building a relationship with the patient (Laine & Davidoff 1996; Kurtz et al. 2003). A typical example of an item of the MG patient is ‘Exploration of expectations and feelings’. Another example is ‘Dealing with emotions’. In order to get a high score on this last item, the doctor should adequately reflect the patient's feelings. This technique aims at making the patient feel understood and as such can be defined as patient-oriented. Task-oriented communication requires communication techniques that help to provide structure in a consultation and not necessarily aim at building rapport with the patient. They reflect the more traditional medical history time line following the journey from complaint to diagnosis and ending up in a treatment plan. An item like ‘Communication about diagnosis and hypothesis’ is typically task-oriented. That these two dimensions are found in an instrument that assesses doctor–patient communication is new. However, it fits in with results found in other studies. Hall et al. (1987) classified physician communications as task-oriented or socio-emotional. They found evidence that physicians tended to adopt either a patient-oriented (giving information, counselling) or a physician-oriented approach (giving directions, asking questions).

The task-oriented sub-scale showed a significant relationship with prior communication skills training. This confirms previous research suggesting that task-oriented behaviour might be better trainable than patient-oriented behaviour. Roter and Larson (2002) have noted that:

the affective dimension of physician behaviour (development of rapport and responsiveness to the patient's emotions) is not generally regarded to have been acquired in medical school. This suggests that the expression of these behaviours (such as empathy, concern, reassurance) may depend on other factors than learned skills.

An interesting finding is that empathy scores correlated positively with the patient-oriented MG sub-scale but were not correlated with the task-oriented sub-scale. This suggests that empathy is a desirable skill in doctors: more empathic doctors tend to be better in patient-oriented communications skills and not worse in task-oriented skills. A similar result was found by Stratton et al. (2005). Because there are indications that empathy is not easily trained, this is valuable information for educators (Stepien & Baernstein 2006; Brunero et al. 2010). Three of the four MG items with the lowest scores (even after undergoing doctor–patient communication skills training) are part of the patient-oriented sub-scale (Clarification of the reason for encounter, Summarizing and Dealing with emotions) and it might, therefore, not be easy to improve these skills (Kramer et al. 2004; Liddell & Koritsas 2004).

The overall scores on the MG showed a relationship with country of birth and prior communication skills training. As for country of birth, this is a confirmation of the earlier reports of Laidlaw et al. (2006) and Liddell and Koritsas (2004) that non-native speakers score lower on communication skills. In our findings, they do so on both sub-scales.

The sub-scales found in this study may be of added value to the MG as it is commonly used in assessing GP speciality trainees for both educational and formative purposes. The original scale has good psychometric properties and according to Schirmer et al. (2005) may be considered as one of the better tools to assess doctor–patient communication skills.

The two-dimensional structure offers a new perspective to its use for educational purposes. For example, trainees show higher average scores on task-oriented communication skills and these skills also seem to be more trainable. This may be used in optimizing tailor-made doctor–patient communications skills programmes by spending less time on the easily trainable skills and focusing more on developing methods that have a positive influence on the less trainable skills. Another possibility is to use personal characteristics and IRI scores to decide which GP speciality trainees seem more likely to benefit from extra attention on certain aspects of doctor–patient communication.

Limitations of the study and further research

There are several limitations to this study. First, sample size was relatively small (68) which restricts the finding of significant effects. In the assessment of group differences in communication skills, for some characteristics (country of birth, prior training, gender), this resulted in very small sub-group sizes, which limits the reliability of the outcomes and may have prevented the detection of differences. On the positive side, the distribution of gender and country of birth is representative of the trend we see in GP training institutes: roughly 25% males and 10–15% born outside of the Netherlands.

Additionally, the small number of participants can potentially have led to unstable factor analysis results. Second, the use of (only) a two-station OSCE may limit generalizability of the results towards live consultations. In the scripts used for these consultations, more emphasis was put on patient-oriented communication skills than on medically oriented skills, which may not be representative of an average live consultation. These limitations suggest replicating the study with live consultations, a larger group or a greater number of consultations.

The relationship of personal characteristics and MG scores also justifies further research. First, to find additional support for the validity of the MG sub-scales. This may result from assessing the relationship between communication skills as measured by the sub-scales and personality characteristics of GP speciality trainees.

If the sub-scales consistently relate to certain personality characteristics, this confirms the idea that the MG measures different communication styles. In addition, these styles could be related to underlying psychological processes (Hall et al. 1987; Manuel et al. 2005; Libert et al. 2007). Second, this relationship is of value in finding ways to tailor educational programmes towards individual needs of GP trainees. If we could use personal characteristics to predict what the needs of individual GP speciality trainees will be at the start of GP training, it could help in realizing truly tailor-made educational programmes in doctor–patient communications skills.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.

Funding: This study was funded by the Dutch foundation for GP Speciality Training. The funders had no role in the design, conduct or analysis and interpretation of the data of the study.