The relationship between medical students’ learning approaches and performance on a summative high-stakes clinical performance examination

Abstract

Background: A learning approach embeds the intention of the student when starting a task and the learning processes and strategies used to carry out a task. Student approaches to learning have been categorized as deep, strategic, and surface.

Aim: To explore the relationships among medical students’ learning approaches, gender, and performance on a summative high-stakes clinical performance examination (CPX).

Methods: We measured medical students’ learning approaches at the beginning of year four using the Approaches and Study Skills Inventory for Students and compared results with CPX scores.

Results: Student scores in the top two quartiles of the CPX were significantly higher on the deep approach than student CPX scores in the bottom quartile, and student scores in the bottom quartile of the CPX were significantly higher on the surface approach than scores for the other three CPX quartiles. CPX patient–physician interaction scores showed a significant positive correlation with deep approach scores, and CPX overall patient satisfaction scores showed a significant positive correlation with deep and strategic approach scores. Surface approach scores correlated negatively with all CPX score categories.

Conclusion: Approach to learning was associated with performance on a high-stakes CPX.

Introduction

There has been a great deal of interest in and research on students’ approaches to learning in higher education as well as medical education. A learning approach embeds the intention of the student when starting a task and the learning processes and strategies used to carry out a task (Marton & Säljö 1976; Entwistle 1991). Coffield et al. (2004) conducted a systematic review of learning style models, and identified 13 major models along a continuum; ranging from one end of the spectrum where learning approaches were viewed as being constitutionally based to the other end, where they were viewed as being more flexible and open to change. The flexible and open models were regarded more as learning approaches or strategies than as learning styles.

Learning approaches derive from Marton and Säljö's (1976) work on deep and surface approaches to learning, and further developed by Entwistle and Ramsden's (1983) description of three approaches to learning: a deep approach where students seek meaning in material, express interest in ideas, relate new ideas to previous knowledge, and use evidence critically; a strategic approach where students emphasize organized studying and good time management, driven by a desire for high achievement; and a surface approach where students memorize without understanding, leading to a fragmented knowledge base.

In medical education research, Hosford and Siders (2010) stated that several studies have explored the association between learning approach and academic performance. Studies showing associations between learning style and academic performance were primarily focused on knowledge acquisition. Using the Kolb Learning Style Inventory (LSI), students who preferred abstract conceptualization, compared to those who preferred concrete experience, performed better in the basic sciences, clinical pathology, general pathology, and physiology, Newland and Woelfl (1992) but not in interviewing skills (Davies et al. 1997). Because the nature of the academic discipline is thought to influence the learning strategies of students, there may be a difference between the learning approaches of students in medical schools compared to that of other students in higher education (Vermunt 2005). Mattick et al. (2004) reported that medical students scored higher on the deep approach when compared to other students in higher education.

In medical students, Lindblom-Ylanne and Lonka (1999) found that meaning-directed learning was positively related to both preclinical and clinical study achievements, and reproduction-directed learning was negatively associated with achievements. Meyer (2000) showed that “dissonant” student learning patterns were associated with low performance in exams. Smits et al. (2004) found that gender and learning style were related to an increase in learning and performance in postgraduate medical education, with the female gender being positively related to both.

In second-year medical students, the Approaches and Study Skills Inventory for Students (ASSIST) was used to determine learning approaches, and student scores on the modified essays and multiple choice questions were positively correlated with deep and strategic approaches and negatively with surface approaches. However, this relationship was not seen with in-course assignments (Reid et al. 2007). Mattick et al. (2004) reported that a deep approach to studying in the first year of medical school was associated with higher scores on tests of applied medical knowledge, and the strength of this relationship was found to increase over time. For first-year medical students learning anatomy, it was found that students used both surface and deep learning approaches, reflecting the use of both memorization, and understanding as key learning strategies (Pandey & Zimitat 2007).

As early as 1998, researchers found that the amount of knowledge that a student gains from clinical experience correlates positively with deep and strategic learning approaches as measured in the final year of medical school and also at the time of admission to medical school (McManus et al. 1998). As a consequence, the relationship between learning style and clinical competence received relatively more attention in medical education research. Van Lohuizen et al. (2009) found no significant relationships between learning strategies used during clerkships and students’ clinical performance. Most students used more than one learning strategy during clerkships, which may reflect the complexity of learning in the clinical setting. In contrast, Martin et al. (2000) found that performance on an objective structured clinical examination (OSCE) was related to a desire to achieve and well-organized study methods, as determined by Entwistle's LSI. The authors also found that female students scored 2.2% higher than the male students on the OSCE.

Since the clinical performance examination (CPX) comprises integration of communication skills, professional attributes, clinical skills, and knowledge in realistic clinical encounters (Howley 2004), we hypothesized that a student's learning approach could affect his or her performance on the CPX. We were unable to find studies reporting the relationship between learning approach and student performance on a high-stakes summative CPX, and therefore, the aim of this study was to explore the relationship between ASSIST learning approaches and CPX performance. Because the CPX measures students’ ability to take a history, conduct a physical exam, and interact with standardized patients, insight into the relationship of students’ learning approaches with aspects of clinical competency might be gained.

Table

Practice points

Deep approach to learning showed significant positive correlation with total CPX performance, primarily the patient–physician interaction and overall patient satisfaction CPX categories. Surface approach to learning showed significant negative correlation with total CPX performance and all CPX categories. Female students performed significantly better on the CPX than males and were more likely than males to use a strategic approach, but there were no significant gender differences in the use of the deep and surface approaches.

Methods

Subjects

Fourth-year medical students at the Keck School of Medicine of the University of Southern California were invited to participate in this study. IRB approval was obtained and all participants consented to the use of their data. A total of 160 of the 164 (97.6%) fourth-year medical students completed the ASSIST inventory. The sample consisted of 91 (56.9%) male students and 69 (43.1%) female students.

Research design

This was a correlational research study designed to explore the relationship between students’ learning approaches and their performance on a CPX. The students were asked to complete the ASSIST inventory at the beginning of year 4, and they took a CPX administered at the same time.

Instruments

The students’ learning approaches were assessed using the short version of the ASSIST. The ASSIST (Entwistle 2006) comprises 52 statements, to which students respond using a five-point Likert scale (5 = agree, 4 = agree somewhat, 3 = unsure, 2 = disagree somewhat, and 1 = disagree). This inventory has three main scales: deep, strategic, and surface, and each scale has a number of subscales. The students were asked to complete this inventory at the time that they took the CPX.

The ASSIST was chosen for its construct validity (Byrne et al. 2004; Coffield et al. 2004; Speth et al. 2007). Internal and external evaluations of the instrument showed satisfactory reliability and internal consistency. The internal consistencies (Cronbach's alpha) of the three ASSIST scales were 0.74 (deep approach), 0.73 (strategic approach), and 0.74 (surface approach). These coefficients are within the range reported in other research studies (Cronbach's alpha = 0.59–0.83).

The CPX is a required examination developed by eight medical schools constituting the California Consortium for the Assessment of Clinical Competence (CCACC). The exam consists of eight stations representing common clinical problems. For each station, students have 15 min to interact with the standardized/simulated patients (SPs) and another 7 min for a post-encounter exercise. This post-encounter could be a SOAP note, an X-ray, EKG, an eye model, etc. SPs scored students’ clinical performance using a predefined checklist with a dichotomous scale for the performance items, and a five-item Likert scale for the patient–physician interaction items. The CPX score was calculated for each student and station and summed across the stations according to five performance categories: (1) history taking, (2) physical examination, (3) information sharing/counseling, (4) patient–physician interaction, and (5) overall patient satisfaction.

Statistical analysis

After stratifying student CPX scores into quartiles, we used one-way analysis of variance (ANOVA) with a Tukey post hoc analysis to test for differences in ASSIST scale scores among the CPX quartiles. We performed student t-tests to determine whether males and females significantly differed on the CPX and in their approaches to learning. Pearson's correlation analyses were used to investigate relationships among ASSIST scales and performance category scores on the CPX. The Bonferroni adjustment technique was applied to control for Type I error across the three approaches to learning on CPX score levels and gender with a p-value of less than 0.017 (0.05/3 = 0.017) required for significance. All analyses were performed using SPSS software (ver. 17.0; SPSS, Chicago, IL, USA).

Results

The relationship between ASSIST scale scores and CPX quartile score levels

The ANOVA analysis resulted in a significant main effect for deep approach F (3,156) = 4.323, p = 0.006 and surface approach F (3, 156) = 9.383, p = 0.000, but not for strategic approach F (3, 156) = 2.535, p = 0.059. Tukey post hoc tests revealed that the deep approach score was significantly higher among students scoring in the top two quartiles of the CPX compared with those scoring in the bottom quartile, and the surface approach score was significantly higher among students scoring in the bottom quartile compared with other quartiles. Table 1 summarizes the ANOVA and Tukey test results.

Table 1  Differences in ASSIST scale scores by quartile of CPX scores

	Quartile of CPX score				Unit: mean ± SD
Approaches to learning	I (Bottom)	II	III	IV (Top)	p-value* Tukey post hoc test
Deep approach	57.40 ± 6.68	60.85 ± 8.29	62.23 ± 7.81	62.70 ± 6.17	0.006 I < (III, IV)
Strategic approach	69.75 ± 8.79	71.65 ± 11.30	71.28 ± 11.93	75.80 ± 8.70	0.059
Surface approach	48.53 ± 10.90	43.10 ± 8.62	41.00 ± 8.65	37.88 ± 8.62	0.000 I > (II, III, IV)

Note: *Indicates significance at p < 0.017 level (0.05/3 = 0.017).

The relationship between student gender, learning approaches and CPX performance

The mean score for the CPX was 72.27 (SD = 5.13). Female students had significantly higher mean scores (73.34 ± SD 5.28) than male students (71.46 ± SD 4.88) on the CPX (t = −2.319, p = 0.022). Of the five CPX categories, female students had a significantly higher score only on overall patient satisfaction (t = −2.620, p = 0.01). Female students scored significantly higher on the strategic approach scale than male students (t = −2.470, p = 0.015), but there were no significant differences between female and male students on the deep approach (t = 0.419, p = 0.676) and surface approach scales (t = 0.210, p = 0.834). See Table 2.

Table 2  Differences in ASSIST scale scores by gender

	Gender			Unit: mean ± SD
Approaches to learning	Male	Female	t	p-value*
Deep approach	61.01 ± 8.06	60.51 ± 6.76	0.419	0.676
Strategic approach	70.37 ± 10.66	74.42 ± 9.72	−2.470	0.015
Surface approach	42.77 ± 10.00	42.43 ± 9.97	0.210	0.834

Note: *Indicates significance at p < 0.017 level (0.05/3 = 0.017).

Correlation between ASSIST scales and CPX category scores

The CPX score for patient–physician interaction was significantly positively correlated with the deep approach score (r = 0.316, p = 0.000). The CPX score for overall patient satisfaction was significantly positively correlated with the deep approach (r = 0.254, p = 0.001) and strategic approach scores (r = 0.192, p = 0.015). The surface approach was significantly negatively correlated with scores for history taking (r = −0.261, p = 0.001), information sharing (r = −0.190, p = 0.016), patient–physician interaction (r = −0.217, p = 0.006), and overall patient satisfaction (r = −0.255, p = 0.001). Although surface approach also had a negative correlation with the score for physical examination, it did not reach a level of significance (r = −0.177, p = 0.025). See Table 3.

Table 3  Correlations between scores on ASSIST scales and CPX category scores

	CPX categories
Approaches to learning	History taking	Physical examination	Information sharing	Patient—physician interaction	Overall patient satisfaction
Deep approach	0.048	0.007	0.154	0.316*	0.254*
Strategic approach	0.126	0.120	0.061	0.059	0.192*
Surface approach	−0.261*	−0.177	−0.190*	−0.217*	−0.255*

Note: *Indicates significance at p < 0.017 level (0.05/3 = 0.017).

Discussion

We found a significant positive relationship between the deep learning approach and CPX scores, primarily those related to doctor–patient relationship and patient satisfaction. In earlier studies, Martin et al. (2000) had found that student performance in an OSCE at the end of their first clinical year was correlated with a well-organized and strategic learning style, but not with a deep approach nor with a surface learning approach. Van Lohuizen et al. (2009) did not find any significant relationships between learning strategies and clinical performance, and postulated that the lack of a relationship between the deep approach and OSCE could be because the deep approach may not be useful for a skill-based examination. Carrick (2010) found that students’ strategic and deep learning approach scores were significantly higher in the clinical learning environment. A deep clinical learning approach had a positive impact on the students’ course grades. In our study, we found that the deep approach correlated significantly positively with the two CPX categories of patient–physician interaction and overall patient satisfaction. The importance of using patient-centered communication in the patient–doctor encounter is well-documented in the literature (Fiscella et al. 2004; Epstein et al. 2005; Saha and Beach 2011). We believe that students who use a deep approach to learning may be able to elicit the patient's problems better, tailor information to what the patient wants to know, and determine how much the patient wants to participate in the decision-making.

The second important finding in our study is the consistent negative correlation between the surface approach and scores on all categories of the CPX. In a recent dissertation, Carrick (2010) reported a negative correlation between the surface approach and achievement in both classroom and clinical settings. This negative correlation between the surface approach and student achievement was also found in other previous studies (McManus et al. 1998; Biggs et al. 2001; Entwistle 2005; Biggs & Tang 2006). The results of this study are consistent with this research.

We found that female students had a higher score overall on the CPX than male students, which was similar to the results found by previous authors (Martin et al. 2000; Haist et al. 2003; Haq et al. 2005). This is similar to the findings by Boulet et al. (2003) who reported that female students scored better than their male counterparts overall, and in the specific area of interpersonal skills. The literature shows that, in clinical skills assessments, female students generally score better on interpersonal skills than males (Vu et al. 1993; Chambers et al. 2001). Wahlqvist (2007) reported that female students scored higher in patient-centeredness, similar to results of earlier studies (Krupat et al. 1999; Roter & Hall 2004). Laidlaw et al. (2006) in their study of first- and second-year residents found that females scored significantly higher in communication skills performance. In our study, female students scored significantly higher than male students on overall patient satisfaction, although there was no significant difference in the other CPX categories. We postulate that the patients were more satisfied with female students, who communicated better and were more patient-centered than the males. On average, female students chose the strategic approach more often than males, and the strategic approach was also significantly correlated with overall patient satisfaction. Further studies are needed to tease out whether the higher patient satisfaction scores are due more to female gender or to strategic learning style.

What are the implications these results have for medical education? Although the literature is mixed on whether deep and strategic learning approaches can be developed in students, faculty can adopt teaching and examination methods that discourage students from adopting a surface learning approach. De Lima et al. (2005) reported that residents in a cardiology training program perceived the Mini Clinical Examination Exercise as favorably influencing a constructive approach to study and learning. Ramsden (2005) stated that “the single most intense effort to be made in course planning, and in the setting of assessment questions, is to avoid presenting a learning context which is perceived by students to require, or reward, surface approaches.” Faculty and testing administrators must work together to ensure an environment that encourages deep and strategic learning approaches and discourages surface learning.

Limitations

The use of a single institution limits the representativeness of this study for the entire population of medical students. In addition, this study only investigated one of the instruments designed to measure student approaches to learning, and other learning approaches instruments may provide different results.

Conclusions

The deep approach to learning, as measured with ASSIST, was significantly related to higher scores on a high-stakes CPX, especially in the areas of patient–physician interaction and overall patient satisfaction. Surface approach to learning scores were negatively correlated with all CPX category scores, indicating that the surface approach to learning may not be an effective strategy for developing clinical skills. Female students scored higher than males on the CPX, and more often chose the strategic approach.

Acknowledgments

The authors acknowledge their thanks to Justin P. Lee, MD, who helped in data entry and preliminary analysis. The authors also thank Elie Kim who assisted in data entry and Josefina Cruz who assisted with the data collection. Dr Chung's work was supported by a grant of the Chonnam National University Hospital (CRI08064-1).

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.