Weak self-directed learning skills hamper performance in cumulative assessment

Abstract

Background: Self-regulated learning is an important determinant of academic performance. Previous research has shown that cumulative assessment encourages students to work harder and improve their results. However, not all students seem to respond as intended. We investigated the influence of students’ behavioral traits on their responsiveness to a cumulative assessment strategy.

Method: The cumulative test results of a third-year integrated ten-week course unit were analyzed. The test was divided into three parts delivered at 4, 8 and 10 weeks. Low starters (below median) with low or high improvement (below or above the median) were identified and compared regarding their behavioral traits (assessed with the Temperament and Character Inventory questionnaire).

Results: A total of 295 students filled out the questionnaire. A percentage of 70% of the students below the median on the first two test parts improved during the final part. Students who were less responsive to improve their test results, scored low only on the TCI scale “self directedness” (t = 2.49; p = 0.011).

Conclusion: Behavioral traits appear to influence student reactions to feedback on test results, with students with low self-directedness scores being particularly at risk. They can thus be identified and should receive special attention from student counselors.

Introduction

“Assessment drives learning” is commonly held to be true. It has recently been shown that a cumulative assessment strategy encourages students who perform poorly at first to improve their performance (Kerdijk et al. 2013) and it encourages students to spend significantly more time on self-directed study. (Kerdijk et al. 2015) However, a small proportion of students do not appear to respond to the stimulus of cumulative assessment on studying and performance. We wonder whether the students’ behavioral traits play a role in any insensitivity to external assessment stimuli. Student counselors expend a lot of effort on poorly performing students. Knowledge about the behavioral reasons behind poor performance in the later years of medical training could help counselors guide students to adapt their study behavior more effectively. Therefore, the aim of this study is to explore the relationship between behavioral traits and responsiveness to external feedback from a cumulative assessment program. Our research question is: Do unresponsive students differ from responsive students with respect to their behavioral traits?

Methods

The cumulative test results of a third-year integrated ten-week course unit were analyzed. The test was divided into three parts delivered at 4, 8 and 10 weeks. The first two test parts contained half of all the test items, and the third part contained the rest. Students scoring below the median after the first two parts were divided into two groups based on whether they showed a high or low improvement (i.e. above or below the median) during the third part.

Table

Practice points

Cumulative assessment is a strong external stimulus for influencing study behavior. Cumulative assessment has a positive effect on student achievement. A minority of students do not benefit from cumulative assessment. Students at risk can be identified using the temperament and character inventory: they score low on self-directedness.

Behavioral traits were measured using the Temperament and Character Inventory (TCI) (Cloninger et al. 1994) which measures behavioral traits such as novelty-seeking, harm avoidance, reward dependence, persistence, self-directedness, cooperativeness and self-transcendence.

The two groups were compared using Students t-test, and the correlation between improvement and behavioral trait was assessed using Pearson’s correlation coefficient.

The TCI was chosen because previous research using the same questionnaire found an association between self directedness and intrinsic academic motivation in medical students (Tanaka et al. 2009) and because it contains dimensions of character that influence personal and social effectiveness by insight learning about self-concepts (Cloninger et al. 1993).

Results

The TCI was completed by 295 out of 383 students. After the first two parts of the test, 133 scored below the median. Of these, 93 (70%) showed an improvement above the median after the third test and 40 (30%) below. These two groups were then compared on their TCI scores. A significant difference was found between the groups only on the self-directedness TCI score (Table 1). Furthermore, there was a significant correlation between change in score and self-directedness (R = 0.219; p = 0.011).

Table 1. TCI dimension scores for students with high and low improvement.

	High (n = 93)	Low (n = 40)	p Value
Novelty-seeking	7.53 ± 0.27	8.10 ± 0.50	0.281
Harm avoidance	8.18 ± 0.44	8.30 ± 0.75	0.887
Reward dependence	10.63 ± 0.21	10.20 ± 0.31	0.262
Persistence	2.43 ± 0.17	2.75 ± 0.27	0.304
Self-directedness	20.35 ± 0.31	18.65 ± 0.69	0.011
Cooperativeness	21.41 ± 0.28	20.85 ± 0.53	0.311
Self-transcendence	1.97 ± 0.22	2.80 ± 0.47	0.068

p values are for independent sample T-tests.

Discussion

We explored the effect of behavioral traits on responsiveness to external feedback as implemented in cumulative assessment. We found that students who scored poorly on their first two test parts and did not improve much on the third had the lowest self-directedness scores. Cloninger et al. (1993) described that “the basic concept of self-directedness refers to self-determination and “willpower,” or the ability of an individual to control, regulate, and adapt behavior to fit the situation in accord with individually chosen goals and values”. The low self-directedness scores found in these students could suggest that students who are less internally driven are unlikely to respond sufficiently to external stimuli to perform better, such as cumulative testing. These students could be members of the group Dweck (1999) categorized as having an entity view. She argues that motivation depends on expectations and values. If students expect their ability to be limited, they will also believe that there is a limit to what they can achieve. In other words, they will live up to their expectations: they interpret failure as a reflection of their low ability and will probably give up more easily. This low self-directedness could also relate to the students’ limited self-regulated/directed learning ability, which is known to influence academic performance (Stegers-Jager et al. 2012). Students thus identified may need additional support to improve study performance. Our findings should be interpreted in the context of certain limitations. Associations between behavior and academic performance do not permit conclusions about causal relationships. However, behavioral traits are stable over time. It is very unlikely that these were influenced by academic performance during such a short period – only ten weeks – and it is more likely that the opposite is true. Furthermore, it is widely accepted that academic performance is influenced by ability (intelligence), opportunity (socioeconomic status) and motivation, which includes cultural norms and behavior (Blumberg & Pringle 1982). Another limitation may be the use of the TCI which is a psychobiological model Cloninger et al. (1993). It has however been used previously in the field of medical education (Tanaka et al. 2009) and it describes three dimensions of character that mature in adulthood and influence personal and social effectiveness (Cloninger et al. 1993) which are important for self directed learning.

Increased participation in a student counseling program is known to improve academic achievement in students at risk of low performance (Stegers-Jager et al. 2013). In addition, it has previously been shown that psychosocial factors have an incremental effect on top of previous achievement and can thus be used to prevent drop out and academic failure (Allen et al. 2009). In line with this previous research, the results of our study strongly suggest that a supportive program for students unresponsive to cumulative testing should consider their behavior.

Notes on contributors

RENÉ A. TIO, MD, PhD, is an Associate Professor, Center for Education Development and Research in Health Professions (CEDAR) and Department of Cardiology, University of Groningen and University Medical Center Groningen.

MARIKEN E. STEGMANN, Msc, MD, BA, has recently started her specialist training in general practice and is working on a PhD about oncology in primary care at the University of Groningen and University Medical Center Groningen.janke koerts, PhD, is a neuropsychologist and Associate Professor at the University of Groningen.

JANKE KOERTS, PhD, is a neuropsychologist and associate professor at the University of Groningen.

TITUS W. D. P. VAN OS, MD, PhD, is psychiatrist, psychoanalyst, and trainer Mental Health Organization Friesland, The Netherlands.

JANKE COHEN-SCHOTANUS, PhD, is a Professor and former Head of the Center for Research and Innovation in Medical Education, University of Groningen and University Medical Center Groningen, The Netherlands.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.