https://orcid.org/0000-0003-0112-880X
Abstract
The aim of this study is to examine whether a preference for specific assessment methods in higher education is associated with personality and character strengths. Two-hundred and seventy Portuguese students completed a survey of character strengths, a Big Five personality test and their preference for each of six higher education assessment methods. Participants most favoured continuous assessment and multiple choice over viva voce and dissertations. Regression analysis showed that demographic factors, character strengths and personality accounted for 3–7% of the variance in the preferred examination method. Findings partially replicate previous investigations. Limitations and further research options are suggested.
How best to evaluate students’ knowledge? What is the fairest and most efficient method? Should students be able to choose their preferred method of assessment? What factors, like personality or ability, determine differences in student preferences?
There are diverse ways of assessing undergraduate students’ learning (Richardson Citation2015; Holzinger et al. Citation2020). Changes in technology and teaching methods have led to an interest in different methods of assessment (Villarroel et al. Citation2019). Having to change both teaching methods and assessment techniques as a function of the Covid-19 pandemic has also led to a renewed interest in assessment methods in educational settings.
The attention given to the preference for evaluation methods is important because student satisfaction with assessment methods may contribute to successful student outcomes (Martinez and Munday Citation1998; Pereira et al. Citation2022). Student dissatisfaction with evaluation methods may result in drop out from higher education (Zeidner Citation1987). The current research will examine students’ preferences for assessment methods. However, it should be recognised that students differ in their preferred assessment method and that it is unlikely that it could be designed or customised at an individual level.
Furnham, Batey, and Martin (Citation2011) described six evaluation methods: multiple choice examinations, essay-type examinations (e.g. answer four questions in two hours), final year dissertation (e.g. supervised thesis, four-month research project), oral examination (viva voce), continuous assessment (e.g. participation in class, attendance, course work, etc.) and group work (team assignment with the same mark for everybody in the group). The current study seeks to replicate the relationship between preference for academic assessments and personality (Chamorro-Premuzic et al. Citation2005; Furnham and Chamorro-Premuzic Citation2005; Furnham et al. Citation2008; Lakhal, Sévigny, and Frenette Citation2013; Wang and Brown Citation2014; Flores et al. Citation2015). It also seeks to extend the investigation in this field by scrutinizing the relationship between character strengths (Peterson and Seligman Citation2004; Furnham and Lester Citation2012; Neto, Neto, and Furnham Citation2014) and preferences for academic assessment method. That is, do students with different self-assessed strengths favour certain assessment methods over others?
Personality
Personality, as defined by the Big Five model, is a unique constellation of enduring traits and dispositions that governs individuals’ consistent cognitive, affective and behavioural patterns (Costa and McCrae Citation1995). Various studies have examined the relationship between personality according to the Big Five model and students’ preferences for their assessment methods. Furnham and Chamorro-Premuzic (Citation2005) testing British students, found stable (non-neurotic) and agreeable students preferred essay-type examinations; stable and extraverted students preferred oral examinations; and conscientious students preferred continuous assessment.
Chamorro-Premuzic et al. (Citation2005) examined the preference for specific assessment methods among Australian students. They showed that the less open students (those closed to experience) favoured multiple choice examinations; agreeable extraverts favoured oral examinations, stable extraverts favoured continuous assessment; and extraverts favoured group work.
Furnham et al. (Citation2008) found that personality, learning style, general knowledge and demographic factors accounted for 5–10% of the variance in preferred assessment methods among British students. Extraversion was positively associated with preference for multiple choice tests, vivas and group work evaluation; openness was positively related to essays and oral examinations, but negatively related to multiple choice tests and group work; agreeableness was positively associated with dissertations but negatively related to multiple choice; conscientiousness predicted a preference for continuous assessment.
In another study, Furnham, Batey, and Martin (Citation2011) tested British students and found openness was positively related to essays and dissertation, but negatively related to multiple choice tests and group work; stable and conscientious students favoured oral examinations. Intelligence was not clearly related to examination method choice.
Lakhal, Sévigny, and Frenette (Citation2013), testing a Canadian sample of 108 students, found extraversion predicted group work and oral examinations; openness to experience predicted group work, oral examinations and simulation; agreeableness predicted case study, group work, written examinations and projects while conscientiousness predicted group work.
In sum, these five studies consistently showed that personality traits significantly predicted student’s assessment preferences, although the explained variance was relatively small (between 2% and 19%). However, these studies did not present a unanimous pattern of those predictors, which may be because of other factors relating to personality.
Character strengths
In addition to personality traits, we also examine the character strength predictors of assessment method preferences. Peterson and Seligman (Citation2004) developed a conceptual framework of character strengths and argued that a relevant part of human functioning relates to strengths of character: ‘Character strengths are the psychological processes or mechanisms that define virtues’ (Park and Peterson Citation2006, p. 983).
Peterson and Seligman (Citation2004) classified 24 personal strengths with six ‘higher order virtues’: (1) Wisdom and Knowledge: cognitive strengths which require acquisition and use of knowledge such as creativity, curiosity, judgment, love of learning and perspective; (2) Courage: strengths which involve willingness to achieve goals despite internal or external confrontation such as bravery, perseverance, honesty and zest; (3) Humanity and Love: interpersonal strengths involving learning and supporting others such as capacity to love and be loved, kindness and social intelligence; (4) Justice: strengths underlying ‘healthy community life’, such as teamwork, fairness and leadership; (5) Temperance: strengths which are preventing from actions beyond what is usual or proper such as forgiveness, modesty, prudence and self-regulation; (6) Transcendence, strengths such as beauty, gratitude, hope, humour and religiosity.
Character strengths have been shown to be related to higher well-being and life satisfaction (e.g. Bdrar and Kashdan Citation2010) and academic achievement (Park and Peterson Citation2005). Preferences for assessment methods may thus mirror students’ character strengths. That is, they match what they believe to be their strengths with their preferences for evaluation methods.
Furnham and Lester (Citation2012) assessed character strengths with a short tool of the six higher order virtues. They showed that personality was more important to predicting self-rated strengths than demographics and ideology. The six virtues appeared related to the Big Five factors: Extraversion was associated with all six virtues; agreeableness was associated with humanity and love; conscientiousness was positively associated with courage and temperance, and negatively related to transcendence.
In this study, using Portuguese students, we tested the following hypotheses:
H1: Students’ personalities will predict preferences for assessment methods. More specifically:
H1.1: Extraverts will favour oral examinations and group work (Chamorro-Premuzic et al. Citation2005; Furnham et al. Citation2008; Lakhal, Sévigny, and Frenette Citation2013).
H1.2: Open individuals will favour viva voce (Furnham et al. Citation2008; Lakhal, Sévigny, and Frenette Citation2013).
H1.3: Conscientious individuals will favour continuous assessment (Furnham and Chamorro-Prezumic 2005; Furnham et al. Citation2008).
H1.4: Stable individuals will favour the dissertation method
H2: Character strengths will predict preferences for assessment methods. However, given the lack of research between character strengths and preferences for assessment methods no specific hypotheses are advanced.
Method
Participants
In all, 270 s year psychology students from the University of Porto participated in this study, of whom 212 were female and 58 male. Their mean age was 20.60 years (SD = 4.81).
The questionnaire
The questionnaire included three scales.
Choice of Assessment Method (Furnham and Chamorro-Premuzic Citation2005). This is a 24-item questionnaire with the following instructions: ‘There are many ways schools, universities and other educational institutions can assess or evaluate a student’s knowledge and learning. Thus, they can have traditional examinations, continuous assessment or even an oral examination. This short questionnaire is about your attitudes to, beliefs about, and preferences for six possible ways to assess people in education.’ It used a 10-point answer format (1 = ‘least preferred’ and 10 = ‘most preferred’). The questions and choices of the assessment method are presented in .
Abbreviated Big Five (McManus et al. Citation2003). This is a 15-item scale that assesses five factors. This scale was used in several publications (e.g. Furnham and McManus Citation2004; Neto, Neto, and Furnham Citation2014; Tan, Mansi, and Furnham Citation2018), and it evidenced to be reliable and valid (Furnham and Lester Citation2012). Participants rated each item on a 5-point Likert scale (from 1 = ‘strongly disagree’ to 5 = ‘strongly agree’). In this sample Cronbach’s alphas were: openness (α = .64), conscientiousness (α = .53), extraversion (α = .59), agreeableness (α = .60) and neuroticism (α = .54). These values are relatively low, but McManus and Furnham (Citation2006, p. 561) have pointed out that ‘they are each based on only three questions but are more than adequate for assessing population-level correlations’.
Self-Rated Character Strengths (SRCS, Furnham and Lester Citation2012; Neto, Neto, and Furnham Citation2014). This 24-item measure, based on the traditional Values in Action Inventory of Strengths (Peterson and Seligman Citation2004), was developed to assess six higher order virtues: wisdom (6 items; α = .68), courage (3 items; α = .54), love (2 items; α = .73), justice (3 items; α = .65), temperance (3 items; α = .63) and transcendence (7 items; α = .76).
Table 1. Mean scores (SDs) for the sample’s preferred assessment methods.
Participants were also asked to indicate personal details such as sex and age.
Procedure
Questionnaires were provided via a paper survey, with the purpose of the research explained at the beginning of the session. Students gave informed consent to participate in the research. The survey required about 15 min to complete. All participants were unpaid volunteers and fully debriefed.
Results
shows means and standard deviations for each type of preference. The rating scores indicate the strongest preferences for continuous assessment and multiple-choice examinations. A timed written essay paper, probably the most-used method in psychology, was also preferred. The students were strongly disinclined to have an oral examination.
The rating scores regarding the accuracy of each method show an identical variability but a different rank order. Continuous assessment and timed written papers were considered the most accurate, followed by the dissertation. Group work was judged to be the least accurate, after multiple-choice tests.
The rating regarding fairness also mirrored the variability of each method. Continuous assessment and dissertation were judged the fairest, followed by written essays. Group work was judged the least fair, after oral examinations.
Finally, responses regarding the percentage of each method students would personally prefer closely mirrored the responses for preferences.
Next, we performed correlations across all ratings. shows the bivariate correlations between the two sets of predictor variables (personality and character strengths). Neuroticism was negatively correlated with courage and transcendence. Extraversion was positively correlated with wisdom, courage, love, justice, temperance and transcendence. Openness showed a positive correlation with wisdom and temperance. Agreeableness was positively correlated with love, transcendence and neuroticism. Conscientiousness was positively associated with courage, justice, openness and agreeableness.
Table 2. Correlational analysis results.
shows the correlations between three of the assessment method ratings (prefer, accurate and fair) and character strengths and personality. A preference for multiple choice was negatively related to openness. A preference for essay-type examinations was reported by those who were conscientious. A preference for oral examinations was positively related to openness, temperance and transcendence. A preference for continuous assessment was associated with those who were courageous. A preference for a dissertation was negatively related to neuroticism, and positively related to temperance and transcendence. Extraverts with courage, love and justice favoured group work the most.
Table 3. Correlations between the three preference ratings, personality and character strengths.
To examine whether personality, character strengths and demographic factors predicted preference for different evaluation methods a set of hierarchical regressions were performed. presents two summarized sets of regressions. In the first set, the Big Five traits were entered in the first block of predictors, followed by sex and age of the participants. In six regressions, the different types of assessment method were the criteria. Personality traits emerged as significant predictors of preferences for oral examinations and continuous assessment. Although the amount of variance accounted for was relatively small (6% in both cases), there were several significant predictors. Age only emerged as a significant predictor of multiple-choice examination preference. This type of assessment was preferred by younger students. Extraversion positively predicted timed written paper and group work. Neuroticism negatively predicted preference for dissertation. Openness to experience positively predicted oral/viva voce examination. Contentiousness positively predicted continuous assessment.
Table 4. Results for multiple regressions analyses with the six assessment methods as the criterion variable: in the upper set of regressions the predictors were personality (big five), and demographic variables, while in the lower set, strength characters and demographic variables were the predictors variables.
The second set of regressions had the character strengths as predictor variables, followed by demographic variables. Character strengths positively predicted preferences for oral examinations and group work. Although the amount of variance accounted for was relatively small (6% and 7%, respectively), various significant predictors emerged. Similar to the results for the personality traits, for character strengths only age emerged as a negative and significant predictor. Wisdom negatively predicted dissertation and group work, whilst justice positively predicted the same assessment methods. Love was a significant and negative predictor of oral/viva voce examination, whilst transcendence was a positive and significant predictor of oral examination. Temperance and sex (female) were positive and significant predictors of timed written paper.
Discussion
This study explored the many differences in how students like to be assessed. Students preferred continuous assessment and indicated that this method is, in their view, particularly accurate and fair for measuring true ability. They also preferred multiple choice examinations, but they did not judge them to be highly accurate. Multiple choice examinations were preferred as students considered them relatively easy to prepare for, and better to obtain higher marks with than other evaluation methods (Van de Watering et al. Citation2008). The traditional timed essay paper was considered to be more accurate than multiple choice examinations. Group work received the lowest ratings for accuracy and fairness, probably because of a belief that the contributions would be unequal, yet the same grade would be given to each person.
Oral/viva voce examinations obtained low ratings for preference, accuracy and fairness. One possible explanation for the very low preference of oral examinations may be related to students’ familiarity with this evaluation method (Chamorro-Premuzic et al. Citation2005). This evaluation method may not be preferred as it is not commonly utilized with undergraduate students, yet many higher degrees (e.g. PhD) are examined by oral interrogation which may last for hours.
Specific individual predictors of preference were found. Overall, age and sex showed little effect, influencing only one preference each: younger students liked multiple choice examinations more than older students, and females liked essays more than males. This probably reflects self-confidence (Chamorro-Premuzic and Furnham Citation2005). Scouller (Citation1998) showed that higher performance in essays was related to adopting deep learning strategies and a perception that essays mobilize higher levels of cognitive processing.
The two global hypotheses of this study were supported, as personality and character strengths emerged as significant predictors of preferences for assessment methods. The findings showed that, except for agreeableness, the Big Five personality traits related to preferences for assessment methods regardless of age and sex. This is consonant with previous results (Chamorro-Premuzic et al. Citation2005; Furnham and Chamorro-Prezumic 2005; Furnham et al. Citation2008; Furnham, Batey, and Martin Citation2011). Nevertheless, our first hypothesis was only partially supported as extraverts did not favour oral examinations as predicted. The other specific hypotheses were supported.
Extraversion predicted a preference for essays and group work. Extraverts are positively inclined to group work, possibly because of their sociability and interpersonal skills and they enjoy working with others. This supports prior research (Chamorro-Premuzic et al. Citation2005; Lakhal, Sévigny, and Frenette Citation2013). Extraversion was also positively associated with preference for essay examinations. Perhaps the social confidence dimension of extraversion explains this preference.
Openness to experience significantly predicted oral examination preference, possibly due to the association between openness, curiosity, imagination and creativity. As openness is associated with originality/creativity (McCrae and Costa Citation1997) it may be anticipated that open personalities favour examinations that facilitate creative production.
Conscientiousness significantly predicted continuous assessment. Conscientious students work hard and are organised, planful and responsible. They tend to have more success at continuous assessment that requires dedication. This is also in agreement with prior work (Furnham and Chamorro-Prezumic 2005; Furnham et al. Citation2008),
Students with more emotional stability favoured a final report. This may reflect that highly neurotic students are not able to cope with the stress associated with a final project. It has been established that anxiety negatively effects assessment, and that neuroticism is negatively associated with intelligent test performance (Furnham Citation2021).
One unique aspect of this study was to explore the relationship between character strengths and preferences for evaluation methods. It was hypothesized that character strengths would predict preferences for assessment methods. This second hypothesis tended to be supported as all the self-rated strengths, except courage, emerged as significant predictors of preferences for assessment methods.
The virtue of wisdom negatively predicted dissertation and group work. In both methods of assessment, it may be that the students favour cognitive strengths involving the acquisition and use of knowledge, as supervisors or colleagues may provide wise counsel and knowledge to them. Humanity (e.g. strengths of character associated with helping others) negatively predicted oral examinations. This is understandable as humanity includes interpersonal strengths involving ‘tending to and befriending’ others, and oral examinations are often adversarial.
The virtue of justice comprises civic strengths underlying a healthy community life and predicted preference for dissertation and group work. Both assessment methods require direct interactions with tutors or peers. This link makes sense, as justice encompasses the strengths of teamwork, or working well as part of a team. Temperance (e.g. having self-control, being prudent, humble and modest) significantly predicted preference for essays. Transcendence, or strengths forging connections to the larger universe and providing meaning, was related with oral examination preference.
As the way in which students are taught has changed over the past few years (e.g. Zoom) so have there been calls to think of new ways of examining students, like ‘open-book’ examinations or 6–12 h opportunities to access and process material. It must also be acknowledged that examination methods vary between subjects, for a variety of reasons (Holzinger et al. Citation2020), and that students have been known to choose courses on how they are examined.
However, it would seem very difficult to design or customise assessments according to the preference of students based on their personality and strengths. Assessment methods are usually chosen by teachers for various reasons: to focus on different aspects of learning; to be maximally efficient and reliable; to fulfil the needs of regulatory authorities. Indeed, it may be that developments in machine learning mean that many forms of assessment are computer scored. The debate about the most accurate, appropriate and efficient way of assessing knowledge and skill gain through education will continue.
Like all studies, this work is not free of limitations. First, the sample was relatively small and there was a sex imbalance, with many more women than men. Although both this and other works have shown few sex and age differences in preferences for assessment methods, it is always desirable to have large samples. Second, the measures used in this work, like others in this area, are limited to self-reports. Third, the variables considered in this study accounted for relatively low percentages of explained variance (7% at most) yet are comparable to those reported in past research. This suggests that other variables could account for the variance. These might include the anxiety which accompanies examinations (Zoller and Ben-Chaim Citation1990), intelligence (Furnham et al. Citation2008), motivation, sense of self-efficacy and learning style (Struyven, Dochy, and Janssens Citation2005).
References
- Bdrar, I., and T. B. Kashdan. 2010. “Character Strengths and Well-Being in Croatia: An Empirical Investigation of Structure and Correlates.” Journal of Research in Personality 44: 151–154.
- Chamorro-Premuzic, T, and A. Furnham. 2005. Personality and Intellectual Competence. New Jersey: Lawrence Erlbaum Associates.
- Chamorro-Premuzic, T., A. Furnham, G. Dissou, and P. Heaven. 2005. “Personality and Preference for Academic Assessment: A Study with Australian University Students.” Learning and Individual Differences 15 (4): 247–256. doi:10.1016/j.lindif.2005.02.002.
- Costa, P. T., Jr., and R. R. McCrae. 1995. “Domains and Facets: Hierarchical Personality Assessment Using the Revised NEO Personality Inventory.” Journal of Personality Assessment 64 (1): 21–50.
- Flores, M. A., A. M. Veiga Simão, A. Barros, and D. Pereira. 2015. “Perceptions of Effectiveness, Fairness and Feedback of Assessment Methods: A Study in Higher Education.” Studies in Higher Education 40 (9): 1523–1534. doi:10.1080/03075079.2014.881348.
- Furnham, A. 2021. Twenty Ways to Assess Personnel. Cambridge: Cambridge University Press
- Furnham, A., and T. Chamorro-Premuzic. 2005. “Individual Differences and Beliefs concerning Preference for University Assessment Methods.” Journal of Applied Social Psychology 35 (9): 1968–1994. doi:10.1111/j.1559-1816.2005.tb02205.x.
- Furnham, A., and D. Lester. 2012. “The Development of a Short Measure of Character Strengths.” European Journal of Psychological Assessment 28 (2): 95–101. doi:10.1027/1015-5759/a000096.
- Furnham, A., and I. C. McManus. 2004. “Student Attitudes to University Education.” Higher Education Review 36: 29–38.
- Furnham, A., M. Batey, and N. Martin. 2011. “How Would You like to Be Evaluated? The Correlates of Students’ Preferences for Assessment Methods.” Personality and Individual Differences 50 (2): 259–262. doi:10.1016/j.paid.2010.09.040.
- Furnham, A., A. Christopher, J. Garwood, and N. G. Martin. 2008. “Ability, Demography, Learning Style, and Personality Trait Correlates of Student Preference for Assessment Method.” Educational Psychology 28 (1): 15–27. doi:10.1080/01443410701369138.
- Holzinger, A., S. Lettner, V. Steiner-Hofbauer, and M. Capan Melser. 2020. “How to Assess? Perceptions and Preferences of Undergraduate Medical Students concerning Traditional Assessment Methods.” BMC Medical Education 20 (1): 312. doi:10.1186/s12909-020-02239-6.
- Lakhal, S., S. Sévigny, and E. Frenette. 2013. “Personality and Preference for Evaluation Methods: A Study among Business Students.” Studies in Educational Evaluation 39 (2): 103–115. doi:10.1016/j.stueduc.2013.02.002.
- Martinez, P, and F. Munday. 1998. 9000 Voices: Student Persistence and Drop out in Further Education. London: Learning and Skills Development Agency.
- McCrae, R. R., and P. T. Costa. Jr. 1997. “Personality Trait Structure as a Human Universal.” The American Psychologist 52 (5): 509–516. doi:10.1037//0003-066x.52.5.509.
- McManus, I. C., and A. Furnham. 2006. “Aesthetic Activities and Aesthetic Attitudes: Influences of Education, Background and Personality on Interest and Involvement in the Arts.” British Journal of Psychology 97 (4): 555–587. doi:10.1348/000712606X101088.
- McManus, I. C., E. Smithers, P. Partridge, A. Keeling, and P. Fleming. 2003. “A Levels and Intelligence as Predictors of Medical Careers in UK Doctors: 20 Year Prospective Study.” BMJ 327 (7407): 139–142. doi:10.1136/bmj.327.7407.139.
- Neto, J., F. Neto, and A. Furnham. 2014. “Gender Differences and Psychological Correlates of Self-Rated Strengths among Youth.” Social Indicators Research 118 (1): 315–327. doi:10.1007/s11205-013-0417-5.
- Park, N, and C. Peterson. 2005. “The Values in Action Inventory of Character Strengths for Youth.” In What Do Children Need to Flourish: Conceptualizing and Measuring Indicators of Positive Development, edited by K. A. Moore & L. H. Lippman, 13–23. New York: Springer.
- Park, N., and C. Peterson. 2006. “Moral Competence and Character Strengths among Adolescents.” Journal of Adolescence 29 (6): 891–909.
- Pereira, D., I. Cadime, G. T. Brown, and M. A. Flores. 2022. “How Do Undergraduates Perceive the Use of Assessment? A Study in Higher Education.” European Journal of Higher Education 12 (1): 1–17. doi:10.1080/21568235.2020.1871393.
- Peterson, C, and M. Seligman. 2004. Character Strengths and Virtues: A Handbook of Classification. Washington, DC: APA Press.
- Richardson, J. T. 2015. “Coursework versus Examinations in End-of-Module Assessment: A Literature Review.” Assessment & Evaluation in Higher Education 40: 439–455.
- Scouller, K. 1998. “The Influence of Assessment Method on Students’ Learning Approaches: Multiple Choice Examination versus Assignment Essays.” Higher Education 35 (4): 453–472. doi:10.1023/A:1003196224280.
- Struyven, K., F. Dochy, and S. Janssens. 2005. “Students’ Perceptions about Evaluation and Assessment in Higher Education: A Review.” Assessment & Evaluation in Higher Education 30 (4): 325–341. doi:10.1080/02602930500099102.
- Tan, S., A. Mansi, and A. Furnham. 2018. “Students’ Preferences for Lecturers’ Personalities.” Journal of Further and Higher Education 42 (3): 429–438. doi:10.1080/0309877X.2017.1301406.
- Van de Watering, G., D. Gijbel, F. Dochy, and J. Van der Rijt. 2008. “Students’ Assessment Preferences, Perceptions of Assessment and Their Relationships to Study Results.” Higher Education 56 (6): 645–658. doi:10.1007/s10734-008-9116-6.
- Villarroel, V., D. Boud, S. Bloxham, D. Bruna, and C. E. Bruna. 2019. “Using Principles of Authentic Assessment to Redesign Written Examinations and Tests.” Innovations in Education and Teaching International 57: 1–49. doi:10.1080/14703297.2018.1564882.
- Wang, Z., and G. T. Brown. 2014. “Hong Kong Tertiary Students’ Conceptions of Assessment of Academic Ability.” Higher Education Research & Development 33 (5): 1063–1077. doi:10.1080/07294360.2014.890565.
- Zeidner, M. 1987. “Essay versus Multiple-Choice Type Classroom Exams: The Student’s Perspective.” The Journal of Educational Research 80 (6): 352–358. doi:10.1080/00220671.1987.10885782.
- Zoller, V., and D. Ben-Chaim. 1990. “Gender Differences in Examination-Type Preferences, Test Anxiety, and Academic Achievements in College Science education—A Case Study.” Science Education 74 (6): 597–608. doi:10.1002/sce.3730740603.