How much is too much? Medical students’ perceptions of evaluation and research requests, and suggestions for practice

Abstract

A combination of institutional requirements and research activity means health profession students receive many requests to complete evaluations or participate in research. This study aimed to understand pre-clinical medical students’ perceptions and attitudes towards these requests. A prospective audit of evaluation and research requests to students during 2022 was undertaken to identify the volume and frequency of requests, and inform survey development. The online survey included questions about request frequency, volume, and timing. Two student cohorts received 42 and 34 evaluation plus 8 and 10 research requests, respectively. Responses (n = 167, aggregated response rate 28%) showed 70% felt they received too many evaluation requests, 76% indicated evaluation request volume should be limited, and 30% indicated receiving evaluation requests was ‘a little stressful’. Students indicated reasonable evaluation and research request frequency was 1–2 per month, with yearly maximums of 10–19 evaluation and 10 research requests. Students preferred receiving requests at the start of semesters; examination and holiday times were least preferred. Findings indicate students feel over-evaluated with the current evaluation schedule; students suggested the Medical School should regulate requests. The current research schedule was viewed as reasonable. Outcomes may guide institutional practice around request delivery.

Keywords:

• Student evaluation
• research request
• student perspective
• teaching evaluation

Introduction

Students undertaking tertiary education are often asked to volunteer their time for activities that are not part of their formal course requirements. They may be asked to evaluate their experience of course content and the quality of teaching (Schiekirka et al. 2012; Flodén 2017; Constantinou and Wijnen-Meijer 2022), or to participate in research projects (Christakis 1985; Forester and McWhorter 2005; Boileau, Patenaude, and St-Onge 2018; Devine et al. 2019). Medical students are often a population of interest for evaluation and research requests as they are an easily accessible cohort of students (Christakis 1985; Dyrbye et al. 2007; Constantinou and Wijnen-Meijer 2022), existing within a course that is often subject to examination, and quality control measures because of the high standards associated with delivering and maintaining a medical degree (Blouin et al. 2018).

Promoting meaningful student engagement with evaluation and research requests is important in order to make such interactions purposeful. Various strategies have been proposed for ensuring good response rates when asking students to undertake evaluations, including providing time within class to complete the evaluations (Kuch and Roberts 2019; Joubert, Steinberg, and van der Merwe 2022), and providing feedback to ‘close the loop’ once data have been analysed (Spencer and Schmelkin 2002). Studies also indicate that students overburdened with evaluation requests tend to participate less (Spooren and Van Loon 2012; Constantinou and Wijnen-Meijer 2022). What remains unclear from these studies is ‘how many is too many’?

Very few studies have described the effect of student response rates in combination with actual data of evaluation request volume. Adams and Umbach (2012) showed response rates for student evaluations of teaching (SET) decreased over sequential semesters if students were administered 11 or more SETs per semester, but previously published ‘guidelines’ and ‘recommendations’ around SET use contain little information regarding request volume, frequency or student perspectives on these variables (Oermann et al. 2018; Kreitzer and Sweet-Cushman 2022). Regarding research requests, studies have indicated student support for being participants in research (Forester and McWhorter 2005; Roberts and Allen 2013), and while medical students have described a professional responsibility to participate in research (Devine et al. 2019) there are indications some medical students feel they are approached ‘too frequently’ to be participants in research projects (Joubert, Steinberg, and van der Merwe 2022), and that scheduling approaches should be considered (Joubert, Steinberg, and van der Merwe 2019).

Despite the observation that ‘too many’ evaluations decreases engagement, no studies appear to have explored what frequency or volume of evaluation requests are viewed as reasonable by medical students. Similarly, no empirical evidence exists to indicate what frequency or volume of research requests is reasonable, an important consideration when medical students are viewed as a ‘vulnerable’ population of research participants (Maggio et al. 2018; Joubert, Steinberg, and van der Merwe 2019; Fisher and Rothwell 2022) and many already experience demanding workloads (Sarpel et al. 2013). Further, it is unknown whether or how research requests and evaluation requests may be viewed differently by the same cohort of students, or how they may react to these requests. The aim of this project was therefore to explore factors around how students perceive and experience evaluation and research requests in the early stages of a medical course. Audits of evaluation and research requests were utilised to inform the development of a cross-sectional survey, acquiring information on student attitudes, and perspectives that is informed by their ‘lived experience’ of request frequency and volume. The outcomes will assist how to appropriately deliver evaluation and research requests to early-course medical students (Kite, Subedi, and Bryant-Lees 2015; Fisher and Rothwell 2022), and may be useful in guiding such processes in other tertiary education areas.

Methods

Study location and description

The University of Otago is a research-intensive tertiary institution located in Dunedin, New Zealand. The Bachelor of Surgery and Bachelor of Medicine (MB ChB) degree is a six-year, undergraduate course with 300 students per cohort/year located on the same campus for the first two years (pre-clinical). When students are accepted into this course, they enter at year two of the programme. This study focused on medical students in the pre-clinical years two and three.

The University of Otago academic year runs congruently with the calendar year (i.e. January to December), and the MB ChB programme for pre-clinical students comprises two semesters of 17 and 13 weeks length, respectively, separated by a two-week holiday period. A mid-semester break of one week is included within each semester, and one week for study pre-examinations is at the end of semester two. A typical teaching week for years 2 and 3 comprises contact time of approximately 10 lecture hours and 12 hours of small-group tutorial or laboratory work. In addition to personal study time, task oriented independent (individual or group) learning comprises 6–10 hours per week. During 2022, the majority of the curriculum was delivered face-to-face in semester two, whilst the first half of semester one was influenced by the COVID-19 pandemic, and teaching was mostly online. Assessment within the programme consists of multiple low stakes events throughout the year and higher stakes assessments at end of year, with the overall delivery of assessment encompassing the principles that underpin programmatic assessment (van der Vleuten et al. 2012).

Three forms of data collection were involved in this study: a survey, a retrospective audit of research ethics approvals and a prospective audit of evaluation and research requests. Audit data allowed for benchmarking of evaluation and research requests students receive, facilitating interpretation of results against the reported student experience, and provided data that informed the development of the survey. A retrospective audit of evaluation requests was not possible as mechanisms to extract these data do not exist.

Survey

Ethical approval for the survey was acquired (University of Otago Human Ethics Committee; reference D21/400). An electronic survey link was distributed to the study cohort via student email. Consent was assumed via engagement with the electronic study hyperlink and survey platform. The survey was open for two weeks with a reminder sent after one week. In addition to email invitation, students were encouraged to participate via word of mouth and social media platforms. The online survey was comprised of multiple choice, Likert-type, and free-text questions. Limited demographic data were acquired (e.g. self-identified gender and admission pathway). Survey responses were anonymous.

Survey data was analysed for differences between student perceptions of research and evaluation requests in relation to volume and frequency, and student recommendations about timing, volume and frequency. Statistical analyses included t-tests for continuous data, and Chi-square or Wilcoxon rank sum for categorical variables; significance was p < 0.05. Free-text entries to the question ‘Are there any other comments you would like to make about evaluation and/or research requests’? were analysed by thematic analysis to explore patterns within the data (Braun and Clarke 2006).

Prospective audit data—evaluation and research requests

An audit of the evaluation and research requests students received from January to December 2022 was performed by a year 2 (author WG) and year 3 student (author JT). Records included request type (evaluation or research), date of request, how the request was delivered, and the estimated duration to complete the request to an adequate standard.

Retrospective audit data—research ethics approvals

Research requests that medical students receive are granted ethical approval by the University of Otago Ethics Committee. A retrospective audit of the titles and home departments of all University of Otago ethics approvals from 2017 to 2021 was performed to identify the potential number of research requests medical students may have experienced in previous years. Research was deemed ‘likely’ to include medical students if the project title included the term ‘medical student’.

Results

Survey—demographic information and response rate

The aggregate response rate was 28% (year 2, n = 76, 26%; year 3, n = 91, 31%). In general, item response distributions showed no significant difference between years 2 and 3, and unless otherwise stated data for both years was combined. Respondents’ gender (p = 0.76) and entry pathway to medical school (p = 0.43) were representative of the cohort.

Survey—evaluation and research request frequency and volume

Evaluation request frequency for year 2 and 3 differed across the year and peaked at different times (Table 1). Both year 2 (50%) and year 3 (51%) students indicated that at these peaks the frequency of evaluations was too high, but was otherwise reasonable throughout the year. This compares to an average across both years of 33% of students who indicated the overall frequency was reasonable and 16% that it was not reasonable. Correlating the frequency data (year 2 peak months March–June, year 3 peak months June–September; average of 6 per month) with the perception data indicates that 50% of students consider a monthly evaluation request of greater than five as too high.

Table 1. Prospective audit data for all evaluation and research requests, and mode of request, for year 2 and 3 medical students at the University of Otago during 2022.

	Request type
	Evaluation		Research
	Year 2	Year 3	Year 2	Year 3
January				1
February			1	1
March	9	1		1
April	3	3
May	4	2	1
June	6	7	1	1
July	1		2	2
August	4	7	1	1
September	9	9	2	2
October	3	3		1
November	2	1
December	1	1
Total	42	34	8	10
	Request mode
Student email	32	24	8	8
During class time	6	15	0	2
Online learning platform	9	9	0	0
Social media pages	1	1	1	2
Total	46	47	9	12

Totals for mode of request differ from total request numbers due to requests sometimes being delivered in multiple ways.

Current research request volume was viewed as reasonable by 96% of students, as was frequency (96%). Enforced limitation on the frequency of requests was also favoured for evaluations over research (69% versus 31%; p < 0.001). Of those who thought the frequency of evaluation or research requests should be limited, 71% indicated the frequency of evaluations should be no more than one per fortnight, whilst for research requests 84% suggested no more than one to two per month was reasonable.

More students thought the number of evaluation requests should be limited by the medical school than research requests (76% versus 32%; p < 0.001). Of those that thought the number of evaluation or research requests should be limited, approximately equal numbers thought up to 10 (41%) or between 10 and 19 (50%) evaluation requests were reasonable across the year, whilst for research up to 10 requests were rated as reasonable by 48% of respondents.

When asked to rank when they would prefer to receive requests, early in both semesters or at the end of semester one were the most favoured, with the end of semester two and during examinations least favoured.

Survey—experiences and impressions of evaluation and research requests

Students consistently viewed research requests differently to evaluation requests. They were more likely to engage with research (44%) than evaluation requests (27%), whilst 23% stated they had no real interest in engaging with evaluation requests compared to 6% for research requests.

Students felt they received too many evaluation requests (70%) compared to too many research requests (12%; p < 0.001), and fewer students rated the number of evaluation requests as reasonable compared to the number of research requests (36% versus 85%; p < 0.001). Students were more likely to rate their first reaction to receiving an evaluation request as stressful (31%) compared to when research requests were received (20%; p = 0.02).

Survey—free text responses

There were n = 19 year 2, and n = 21 year 3 free text responses; responses from both years were pooled for analysis. The most dominant theme, associated with the majority of comments, centred on improving the delivery of evaluations, including suggestions to promote engagement with or delivery of the evaluations (Table 2). This included comments associated with evaluation length, delivery and timing. Other minor themes included facilitating access to research opportunities, and the negative personal impact of receiving evaluations including guilt and stress.

Table 2. Examples of free-text comments and associated themes from year 2 and 3 students to the question ‘Are there any other comments you would like to make about evaluation and/or research requests’?.

Student comment	Theme
Instead of sending requests, could I send feedback through a form if I have anything to say? Otherwise y’all are just spamming my inbox (sometimes at 3am)	Delivery of evaluations (Major theme)
I feel evaluations about lectures we have for 1 lecture is unreasonable, we barely have time to get to know them. Maybe they have to give 4+ lectures to be able to send out an evaluation, or some kind of threshold
Maybe an alternative way of delivering the requests. They can often get lost in emails as they are not usually super urgent/important compared to other emails.
There should just be an open evaluation box (online)—if someone feels like they have valuable feedback to contribute they can submit it through that box. It means you can submit if you would like to, don’t have to if you don’t want to
If you want feedback on a lecture/lecturer/and or course, a good way would be to have a satisfaction stand outside the lecture theatre. Then, only follow up on negative results with further surveys.
I would like to be contacted for more research requests—being part of research is something that interests me.	Facilitating access to research opportunities (Minor theme)
I barely ever engage with evaluation requests but I always participate in research when I see it.
It’s not as much stress about receiving requests, it’s more the guilt when ignoring a request (because you are busy/don’t have time)	Negative personal impact of receiving requests (Minor theme)
Way too many evaluations. As soon as I get an email where I see it in the subject matter I delete it without opening. Very stressful when you’re already inundated with emails and work to get these emails which will not benefit me but are for the lecturer.

Prospective and retrospective audit data—evaluation and research requests

During 2022, year 2 students received 41 evaluation and 8 research requests, while year 3 students received 33 evaluation and 10 research requests (Table 1). Requests were delivered to students via multiple delivery modes; student email was most prevalent for both evaluation and research requests (Table 1), with social media invitations least frequently used.

The estimated duration to complete all evaluation requests was 290 minutes (year 2) and 235 minutes (year 3) (average duration 6.9 minutes; range 5–25 minutes). The estimated time to complete all research requests was 220 minutes (year two) and 200 minutes (year three) with an average duration of 23.3 minutes (range 5–70 minutes).

From 2017 to 2021 there were 2950 University of Otago ethics approvals for research projects. On average, 10.8 approvals per year ‘likely’ included medical students as participants (Table 3). Data indicate research requests for 2022 likely being consistent with previous years.

Table 3. Retrospective audit data of University of Otago research ethics approvals to identify projects where year two and three medical students may have been participants.

University of Otago research ethics approvals involving medical students as participants
Year	Likely
2017	9
2018	16
2019	11
2020	11
2021	7
Total	54
Average	10.8

‘Likely’ involved identifying projects for medical students (e.g. having ‘medical student’ in the title of the project). The total number of ethics approvals from 2017 to 2021 was 2950.

Discussion

This study examined pre-clinical medical student perceptions of the volume and frequency of evaluation and research requests, finding many feel over-evaluated and that the current delivery of evaluation requests was not reasonable. Conversely, existing trends around research requests were mostly viewed as reasonable. Interestingly, some students find the receipt of these requests stressful, and many suggested the Medical School needs to regulate how evaluation and research requests are delivered. Students suggested volumes and frequencies of requests they regard as reasonable, as well as indicating preferences for when and how these are delivered throughout the year.

Student perspectives on evaluation requests

While the use of evaluations to gauge faculty performance continues to be debated (Deaker, Stein, and Spiller 2016; Hornstein 2017), they are accepted by staff despite their limitations (Stein et al. 2013) and remain ubiquitous within the tertiary environment for purposes such as assessing teaching performance (Flodén 2017) and undertaking course evaluation (Spooren and Van Loon 2012). However, getting students to engage with evaluation requests remains an issue and a lack of student participation may adversely influence the validity of the information gathered. Leaving students to identify which requests to engage with is currently the status quo, and global observations of poor student response rates to evaluation requests (Adams and Umbach 2012; Spooren and Van Loon 2012; Constantinou and Wijnen-Meijer 2022) suggest change is required to increase participation. It is necessary to ensure student response rates are reasonable, and while students see the necessity to engage with evaluations (Kite, Subedi, and Bryant-Lees 2015), there are many reasons why they choose not to, including perceptions around subsequent improvement in teaching or the course (Spencer and Schmelkin 2002; Spooren and Christiaens 2017; Monzón, Suárez, and Paredes 2022), the mode of evaluation delivery (e.g. online) (Plante, Lesage, and Kay 2022), student characteristics (Spooren and Van Loon 2012), lack of knowledge around how responses are used (Stein et al. 2021), and a lack of time (Adams and Umbach 2012; Kite, Subedi, and Bryant-Lees 2015; Stein et al. 2021). Responses in this study are congruent with previous findings, including reference to number of requests, timing and mode of delivery, with email requests being singled out as problematic despite reports of some students preferring this mode of delivery (Schiekirka et al. 2012).

Yet, despite the volume of research undertaken around student engagement with evaluations, few studies have explored how the volume or frequency of evaluation requests is perceived by students. A single study (Adams and Umbach 2012) shows student response rates to evaluations drop off when more than 11 per semester are delivered, and findings from this current study indicate a similarly pessimistic trend: the majority of students (63%) perceived 30 to 40 evaluations per year as not reasonable. At the present volume and frequency, 23% of medical students indicated they had no real interest in engaging with evaluation requests, with 70% believing they receive too many. These data are in agreement with others that indicate students overburdened with evaluation requests tend to participate less (Spooren and Van Loon 2012).

How can evaluations be delivered in a manner that is in keeping with student perspectives? Interestingly, no empirical evidence exists to illuminate the student voice around evaluation request frequency. Here, many suggested the Medical School should limit evaluation numbers to between 10 and 19 requests a year; students also suggest a frequency of no more than one request a fortnight was reasonable, and indicated that the peak frequencies of evaluation request they experienced (different between years 2 and 3; on average more than 5 per month) were too high (Table 1).

An interesting aspect of the data is that some students (31%) indicated receiving these requests was stressful, a finding supported by free-text responses. In an environment whereby the contribution of medical schools to adversely influencing student mental health remains under scrutiny (Smith et al. 2007; D’Eon et al. 2021), tertiary institutions must remain mindful of perceived stressors (Slavin, Schindler, and Chibnall 2014; Slavin 2016) and this finding requires further exploration to determine how such stress manifests and could be mitigated. Trialling information sessions to students at the inception of a course to raise their awareness around evaluation and research requests, and any responsibilities around participation in these, may help alleviate any uncertainty around student participation and potentially assist in decreasing any stress experienced by receiving them.

Student perspectives on research requests

Students are often willing to be participants in research (Brewer and Robinson 2018; Joubert, Steinberg, and van der Merwe 2022), and may be motivated to participate in research by the potential benefits (Roberts and Allen 2013), while the time required will also affect participation (Sarpel et al. 2013; Devine et al. 2019). In this study, student free-text responses suggest interest in receiving research requests, highlighting that some students are genuinely interested in participating in research, a response which has also been noted elsewhere (Fisher and Rothwell 2022).

Findings indicate that the majority of students (85%) felt they were receiving a reasonable number of research requests throughout the year (roughly one per month), and only 20% of students found receiving these research requests stressful. Does this indicate increased attention should be paid to how students receive requests? Arguments for overseeing research requests include medical students being a vulnerable population (Ridley 2009; Bradbury-Jones et al. 2011; Walsh 2014; Maggio et al. 2018; Joubert, Steinberg, and van der Merwe 2019) that has the potential to be viewed as under pressure to participate (Bartholomay and Sifers 2016; Joubert, Steinberg, and van der Merwe 2022) or experience ‘undue influence’ (Callahan, Hojat, and Gonnella 2007; van den Broek, Wouters, and van Delden 2015), despite some students sometimes not perceiving themselves this way (Sarpel et al. 2013). Yet despite the widespread identification of students being ‘vulnerable’ in their role as research subjects, there exist no guidelines or data to indicate what frequency or volume of research requests is reasonable, even with the suggestion that scheduling of approaches needs to be considered (Joubert, Steinberg, and van der Merwe 2019, 2022).

Approximately one third of students thought the Medical School should limit the number of research requests to not more than ten per year, and that frequency should be limited to one to two requests per month. However, 85% suggested their experienced volume (between eight and ten per year) and frequency (Table 1) was reasonable, indicating that students are not concerned about the current volume or frequency of research requests. Therefore, to decrease the negative impact of receiving research requests (Fisher and Rothwell 2022), it can be suggested that research requests of between one and two per month, with a maximum of ten requests per year, is recommended for similar cohorts of students. The indication that some students are stressed by receiving these requests requires further exploration to see how this may inform request delivery.

Are evaluation and research requests viewed differently by students?

It is clear that decision-making by students around engaging with both research (Fisher and Rothwell 2022) and evaluation (Hoel and Dahl 2019) requests is multifactorial. Certainly, timing of requests likely contributes (Sarpel et al. 2013; Kite, Subedi, and Bryant-Lees 2015), and participants indicated that periods of the year that are academically quieter periods were preferred for receiving requests. This seems logical, as students may not feel burdened or overloaded in these periods, and fits with previous suggestions around avoiding busy times (Spooren and Van Loon 2012; Fisher and Rothwell 2022). However, there are likely other factors that contribute to student perspectives around receiving requests.

Students consistently indicated more negative views on engaging with evaluation requests with almost a quarter indicating they had no real interest in engaging with them. Conversely, students preferred to engage with research requests. Contributions to student perceptions from request volume and frequency appear likely; students indicated the current number of evaluation requests as being too many and unreasonable compared to research, despite the total time to complete all evaluation or all research requests being not dissimilar (290 or 235 versus 220 or 200 minutes, respectively, for the two student cohorts), supporting the notion that low frequency and volume of request contribute to perceptions of reasonableness. Consistent with this, students also more strongly favoured the Medical School regulating the number and frequency of evaluation requests, despite the average time for participating in research requests (23.3 minutes) being much larger than evaluation requests (6.9 minutes). Currently, at this institution, the volume of evaluation and research requests to students is not restricted.

Findings also raise important questions and challenges. Students were significantly more likely to indicate receipt of evaluation than research requests as stressful, underscoring an important and novel observation around student reactions. How this ‘stress’ should be interpreted is unclear, and requires further exploration in the context of student health.

There is little available evidence to support whether the effect on students of these two different types of request should be viewed concurrently or separately; however, at face value these data suggest decisions around engaging with either type of requests may be distinctly different. Comparison between evaluation and research requests in this study is hindered by the disparity between the numbers of evaluation and research requests, though findings indicate volume and frequency as likely metrics which influence student perspectives, with student stress, spare time and assessment proximity all being factors that may affect student perceptions around request delivery. Furthermore, inter-institutional differences, with variation in and across these variables, including different ethnic, cultural and social influences, may also affect student perceptions. These data are, however, a useful benchmark in an area where few currently exist, and may serve as a convenient proxy for justifying the regulation of requests.

Suggestions for good practice in similar student cohorts

In the light of these findings, further areas that may improve the practice of request delivery are presented. There has been a call for international standards for research involving students (Leentjens and Levenson 2013), and further guidance around the use of evaluations (Turner, Hatton, and Valiga 2018), and perhaps within such standards the frequency and volume of requests should be considered. Based on these current data it can be suggested that, for students experiencing similar coursework, load and environmental stressors, the following may guide good practice around evaluation and research requests to students:

• a frequency of evaluation requests above five per month is not recommended,

• the frequency of evaluation requests is limited to one per fortnight,

• a reasonable number of evaluation requests is not more than 10 to 19 per year,

• the frequency of research requests is not more than one per fortnight,

• a reasonable number of research requests is not more than 10 per year.

Email approaches are problematic with students suggesting these are easily overlooked, and other solutions are required to facilitate an increase in engagement. Timing of approaches should not be in busy or stressful times of the year (Spooren and Van Loon 2012; Joubert, Steinberg, and van der Merwe 2022). Regardless of any decision to limit requests, as good practice, a register of requests should be maintained and monitored, to evaluate volume/frequency of requests in the context of the extant learning environment.

Limitations

The students who collected the prospective audit data (authors WG, JT) were assumed to have received a similar number of requests as other students in their cohort. However, some requests may have been sent to specific students, meaning prospective audit data may be slightly over or underestimated. The retrospective audit is limited to University of Otago ethics applications only, and so may not have included requests from sources external to the University; however, anecdotally very few requests for medical student involvement are of this type. It was also not possible to retrospectively audit evaluation requests as there is no record or screening of evaluation requests, meaning it is unclear whether this year is wholly representative of other years. There is also the potential for non-response bias (Plante, Lesage, and Kay 2022), a consideration for such studies where response rates may be modest.

Conclusion

This work is the first to explore the impact, perceptions and attitudes around evaluation and research requests in the same cohort of health professional students. It supplements and extends understanding around evaluation and research requests to students, exploring situational perspectives based on extant audit data to illuminate differences in how students view evaluation and research requests. Data indicate that yearly volumes of 30 to 40 evaluation requests, and evaluation frequencies of more than five per month, are too high and are viewed by students as not reasonable. Student-nominated ‘reasonable’ request parameters include frequency and volume being no more than 19 evaluation or 10 research requests per year, with a frequency of no more than one a fortnight for either. Students indicated email requests can be problematic, and request delivery in academically quieter periods of the year. Interestingly, some students indicate their first reaction to receiving requests can be stressful, suggesting that institutions should perhaps be aware of how many and what type of requests students receive. Findings may provide a useful benchmark and inform institutional guidelines around request delivery, in concert with other data around programme delivery, though how transferable these findings are to other programs, institutions or student cohorts is not yet clear. Data have shown that evaluation and research requests are viewed differently, but not exactly why, and further studies are required to elucidate how these two types of request may affect student perceptions of requests.

Disclosure statement

The authors declare no potential conflicts of interest.