Developing good practice by understanding how UK medical schools address low level concerns: a survey study

ABSTRACT

In the literature, a distinction is made between low-level concerns and what is regarded as fitness to practise concerns. The General Medical Council expects all UK medical schools to have a transparent process in place about how concerns about its medical students are identified monitored and responded to. However, internationally, there is currently no well-established consensus on what is good practice in managing low-level concerns. Furthermore, currently, there is little information on how the UK medical schools vary in the processes they implement to monitor and respond to low-level concerns of their students. An online survey was developed and informed by the literature and sent to all UK medical schools to better understand their low-level concerns process. Of 39 medical schools invited, 25 participated. The data indicate variations between medical schools in the processes implemented. These variations can potentially influence the quality of the data; for example, whether there is a named person co-ordinating concerns between medical schools and placement providers. Furthermore, the data identify primary-care-based learning as offering missed opportunities where low-level concerns could be picked up. Key areas identified within the data for further work include how to quality assure that processes are equitable and how to bring more consistency to what sanctions are common and how these are decided up on.

KEYWORDS:

• Professionalism
• low-level concerns
• surveys

Introduction

Low-level concerns are valid predictors of later fitness to practise issues [1,2]. The General Medical Council (GMC) requires that the UK medical schools have a formal process in place to monitor and support remediation and development. The GMC does not stipulate particular requirements but asks the UK medical schools for transparency in how concerns will be identified and responded to.

Neither the international literature nor the GMC guidance give a single definition of what a low-level concern is. However, there are several examples provided of what this could constitute of. These include non-attendance, not handing in work on time or not following expected process during learning events [3]. Furthermore, broad guidance is given on the importance of having a process in place for identifying low-level concerns and supporting student development [3]. Even though not everyone who are identified as having low-level concerns during undergraduate training will go onto have difficulties with professionalism, evidence suggests that early identification, intervention and support is important in developing doctors who uphold professionalism and are safe practice [4–6].

Predictors of fitness to practise concerns and how to manage these have had some attention in the postgraduate setting [7–9]. However, despite being acknowledged as a key predictor of later professionalism and fitness to practise issues [1,2], within undergraduate settings, there is limited literature on predictors [10] and processes surrounding low-level concerns [6], with exceptions [11].

As professionalism educators, we gathered that there is differing reported practice across the UK medical schools. A literature review indicated currently there is no international consensus about good practice in recording and managing low-level concerns.

A shared national understanding can be a first step in achieving international consensus in good practice essential for supporting the development of medical professionalism. To respond to this gap, we implemented an online UK survey to identify what the common features and differences are across UK medical school with regard to processes to identify and manage low-level concerns for medical students.

Methods

The intended respondent groups are senior staff leading on professionalism and low-level concerns processes across UK medical schools. The aim was to survey the whole small population (all UK medical schools) to map current practice. The survey items were kept simple without developing complex scales. As no sampling was involved, it was unlikely the survey will be repeated within a reasonable time. The survey replaces a highly structured interview due to practical considerations (flexibility and resource constraints). For these reasons, the statistician advised there is little to be gained from examining the internal structure of the survey and that face validity and cognitive testing is sufficient [12].

The GMC requirements on monitoring low-level concerns [3] and a literature review (not UK specific) informed survey development. For example, the literature and GMC guidance identifies promptly responding to low-level concerns as important. Therefore, questions were added to discern what the process is for identifying concerns in a timely manner (Figure 1, questions 9 and 10). The literature identifies support processes as essential for the development of the future professional [13]; hence, for example question 11 on Figure 1.

Figure 1. The caption repeats all the information in the figure. Caption should only read- Figure 1- Survey Questions.

If fixed choice answers would restrict responses, these were left as open questions. It was checked that each survey item only addresses one question. Two academics involved in professionalism curriculum and the low-level concerns process cognitive tested the survey. They checked that the final questions were relevant and nothing significant was omitted; questions were easily understood as intended and had face validity [12]. Initially, the survey was sent out to a handful of professionalism leads who could comment on any issues around questions and structure but no such revisions were received. Then, it was sent out widely to all UK medical school professionalism leads.

Ethics Approval was granted by the lead author’s Medical School Research Ethics Committee (Reference: 024754). The final version was distributed via Survey Monkey to professionalism leads in UK medical schools through direct email during summer 2019. At this time, there were 39 medical schools in the UK. A single reminder was sent about four week after the first e-mail to those who had yet to respond.

Responses were anonymised by allocating a unique identifying number for each UK medical school at the stage of inviting to complete the survey; and removing any personally identifiable detail from responses.

The survey questions are presented in Figure 1.

Data analysis

For questions with categorical responses, analysis was by summation of individual response categories. Free text comments were grouped by particular themes. These themes inevitably corresponded to the key areas addressed by the survey. The groupings were verified by the co-authors. Frequency of free text responses were not taken into account; regardless of the number of times or number of respondents making a point, they are integrated into results presentation if relevant to the aims of the survey.

Results

Of 39 medical schools, 25 participated (64%). As the survey was emailed to the professionalism lead in each school respondents mostly identified themselves as having various academic or related senior roles within professionalism or fitness to practise teams. One respondent identified themselves as ‘the manager’. Of the non-responders, one school reported lack of time as the reason. The other schools simply did not respond to the initial email or reminder. There were no visible characteristics that separated the non-responders from the responders, such as geographical region or whether the school was an established or a newer medical school.

All schools that responded confirmed they have a low-level concerns process in place.

The results are structured through key aspects of the low-level concerns process and include [1]: Low-level concerns data management [2], Identifying concerns in a timely manner [3], Supporting students [4], common sanctions for low-level concerns [5], Perceptions of how low-level concerns link to fitness to practise concerns; and [6] communication of low-level concerns process to students.

Low-level concerns data management

Concerns are raised through a variety of mechanisms in place. In some schools, the concerns are raised through a form housed within the virtual learning environment; in others, through paper forms. In all schools, concerns could be raised by a variety of stakeholders including peers, staff or patients. Most schools have an online recording system, but this was not specified by all the responders. Five schools reported additional methods in place to gather information about concerns. This included yearly emails to year leads and administrators to prompt reporting of concerns, end of placement assessment of professionalism; and monitoring engagement during small group teaching.

Regarding staffing, only 15 schools had a dedicated person to process low-level concerns (15/25:60%). Where there is no dedicated lead the concerns are initially processed by year administrators or year leads.

Across the 25 schools, 13 audited their data, eight did not (one did not respond to this question.) A further three schools reported doing some form of review of the data as follows. One school said currently they don’t have ‘sufficient numbers to make this work; but rather depends on how you define audit. It gets reviewed formally on a regular basis – that’s audit too … ’; one was unsure whether or not they audit, one reported ‘it does get audited for formal committee appearances’; one school reported relevant data is submitted to the Board of Examiners and Board of Studies (anonymised numbers and reasons) and this data is considered in the context of Equality Diversity and Inclusion review of assessment processes.

Two schools reported that linking data to student records allows for a longitudinal tracking of concerns; thereby facilitating the monitoring of recurrence and patterns of behaviour. One school described a policy encompassing a cumulative points-based system. The points reset at the end of each academic year, but if they reach a certain threshold, students are referred to the fitness to practise process. Another school outlined fixed census points where feedback is provided and students at risk of failing are offered an intervention meeting. The outcome of such a meeting could include a fitness to practise hearing, failing the academic year and being referred for support to study programme.

One school specified professionalism concerns being integrated into the GP placement assessment forms. Furthermore, checking student wellbeing was reported as a first step when low-level concerns are raised (e.g. with attendance). Written reflections were outlined as a way of eliciting active engagement and insight from students about developmental needs.

Staff development workshops were identified as effective in raising awareness across faculty as well as improving consistency and handover between different year groups and teams.

Identifying concerns in a timely manner

The response to this question was mixed. Overall, responding schools were confident that a ‘reasonable’ timeliness and accuracy is met. However, there were a number of key factors that promotes or prevents this, and these are summarised below in Table 1.

Table 1. Facilitators and barriers for identifying concerns in a timely manner

Facilitators

Barriers

Have a ‘go-to’ person for concerns who is visible across the medical school and placement providers The process is not just about identifying concerns, but support mechanisms need to be in place to ensure students can develop and improve. A raising concerns process needs to have features that enable students to feel the staff are approachable so underlying health or welfare issues can be elicited. In some schools, PBL/small group tutors and GP tutors were identified more than once as better placed to pick up non health related concerns which are harder to identify; due to the how these workplaces are structured. On the other hand, hospitals where students are moving to different teams with short turnarounds were identified as places where concerns are hard to pick up on.

Clinical leads may not always understand the importance and reason for low threshold and timeliness of concerns The large number of students spread over several trusts means implementing a process that is reliable can be a challenge A process that is heavily reliant on the students coming forward is unreliable. Sensitivity of the process can be improved. There is a grey area where concerns could fall where it’s hard to differentiate between minor and major concerns; and separating academic/progress concerns from professionalism issues Level of Staffing Support from leadership so concerns are managed centrally and off site staff are given specific instructions about reporting, so their reporting is low threshold

Supporting students with concerns

Responses indicated that students are supported by a multi-layered staff including educational supervisors, personal tutors and pastoral team. Schools reported accessing university counselling services, general practitioners and occupational health. One school described a student support group, but it was not known how the sustainability of this group is ensured and what resources or training this group receives. Another school identified mentoring by foundation doctors on professionalism as a key support measure.

One school reported offering one-to-one support to students undergoing professionalism investigation by explaining the process, managing expectation and helping the student to develop professionalism specific to the situation. Another school recognised the importance of identifying any patterns in student behaviour and understanding the student’s motivation to make the necessary changes.

Challenges included not all students engaging with the support offered and finding mentors who offer ongoing help to students who need them. It was identified that students should not feel any support on offer has a punitive element. It was recognised that improving the communication between the different teams (e.g. between professionalism and student support) would lead to improved experience for students. Other key factors that can improve support and effectiveness of interventions included better triangulation of concerns through data from various sources, and clear definitions of thresholds for reporting concerns. With regard to this latter point, one respondent highlighted the fairness issue of students feeling that they have been ‘caught’; whereas their peers hadn’t been. Attendance and sickness recurred as an issue that is harder to manage through a low-level concerns process. Responses suggested that the focus of low-level concerns is to monitor emerging patterns of behaviour.

Common sanctions for low-level concerns

A common outcome of the low-level concerns process is for students to complete reflective writing. Academic interviews leading to a permanent record on the student’s progress file or a written warning were also common outcomes. Where concerns related to attendance issues, remediation may take the form of additional attendance. One school reported that a sanction through the ‘doctor as a professional’ assessment domain can lead to the student not progressing; if a borderline grade is received then the student needs to achieve an acceptable grade in the next academic year.

One respondent commented that ‘Sanctions can be problematic and biased’ and that it is ‘Important to recognise that although low level concerns are important in predicting future issues a vast majority don’t’ (lead to this). In rare cases, sanctions can include being barred from sitting an exam or being subject to an additional period of scrutiny to ensure they are meeting particular targets such as level of attendance.

Low-level concern process and fitness to practise processes

Responders were asked whether the low-level concerns process and the fitness to practise processes were separately administered. This was the case in 20 schools (20/25; 80%) with five schools having an integrated process. Related free text responses included – Good grief – I’m slightly horrified that these are regarded as separable. Another responded – Yes – in that serious things like criminal conviction go straight to FtP [fitness to practise]– but LLCs [low level concerns] can refer on to FtP too. But they are separate.

More than one school clarified that low-level concerns and fitness to practise are part of the same escalating pathway but are dealt with by different teams.

Communicating low-level concerns process to students

Schools utilised a number of mechanisms for communication. These included lectures, workshops, handbooks and the virtual learning environment. One school reported inviting students in senior years to offer lectures and workshops about fitness to practise and professionalism. It was recognised by respondents that communication about the process needs to occur at least yearly.

One respondent reported that their school did not frame the process as ‘low level concerns’ monitoring. Nevertheless, they assess students on their professionalism through weekly small group learning sessions. Another school highlighted the presence of specific professionalism groups for students each fortnight. One responded that their school did not disseminate formal information regarding low-level concerns, but that student study guides contained material related to expectations. More than one school reported utilising relevant e-learning hosted by the GMC. One school reported that students were informed of the existence of a central progress record.

Discussion

This paper gives a picture of the key feature of the processes followed by UK medical schools and thereby starts to consider how to effectively manage low level concerns about medical students.

All responders recognise the importance of addressing low-level concerns as advocated within the medical professionalism literature [13,14] and have a process in place. However, there are variations in the process between different medical schools. This may be a sign of local resourcing needs. Some variations such as whether or not there is a named person to manage the low-level concerns process or whether the system is paper or online-based may influence the quality and reliability of the low-level concerns data held; we are unable to explore this through the current data set. Nevertheless, we argue that good practice in record keeping in healthcare practice [15,16] applies here also. For example, a purposeful online system will enable intelligent long-term tracking and building a cross-sectional picture; a feature discerned to be important for picking up concerns early and supporting students effectively [2,17]. The data also help identify key questions where stakeholders need to aspire for consensus as these can influence the quality of the low-level concerns process. Examples include how sanctions are applied and how students are supported.

With literature indicating that non-caucasian medical students are more likely to experience fitness to practice processes and concerns raised about them [18] how do we determine that the processes in place do not deliberately disadvantage particular group of students? The data indicate that not all medical schools audit the data using a standard process and may understand the purpose of audit differently. Furthermore, currently there is no precedence about effective ways in which to do this.

The data indicate currently UK Medical schools do not always utilise best practice in the process for detecting low-level concerns. For example, even though the data indicate that small group teaching opportunities are recognised as effective ways to detect low-level concerns and especially concerns about health, currently not all schools may be utilising this opportunity. However, this maybe because of worries around balancing the importance of eliciting low-level concerns against maintaining an open and safe space that facilitates transformative learning. Therefore, primary-care education and other small group learning leads should consider how these key findings can be translated to an effective low-level concerns raising policy within such learning environments so valuable information that may help to support the student development is not missed. This must be done without compromising the perceived safety of the learning environment.

One of the many challenges of improving consistency in the process may be staff development efforts that contributes to developing a shared mental model of why low level concerns are important and how to identify and respond to these. Currently investment into staff development is variable.

The survey did not explore the nature of reported concerns. This would be useful to ensure consistency of reporting across medical schools: what one school considers a ‘low level concern’ may not even be reportable in another. Collation of such data would be useful to inform curriculum development in teaching and to support student professionalism development; particularly of students who find their journeys to becoming professionals a challenge.

As this is an online survey, the conversations that are possible through qualitative interviews were missed; particularly, there were no opportunities to clarify responses offered. For example, gathering more information on variations on how data are audited, by asking follow-up questions was not possible. Furthermore, it would have been useful to elicit reflections from respondents about the value of the interventions commonly employed and how they are decided upon both in terms of choice and implementation. For example, reflective writing was commonly cited as an outcome of a low-level concerns investigation. Reflection clearly has a role in medical education, and the development of the reflective practitioner is championed by many regulatory bodies [19,20]. Reflective practice is, however, a broad church, and one must be wary of what has been described as the ‘reflective zombie’ [21], who may simply go through the motions of reflection without discernible output. Greater interrogation of the use, and indeed effectiveness of reflective writing in the low-level concerns process would be greatly beneficial.

In the future, the survey conclusions can be used as a basis to develop complex mixed method work that will allow in-depth exploration of focused issues. These could include establishing which interventions are effective to support students with low-level concerns and synthesising best practice in implementation with the view of achieving a consensus on these amongst medical educationalists. Such developmental work also needs to consider how best to audit data for quality and equity purposes. These processes can be used to ensure that low-level concerns monitoring is fair and comparable across institutions.

Acknowledgments

To all respondents who responded to the survey.

Disclosure statement

No potential conflict of interest was reported by the author(s).