Tell me if there is a problem: safety in early general practice training

ABSTRACT

In contrast to other comparable countries, trainees commencing general practice in Australia can see patients without being required to contact their supervisor. To understand how patient safety in early training is managed a qualitative study design using semi-structured interviews was used. A lead medical educator from each of the nine Australian Regional Training Organisations (RTOs) was interviewed. Transcriptions of interviews were analysed to identify themes.

RTOs do not mandate a period of direct observation of trainees and the use of safety checklists for supervision is variable and not monitored. The oversight of training practices by RTOs mirrors that of trainees by supervisors. The onus falls on those being supervised to identify the need for assistance. Despite this, lead medical educators still consider the commencement of general practice training to be safe.

Other factors found potentially to impact on safety include the variability of training practices and supervision; the complex RTO-practice relationship; quota-driven selection of doctors into general practice; and the negative impact on education of the funding model.

Patient safety may be improved by a period of direct observation of potential trainees prior to the commencement of general practice training and the use of checklists to encourage supervision of high risk activities.

KEYWORDS:

• General practice
• graduate medical education
• clinical supervision
• patient safety
• health care economics and organisations

Introduction

Imagine a trainee surgeon at the commencement of training being told by their supervisor, ‘go into theatre, start operating on whatever comes in, call me if you think you need some help’. This is obviously a ludicrous scenario, and yet it is akin to what can occur to Australian General Practice (GP) registrars (trainees).

In the Australian GP Training (AGPT) Program, after at least two postgraduate years of hospital training, registrars are placed in general practices. Training is divided geographically into regions that were reconfigured in 2016. Each Regional Training Organisation (RTO) is accredited to deliver training by the two GP training colleges: Royal Australian College of General Practitioners (RACGP) and Australian College of Rural and Remote Medicine. Fifty percent of training must occur in rural, regional or remote locations [1].

The AGPT Program is funded by the Australian Department of Health, but registrars also bill patients for the services they provide. This billing income is retained by the practice. A registrar is paid the greater of either a percentage of patient billings or a fixed salary [2].

An international medical graduate commencing GP in Australia is usually required to commence under the Medical Board of Australia’s Level 1 supervision standard. This requires a doctor to confirm their assessment and management of each patient with their supervisor prior to the patient leaving the facility [3]. In contrast, an Australian-trained  GP registrar can commence practice being required only to call for assistance when they consider it necessary.

The (RACGP) Standards for GP Training are outcome standards [4]. The standards require registrars to be supervised for procedures, or management of high risk situations, that they are not yet competent to undertake but do not specify the level of supervision required to achieve this outcome. This is an unusual position among the comparable international GP training programs of New Zealand, Ireland, Canada, Netherlands and the United Kingdom [5].

There is a tension between the lack of Level 1 supervision and the requirement for registrars to be supervised when managing high risk situations. This may put the onus on registrars to determine when the situation is high risk. In a recent Australian study that reviewed records of registrar consultations, patient safety concerns were uncovered by 30% of supervisors, with 16% needing to consequently contact the patient [6]. These findings raise concerns about patient safety in AGPT.

Our research seeks to understand, by investigating the current approaches of RTOs, how high risk situations in early GP training are managed. We also aimed to ascertain the views of lead medical educators (LMEs) in RTOs on the safety of AGPT.

Methods

The research team consisted of a GP supervisor and medical educator (GI), a GP registrar and junior medical educator (KP), a PhD research academic with experience in qualitative research (RK), and a research assistant and RTO administrator (NW).

A qualitative study design was selected as the best way to explore the attitudes, beliefs and experiences of those involved in implementing AGPT [7]. Individual interviews enabled a deep exploration of the views of each participant without contamination from the group [8]. An inductive thematic approach where ‘themes derive from the content of the data themselves was chosen [9].

We purposively sampled the lead medical educator of each of the nine RTOs across Australia. Where the lead medical educator was not available, we approached the deputy-lead medical educator. Respondents were invited by email to participate in a semi-structured interview on ‘supervision of high risk early consultations in GP training’ via videoconference. Participants provided consent for the interview to be recorded and transcribed.

Seven lead medical educators and two deputy-lead medical educators were interviewed in July and August 2018. All were interviewed by one researcher (GI) who, as an experienced medical educator, was known to the participants as a peer able to understand the complexity of their role. This was considered likely to encourage greater openness among respondents. Each interview lasted between 36 and 56 minutes. Interview questions focused on the roles of RTOs, training practices, and GP supervisors in ensuring registrar patient safety, and how this is monitored. The interview questions also explored perceived barriers to patient safety and ways that this could be improved.

Although an interview schedule (Box 1) developed by the researchers was used as a basis for questions, participants were encouraged to talk widely across the topic. Interviewees were invited to share any relevant resources including lists of high risk encounters in early GP training.

Box 1. Interview schedule.

Each research team member listened to the interviews, read the transcripts and independently identified content themes on hard-copy transcripts prior to meeting to compare and discuss. In general, there was concordance about the themes. Debate was encouraged, and transcripts were referenced to consider disagreements. The research team was mindful of the risk of the sole interviewer influencing the views of the others. The discussion about themes was limited until all transcripts were available, ensuring all researchers identified themes independently. The interviewer (GI) was last to share his discovered themes. Bias was also reduced by having a researcher (RK) not involved in AGPT lead the discussion.

The analysed data were presented back to the lead medical educator group at an annual meeting in November 2018 to obtain respondent validation and increase interpretative rigour. There was agreement with the themes identified.

Results

Seven themes were identified from the interviews. These are listed in Box 2 and discussed below.

Box 2. Themes.

Clinical supervision responsibility is delegated to training practice

While all RTOs consider they have a role in ensuring GP registrars only manage patients they are competent to manage, this is seen ultimately as the responsibility of the supervisors and the training practice. Each RTO supports practices to provide safe care through supervisor training and by conducting initial and ongoing assessments of the registrar, but safety can only truly be managed in the training practice.

The supervision must match the competence of the registrar – the RTO is not in a position to make that call, but the supervisor who can see the registrar is. LME1

Early assessment of registrars varies widely in type and extent between RTOs. There is debate about its value. Assessments used extend from relying mostly on hospital supervisor reports prior to GP placement through to detailed written examinations and OSCEs (objective structured clinical examinations) conducted by the RTO. Early assessment is considered more useful in identifying registrar learning needs than in determining registrar safety in practice. Even RTOs that conduct extensive assessments concede they could not deem a registrar safe for any specific clinical scenario. This must be determined by observation in practice.

We don’t have a test that says you’re safe or you’re not safe. LME3

All RTOs ensure training practices are accredited and expect supervisors to undertake training before accepting their first registrar. The supervisor induction training includes instruction on how registrars should be safely supervised.

All RTOs encourage supervisors to directly observe a registrar consulting at the start of the first GP placement before allowing them to commence seeing patients, only asking for help when the registrar considers it necessary. Generally, the expectation is that a registrar will progress beyond Level 1 supervision by the second week of training. Some RTOs provide a list of high risk activities to prompt discussion between the registrar and supervisor about supervisory practice. Each practice is free to determine the duration of Level 1 supervision or how to use the lists of high risk activities. Neither is mandated or routinely monitored.

We have had some registrars who have swum outside of the flags and it’s the supervisors who have brought that to our attention. Whether that’s happened all the time? That’s unknowable to me. LME5

There is no specific monitoring by RTOs of whether practices ensure that registrars only manage patients they are competent to manage. Major adverse events involving a registrar are expected to be reported by the practice to the RTO. There is uncertainty about how accurately this reflects the true incidence of adverse events.

Variability of training practices and supervision

Many RTOs are concerned about the variable quality of registrar supervision between training practices. Where it is thought a practice’s involvement in GP training is mostly motivated by workforce needs, it is believed there is a greater chance of poor supervision and teaching.

You do your best to accredit them and provide the professional development but yeah, we’re tending to put a few fires out sometimes. LME3

RTOs rely on registrar feedback to evaluate the teaching and supervision in practices. However, this feedback might be compromised by registrar reticence to provide criticism that might influence their future career prospects.

One of our greater challenges about managing, what I’m going to call less than good supervision, is getting it on the record or documented. LME6

Some RTOs are less concerned about variability of supervision. They attribute this to the close relationships they have with their training practices. Close relationships are considered more likely in RTOs that visit their practices frequently, are smaller, or largely unchanged by the government alterations to training regions in 2016.

The interviewees identified the need to reduce supervision variability through investment in quality improvement. Novel ideas included the recognition and reward of higher-quality practices and the development of expert supervisor medical education teams to visit training practices and facilitate quality improvement in the practice.

The complex RTO-practice relationship

Each RTO is reliant on independent training practices to deliver the training program to their enrolled registrars. The interviewed medical educators are keen to protect practices and supervisors from unnecessarily burdensome change. Any change to the training program must be meaningful and simple to implement.

If you look at it from a business perspective our clients are our practices; they’re our long-term partners in this industry. So, it’s critical that we support the registrars, but I would say it’s vital that we support the practices. LME7

Many RTOs are concerned about losing training practices, particularly practices in rural areas, as this might impact the RTOs’ ability to meet rural training targets. Some acknowledge this might influence them to continue working with practices providing poorer teaching and supervision, rather than removing them.

Part of the problem we have is particularly in the rural areas, where the quotas are high, and the capacity is low. LME8

Training funding model negatively impacts on education and supervision

Several of the lead medical educators interviewed believe that the current AGPT funding model impacts on the level and quality of supervision. Practice subsidies do not cover the registrar’s wage, so registrars need to see and bill patients to fund their training. Consequently, service delivery can trump education in structuring the registrar’s clinical and educational experience.

If I had endless buckets of money and I could change the system, the first thing I would do is get rid of Medicare and tagging earnings to training. So that seeing patients is all about a learning experience and not about making money. That would be my number one. LME4

In the UK whether you spent an hour with a patient or 20 minutes with a patient, it made no difference. And I think that’s the huge difference here in Australia. LME5

Safety risk of quota-driven selection

Participants consider selection into GP training to be an important determinant of safety in GP training. A drop in the number or standard of training program applicants would create difficult decisions for lead medical educators, who are expected to fill the quota of GP training places in their region. Not all medical educators consider they have the option to only accept applicants they can safely train.

Junior doctors are coming out of our system without an awful lot of clinical exposure, and we know that. It’s getting diluted as a student, and as a junior doctor. LME9

We have a KPI to place those registrars. And if we don’t place those, that negatively impacts our reporting. LME4

High risk management lacks consistency

Specific lists of high risk clinical activities for registrars are provided to supervisors by the majority of, but not all, RTOs. Lists differ between RTOs. The intent is to prompt a discussion between registrar and supervisor about closer supervision of the listed clinical activities. Those RTOs providing lists explain their use during introductory supervisor workshops and occasionally in ongoing supervisor education. No RTO routinely monitors whether the lists are being used.

We don’t mandate that it is completed and uploaded. We provide it as a resource. LME4

Current system is not broken

None of the lead medical educators consider there are major shortfalls in the current GP training arrangements in ensuring patient safety. When offered the potential to hypothetically change the training program, they were mostly inclined to improve the current arrangements, rather than overhaul them.

You know 95% of our registrars realistically are sailing through without any issues over competency, or problems, or medical board reports, or anything. LME9

I would be confident the vast majority of registrars are practising in a safe way. I wouldn’t be 100% confident that that was always the case, and it would be more of a sub-set of practices where the supervisors weren’t taking their responsibility as seriously as I would like them. LME8

There is caution about a proposed compulsory period of Level 1 supervision early in GP training. Some RTOs consider they might not have the capacity to deliver such a term. All agreed that it would not be possible without different funding. There was further concern that it might delay registrar progression to experiencing responsibility for making clinical decisions. This would make the transition after training to independent practice more challenging. They considered any period of closer supervision should either be brief (a few months at most) or be able to be shortened in response to registrar progression, lest it frustrate the advanced registrar.

There is mostly acceptance of the use of high risk lists to improve safety. Those reluctant to adopt the use of lists are concerned that an overly long list can be burdensome for supervisors and registrars. Clinical activities not on the list might be expected to be independently managed by the registrar, paradoxically leading to less safe practice.

One of the real challenges when you codify something is as soon as you start to say, “This is,” then by definition you start to say, “Everything else isn’t.” LME7

I think that would, for want of a better word, ‘upset’ our supervisors out there who feel they are experienced supervisors and they have their own methods of making sure that a registrar is safe. LME9

Discussion

We interviewed lead medical educators from every RTO in Australia, capturing the clinical safety strategies used in GP training across the country. Fundamentally, those interviewed consider current GP training to be safe for the registrar’s patients.

It is difficult to say whether training is safe in Australia. GP registrar surveys report high levels of appropriate supervision [10]; levels higher than those in the hospital-based specialties [11]. Studies examining safety in GP training are few [12,13], and measurement of safety in GP is less advanced than in other areas of healthcare. This makes the design of such studies difficult [14].

However, there appears to be a disconnect between interviewees’ certainty of safe registrar practice and the finding that RTOs delegate the safety responsibility to supervisors. RTOs do not directly monitor whether supervisors ensure registrars only see patients they are competent to manage. In this regard, the supervision of practices in the AGPT Program appears to mirror the supervision of registrars. The onus is on those being supervised identifying and notifying problems.

The variable amount and quality of supervision reported by some of the lead medical educators has been noted in other papers [15–18]. Suggested solutions to this problem have included closer engagement by RTOs with practice owners [19], supporting the supervisor-registrar educational alliance [20,21], and development of a more detailed and consistent supervisor education program [22].

The need to retain poorly performing practices to meet program requirements identified by some educators relates to a requirement that 50% of AGPT must occur in rural, regional or remote locations [1]. We believe that consideration should be given to relaxing workforce requirements in those RTOs that identify difficulty in meeting the rural training target due to a lack of suitable training practices.

There is evidence to support the view that current Australian GP funding arrangements may impact the quality of GP training, prioritising service delivery over education. For both the registrar and the practice there is a financial incentive for a registrar to see more patients. Once a billing threshold is reached, the registrar is paid a percentage of patient billings. This is occurring in nearly three quarters of training practices [2]. There is at best a small financial gain for training practices in training registrars [23]. This might be jeopardised by providing more direct supervision.

The number of places in AGPT has remained stable at 1500 since the 2015 intake [1,24]. Despite the concerns of our interviewees, the number of eligible applicants has remained above 2000 for each year of intake (personal communication, Dr Mark Rowe, General Manager Education Services at RACGP). Nevertheless, safety would be supported by having a flexible approach to training places that did not require RTOs to reach quotas if this meant accepting doctors likely to need closer supervision.

Our interviewed lead medical educators doubted that the current RACGP-required early registrar assessments can determine safety to commence GP. They considered that direct observation of the registrar was the best and ultimate determination of whether the registrar was safe for practice. Recently concern has been raised that Australian hospital experience may not prepare doctors for GP training as well as in the past [25]. If true, the argument for a period of mandatory Level 1 supervision at, or prior to, the commencement of GP training is stronger.

The Prevocational GP Placements Program (PGPPP) that ceased in 2014 did enable junior doctors to experience general practice under direct observation. PGPPP was designed to encourage rural GP recruitment. Placements were available to all junior doctors prior to their nomination of a vocational training pathway. It was ceased because of concerns regarding its cost [26]. A redesign of the program with the principal aim of assuring safety in early GP training rather than rural recruitment is worth considering. This could be used to replace the current entry into GP training assessment. A program only available to doctors considering GP training would be less costly.

There is a lack of consistency across the RTOs in the use of lists for supervising high risk activities in early GP training, and no apparent evidence base for the lists used. The authors are currently developing an approach to supervision of high risk activities in an Australian context.

Our study is limited to the views of a single, senior educator in each RTO. It may be that some senior educators were unaware of aspects of the education program and supervision in their region. The concern that our findings may not reflect the views of those at the coalface of GP training is being addressed by further research involving focus groups of supervisors and registrars about the safety of GP training in Australia.

Conclusion

The current model of GP training stands out from other Australian specialties and international GP training programs in not having a mandatory period of direct observation or Level 1 supervision at the commencement of training [5]. Despite this, lead medical educators consider the safety of the current Australian model to be adequate. Improvements to safety might be achieved through reconsideration of the rural training requirements, reintroduction of a targeted short GP placement with Level 1 supervision for hospital residents who are intending to apply for GP training, and refinement through research of a more consistent approach to the management of high risk activities. More radical reform would require changing the GP training model to enable a greater focus on education over service delivery.

Ethical statement

ETHICS APPROVAL for the study was obtained from the Monash University Human Research Ethics Committee [project 12673].

Acknowledgments

We thank the medical educators from each of the nine Australian Regional Training Organisations who gave their time and expertise in interviews, and in giving feedback to validate the study findings.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by the Royal Australian College of General Practitioners with funding from the Australian Government under the Australian General Practice Training program.