187
Views
12
CrossRef citations to date
0
Altmetric
Original Research

Practically prepared? Pre-intern student views following an education package

&
Pages 111-120 | Published online: 27 Jan 2017

Abstract

Background

Graduating medical students enter their internship with varied levels of practical experience in procedural skills. To address this problem, many medical schools have introduced intensive skill training courses immediately prior to graduation. This study examines the impact of a pre-intern (PrInt) education package, consisting of a short intensive course, followed by a one-month clinical attachment.

Methods

In September 2014, all PrInt students (n = 53) at the Central Clinical School (Sydney, NSW, Australia) attended three days of intensive training. This included a didactic introduction, case-based scenarios, and interactive workshops. This was followed by four weeks of targeted, experiential learning during a clinical attachment (PrInt term). Immediately prior to training and following PrInt, all students were invited to complete a six-domain questionnaire containing 40 subscale closed questions to assess their knowledge, experience, and confidence in key practical skills essential for a successful internship.

Results

A total of 41/53 (77%) students completed an identical questionnaire prior to PrInt, and 37/53 (70%) immediately following PrInt. Respondents reported statistically significant increases in their experience, ability, knowledge, and confidence in a number of domains. The key changes were the following: knowledge of pharmacy skills (mean improvement = 26.48, confidence interval 95% [CI 95%] = 17.29–35.66, p ≤ 0.0001) and management of procedural skills (mean = 24.46, CI 95% = 16.58–32.34, p ≤ 0.0001). Despite the positive overall increase in most domains, some subscale results remained low following the educational package; only 44% students had inserted a nasogastric tube; only 44% reported confidence in commencing patients on warfarin; and only 42% in managing a hospital emergency. Surprisingly, there was a slight decline both in confidence in communicating with members of the hospital team (10%) and in awareness of the causes of hypoglycemia (7%).

Conclusion

Final year students perceived substantial benefit from an educational package specifically aimed at improving their practical skills immediately prior to internship.

Background

Medical graduates’ readiness for internship entails an awareness of their abilities (and limitations) in basic practical skills in the clinical environment, as well as their confidence to safely commence internship.Citation1 It is not surprising that this “readiness” is strongly linked to their previous “hands on experience” as students,Citation1,Citation2 which is unfortunately opportunistic and, therefore, highly varied.Citation3 Consequently, new graduates commence their internship with vastly differing levels of both knowledge and experience in practical skills, required to safely undertake their new role in patient care.Citation3 Intern knowledge gaps have been particularly identified in the management of the acutely ill patients, prescribing,Citation4Citation6 and non-technical skills, including communication skills,Citation7 team work, and task management.

To address this variability in readiness for internship, preparatory courses such as “Coda” accelerated learningCitation8 “Boot Camp”,Citation9 Student assistantships,Citation10 and “Shadowing” are offered in many medical schools immediately prior to graduation. To ease the transition into internship, Sydney Medical School implemented a pre-internship term in 2000. This comprised activities such as allowing students to “shadow” an intern, perform procedural skills under supervision, gain experiential knowledge in non-technical skills such as team work, and to gain an awareness of hospital protocols crucial to patient safety.

While the PrInt term was seen as useful for internship, we felt there was room for further improvement in core procedural skills and designed a new educational package to cover obvious gaps. The aim of our study was to assess the impact of this new educational package on students’ self-reported ability, confidence, knowledge, and awareness of the key practical skills required for internship.

Methods

The Sydney Medical Program is a four-year graduate entry course with problem-based learning as the key teaching strategy. This study took place at the CCS Royal Prince Alfred Hospital, a teaching hospital for The University of Sydney, Sydney Medical School. In the last four weeks of the medical program, students enter their pre-internship term.

In September 2014, all PrInt students at CCS (n = 53) who had completed their final exams attended a three-day PrInt induction course. The course had been designed and piloted previously as a five-day course in 2011.Citation11 The course described here is a three-day modification of the pilot study.

Education package

Didactic teaching

This three-day intensive teaching program included key situations that new interns were very likely to encounter,Citation12 including core procedural skills, such as “managing” procedural skills, identifying N/G tube placement on X-ray, recognition of the acutely unwell patient, complex communication skills – presented as ward-based scenarios, interpreting common X-rays, suturing, and effective handover utilizing ISBAR (a mnemonic created to improve safety in the transfer of critical information).Citation13

Small group teaching sessions

These were designed to provide students with individualized practice, with real-time feedback from experienced clinicians. Students were also given the opportunity to clarify concerns in performing practical skills and were provided with strategies for identifying and dealing with common clinical challenges. This style of learning encouraged reflection and integration of skills and supported their transition to real life as a junior doctor.Citation14

Case-based teaching

We used case-based scenarios, known to be associated with errors/adverse events, such as test ordering, prescribing medications, recognition of the deteriorating patient, and specimen collection/labeling. Specific education was delivered by pharmacists in “pharmacy/prescribing skills”, and by emergency physicians in “emergency skills”. In addition, while on their subsequent clinical attachments, they had weekly bedside tutorials conducted by hospital clinicians.

Clinical attachment

This component of the package involved four weeks when the students were assigned to a medical or surgical team, where they “shadowed” the intern and became active members of the team. PrInt students are required to perform various mandatory practical skills under supervision, and their overall performance as part of the team must be deemed “satisfactory” by their PrInt supervisor before they can graduate.Citation15

Instrument

To develop our instrument, focus groups were conducted at CCS with hospital clinicians. Following data analysis, we generated a novel (unvalidated) six-domain questionnaire. We were mindful of including key procedural skills that interns would be expected to perform. The scores ranged from 1 to 100, and we arbitrarily considered 5 points (i.e., 5%) to be the minimal relevant change. Each of the six domains consisted of additional subscales, giving a total of 40 questions answered by either “Yes” or “No”.

The domains are as follows:

  1. Performance of core procedural skills

  2. Knowledge and confidence in managing procedural skills

  3. Clinical recognition scores (recognition of the acutely ill patient)

  4. Pharmacy skills/prescribing skills

  5. Communication skills

  6. Organizational skills

Students anonymously completed two identical questionnaires – the first, immediately prior to the three-day induction course (in late September 2014), and the second, immediately following four weeks of PrInt (in October 2014). Questionnaires were distributed and collected by a third party not involved with the study.

Statistical analysis

The two questionnaires were compared using multiple regression analysis. We were not able to compare individual student results due to the anonymous nature of the research.

Ethical considerations and participant consent details

Ethics approval was granted by The University of Sydney Ethics committee. Permission to undertake this research has been granted by Human Ethics approval number 2013/211 at The University of Sydney, NSW, Australia. Participants received an accompanying statement informing them that lodgment of forms was considered to be their consent for future use of data.

Results

A total of 41/53 (77%) PrInt students, attached to CCS, completed identical questionnaires prior to their PrInt induction session and 37/53 (70%) at the completion of four weeks’ PrInt. Of the 41 preinduction respondents, 24/41 (59%) were male, 15/41 (37%) were female, and 2/41 (5%) did not report their gender. Respondents’ ages ranged from 23 to 34 years.

The mean scores and mean differences in each domain of the six closed questions concerning both experience and knowledge in key skills are presented in . Respondents reported significant improvements in knowledge of pharmacy skills and managing procedural skills. However, because of preexisting high baseline scores, there was little improvement with respect to other domains, including communication, organizational, and procedural skills.

Table 1 Report of the practical skills ability of PrInt students in order of magnitude of improvement following pre-internship

Subscale results

The subscale results in the order of magnitude of improvement are demonstrated in .

Figure 1 Subscale results of PrInt students’ self-reported knowledge and awareness in safe prescribing (pharmacy skills) in 2014 before the educational package and immediately after pre-internship. Note that students’ confidence in commencing patients on warfarin remains low.

Abbreviation: PrInt, pre-intern.
Figure 1 Subscale results of PrInt students’ self-reported knowledge and awareness in safe prescribing (pharmacy skills) in 2014 before the educational package and immediately after pre-internship. Note that students’ confidence in commencing patients on warfarin remains low.

Figure 2 Subscale results of PrInt students’ self-reported confidence and awareness in their experience of management of procedural skills in 2014 before their educational package and following PrInt.

Abbreviations: N/G, nasogastric; PrInt, pre-intern.
Figure 2 Subscale results of PrInt students’ self-reported confidence and awareness in their experience of management of procedural skills in 2014 before their educational package and following PrInt.

Figure 3 Subscale results demonstrating PrInt students’ self-reported knowledge in clinical recognition skills prior to and following PrInt in 2014. There was reported decline in knowledge in the precipitating causes of hypoglycemia; recognition of diabetic ketoacidosis also declined by 5%, whereas knowledge of the management of risk factors leading to hyponatremia improved.

Abbreviation: PrInt, pre-intern.
Figure 3 Subscale results demonstrating PrInt students’ self-reported knowledge in clinical recognition skills prior to and following PrInt in 2014. There was reported decline in knowledge in the precipitating causes of hypoglycemia; recognition of diabetic ketoacidosis also declined by 5%, whereas knowledge of the management of risk factors leading to hyponatremia improved.

Figure 4 Subscale results demonstrate the decline in respondents’ reported confidence in working with the hospital team on ward rounds by 25% (from 98% to 73%) and a small decline by 5% (from 88% to 83%) in prioritizing patients to minimize the possibility of adverse events.

Abbreviation: PrInt, pre-intern.
Figure 4 Subscale results demonstrate the decline in respondents’ reported confidence in working with the hospital team on ward rounds by 25% (from 98% to 73%) and a small decline by 5% (from 88% to 83%) in prioritizing patients to minimize the possibility of adverse events.

Figure 5 Subscale results demonstrating PrInt students’ self-reported awareness and confidence in organizational preparedness before the education intervention and following PrInt in 2014. There was a decreased awareness of 11% (from 99% to 88%) reported for knowledge of writing in the patients’ notes, and students’ ability to perform an effective clinical handover increased by 12% (from 61% to 73%) as did performing an effective verbal consult to a senior colleague by 10% (from 68% to 78%).

Abbreviation: PrInt, pre-intern.
Figure 5 Subscale results demonstrating PrInt students’ self-reported awareness and confidence in organizational preparedness before the education intervention and following PrInt in 2014. There was a decreased awareness of 11% (from 99% to 88%) reported for knowledge of writing in the patients’ notes, and students’ ability to perform an effective clinical handover increased by 12% (from 61% to 73%) as did performing an effective verbal consult to a senior colleague by 10% (from 68% to 78%).

Figure 6 Subscale results demonstrating PrInt students’ self-reported performance in a range of procedural skills in 2014 before the educational package and following PrInt. Low pre- and post-scores were recorded for their practical experience of the insertion of a nasogastric tube.

Abbreviations: ECG, electrocardiogram; PrInt, pre-intern; BSL, blood sugar level.
Figure 6 Subscale results demonstrating PrInt students’ self-reported performance in a range of procedural skills in 2014 before the educational package and following PrInt. Low pre- and post-scores were recorded for their practical experience of the insertion of a nasogastric tube.

Pharmacy skills

The results of students’ perceived knowledge and awareness and knowledge of pharmacy skills are shown in . There were statistically significant improvements in five areas: safety awareness in pharmacy skills in writing the prescriptions increased by 58% (from 34% to 92%), knowledge in the use of the medication management plan increased by 51% (from 10% to 61%), confidence in prescribing low molecular weight and standard weight heparin increased by 42% (from 22% to 64%), confidence in commencing patients on warfarin therapy increased by 22% (from 22% to 44%), and recognition and treatment of opiate-induced respiratory depression increased by 16% (from 78% to 94%)

Managing procedural skills

Respondents’ perceptions of their ability to manage situations related to procedural skills are shown in . The most significant improvements in students’ confidence were as follows: managing common problems associated with indwelling catheters increased by 39% (from 39% to 78%), intravenous cannulas increased by 37% (from 49% to 86%), identification of correct placement of peripherally inserted central catheter-lines on X-ray increased by 32% (from 36% to 78%), and confidence in managing clinical emergencies on the ward increased by 32% (from 10% to 42%).

Clinical recognition skills (recognition of the acutely unwell patient)

Ten questions were asked to participants regarding their ability in the recognition, awareness, and management of the acutely ill patient. The results are shown in . There were significant improvements in the following: students’ perceived confidence in recognizing respiratory acidosis increased by 17% (from 51% to 68%), confidence in managing the fluid status of a patient with hyponatremia increased by 12% (from 49% to 61%), confidence in the assessment and management of patients with oliguria increased by 13% (from 63% to 76%). There was a small decline in knowledge regarding the precipitating causes of hypoglycemia, which decreased by 7% (from 95% to 88%); awareness of the risks of ceasing oxygen for a patient with hypoxemia and hypercapnea also declined by 7% from (from 90 to 83%).

Communication skills

As seen in , there was a decline in respondents’ reported confidence in working with the hospital team on ward rounds by 25% (from 98% to 73%) as well as a small decline by 5% (from 88% to 83%) in prioritizing patients to minimize the possibility of adverse events both of which had relatively high preexisting baseline scores.

Organizational preparedness

Three closed questions asked respondents to self-report their organizational preparedness. demonstrates a decreased awareness of 11% in respondents’ reported knowledge of writing in the patients’ notes, and students’ ability to perform an effective clinical handover increased by 12% (from 61% to 73%) and their ability to perform an effective verbal consultation with a senior colleague increased by 10% (from 68% to 78%).

Procedural skills

The results of seven closed questions related to students’ reported performance of procedural skills are shown in . Respondents indicated very high baseline levels of performance in cannulation (98%) and venipuncture (98%), and there was no room for further improvement. However, there were significant increases to IDC insertion by 15% (from 78% to 83%), electrocardiogram performance by 13% (from 59% to 72%), and N/G tube insertion by 15% (from 29% to 44%).

Discussion

The results of this study demonstrate that PrInt students perceived a substantial benefit from a combined educational package and a month of experiential learning in their hospital attachment as PrInt students, in particular, improvements in pharmacy skills and management of procedural skills. However, despite the positive overall increase in most domains, two subscales remained low with only 44% of students reporting confidence in commencing patients on warfarin therapy, and while most respondents had attended an emergency on the ward, only 42% felt confident about handling emergencies. This is despite having had extensive experience in simulated cardiopulmonary resuscitation throughout their medical course and advanced life support skills training in the simulation laboratory during year three, and PrInt. This clearly indicates that respondents felt that they had not had enough practical experience.

While students reported increased performance of N/G tube insertion by 15% (from 39% to 44%), the score remained low, suggesting that the students had low exposure to the procedure in both year three and PrInt. The 15% increase (from 66% to 81%) in taking BSLs was attributed to “hands on” use of the BSL equipment in the PrInt education course.

Students reported a moderate level of confidence in their ability to recognize the acutely ill patient (clinical recognition skills). Interestingly, there was a slight decline in students’ reported ability to manage and treat diabetic patients, as well as in communicating with other members of the medical staff following PrInt (). The likely explanation for this is that students had overestimated their knowledge in the aforementioned skills prior to engaging in further education and learning on the wards. This finding is in accordance with Kruger and Dunning who wrote that “those who are unskilled may over estimate their abilities as they are unaware of their own incompetence”.Citation16

Our findings are similar to those of other recent researchers. Coombes et al in their questionnaire study on pharmacy skills found that medical graduates when commencing internship had little confidence in commencing patients’ warfarin, believing initiation of anticoagulants as being “too great a responsibility”.Citation17 They found that students also felt less confident in the management of patients with diabetes.Citation17 A study by Jones et al also supports our findings in that medical graduates rated themselves well prepared in venipuncture and working in a team but least prepared with N/G tube insertion.Citation18 Wall et al also found that medical graduates are well prepared for working in a team, but in contrast to our findings, students reported to be less prepared in practical skills and pharmacy skills.Citation19

Our students reported lack of confidence in managing an emergency is a well-known, worldwide concern. Citation20,Citation3,Citation21 Clearly, while simulation can shorten the learning curve, these programs do not replace real “hands on experience”.Citation3

Clinical attachment

While it is necessary for new interns to be equipped with sufficient knowledge and ability to perform practical skills, safe internship is also dependent on non-technical skills, such as working environment and ward protocols. Previous cohort studies have found that graduate entry PBL medical courses along with “shadowing“ were associated with improvements in preparedness for internship.Citation1 This was described by Wenger as “legitimate peripheral participation in a community of practice”,Citation22 and this occurs when students work under supervision and integrate their clinical knowledge into practice while learning how the hospital community operates.Citation23 Conversely, it has been reported that “shadowing” can be limiting as an observational role, leaving the student feeling demotivated and frustrated.Citation24 Ideally, “shadowing” should be in the same hospital that they will be allocated to as interns (i.e., their new work environment).Citation23 While PrInt placements are currently held at the “parent” hospital where they were students, unfortunately their intern placement is usually not in their “parent” hospital. To address this issue, all new interns in NSW (Australia) now attend a short orientation, which includes “shadowing” the outgoing intern for one week, prior to commencing their new independent clinical role as interns.

Limitations

This study has several limitations. The use of a control group would have defined if specific gains were due to the education package immediately before PrInt or experiential learning during the four weeks of PrInt.

We acknowledge that this is a relatively small, singlecenter cohort study. Moreover, the questions in this study pertain to self-reported knowledge and confidence, and in the absence of observational feedback, there are obvious concerns of its reliability and validity.Citation25 Instructional courses may improve the learner’s perception of confidence, including their competence in performing the skill. However, while students reported that their knowledge had increased, it may not be an accurate indicator of their actual skills, knowledge, or performance ability.Citation26 A further limitation to this study was its failure to find out how many times a procedure had been performed and whether any feedback had been given.

Further investigation

Results from our findings are important to patient safety, and we are currently conducting further research to determine whether our educational intervention has helped interns to safely care for patients. Medication safety is of particular interest as interns initiate medications, including those which are of high risk, such as warfarin,Citation27 without the counter-signing by an experienced clinician. Students’ lack of confidence in the management of an emergency on the ward is of real concern, which suggests that extensive simulation training throughout their course is not sufficient to overcome this problem. The low frequency of N/G tube insertion is a further concern, which highlights the need for collaboration with nursing staff, who are more likely to perform this skill.

Recommendations

We recommend that a short, intensive transition course be implemented in the curriculum prior to students commencing their PrInt clinical attachment. In Sydney Medical School, the week before PrInt (so called “conference week”) is an ideal time to present the learning package, as students are not involved in other teaching, nor exams, and can shift their focus to their procedural skills, and patient safety issues, prior to transiting their first clinically independent role as interns.

Conclusion

Our results indicate that students do perceive positive benefits from an educational package specifically aimed at improving their practical skills immediately prior to internship. The need for intensive education prior to internship is also highlighted, to address the variation in students’ experiences during clinical rotations to enable students’ awareness of their abilities and limitations prior to internship.

Author contributions

Susan McKenzie is a Clinical Lecturer at Sydney Medical School – Central and is currently a PhD candidate. Susan is responsible for the study concept, design, analysis, and interpretation of the data and drafting of the manuscript. Craig Mellis is a pediatric chest physician, former Associate Dean and Head of Central Clinical School and is now Emeritus Professor at Sydney Medical School. Professor Mellis is responsible for the analysis and interpretation of data and critical revision of manuscript for important intellectual content. Both authors have read and approved the final version of the manuscript.

Abbreviations

PrInt=

pre-intern

N/G=

nasogastric tube

CCS=

Central Clinical School

ISBAR=

identify yourself, situation, background, assessment, and recommendation

BSLs=

blood sugar levels

Acknowledgments

The authors wish to thank the pre-intern students of 2014 who participated in this study.

Disclosure

The authors report no competing conflicts of interest in this work.

References

  • CaveJWoolfKJonesADacreJEasing the transition from student to doctor: How can medical schools help prepare their graduates for starting work?Med Teach200931540340819142797
  • IllingJMorrowGKergonCHow prepared are medical graduates to begin practice? A comparison of three diverse UK medical schools (unpublished report). Final Project Report for the GMC Education CommitteeLondonGeneral Medical Council2008 Available from: http://www.gmc-uk.org/FINAL_How_prepared_are_medical_graduates_to_begin_practice_September_08.pdf_29697834.pdfAccessed January 3, 2017
  • BootsRJEgertonWMcKeeringHWinterHThey just don’t get enough! Variable intern experience in bedside procedural skillsIntern Med J200939422222719402860
  • BleakleyABrennanNDoes undergraduate curriculum design make a difference to readiness to practice as a junior doctor?Med Teach201133645946721609175
  • TallentireVRSmithSEWyldeKCameronHSAre medical graduates ready to face the challenges of Foundation training?Postgrad Med J201187103159059521690255
  • HilmerSNSealeJPLe CouteurDGCramptonRLiddleCDo medical courses adequately prepare interns for safe and effective prescribing in New South Wales public hospitals?Intern Med J200939742843419383059
  • MathesonCMathesonDHow well prepared are medical students for their first year as doctors? The views of consultants and specialist registrars in two teaching hospitalsPostgrad Med J200985100958258919892893
  • LaackTANewmanJSGoyalDGTorsherLCA 1-week simulated internship course helps prepare medical students for transition to residencySimul Healthc20105312713220651473
  • BlackmoreCAustinJLopushinskySRDonnonTEffects of postgraduate medical education “boot camps” on clinical skills, knowledge, and confidence: a meta-analysisJ Grad Med Educ20146464365226140112
  • BraniffCSpenceRAStevensonMBoohanMWatsonPAssistant-ship improves medical students’ perception of their preparedness for starting workMed Teach2016381515826037743
  • CETIPreparing for Safe Practice as an Intern (PSPI) Program. [Curriculum guide]SydneyHealth Education and Training Institute (HETI)2012 Available from: http://www.heti.nsw.gov.au/Global/HETI-Resources/team-health/preparation-for-safe-practice-intern-pspi.pdf
  • GrahamISGleasonAJKeoghGWAustralian Curriculum Framework for junior doctorsMed J Aust20071867 SupplS14S1917407415
  • PettittBJMedical student concerns and fears before their third-year surgical clerkshipAm J Surg2005189449249615820468
  • BrennanNCorriganOAllardJThe transition from medical student to junior doctor: today’s experiences of tomorrow’s doctorsMed Educ201044544945820518984
  • Sydney Medical School, UoSPre-Internship (PRINT) Stage 3 Handbook. Vol Version 142014SydneySydney Medical School, UoS2016
  • KrugerJDunningDUnskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessmentsJ Pers Soc Psychol19997761121113410626367
  • CoombesIDMitchellCAStowasserDASafe medication practice: attitudes of medical students about to begin their intern yearMed Educ200842442743118338996
  • JonesAMcArdlePJO’NeillPAHow well prepared are graduates for the role of pre-registration house officer? A comparison of the perceptions of new graduates and educational supervisorsMed Educ200135657858411380861
  • WallDBolshawACarolanJFrom undergraduate medical education to pre-registration house officer year: how prepared are students?Med Teach200628543543916973456
  • WakefordRRobertsSAn evaluation of medical students’ practical experience upon qualificationMed Teach19824414014324483765
  • TallentireVRSmithSESkinnerJCameronHSUnderstanding the behaviour of newly qualified doctors in acute care contextsMed Euc2011459951005
  • WengerECommunities of practice and social learning systems: the career of a conceptBlackmoreCSocial Learning Systems and Communities of PracticeLondonSpringer London2010179198
  • JonesAWillisSCMcArdlePJO’NeillPALearning the house officer role: reflections on the value of shadowing a PRHOMed Teach200628329129316753731
  • DrinkwaterJCan a medical student be of use while still learning?Clin Teach200744189192
  • GordonMJA review of the validity and accuracy of self-assessments in health professions trainingAcad Med199166127627691750956
  • BarnsleyLLyonPMRalstonSJClinical skills in junior medical officers: a comparison of self-reported confidence and observed competenceMed Educ200438435836715025637
  • CoombesIMitchellCStowasserDSafe medication practice tutorials: a practical approach to preparing prescribersClin Teach200743128134