Factors associated with infectious diseases fellowship academic success

ABSTRACT

Background

A multitude of factors are considered in an infectious disease (ID) training program’s meticulous selection process of ID fellows but their correlation to pre and in-fellowship academic success as well as post-fellowship academic success and short-term outcomes is poorly understood. Our goal was to investigate factors associated with subsequent academic success in fellowship as well as post-fellowship short-term outcomes.

Methods

In 2022, we retrospectively analyzed deidentified academic records from 39 graduates of the Mayo Clinic Rochester ID Fellowship Program (1 July 2013- 30 June 2022). Data abstracted included demographics, degrees, honor society membership, visa/citizenship status, medical school, residency training program, United States Medical Licensure Exam (USMLE) scores, letters of recommendation, in-training examination (ITE) scores, fellowship track, academic rank, career choice, number of honors, awards, and abstracts/publications prior to fellowship, during training, and within 2 years of graduation.

Results

Younger fellows had higher USMLE step 1 scores, pre and in-fellowship scholarly productivity, and higher ITE performance. Female fellows had significantly higher USMLE step 3 scores. Prior research experience translated to greater in-fellowship scholarly productivity. Higher USMLE scores were associated with higher ID ITE performance during multiple years of fellowship, but USMLE step 2 clinical knowledge and 3 scores were associated with higher pre and in-fellowship scholarly productivity and receiving an award during fellowship. The USMLE step 1 score did not correlate with fellowship performance beyond year 1 and 2 ITE scores.

Conclusions

Multiple aspects of a prospective fellow’s application must be considered as part of a holistic review process for fellowship selection. USMLE step 2 CK and 3 scores may predict fellowship performance across multiple domains.

KEYWORDS:

• Infectious diseases fellowship
• academic success
• medical education
• outcomes
• infectious diseases

Background

Dedicated subspecialty training in Infectious Diseases (ID) is designed to provide a comprehensive experience in clinical practice, research, and education to allow graduates to practice independently in a variety of settings. ID physicians can practice in academic medical centers, where in addition to clinical practice, there is an emphasis on scholarly productivity and medical education of learners, or in community or private practice, where positions tend to be more clinically oriented. Depending on the individual’s career interest, fellows may seek training in an academic or clinical fellowship training program. Most of the ID fellowship training programs in the US offer two years of intensive clinical training, while some programs offer an additional year of studies focused on a career as a physician scientist or an ID subspecialist (e.g., transplant ID). In 2023, there were 189 certified ID fellowship training programs in the United States, and the majority offer two years of clinical training at academic medical centers. These ID fellowship programs interviewed prospective applicants to fill 450 certified fellowship positions through the National Resident Matching Program (NRMP), in an ‘all-in’ match on a yearly basis. The application, interview, and ranking process is similar to prerequisite Internal Medicine (IM) residency training [1].

As part of this process, ID fellowship selection committees are tasked with comprehensively reviewing multiple aspects of a prospective fellow’s application including pre-medical, medical school and IM residency performance, United States Medical Licensing Examination (USMLE) step 1, 2 clinical knowledge (CK), and 3 scores, program director and faculty letters of recommendation, personal statement, as well as extracurricular, volunteer, and scholarly activities. Following this rigorous initial holistic review, candidates are selected to interview for available positions through the NRMP match. During the interview, the selection committee will assess the applicant’s career goals as well as the overall fit into the program. At the conclusion of the interview process, programs’ rank order lists of applicants are created after taking into consideration the pre-interview application strength, interview interactions with program director and selection committee members, faculty, staff, and current trainees, as well as institutional fit (to ensure that their career goals will be supported by the program). This list is then submitted to the NRMP to facilitate the match process.

Despite this approach to the review of prospective fellows’ applications, selection of interview candidates and subsequent creation of a program rank order list, the factors taken into consideration in the selection and ranking of applicants and their association with academic success have not been well characterized. For the purpose of this study, academic success was measured by performance during standardized testing, scholarly productivity, career choices, and academic productivity within 2 years of graduation. Factors associated with fellowship academic success have been identified in larger IM subspecialties, but not for ID fellowships [2,3]. The comparatively lower number of ID fellows relative to other IM subspecialty training programs, such as gastroenterology, cardiology, and hematology/oncology may have limited the feasibility of such a study. Additionally, this is further compounded by historically lower fill rates of ID fellowship programs over the years [4–6]. In 2023, about half (50.8%) of the ID fellowship training programs completely filled their certified positions, while the rest (49.2%) have at least one unfilled position; only 303 of the 450 (67.3%) certified positions were filled successfully through the NRMP [1].

Given the paucity of data surrounding factors associated with fellowship academic success in the current literature, we sought to perform a retrospective review of trainees to identify factors associated with academic success among ID fellows in our program.

Methods

The Mayo Clinic School of Graduate Medical Education (MCSGME) ID Fellowship Program, founded in 1961 in Rochester, Minnesota, is a three-year training program that provides a robust and immersive clinical experience for trainees coupled with at least 12 months of protected research time. Currently, the program recruits fellows to fill 6 available positions each year through the fellowship recruitment and NRMP match process. The additional academic year guarantees institutional-based funding for research, which is not necessarily linked to a federal training grant. The three-year training program allows for individualized career development through a variety of subspecialty ID tracks to tailor the training according to the trainee’s career goals. These individualized fellowship tracks provide focused clinical and research experiences in general ID, intensive care infections, orthopedic infections, global health, antimicrobial stewardship, and infection control, and infections of transplant and immunocompromised patients. These tracks allow for clinical elective time to be focused within the desired training track to build a strong foundation within a specific subspecialty of ID. Occasionally, one of the fellowship slots is filled by a Clinician Investigator, who are recruited prior to or during their IM residency training; the Clinician Investigator Training Program is a unique program for internal Mayo Clinic applicants as the fellow starts their ID research during the last 6 months of their IM residency training and this continues on during ID fellowship for a total of 2 years of dedicated research time. Mentorship throughout a fellow’s select track is facilitated by the Program Director in careful coordination with career and research mentors through the Career Development Council. Depending on their clinical, academic, and scholarly performance, ID fellows are also eligible for nomination to the Academic Appointment and Promotions Committee for instructor and assistant professor of medicine academic ranks, with the designation commensurate to their accomplishments. The program has a 100% American Board of Internal Medicine (ABIM) ID Certification Exam pass rate.

Following Mayo Clinic Institutional Review Board approval, we performed a retrospective cohort study of graduates of the Mayo Clinic Rochester ID fellowship program over a decade-long period between 2013 and 2022 by reviewing de-identified trainee academic records. During this study period, the number of ID fellows increased from 3 per year to 6 per year [1]. Prior to data collection and analysis, the study protocol was reviewed by the Educational Research Committee of the Mayo Clinic Institutional Review Board and found to be in compliance with the learners as research subjects policy at Mayo Clinic. Data abstracted included demographics, medical degree, additional advanced degrees, honor society membership, visa/citizenship status, medical school, residency training program type and size, USMLE step 1, 2CK, and 3 scores, letter of recommendation with note of a top qualifier, in-training examination (ITE) scores, fellowship training track, academic rank at graduation, career choice, number of abstracts, honors, awards, and publications prior to matriculation into fellowship, during training and within 2 years of graduation.

Publication data were collected following a search of indexed publications on PubMed accounting for name changes of fellows in addition to a review of an internal Mayo Clinic database of scholarly activity including peer-reviewed articles, abstracts, editorials, letters, commentaries, and book chapters. Data on awards were obtained from de-identified academic records as outlined above at the following levels: divisional awards given to fellows in recognition of clinical performance or scholarly productivity, awards given at the institutional level (for work related to clinical care, safety, research), and national awards (including travel grants that recognize abstracts of distinction). The Geraci Award is a divisional award that is voted on by faculty and is given to one or two graduating fellows each year in recognition of excellent performance in clinical care, education, and research during ID fellowship.

Data in the accompanying tables are displayed as either frequency (%) for nominal variables or median (range) for continuous variables. Spearman’s correlations were used to test associations between two continuous variables. Associations between continuous predictors and binary outcomes were tested by univariable logistic regression. Two-sided t tests or analysis of variance models were used to test associations between binary or multinominal predictors, respectively, and continuous outcomes. Fisher’s exact test was used to test associations between two binary variables. All analyses were performed on RStudio statistical software, version 2022.07.1.554 (RStudio Team, PBC, Boston, Massachusetts, United States).

Results

Over the study time frame, there were thirty-nine ID fellowship program graduates with the accompanying characteristics (Table 1). Twenty-one fellows were male (53.8%). The median age at the beginning of ID fellowship was 31 years (range 26–42). Sixteen (41%) trainees were United States citizens, whereas fourteen fellows held a J1 visa (35.9%), 8 held a H1b visa (20.5%) and one fellow (2.6%) was a permanent United States resident. Twenty-five fellows (64.1%) were international medical graduates. Median USMLE step 1, 2 CK, and 3 scores were 240, 246, and 233, respectively. Twenty-three fellows (59%) completed IM residency training at an academic medical center and 33% of trainees attended a residency program with more than fifty trainees per residency class. Five fellows (12.8%) completed additional training as a Chief Medical Resident. The median number of abstracts and publications prior to fellowship were 5 and 3, respectively.

Table 1. Pre-fellowship characteristics of 39 infectious diseases Fellows.

Variable	Overall (N = 39)
Sex
– Female	18 (46.2)
– Male	21 (53.8)
Age at start of fellowship, years, median (range)	31 (26–42)
Medical Degree
– M.D.	26 (66.7)
– M.B.B.S.	11 (28.2)
– M.B., B.Ch., B.A.O.	1 (2.6)
– M.B., Ch.B.	1 (2.6)
Ph.D.	2 (5.1)
Medical School Location
– International	25 (64.1)
– United States	14 (35.9)
Visa/Resident/Citizenship Status
– United States Citizen	16 (41.0)
– Permanent US Resident	1 (2.6)
– J1 Visa	14 (35.9)
– H1B Visa	8 (20.5)
Alpha Omega Alpha status (N = 14)	4 (28.5)
Gold Humanism Honor Society status (N = 14)	1 (7.1)
USMLE Step 1, median (range)	240 (198–269)
USMLE Step 2 clinical knowledge, median (range)	246 (216–275)
USMLE Step 3, median (range) (N = 38)	233 (197–253)
Residency Program Type
– Academic/University	23 (59.0)
– Community	16 (41.0)
Residency Program Size, number of residents per year
– <15	9 (23.1)
– 15–24	7 (17.9)
– 25–34	6 (15.4)
– 35–44	4 (10.3)
– >50	13 (33.3)
Chief Medical Resident	5 (12.8)
Clinical Investigator	3 (7.7)
Letter of Recommendation with Top Qualifier (N = 38)	4 (10.5)
Abstracts Pre-Fellowship, median (range)	5 (0–14)
Publications Pre-Fellowship, median (range)	3 (0–18)

Abbreviations: USMLE, United States Medical Licensing Examination.

^aData are N (%) unless otherwise specified.

During training, the median number of abstracts was 6 (range, 1–21) and the median number of publications was 6 (range, 0–54). The most common fellowship tracks were the traditional clinician scholar followed by intensive care (paired with antimicrobial stewardship and infection control), transplant and immunocompromised host and research tracks (Table 2). Thirty-six (92.6%) of fellows received an award during training and 97.4% of graduates took ID consultant positions at academic institutions. Additional cohort characteristics are available in Tables 1 and 2.

Table 2. Intra-fellowship and post-fellowship characteristics of 39 infectious diseases Fellows .

Variable	Overall (N = 39)
Year 1 Infectious Diseases ITE, median (range) (N = 38)	512 (359–727)
Year 2 Infectious Diseases ITE, median (range)	573 (455–800)
Year 3 Infectious Diseases ITE, median (range) (N = 31)	586 (436–800)
Infectious Diseases Board Examination Score, median (range) (N = 28)	591.5 (393–800)
Publications during Fellowship, median (range)	6 (0–54)
Abstracts during Fellowship, median (range) (N = 38)	6 (1–21)
Award during Fellowship	36 (92.3)
– Geraci Award	13 (33.3)
– Divisional	22 (56.4)
– Institutional	9 (23.1)
– National	32 (82.1)
Awards during Fellowship, median (range)	3 (0–7)
Chief Fellow designation	15 (38.5)
Fellowship Track
– Intensive Care	9 (23.1)
– Orthopedic	2 (5.1)
– Research	9 (23.1)
– Traditional Clinician Scholar	10 (25.6)
– Transplant/Immunocompromised host	9 (23.1)
Academic Appointment at Graduation
– Assistant Professor of Medicine	22 (56.4)
– Instructor of Medicine	14 (35.9)
– None	3 (7.7)
Career Choice
– Academic	38 (97.4)
– Private Practice	1 (2.6)
Publications within 2 years post-fellowship, median (range) (N = 36)	4 (0–26)

Abbreviations: ITE, In-Training Examination; US, United States; USMLE, United States Medical Licensing Examination.

^aData are N (%) unless otherwise specified.

Factors associated with pre-fellowship performance

Female gender was associated with a higher USMLE step 3 score (p = .041), while a younger age was associated with a higher USMLE step 1 score and a higher number of pre-fellowship publications (p = .021 and .032, respectively). Further, a higher USMLE step 2 score was associated with a higher number of pre-fellowship publications (p = .031). The results of this analysis of the effect of demographic factors and pre-fellowship variables are shown in Table 3.

Table 3. Associations between demographic factors and pre-fellowship Variables.

Pre-Fellowship Variable
Demographic Variable	USMLE 1	USMLE 2 CK	USMLE 3	LOR Top Qualifier	Abstracts Pre-Fellowship	Publications Pre-Fellowship
Sex	0.271	0.283	0.041	0.606	0.978	0.520
Age	0.021	0.223	0.298	0.969	0.066	0.032
Citizenship Status	0.911	0.666	0.759	0.841	0.959	0.388
Medical School Location	0.899	0.512	0.649	0.278	0.280	0.247
Early Pre-Fellowship Variable		Late Pre-Fellowship Variable
		LOR Top Qualifier		Abstracts Pre-Fellowship	Publications Pre-Fellowship
USMLE 1		0.320		0.256	0.483
USMLE 2 CK		0.581		0.258	0.031
USMLE 3		0.395		0.246	0.849

Abbreviations: ITE, In-Training Examination; US, United States; USMLE, United States Medical Licensing Examination; CK, Clinical Knowledge; LOR, Letter of Recommendation.

^aData are N (%) unless otherwise specified.

Factors associated with performance during ID fellowship

The results of the analysis of associations between pre-fellowship and in-fellowship variables are displayed in Table 4. A younger age was found to be associated with higher year 1 and year 2 ITE scores (p = .027 and .025, respectively) and higher number of publications during fellowship (p = .005). Graduating from an academic/university-based residency program was associated with higher likelihood of receiving the Geraci award (p = .037), while a higher number of pre-fellowship abstracts were associated with higher numbers of abstracts and publications during fellowship (p = .026 and .002 respectively).

Table 4. Associations between pre-fellowship and In-fellowship Variables .

Pre-Fellowship Variable	In-Fellowship Variable
	Year 1 ITE	Year 2 ITE	Year 3 ITE	Abstracts during Fellowship	Publications during Fellowship	Award during Fellowship	Geraci Award
Sex	0.590	0.688	0.540	0.322	0.654	1.000	1.000
Age	0.027	0.025	0.789	0.091	0.005	0.200	0.164
Citizenship Status	0.298	0.296	0.811	0.723	0.521	0.215	1.000
Medical School Location	0.745	0.236	0.773	0.185	0.413	1.000	0.482
USMLE 1	0.002	0.041	0.069	0.523	0.709	0.497	0.350
USMLE 2 CK	0.003	0.010	0.003	0.456	0.454	0.114	0.015
USMLE 3	0.008	0.045	0.051	0.454	0.042	0.046	0.022
LOR Top Qualifier	0.652	0.388	0.454	0.792	0.518	1.000	0.730
Residency Program Type	0.161	0.796	0.986	0.666	0.282	0.557	0.037
Residency Program Size	0.940	0.161	0.759	0.166	0.782	1.000	0.863
Chief Resident	0.195	0.374	0.332	0.803	0.422	1.000	1.000
Abstracts Pre-Fellowship	0.570	0.135	0.990	0.026	0.002	0.428	0.910
Publications Pre-Fellowship	0.901	0.973	0.968	0.061	0.088	0.658	0.683

Abbreviations: ITE, In-Training Examination; US, United States; USMLE, United States Medical Licensing Examination; CK, Clinical Knowledge; LOR, Letter of Recommendation.

^aData are N (%) unless otherwise specified.

A higher USMLE step 1 score was associated with higher year 1 and year 2 ITE scores, whereas a higher USMLE step 2 score was associated with higher ITE scores across all 3 years and higher likelihood to receive the Geraci award (Table 4). Additionally, USMLE step 3 performance was associated with higher year 1 and year 2 ITE scores, a higher number of publications during fellowship, higher likelihood of receiving an award during fellowship, and higher likelihood to receive the Geraci award (Table 4).

Factors associated with post-fellowship outcomes

With regard to the association of in-fellowship variables and outcomes data, fellows who received an award during fellowship had higher numbers of publications both during fellowship and during their first two years after fellowship graduation (p = < .001 and .002, respectively). Post-hoc pairwise analysis showed those appointed to the academic rank of Assistant Professor of Medicine had higher numbers of abstracts and publications during fellowship as compared to those appointed to instructor of medicine rank (Table 5). In addition, years 1–3 ITE performance as well as the receipt of the Geraci award were associated with a higher score on the ABIM ID Certification Exam (Table 5).

Table 5. Associations between In-fellowship variables and Outcomes .

In-Fellowship Quantitative	Research Productivity			Board Exam
	Abstracts during Fellowship	Publications during Fellowship	Publications within 2 years Post-Fellowship	Infectious Diseases Board Examination Score
Year 1 ITE	0.825	0.327	0.132	0.002
Year 2 ITE	0.479	0.120	0.408	<0.001
Year 3 ITE	0.465	0.417	0.373	0.003
Chief Fellow	0.145	0.229	0.236	0.523
Fellowship Track	0.100	0.069	0.888	0.712
Award during Fellowship	0.288	<0.001	0.002	0.903
Geraci Award	0.263	0.967	0.744	0.044
Academic Appointment	0.008	0.017	0.296	0.537

Abbreviations: ITE, In-Training Examination.

^aData are N (%) unless otherwise specified.

Discussion

The benefit of ID subspecialty evaluation has previously been highlighted through studies demonstrating improved survival and outcomes in patients with blood stream infections and drug-resistant organisms [7–9]. The importance of ID expertise in routine clinical care and ongoing public health efforts has been further accentuated throughout the COVID-19 pandemic and moving forward, the need for subspecialty training in ID continues to be of the utmost importance for ongoing pandemic preparedness [10]. Given the ongoing low fill rates and the challenges with recruiting trainees to the ID specialty, it is imperative to develop a deeper understanding of how fellows can be most successful academically throughout their training experience [1,4].

Prior studies have characterized factors associated with academic success and career choice within hematology and oncology fellowship training, which concur with our observed correlations between USMLE examination and ITE performance but were not predictive of academic productivity. In contrast, we observed that age and gender were associated with ITE performance among our ID fellows [2]. Hematology and oncology fellows who were non-US citizens and/or international medical graduates (IMGs) were found to be more academically productive, but this was not seen in our cohort. These differences in observations highlight the need to assess factors that are unique or associated with specific training programs. To our knowledge, our study is the first study of its kind to identify factors associated with ID fellowship academic success, taking into consideration data provided during the fellowship application process and the association on subsequent fellowship achievement and academic success.

Several pre-fellowship variables were significantly correlated with academic performance during ID fellowship training. Having a prior research experience translated to significantly higher in-fellowship scholarly productivity. It is possible that satisfaction in research experience ignited the passion among these ID fellows to develop a career as a clinician investigator or scholar. There was also association among scores on pre-fellowship standardized examinations (USMLE) and subsequent performance in-fellowship examinations (ITE). In our study, higher USMLE scores were associated with higher performance on the ID ITE during multiple years of fellowship, confirming findings from prior studies demonstrating this association [11]. Beyond their association with ITE performance, higher USMLE step 2 CK and 3 scores were also associated with higher pre and in-fellowship scholarly productivity as well as likelihood of receiving an award during fellowship. Indeed, the collection of these objective measures could reflect the multifaceted excellence of fellows that eventually gets recognized with these awards during fellowship, particularly the Geraci Award. Interestingly, in contrast, USMLE step 1 score did not significantly correlate with fellowship performance beyond the year 1 and 2 ITE scores. Similar correlations of USMLE examination performance to ID ITE equivalents have been observed across other non-IM specialties including orthopedic surgery, plastic surgery, anesthesiology, emergency medicine, and obstetrics and gynecology residencies [12–16].

Applications of this work include its potential to guide both ID fellowship selection committees and prospective applicants through subsequent application cycles [1]. The association between pre-fellowship research experience and in-fellowship research productivity may be useful for training programs that are geared towards recruiting and training the academic clinician-scientists, the importance of which cannot be overstated, as they are often relied upon to move the field with their cutting-edge scientific investigations. The association between USMLE step 2 CK and 3 performance and in-fellowship success variables across multiple domains suggests that these examinations may serve as additional tools for selection committees when considering a candidate’s potential for in-fellowship academic success. The importance of the USMLE step 2 CK and 3 examinations may be further underscored in future application cycles as the USMLE step 1 recently transitioned to a pass/fail examination in 2022. While unique to our study, previous research outside of ID to evaluate the utility of USMLE step 2 CK on prediction of training performance has been mixed. This examination has previously been shown to be a predictor of multimodal performance during IM residency in the ambulatory setting, whereas this examination was not found to be a reliable predictor American Board of Neurological Surgery written exam performance [17,18].

Limitations of this study include its generalizability across ID fellowship programs. First, this retrospective study was conducted at a large academic medical center in the Midwestern United States. Secondly, the Mayo Clinic Rochester ID Fellowship Program is a three-year program for all its general ID fellows, providing them with the opportunity for academic advancement. Our results may therefore reflect programs with a similar length of training. Admittedly, this is not the same structure for most ID fellowship programs in the country, where 2 years is the standard duration of training for eligibility for certification by the ABIM. Accordingly, measures of scholarly productivity in this study may not be applicable to most two-year clinical training programs. However, there are other findings that may be pertinent to all programs, such as the strong association among standardized examinations, and other factors that predict excellence in clinical skills, research, and education (as measured by the Geraci award in this program). Future investigations may be targeted towards elucidating the effect of pre and in-fellowship variables on in-fellowship clinical performance as determined through review of standardized evaluations completed during fellowship training mapped to the Accreditation Council for Graduate Medical Education (ACGME) Infectious Disease Milestones.

In conclusion, this study of a relatively large academic training program in ID observed several variables that were associated with academic success during fellowship training and short-term outcomes after graduation from the ID fellowship training program.

List of abbreviations

ABIM	=	American Board of Internal Medicine
CK	=	Clinical Knowledge
ID	=	Infectious Diseases
IM	=	Internal Medicine
ITE	=	In-Training Examination
MCSGME	=	Mayo Clinic School of Graduate Medical Education
NRMP	=	National Resident Matching Program
USMLE	=	United Stated Medical Licensing Examination

Author contributions

R.B.K. contributed to the conception, design, writing of the man manuscript text, preparation of figures and tables, and review of this manuscript. Z.A.Y. contributed to the conception, design, statistical analysis, preparation, and manuscript review. C.L.D. contributed to the data collection for the manuscript. R.R.R contributed to the conception, design, and manuscript review.

Ethical approval and consent to participate

The Mayo Clinic Institutional Review Board reviewed the study protocol and granted it an exempt status (#22–004105) on 6/20/2022. Informed consent was obtained for subjects and found to be in compliance with the learners as research subjects policy at Mayo Clinic. Additionally, all methods and research/study protocols were carried out in accordance with relevant guidelines and regulations which were reviewed by the Educational Research Committee of the Institutional Review Board and found to be in compliance with the learners as research subjects policy at Mayo Clinic.

Disclosure statement

R.R.R. receives research support (funds provided to the institution) from Gilead, Regeneron, Roche, the MITRE corporation, and Nference, Inc. R.R.R. serves on the advisory board for Glaxo Smith Kline and on the Data Safety Monitoring Board for Novartis and the Endpoint Adjudication Committee of Allovir. R.R.R. also serves as a member of the Board of Directors of the American Society of Transplantation. None of these entities have provided support for this current study. Otherwise, R.B.K., Z.A.Y., and C.L.D. have no conflicts of interest to report.

Data availability statement

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Additional information

Funding

This project was supported by Grant Number [UL1 TR002377] from the National Center for Advancing Translational Sciences (NCATS). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the National Institutes of Health.