https://orcid.org/0000-0001-6923-0101
ABSTRACT
For diversity to exist in the medical graduate workforce, students from all backgrounds should have equitable opportunities of employment. Specialties have utilized a minimal threshold for USMLE Step 1 score when screening applicants for residency interviews. The OHSU SOM class of 2021 completed a 14-question voluntary survey on their Step 1 score and the following non-modifiable risk factors: Adverse Childhood Experience score (ACEs), sex, gender, Underrepresented in Medicine status (URiM), family income during adolescence, highest degree held by a guardian, discrimination experience during medical school, federal/state assistance use, and rural versus urban primary home. Descriptive statistics and unadjusted risk ratios were applied to study the relation between Step 1 score and non-modifiable risk factors as well as certain non-modifiable risk factors and ACEs ≥ 3. The mean Step 1 score was 230 (213, 247). Of the students, 28.2% identified ACEs ≥ 3, 13.6% were considered URiM, and 65.4% were female. URiM were 2.34 (1.30, 4.23), females were 2.77 (1.06–7.29), and those who experienced discrimination in medical school were 4.25 (1.85, 9.77) times more likely to have ACEs ≥ 3. Students who had ACEs ≥ 3 were 3.58 (1.75, 7.29) times less likely to meet a minimal threshold for residency interviews of 220. These are the first results to demonstrate a relationship between Step 1 score and ACEs. Those who identified as URiM, females, and those who experienced discrimination in medical school were at a higher risk of ACEs of ≥ 3. Step 1 transitioned to pass/fail in January 2022. However, the first application cycle that residencies will see pass/fail scoring is 2023–2024, and fellowships will continue to see scored Step 1 until, at the earliest, the 2026–2027 application cycle. These data contribute to a foundation of research that could apply to Step 2CK testing scores, and help to inform decisions about the diversity and equity of the residency interview process.
Background
For there to be diversity in the physician workforce, students from all backgrounds should have equitable opportunities of employment. Residency programs have historically utilized United States Medical Licensing Examination (USMLE) Step 1 scores to determine interview offers. The use of minimum thresholds raises the issue of diversity and equity in residency training [Citation1].
During the medical residency application process, many specialties use minimum thresholds for boards examination scores when deciding whether to offer a candidate an interview for residency. Often, if a candidate does not meet that minimum threshold, they will not be offered an interview. Specialities considered more competitive, such as radiology, orthopedic surgery, and dermatology have higher minimum thresholds for boards examinations. When reviewing the Fellowship and Residency Electronic Interactive Database System (FREIDA), updated yearly by the American Medical Association (AMA), minimum thresholds can range from as low as the passing score of the exam, 194 (in 2021), to as high as 256, or approximately the 90th percentile [Citation2]. Internal medicine residency programs were 3–6 times less likely to interview a Black candidate when using Step 1 minimum thresholds [Citation3]. No study has examined a breadth of non-modifiable risk factors and their relation to Step 1 scores [Citation3,Citation4].
The cultural competency of the physician workforce was set as a major goal for the AMA in 2017 [Citation5]. Saha describes how social class and socioeconomic differences could worsen the ‘cultural distance’ between physicians and their patients [Citation6]. Racial concordance between patients and their physicians has also been shown to increase communication, and decrease mortality rate among patients care for by female compared to male physicians [Citation7,Citation8]. In order to achieve a medical education, medical students who identify as Underrepresented in Medicine (URiM) must cross a wide divide. Matriculants to MD programs increasingly come from households with higher than average household incomes, and as racial and ethnic minorities become more well-represented in medical school applications, they continue to be less likely to matriculate [Citation9,Citation10]. The divide continues through graduate medical education. Residencies considered more competitive such as orthopedic surgery, also have less representation from individuals who identify as URiM [Citation11]. To best explore the cultural competency of this workforce – including providing more support for individuals who identify as URiM – we must understand the adversity that these individuals go through on their path to and through graduate medical education. One method of identifying adversity in childhood is through the Adverse Childhood Experience Score (ACEs). No study has analyzed ACEs in the undergraduate or graduate medical education population.
We aim to understand the relationship between non-modifiable risk factors and Step 1 exam scores for a sample of medical students at the Oregon Health & Science University School of Medicine (OHSU SOM), what these findings may elucidate regarding the disparities in the early academic preparedness for medical school boards examinations, and ultimately the diversity of the medical workforce. We will also explore childhood adversity through the ACEs, non-modifiable characteristics that may be related to childhood adversity, and how this adversity could play a role in the cultural competency or ‘cultural distance’ of the physician workforce.
Materials and methods
A 14-question online survey asked participating students within the class of 2021 at OHSU SOM about the following non-modifiable risk factors: ACEs, sex, gender, URiM status, family income during adolescence, highest degree held by a guardian, discrimination experience during medical school, use of federal or state assistance, and rural versus urban primary home. Those who are URiM include Black, American Indian or Alaskan Native, Pacific Islander or Native Hawaiian, Hispanic/Latino, or Multiracial individuals. Those who are overrepresented in medicine (ORiM) include White and Asian individuals. Of the 160 persons who were invited to participate in the survey, 81 answered the survey. The study sample was representative of the OHSU student body, as the majority of the student body is White, Non-Hispanic/Latino, and female. A 10-item ACEs questionnaire was applied [Citation12,Citation13]. Prior literature has utilized ACEs in the setting of identifying disparities in mental and physical health. In these studies, a score of ≥ 3 identified a participant with a high risk childhood [Citation14,Citation15]. ACEs were then converted to a categorical variable. No data was missing, and all survey data was analyzed.
USMLE identification number, and the month and year students took Step 1 were matched to Step 1 score by author, TB, in her role as Associate Dean for Undergraduate Medical Education. Student researchers received deidentified data.
We report descriptive statistics for Step 1 score and non-modifiable risk factors. We examined the relationship between Step 1 score as a binary variable (<220 or ≥ 220), to mimic minimum thresholds, found on many residency applications filters on FREIDA, and non-modifiable risk factors through unadjusted risk ratios with 95% CI.
This study was approved by the OHSU Institutional Review Board in December of 2020.
Results
The mean Step 1 exam score was 230.6 (213.4, 247.8). All descriptive statistics and analysis for the respondents are summarized in .
Table 1. Descriptive statistics non-modifiable risk factors and analysis with unadjusted risk ratios.
Analysis through unadjusted risk ratios found that Step 1 score and ACEs (p < 0.01) were associated. Students who had an ACEs of ≥ 3 had a 3.58 (95% CI [1.75–7.29]) times increased risk of not meeting a Step 1 exam minimum threshold of 220. Students who identified as URiM (2.34 (95% CI [1.30–4.23])), female (2.77 (95% CI [1.06–7.29])), and those who experienced discrimination in medical school (4.25 (95% CI [1.85–9.77])) had significantly increased risk of ACEs of ≥ 3. The relationship between Step 1 score and ACEs is no longer significant when assessed at a minimum threshold or passing score of 194.
Discussion
These are the first results, to date, which demonstrate a relationship between Step 1 exam score and ACEs. This is also the first study to analyze a breadth of non-modifiable risk factors and their relation to Step 1 exam scores. Students who identify as URiM, female, or who have experienced discrimination in medical school are more likely to have an ACEs of ≥ 3. This relationship may have cascading effects (1) putting certain students at higher risk of not meeting minimum thresholds for residency interviews, and (2) exacerbating the existing problem of inadequate diversity in the physician workforce.
This survey was limited due to a lack of inclusion of questions about disability, sexual orientation, or modifiable risk factors like duration of time spent studying for the exam. The results should also be analyzed considering the change of Step 1 to pass/fail scoring. While Step 1 was made pass/fail in January 2022, the majority of students take Step 1 in their second year of medical school. This makes the application cycle of 2023–2024 the first in which the majority of values provided to residencies will be in a pass/fail format. A majority of current residents have Step 1 scores, and these scores will continue to follow them through further training endeavors, such as fellowship applications. Minimum thresholds may continue to impact fellowship applications. Further, these results also hold implications for other USMLE examinations such as the Step 2 Clinical Knowledge (Step 2CK) test. While scores for Step 2CK were not gathered in this study, a foundation has been laid regarding how childhood experiences may impact graduate medical education and the diversity, equity, and holistic review of applicants to residency programs. Future studies could develop ways to identify students who are at risk of not meeting minimum thresholds and provide them with additional resources that may create a more equitable field prior to taking the Step 1 or Step 2 CK. These studies could also examine what a holistic review of an applicant means. Other items on a residency application like paid work, volunteering, leadership, and research may then become higher factors in how a residency may view an applicant. The equity of these aspects of the application would need to be studied among those with childhood adversity, URiM status, females, and other underprivileged individuals. We hope that these results move institutions to take greater care in the residency interview process.
Availability of data and material
The datasets used and analyzed during the current study are available from the corresponding author upon a reasonable request.
Consent for publication
All participants of the study provided informed consent to participate in the study and publish its findings.
Ethics approval and consent to participate
The Oregon Health & Science University Institutional Review Board approved the present study (Study ID: 00022312). All methods were performed in accordance with the Declaration of Helsinki. We only analyzed the data of the participants that provided informed consent. We did not collect sensitive information from participants and data was anonymized prior to data processing.
Acknowledgments
We would like to thank the students who participated in this project and helped bring it to fruition.
Disclosure statement
No potential conflict of interest was reported by the author(s).
Correction Statement
This article was originally published with errors, which have now been corrected in the online version. Please see Correction (http://dx.doi.org/10.1080/10872981.2024.2334472).
Additional information
Funding
References
- Williams M, Kim E, Pappas K, et al. The impact of United States medical licensing exam (USMLE) step 1 minimal threshold scores on recruitment of underrepresented minorities in medicine: a retrospective cross‐sectional study. Health Sci Rep. 2020;3(2). doi: 10.1002/hsr2.161
- Tools To Help You. FREIDA. https://freida.ama-assn.org/
- Edmond MB, Deschenes JL, Eckler M, et al. Racial bias in using USMLE step 1 scores to grant internal medicine residency interviews. Acad Med. 2001;76(12):1253–4. doi: 10.1097/00001888-200112000-00021
- Rubright J, Jodoin M, Barone M. Examining demographics, prior academic performance, and United States medical licensing examination scores. Academic Medicine. 2019;94(3):364–370. doi: 10.1097/ACM.0000000000002366
- Murphy B. AMA To Aim For More Diverse, Better Prepared Physician Workforce, American Medical Association; 2017. https://www.ama-assn.org/education/medical-school-diversity/ama-aim-more-diverse-better-prepared-physician-workforce
- Saha S. The relevance of cultural distance between patients and physicians to racial disparities in health care. J Gen Intern Med. 2006;21(2):203–205. doi: 10.1007/s11606-006-0262-1
- Shen MJ, Peterson E, Costa-Muniz R, et al. The effects of race and racial concordance on patient-physician communication: a systematic review of the literature. J Racial Ethn Health Disparities. 2017;5(1):117–140. doi: 10.1007/s40615-017-0350-4
- Basky G. Gender may not be the biggest factor in women doctors’ lower patient mortality rates. Can Med Assoc J. 2021;193(33):E1308–E1309. doi: 10.1503/cmaj.1095957
- Katherine B. Medical schools continue to favour white, middle class candidates. BMJ. 2004;328(7455):1518.3. doi: 10.1136/bmj.328.7455.1518-b
- Nguyen M, Desai M, Fancher T. Temporal trends in childhood household income among applicants and matriculants to medical school and the likelihood of acceptance by income, 2014-2019. JAMA. 2023;329(21):1882–1884. doi: 10.1001/jama.2023.5654
- Shah K, Zhuang T, Scott B, et al. What program characteristics are associated with resident racial diversity in orthopedic surgery. An analysis of association of American medical colleges data. J Of Bone And Joint Surg. 2023;8(1). doi: 10.2106/JBJS.OA.22.00056
- Anda R, Butchart A, Felitti V, et al. Building a framework for global surveillance of the public health implications of adverse childhood experiences. Am J Prev Med. 2010;39(1):93–98. doi: 10.1016/j.amepre.2010.03.015
- Centers for Disease Control and Prevention. Behavioral Risk Factor Surveillance System. https://www.cdc.gov/brfss/questionnaires/pdf-ques/2020-BRFSS-Questionnaire-508.pdf
- LaBrenz CA, O’Gara J, Panusch L, et al. Adverse childhood experiences and mental and physical health disparities: the moderate effect of race and implications in social work. Soc Work Health Care. 2020;59(8):588–614. doi: 10.1080/00981389.2020.1823547
- Cheong E, Sinnott C, Dahly D, et al. Adverse childhood experiences (ACEs) and later-life depression: perceived social support as a potential protective factor. BMJ Open. 2017;7(9):e013228. doi: 10.1136/bmjopen-2016-013228