Do certain phrases in letters of recommendation predict success in fellowship performance?

Abstract

Background

Letters of recommendation (LORs) are a valued, yet imperfect tool. Program directors (PDs) score phrases such as give my highest recommendation and top 5 to 10% of students as positive. Although positive phrases are valued by PDs, there is no evidence that these phrases predict performance. We attempt to identify whether 12 specific phrases found in letters of recommendation predict future performance of fellows.

Methods

LORs were evaluated for 12 select phrases and statements. Alpha Omega Alpha (AOA) status, Step 2 Clinical Knowledge (CK) score, and whether the letter writer was personally known to our admission’s committee were also categorized. Logistic regressions were performed to evaluate the relationship of the independent variables with fellow performance.

Results

Using multivariate logistic regression, one of the best residents (OR = 4.02, 95% CI (1.0, 15.9), p < 0.05), exceeds expectations (OR = 4.74, 95% CI (1.4, 16.3), p = 0.01), and give my highest recommendation (OR = 3.87, 95% CI (1.3, 11.7), p = 0.02) predicted positive performance. Highly recommend (OR = 0.31, 95% CI (0.1, 1.0), p < 0.05) and top 5 to 10% (OR = 0.05, 95% CI (0.0, 0.6), p = 0.02) predicted negative performance. The remaining phrases did not correlate to fellowship performance.

Conclusion

The current LOR evaluation process may place undo importance on phrases that have limited bearing on a candidate’s success in training. Training both letter readers and writers to avoid using coded language or avoid assigning improper importance to select phrases may help improve the candidate selection process.

Keywords:

• Fellowship application
• fellowship performance
• letters of recommendation
• medical education

Acknowledgements

The authors would like to thank Dani Lavage for her collaboration and help with the statistical review of this project.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Notes on contributors

Niteesh Sundaram

Niteesh Sundaram, MD, MS, Resident Physician Postgraduate Year 2, Department of General Surgery Crozer Health.

K. Karisa Walker

K. Karisa Walker, MD, MEd, Associate Professor of Anesthesiology, Department of Anesthesiology University of Florida, Gainesville.

Franklyn P. Cladis

Franklyn P. Cladis, MD, MBA, Professor of Anesthesiology, Department of Anesthesiology and Perioperative Medicine University of Pittsburgh Medical Center.