Perspectives on the quality and utility of letters conforming to the AAIM guidelines

Figures & data

Table 1. Comparing respondents to survey population based on program type

	Total Eligible Population (Pre-Survey)^a	Total Eligible Population (Both Post-Surveys)^b	Pre (2108)	First Post	P-value^c	Second Post	P-Value^c
Qualifying Question			130 (100%)	135 (100%)		116 (88%)
Program Type
Community-based	42 (11%)	55 (14%)	–	22 (17%)	0.4	22 (19%)	0.19
Community-based, University Affiliated	190 (51%)	202 (52%)	–	56 (44%)	0.02*	47 (42%)	0.04*
Other	10 (3%)	8 (2%)	–	1 (<1%)	0.31	0 (0%)	0.12
University-based	131 (35%)	127 (32%)	–	49 (38%)	0.53	44 (39%)	0.23

^aN = 373.

^bN = 392.

^cTwo-sample test of proportions used to compare survey respondents and survey population.

*P < 0.05.

Table 2. Use of AAIM Guidelines and Their Individual Components

	Pre	Post (2018)	Post (2019)
Respondents	116	119	104	P-value*
Use of Guidelines
Used Guidelines (every time, mostly, occasionally)		93 (78%)	93 (87%)
Did not use guidelines (almost never, never)		26 (22%)	14 (13%)
Components of Guidelines Included:
Applicant Learning Milieu
Applicant charateristics	0 (0%)	112 (93%)	99 (95%)
Residency program characteristics	44 (38%)	104 (87%)	92 (88%)	0.001**
Competencies (ACGME)
Interpersonal and Communication Skills (IPC)	114 (99%)	118 (98%)	101 (98%)	0.79
Medical Knowledge (MK)	110 (96%)	115 (96%)	101 (98%)	0.57
Patient Care and Procedures (PC)	113 (100%)	117 (98%)	100 (98%)	0.26
Practice-Based Learning and Improvement (PBLI)	67 (60%)	90 (76%)	79 (77%)	0.009**
Professionalism (P)	111 (96%)	118 (98%)	100 (97%)	0.68
Systems-Based Practice (SBP)	63 (57%)	81 (69%)	78 (77%)	0.006**
Performance Management
Curtailment of clinical privileges	83 (75%)	80 (72%)	71 (73%)	0.85
Formal probation	85 (77%)	83 (75%)	76 (78%)	0.88
Training extension	75 (67%)	75 (68%)	76 (78%)	0.15
Special Notes
Scholarly activity	111 (96%)	116 (97%)	97 (93%)	0.47
Skills sought to master beyond minimum requirements	40 (36%)	69 (58%)	53 (52%)	0.003**
Suitability for fellowship	94 (84%)	98 (83%)	90 (87%)	0.76
Standardized Test Scores
In-Training Exam (ITE)	7 (6%)	5 (4%)	3 (3%)	0.5
USMLE Steps	5 (5%)	4 (3%)	2 (2%)	0.59

Table 3. Effects of Guidelines on Letter of Recommendation Writers

	First Postsurvey (2018)			Second Postsurvey (2019)
	n=107			n = 95
	Major or Moderate Negative Effect n(%)	Neutraln(%)	Major or Moderate Positive Effect n(%)	Major or Moderate Negative Effect n(%)	Neutral n(%)	Major or Moderate Positive Effect n(%)	P-value
Ability to comment on each of the six ACGME competencies	7 (7)	63 (59)	37 (35)	6 (6)	48 (51)	41 (43)	0.449
Ability to distinguish applicants	18 (17)	59 (55)	20 (28)	10 (10)	50 (52)	36 (38)	0.224
Ability to evaluate candidates for fit for a particular fellowship	10 (9)	82 (76)	16 (15)	5 (5)	70 (74)	20 (21)	0.327
Ability to highlight distinctive spects of my program or hospital	4 (4)	50 (47)	53 (50)	1 (1)	41 (43)	53 (56)	<0.001
Ability to structure letter	10 (9)	49 (46)	48 (45)	3 (3)	35 (37)	57 (60)	0.045
Number of direct observations performed on resident	4 (4)	97 (92)	5 (5)	0 (0)	89 (94)	3 (3)	0.144