Assessment of cognitive biases and biostatistics knowledge of medical residents: a multicenter, cross-sectional questionnaire study

Figures & data

Table 1 Resident demographic, educational, and residency profile

1. Age	Median: 32 years Range: 24–44 years		2. Gender^a	Male	Female	3. Year of residency	Median: 2 years Range: 1–7 years
				112 (73.2%)	41 (26.8%)
4. Hospital^a,b	1st I.K.A. Hospital	251 G.N.A. Hospital	Aglaia Kyriakou Hospital	Alexandra Hospital	Asklipiio Hospital	Elpis Hospital	Evangelismos Hospital	Metaxa Hospital
	18 (11.8%)	17 (11.1%)	21 13.7%)	21 (13.7%)	21 (13.7%)	19 (12.4%)	15 (9.8%)	21 (13.7%)
5. Residency field^a	Medical specialties			Surgical specialties	Diagnostic and laboratory specialties
	71 (46.4%)			67 (43.8%)	15 (9.8%)
6. Other graduate degrees ^a,c	No 136 (88.9%)		PhD 6 (3.9%)	MSc 14 (9.2%)		7. Residency program type ^a	University based 28 (18.3%)	Community based 125 (81.7%)
8. Education abroad ^a,d	No		Primary or secondary education	Medical School		Prior to beginning residency	During Residency	Other
	96 (62.7%)		2 (1.3%)	45 (29.4%)		4 (2.6%)	10 (6.5%)	3 (2%)
9. Biostatistics training ^a,e	No		Primary or secondary education	Medical School		Prior to beginning residency	During Residency	Other
	28 (18.3%)		7 (4.6%)	119 (77.8%)		0	6 (3.9%)	11 (7.2%)

^aPercentages in parentheses represent the rate percent to the study's 153 respondents.

^bPercentages in parentheses do not add up to 100% due to rounding.

^cPercentages in parentheses do not add up to 100% due to rounding and overlapping of degrees (three respondents had both PhD and MSc degrees).

^dPercentages in parentheses do not add up to 100% due to rounding and overlapping of foreign educational experiences.

^ePercentages in parentheses do not add up to 100% due to rounding and overlapping of past biostatistics training settings.

Table 2 Biostatistics knowledge questions

	Scenario description	Multiple choices (select only one)	Response frequencies^a (%)
SD vignette	You read in a medical paper that 100 patients had fasting serum glucose levels of 153 mg/dl±6 mg/dl (mean±SD). Which of the following statements is the most correct?	• It is approximately 95% certain that the true mean value lies between 147 and 159 mg/dl	80 (52.3)
	• Approximately 50% of patients had fasting serum glucose 153 mg/dl	12 (7.8)
		• Approximately 95% of patients had fasting serum glucose 141–165 mg/dl	49 (32)^b
		• I do not know and do not wish to guess	12 (7.8)
SEM vignette	You read in a medical paper that 2 hours following administration of a drug the mean systolic pressure of the patients was 138 mm Hg±8 mm Hg (mean±SEM). Which of the following statements is the most correct?	• It is approximately 95% certain that the true mean value lies between 122 and 154 mm Hg	35 (22.9)^b
	• Approximately 50% of patients had mean systolic pressure 138 mm Hg	11 (7.2)
		• Approximately 95% of patients had mean systolic pressure 130–146 mm Hg	85 (55.6)
		• I do not know and do not wish to guess	22 (14.4)
p-value vignette	You read in a clinical trial report that the treatment under investigation showed statistically significant increase in overall survival compared to the placebo-control group (p<0.05). Which of the following statements is most correct?	• The treatment is definitely better compared to placebo	58 (37.9)
	• The observed increase in overall survival is so large that there is less than a 5% chance that placebo is equal to the treatment	30 (19.6)
		• If the treatment does not actually increase survival, then the chance of obtaining the observed (or even greater) increase in survival is less than 5%.	41 (26.8)^b
		• I do not know and do not wish to guess	24 (15.7)
CI vignette	You read in a medical paper that the RR for a disease in individuals who are exposed to a risk factor is 1.64 (95% CI=1.10–2.45) compared to otherwise similar individuals who are not exposed to this risk factor. Which of the following statements is most correct?	• 95% of those exposed to the risk factor had RR between 1.10 and 2.45	44 (28.8)
	• There is a 95% likelihood that the true RR falls between 1.10 and 2.45	51 (33.3)^b
	• Approximately 50% of individuals exposed to the risk factor had a RR of 1.64	9 (5.9)
		• I do not know and do not wish to guess	49 (32)
Correlation coefficient vignette	You read in a medical paper that there was a highly statistically significant positive correlation between atrial natriuretic peptide blood levels and treatment outcomes (r=0.29, p<0.001). Which of the following statements is most correct?	• There was a strong correlation between atrial natriuretic peptide blood levels and treatment outcomes	97 (63.4)
	• The correlation between atrial natriuretic peptide blood levels and treatment outcomes was weak	23 (15)^b
	• No correlation between atrial natriuretic peptide blood levels and treatment outcomes was detected	4 (2.6)
		• I do not know and do not wish to guess	29 (19)
Sensitivity vignette	An article is describing the effectiveness of a novel diagnostic technique for the detection of a disease. The sensitivity of the technique is defined as:	• The ratio of true positives to the number of subjects who have the disease	79 (51.6)^b
	• The ratio of true positives to the number of subjects who had positive test results	54 (35.3)
		• The ratio of false positives to the number of subjects who had positive test results	8 (5.2)
		• I do not know and do not wish to guess	12 (7.8)
RR vignette	You read in a medical paper that a new screening test for a disease reduces by 25% the risk of death from the disease in high-risk individuals compared to those who do not undergo the screening test (RR=0.75). On the other hand, you are also aware that 12 out of 1,000 high-risk individuals who undergo an older screening test will die from the disease compared to 20 out of 1,000 high-risk individuals who are not screened with any method. Which of the following statements is most correct?	• The data indicate that the new screening test may achieve greater reduction of the risk of death from the disease compared to the old screening test	49 (32)
	• The data indicate that the old screening test may achieve greater reduction of the risk of death from the disease compared to the new screening test	43 (28.1)^b
	• The data indicate that both screening tests have similar efficacy in reducing the risk of death from the disease	10 (6.5)
		• I do not know and do not wish to guess	51 (33.3)

^aPercentages in parentheses represent the rate percent to the study's 153 respondents.

^bCorrect answer.

SD, standard deviation; SEM, standard error of the mean; CI, confidence interval; RR, relative risk.

Table 3 Cognitive biases questions

	Scenario description	Multiple choices (select only one)	Response frequencies^a (%)
Gambler's fallacy vignette	Approximately 10 patients with hemoptysis per month attend a primary care center. Epidemiological data from this center indicate that approximately 1 out of 10 hemoptysis patients will have TB. Near the end of the month, nine patients with hemoptysis have attended the center and all were TB-negative. Which of the following statements about the next patient with hemoptysis who will attend the primary center before the end of the month is most correct?	• It is almost 100% likely that the patient will have TB	4 (2.6)
• The patient's chances of having TB will be approximately 50%	7 (4.6)
• The patient's chances of having TB will be approximately 10%	83 (54.2)^b
	• The patient's likelihood of having TB will be increased compared to the previous 9 patients with hemoptysis but more data will be required to calculate it	30 (19.6)
	• No conclusions about the patient can be derived from the scenario's data	29 (19)
Conjunction fallacy vignette	John is a 42 years old, married office worker with a BMI of 35.6. He complains of angina-like chest pain. His father had CAD and died from myocardial infarction 7 years ago. John's brother also suffers from CAD. Based on the above data, rank the following statements from one to six where one is the most probable and six is the least probable statement:	• John has depression	48 (31.4) respondents ranked the likelihood that John has Huntington's disease higher compared to the likelihood that he has both CAD and Huntington's disease
• John is suffering from osteosarcoma and pneumonia
• John has CAD
	• John has Huntington's disease
	• John has pre-diabetes
	• John has CAD and Huntington's disease

^aPercentages in parentheses represent the rate percent to the study's 153 respondents.

^bCorrect answer.

BMI, body mass index; TB, tuberculosis; CAD, coronary artery disease.

Table 4 Multivariate association (multiple linear regression) between resident variables and total biostatistics knowledge score

	Regression coefficient^a	p
Age (years)	0.046 (−0.059 to 0.150)	0.389
Year of residency	−0.227 (−0.429 to −0.026)	0.027
Education abroad^b	−0.846 (−1.426 to −0.266)	0.005^c
Total biostatistics familiarity score	0.047 (0.026 to 0.068)	<0.001^c

^aNumbers in parentheses represent the 95% confidence intervals (CI).

^b‘No education abroad’ is used as the reference category.

^cStatistically significant association set at p<0.01 to adjust for multiple pairwise comparisons.

Table 5 Perceived familiarity rating by medical residents for the biostatistical concepts of SD, SEM, p-values, CI, correlation coefficients, RR, sensitivity, and PPV

Familiarity rating^a,b	1 (none) (%)	2 (%)	3 (%)	4 (%)	5 (%)	6 (%)	7 (excellent) (%)
SD	24 (15.7)	7 (4.6)	26 (17)	11 (7.2)	28 (18.3)	26 (17)	31 (20.3)
SEM	37 (24.2)	10 (6.5)	23 (15)	24 (15.7)	24 (15.7)	19 (12.4)	16 (10.5)
P	34 (22.2)	13 (8.5)	16 (10.5)	21 (13.7)	19 (12.4)	29 (19)	21 (13.7)
CI	46 (30.1)	16 (10.5	23 (15)	19 (12.4)	24 (15.7)	14 (9.2)	11 (7.2)
Correlation coefficients	54 (35.3)	21 (13.7)	30 (19.6)	14 (9.2)	11 (7.2)	13 (8.5)	10 (6.5)
RR	33 (21.6)	11 (7.2)	26 (17)	23 (15)	23 (15)	36 (23.5)	1 (0.7)
Sensitivity	14 (9.2)	7 (4.6)	14 (9.2)	16 (10.5)	14 (9.2)	36 (23.5)	52 (34)
PPV	20 (13.1)	12 (7.8)	15 (9.8)	24 (15.7)	22 (14.4)	30 (19.6)	30 (19.6)

^aPercentages in parentheses represent the rate percent to the study's 153 respondents.

^bPercentages in parentheses do not add up to 100% due to rounding.

SD, standard deviation; SEM, standard error of the mean; CI, confidence intervals; RR, relative risk; PPV, positive predictive value.

Table 6 Multivariate association (multiple linear regression) between resident variables and total biostatistics familiarity score

	Regression coefficient^a	p
Biostatistics training^b	6.054 (1.205 to 10.904)	0.015
Other graduate degrees^c	11.349 (5.382 to 17.316)	<0.001^d

^aNumbers in parentheses represent the 95% confidence intervals (CI).

^b‘No prior biostatistics training’ is used as the reference category.

^c‘No other graduate degrees’ is used as the reference category.

^dStatistically significant association set at p<0.01 to adjust for multiple pairwise comparisons.