Probability of minority inclusion is underestimated

Figures & data

Figure 1. Overview of the conditions and procedures in Experiment 1. Each participant estimated p_inc (probability of inclusion) for five problems (Q1–Q5). The topics of Q1–Q4 varied across conditions. The original questions were in Japanese. q, prevalence presented in the problems. n, group size presented in the problems. N, sample size.

Table 1. The topics used in Experiment 1 and the results of the prevalence estimation task. The prevalence of each topic was estimated by the Experiment 1 participants who did not see the topic in the preceding p_inc problems (see Figure 1). Medians of the estimated prevalence are shown with 95% CIs. The actual prevalence values were adopted from the references. In reality, it is often impossible to determine the prevalence of the traits definitively. These traits have a wide continuum of individual differences and may even be multidimensional (Bermudez & Zatorre, 2009; Bosten, 2019; Epstein et al., 2012; Simner, 2012).

Topic	N	q (%)	Median estimate [95% CI]	Actual percentage prevalence in Japan (otherwise noted)	Reference
Colour deficiency	260	3	5 [3, 5]	3.8–5.9 (male), 0.2–0.6 (female)	Birch (2012)
Gay/bisexual	260	7	6 [5, 8]	7.0 (student sample, age 18–22) 1.9 (age 18–64)	Ikuta et al. (2017) Saitama Prefecture (2021)
Synaesthesia	351	3	2 [2, 3]	4.4 (UK student sample)	Simner et al. (2006)
Absolute pitch	351	7	5 [5, 6]	7.8 (non-music major student sample, 80–100% accuracy of pitch class naming)	Miyazaki et al. (2012)
Foreigners	335	3	8 [7, 10]	2.2	Statistics Bureau of Japan (2021)
Police women	335	7	18 [15, 20]	10.6	National Police Agency of Japan (2021)

Note: N, sample size in Experiment 1. q, percentage prevalence presented in the Experiment 1 problems.

Figure 2. Results of Experiment 1, participants’ estimates of the probability of inclusion (p_inc) at various settings of q and n. A histogram of the estimated p_inc for the blood type problem (Q5) is shown in a. The blood type problem was presented to all the participants of Experiment 1. The results of the other four p_inc problems (Q1–Q4) of the conditions other than the Majority condition are shown in b to e. The topics used in these problems varied among the conditions. The results of the Majority condition are shown in f to i. Estimates less than 0 or larger than 100 are not shown in this figure. q, prevalence presented in the problems. n, group size presented in the problems. N, sample size. EV, expected value.

Table 2. Descriptive statistics of the estimated probability of inclusion in Experiments 1 and 2. Prevalence (q) and group size (n) were presented in the problems. Underestimation was defined as estimates less than the normative p_inc (probability of inclusion) for each problem.

Topic	q (%)	n	Normative pinc (%)	Median estimate [95% CI]	Underestimation (%)	Estimate ≤ q (%)
Negative condition (N = 88)
Q1, colour deficiency	3	30	59.9	2 [1, 3]	86.4	71.6
Q2, colour deficiency	3	80	91.3	3 [2, 3]	90.9	62.5
Q3, gay/bisexual students	7	30	88.7	3 [2, 5]	93.2	75.0
Q4, gay/bisexual students	7	80	99.7	6 [5, 7]	95.5	70.5
Positive condition (N = 78)
Q1, synaesthesia	3	30	59.9	3 [1, 3]	88.5	64.1
Q2, synaesthesia	3	80	91.3	3 [2, 8]	93.6	57.7
Q3, absolute pitch	7	30	88.7	3 [2, 7]	91.0	66.7
Q4, absolute pitch	7	80	99.7	7 [6, 9]	96.2	59.0
Visible condition (N = 94)
Q1, foreigners	3	30	59.9	3 [3, 9]	85.1	52.1
Q2, foreigners	3	80	91.3	6 [3, 15]	88.3	44.7
Q3, police women	7	30	88.7	7 [4, 20]	85.1	52.1
Q4, police women	7	80	99.7	15 [9, 56]	90.4	36.2
Neutral condition (N = 88)
Q1, fictional sweat content	3	30	59.9	3 [1, 3]	80.7	62.5
Q1, fictional sweat content	3	80	91.3	3 [3, 6]	89.8	54.5
Q3, fictional gene	7	30	88.7	3.5 [2, 7]	87.5	65.9
Q4, fictional gene	7	80	99.7	6.5 [6, 8]	95.5	60.2
Majority condition (N = 81)
Q1, normal colour vision	97	30	≃ 100	97 [95, 99]	76.5	56.8
Q2, normal colour vision	97	80	≃ 100	97 [97, 99]	75.3	59.3
Q3, heterosexual students	93	30	≃ 100	93 [90, 93]	75.3	61.7
Q4, heterosexual students	93	80	≃ 100	93 [93, 95]	75.3	59.3
Experiment 2 (N = 48 students with PCs)
Q1, colour deficiency	3	30	59.9	20 [3, 50]	77.1	37.5
Q2, colour deficiency	3	80	91.3	68.5 [5, 80]	83.3	33.3
Q3, gay/bisexual students	7	30	88.7	24.5 [7, 50]	85.4	37.5
Q4, gay/bisexual students	7	80	99.7	53 [10, 90]	93.8	33.3

Figure 3. Results of Experiment 2, the estimated probability of inclusion for the student sample. Students (N = 48) with PCs estimated the probabilities of inclusion for four problems (Q1–Q4) identical to those in the Negative condition of Experiment 1 (colour deficiency problems, gay/bisexual student problems). Estimates larger than 100 are omitted in this figure. EV, expected value. q, prevalence presented in the problems. n, group size presented in the problems.

Figure 4. Overview of the conditions and procedures in Experiment 3. The content of the critical question (Q2) varied between the conditions. The original questions were in Japanese. q, prevalence presented in the problems. n, group size presented in the problems. N, sample size.

Figure 5. Results of Experiment 3 critical question (Q2). In the Control condition, (a) the participants estimated p_inc for q = 7% and n = 30 (Q2, gay/bisexual student problem with revised wording). For reference, b shows the result of the gender problem (Q1), which appeared only in the Hint condition. The results of the Control condition are plotted together with the results of the other conditions (c, e, f, and g) for comparison purposes. EV, expected value. q, prevalence presented in the problems. n, group size presented in the problems.

Table 3. Percentage frequency of the heuristics used in Experiments 2 and 3. The analyses of the estimates and participants’ retrospective verbal reports identified seven heuristics used in the estimation tasks in Experiments 2 and 3. Raw frequencies are in parentheses. In Experiment 3, the frequency of the heuristics was compared between the Control (p_inc estimation) and Cumulative Risk conditions.

Heuristic	Typical estimate (q = 7%, n = 30)	Experiment 2		Experiment 3
				Control		Cumulative Risk
Normative*	88, 89	8.3%	(4)	2.3%	(2)	10.3%	(9)
EV	2	25.0%	(12)	10.3%	(9)	5.7%	(5)
EV-to-probability translation*	100, 90	20.8%	(10)	20.7%	(18)	5.7%	(5)
1/n*	3	0.0%	(0)	10.3%	(9)	0.0%	(0)
Constant*	7	8.3%	(4)	6.9%	(6)	28.7%	(25)
Additive*	210, 100	2.1%	(1)	0.0%	(0)	10.3%	(9)
Guess without calculation	–	29.2%	(14)	31.0%	(27)	23.0%	(20)
Other/unknown heuristics	–	6.3%	(3)	18.4%	(16)	16.1%	(14)
Total		100.0%	(48)	100.0%	(87)	100.0%	(87)

Note: *Statistically significant difference (p < .05) between Control and Cumulative RIsk conditions in Experiment 3.

Figure 6. Results of Experiments 4a and 4b. Each participant estimated p_inc for a real group of people in the classroom. The only difference between the two experiments was n (i.e. the group size presented in the problems). EV, expected value. q, prevalence presented in the problems. N, sample size.

Figure 7. Results of Experiment 5. The participants provided their attitude ratings twice, namely, before (blue plot) and after (red plot) a middle task. The vertical axis represents the attitude score (the average of the three rating items). A high attitude rating score indicated strong agreement with the inclusive statements on colour deficiency. The middle task varied by condition: p_inc estimation (a), EV estimation (b), and reading about the normative p_inc value and rating how believable the value was (c). The open dots with thin lines represent the participants’ scores and the filled dots with thick lines represent the averages. The p-values were adjusted for the triplet of t-tests. EV, expected value.

Table A1. Results of Experiment 3 critical questions (Q2). Prevalence (q) and group size (n) were presented in the problems. Underestimation was defined as estimates less than the normative answer for each problem.

Condition	q (%)	n	Normative	Median estimate [95% CI]	Underestimation (%)	Estimate ≤ q (%)
Control (N = 87)	7	30	88.7	7 [ 3, 10]	85.1	55.2
Hint (N = 82)	7	30	88.7	7 [ 5, 10]	86.6	57.3
Complementary (N = 90)	7	30	11.3	40 [20, 60]	33.3	24.4
No Group Size (N = 90)	7	–	–	9.5 [ 7, 20]	–	47.8
Frequency (N = 90)	7	30	88.7	10 [ 7, 20]	76.7	44.4
Cumulative Risk (N = 87)	7	30	88.7	11 [ 7, 20]	78.2	46.0

Figure A1. Results of the middle task of the p_inc estimation condition (N = 99), Experiment 5.

Bermudez, P., & Zatorre, R. J. (2009). A distribution of absolute pitch ability as revealed by computerized testing. Music Perception, 27(2), 89–101. https://doi.org/10.1525/mp.2009.27.2.89

 Web of Science ®Google Scholar

Bosten, J. (2019). The known unknowns of anomalous trichromacy. Current Opinion in Behavioral Sciences, 30, 228–237. https://doi.org/10.1016/j.cobeha.2019.10.015

 Web of Science ®Google Scholar

Epstein, R., McKinney, P., Fox, S., & Garcia, C. (2012). Support for a fluid-continuum model of sexual orientation: A large-scale internet study. Journal of Homosexuality, 59(10), 1356–1381. https://doi.org/10.1080/00918369.2012.724634

 PubMed Web of Science ®Google Scholar

Simner, J. (2012). Defining synaesthesia. British Journal of Psychology, 103(1), 1–15. https://doi.org/10.1348/000712610X528305

 PubMed Web of Science ®Google Scholar

Birch, J. (2012). Worldwide prevalence of red-green color deficiency. Journal of the Optical Society of America A, 29(3), 313–320. https://doi.org/10.1364/JOSAA.29.000313

 Google Scholar

Ikuta, N., Koike, Y., Aoyagi, N., Matsuzaka, A., Fuse-Nagase, Y., Kogawa, K., & Takizawa, T. (2017). Prevalence of lesbian, gay, bisexual, and transgender among Japanese university students: A single institution survey. International Journal of Adolescent Medicine and Health, 29, Article 20150113. https://doi.org/10.1515/ijamh-2015-0113

 Google Scholar

Saitama Prefecture. (2021). Tayosei o soncho-suru kyoseishakai-zukuri ni kansuru chosa hokokusho [Report of the survey for a coexisting society that respects diversity]. https://www.pref.saitama.lg.jp/documents/183194/lgbtqchousahoukokusho.pdf.

 Google Scholar

Simner, S., Mulvenna, C., Sagiv, N., Tsakanikos, E., Witherby, S. A., Fraser, C., Scott, K., & Ward, J. (2006). Synaesthesia: The prevalence of atypical cross-modal experiences. Perception, 35(8), 1024–1033. https://doi.org/10.1068/p5469

 PubMed Web of Science ®Google Scholar

Miyazaki, K., Makomaska, S., & Rakowski, A. (2012). Prevalence of absolute pitch: A comparison between Japanese and Polish music students. The Journal of the Acoustical Society of America, 132(5), 3484–3493. https://doi.org/10.1121/1.4756956

 PubMed Web of Science ®Google Scholar

Statistics Bureau of Japan. (2021). 2020 population census. https://www.stat.go.jp/english/data/kokusei/2020/summary.html

 Google Scholar

National Police Agency of Japan. (2021). Reiwa 3 nen ban keisatsu hakusho [The white paper on police 2021].

 Google Scholar

Data availability statement

Anonymized data, questionnaires, and analysis code are publicly available at Open Science Framework repository, https://doi.org/10.17605/OSF.IO/GXWN7

 Google Scholar

.