Do Reading Goals Make a Difference for Refutation Text Effectiveness?

Figures & data

Table 1. Number and percentage of participants in each condition who changed their misconception to true belief from t1 to t2.

Belief change	Traditional		Refutation
	Entertainment	Explanation	Entertainment	Explanation
No	25 (53.19%)	32 (74.42%)	10 (27.78%)	18 (42.86%)
Yes	22 (46.81%)	11 (25.58%)	26 (72.22%)	24 (57.14%)
Sum	47 (100%)	43 (100%)	36 (100%)	42 (100%)

Note. N = 168.

Table 2. Final logistic regression models of belief change and misconceptions in written justifications.

	B (SE)		OR	95% CI for OR
		p		Lower	Upper
Belief change
Intercept	−0.18 (0.26)	.493	0.83	0.50	1.40
Text type	1.22 (0.33)	<.001	3.40	1.79	6.62
Reading goal	−0.81 (0.33)	.014	0.44	0.23	0.84
χ²(2) = 18.94, p<.001, R²=0.08 (Hosmer-Lemeshow), 0.11 (Cox-Snell), 0.14 (Nagelkerke)
Misconceptions in written justifications
Intercept	−0.54 (0.29)	.066	0.58	0.32	1.03
Text type	1.27 (0.40)	.002	3.57	1.65	8.07
Reading goal	−1.18 (0.41)	.004	0.31	0.13	0.67
χ²(2)=16.88, p<.001, R²=0.10 (Hosmer-Lemeshow), 0.10 (Cox-Snell), 0.15 (Nagelkerke)

Figure 1. Confidence accuracy before and after reading by condition. Note. Mean confidence accuracy scores before and after reading the text are shown for different text types and reading goals. Small confidence accuracy values indicate inappropriate confidence in a misconception, and high values indicate appropriate confidence in a correct belief. Error bars show 95% confidence intervals.

Table 3. Three-way ANOVA statistics for confidence accuracy.

Predictor	F (1, 163)	p	ɳ^2part	95% CI for ɳ^2
				Lower	Upper
Intercept	871.77	<.001	0.84	0.80	0.87
Text type	20.14	<.001	0.11	0.03	0.20
Reading goal	7.59	.007	0.04	0.003	0.12
Time	62.63	<.001	0.28	0.17	0.38
Text type × reading goal	0.05	.820	0.00	0.00	0.02
Text type × time	29.54	<.001	0.15	0.06	0.25
Reading goal × time	6.51	.012	0.04	0.002	0.11
Text type × reading goal × time	0.71	.400	0.00	0.00	0.04

Figure 2. Relative frequency of misconceptions and correct answers in participants’ written justifications.

Table 4. Exemplary quotations from participants’ justifications.

Code	Label	Sample quotation (translated from German)
0	Misconception	“The brain hemispheres differ in such a way that during creative activities, such as painting, other brain regions are activated than, for example, when solving a mathematical task. Therefore, if the brain hemisphere for analytical thinking is more pronounced, a learner is more likely to achieve good performance in subjects such as mathematics.”
1	Correct	“Many learning processes require the cooperation of several parts of the brain, so that both hemispheres of the brain have to work together, for example in reading. No cognitive performance is possible only through processing in one brain hemisphere. Therefore, differences in hemisphere dominance cannot explain why students learn certain content differently.”
NA	Not assigned	“I guessed or rather the answer seemed logical to me—after the explanation just now, however, I can well understand the other perspective.”

Data availability statement

This study was registered with the OSF. The data that support the findings of this study are openly available at the OSF.