Automated and controlled processes in comprehending multiple documents

Figures & data

Figure 1. Example unit of the MDC test.

Table 1. Overview of the study variables.

Variable	Operationalization	Level of aggregation	Level of measurement	Value description	M/RF	SD	Skew	Kurtosis
Indicators of multiple document (MD) reading
MDC task success	item scores in the MDC test	item	dichotomous	0 = incorrect item response 1 = correct item response	0.66	-	-	-
time allocation	unit processing time	unit	continuous	high values indicate longer processing times (minutes)	23.71	8.86	0.25	−0.53
corroboration	number of switches between documents	unit	continuous	high values indicate more frequent document switches	11.76	7.36	1.16	2.37
proactive sourcing	indicator of whether the sources of at least two documents were accessed within the first 10% of the processing time of the respective document	unit	dichotomous	0 = less than two sources in a unit were visited proactively 1 = at least two sources in a unit were visited proactively	0.20	-	-	-
Fundamental cognitive skills
reading speed	sum scores of correctly evaluated sentences within a 2-minute time limit	person	continuous	high values indicate higher speed	45.58	5.81	−1.24	1.00
working memory	factor scores from a confirmatory factor model	person	continuous	high values indicate higher proficiency	0.05	5.24	−2.66	8.79
reading comprehension	weighted likelihood estimates from a Partial Credit Model	person	continuous	high values indicate higher proficiency	−0.02	0.94	0.57	5.60
Control variables
general cognitive performance	German school graduation grades (‘Abiturnote’)	person	continuous	small values indicate higher proficiency	2.13	0.66	0.07	−0.94
familiarity with study-related MD tasks	study level	person	dichotomous	0 = Bachelor program 1 = Master program	0.43	-	-	-
unit	assigned unit of the MDC test	unit	nominal	0 = MDC unit ‘2134’ 1 = MDC unit ‘Catalano’ 2 = MDC unit ‘Nothing’ 3 = MDC unit ‘Animals’ 4 = MDC unit ‘Universe’	-	-	-	-
unit position	position of the assigned unit within the MDC test part	unit	dichotomous	0 = first unit 1 = second unit	0.50	-	-	-
test position	position of the MDC test part within the overall test procedure	unit	ordinal	0 = first position 1 = second position 2 = third position	-	-	-	-

Notes. M and SD are the mean and standard deviation of the unstandardized variables. RF is the relative frequency of category ‘1’ of dichotomous variables.

Table 2. Correlations on person level (n = 401 students).

Variables		(1)	(2)	(3)	(4)	(5)	(6)	(7)
(1)	general cognitive performance (graduation grades)^1	-
(2)	familiarity with study-related MD tasks (study level)	.05	-
(3)	reading speed	−.21***	−.08	-
(4)	working memory	−.13*	.03	.21***	-
(5)	reading comprehension	−.32***	.17***	.29***	.35***	-
(6)	time allocation^2	−.09	−.09	−.04	.15**	.16**	-
(7)	corroboration^2	−.17***	.14**	.16**	.19***	.29***	.55***	-
(8)	proactive sourcing^3	−.14**	.21***	.10*	.05	.25***	.10	.22***

Notes. ¹Small values indicate higher proficiency. ²For determining correlations on person level, these variables were aggregated by averaging log-transformed values over units. ³For determining correlations on person level, this variable was aggregated by the sum of values over units.

*p < .05, **p < .01, ***p < .001.

Table 3. Prediction of time allocation (LMM), corroboration (LMM), and sourcing (GLMM).

	Time allocation (H1)		Corroboration (H2)		Proactive sourcing (H3)
	Est. (SE)	β	Est. (SE)	β	Est. (SE)	OR
intercept	−0.16 (0.08)		0.25 (0.10)		−7.32 (0.51)
unit position 2	−0.19 (0.04)	−.09***	−0.06 (0.05)	−.03	3.01 (0.26)	20.39***
test position 2	−0.25 (0.09)	−.12**	−0.12 (0.09)	−.06	−1.29 (0.38)	0.28
test position 3	−0.79 (0.09)	−.37***	−0.57 (0.10)	−.27***	−2.65 (0.42)	0.07***
graduation grades	−0.03 (0.04)	−.03	−0.08 (0.04)	−.08	−0.48 (0.19)	0.62
study level Master	−0.17 (0.07)	−.09*	0.20 (0.08)	.10*	2.16 (0.36)	8.64***
Cognitive skills
reading speed	−0.10 (0.04)	−.09*	0.08 (0.04)	.08*	0.31 (0.18)	1.36
working memory	0.13 (0.04)	.13**	0.06 (0.04)	.06	−0.27 (0.17)	0.76
reading comprehension	0.15 (0.04)	.15***	0.18 (0.05)	.17***	1.16 (0.21)	3.19***
% variance explained	10.5		19.5		57.9

Note. Unit effects are not reported. Est. = unstandardized coefficients; β = standardized coefficients; OR = Odds ratio.

*p < .05, **p < .01, ***p < .001.

Table 4. Prediction of the probability of MDC task success (GLMMs).

	H4a		H4b		H4c
	Est. (SE)	OR	Est. (SE)	OR	Est. (SE)	OR
intercept	0.88 (0.29)		0.93 (0.29)		0.88 (0.29)
unit position 2	0.16 (0.05)	1.17***	0.16 (0.05)	1.17***	0.17 (0.05)	1.19***
test position 2	−0.09 (0.08)	0.91	−0.04 (0.07)	0.96	0.00 (0.07)	1.00
test position 3	−0.25 (0.08)	0.78**	−0.27 (0.08)	0.76***	−0.10 (0.08)	0.90
graduation grades	−0.20 (0.03)	0.82***	−0.12 (0.03)	0.88***	−0.11 (0.03)	0.89***
study level Master	0.29 (0.07)	1.34***	0.18 (0.06)	1.19**	0.16 (0.06)	1.17*
Cognitive skills
reading speed	0.14 (0.04)	1.15***	0.07 (0.03)	1.08*	0.07 (0.03)	1.08*
working memory	0.10 (0.03)	1.11**	0.02 (0.03)	1.02	0.00 (0.03)	1.00
reading comprehension	-		0.33 (0.04)	1.40***	0.29 (0.04)	1.33***
Indicators of behavior
time allocation	-		-		0.12 (0.04)	1.12**
corroboration	-		-		0.09 (0.03)	1.10**
proactive sourcing	-		-		0.15 (0.07)	1.16*
% variance explained	33.7		54.2		59.6

Note. Unit effects are not reported. Est. = unstandardized coefficients; OR = Odds ratio.

*p < .05, **p < .01, ***p < .001.