Test administrator effects on cognitive performance in a longitudinal study of ageing

Figures & data

Table 1. Testing specific factors in the GÅS-study, implementation reason and suggested latent factors

Testing specific factors	Implementation reason	Latent factors
Multiple test administrators	Practical: Change in personnel throughout the study	Test administrator influence, e.g. experience of test administrator
Different test versions	Reduce practice effects on subsequent testing occasions	Different difficulty levels of the test versions
Different orders of test battery administration	To reduce potential systematic fatigue effects	Fatigue due to serial testing or alertness in the beginning of the testing session.
Time of day of testing	Practical: To test many participants in the same day	Fatigue due to differences in circadian rhythms.
Testing setting	To boost participant rate/minimise selection bias	Discomfort/comfort of the participant

Figure 1. Flow chart of the sample with the exclusion criteria.

Notes: MMSE = Mini mental state examination.

Table 2. Number of examination times for participants

	Number of participants	Percentage
Examination times
1	2,611	53
2	1,523	31.1
3	542	11.1
4	184	3.8
5	37	.7
Total	4,897	100

Table 3. Sample characteristics after exclusion and descriptive statistics of participant observations and testing specific factors

	Mean	Standard deviation	Number of observations	Percentage
Age (years)	71.34	10.1
Education (years)	10.90	3.79
MMSE	27.62	1.69
Sex
Women			3,602	53.9
Men			3,084	46.1
Previous experience
No experience at the time of examination			4,090	61.2
Previous experience			2,596	38.8
Time of day
Morning			2,243	33.5
Prior lunch			2,820	42.2
Afternoon			1,618	24.2
Test order
1			3,390	50.7
2			3,296	49.3
Test version
1			2,248	33.6
2			2,219	33.2
3			2,219	33.2
Testing setting
Test centre			5,921	88.6
Home visit			492	7.4

Notes: The 6,686 observations presented here were acquired from 4,090 distinct participants. MMSE = mini mental state examination.

Table 4. Variance estimates for different tests and variance components and percentage of variance corresponding to test administrator

Random effects	Variance estimate	Standard error	Wald Z	p-value	Percentage of variance
Digit cancellation
Test administrator	.369	.129	2.86	.004**	2.8
Participant	9.07	.282	32.1	.001***
Residual	3.92	.118	33.2	.001***
Pattern comparison
Test administrator	1.32	.437	2.82	.005**	3.5
Participant	22.3	.746	29.9	.001***
Residual	11.7	.355	32.9	.001***
Free recall
Test administrator	.065	.029	2.26	.024*	1.4
Participant	1.92	.099	19.4	.001***
Residual	2.59	.077	33.8	.001***
Recognition
Test administrator	.021	.008	2.52	.012*	2.6
Participant	.314	.017	18.4	.001***
Residual	.473	.014	34.1	.001***
Speed of mental rotation
Test administrator	1.64	.573	2.68	.007**	1.5
Participant	27.8	.851	32.7	.001***
Residual	71.6	.569	125.9	.001***
Accuracy of mental rotation
Test administrator	.0006	.0002	2.49	.013*	1.9
Participant	.008	.0007	11.6	.001***
Residual	.024	.0007	34.5	.001***

^* Significant at the p < .05 level.

^** Significant at the p < .01 level.

^*** Significant at the p < .001 level.

Table 5. Statistics for time of day with estimated means, F-values and p-values for fixed effects and effect sizes of pairwise comparisons

Tests	Estimated marginal means† (SE)			df			F-value	p-value	Cohen’s d
	Morn.	P. lunch	Aftern.	Morn.	P. lunch	Aftern.			Morn. vs. P. lunch	Morn. vs. Aftern.	P. lunch vs Aftern.
Digit cancellation	15.3 (.172)	15.4 (.171)	15.5 (.173)	48.9	48.0	50.5	1.32	.268	.012	.014	.029
Pattern comparison	23.5 (.301)	23.6 (.3)	23.8 (.303)	40.8	49.1	42.2	1.54	.214	.081	.047	.029
Free recall	6.15 (.09)	6.11 (.89)	6.02 (.091)	68.1	66.2	73.4	1.81	.164	.021	.059	.038
Recognition	2.75 (.043)	2.67 (.43)	2.6 (.044)	46.9	45.8	49.4	14.4	.001***	.089	.168	.079
Speed of mental rotation	16.7 (.335)	16.2 (.333)	16.2 (.337)	36.4	35.8	37.6	6.28**	.002**	.046	.051	.006
Accuracy of mental rotation	.604 (.008)	.596(.008)	.598 (.008)	62.3	60.4	67.3	1.28	.278	.045	.033	.011

Notes: Morn. = Morning, P. lunch = Prior lunch, Aftern. = Afternoon.

^† Estimated means are evaluated at 10.9 years of education for all tests except speed of mental rotation which is evaluated at 11.14 years.

*** Significant at the p < .001 level.

** Significant at the p < .01 level.

*** Significant at the p < .001 level.

Table 6. Statistics for test order with estimated means, F-values and p-values for fixed effects and effect sizes of pairwise comparisons

Tests	Estimated marginal means† (SE)		df		F-value	p-value	Cohen’s d
	Order 1	Order 2	Order 1	Order 2			Order 1 vs. 2
Digit cancellation	15.3 (.169)	15.6 (.17)	46.1	46.8	12.4	.001***	.082
Pattern comparison	24.3 (.297)	23.0 (.298)	38.6	39.0	87.8	.001***	.226
Free recall	6.07 (.086)	6.11 (.087)	59.1	60.4	.425	.515	.017
Recognition	2.66 (.042)	2.69 (.42)	41.9	42.6	1.99	.158	.036
Speed of mental rotation	15.8 (.331)	16.9 (.331)	34.9	35.0	42.8	.001***	.010
Accuracy of mental rotation	.598 (.008)	.6 (.008)	54.15	54.2	.228	.633	.011

^† Estimated means are evaluated at 10.9 years of education for all tests except speed of mental rotation which is evaluated at 11.14 years.

*** Significant at the p < .001 level.

Table 7. Statistics for test versions with estimated means, F-values and p-values for fixed effects and effect sizes of pairwise comparisons

Tests	Estimated marginal means† (SE)			df			F-value	p-value	Cohen’s d
	Version 1	Version 2	Version 3	Version 1	Version 2	Version 3			Version 1 vs. 2	Version 1 vs. 3	Version 2 vs. 3
Digit cancellation	15.5 (.169)	15.4 (.17)	15.3 (.17)	46.4	46.8	47.0	6.02	.001***	.039	.071	.034
Pattern comparison	23.6 (.297)	23.9 (.298)	23.4 (.298)	39.0	39.4	39.4	8.52	.001***	.052	.038	.09
Free recall	6.3 (.088)	5.8 (.088)	6.17 (.88)	64.8	64.8	64.8	42.2	.001***	.234	.06	.173
Recognition	2.65 (.043)	2.72 (.043)	2.66 (.043)	45.0	45.1	45.2	5.24*	.005**	.078	.007	.071
Speed of mental rotation	N/A	N/A	N/A	N/A	N/A	N/A	2,236	.001***	N/A	N/A	N/A
Accuracy of mental rotation	.593 (.008)	.605 (.008)	.599 (.008)	60.0	60.6	60.6	2.38	.93	.061	.033	.033

Note: The covariate difficulty level is only relevant for mental rotation speed, all remaining tests have test version as covariate instead.

^† Estimated means are evaluated at 10.9 years of education for all tests except speed of mental rotation which is evaluated at 11.14 years.

** Significant at the p < .01 level.

*** Significant at the p < .001 level.

Table 8. Statistics for testing setting with estimated means, F-values and p-values for fixed effects and effect sizes of pairwise comparisons

Tests	Estimated marginal mean^† (SE)		df		F-value	p-value	Cohen’s d
	Test centre	Home visit	Test centre	Home visit			Test centre vs. home visit
Digit cancellation	16.1 (.159)	14.8 (.204)	36.1	96.6	64.9	.001**	.356
Pattern comparison	24.9 (.28)	22.4 (.354)	30.8	77.51	87.2	.001**	.43
Free recall	6.34 (.077)	5.85 (.118)	38.1	199.1	19.2	.001**	.173
Recognition	2.74 (.039)	2.61 (.054)	30.0	114.9	6.87	.001**	.139
Speed of mental rotation	16.2 (.313)	16.5 (.394)	27.9	69.9	1.23	.268	.034
Accuracy of mental rotation	.598 (.007)	.6 (.011)	35.5	174.5	.862	.011*	.862

^† Estimated means are evaluated at 10.9 years of education for all tests except speed of mental rotation which is evaluated at 11.14 years.

^* Significant at the p < .05 level.

^** Significant at the p < .001 level.

Table 9. F-values and p-values for fixed effects for individual differences

Tests	Individual differences
	Age		Sex		Education		Previous experience
	F-value	p-value	F-value	p-value	F-value	p-value	F-value	p-value
Digit cancellation	107.1	.001***	44.2	.001***	97.66	.001***	9.8	.002**
Pattern comparison	192.2	.001***	19.7	.001***	255.1	.001***	3.45	.064
Free recall	46.5	.001***	225	.001***	169.5	.001***	.53	.467
Recognition	3.53	.001***	78.4	.001***	66.5	.001***	.177	.674
Speed of mental rotation	31.9	.001***	5.69	.001***	31.36	.001***	22.6	.001***
Accuracy of mental rotation	16.7	.001***	320	.001***	104	.001***	1.37	.242

^** Significant at the p<.01 level.

^*** Significant at the p < .001 level.